Search for tag: "data"

Interviewer: What's becoming loud and clear is that most scientific studies are, well, unreliable. But there's help out there for scientists. We'll talk about that next on The Scope.

Announcer: Examining the latest research, and telling you about the latest breakthroughs. "The Science and Research Show" is on The Scope.

Interviewer: I'm talking with Darell Schmick, research librarian at the Eccles Health Sciences Library at the University of Utah. So there are a number of reasons for what some people are calling a research reproducibility crisis, including fraud. But even scientists with the best intentions are at risk for doing sloppy work and there are a lot of reasons for that as well. One of the things that you're interested in looking at is data management. I really like this example of the Postdoc who leaves the lab.

Schmick: So it was it an age-old issue where you do a lot of work and it happens to be on your personal computer. It happens to be in a folder with poorly-named files and you produce a bunch of research on behalf of the institution, but then, once you're done, you obviously take your computer with you, right, and then head off to that next position. Now, that seemingly innocuous, however, has a lot of implications. The data that you produce on behalf the university has ownership concerns. Is it the Postdoc's? Is it the University's?

Interviewer: How can that lead to issues with research reproducibility?

Schmick: So if the Postdoc takes all that data with him or her and hasn't been saved into the department bio or anything like that, how can you ensure that you have records of all the work that Postdoc has done? They could have taken just a little bit of it, they could have taken a substantial chunk of it. And it really leaves the PI as well as the rest of the members of the lab with potentially a significant disadvantage.

Interviewer: So there's a lot of knowledge that can be lost?

Schmick: Absolutely.

Interviewer: So what are some ways to avoid that?

Schmick: We do teach a research administration training class that talks about just the basic fundamentals of just good data management, which involves things like where to properly back up your files and how often to do that. Myself and a couple of the librarians on campus have been working on a pilot for electronic lab notebook technology.

So if, say for instance, you happen to have the perennial issue of lab members recording data in their own personal computers because it's inconvenient to share it, this sort of technology allows for a lab to share in a collaborative notebook technology something that has all those questions that we're talking about answered, like how frequently will it be backed up? Is it going to be backed up in a trustworthy source?

Interviewer: Right. Well, and not to mention that most lab notebooks that I've seen are kind of a disaster.

Schmick: Are you saying that scientists don't uniformly have amazing handwriting?

Interviewer: Exactly. Right, we're all human. And even how the information is recorded is varied from person to person. So the idea is that this would become more standardized?

Schmick: You hit the nail on the head there.

Interviewer: Any other approaches to make their data more accessible or reliable?

Schmick: When we're talking about optimizing the mechanics of anything in the research process, we want to ensure that we're doing it in a way that is not only accessible by us because we can look at our notes, presumably, and be able to understand what we were saying, understand what we were recording, understand what we were encountering in that process. But to think about how the results that you're producing are going to be read by somebody else that's not in that same context. So if you're doing an experiment, you do it for you, but you also ensure that you're doing it in a way that if somebody wants to reproduce that experiment, they can do that.

Interviewer: You've talked about ways about preserving information within a lab group, for example, a research group. What about sharing information more broadly with the scientific community?

Schmick: That's a great question, Julie. And a lot of people think that all you are able to really produce is that end product, that finalized article. And we don't realize, many times, that when we're doing experiments, when we're producing all this data that we're talking about, that data could be good data. It could be good information and good intel for another scientist that's stumbling across that same issue.

If you're embarking on answering a research question and you come across a dataset that has already sought to ask that question, you find out that maybe those results weren't satisfactory enough to produce something into a finished article, that could potentially save you years in otherwise reinventing the wheel.

So another thing that we like to talk about is the idea of ensuring that researchers know that the data that they produce is of value and there are places that you can store that. So one example that comes to mind is figshare. And figshare is a repository that you can actually assign a DOI to the data sets that you're uploading on there. Figshare are all about open science so they say as long as you're making it public, I mean, you can upload it for free.

Interviewer: So how can sharing data with the scientific community help with research reproducibility?

Schmick: There's a lot of news as of late in the way of that openness toward science where folks on a peer review panel want to see the steps you were able to take in order to draw the conclusions that you were able to take or able to make. And if they're able to go ahead and see that data right there from the start, it answers all those questions.

It's when things like that data being withheld presents a larger problem not only for you as author but greater implications for science in general. When we start to withhold that data, when we start to conceal certain steps in that recipe toward what we ended up with the final product, it leads towards a slippery slope of was it not open science and that closed scholarly environment, I think, is something that is well worth fighting against.

Interviewer: Where can people go to learn more about best data management practices?

Schmick: There are a lot of places. If you're embarking on a data management plan there's a great tool called DMPTool. That's dmptool.org, and it'll take you through the steps of the processes of, "Let's take you through the steps of the data management process." By asking you in a 20 questions format, "What's this data going to be for? Which agencies are going to be seeing this data?" And it'll give you recommendations at the end from there. If you're at the University of Utah's campus, I'd encourage you to talk to me or one of our other fine staff at Eccles Health Sciences Library. I'd be delighted answer any questions regarding data management plans.

Announcer: Interesting, informative and all in the name of better health. This is The Scope Health Sciences Radio.

Julie: One day, each swipe of your grocery store loyalty card could help you lead a healthier life, up next on The Scope.

Announcer: Examining the latest research and telling you about the latest breakthroughs. The science and research show is on The Scope.

Julie: I'm talking with Dr. John Hurdle, professor of biomedical informatics at the University of Utah. Dr. Hurdle, if you would see me at the grocery store yesterday, you would have looked in my shopping cart and seen some fruits and vegetables, some chicken and beef and some cookies. But it turns out that this snapshot is very valuable. Why is that?

Dr. Hurdle: Well, that snapshot is valuable because it says something about the patterns of the foods you consume and we know that over time, that pattern is indicative of what we call the diet quality shoppers. And if you look at that over time, you have a nice picture of what it is that households bring into their house. And we know that about 60% to 70% of the calories that people eat, almost certainly 70% of the calories that children eat, come from the retail grocery stores. We think it's a useful source of data if we could follow it over time. We can perhaps detect important healthy or unhealthy patterns, and then try to find a way to improve it.

Julie: And you've actually come up with a way of monitoring exactly this kind of data.

Dr. Hurdle: Julie, there are a lot of tools out there a lot of people would be familiar with. My Fitness Pal where you have an app where you enter the foods that you eat. There are other systems that are based on grocery stores where you put stickers on food, sort of rank individual cereals, for example, like this one is more healthier than that one. There's nothing necessarily wrong with that except that it puts a lot of burden on the shopper to have to examine...find the best item of everything that they're buying and it's just looking at a very narrow snapshot.

One of the beauties of our tool, QualMART, is they can look at the entire market basket and do so over any length of time. You can do that without asking anything special of the shopper except to shop.

Julie: So how does your tool go about measuring how healthy the things that you're buying from the store are?

Dr. Hurdle: So we figured out a way to map those little barcodes that appear on all the products that you buy into various nutrition databases at the USDA. Using that, we can use the Dietary Guidelines for Americans, which is what the USDA is recommending the people eat to maintain a healthy diet, and then assess the quality of a market basket for household or thousands of households over time.

Julie: So your tools can be a way for families to track the healthfulness of what they're buying at the grocery store, with the hopes of being able to change bad habits over time, I would think.

Dr. Hurdle: I think the first step, and it's probably a baby step, is to have the household look at their own patterns just like we do now when we get an electric bill. Not sure about you, but when I get an electric bill, it shows me how I'm doing over the year and how that compares to my neighbors and how that compares to my really efficient neighbors. One could imagine the same thing with food, and that's a very self-motivated approach. Because we know the kinds of foods that these households prefer to eat and we know the gaps and what they are eating now, we envision building a recommendation system that would move that one more step beyond just self-motivation and actually say, "If you bought more of this and bought less of that, your overall food quality would be even better." But I would like to stress, we're only envisioning doing this with families that opt-in to a program.

Julie: And I could imagine that in partnering with the grocery store, you can take that a step further rather than just having a list of recommendations. You can have coupons that encourage people to buy more leafy greens.

Dr. Hurdle: Exactly right. It's both good for their family, it's good for the store. It's this one plan we've been discussing with University of Utah; health plans where we actually would provide rewards program if you can maintain a consistently more healthy diet than you used to have. Maybe we can send you a shopping card for the grocer and you could buy even more stuff, so you would have both the incentive in terms of the card as well as the information you would need to act in a more healthy shopping way.

Julie: And you've developed a partnership with the grocery store chain, Smith's. Why did they decide to become involved?

Dr. Hurdle: It's just very much part of their mission; watch out for the health and welfare of their customers. I think, it's mainly altruistic, but it's also good business if you can convince customers that you're interested in their health, and that might encourage them to come more to your store.

Julie: You had told me about one project; that you're performing with the Diabetes Prevention Program here and I think that's a really good example of how you can use this type of information.

Dr. Hurdle: That's right. You have a good memory, Julie. So we are partnering with Dr. Tim Graham and Dr. Julie Metos who are running a Diabetes Prevention Program. So they're seeing if they can find ways to motivate through improving exercise and diet these pre-diabetic individuals. So they have all clinical markers for a person who may tip over into full-blown adult-onset diabetes, which is a huge problem in this country. So we are planning on recruiting a subset of those patients, get them to opt into our program. And of course, they have to be primarily shoppers at Smith's because that's our local contact at the moment. And we will be able to tell Dr. Graham and Dr. Metos just how much effect they had, if any, on the diet, at least, what people are buying. So we would look back at what they did buy, what they bought during the year of the program. Then we could follow them after to see if that change persists and that's evaluation, which is pretty much impossible to do any other way.

Julie: The beauty of all this is that this data already exists. You're just capturing it in a different way and processing it in a different way so that you can get this valuable nutrition data out of it.

Dr. Hurdle: That's right. That's why this data is so appealing, because it's there anyway. It's cheap to collect. So we would like to see wellness plans sign up and as soon as people are getting incentives to go to a particular store, that would be good for the store because it would tend to drive customers into their doors. And our long-term goal is to recruit a whole variety of grocery store chains so that we can scale this up. This, I think, is something that could profit real people in real a demonstrable way and as you said, the data is there, so why not try to find a way to use it?

Announcer: Interesting, informative, and all in the name of better health. This is The Scope Health Sciences Radio.

The burning platform in health care is that the health care system really is not working. In fact, to even call it a system does an injustice to the word system. In the U.S. we spend perhaps 50% more per capita than anyone else in the world, and currently we have worse outcomes. So this is not a good situation when we're spending more than $3 trillion a year, 18% of our gross domestic product in an industry that is basically sick.

This is work I'm doing in collaboration with my colleague and friend, Michael Porter at Harvard Business School. Michael's a professor of strategy, and in fact, founded the field and he got interested in health care 10 to 15 years ago because he wondered, "How do you have a system like this sustain itself in such an inefficient way?" As he sort of threw it, he realized that the fundamentals for healthy competition, which ultimately is what drives value to consumers, does not exist in health care that we don't even agree on what goal of the system is.

So Michael has proposed, and I fully agree, that the basis for organizing the health care sector is around value, but we have a very specific definition of value is that for each medical condition, how do we improve the outcomes that are delivered and experienced by the patient and how do we lower the cost of delivering those outcomes, which doesn't seem revolutionary. But it is revolutionary because no one has defined the problem and the solution in that way. They've been all around this thing, how do we use checklists, how do we use lean and process improvement, how do we deliver better outcomes to patients and lower the cost of doing that?

So you think about a $3 trillion a year industry in the U.S. that doesn't know what its outcomes are and doesn't measure its cost, that explains a lot about the system that we have. And I got involved because I have developed a technology of how to measure cost and Michael's developed an approach on how to measure outcomes.

And then there's a third part, is when you have outcome measurements and you have cost measurements, you can reimburse providers in a very different way and right now, providers are reimbursed, in the U.S. mostly, by fee for service, and that rewards volume of stuff but not value, not good outcomes because the fees are independent of what outcomes get produced. And in fact, if you don't do a very good job and the patient has to come back for more work, you get more revenues. That's not a very good way of paying for health care.

So we're advocates of an approach that's beginning to come in called bundled payment reimbursement where you make a single payment to cover the treatment to entire medical condition. But we are actually doing it with [inaudible 00:03:04] flavor which is base it on, make the payment contingent on delivering good outcomes, which seems normal because if you buy a car and it doesn't work, you return it because it doesn't work. Every other industry that straight forward in health care, this is revolutionary. My role in this agenda primarily at this stage is the costing element and what I found does exist is that costs are not measured well in health care at the specific medical condition level.

Obviously, hospitals know how much they're spending. They can trace some of their spending to various clinical departments like orthopedics and cancer care, neurosurgery. They can't take the cost of that department level and drive it down to the patient level treating the patient for the clinical condition. I've helped to develop an approach called time-driven activity based costing which enables that assignment of expenses to be done well at the patient level. It just consists of really two types of two estimates and one of them is to understand the clinical pathway that is used to treat a patient with a medical condition.

What actually happens when the time they show up with a complaint until you diagnose it, treat it, through the surgery, the recovery and then the therapy and release them. What are all the clinical and administrative processes that are performed to get the patient through the entire cycle of care? And when we do that we try to find out at each process step, who is doing it. We can actually calculate how many minutes per year that a person or piece of equipment is available for patient work. That typically comes out to be 90 or 100,000 minutes a year. As you divide their annual expenses by their capacity, you get a cost per minute for every single individual.

And what's really fascinating about health care is the variation and cost rates. Until we have the MDs, whose cost rates could be somewhere between three and a half to six or eight dollars a minute. That would be typical rates, but you have a nurse, might be a dollar or a dollar and a half a minute and you have clinical assistants or scribes or receptionists for around $0.50 or $0.60 a minute. So you get this 10, 12 to 1 variation between the least highly and the most highly paid person which is really interesting because it says even if everybody is busy all the time and they're doing work, if you have a $5 a minute person doing something, that an 87 minute person could do equally well or slightly less efficiently, that's an inefficient use of the resources.

So there's an expression in medical care called have people work at the top of their license. But they don't really see the rationale or the economics behind it until you get to this caustic procedure. So combination of . . . they would take the process times for each person multiplied by their cost rate, cost per minute that tells you the cost of that person or the process step, and we just add it up across all the process steps over the whole cycle of care, and that tells you the total cost. This is not a quantum mechanics or genomic sequencing, it's really simple. What makes it difficult is just there's so many process steps.

So Michael and I are sitting there at the Harvard Business School four or five years ago and we realized, okay, this looks like this could work. But how do we get people other than the two of us to realize that this really could work in practice, there's a lot of good theories that don't work in practice. I said, well, the only way we're going to do this is to test it in practice and so we have to find pilot sites that are willing to try this out because we think we'll be able to create the change by demonstrating that it works. We want to work with leading providers. We're really excellent and they want to be better and they also want to prepare for the new environment where there will be fewer resources available to pay for health care.

And so the ones that are proactive and thinking ahead that they realize that this very forgiving fee for service environment will phase out over the next five years and they want to be ahead of the curve and see what can replace them. Currently, we're working with at least two dozen organizations and they're really distinguished organizations. They are leaders, the Boston-based hospitals and Yale Clinic and Cleveland Clinic, then about six or eight months ago, got contacted by Dr. Vivian Lee here at the University of Utah Health Center and Dr. Lee is a great visionary leader and those are the kinds of people I like working with.

So starting in January, I and my team came out here, we presented the ideas and we formed five teams to work on applying the time-driven ABC in three clinical areas and two support areas. We had the benefit of Dr. Lee, had already put into motion, with some terrific staff here, an emerging system called value-driven outcomes. We're getting outcome measurements very well at the clinical level as well as tracing starting to trace costs to the clinical level as well. And we had to just refine the costing module within that and to build upon this value driven outcomes approach that was being implemented.

It's been a great place to collaborate. We have the opportunity, the UHC has the opportunity maybe with some of our systems to really be one of the national leaders in implementing this value-based health care delivery approach. And so if we can use this as a shine, light, the example, this really can be done and it can deliver major improvements, and patients are better off and we're doing this at lower cost, people will see that this is really a successful way for transforming the way we deliver health care in the United States. There's a lot of skepticism in health care and they'll tell you 200 reasons why any innovation approach won't work and my role is to show it can work.