Search for tag: "data"

Interviewer: What's becoming loud and clear is that most scientific studies are, well, unreliable. But there's help out there for scientists. We'll talk about that next on The Scope.

Announcer: Examining the latest research, and telling you about the latest breakthroughs. "The Science and Research Show" is on The Scope.

Interviewer: I'm talking with Darell Schmick, research librarian at the Eccles Health Sciences Library at the University of Utah. So there are a number of reasons for what some people are calling a research reproducibility crisis, including fraud. But even scientists with the best intentions are at risk for doing sloppy work and there are a lot of reasons for that as well. One of the things that you're interested in looking at is data management. I really like this example of the Postdoc who leaves the lab.

Schmick: So it was it an age-old issue where you do a lot of work and it happens to be on your personal computer. It happens to be in a folder with poorly-named files and you produce a bunch of research on behalf of the institution, but then, once you're done, you obviously take your computer with you, right, and then head off to that next position. Now, that seemingly innocuous, however, has a lot of implications. The data that you produce on behalf the university has ownership concerns. Is it the Postdoc's? Is it the University's?

Interviewer: How can that lead to issues with research reproducibility?

Schmick: So if the Postdoc takes all that data with him or her and hasn't been saved into the department bio or anything like that, how can you ensure that you have records of all the work that Postdoc has done? They could have taken just a little bit of it, they could have taken a substantial chunk of it. And it really leaves the PI as well as the rest of the members of the lab with potentially a significant disadvantage.

Interviewer: So there's a lot of knowledge that can be lost?

Schmick: Absolutely.

Interviewer: So what are some ways to avoid that?

Schmick: We do teach a research administration training class that talks about just the basic fundamentals of just good data management, which involves things like where to properly back up your files and how often to do that. Myself and a couple of the librarians on campus have been working on a pilot for electronic lab notebook technology.

So if, say for instance, you happen to have the perennial issue of lab members recording data in their own personal computers because it's inconvenient to share it, this sort of technology allows for a lab to share in a collaborative notebook technology something that has all those questions that we're talking about answered, like how frequently will it be backed up? Is it going to be backed up in a trustworthy source?

Interviewer: Right. Well, and not to mention that most lab notebooks that I've seen are kind of a disaster.

Schmick: Are you saying that scientists don't uniformly have amazing handwriting?

Interviewer: Exactly. Right, we're all human. And even how the information is recorded is varied from person to person. So the idea is that this would become more standardized?

Schmick: You hit the nail on the head there.

Interviewer: Any other approaches to make their data more accessible or reliable?

Schmick: When we're talking about optimizing the mechanics of anything in the research process, we want to ensure that we're doing it in a way that is not only accessible by us because we can look at our notes, presumably, and be able to understand what we were saying, understand what we were recording, understand what we were encountering in that process. But to think about how the results that you're producing are going to be read by somebody else that's not in that same context. So if you're doing an experiment, you do it for you, but you also ensure that you're doing it in a way that if somebody wants to reproduce that experiment, they can do that.

Interviewer: You've talked about ways about preserving information within a lab group, for example, a research group. What about sharing information more broadly with the scientific community?

Schmick: That's a great question, Julie. And a lot of people think that all you are able to really produce is that end product, that finalized article. And we don't realize, many times, that when we're doing experiments, when we're producing all this data that we're talking about, that data could be good data. It could be good information and good intel for another scientist that's stumbling across that same issue.

If you're embarking on answering a research question and you come across a dataset that has already sought to ask that question, you find out that maybe those results weren't satisfactory enough to produce something into a finished article, that could potentially save you years in otherwise reinventing the wheel.

So another thing that we like to talk about is the idea of ensuring that researchers know that the data that they produce is of value and there are places that you can store that. So one example that comes to mind is figshare. And figshare is a repository that you can actually assign a DOI to the data sets that you're uploading on there. Figshare are all about open science so they say as long as you're making it public, I mean, you can upload it for free.

Interviewer: So how can sharing data with the scientific community help with research reproducibility?

Schmick: There's a lot of news as of late in the way of that openness toward science where folks on a peer review panel want to see the steps you were able to take in order to draw the conclusions that you were able to take or able to make. And if they're able to go ahead and see that data right there from the start, it answers all those questions.

It's when things like that data being withheld presents a larger problem not only for you as author but greater implications for science in general. When we start to withhold that data, when we start to conceal certain steps in that recipe toward what we ended up with the final product, it leads towards a slippery slope of was it not open science and that closed scholarly environment, I think, is something that is well worth fighting against.

Interviewer: Where can people go to learn more about best data management practices?

Schmick: There are a lot of places. If you're embarking on a data management plan there's a great tool called DMPTool. That's dmptool.org, and it'll take you through the steps of the processes of, "Let's take you through the steps of the data management process." By asking you in a 20 questions format, "What's this data going to be for? Which agencies are going to be seeing this data?" And it'll give you recommendations at the end from there. If you're at the University of Utah's campus, I'd encourage you to talk to me or one of our other fine staff at Eccles Health Sciences Library. I'd be delighted answer any questions regarding data management plans.

Announcer: Interesting, informative and all in the name of better health. This is The Scope Health Sciences Radio.

The burning platform in health care is that the health care system really is not working. In fact, to even call it a system does an injustice to the word system. In the U.S. we spend perhaps 50% more per capita than anyone else in the world, and currently we have worse outcomes. So this is not a good situation when we're spending more than $3 trillion a year, 18% of our gross domestic product in an industry that is basically sick.

This is work I'm doing in collaboration with my colleague and friend, Michael Porter at Harvard Business School. Michael's a professor of strategy, and in fact, founded the field and he got interested in health care 10 to 15 years ago because he wondered, "How do you have a system like this sustain itself in such an inefficient way?" As he sort of threw it, he realized that the fundamentals for healthy competition, which ultimately is what drives value to consumers, does not exist in health care that we don't even agree on what goal of the system is.

So Michael has proposed, and I fully agree, that the basis for organizing the health care sector is around value, but we have a very specific definition of value is that for each medical condition, how do we improve the outcomes that are delivered and experienced by the patient and how do we lower the cost of delivering those outcomes, which doesn't seem revolutionary. But it is revolutionary because no one has defined the problem and the solution in that way. They've been all around this thing, how do we use checklists, how do we use lean and process improvement, how do we deliver better outcomes to patients and lower the cost of doing that?

So you think about a $3 trillion a year industry in the U.S. that doesn't know what its outcomes are and doesn't measure its cost, that explains a lot about the system that we have. And I got involved because I have developed a technology of how to measure cost and Michael's developed an approach on how to measure outcomes.

And then there's a third part, is when you have outcome measurements and you have cost measurements, you can reimburse providers in a very different way and right now, providers are reimbursed, in the U.S. mostly, by fee for service, and that rewards volume of stuff but not value, not good outcomes because the fees are independent of what outcomes get produced. And in fact, if you don't do a very good job and the patient has to come back for more work, you get more revenues. That's not a very good way of paying for health care.

So we're advocates of an approach that's beginning to come in called bundled payment reimbursement where you make a single payment to cover the treatment to entire medical condition. But we are actually doing it with [inaudible 00:03:04] flavor which is base it on, make the payment contingent on delivering good outcomes, which seems normal because if you buy a car and it doesn't work, you return it because it doesn't work. Every other industry that straight forward in health care, this is revolutionary. My role in this agenda primarily at this stage is the costing element and what I found does exist is that costs are not measured well in health care at the specific medical condition level.

Obviously, hospitals know how much they're spending. They can trace some of their spending to various clinical departments like orthopedics and cancer care, neurosurgery. They can't take the cost of that department level and drive it down to the patient level treating the patient for the clinical condition. I've helped to develop an approach called time-driven activity based costing which enables that assignment of expenses to be done well at the patient level. It just consists of really two types of two estimates and one of them is to understand the clinical pathway that is used to treat a patient with a medical condition.

What actually happens when the time they show up with a complaint until you diagnose it, treat it, through the surgery, the recovery and then the therapy and release them. What are all the clinical and administrative processes that are performed to get the patient through the entire cycle of care? And when we do that we try to find out at each process step, who is doing it. We can actually calculate how many minutes per year that a person or piece of equipment is available for patient work. That typically comes out to be 90 or 100,000 minutes a year. As you divide their annual expenses by their capacity, you get a cost per minute for every single individual.

And what's really fascinating about health care is the variation and cost rates. Until we have the MDs, whose cost rates could be somewhere between three and a half to six or eight dollars a minute. That would be typical rates, but you have a nurse, might be a dollar or a dollar and a half a minute and you have clinical assistants or scribes or receptionists for around $0.50 or $0.60 a minute. So you get this 10, 12 to 1 variation between the least highly and the most highly paid person which is really interesting because it says even if everybody is busy all the time and they're doing work, if you have a $5 a minute person doing something, that an 87 minute person could do equally well or slightly less efficiently, that's an inefficient use of the resources.

So there's an expression in medical care called have people work at the top of their license. But they don't really see the rationale or the economics behind it until you get to this caustic procedure. So combination of . . . they would take the process times for each person multiplied by their cost rate, cost per minute that tells you the cost of that person or the process step, and we just add it up across all the process steps over the whole cycle of care, and that tells you the total cost. This is not a quantum mechanics or genomic sequencing, it's really simple. What makes it difficult is just there's so many process steps.

So Michael and I are sitting there at the Harvard Business School four or five years ago and we realized, okay, this looks like this could work. But how do we get people other than the two of us to realize that this really could work in practice, there's a lot of good theories that don't work in practice. I said, well, the only way we're going to do this is to test it in practice and so we have to find pilot sites that are willing to try this out because we think we'll be able to create the change by demonstrating that it works. We want to work with leading providers. We're really excellent and they want to be better and they also want to prepare for the new environment where there will be fewer resources available to pay for health care.

And so the ones that are proactive and thinking ahead that they realize that this very forgiving fee for service environment will phase out over the next five years and they want to be ahead of the curve and see what can replace them. Currently, we're working with at least two dozen organizations and they're really distinguished organizations. They are leaders, the Boston-based hospitals and Yale Clinic and Cleveland Clinic, then about six or eight months ago, got contacted by Dr. Vivian Lee here at the University of Utah Health Center and Dr. Lee is a great visionary leader and those are the kinds of people I like working with.

So starting in January, I and my team came out here, we presented the ideas and we formed five teams to work on applying the time-driven ABC in three clinical areas and two support areas. We had the benefit of Dr. Lee, had already put into motion, with some terrific staff here, an emerging system called value-driven outcomes. We're getting outcome measurements very well at the clinical level as well as tracing starting to trace costs to the clinical level as well. And we had to just refine the costing module within that and to build upon this value driven outcomes approach that was being implemented.

It's been a great place to collaborate. We have the opportunity, the UHC has the opportunity maybe with some of our systems to really be one of the national leaders in implementing this value-based health care delivery approach. And so if we can use this as a shine, light, the example, this really can be done and it can deliver major improvements, and patients are better off and we're doing this at lower cost, people will see that this is really a successful way for transforming the way we deliver health care in the United States. There's a lot of skepticism in health care and they'll tell you 200 reasons why any innovation approach won't work and my role is to show it can work.

Ninety percent of the data that currently exists in the world is less than two years old, which means everything is new. For the majority of physicians, what they read, is two or three journals that they subscribe to or think are most relevant to their practice. In 2010 the National Library of Medicine cataloged 700,000 new articles. There is information there that would be helpful to you that you never will read, see, learn, or memorize.

We need support such as Watson and other analytical tools because the amount of information that is out there, it overwhelms us. We can't even conceive of it let alone use it. So Watson is a computer system that reads and understands English, which is a major advance. There are lots of tools that can recognize certain words or patterns of words, but Watson actually understands the meaning of an English statement or an English document.

Any clinician patient-relationship is a key to the future of health care. Disrupting that would be distinctly disadvantageous. Patients really aren't interested in having tools make decisions for them but providing them advice or ideas, along with their clinician, to help both of them make a better decision.

So that's what Watson does, it understands the nature of the question to be addressed then does all this reading and says, "'These are the important concepts, I think, for you to consider in making your decisions."' That's the key part, it helps you decide by giving you information that you can use with your skills and expertise to make it more likely to have an evidence-supported decision.

So that's where these algorithms come in. Recognizing that in this complex data, there are patterns and clues and signs that we can use to make better decisions that we can't process ourselves. At the heart of Watson is hundreds and hundreds of such algorithms.

In organizations such as the University of Utah are developing this algorithmic approach because we know that if we are going to use that data effectively we have to develop the mathematical and computer tools that collect and process that information for us and present it to us in a way that we can use when the mass amount of data is something we can't use.

I think the regulatory agencies can use their platform to help inspire science and development of new tools that will provide better, safer, more effective, higher quality products, which is really what we're trying to do in terms of getting good therapies to patients.

The major barrier to entry in the market is the cost and the time that it takes to bring a discovery that's made in the laboratory actually to the clinic.

On average, right now, to bring a new drug to the market, it's probably somewhere between one and two billion dollars. If you've made a $2 billion investment in a new product, what do you have to price it at in order to get your return on investment?

If we were to try to support cures for all the orphaned diseases, it would consume the entire U.S. GDP. Clearly, we need to do a better job in driving down development costs so that we can drive down what these products are listed at in order to not completely break the healthcare system.

All of these different trends that are happening in the industry, the kind of merger and acquisition, the loss of capital, the direction that science is driving toward smaller markets, all of this is really converging to a point where this industry is under significant stress.

I think one of the things we need to do is think about establishing a national infrastructure for clinical trials. The other thing we need to think about is how we design smarter, I'll call them leaner, less expensive clinical trials that allow us to get just as much information about a therapy out of a much smaller trial.

We spend little to no effort on actually taking what knowledge we have, in terms of real world performance and things that we know about in prior studies, and integrating it back, or reverse engineering into the discovery or to the next generation product development.

Probably, most importantly, we don't include the patient and the consumer perspective. What are their opinions? What are their experiences, which are going to be critically important.

I think the academic medical centers are the drivers, or the engines of discovery and innovation, in terms of bringing basic fundamental mechanisms of knowledge of disease forward.

I think what academic medical centers could do, that might improve the situation, is take those findings a little bit further down that path of product development.

One of the problems, of course, is that they're not funded to do so. So the NIH isn't really funding product development, and that's something that I think, that the FDA is trying to change through its Regulatory Science Initiative, but it may necessitate other types of partnerships between academic institutions and private industries.

We like stories, we like storylines. When I write a paper, I always write it in terms of a story. Not because that's the way life is, but because that's the way we think. When we started gene targeting what it does is inactivate one gene at a time. No story is one gene. It's always the interaction of many genes together that has a beginning, a middle, and an end.

My end is always understanding. I'm excited when all of a sudden I understand something that I didn't understand before. And that is what makes science exciting, when you know all of a sudden, you see something that nobody else has seen before. My feeling is always push, push, push understanding. Push basic research and translation will automatically follow.

Science is interesting because it entails almost two completely opposite skills. One is flights of imagination. You have to think about things that don't exist and why they don't exist. It's that kind of mentality that allows you to jump into areas where the solutions aren't clear, that they look impossible. You simply have to extend yourself into them and then make it possible. On the other hand, if you want experiments to work, you have to work extremely diligently and pay attention to detail which is a completely opposite skill. A successful scientist has to have both.

Each one of us sees the world from our own eyes because of our training and because of our experiences, and everybody else is seeing it from slightly different eyes. I think the cross mashing of that is where new things can arise.

If you want to really be innovative, you're almost working at the edge of science fiction because at that point people are willing to have flights of imagination. One of the aphorisms that I was raised on, "The difficult we do right away, the impossible takes a little longer." It takes the same amount of effort to work on big questions as little questions. So why not work on big questions?

I think we don't really know what patients want. So this is from the perspective of introducing technology and assuming all our patients are techno savvy, and I think you have to ask the patients what they want and how digitalized they are. A lot of our patients are older and may not adapt as quickly to this technology.

I think we need more data on what patients want, and I think patients need to be brought into the discussion. I think the movement for patient and family centered care is increasing at the same time that this digital stuff is going on, and I think if we bring patients into the conversation, we can do this intelligently and thoughtfully.

I don't know that we've ever, in the course of human history, designed anything that was perfect and could last forever. We need to be ready for change.

As the old joke goes, people will say, "Change is good. You go first." What I see being different now is the people not only acknowledge that change is important and it's good, but more and more of them are willing to go first.

I see groups of institutions forming coalitions that will work on big change, transformational efforts, in their own institution. When that starts to happen, it will ripple throughout academic medicine, and healthcare in general.

The issue isn't competing against each other. The issue is we all have very similar mission statements. We want to improve the health of the public, and that's a big challenge.

Sadly, in many ways, the U.S. healthcare system is broken. We have great medicine. We have great surgeries, great medications and tests. But the system isn't working. And so, more and more what I see the people at this meeting thinking about isn't, "Where do I stand relative to anybody else," but, "Are we innovating? Are we changing ourselves in a way that we really can start to improve the health of the country."

We bring together people from all parts of the medical school, research, teaching, the administration, together with the health system. It's a place where the community really can come together and share ideas. I would argue there's never been a time when we need more new ideas to help us meet the challenges we face.

People who are coming to this meeting are going to leave more ready to push for change, to exert leadership, to argue for innovation on their home campus. All boats will rise together, and eventually we will create a tipping point where I think people will look at academic medical centers in this country ten years from now and say, "We were in trouble when it came to healthcare, and they led the solution.”