Taking a risk on go-for-it research
Can we afford to fund 'high-risk, high-reward' health research when it comes with such a high failure rate? Perhaps more importantly, can we afford not to?
When it comes to funding health research, it's crucial to support work based on sound science that can increase knowledge, improve health and diminish disease. At the Canadian Institutes of Health Research (CIHR), peer-review panels ensure that grants go to exceptional scientists who are proposing excellent investigations that will have impact.
But does this strategy produce the best results? Does investing in projects that have the best chance of success lead to the kind of breakthroughs needed to solve the health puzzles of the 21st century? Or by avoiding high-risk, high-reward research, are we limiting ourselves to slow-and-steady, incremental advances? If so, how do we, as Canadians, support high-risk research that has the potential to hit home runs but comes with a very high strikeout rate?
In this case, risk does not refer to any potential risk to patients. It refers to the gamble of taking speculative leaps into unknown territories of scientific investigation, and the possibility that an experiment may fail or a hypothesis may be proven wrong.
Is it important to fund research that comes with a large potential for failure but also holds the possibility of amazing breakthroughs? How can such speculative work be prudently supported? In June of 2011, we asked two of Canada's leading health researchers – one in biomedicine, the other in health systems and services – to share their thoughts on high-risk, high-reward research. Their exchange touched on a few ideas that also emerged in the talks surrounding CIHR's proposed design reforms.
Dr. Brett Finlay of the University of British Columbia is an internationally renowned leader in the field of microbial pathogenesis and a member of the CIHR Governing Council. His leading-edge research focuses on the interaction between host and pathogenic bacteria and explores links between the state of gastrointestinal microbes and a range of health issues including diarrhea and asthma.
Dr. Steve Morgan is Associate Director of the Centre for Health Services and Policy Research at the University of British Columbia. He is also a leader of the CIHR-funded Pharmaceutical Policy Research Collaboration, a national network of university-based experts committed to conducting and communicating policy-relevant research with integrity. Co-author of The Canadian Rx Atlas, he puts a high priority on experimenting with various forms of knowledge exchange to inform and improve pharmaceutical policy.
Question: In researching the causes of disease and ways of preventing or treating it, sometimes there is a failure – or maybe just a reluctance – to look beyond the usual suspects. A recent op-ed piece in the New York Times pointed out that when AIDS first emerged many leading, peer-review medical journals published flawed papers. It cited a December 1981 New England Journal of Medicine editorial that never considered the possibility of a previously unknown microbe as the possible villain. Is there a failure among the science establishment to think outside the box?
Finlay: It's part of the culture, wrapped up with how you get your work funded. My lab has experienced this, when we proposed that strange microbes cause diseases. You get laughed at. Look at Dr. Barry Marshall who investigated the bacterium Helicobacter pylori and stomach ulcers, and won a Nobel Prize in 2005 for it. He was a laughingstock until he turned out to be right. Grants and funding are based on progress and there is no mechanism right now to provide funding for creative, out-of-the-box, go-for-it proposals. Those are the ones that get slashed first. The way I drive research is to treat it like an investment portfolio with lots of blue-chip stuff with guaranteed papers. But we're also going to have some mining stock equivalents that are high flyers with riskier ideas. I'm fortunate because I have enough grants that I can carve off pieces to go try things. If you find something novel, once you have enough data and you have shown it works, you can apply for funding. It's kind of a backwards way of doing it, but the whole culture and the way it has developed to get funding is not conducive to high-risk research.
Question: Regarding that culture, Michael Pollan, in his book, In Defense of Food, points out how, since the 1950s, researchers have focused lots of attention on cholesterol largely because it was the only factor linked to heart disease that they had the tools to study. He talks about 'parking lot science' based on the notion that when a person loses their keys in a parking lot they first look for them under the streetlight, not because they lost them there but because that's the easiest place to see. In your field, do you see that kind of parking lot science?
Finlay: Until recently, science was driven by what you physically could do with limited techniques and limited technologies. That has changed since the explosion of molecular biology and genomics. Now I say to my students, 'It's not what can you do – it's what should you do.' That, actually, is much, much harder. Because, in molecular biology, you can do nearly anything these days. The whole next generation of scientists is coming up into this and they will need to have creative ideas.
Question: There is tremendous pressure, even on those young scientists, to show a return on the investment made in their work. How do you approach the idea of doing riskier research that may not yield immediate results?
Finlay: That's a huge issue. If you're up for tenure, you've got five years to make or break it. You're not going to be trying the weirdest, wackiest longshots. High-risk research, like interdisciplinary research, is not something you're going to really tackle until you're more stable in your career – mostly because of the way the system works.
Morgan: It's a bit like the opposite of investing for retirement. Early in a research career, you're on the tenure clock; you need to publish and to meet deadlines; and, therefore, you need to have some level of security. Later on, post-tenure, you can start to take on more risks because your career progression will not be as hamstrung by failure. To me, this suggests a need to invest in people, labs and teams where high-failure-rate activity is coordinated by mid- to senior-level academics mentoring junior investigators who thereby have access to the full research portfolio without taking the big hits on the losses.
Question: Have there been points in your careers when taking on more risk meant that you had to minimize or modify your research plans because you couldn't secure funding for work that you felt was important to do?
Finlay: Well you can't get risky work funded, so you just get the blue-chip funds and use the money to do risky work. When I write my grants, I already know the answer to many of the proposed things. We're moving on. It's really hard for anyone to argue when you've already got powerful preliminary data. We're always looking at where we can push the envelope next with the funds we get. But that's a hard position to get to in a career.
Morgan: In my field of health policy, it's more difficult to leverage the grants that you hold to do high-risk investments. That is due in part to the fact that you can't readily pre-do the studies that we might be planning. Moreover, our research community has an intensely self-critical culture, arguably more so than in the basic or clinical sciences. Health services researchers or social scientists are risk averse. There is a culture of almost stifling some innovation because we are not willing – particularly at peer review – to accept the fact that the study may not be perfect even though the potential impact might be great if it succeeds. Overcoming that culture may require new mechanisms to fund research so that the peer review process can't get bogged down in, 'Let's just go with what we know is going to succeed ... asking the same old questions with the same old methods to get the same old answers.'
Question: That's disconcerting because isn't policy research the one area where you really should take a flyer now and then?
Morgan: There are arguments for it, for taking some big risks and big gambles on certain kinds of health services and health policy work. You can imagine the potential benefits of trying to experiment with different ways of organizing and delivering health services. But it is high-risk research. There should be some mechanism for this and for other types of research that are innovative in terms of transforming how we think about policy and health system problems. With regards to the 'parking lot science' you referred to, in our field a lot of what people do is quite basic research ... the kind of stuff that shines a light before you search for your car keys. It's work to understand fairly basic ideas and phenomena such as behavioural responses of actors in health systems or organizational theory and change management. There is a lot of this kind of policy work that might have important impacts but it's longer term than the discovery of a new technology that can be put into place rapidly.
Question: So it sounds as though, in your experience, there is too much emphasis on safe research that will produce predictable results, and that to do the higher-risk research that might produce amazing results really requires a bit of a work-around approach. Would that be accurate?
Finlay: Everyone wants to hit the home run. But it's usually unsuccessful. We all have tried many, many things and if they had worked we would have been famous a long time ago. It's not for lack of trying. You do the safe stuff because that's how you get the money to try other things. I can list 100 cool ideas off the top of my head, but I don't know which one is going to work.
Question: What are your thoughts on how research should be funded to allow high-risk investigations?
Finlay: There is no way you can adjudicate it, if you believe in peer-review. I personally think there should be more emphasis on the person. If you've got a proven track record and you've published 20 papers a year for the last 20 years, you're going to keep doing that. So why not give that person a bit of freedom and say, 'OK, go try something; you don't need more papers at this stage of your career so why not take a flyer?' If you look at Howard Hughes Medical Institute's (HHMI) Janelia Farm concept, they give people 10 years to do something and they hope to generate Nobel Prizes from it. That's a very risky proposition. But I think research is both the stuff you do and the person you are. It's been said that if you pick good people, good things are going to happen.
Question: The HHMI's approach is based on that idea isn't it, investing in the investigator?
Finlay: Not quite. I've sat on their selection committees and it's the investigator and what they've done. It goes hand-in-hand. You're picking really good people who do really good research. It's the whole program. I like that idea and that concept.
Morgan: Yes, but it should not just be throwing money at the rich so they can do what they already do very well, the same-old, same-old stuff. It has to be clearly framed: 'We need you to think differently.' If you're going to invest in programs and people and tell them, 'Here's 10 years of funding,' the spirit of it should be that it's for game-changing work. The expectation should be that that investigator is going to get off the 'paper mill' and stop producing 20 slice-of-bologna papers a year and start taking more risks. You should also document and celebrate the failures in those risky investments – because that's part of the process of generating real innovation – in the hope that over the 10 years you observe a positive success rate trajectory.
Finlay: Let me give a personal example: my whole career has been based on studying how bacteria interact with mammalian cells and cause disease. A few years ago, I was getting frustrated because we weren't getting anywhere in terms of the actual disease. We were getting beautiful papers about molecular mechanisms, but it had nothing to do with disease. I took my lab for a retreat on Galiano Island. We had a very serious brainstorming session about where we thought the field should go. One of the things that came up was, 'OK, these are diarrhea-causing bacteria. They're in the gut, which is full of microbes. Could the microbes be contributing to this?' Nothing had been done on this. We thought, maybe we should embrace the concept of investigating not only the bacteria in the host but also the microbiota (naturally-occurring microbes in the body). So I put in a grant application to CIHR. Every other grant I'd ever put in had been ranked very well by CIHR panels. I had never not gotten a grant I'd put in. I put this thing in and it got totally thrown out. I said, 'This is so important I'm going to do it anyway.' And, yes, my productivity took a hit; we had fewer papers for a couple of years. But now we are a leader in this field. We've had spectacular findings. It's now translating into the idea that you can shift people's microbiota to make them more resistant to diarrheal diseases in developing countries.
Question: All because you took a chance on a risky idea?
Finlay: It has also led us into the world of asthma. My wife, who's a pediatrician, came home one day and said, 'Brett, kids who get antibiotics in the first year of life have higher levels of asthma.' I convinced a grad student to do an experiment where we treated mice with antibiotics and induced asthma. No funding whatsoever; I did it on the side for two years. I tried the grant route but got laughed out of there. And it's been spectacular. Now we have great data. Now we're working in the asthma field because of this whole new theory that you need certain bacteria early in life; if you don't get them you get allergies and asthma. Once I proved it, funding agencies said, 'Fantastic, here's some more money.'
Question: What's your experience, Dr. Morgan?
Morgan: In my career I've had a lot of success in writing grants and doing traditional, data-driven health services research. That's a 'safe' way of getting funding and papers published. So, in the early part of my career, I never had trouble getting a grant funded. But my interests started to shift; I became tired of describing the problem and more interested in finding solutions. I wanted to do more comparative policy research within Canada and abroad and, most importantly, to bring policy makers along with me when we did that work. It was a big-risk investment to say, 'Wouldn't it be interesting if policy makers from Canada actually talked with policy makers abroad?' Well, proposals for work of that nature failed miserably. It was the first time I'd ever received a grant rating that wasn't even in the fundable range. I got a 1.9 out of a theoretical 4.9. I didn't think scores that low even existed! It was a slap in the face and a clear signal from the committee to say, 'Don't ever bring this stuff back to us again.' Fortunately, CIHR came through with some Emerging Team funding that allowed me to carve off a small component of the funding to do some of this work and, in terms of policy impact, it has turned out to be one of the best investments we have made in better understanding policy and in creating linkages and exchange across the country. Peer reviewers just wouldn't go down that path whatsoever, but I know that the Canadian policy audience is appreciative of what we're now doing. The rewards have been great. I'm absolutely sure it has had an impact on Canadian policy, in ways that I have not had with the more traditional inquiry that we were doing in the past.
Question: How important is failure? To try something and just have it fail on you?
Morgan: It's only important if you learn from it. There is failure in terms of achieving the funding you're seeking and then there's the failure of the project. Many funded projects fail. We should respect that if the investigator has done what they proposed and it failed, it's still a success … provided you extract whatever lessons you can from the failure.
Finlay: Failure is a major part of science. In my area, probably 90-95% of what we do doesn't work. It's the rare exception when something actually works, experimentally anyway.
Question: To use the baseball analogy, if a player gets a hit every fourth time at bat, he is considered to be really good. That means failing 75% of the time. Is the same thing applicable in science?
Finlay: Absolutely. If you're going for groundbreaking work, your failure rate is much higher than that.
Morgan: It's important to remember that you don't want to put just anyone at the plate swinging for the fences. You want the people at the plate who probably have a chance that if they connect it's going to go there. We should not simply throw money at any high-risk ideas. You want to invest in people who have the capacity to actually get that home run.
Question: So it sounds like you both agree that there needs to be some sort of funding mechanism that allows high-risk, high-reward research, but that there is no such mechanism at the moment. Is that right?
Morgan: I would reiterate that it should be part of the overall investment portfolio for an organization like CIHR.
Finlay: It's there now, but it's not named as such. In terms of formalized programs to do really cool stuff, that does not exist. But if you hit a home run, when you put in your next grant it will get funded much more easily. So you are trying to hit home runs – it's just that you don't dare put it in a grant. I always tell my students that there is grant writing and there is doing science and never the twain shall meet. They are separate issues. You get the money and then you try to do really good science.