New tool can help countries measure return on investment from research

Measuring tangible benefits

Research organizations worldwide have long struggled to demonstrate — in empirical ways — how public funding of research results in tangible benefits. Now Canada is poised to take a leadership role in dealing with this issue with the development of a sophisticated tool that measures the return on investment for health research, and potentially other sectors such as energy and even the social sciences. This new tool or framework is the result of the first expert panel report from the Canadian Academy of Health Sciences (CAHS).

Fifteen months in development, the massive 500-page report details how the framework can be customized by organizations to determine the impact of the funding they receive, providing a potent tool to respond to governments' demand for increased accountability.

The expert panel was comprised of 13 mainly international experts and chaired by Dr Cy Frank, a professor of joint injury and arthritis research at the Univ of Calgary and executive director of the Alberta Bone and Joint Health Institute. Frank says Canadian funding organizations that have seen the report are enthusiastic at the prospect of having a common approach to determining ROI.

Entitled Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research, the $500,000 report is the first expert panel report not conducted by the Canadian Council of Academies (CCA) since its formation three years ago. The expert panel reviewed ROI models developed in other nations, synthesized the world's literature on the subject and commissioned seven papers to come up with the best framework for Canada

The framework is an enhanced version of the existing Payback Framework developed in the 1990s by Dr Martin Buxton, a professor of health economics at Brunel Univ in London UK and a member of the CAHS expert panel.

"This could be the beginning on an international exercise if it's used globally. That would put this group (the expert panel) in a leadership position," says Frank. "The majority of panel members are international. They have networks of people internationally and have links into other international exercises ... If this report helps advance the field, we'll all feel gratified."

Once the panel agreed upon adopting the Payback Framework, it sorted through an exhaustive list of more than 300 indicators, whittling it down to 66 which will be used to initially populate the framework. Frank says the list of indicators will eventually be added to as others are validated and adopted.

Frank says the beauty of the model is its flexibility and potential for use in different types of organizations in various sectors. It can be used to demonstrate ROI for advancing knowledge, capacity building, informing decision making, health benefits or economic and social benefits. The CAHS model is designed to provide ROI insight into any of the four pillars of health research: biomedical, clinical, health services and population and public health.

"We had the evaluators for most of the organizations in the country together and they were very enthusiastic. They think this is the answer to their dreams. They understood the model very quickly and we got a lot of positive feedback on it," says Frank. "This report takes the model to a new level of granularity and practical application."

Nearly two years in the making, the panel's development of the Payback Framework comes at particularly opportune time when the federal government is emphasizing increased accountability. The 2007 S&T strategy called for improved metrics and international benchmarking — tasks the model is especially suited for.

Senator Wilbert Keon says there's another reason the report is timely.

"(US) president (Barack) Obama said yesterday (January 20) there will be a huge stimulus for science in America and there will be one for Europe also," says Keon. "The prime minister (Stephen Harper) has already indicated that he is very anxious to keep up with America ... I think there will also be a major stimulus for research in Canada and it's extremely important that we have this framework to measure what we're accomplishing. It's a giant step forward."

The CAHS has already started on its next expert panel report, entitled Health System Transformation to Meet the Challenge of Chronic Diseases. Several other assessments are in various stages of development including access to oral health care, the interaction between industry and academia and the issue of research integrity.

The Royal Society of Canada has also indicated that it will soon return to the task of undertaking expert panel assessments. Although both are members of the Council of Canadian Academies, they do not have access to any of its funding.

"Unfortunately, the model that was essentially prescribed by the federal government when CCA was formed makes it a separate entity." says CAHS president Dr Martin Schechter, professor and head of the department of health care and epidemiology at the Univ of British Columbia. "But in our view, there are a lot of other stakeholders in health other than the federal government, particularly the provinces."

R$