New report measures effectiveness and efficiency of university publication activity

Institutional performance varies widely

Not all universities are the same when it comes to generating value for research investment. A new report on Canadian University Publications has developed two new indictors to gauge how much institutions pay for the publications its researchers publish and whether they are getting value for money when compared to other institutions domestically and internationally. These impact and efficiency indictors reveal that, while Canadian institutions rank highly on the world stage, there is a considerable variation between institutions and disciplines.

For the year 2004, the report - published by Research Infosource Inc, a division of The Impact Group and an affiliate to RE$EARCH MONEY - shows that Canada ranked 7th in world scientific publications, after ranking 6th between 1999 and 2002. Canada has maintained a 4.6% share of global publishing output and ranks 4th along with Australia on a per capita output basis with 0.98 papers per 1,000 people. The maintenance of Canada's share is particularly impressive, considering the major gains in publication made by countries like China, which moved from 12th to 6th between 1999 and 2004.

When taken alone, the university sector - which accounts for 90% of Canada's total research publishing output - experienced a 3.6% increase in published articles between 1999 and 2004. During that same period, Canadian universities captured 4.1% of world publishing output. When examined by fields of study, humanities publishing output was 6.1% of the global total, followed by social sciences (5.7%), health sciences (4.6%) and natural sciences and engineering (3.1%).

In nearly 600 pages of data and commentary, the report's analysis of research effectiveness (cost advantage) reveals that many universities or affiliated research hospitals fall below the average for their respective categories. It is this ranking element that is likely to set off a flurry of institution-by-institution comparisons, although the report notes that this is not the intent of the report and should not be attempted without referring to the raw data for publication and faculty counts as well as research income data.

"The real value of the projects is to determine where do we go from here? Do we spend more money, spend it in different ways or in different areas?" says Ron Freedman, the report's author, CEO of Research Infosource and co-publisher of RE$EARCH MONEY. "We need a debate now that we have rejuvenated university research in Canada and we hope this report will inform the debate."

Where the report is likely to be controversial is in the data for publication efficiency, essentially the cost at each Canadian university for producing a publication. The average cost per paper published over the 2001-2004 period is $111,300. Average publication intensity across the system is 0.85 papers per full-time faculty.

UBC MOST EFFICIENT

For the period 2001-2004, the Univ of British Columbia is the most efficient among 16 major institutions with medical/doctoral programs, Simon Fraser Univ is tops for comprehensive universities and the Univ of Prince Edward Island ranks #1 for undergraduate universities. At the bottom end of the scale were the Univ of Montreal (medical/doctoral), the Univ of Quebec and Montreal (comprehensive), and the Univ of Quebec at Abitibi-Temiscaninque (under- graduate).

Reaction based on the report's preliminary data swung wildly from supportive to damning. In the latter, criticism has ranged from dismissal of the methodology to questioning the timing of the report's release as the new federal government is preparing a new S&T strategy. Freedman defends the methodology, asserting that the data are robust and underpinned by sound philosophy. As for concerns that the report's findings may promote the government to cut university research funding or suspend further increases, Freedman says the comments are unfounded or politically motivated.

"There's no way these findings should be taken as an indication that research support should go down. It has too many important benefits, although an increase in publication output is not one of them" he says. "A number of people have said that we should not have come out with this report and in most cases it's an over-reaction on their part. I don't accept those concerns."

FMI: www.researchinfosource.com

R$