Environmental watchdog solicits science topics for future AG audits

In-house S&T could be on federal auditor's radar

By Debbie Lawes

If the federal government has made a commitment on anything related to science or innovation, Julie Gelfand wants to know about it. The new Commissioner of the Environment and Sustainable Development — part of the Auditor General's (AG) office — took her fact-finding mission to the Canadian Science Policy Conference (CSPC) in Halifax October 17 to solicit ideas for what may become a series of science audits.

"We want input from you to help us develop a Strategic Audit Plan around science and technology," said Gelfand, who just one week earlier released a scathing report on the federal government's failure to meet its 2020 targets to reduce greenhouse gas emissions, and its lack of marine charts, navigation aids and ice-breaking services for many high-traffic, high-risk areas in the Arctic.

For future audits on S&T, she wants to know "what is working well now; what are the areas that need improvement; what are the biggest risks facing the federal science and technology function; and what criteria would be used to assess how well they (government) are doing."

The AG is responsible for coming up with a plan for three science audits. "We could be looking at a series of audits over time, they could be built up, they could be on different topics, they could be thematic," she explained.

Auditing grants and contributions made to organizations like the research granting councils are "really easy" and straightforward, said Gelfand, who has worked for industry (Rio Tinto, Mining Association of Canada) and NGOs (Nature Canada). "I'm more interested in the harder stuff to audit, which could include things like the Industrial Research Assistance Program (last audited in 1999) or "how well we are doing on science and technology (S&T) strategies, government, how well S&T is governed in Canada and whether or not we can assess that and audit that."

The 1999 audit was conducted within the context of a larger review of the Industry Canada portfolio and its role in fostering innovation. In addition to IRAP, it also focused on the Natural Sciences and Engineering Research Council's Research Partnerships Program, the Networks of Centres of Excellence (NCE) and the former Technology Partnerships Canada program. (R$, Dec. 8/99). In 2002, the AG launched a review of Canada's 1996 S&T strategy but shelved it in 2007 when the Conservative government released a new S&T framework (R$, Nov 12/07 & May 31/07).

A "portrait of unmanaged decline"

Panel member Dr Peter Nicholson suggested an audit of the government's in-house S&T activities is overdue. He described intramural S&T as "by far the most significant weakness in the S&T envelope".

The federal government has reduced its spending on S&T by $1.7 billion or 14.4% over the past five years, according to Statistics Canada (R$, June 16/14). The agency projects that federal S&T outlays will decline for the fourth year running to $10.3 billion in FY14-15 — a year-over-year decline of $587 million or 5.4%. The picture is even gloomier when cast in 2007 constant dollars — $9.8 billion in FY13-14 or back to the same level as FY04-05.

The cuts are spread across the spectrum of S&T-related activities, from data collection and information services to education support, administration of extramural programs and capital. Administration of extramural programs declined 10.6% from $85 million to $76 million.

The cuts have inevitably led to an across-the-board downsizing of federal scientific personnel. In the same five-year period, S&T staff was cut by 3,405 or 8.8% from 39,594 in FY10-11 to 35,189 in FY14-15.

The steady progression of federal S&T expenditure reductions is in stark contrast to assertions by Ed Holder, the minister of state for science and technology and his predecessors Gary Goodyear and Greg Rickford, who have stated repeatedly that the Conservative government is making record investments in science, technology and innovation.

"This is a portrait of unmanaged decline," that began with the previous Liberal administration, said Nicholson, who has held senior policy positions in government and industry. "It really does signal a vacuum of leadership and it's a very serious problem because we definitely need a healthy and well-motivated scientific capacity to support the mandates of government departments and agencies."

These mandates include policy, regulation and knowledge mobilization and while some of these activities can be delegated to universities, NGOs and consultants, Nicholson said "the government still needs first-rate internal, substantive knowledge in order to manage that outsourced process."

CSPC delegate Dr Ron Pelot, the associate scientific director at the MEOPAR network, suggested the AG could audit a Cabinet directive requiring that all memoranda to Cabinet include a strategic environmental assessment to determine whether departments have enough scientific capacity — in terms of people and resources — to provide complete information on the environmental impacts of proposed programs. The Environmental Commissioner's most recent report noted that while processes have improved, there is still a risk that ministers are not getting complete information.

"I don't understand how you can understand risks and deal with them at the government level if you don't have enough people, resources and studies to understand all those issues," said Pelot.

The challenge for Gelfand's office will be separating actual commitments from rhetoric. Nicholson suggested "there's a constant rhetoric about how much we're doing to promote innovation in business. But look at the results — there's a gap. Ask why."

Gelfand said while there's potential for the AG to audit rhetoric, it's better to have a clear, preferably written commitment. "Are there commitments the federal government has made to say we are going to improve innovation? If so, tell us where those commitments are because we need to audit to something we can see and measure."

Conference delegate David Watters, president and CEO of Global Advantage Consulting, agreed Canada's science enterprise is in decline, but questioned if an audit is the right public policy instrument to encourage the type of change that is needed. "How seriously are the results of these audits taken or is there some other mechanism that might be appropriate," he asked.

What can be measured?

Dr Martha Crago, VP research at Dalhousie Univ, questioned if an audit can capture how national science advice is used, and if the research results lead to intended societal outcomes, whether economic, social, intellectual, scientific or cultural.

For example, she points to the NCE which has a mandate to integrate the social sciences into its research programs. "Is one-third of their money being spent on things with social sciences being included in them? … It's not simple to do performance data in the social sciences and humanities," she noted.

Panel member Dr Alan Bernstein, president and CEO of the Canadian Institute for Advanced Research, says there are inherent challenges in conducting an audit of Canada's S&T system. These include identifying a causal link between a research investment and an outcome; the lengthy time lags between that investment and results; understanding the role of other funders in a complex S&T ecosystem; and recognizing that more science is now global.

"I would argue that the most important science done today is international," said Bernstein. "Therefore, something someone would want to measure in a performance audit is how are we doing at participating in global networks."

Another issue worth auditing, added Genome Canada president and CEO Dr Pierre Meulien, is how to measure whether state-of-the-art science is being used when environmental monitoring, food safety or health assessments are conducted. He added that measuring success has to take into account the incentives and impediments to success related to: how science is done ("multidisciplinary is key", including the social sciences and humanities); how collaboration is encouraged (overcoming jurisdictional issues and having the right incentives in place); and the extent of international collaborations.

On the international question, Gelfand said it's possible to conduct an audit involving auditors general from several countries. More generally, she stressed that any topic that's proposed has to be of interest to , as it's ultimately Parliament that decides to hold departments accountable.

R$