UCalgary signs Declaration on Research Assessment

A global initiative to support improving assessment of scholarly research seeks to temper reliance on bibliometrics. The San Francisco Declaration on Research Assessment (DORA) asks academic institutions, funding agencies, and other organizations to re-think their approach to evaluating research, by recognizing that journal-based metrics alone do not accurately assess the quality and impact of research.

DORA encourages a shift toward considering research on its own merits. This can be done by including in assessments scholarly activities that are typically not captured in bibliometrics, such as community engagement, reports to government, research creation, and outputs such as datasets, software, and performances.

UCalgary recently signed the declaration, joining more than 2,000 organizations around the world. Pursuing changes to the way the university assesses research will help to align policies and practices with those of leading funders, many of whom have also signed DORA.

An enhanced focus on research content, rather than on bibliometrics alone, is expected to benefit all types of research, from curiosity-driven inquiry to the most applied forms of investigation.

Canadian signers include SSHRC, NSERC, CIHR, Canadian Foundation for Innovation, Genome Canada, Canadian Association of Research Libraries, Canadian Science Publishing/NRC Research Press, and the Canadian Association of University Teachers (CAUT). Beyond Canada, DORA has also been signed by organizations such as the Wellcome Trust, National Health and Medical Research Council UK, Howard Hughes Medical Institute, Bill and Melinda Gates Foundation, and Swiss National Science Foundation.

“Joining DORA is another step toward more meaningfully evaluating research output and impact,” says Dr. Marcello Tonelli, associate vice-president (research). “Our scholars are increasingly engaged in new forms of knowledge creation, and as a university we have emphasized the need to better demonstrate the impact of our research. Including an assessment of impact along with more traditional metrics will help us to fully recognize the value of their work.” 

The declaration includes two specific commitments which universities are expected to address:

• Be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.
• For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.

Now that UCalgary has signed the declaration, plans are under way to develop a framework and the required tools, platforms, and resources to pursue a new paradigm in research and innovation impact assessment.

“An enhanced focus on research content, rather than on bibliometrics alone, is expected to benefit all types of research, from curiosity-driven inquiry to the most applied forms of investigation,” says Dr. William Ghali, vice-president (research). “Pursuing research and scholarship impact assessment is intended to not only support our scholars in their careers, but also to establish a clear process through which we can demonstrate the benefits and positive impacts our research has in our communities.”

Preliminary work is occurring through the spring to gather information about existing frameworks and published work on the subject of research and innovation impact assessment. The Office of the Vice-President (Research) plans to invite members of the academy to participate in discussions around impact assessment later this year. Faculty members interested in engaging are welcome to contact vprcomms@ucalgary.ca.