Responsible metrics

The term 'responsible metrics' was coined by the Metric Tide review (Wilsdon et al., 2015) and operationalised through the Forum for Responsible Research Metrics in the UK. The five principles are: robustness (use the best possible data in terms of accuracy and scope); humility (recognise that quantitative evaluation should support, not supplant, qualitative expert assessment); transparency (keep data collection and analytical processes open and simple); diversity (account for variation by field and use a range of indicators); reflexivity (recognise and anticipate the systemic and potential effects of indicators and update them in response). The framework is now embedded in CoARA, the R4RI, and many institutional and funder policies.

References

• Wilsdon et al. 'The Metric Tide', HEFCE, 2015. Forum for Responsible Research Metrics (frrm.ac.uk).

What adopting responsible metrics actually involves

The principle is easy to endorse and hard to operationalise. The instruments that define it — the Metric Tide and its 2022 follow-up, DORA, and the Leiden Manifesto — set out what good practice looks like, but each institution has to translate that into its own promotion criteria, hiring processes and internal reporting. In practice, adoption usually means some combination of the following.

• Writing an institutional statement that says which indicators may and may not be used in evaluation, and at what level of aggregation. Signing DORA is the visible step; the statement is what gives it effect.
• Removing journal-level indicators from individual assessment. This is the concrete commitment most frameworks share: a journal-level metric describes a journal, not a paper or a person, and using it as a proxy for either is the specific practice DORA was written to stop. Alternative journal metrics do not resolve that objection — the problem is the level of aggregation, not the vendor.
• Pairing any indicator with expert judgement. Peer review remains the primary mechanism; metrics inform it rather than replace it.
• Broadening what counts as an output, so that datasets, software, protocols and public-facing work can be assessed rather than ignored by default.
• Auditing for disparate effect. Indicators distribute differently across fields, career stages and demographics, and a policy that is neutral in wording can be uneven in effect. Checking this is the reflexivity dimension in practice.

Where the obligation comes from

For most institutions responsible metrics is no longer purely voluntary. Funders increasingly ask how research is assessed as a condition of funding, CoARA commits its signatories to a reform timetable with public reporting, and national assessment exercises set expectations that flow down into institutional policy. The practical consequence is that an institution may need to evidence its approach, not merely hold one.

The recurring failure mode

The commonest gap is between a signed commitment and an unchanged promotion form. A statement that rules out journal-level metrics has no effect if committee papers still display them, if the internal dashboard still ranks staff by them, or if reviewers are never told what to use instead. Changing the process — the forms, the guidance to panels, the data shown to decision-makers — is what makes the commitment real, and it is the part most often deferred.