How can I make sure I'm evaluating a Researcher fairly?

First you need to understand what you are trying to measure and why. The chart below illustrates the risk associated with each measurement reason for different entity types.

Note how metric use when looking at individuals (researchers) carries a medium to high risk for all measurement types. This is because even a small amount of missing or incorrect content can have a potentially large impact on the metric outcomes. It is therefore very important that care is taken when choosing metrics to complement expert opinion when evaluating researchers.

• Understand – e.g. the “science of science.” It helps you understand science based on the database you’re using
• Show off – e.g. to market an institution on promotional material or to provide evidence for a grant application
• Monitor – e.g. plot progress against an objective
• Compare – e.g. university rankings
• Incentivize – e.g. measure OA content for a researcher to show how much they support open science
• Reward – e.g. for a job promotion, grant, prize or award

Chart taken from The Bibliomagician blogopens in new tab/window.

HEI = Higher Educational Institution

• How long has the researcher been actively publishing?
• What point are they in their career?
• Have they had an absence due to illness, children or other reasons?
• Which subject area are they working in?

Metrics advice

Why should you not use the FWCI to compare researchers or small entities?

It is not recommended to use Field-weighted Citation Impact (FWCI) to compare researchers or small entities. Researchers usually have a small number of publications, which can be affected by outlier publications or publications that do not fit the general profile of the entity being analyzed.

For example, let’s say each publication of an author has an FWCI of 0.5 but one publication out of a total of 5 gained several citations and had an FWCI of 15. This would disproportionately affect and elevate the overall FWCI of the author.

For a small entity, such as a group comprising a few researchers, the concerns are the same: if the publication numbers are small, there is a risk of a significant outlier effect.

Which metrics can you use instead of the FWCI?

There are a number of field-normalized alternatives to the FWCI:

• Use the FWCI per publication instead of a single average metric

  • It’s absolutely fine to say that 2 of the researcher’s 10 publications have a FWCI of at least 2
• Use the Field-weighted Outputs in Top Citation Percentiles (1, 10 or 25%)

  • This metric cannot be skewed by very large FWCI values as it is a count, which is normalized by the researcher’s total publications, by turning it into a percentile

• Scholarly Output – useful as a size indicator and to set the scene
• Outputs in Top Journal Percentiles - since journal data underpin this metric, and not the citations received by individual publications themselves which would require time to accumulate
• Percentage of international or corporate collaboration – as typically these 2 types of collaboration lead to a higher publication FWCI
• Field-weighted Views Impact – this can give an early indication of interest for the individual publications
• Field-weighted Citation Impact – at an individual publication level

Understanding Scopus author profiles

Scopus has invested in automatically grouping the publications it indexes into those published by a single author. This algorithm looks for similarities in author name, as well as affiliation, journal portfolio, and discipline to match publications together. The matching algorithm is very good, but can never be 100% correct, because the data it is using to make the assignments are not 100% complete or consistent.

The Scopus author profiles are also a partial view of the researcher’s work, as it does not contain aspects like teaching and learning, administrative experience, all funding, PhD completions and esteem indicators such as being an editor, invited keynote speaker, policy consulting etc.