The Impact Factor (IF), or Journal Impact Factor (JIF), is a widely used metric intended to gauge the relative importance of academic journals within their respective fields. While it offers a seemingly straightforward numerical value, its application and interpretation are fraught with complexities and criticisms. Originally designed as a tool for librarians to make purchasing decisions, the JIF has evolved into a powerful, yet often misleading,
benchmark for evaluating research, journals, and even individual scientists. Understanding its limitations, from statistical flaws to the potential for manipulation, is crucial for a nuanced perspective on journal evaluation.
Statistical Flaws and Misleading Averages
One of the most fundamental criticisms of the Impact Factor lies in its statistical methodology. A 2007 study pointed out that Impact Factors present the mean of data that are not normally distributed. Citation counts typically follow a highly skewed distribution, meaning a small number of highly cited papers can disproportionately inflate a journal's average Impact Factor. For example, approximately 90% of *Nature*'s 2004 Impact Factor was attributed to only a quarter of its publications. This implies that the actual number of citations for most individual articles within a journal is often much lower than the reported mean Impact Factor suggests.This skewed distribution makes the arithmetic mean, as used in the Impact Factor calculation, potentially misleading when trying to assess the typical impact of articles within a journal, rather than the overall impact of the journal itself. Critics argue that using the median of citation data would be a more appropriate statistical measure, as it would better reflect the central tendency of citations for most articles. The strength of the relationship between a journal's Impact Factor and the citation rates of its individual papers has also been observed to decrease steadily since articles became digitally available, further highlighting the disconnect between the journal-level metric and article-level impact.
Manipulative Editorial Policies and Practices
The widespread acceptance of the Impact Factor as a proxy for research quality has unfortunately incentivized some journals to adopt editorial policies and practices aimed at artificially boosting their scores. These tactics can range from acceptable to highly dubious. For instance, journals may increase the proportion of review articles they publish, as these generally receive more citations than original research reports. Research in 2020 on dentistry journals confirmed this, finding that systematic reviews significantly influenced the Journal Impact Factor, while clinical trials did not.Another strategy involves manipulating the denominator of the Impact Factor equation, which is the number of "citable items." Journals might decline to publish articles deemed unlikely to be cited, such as case reports in medical journals, or alter article formats to prevent them from being counted as citable items by Clarivate Analytics. Negotiations between publishers and Clarivate over what constitutes a "citable item" can lead to "negotiated values" for Impact Factors, resulting in substantial changes in observed scores for numerous journals. Items considered uncitable, if cited, can still contribute to the numerator of the equation, further complicating the calculation and potentially inflating the score. More overtly, some journals have engaged in self-citation campaigns, as seen with *Folia Phoniatrica et Logopaedica* in 2007, which cited all its previous articles in protest, leading to a temporary surge in its Impact Factor and subsequent exclusion from the Journal Citation Reports.
