Ilka Agricola, Lynn Heller, Wil Schilders, Moritz Schubotz, Peter Taylor, and Luis Vega

1. Introduction

This report is the first of two publications of a joint working group of the International Mathematical Union (IMU) and the International Council of Industrial and Applied Mathematics (ICIAM). In it, we shall analyze the current state of publishing in the mathematical sciences and explain the resulting problems. Our second publication will offer concrete recommendations, guidelines, and best practices for researchers, policymakers, and evaluators of mathematical research [43]. It will explain how to detect and counteract attempts to game bibliometric measures, empowering the community to reclaim control over research evaluation and drive necessary change.

As students, teachers, authors, reviewers, editors, and committee members, we contribute to the creation, publication, consumption, and evaluation of research outputs. In the current academic environment, the assessment of research quality is heavily influenced by bibliometric analysis. As a result, there is a strong incentive for researchers to optimize their bibliometric data. Unfortunately there is a growing number who are using fraudulent methods to do this. New (and to many unknown) threats such as paper mills, predatory journals, and citation cartels have been invading the “ecosystem” of scientific publishing and putting the integrity of our profession at risk. A detailed glossary is provided in Appendix A, with an extensive list of references pointing to further reading that supports the claims made in this article. As a main source, we recommend the excellent book [5]. It is worth noting that most of the articles in the reference list have appeared in the last five years, reflecting the urgency of the problem. It will become clear that two driving forces behind the problems we address are the advent of Article Processing Charges (APCs) for Open Access as a business model for scientific publishing and the pervasive demand for quantitative research output assessment. Also intricately involved are the companies that collect citation data and use it to produce measures such as the journal impact factor, the h-index, and the lists of highly cited researchers (HCRs) that are derived from this data. We will argue that products such as these are having detrimental effects on the publication enterprise.

This is the print version of this article with a selection of the most important references. A version with complete clickable references is available on arXiv [2].

2. Bibliometrics in the Mathematical Sciences

When a measure becomes a target, it ceases to be a good measure.

—Charles Goodhart, British economist (1975)

In 2009, Adler, Ewing, and Taylor were engaged respectively by the Institute of Mathematical Statistics (IMS), the IMU, and the ICIAM to look into the effects that quantitative assessment of research was having on their disciplines. The charge that they were asked to deliver on was:

The drive towards more transparency and accountability in the academic world has created a “culture of numbers” in which institutions and individuals believe that fair decisions can be reached by algorithmic evaluation of some statistical data; unable to measure quality (the ultimate goal), decision-makers replace quality by numbers that they can measure. This trend calls for comment from those who professionally “deal with numbers”—mathematicians and statisticians.

The result was the report [1], which was published in a number of outlets. While the report remains highly relevant, a new phenomenon has emerged that merits a similar discussion, namely the widespread gaming of quantitative metrics. To provide background, we shall in this section briefly recall what [1] and other papers [12,24] say about the publishing culture of the mathematical sciences and pose some questions to motivate readers’ thinking.

2.1. How mathematicians publish. Mathematicians typically publish smaller numbers of papers, with fewer coauthors and citations than most other scientists. This renders meaningless any comparison of mathematicians with other scientists according to bibliometric measures. Our impression is that this point is largely understood by the proponents of bibliometric analysis: They are usually willing to concede that the disciplinary context matters. What is less understood is that there can be very different citation cultures between different subdisciplines. Furthermore, the small absolute numbers of citations that are a result of the mathematical science referencing culture raise the question of whether it is possible to discern any signal of “quality” from the background noise. For us, the difficult questions revolve around how relevant bibliometrics is to evaluating research within the mathematical sciences. For example:

Q1: Does a relatively high number of citations tell us that a paper in the mathematical sciences is better than another paper?

Q2: Does the impact factor of a journal in the mathematical sciences reflect its quality?

Q3: Does a relatively high number of papers tell us that a researcher in the mathematical sciences is better than another researcher?

Q4: Should citation analysis over a period of time be used to evaluate the careers of mathematical scientists?

Q5: Should aggregated citation data be used to rank institutions in the mathematical sciences?

Q6: Does the existence of bibliometric measures influence the behavior of researchers, for good or for bad?

Crucial for our current discussion is the fact that having a culture that produces a small absolute number of citations renders the discipline particularly vulnerable to manipulation of bibliometrics. With that in mind, we argue below that the answer to Q6 is yes, and that the influence is generally not positive. Specifically, there is very good evidence that some individuals, groups, institutions, and editorial boards are conspiring to tailor their publication behavior to manipulate rankings made on the basis of bibliometric analysis.

2.2. Bibliometrics. Citation databases were first created as a research tool to help scholars search the academic literature by listing papers that have cited a particular, often

seminal, paper. This activity has a long history dating back to the beginning of the twentieth century [23]. However, it is generally acknowledged that the modern form of bibliometric analysis dates from the work of Garfield in the 1950s and 1960s [39]. The growth of this worthy endeavor into a structure whose primary purpose is the quantitative assessment of research is described in many places, one of which is [32], written by David Pendlebury, a long-term employee of one of the main providers of citation data, Clarivate. Other descriptions appear in [23] and [17].

As Goodhart hints at in the quote at the beginning of this section, when the activity being measured involves some aspect of human behavior, there is a large incentive to game systems. We are seeing this playing out in the field of bibliometrics across all disciplines. For example, the vulnerability of a system of research assessment based upon bibliometrics is the topic of Macdonald and Kam’s paper [27]. For the reasons mentioned above, the effects are particularly pernicious in the mathematical sciences. There are ongoing efforts to develop indices that are not prone to gaming, but so far none of them have had any impact on the situation.

2.3. The academic publishing ecosystem. We start with a quick look at a simplified version of the academic publishing ecosystem, illustrated in Figure 1. At the center of all research are, of course, the individual researchers. They work at universities or other research institutions, be they public or private. These institutions are subject to some kind of evaluation or assessment by authorities such as governments, ministries, and grant agencies. Researchers publish their findings in journals, and many of them at the same time serve as editors and reviewers for thesejournals. Journals are published by publishers, as the name indicates. Some of these are learned societies or other not-for-profit entities but, increasingly, many of them are exceedingly profitable commercial enterprises. For simplicity, we have omitted professional societies and conferences from Figure 1 because they usually don’t contribute to the problems that we are addressing.

Before the advent of open access (OA), these were the key players. University libraries would subscribe to single journals based on the recommendations of their researchers, which limited the number of low-quality journals included in their collections. Digitization, OA, and the growth of the scientific community have led via a long process to different questions, depending on the perspective:

• For researchers: How to find the relevant research outlet, and how to increase visibility of their work?

• For policymakers in governments and grant agencies: How to quantify research evaluation?

• For commercial publishers: How to create (more) revenue?

A first decision by large publishing houses was to promote the purchase of subscriptions in large bundles of journals instead of individual journals—the advantage being clear, as long as the bundle contained the key journals of a field, the quality of a single journal mattered less, and a constant revenue was secured, as canceling single subscriptions was no longer possible. This is the policy of Springer Nature and Elsevier, who currently make up 40% of mathematics articles indexed for example by zbMath Open [42]. OA saw the appearance of (sometimes rather substantial) APCs that authors pay to journals in order to publish their articles, thus creating an incentive for journals to publish as many articles as possible. Indexing companies, with their databases, previously existed to help researchers discover relevant papers. But the advent of quantitative assessment of research has made their databases and bibliometric products vastly more valuable. They sell these products to all larger institutions with the claim that they enable objective research assessment.

Bibliometric analysis is possible only when a database is available that, for a sufficiently extensive set of papers 𝑆, records the set 𝐶(𝑥) of papers cited by each 𝑥 ∈ 𝒮. The derivation of measures such as a journal’s impact factor, an author’s citation count or h-index, a field’s list of HCRs or, indeed, more sophisticated statistics such as a journal ranking or an author’s empirical citation distribution, is then a computational task. Since the nature of the database can have a significant effect on the measures that are computed, we shall make a few brief comments about the most important citation databases.

Clarivate (known as Thomson Scientific before 2008) is a British-American publicly traded analytics company with over 10,000 employees and a revenue of US$2.66 billion in 2022. It is the company that computes the journal impact factor and creates the list of HCRs discussed below. It offers these via the paid platform Web of Science (WoS). Every year it publishes the (paywalled) journal ranking JCR (Journal Citation Reports) based on its Science Citation Index Expanded (SCIE) database of journals and publications. The list of journals indexed is not freely available (which in itself is already problematic), but the small number of delisted journals per year underlines the fact that many predatory journals and low-quality megajournals remain in the database [7]. As a rule of thumb, any journal having an impact factor is contained in WoS.

Scopus is a product of the Dutch publishing company Elsevier. It provides journal, article, and author metrics; researcher profiles; and analytical services. As the bibliometrics arm of a major publishing company, the question of whether there is a conflict of interest obviously arises. It is often claimed that Scopus’ database contains questionable journals [16, 26]. It is the database used to compute the commercial journal ranking SJR (SCImago Journal Rank). A quick analysis of the 2023 SJR listing in the general category “mathematics” shows that about 20 % of the Q1 and Q2 journals are not listed by zbMath Open— because they have little relevance for mathematics, because they are known to be of poor quality, or because they are falsely classified as being in mathematics.

Google Scholar is the part of Google’s search engine that concentrates on the scholarly literature. It also, however, computes and publishes metrics. It is often criticized for a lack of control over what is included in its database, with duplications of the same article being noticeable. The recent study [25] shows that the citation data of Google Scholar are easily manipulated.

zbMATH Open is a free abstracting and reviewing service for mathematical literature and software, edited by the European Mathematical Society and FIZ Karlsruhe. The indexing policy is to cover all available published and peerreviewed articles, books, conference proceedings, as well as other publication formats (like ArXiv preprints) pertaining to the scope defined by the Mathematics Subject Classification (MSC) that present a genuinely new mathematical point of view. Mathematical Reviews provides a subscription based service similar to zbMATH (except software and preprints) published by the American Mathematical Society and is held in similar high regard within the mathematical science community. Mathematical Reviews and zbMATH Open currently cover about 1600 journals (as well as many old journals and book series) and index a total of about 5 million items.

Figure 1. A simplified depiction of the academic publishing ecosystem.

The ArXiv is a service provided by Cornell University with some support from other organizations. It is a repository that contains nearly 2.4 million scholarly articles in the fields of physics, mathematics, and computer science (as well as a few others). “Putting a paper on the ArXiv” is a first step toward publication for many mathematical scientists and a large amount of significant research can be found there.

ORCID is a global, not-for-profit organization that provides a unique, persistent identifier for individuals to use as they engage in research, scholarship, and innovation activities. Such an identifier is a useful tool for author disambiguation. ORCID has a strong set of founding principles, which we support. The relationship between ORCID and the commercial databases mentioned above is something that should be monitored as time goes by. We have already noticed instances where an ORCID identifier in, say, a login procedure has been replaced by an identifier controlled by one of the the abovementioned commercial companies.

3. The 2023 Clarivate Exclusion

3.1. The Clarivate exclusion. In November 2023, Clarivate announced that it had excluded the entire field of mathematics from the most recent edition of its influential list of highly cited authors. Justifying this decision, it said [33]

(Mathematics) is a highly fractionated research domain, with few individuals working on a number of specialty topics. The average rate of publication and citation in Mathematics is relatively low, so small increases in publication and citation tend to distort the representation and analysis of the overall field. Because of this, the field of Mathematics is more vulnerable to strategies to optimize status and rewards through publication and citation manipulation.

We agree with this statement. It is interesting to see a company that supplies citation data specifically concede that such data is vulnerable to strategies to optimize status and rewards.

Some authors [14] have argued that the exclusion occurred after it became obvious that many of the people listed as HCRs in the immediately preceding years were not, in the view of the scientific community, top-level mathematicians (see also [35] for further details on the exclusion). Rather they seem to be people who had manipulated their bibliographic parameters in a way that could not be ignored any more. Table 1 contains a list of the most frequent primary affiliations of the 2019 HCRs in the mathematical sciences. Note that the top listed institution, Medical University Taiwan, does not have a program in mathematics. In the list published in November 2024, mathematics was still excluded, but without further comment. In a blog entry [34], Head of Research Analysis David Pendlebury concedes that they excluded more than 2000 individuals for integrity concerns (theformulation is rather vague). Some remaining 6,636 individuals were listed as HCRs, but 48% of them were not associated with any definite research area (they were designated “cross-field”). At least one of the people listed as cross-field in 2024, J. A. Tenreiro Machado, died in 2021, raising (even more) questions about the quality of the underlying data. A recent study [15] of the set of HCRs argues that this list (as a whole, not limited to mathematics) has entered a new “phase” of development in 2019– 2023: “This phase is characterized by the constant increase in profiles suspected of scientific misconduct, which challenges the ability of this list to identify truly influential researchers.”

It has been argued [1] for a long time that the commonly used bibliometric metrics are inappropriate for the mathematical sciences, and probably for all science. Whatever your view of the reasons behind the Clarivate exclusion, its occurrence has forced the wider mathematical community to pay attention to this effect.

Table 1. Institutions having more than one primarily affiliated HCR in mathematics (from a total of 89 HCRs in math in 2019) [40].

3.2. A quick look at other sciences. No one should draw the false conclusion from Clarivate’s decision that mathematics has become a dubious science infiltrated by fraudsters. Fraudulent science occurs in other fields, not only with the aim of manipulating bibliometrics, but also in the form of claiming false results. In fact, the latter is more common in the broader sciences than in the mathematical sciences, because research papers in these disciplines often report the results of experiments, fieldwork and surveys that have to be taken on trust. Cases of massive data falsification in the experimental sciences tend to be widely reported in the media and thus are the first that come to the public’s mind. As an example, we mention the controversy resulting in the resignation of the former president of Stanford University, neuroscientist Marc Tessier-Lavigne, in August 2023 [31].

Overall, the fact that research communities in, say, the life sciences are much larger than in mathematics, combined with the high level of competition for substantial funding, suggests that it would not be surprising if there were a higher number of problematic publications. Several studies support this claim [8]. A 2020 study of retracted papers in the field of neuroscience [11] concluded that “Papers retracted originating from paper mills are increasing in frequency, posing a problem for the research community.”

Systematic studies of fraudulent publishing are rare, but if one considers many events together, typical patterns and a general picture start to emerge. For example, several scandals have shaken Spanish science in the last two years, as documented in the international edition of the newspaper El Pais. The president of the old University of Salamanca, computer scientist Juan Manuel Corchado, was at the center of a very fruitful citation cartel. He now has had an unprecedented 75 papers retracted. There were alleged paper mill purchases by Spain’s most prolific scientific academics, and some universities from Saudi Arabia paid Spanish scientists to lie about their primary affiliation. One of these universities, King Abdulaziz University, is ranked second in Table 1. Similar scandals have been reported from other countries, including in alphabetical order: Canada, China, Hungary, India, Poland, and Vietnam. One of the rare exceptions of a truly systematic study is a detailed investigation by the German newspaper Süddeutsche Zeitung and the public broadcasters NDR and WDR in 2018.

To summarize: Fraudulent publishing is a worldwide problem of substantial size in all scientific fields; it concerns researchers at all seniority levels in a variety of contexts.

3.3. The extremes: HCRs and retractions—and how they correlate.

A closer look at HCRs in mathematics. In 2019, 89 researchers were listed as highly cited researchers in mathematics. In 2021, Edward Dunne, then executive editor of Mathematical Reviews, conducted a thorough investigation of these researchers [19], which is why our analysis will also be based on the same year. First, let us start with a summary of Dunne’s results:

1. The fields in which the group of HCRs work are not representative of the mathematical sciences: for example, none is working in geometry or algebra, and only 16 out of 60 (26%) possible mathematical science subjects actually appear as primary subject of the majority of their publications.

Table 2. Self-citing score (SCS) and self-referencing score (SRS) for different cohorts of mathematicians (cited from [19]).

2. The most common primary subjects of HCRs are: partial or ordinary differential equations (37%), statistics (16%), numerical analysis (15%), and operator theory (10%).

3. Instead of merely counting citations, one could use major math prizes as indicators of mathematical influence (Dunne took these to be the prizes awarded by the AMS, SIAM, EMS, and the IMU including the Fields Medal): only five of the 2019 HCRs have won any of these. The lists of HCRs and prizewinners are thus almost disjoint, despite the fact that there are 636 such winners of a total of 29 prizes, over all time.

4. The topics of the prizewinners are broader, they make up about 61% of the mathematical science subjects. They are not concentrated in analysis and applications, and their most common topics (algebraic geometry and number theory) do not appear at all among the group of HCRs.

5. The mode of “quality assessment” is totally different: HCRs are established solely on an absolute citation count of a very large, uncurated list of journals, while prizes are awarded by selection committees appointed by professional societies.

Furthermore, Dunne detected a clear difference in citation patterns. To explain it, let us introduce two similar, but crucially different numbers (see also [30] and the critique of their work in [19]):

Self-citing score (SCS): Percentage indicating the proportion of citations of an authors works that come from the author’s own publications. The SCS is about citations to an author’s papers. Specifically how many come from other papers by that author?

Self-referencing score (SRS): Percentage indicating the proportion of the citations in an author’s paper that are to their own previous papers, as compared to citing other authors. The SCS is about citations from an author’s papers. Specifically how many are to other papers by that author?

Care is needed for computing these numbers, and it is methodologically reasonable and in fact necessary to restrict attention to people who won their prizes after the year 2000 (some older winners are long retired and hence no longer publish, old citation data is often incomplete, and citation patterns could be time dependent). There is a total of 365 such mathematicians. As a third cohort for comparison, Edward Dunne and the IT team at Mathematical Reviews determined the 1000 top-cited authors based on journal articles published between 2010 and 2020 and contained in the MR database [19].

The differences are dramatic; we summarize them in Table 2. While the median SCS of the top 1000 cited mathematicians is similar to that of the prizewinners, the SCS of HCRs is more than twice as large. For the median SRS, the same effect is visible and it is noticeable that prizewinners reference themselves in their subsequent work very modestly (SRS of only 0.22).

To summarize: HCRs cite and reference themselves about twice as often as the other two cohorts of mathematicians. Their publication data differs drastically from that of other established mathematicians. With few exceptions, the indicator HCR is useless for detecting mathematics of good quality.

3.4. Retractions. In the past, retractions were an unknown phenomenon; articles that were published were so for eternity, and mistakes were corrected in errata published in the same journal. Then articles with falsified data began to be retracted by publishing houses to prevent the spread of incorrect data until, now, retractions are viewed as a necessary ultimate scientific correction mechanism.

As with all fraud, only the tip of the iceberg is visible. As of December 2024, the database of Retraction Watch [47] listed a total of 1009 articles retracted in the primary field of mathematics, and more than 3,000 who listed mathematics as a secondary topic. With a positive lens, a retraction can be a useful correction mechanism of a publishing house; as we all know, mistakes can happen. But it is fair to be suspicious when a journal or author has multiple retractions, as it could be an indicator of poor peer review or systematic scientific misconduct.

In the database, mathematics journals that have multiple retractions are mostly published by Hindawi (Journal of Function Spaces, Mathematical Problems of Engineering, Complexity), which is partly due to the 2022 acquisition of Hindawi by John Wiley & Sons and the ensuing efforts to “clean up” the journals and increase their quality. To a far lesser extent, Sciendo Publ (Applied Mathematics and Nonlinear Sciences) and MDPI (Symmetry, Entropy) are affected. Given how rare they are, it is interesting to relate the database of retractions to the list of mathematics HCRs.Seven out of the 89 HCRs appear in the database of retractions, mainly for paper mill suspicion, plagiarism, or duplication.

Again, the problem is not limited to mathematics—and it is quickly growing. In 2023, more than 10,000 articles were retracted over all sciences, and (in decreasing order) the countries with most retractions per 100,000 published articles were Saudi Arabia, Pakistan, and China, and the percentage of published papers being retracted has increased from about 0.02% in 2002 to 0.2 % in 2023 [29].

3.5. HCRs and university rankings. In addition to serving the personal vanity of some researchers and sprucing up their CV for grant applications and salary negotiations, admission to the “Hall of Fame” of HCRs has a decisive advantage for their university: it impacts their rank in the “Academic Ranking of World Universities” (ARWU, published since 2003), also nicknamed “Shanghai ranking.” The computation of this rank relies on a remarkably small number of indicators:

1. Excellent researchers: Nobel Prizes and Fields Medals and some more.

2. Excellent papers: papers published in Nature and Science.

3. Research output: papers indexed in major citation indices, and the “per capita academic performance of an institution.”

The second item explains part of the hype about publications in Nature and Science; for mathematics, these journals play no role. The critical points are those emphasized in the first and third items; both rely on products of Clarivate. As the total number of Nobel Prizes and Fields Medals is rather low, the company’s databases (WoS, SCIE) and bibliometrics (IF) have been used to create a new class of researchers deemed excellent, the Highly Cited Researchers; needless to say, the major citation indices are their own. Since the database SCIE includes virtually all existent journals and judges them solely by their impact factor, predatory journals and paper mills are, by construction, included as well as reputable journals. The incentive for researchers and universities alike to manipulate their publication data is thus complete: a good ranking increases a university’s prestige; in many countries, it is very important for students’ choices of a university. It attaches a label of excellence that is rarely questioned. The problem is seriously aggravated by the fact that it is surprisingly easy to cheat with one’s affiliation, as Clarivate basically contacts HCRs and asks for them.

And in the end, the approach is successful, at least for some. The top listed university, China Medical University, Taiwan, from Table 1 was, in 2022, listed in the top 300 universities of the world. The second ranked institution welcomes visitors on their webpage with the opening sentence: “King Abdulaziz University maintained its Arab leadership in international rankings, as it ranked first in the Arab world and ranked 101–150 among the best universities in the world, according to the Shanghai International Classification for the year 2021–2022, and first in the Arab world in the British QS classification index for the third year in a row.” Its mathematics program is deemed to be ranked 31st in the world. The temptation to influence an institution’s own position in the rankings through strategic choices is also present at prestigious traditional universities: the University of Paris-Saclay (a merger of four “grandes ´ecoles” and several other institutions) was founded in 2019 with the aim of becoming a top-ranking, research-focused French university. This, of course, works solely because the absolute size of an institution matters.

Other rankings suffer from the same problems, be they international (like the Times Higher Education [THE] World University Ranking, which weights the indicator “citations” by 30% and uses Elsevier’s Scopus) or national (like the National Institutional Ranking Framework [NIRF] in India). All are produced by private companies, based on noncurated databases riddled with mistakes and of poor quality, based on arbitrary easily gamed criteria.

To summarize: Rankings of universities are too easily gamed to be given any real meaning. They create false incentives and do more harm than good.

4. The Many Heads of the Hydra of Fraudulent Publishing in Mathematics Bibliometrics are an attempt to quantify the quality of scientific achievements. Within the mathematical sciences, it is likely that bibliometric manipulation is the most significant form of scientific fraud. Below we discuss the issues in more detail.

Occasional dishonesty is the most widespread form of scientific misconduct. It is usually recognized only by expert peers and not approved of, but also not pursued further due to the perceived “insignificance” and difficulty of doing so. It is what we could call the “zone of occasional poor practice.” Included are actions like:

• Papers split into least publishable units instead of a larger, more substantial article (“salami-slicing”)

• An exaggerated, but still (more or less) tolerable number of self-citations or citations of mathematical friends

• Recycling of previous text blocks, such as a standard introduction to the field (most journals now use similarity checkers and will detect these anyway)

• Dishonest or sloppy attribution of previous results (exaggerating one’s own contribution, “forgetting” a paper with similar content, citing a textbook instead of the original article.. . )

• Sloppy checking of existing literature

• Reviewers asking authors to cite the reviewers’ papers

But as D. Docampo says in [18]: “... In the complex landscape of modern academe, the maxim ‘publish or perish’ has been gradually evolving into a different mantra: ‘Get cited or your career gets blighted.’ (.. . ) Citation has become so important that it has driven a novel form of trickery: stealth networks designed to manipulate citations.. . ”. It is what we could call the “zone of systematic bad practice” that includes:

• Citation manipulation, see for example [6] where instances of manipulation through injection of meaningless texts containing a fixed set of references and addition of references during the peer-review process are documented

• Editors asking authors to include references to other articles in their journal after acceptance as a condition for publication

• Translation and “copy-paste” plagiarism

• Academic superiors claiming coauthorship despite not having contributed to a publication or influencing the attribution of authorship in any other way

• Giving incorrect affiliations of authors on purpose

• Including authors without their consent

• Giving incorrect funding information. While conflicts of interest because of industry-financed research is rare in mathematics, wrong information is sometimes used to suggest high-quality research

• Predatory conferences as part of a citation cartel

Unfortunately, a new level of fraud can be reached when money is involved, the scientific reputation or even employment of people is directly at risk, or a substantial amount of “criminal energy” is required for the misdeeds. Nick Wise, a famous science sleuth, said on the Retraction Watch Blog: ”There’s this entire economy, ecosystem of Facebook groups, Whatsapp groups, Telegram channels selling authorship for papers, selling citations, selling book chapters, selling authorship of patents”. It is what could be called the “zone of fraudulent behaviour.” Here are some examples:

• Citation sales: A professional citation broker (with an anonymous email address) presents a list of papers that need citations and offers a “thank you fee”

• Paper mills: It should go without saying, but buying papers from professional ghostwriters to publish under one’s own name is a serious case of scientific fraud [11]

• Authorship sales: The revenue of paper mills can be increased (or the costs for the buying person reduced) if one sells authorship on a ready-to-publish article

• Blackmailing: Several instances of blackmail have been documented. These include paper mills that threaten authors that they will reveal their scientific misconduct to journals, employers, or grant agencies

• Abuse of power: Bullying of “noncompliant” researchers is mainly carried out by scientific seniors. Several cases of leaders in top academic institutions who lost their positions for bullying have been prominent in the media in recent years

• Identity fraud: Several cases are reported where predatory journals have included researchers as editors without their consent, and sometimes didn’t even remove their names from their webpage after complaints

• Plagiarising articles and publishing them with an incorrect date to make them look as if they were the original publication

• Use of pseudonyms for publications, for example to avoid being connected to previous scientific misconduct

• Nondisclosure of serious conflicts of interest

• Creating email accounts similar to those of established researchers and suggesting them as reviewers of an article, hence being able to review one’s own work.

Only rarely are fraudulent activities pursued in court. One of the few exceptions is the India-based company OMICS Group Inc, which created an intricate network of predatory journals (about 700, some of them shortlived, often rebranded) and predatory conferences (about 3000 every year). In 2016, the US Federal Trade Commission (FTC) filed a suit against OMICS. The final judgment from May 2022 [41] found that the company had used “deceptive marketing practices” and documents many of the frauds listed above . Despite its name suggesting roots in the life sciences, the company also publishes several journals in mathematics.

5. Instead of a Conclusion: A Few Words about AI While AI can be an invaluable tool for a variety of tasks, it also introduces new risks and challenges that the research community is only beginning to comprehend.

Most journal editors and publishers concur that authors may use AI-driven tools such as DeepL, ChatGPT, or DeepSeek for copyediting and proofreading purposes. This practice can be particularly beneficial for nonnative English speakers. However, it is critical that authors take responsibility for ensuring that the content of their publications (including citations, tables and figures) is not altered by automated translation or editing. Under normal circumstances, the use of these AI services for copyediting does not need to be formally acknowledged in the publication. In contrast, when AI is used for more substantive tasks—such as generating or interpreting research content—many publishers now require an explicit statement disclosing this use, and no AI may be listed as a coauthor. This policy shift underscores a growing concern that AI-generated text could be used to fabricate or inappropriately alter scientific claims, thereby undermining scientific integrity. The appendix to [22] provides a valuable compilation of excerpts from AI policy statements by publishers and scientific organizations worldwide, demonstrating a certain convergence on the key principles outlined above.

The advent of sophisticated AI language models has made it cheaper and easier to produce fake research. Paper mills are likely to exploit AI to generate plausiblesounding texts with minimal human oversight [28]. It also presents a major challenge for fraud detection: As AIgenerated texts become more coherent and context-aware, it becomes increasingly difficult for traditional screening methods to distinguish them from human work. This has spurred investment in new “integrity software”—AI tools being developed to catch signs of automated content generation. However, this will almost certainly lead to an arms race: as detection tools improve, so do the generative models, potentially creating a never-ending cycle of innovation on both sides. At present, tortured phrases (unusual or contorted language constructions) remain a common red flag which can be used to identify AI-generated text [9] (see Table 3). Yet, relying on textual anomalies alone may not be sustainable as AI models become more refined. The community is thus calling for robust guidelines, shared databases of fraudulent works, and transparent protocols for both detection tools and editorial processes.

A significant turning point was Wiley’s experience with Hindawi, which reportedly cost the company substantial resources in handling fraudulent submissions. Consequently, Wiley announced a pilot “AI-powered Paper Mill Detection” service. Other major publishers, such as Springer, are following suit, looking for comprehensive strategies that combine algorithmic checks, expert review, and community-based reporting. In his recent investigation of undeclared AI usage in scientific publications [22], the author Alex Glynn concludes: “This analysis [.. . ] reveals that the problem is widespread, penetrating the journals and conference proceedings of highly respected publishers.”

Beyond publishers, funding agencies have also begun to issue guidelines on AI. The US National Science Foundation (NSF), for instance, has released statements emphasizing the importance of transparency in AI usage and the need for responsible use, including the prohibition of “uploading any content from proposals, review information and related records to nonapproved generative AI tools” [44]. This raises the even more fundamental question of what it means for the mathematical community if LLMs are trained with our papers, with or without our consent [46].

To summarize: AI will exacerbate the existing problems of scientific publishing, and the options for cheating will multiply. It remains to be seen whether the scientific community has the resources necessary to counteract this. In the meantime, we appeal to everyone to stand up for the strengthening of ethical standards in their fields and to support honest scientists in this unequal struggle.

Table 3. Collection of Tortured phrases.

A. Glossary

Citation cartel: Group of individuals that agree to cite authors from the cartel and their preferred journals, regardless of their relevance for their work.

Citation broker: Anonymous emailer who offers a “thank you fee” for each citation of a given list of articles. Payment is typically in cryptocurrency.

Citejacked journal: Established journal citing articles in a Hijacked journal, v1.

Mega-journal: Journals publishing huge numbers of articles per year; they did not exist before the advent of Open Access. One of the most prolific megapublishers is MDPI, founded in Basel, Switzerland. It currently owns 433 journals. Its top publication International Journal of Environmental Research and Public Health publishes nearly 17,000 studies each year (it was delisted from WoS in 2023 and will hence lose its impact factor [7]). Its largest math journal Mathematics has more than 1000 people in the editorial board (27 of them are HCRs) and publishes over 6,000 articles in 2023—compared to a total of about 120,000 papers reviewed in zbMath in the same year. Its yearly revenue is estimated at approximately 10 million SF. Multiple scandals and retractions prove that it is just not possible to maintain high scientific quality in this huge quantity of publications (see for example [38]; it also illustrates again how articles of bad quality manage to make it into newspapers).

Hijacked journal, v1: A fake website that uses the titles, ISSNs, and metadata of existing established journals. These first emerged around 2010; they are closely related to Citejacked journal. So far, about 250 cases are documented, none from mathematics; recently, journals from Springer Nature and Elsevier have been targeted. In a recent study, about 66 % of articles in hijacked journals contained evidence of plagiarism [4].

Hijacked journal, v2: A journal whose editorial board has been overtaken or infiltrated by members of a citation cartel. Sometimes, the journal becomes permanently predatory.

Paper mill: A commercial organization that engages in the large scale production and sale of papers to researchers, academics, and students for publication in peer reviewed journals. Many paper mill papers include fabricated data [11]. A detailed analysis of a Russian paper mill is given in [3].

Predatory journal: Journal exploiting the open-access publication model to deceive authors into paying them a fee. These publishers often lie about the journal’s impact factor, have poor editorial standards and falsely claim to provide a rigorous peer-review process.

A famous example is the journal Advances in Difference Equations; zbMath statistics clearly reveal the presence of a strong citation cartel. After several failed attempts to renew it, SpringerNature decided to close it. The last paper was published in 2021. There is no list of predatory journals; however, recommendations on this issue may be found in [43].

Predatory conference: A conference with weak or no peer review for presentations, poor organization and a focus on making money for the organizers. They are typically taking place in large conference venues, with no or very few people actually present. Emphasis is often on virtual participation options and the promise of subsequent publication indexed in reputable citation databases.

Scientific Sleuth: Persons looking for problems in the scientific literature, often in their spare time.

Tortured phrases: Incorrect scientific vocabulary that stems from machine translation / paraphrase / generation of texts [9, 10]. They are the indicator used by the Problematic Paper Screener [45]. A collection of tortured phrases from mathematics and data science (collected from real papers flagged by the Problematic Paper Screener) may be found in Table 3.

References

[1] Robert Adler, John Ewing, and Peter Taylor, Citation statistics: a report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS), Statist. Sci. 24 (2009), no. 1,1–14, DOI 10.1214/09-STS285. MR2561120

[2] Ilka Agricola, Lynn Heller, Wil Schilders, Moritz Schubotz, Peter Taylor, and Luis Vega, Fraudulent Publishing in the Mathematical Sciences, digital version with all references, 2025, https://arxiv.org/abs/2509.07257.

[3] Anna Albakina, Publication and collaboration anomalies in academic papers originating from a paper mill: Evidence from a Russia-based paper mill, Learned Publishing (2023), 689–702, https://doi.org/10.1002/leap.1574.

[4] Anna Albakina, Prevalence of plagiarism in hijacked journals: A text similarity analysis, Accountability in Research, 2024, https://doi.org/10.1080/08989621.2024.2387210.

[5] Mario Biagioli and Alexandra Lippman (eds.), Gaming the Metrics: Misconduct and Manipulation in Academic Research, The MIT Press, 2020 (OA), https://doi.org/10.7551/mitpress/11087.001.0001.

[6] Lonni Besan¸con, Guillaume Cabanac, Cyril Labb´e, and Alexander Magazinov, Sneaked references: Fabricated reference metadata distort citation counts, J. Assoc. Inf. Sci. Technol. 75 (2024), no. 12, 1368–1379, https://doi.org/10.1002/asi.24896.

[7] Jeffrey Brainard, Fast-growing open-access journals lose impact factors, Science 379 (2023), no. 6639, 1283–1284, https://doi.org/10.1126/science.adi0098.

[8] Jennifer A. Byrne, Yasunori Park, Reese A.K. Richardson, Pranujan Pathmendra, Mengyi Sun, and Thomas Stoeger, Protection of the human gene research literature from contract cheating organizations known as research paper mills, Nucleic Acids Research 50 (2022), 12058–12070, https://doi.org/10.1093/nar/gkac1139.

[9] Guillaume Cabanac, Chain retraction: how to stop bad science propagating through the literature, Nature 632 (2024), 977–979, https://doi.org/10.1038/d41586-024-02747-1.

[10] Guillaume Cabanac, Cyril Labb´e, and Alexander Magazinov, Tortured phrases: A dubious writing style emerging inscience. Evidence of critical issues affecting established journals, 2021, https://arxiv.org/pdf/2107.06751.

[11] C. Candal-Pedreira, J. S. Ross, A. Ruano-Ravina, D.S. Egilman, E. Fern´andez, and M. P´erez-R´ıos, Retracted papers originating from paper mills: cross sectional study, BMJ 379 (2022), e071517, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9703783/.

[12] Alan Carey, Michael G. Cowling, and Peter Taylor, Assessing research in the mathematical sciences, Gazette of the Australian Mathematical Society 34 (2007), no. 2.

[13] Minal Caron, Carolin Lye, Barbara Bierer, and Mark Barnes, The PubPeer conundrum: Administrative challenges in research misconduct proceedings, Accountability in Research (2024), 1–19, https://doi.org/10.1080/08989621.2024.2390007.

[14] Michele Catanzaro, Citation cartels help some mathematicians—and their universities—climb the rankings, Science, 30 Jan. 2024, https://doi.org/10.1126/science.zcl2s6d.

[15] Lauranne Chaignon, Highly Cited Researchers: anatomy of a list, Quantitative Science Studies 6 (2025), 305–327, https://doi.org/10.1162/qss_a_00351.

[16] Dalmeet Singh Chawla, Hundreds of ‘predatory’ journals indexed on leading scholarly database, Nature News article, 8 February 2021, https://doi.org/10.1038/d41586-021-00239-0.

[17] Farshid Danesh and Ali Mardani-Nejad, A Historical Overview of Bibliometrics, in: Handbook Bibliometrics, ed. Rafael Ball, De Gruyter, 2021, https://doi.org/10.1515/9783110646610-003.

[18] Domingo Docampo, The dark world of ‘citation cartels’: Predatory journals and bad faith scholars are gaming the system—at scale, The Chronicle of Higher Education, March 6, 2024, https://www.chronicle.com/article/the-dark-world-of-citation-cartels.

[19] Edward Dunne, Don’t count on it, Notices Amer. Math. Soc. 68 (2021), no. 1, 114–118. MR4658679

[20] Erik von Elm, Greta Poglia, Bernhard Walder, and Martin R. Tramèr, Different patterns of duplicate publication: An Analysis of articles used in systematic reviews, JAMA 291 (2004), no. 8, 974–980, https://doi.org/10.1001/jama.291.8.974.

[21] Susan A. Elmore and Eleanor H. Weston, Predatory Journals: What They Are and How to Avoid Them, Toxicol. Pathol. 48 (2020), no. 4, 607–610, https://doi.org/10.1177/0192623320920209.

[22] Alex Glynn, Suspected Undeclared Use of Artificial Intelligence in the Academic Literature: An Analysis of the Academ-AI Dataset, 2024, https://arxiv.org/abs/2411.15218.

[23] Benoît Godin, On the origins of bibliometrics, Scientometrics 68 (2006), 109–133, https://doi.org/10.1007/s11192-006-0086-0.

[24] Klaus Hulek and Olaf Teschke, How do mathematicians publish?—Some trends, Eur. Math. Soc. Mag. 129 (2023), 36–41, https://doi.org/10.4171/MAG/160.

[25] Hazem Ibrahim, Fengyuan Liu, Yasir Zaki, and Talal Rahwan, Citation manipulation through citation mills and pre-print servers, Sci. Rep. 15 (2025), 5480, https://doi.org/10.1038/s41598-025-88709-7.

[26] V´ıt Mach ´aˇcek and Martin Srholec, Predatory publishing in Scopus: Evidence on cross-country differences, Quantitative Science Studies 3 (2022), 859–887, https://doi.org/10.1162/qss_a_00213.

[27] Stuart Macdonald and Jacqueline Kam, Aardvark et al.: quality journals and gamesmanship in management studies, Journal of Information Science 33 (2007), no. 6, https://doi.org/10.1177/0165551507077419.

[28] Layal Liverpool, AI intensifies fight against ‘paper mills’ that churn out fake research, Nature News, May 2023, https:// www.nature.com/articles/d41586-023-01780-w.

[29] Richard Van Noorden, More than 10,000 research papers were retracted in 2023—a new record, Nature 624 (2023), 479–481, https://doi.org/10.1038/d41586-023-03974-8.

[30] Martin Szomszor, David A. Pendlebury, and Jonathan Adams, How much is too much? The difference between research influence and self-citation excess, Scientometrics 123 (2020), 11191147, https://doi.org/10.1007/s11192-020-03417-5.

[31] Ivan Oransky and Adam Marcus, There’s far more scientific fraud than anyone wants to admit, The Guardian, August 9, 2023, https://tinyurl.com/mruef5ku.

[32] David A. Pendlebury, White Paper: using bibliometrics in evaluating research, Thomson Reuters, Philadelphia, 2008.

[33] David A. Pendlebury, Highly Cited Researchers 2023: The evolution of our evaluation and selection policy to support a robust scholarly landscape, blog entry, Nov. 15, 2023, https://tinyurl.com/yn355s5f.

[34] David A. Pendlebury, Citations and stature: Not so simpleanymore, blog entry, Nov. 19, 2024, https://clarivate.com/academia-government/blog/citations-and-stature-not-so-simple-anymore/.

[35] Vicente Saf ´on, Domingo Docampo, and Lawrence Cram, Screening articles by citation reputation, Quantitative Science Studies (2025), https://doi.org/10.1162/qss_a_00355.

[36] Marilyn Strathern, ‘Improving ratings’: audit in the British University system, European Review 5 (1997), no. 3, 305–321.

[37] Alexander Yong, Critique of Hirsch’s citation index: a combinatorial Fermi problem, Notices Amer. Math. Soc. 61 (2014), no. 9, 1040–1050, DOI 10.1090/noti1164. MR3241560

[38] BishopBlog, Collapse of scientific standards at MDPI journals: a case study, July 2024, https://deevybee.blogspot.com/2024/07/collapse-of-scientific-standards-at.html.

[39] Clarivate, History of citation indexing (not dated/signed), https://clarivate.com/webofsciencegroup/essays/history-of-citation-indexing/.

[40] Clarivate, HCRs from past years, https://clarivate.com/highly-cited-researchers/past-lists/.

[41] Federal Trade Commission, Case 2:16-cv-02022-GMNVCF, Document 86, FTC’s motion for summary judgment and memorandum in support thereof, May 2022, https://www.ftc.gov/system/files/documents/cases/omics_de_86_-_ftc_motion_for_summary_judgment.pdf.

[42] IMU Committee on Permissions, Ensuring widespread and equitable access to back issues of mathematics journals, Report, June 2024, https://www.mathunion.org/fileadmin/IMU/Report/2024-IMU_Committee_on_Permissions-FinalReport.pdf.

[43] IMU-ICIAM joint statement & recommendations, How to Fight Fraudulent Publishing in the mathematical sciences, 2025.

[44] National Science Foundation, Notice to research community: Use of generative artificial intelligence technology in the NSF merit review process, December 2023, https://new.nsf.gov/news/notice-to-the-research-community-on-ai?ref=exo-insight.ghost.io.

[45] The Problematic Paper Screener, https://www.irit.fr/~Guillaume.Cabanac/problematic-paper-screener/.

[46] Michael Harris, The Silicon Reckoner: What computers will do for, or to, mathematics and mathematicians, blog, https://siliconreckoner.substack.com.

[47] Retraction Watch Blog, Category: math retractions (updated regularly), https://tinyurl.com/2papjbps.

Original link: https://www.ams.org/journals/notices/202509/rnoti-p1038.pdf