Source Nature

Detailed analysis of misconduct investigations into huge research fraud suggests institutional probes aren’t rigorous enough.

By day, Andrew Grey studies bone health. But over the past few years, he’s developed another speciality: the case of one of science’s most prolific fraudsters.

From 1996 to 2013, Yoshihiro Sato, a Japanese bone-health researcher plagiarized work, fabricated data and forged authorships — prompting retractions of more than 60 studies in the scholarly literature so far. Grey and colleagues at the University of Auckland in New Zealand and the University of Aberdeen, UK, are among the researchers who have raised concerns about Sato’s work over the past decade or so, and they have studied the case in detail — in particular, how universities involved in the research investigated concerns about his work and allegations of misconduct.

At the World Conference on Research Integrity in Hong Kong from 2 to 5 June, Grey’s team described its years-long efforts to clean up Sato’s literature, and presented its analysis of the inquiries conducted by four universities in Japan and the United States ensnared in the scandal (the team published its analysis of three investigations in a paper in February). Grey says their findings provide evidence to support a growing view in the academic community: that university investigations into research misconduct are often inadequate, opaque and poorly conducted. They challenge the idea that institutions can police themselves on research integrity and propose that there should be independent organizations to evaluate allegations of research fraud should.

The analysis is one of just a few to look closely at research-misconduct investigations, and the first to use a systematic approach to rate them, says C. K. Gunsalus, a specialist in research integrity at the University of Illinois at Urbana–Champaign, who was not part of the analysis. Too many research-misconduct investigations turn out to be inadequate or flawed, says Gunsalus, who had a hand in creating a 26-point checklist that university officials can use to guide probes into research misconduct, which Grey’s team used to rate the investigations.

The checklist questions an investigation’s scope, reliability and impact — for instance, whether the investigating committee included external members and whether evidence could have been tampered with. The team independently assessed each investigation report using the checklist; one report had addressed none of the points adequately and two others properly addressed only two or three points. “Overall, each report was considered unacceptable,” say Grey and colleagues.

Alarm bells

Sato, who died in 2016, studied and ran clinical trials of drugs and supplements that might help to prevent bone fracture. Researchers in the field began raising concerns about Sato’s work in the mid-2000s, when some questioned the speed at which Sato had recruited and assessed participants for some of his studies. Sato later apologized for not disclosing all the hospitals from which he had recruited participants, and admitted a mistake in one paper. But more researchers flagged irregularities about his papers to journals, and in 2016, Grey and colleagues published an analysis in Neurology that raised concerns about 33 of Sato’s studies. Sato admitted that three of these studies were fraudulent, asked for them to be retracted, and cleared his co-authors of any wrongdoing. Twenty-seven of those studies have now been retracted.

In 2017, Grey’s team also flagged their concerns about hundreds of Sato’s papers to four institutions that had co-authors on these studies — Kurume, Hirosaki and Keio universities in Japan and New York University’s Winthrop Hospital; Sato had been a researcher at Kurume and Hirosaki universities. Two institutions had already launched investigations into some of the work when Grey contacted them, and the others began investigations.

The researchers asked the institutions for the reports of their investigations to understand how they had responded to the allegations. None of the reports revealed exactly who or which papers had been investigated; one found that an unnamed researcher had committed misconduct, and two reports recommended that papers be retracted.

Grey’s team rated each report as inadequate overall. The researchers also suggested that the investigations focused too much on determining whether research misconduct had occurred, rather than on understanding the validity of the research in question and correcting or retracting unreliable articles. Grey and colleagues argue that protecting the integrity of the literature should be the priority of any investigation — because integrity can be compromised without evidence of misconduct.

Grey says that his and his colleagues’ motivation to pursue the case for so long is to correct the literature that clinicians and patients rely on. Academic institutions, publishers and journals have not been willing or able to do a comprehensive job, he says, so his team has persisted in raising concerns.

Further review

Gunsalus agrees that the Sato case highlights some of the problems with misconduct investigations, and says that if shortcomings emerge, further reviews may be needed. She suggests institutional panels should include external members and that officials should also use a standardized checklist to strengthen their processes. “There should be some way for journals, funders, patients and others to be assured of the credibility and thoroughness of university reviews,” says Gunsalus.

Grey’s findings also suggest that institutions in Japan — which has seen several high-profile research misconduct cases in recent decades — should review their processes for investigating misconduct, says Alan Price, a research-misconduct consultant in Texas.

The universities did not respond directly to criticisms of the investigations, which Nature flagged to them, but offered further details about their inquiries and the outcomes. Winthrop Hospital said that it spent more than a year investigating the concerns, including digging up receipts for lab equipment, but found no misconduct. Keio University said that its investigation included external experts and statistical analysis of data; it found no research misconduct, but some errors in methods and typos in studies.

Kurume University asked a committee of statisticians and medical researchers to investigate 39 papers authored by Sato, and found some data falsification and inappropriate authorships. It said that it cannot conclude whether fraud was involved in another 32 papers because Sato is dead and records for these experiments no longer exist. Hirosaki University — whose 2017 investigation found “research irregularities” in 14 research papers, 7 of which had already been retracted — did not respond to Nature’s request for comment.