"I may have committed some light fraud."
"I may have committed some light fraud."
This is the 27th installment of 774: Weekly lessons from history about science, technology, and the miscellaneous.
This Week:
Where to work if you want to commit fraud
A month ago The Atlantic released an article titled “Why Are Gamers So Much Better Than Scientists at Catching Fraud?”1 The author, Stuart Ritchie, juxtaposes two subsets of people vying for clout within their professional or social circles. The first are scientists. Researchers who perform various studies in areas like social science and medicine. The second group is gamers. Particularly video game players who do what is called a “speedrun,” completing a game in as short a time as possible. Individuals in both these groups have occasionally been found to conduct some form of fraud to polish their reputation.
The way a speedrunner will commit fraud is to either cut together various clips of the successful completion of each level in a video game until the video appears to be an uninterrupted stream of gameplay from start to finish. A speedrunner can also manipulate the game’s code to maximize favorable outcomes in the game, making it easier to beat. The way researchers commit fraud is generally to either make up data and data sources or to manipulate data in such a way as to represent it as something it is not. Ritchie claims that although the importance of honest science far outweighs that of honest speedruns, viewers of speedruns seem to be better at identifying fraud than scientists are, even though it is not any easier to do, and in some cases is harder. The signals of possible fraud in speedrunning and scientific research are similar to those in many other industries: results that seem too good to be true. In speedrunning the signal of possible fraud would be completing a game with just a bit too much luck at a speed that seems almost impossible. In science, fraud could be data sets that all too cleanly fit a hypothesis that a scientist has conjectured. In accounting, it might be reporting revenues that seem astoundingly good for the industry. In the case of writing, it might be having an amazing quote that fits a narrative quite well, but has never been cited anywhere else before. In theory, the identifying factors of something that might be fraudulent are pretty easy to find. It’s actually determining if something is in fact fraudulent instead of just “too good to be true” that can get tricky.
What about speedrunning lends itself to catching fraudsters faster? Is there a rank order to the kinds of industries that are easier to commit fraud in? Speedrunning, science, writing, and accounting all have something in common when it comes to fraud in their ranks. Their success is process-oriented. Accounting isn’t the balance sheet that an accountant creates, it’s the process used to create a balance sheet. Science isn’t conclusions about the world. It’s the process of finding out what those conclusions are. The same goes for speed runs and writing. In reality though, while these are all process-oriented, some are viewed as being more of a process than others. If you hold a book in your hands, you can follow along with the author’s writing process to some degree and experience it as they did. With speedrunning, those who are really into it actually watch the videos on YouTube of people completing Minecraft in record times, instead of just looking at the times. If you’re following along with the process, step by step, it’s easier to spot an anomaly. Science and accounting, though, are more about their results. The average person isn’t going to dig past a company’s income statement or read past the abstract of a published paper. It’s not fun for most people to read the methodology section of a paper saying that X causes cancer; they just want to know if X causes cancer. All four categories are processes to those who do them but don’t necessarily appear that way to those who observe them from the outside of them. The more process-driven a venture is to the casual viewer, the less “acceptable” fraud will be, even if fraud isn’t that big of a deal. The less process-driven, the more acceptable fraud will be, even if it’s a matter of life and death.
So you’ve been caught committing fraud
You’re an accountant, writer, speedrunner, or scientist and you’ve just been caught committing fraud.
Accountant: You cooked the books for your client and got caught and reported by an attentive employee. You lost your CPA license, got slapped with a fine, landed in a prison cell, or all of the above.
Writer: You recently published the hottest social science book of the Summer. You’re bumping elbows with Malcolm Gladwell at the New Yorker Festival. Life is good until a scrappy reporter in Queens named Michael Moynihan finds some of your quotes attributed to Bob Dylan to be dubious. He does some digging to find out if the quotes are real. They aren’t. The reporter publishes the story and it makes national news. The book is pulled from the shelves. Future book deals and speaking gigs are canceled. It is a career-ending event.2
Speedrunner: You’re a hobbyist. You don’t have much to lose from the fallout, save the clout you derive from online popularity and you may get a fair amount of supplementary income from your fans through Twitch. Both of these can be instantly curtailed if you’re deemed a fraud. Not world-shattering in most instances, but certainly not an enjoyable experience. Nobody wants to give tips to a fraud. So you lose your internet street cred and possibly some extra cash.
Scientist: The previous three people experienced some moderate to serious loss from engaging in fraud and it stands to reason the scientist would too. First, let’s start with what counts as a successful scientist. There are two primary metrics:
Number of published papers
The number of times someone cites your paper
There are also titles and awards, but the most objective way to measure scientific prestige are the two metrics above. Imagine again that you’re a scientist committing fraud. You get busted. Some papers you published, generally the ones in which the fraud occurred, will be redacted, but many of your other papers will remain in print. Researchers from MIT found that of the papers by fraudsters that remain, there’s an average of a 10% decrease in citations.3 A 10% increase is a small price to pay compared to the penalty the other fraudsters are subjected to. Particularly when considering some of the kinds of papers that fraud has been identified in. Ritchie highlights “Investigation of Force Received at the Upper Teeth by Video Laryngoscopy” and “Below-Knee Amputation Performed With Pericapsular Nerve Group and Sciatic Nerve Blocks.” Both papers are by Hironobu Ueshima, an anesthesiology researcher at Showa University in Japan.
Ueshima had turned out to be one of the most prolific scientific frauds in history, having partly or entirely fabricated records and data in at least 84 scientific papers, and altered data and misrepresented authorship on dozens more… Ueshima would eventually come clean and apologize—but only after a data sleuth had spotted strange anomalies in his publications
While Ueshima was eventually caught, his string of scientific fraud went far too long without catching the ire of the scientific community.
… it took 12 years for any action to be taken against the world’s most prolific scientific fraudster, Yoshitaka Fujii (coincidentally, another Japanese anesthesiologist), even after very convincing analyses of his dodgy data were published. Like Dream’s speedrun, Fujii’s data were just too good to be true: The fraud-spotters wrote, with admirable literalness, that they were “incredibly nice!”
There’s a strange asymmetry in certain fields between the real-world cost for people affected by fraud and the actual price for committing fraud.
Science Banana regularly reviews the methodology of published research that makes bold claims using dubious research.