Rawit Longsaward is a Ph.D. student at Mahidol University in Thailand. In 2021/2022, he was awarded a scholarship to spend a year in the Plant Chemetics Laboratory of Renier van der Hoorn at the University of Oxford. His objective was to study the extensively researched ribonuclease activity of plant defense-induced proteins belonging to the PR-10 family. Things didn’t go to plan, and his account of the events is detailed below. It is disheartening to note that his experience – wasting time and money due to irreproducible published results – is not unique.
The prevalence of bad science can be expensive and carries a significant cost for those attempting to build upon it, both in terms of financial resources and wasted effort. Scientific research that is flawed, incomplete, or based on inaccurate assumptions can lead to incorrect conclusions and misguided recommendations. In addition to wasting time and resources, bad science can also have serious consequences for public health, safety, and the environment.
And getting journals to publish contradictory data is hard, meaning the incentives for early career scientists to work on fixing the scientific record are limited. At least, these days we have the preprint server bioRxiv to highlight “Contradictory Results”.
Enough said. Let’s hear it in Rawit’s own words.
Cost of bad science: a case study from my cautionary note on PR-10 RNase activity
Hi, my name is Rawit Longsaward. I just finished my Ph.D. in the Botany program at Mahidol University, Thailand. I have been working on a project involving how rubber tree systemically responds to the white root rot pathogen. Since I was an undergrad student, I identified the novel uncharacterized PR-10 protein in rubber tree leaves as a sharply upregulated protein under white root rot disease condition (the article has been published in a journal, BMC Plant Biology).
During my Ph.D. from August 2021 to July 2022, I got a scholarship from the Thai government to do one-year research at the Plant Chemetics Laboratory, University of Oxford, under the main supervision of Prof.Renier van der Hoorn and Dr.Nattapong Sanguankiattichai or Tee. We have planned to test the role of the novel PR-10 protein from rubber tree in plant immunity using the Agromonas assay and test the well-known activity of PR-10 protein which is the RNase activity.
In the RNase activity validation of PR-10 proteins, most of the reports incubated the RNA with the purified PR-10 protein, either extracted from the native plant tissue or purified from the recombinant production in E. coli system, followed by visualizing the RNA degradation in the agarose gel electrophoresis. The negative controls varied from the buffer, the heat-inactivated PR-10 protein, the RNase inhibitor, the incubation with other proteins, etc. It seems easy to do the assay as we can just express the protein, purify it, and then perform an enzymatic assay with the RNA. I started to spend time on gene cloning, recombinant expression, optimization of histag purification, and the RNase assay itself. At least nine of the previously reported articles published in peer-reviewed journals included only the heat-inactivated protein as a negative control and I decided “wrongly” to follow them. I believed that the degraded RNA I observed from the assay and the loss of RNA degradation in the heat-inactivated control, is enough to presume that the rubber tree PR-10 has the ability to cleave RNA.
Improper negative control took us the material, expenses, and the most valuable limited “time”. As a younger scientist, I do need more expertise to be more criticizable with the assays, controls, and results from the number of peer-reviewed articles reporting RNase activity of PR-10 protein. No one ever shouts out to the community which negative control assay is the best for RNase characterization, and which one should be reconsidered. I spent months after that to see deeper in the degradation over time, converted the slightly degraded RNA to a set of cDNA, and amplified each of the RNA types remaining there using the own-designed primer pairs in RT-PCR. I also constructed the mutant protein by changing a potentially significant residue for RNase of the PR-10 of rubber tree. All RNA types were degraded four times when I replicated the RT-PCR, and the purified K52N mutant also showed activity.
Luckily that Prof.Renier and Tee asked me to consider adding incubation with the protein without RNase activity as the negative control. As mentioned in the cautionary note we preprinted, we recombinantly expressed and purified LacZ-His, GFP-His, and PRp27-His proteins using the same system and used the purified negative control proteins in the RNase assay. The false positive noticed in the negative controls were hypothesized to come from the contamination of RNase from the purification system, which is the same system as several previous reports. We also tried the In-gel RNase assay to prove whether the PR-10 of the rubber tree has RNase activity or not. The In-Gel RNase assay is tricky in the protein refolding and enzymatically activating in the gel, we need to optimize it under the limited time of my visit there. Unfortunately, I still have no solid answer to the RNase activity of the rubber tree PR-10 protein.
One of the bad science for me in this situation is I know already that several reports with unsolid results are still there, still being cited, and will be replicated as I did in the earlier period of time I mentioned. The cautionary note was, therefore, written to be a lesson to the community about this sensitive assay and preprinted via BioRxiv. And another one is that contradictory results like this are hard to be published in a regular platform like an impactable scientific journal.
Acknowledgements
The prologue was written with assistance from ChatGPT. Thanks to Rawit Longsaward for sharing his testimonial.
This article is available on a CC-BY license via Zenodo. Cite as: Kamoun, S. (2023) The cost of bad science: A PhD student testimonial. https://doi.org/10.5281/zenodo.7855424
