Directly or indirectly, healthcare statistics and Health Services Research have a lasting influence on society. To foster responsible interpretation and reporting of research findings to policy makers, professionals and the public, a reflection of the conduct and communication of research is required. This thesis first explores the interpretation of publicly reported statistics and proposes a method to improve the interpretation of publicly reported statistics on health and healthcare. Second, this thesis explores the responsible reporting of Health Services Research in scientific and societal publications. These topics are addressed using a combination of quantitative and qualitative data collected through semi-structured interviews, focus groups, document analyses, and surveys amongst researchers, policy makers, science journalists/communicators, knowledge integration specialists, and students.
Research integrity is one of the main pillars constituting trustworthy science. However, fuelled by large, often spectacular cases of research misconduct, research integrity has come under the attention of scholars, science policymakers and the larger public alike. Combined with increasing concerns about the (ir)reproducibility of scientific experiments and an apparent decline in public trust in science, some consider the global research enterprise to be in a status of crisis. This dissertation aims to study research integrity, by describing how it may be threatened and how it may be protected.
In three empirical parts, it considers three questions regarding research integrity and the scientific publication system. With these, it aims to contribute to an understanding about: How problematic research gets spilled into the research literature, how self-regulating mechanisms may filter such articles from the literature, and how whistleblowers and research performing organisations may clean up cases that slipped through.
Psychology is facing a “replication crisis”. Many psychological findings could not be replicated in novel samples, which lead to the growing concern that many published findings are overly optimistic or even false. In this dissertation, we investigated potential indicators of problems in the published psychological literature. In Part I of this dissertation, we looked at inconsistencies in reported statistical results in published psychology papers. To facilitate our research, we developed the free tool statcheck; a “spellchecker” for statistics. In Part II, we investigated bias in published effect sizes.
We showed that in the presence of publication bias, the overestimation of effects can become worse if you combine studies. Indeed, in meta-analyses from the social sciences we found strong evidence that published effects are overestimated. These are worrying findings, and it is important to think about concrete solutions to improve the quality of psychological research. Some of the solutions we propose are preregistration, replication, and transparency. We argue that to select the best strategies to improve psychological science, we need research on research: meta-research.
There is little research on research integrity in low- and middle-income countries (LMICs). This thesis investigates perceived and actual research reporting practices in relation to authorship, plagiarism, redundant publication and conflicts of interest amongst LMIC health researchers.
Replication is one of the central aspects of the scientific process. In science, we replicate research findings in order to test their reliability. In cognitive neuroscience as well, replication is a powerful tool aimed at filtering out individual and other biases, through mutual agreement between results of similar experiments performed by independent scientists. In the fields of experimental psychology and cognitive neuroscience, the suggestion has been made that replication attempts are scarce, and erroneous findings may indeed exist. As a consequence, some researchers suggest that the field is experiencing a ‘crisis of confidence’.
In this thesis Wouter Boekel discusses three aspects in terms of their influence on reliability; Questionable research practices (QRP), the file drawer problem, and low sample sizes.
Recent studies have highlighted that not all published findings in the scientific literature are trustworthy, suggesting that currently implemented control mechanisms such as high standards for the reporting of research methods and results, peer review, and replication, are not sufficient. In psychology in particular, solutions are sought to deal with poor reproducibility and substandard replicability of research results. These problems are believed to be due to bias resulting from failure to publish all relevant research findings, common errors in the reporting of statistical results, and flexibility in the way data are analyzed.
In this dissertation these problems are considered from the perspective that the scientific enterprise must better recognize the human fallibility of scientists. Therefore, Coosje Veldkamp studied perceptions of the characteristics of scientists and examined the prevalence of statistical reporting errors. Finally, she presents an overview of many potential biases in hypothesizing, designing, collecting, analyzing, and reporting of psychological experiments, and evaluated the promising method of pre-registration as a way to deal with these biases.
Over the last few years the ethics of reporting and dissemination of study findings has been given more attention. This seems to be especially the case in the field of biomedical research. The Helsinki Declaration now states that ‘Researchers, authors, sponsors, editors and publishers all have ethical obligations with regard to the publication and dissemination of the results of research’.
For his dissertation on diagnostic research, Daniel Korevaar (University of Amsterdam) analysed the overall quality of diagnostic accuracy studies from the perspective of reporting and dissemination. Korevaar concludes that we have reason for modest optimism; although more steps should be taken in order to increase the value of diagnostic research, there is already movement within the field itself.
The thesis describes aspects of the current publication culture that have negative effects: publication pressure, overwrought professors and complaining scientists. It is mainly about the negative aspects of the publication culture and about scientists who fiddle with research results under publication pressure and possibly as a result of personality traits.
A one-off science glossy called ‘The Joeri’ was made to go with the thesis (in Dutch).
Download the pdf of the PhD thesis of Joeri Tijdink here.