Blog

Reproducibility is important as it shows that research results are reliable, not random or biased. In a previous blog post, we discussed the benefits of reproducibility in open research. However, many researchers find it challenging to make their work reproducible.

In this blog post, we examine the barriers to reproducibility and offer insight into making your work as reproducible as possible.

The reproducibility crisis

Many researchers have alluded to a “reproducibility crisis” in recent years. The reproducibility or replicability crisis refers to a current state in research in which the results of many studies are difficult or impossible to reproduce. It raises important questions about research practice and the validity of research findings. As such, the reproducibility crisis has been a prominent topic of conversation in recent years, particularly within psychology and the life sciences. 

Notably, a study found that over 70% of life sciences researchers could not replicate the findings of others, and about 60% could not reproduce their own results.

Barriers to reproducible research

What has caused the reproducibility crisis? Despite the apparent benefits of reproducible research, researchers encounter various challenges that hinder reproducibility. These barriers include:

• Lack of recognition or incentives: Researchers are often rewarded for publishing novel findings, while null or confirmatory results receive little recognition. This creates an environment where researchers are less motivated to invest more effort to reproduce studies with seemingly insignificant results.
• Unwillingness to share methods, data, and research materials: Some researchers are reluctant to share their data. They may choose not to share the data out of fear of being scooped by other researchers.
• Reproducibility requires additional time and skills: Achieving reproducibility often requires a significant time investment and skills not always covered in university education. To conduct research reproducibly, researchers may need to learn new software and tools, develop data and software engineering skills, and upskill in project management.  
• Poor research practices and study design: Poor research practices can lead to irreproducible research, including unclear methodologies, inaccurate statistical or data analyses, and insufficient efforts to minimise biases. Poor study design also makes it less likely that research will be reproducible.

How to make your work more reproducible

Creating reproducible research starts during the project planning phase and continues through the publication stage. You can increase the reproducibility of your work by following the recommendations provided below.

Sharing data, software, materials, workflows, and other tools

One of the most fundamental barriers to reproducibility is data unavailability. Independent analysis cannot be performed if the original dataset is not openly available. Researchers must access the original data, protocols, and key research materials to reproduce published work. Without access to all the research outputs, reproducibility is nearly impossible. This is why HRB Open Research has an open data policy that adheres to the FAIR data guidelines.

Some researchers avoid sharing their data and research materials because they fear being scooped by other researchers. However, you can share your data for reuse without fear of scooping by publishing your data in a repository. An open access data repository allows researchers to deposit and store research datasets. With repositories, you can also establish an embargo period for reuse, which is a period during which only the data owner can use the data, ensuring that you have the first opportunity to publish your findings. Read more about open data here.

 Additionally, repositories create a Digital Object Identifier (DOI) that enables your research to be more readily discovered and cited after the embargo period. By depositing your data in repositories, you allow your data, code, and other tools to be reused. These repositories store data in a way that allows immediate user access to anyone, so there are no access restrictions. Furthermore, describing your data with rich, meaningful, machine-readable metadata makes it easier for other researchers to find and replicate.

Publishing research intent before research begins

Publicly registering research ideas and plans increases the integrity of the results by clearly establishing authorship, ensuring that authors receive the recognition they deserve. This increases the study design quality and the results’ reliability and reproducibility. It also provides a solution to publication bias–where the decision to publicise or disseminate research is based on the perceived significance or interest of the results.

Publishing proposed research studies before initiating a study allows reviewers to evaluate and verify your research approach. This helps ensure the research is reproducible by ensuring that the research information gathered, interpreted, and reported is unbiased and easily replicated by other researchers.

Additionally, registration also provides opportunities for collaboration and reduces duplication of research efforts. To make your research more reproducible, you can publish your research intent as Method Articles and Registered Reports.

Publish negative data and confirmatory results

Both positive and negative (or null) results are essential for the progression of science. However, there is a general reluctance to publish negative results.

Promotion criteria for researchers often relies on noteworthy positive results, where emphasis is placed on publishing in high-impact publications. As a result, researchers aren’t typically rewarded for publishing negative or null results; instead, they are rewarded for publishing novel findings in the form of higher volumes of citations. This makes it challenging to encourage researchers to go to the extra effort of reproducing research and leads to an under-reporting of studies that produce seemingly insignificant results. As such, reproducibility in research is hindered by the under-reporting of studies that yield negative or null results.

By publishing your negative and null results, you prevent other researchers from wasting funding and resources trying to replicate studies that cannot be replicated. Your negative findings can also lead to new discoveries as others cite your research and adjust their experimental designs based on your findings.

This is why HRB Open Research supports the publication of negative or null results and allow submissions of this type of papers.

Incorporating new technology and tools into existing workflows

Another way you could champion reproducibility is by using new technology and tools to share your research data. Laboratory researchers increasingly use Electronic Laboratory Notebooks (ELNs) to record and access notebook entries. Recording, accessing, and preserving paper records can be slow, inefficient, and challenging to integrate with modern computer-controlled data capture systems. Alternatively, ELNs allow researchers to digitise their lab entries so they sit seamlessly alongside research data. This makes it easier for researchers across experiments to readily access, use, and share notebook data and quickly interpret meaning from results. This also helps facilitate reproducibility across experiments.

By sharing your laboratory entries digitally, you allow multidisciplinary research to occur more easily and is reproducible at scale. ELN entries contain important research metadata. By letting other researchers from diverse disciplines access, use, and share your laboratory data, you can assist in transferring experimental details across different research groups.

Furthermore, version control is an excellent tool to increase the reproducibility of your data and code. It’s difficult to reproduce research when data is disorganised or missing, or it’s impossible to determine where or how data originated. Using version control will allow you to manage your files better. Moreover, by sharing multiple versions of your research, you record how your data and code evolved over time. This allows other researchers to access, analyse, and reuse your data or code at a specific point in time, enabling greater reproducibility. HRB Open Research’s publishing process encourages article versioning, letting authors update their Articles after publication. Readers can access previous versions of published Articles to see how the research has developed since its original publication.

We hope you found the tips above helpful in making your research as reproducible as possible. To submit your work to HRB Open Research, visit our submissions page.

• Follow us on Twitter
• Sign up to our newsletter