Tuesday, 07 April 2026
Two researchers from the 海角黑料 are part of an international collaboration of 865 researchers behind the SCORE (Systematising Confidence in Open Research and Evidence) programme, which has released major new findings on research credibility in three papers published in the world’s leading science journal Nature.
SCORE is a large-scale, multi-method initiative offering new evidence on the reproducibility, robustness and replicability of research, and is designed to improve how scientific credibility is assessed in the social and behavioural sciences. SCORE is funded by the US Defence Advanced Research Projects Agency (DARPA).
The programme was led by an international team from the Centre for Open Science, the University of Melbourne, and Pennsylvania State University. It brought together teams from multiple institutions – including the 海角黑料’s Dr Joris Schroeder, Research Fellow in the School of Economics, and Dr Christopher Madan, Assistant Professor in the School of Psychology – to evaluate research using approaches such as reproduction and replication studies, expert assessments, and machine-learning models, while also creating one of the largest open datasets on research credibility.
When we read a published study, how confident can we really be in its findings? SCORE takes on this question at an unprecedented scale, and I'm glad to have been a small part of it.
Analysing claims from 3,900 papers across 62 journals (2009–2018), SCORE assessed the reliability of research (based on the three dimensions of reproducibility, robustness, and replicability), and the programme also tested whether humans and machines can predict which findings will replicate.
Key findings include:
- Reproducibility (same data, same analysis): Limited transparency hindered assessment – data was available for only 24 per cent of papers. Among those tested, 72 per cent reproduced approximately and 53 per cent exactly, with higher success when data and code were shared.
- Robustness (same data, different analyses): Results varied depending on analytical choices. Only 34 per cent of re-analyses closely matched original results, though 74 per cent reached similar overall conclusions.
- Replicability (new data): About 49 per cent of studies replicated successfully, with replication effects less than half the size of original findings.
It was great to be able to contribute to this project. Large field-wide collaborative projects like this are what truly move the field forward.
Additional findings show that credibility is multi-dimensional, with weak links between measures. Human predictions of replicability are fairly accurate, while machine predictions are less reliable. No single field consistently outperforms others, and there is no single metric for research credibility.
Overall, the results highlight that verifying research is complex but essential. The SCORE programme provides openly available data, tools, and materials to support future research and improve how scientific credibility is assessed.
The three research papers have been published in Nature.
Story credits
More information is available from Dr Joris Schroeder, via joris.schroeder@nottingham.ac.uk; or Liz Goodwin, Media Relations Manager for the Faculties of Arts and Social Sciences, via liz.goodwin@nottingham.ac.uk or 0115 748 5133.
Notes to editors:
About the 海角黑料
Ranked 97 in the world and 17th in the UK by the , the 海角黑料 is a founding member of Russell Group of research-intensive universities. Studying at the 海角黑料 is a life-changing experience, and we pride ourselves on unlocking the potential of our students. We have a pioneering spirit, expressed in the vision of our founder Sir Jesse Boot, which has seen us lead the way in establishing campuses in China and Malaysia - part of a globally connected network of education, research and industrial engagement.
Nottingham was crowned Sports University of the Year by 鈥� the third time it has been given the honour since 2018 鈥� and by the .
The university is among the best universities in the UK for the strength of our research, positioned seventh for research power in the UK according to REF 2021. The birthplace of discoveries such as MRI and ibuprofen, our innovations transform lives and tackle global problems such as sustainable food supplies, ending modern slavery, developing greener transport, and reducing reliance on fossil fuels.
The university is a major employer and industry partner - locally and globally - and our graduates are the third most targeted by the UK's top employers, according to The Graduate Market in 2024 report by High Fliers Research. Alongside Nottingham Trent University, we lead the , a pioneering collaboration between the city鈥檚 two world-class institutions to improve levels of prosperity, opportunity, sustainability, health and wellbeing for residents in the city and region we are proud to call home.