BioModels Reproducibility Scorecard

Registration for this event has closed

Please note, by joining this workshop you are agreeing to follow the UKRN code of conduct.

Overview

Systems biology modelling involves the mathematical representation of biological processes to study complex system behaviour and was expected to be least affected by the reproducibility crisis. However, models often fail to reproduce and the reasons for the failure and prevalence were not fully understood. In a recent study [1], we analyzed 455 kinetic models published in 152 peer-reviewed journals. Most of these models were manually encoded from scratch to assess the reproducibility. Our investigation revealed that 49% of the models could not be reproduced using the information provided in the manuscripts. Among the corresponding authors, we contacted over 70% did not respond. Models across several life science journals failed to reproduce, revealing a common problem in the peer-review process. As a solution, we proposed a simple easy-to-use reproducibility scorecard [2] with 8 yes-or-no questions that can be used by model authors, reviewers and journal editors during the peer-review process. We demonstrated that the scorecard is a good indicator of model reproducibility. In this workshop we discuss about our study in details, run a hands-on training to use the scorecard to assess models during the peer-review process. The reproducibility crisis in systems biology modelling can be tackled as a community, where model authors, reviewers, journal editors and funding bodies embrace reproducibility more proactively than before.

[1] Tiwari K, Kananathan S, Roberts MG, Meyer JP, Sharif Shohan MU, Xavier A, Maire M, Zyoud A, Men J, Ng S, Nguyen TVN, Glont M, Hermjakob H, Malik-Sheriff RS. Reproducibility in systems biology modelling. Mol Syst Biol. 2021 Feb;17(2):e9982. https://doi:10.15252/msb.20209982
[2] Tiwari, Krishna, Waltemath, Dagmar, Hermjakob, Henning, & Malik-Sheriff, Rahuman S. (2021). Reproducibility Scorecard to Assess Systems Biology Models (1.1). Zenodo. https://doi.org/10.5281/zenodo.5786693

Speaker

Dr Rahuman Sheriff is a Project Leader at EMBL-EBI, Cambridge, UK. He develops tools and resources for systems modelling, and chemoinformatics. He leads BioModels repository, a leading resource for curated mathematical models of biological processes. He is an editorial board member of Systems Biology Markup Language (SBML) a community standard for encoding and exchanging models. He has over 13 years of experience in data analysis, integration, modelling and management working at EMBL-EBI, MRC-LMS, Imperial College London, and Max Planck Institute, Germany.

Programme (GMT+1/BST)

13:00 – Welcome
13:05 – Talk on our initiative: “BioModels and reproducibility study”
13:45 – Q&A
13:55 – Break (10 min)
14:05 – Introduction to reproducibility scorecard
14:15 – Participants score published models using the reproducibility scorecard
14:45 – Discussion and wrap-up
15:00 – End