Reproducibility in Research: Systems, Infrastructure, Culture04 Oct 2017
The reproduction and replication of research results has become a major issue for a number of scientiﬁc disciplines. In computer science and related computational disciplines such as systems biology, the challenges closely revolve around the ability to implement (and exploit) novel algorithms and models. Taking a new approach from the literature and applying it to a new codebase frequently requires local knowledge missing from the published manuscripts and transient project websites. Alongside this issue, benchmarking, and the lack of open, transparent and fair benchmark sets present another barrier to the veriﬁcation and validation of claimed results. In this paper, we outline several recommendations to address these issues, driven by speciﬁc examples from a range of scientiﬁc domains. Based on these recommendations, we propose a high-level prototype open automated platform for scientiﬁc software development which eﬀectively abstracts speciﬁc dependencies from the individual researcher and their workstation, allowing easy sharing and reproduction of results. This new e-infrastructure for reproducible computational science oﬀers the potential to incentivise a culture change and drive the adoption of new techniques to improve the quality and eﬃciency – and thus reproducibility – of scientiﬁc exploration.