Piloting peer code review in a research consortium

ABSTRACT
Code for research is more flexible than point-and-click statistical softwares, but can be more error-prone. Although peer review is part of the scientific process, it rarely (though increasingly) involves review of research code. Part of the difficulty with implementing formal peer review of code is the time, expertise, and supportive environment required to successfully execute it. We present a pilot program for peer code review conducted within a research consortium setting, which may represent a useful model to overcoming these challenges. The Australia-Aotearoa Consortium for Epidemic Forecasting & Analytics (ACEFA) hub aims to support timely and effective responses to epidemic diseases in Australia and New Zealand through real-time data analytics, modelling, and forecasting. In parallel with a methods review, we have conducted peer code review of models. We also conducted pre- and post- evaluation surveys to evaluate the program, and to provide evidence for iterative improvements for future rounds of code review. In this presentation I will share what we learned from the program, what worked, and how it can be replicated by other groups.