
==== Front
bioRxiv
BIORXIV
bioRxiv
2692-8205
Cold Spring Harbor Laboratory

10.1101/2024.09.04.611290
preprint
1
Article
Putting computational models of immunity to the test - an invited challenge to predict B. pertussis vaccination outcomes
Shinde Pramod http://orcid.org/0000-0002-1071-2744

Willemsen Lisa
Anderson Michael
Aoki Minori
Basu Saonli http://orcid.org/0000-0003-1200-4546

Burel Julie G http://orcid.org/0000-0003-1692-2758

Cheng Peng
Dastidar Souradipto Ghosh
Dunleavy Aidan
Einav Tal
Forschmiedt Jamie
Fourati Slim
Garcia Javier
Gibson William
Greenbaum Jason A
Guan Leying
Guan Weikang
Gygi Jeremy P
Ha Brendan
Hou Joe
Hsiao Jason
Huang Yunda
Jansen Rick
Kakoty Bhargob
Kang Zhiyu
Kobie James J http://orcid.org/0000-0001-8069-0272

Kojima Mari
Konstorum Anna
Lee Jiyeun
Lewis Sloan A
Li Aixin
Lock Eric F
Mahita Jarjapu
Mendes Marcus
Meng Hailong
Neher Aidan
Nili Somayeh
Olsen Lars Rønn
Orfield Shelby
Overton James A.
Pai Nidhi
Parker Cokie
Qian Brian
Rasmussen Mikkel
Reyna Joaquin
Richardson Eve
Safo Sandra
Sorenson Josey
Srinivasan Aparna
Thrupp Nicky
Tippalagama Rashmi
Trevizani Raphael
Ventz Steffen
Wang Jiuzhou
Wu Cheng-Chang
Ay Ferhat http://orcid.org/0000-0002-0708-6914

Grant Barry
Kleinstein Steven H
Peters Bjoern
08 9 2024
2024.09.04.611290https://creativecommons.org/licenses/by-nc-nd/4.0/ This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which allows reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the creator.
http://biorxiv.org/lookup/doi/10.1101/2024.09.04.611290
nihpp-2024.09.04.611290.pdf
Abstract

Systems vaccinology studies have been used to build computational models that predict individual vaccine responses and identify the factors contributing to differences in outcome. Comparing such models is challenging due to variability in study designs. To address this, we established a community resource to compare models predicting B. pertussis booster responses and generate experimental data for the explicit purpose of model evaluation. We here describe our second computational prediction challenge using this resource, where we benchmarked 49 algorithms from 53 scientists. We found that the most successful models stood out in their handling of nonlinearities, reducing large feature sets to representative subsets, and advanced data preprocessing. In contrast, we found that models adopted from literature that were developed to predict vaccine antibody responses in other settings performed poorly, reinforcing the need for purpose-built models. Overall, this demonstrates the value of purpose-generated datasets for rigorous and open model evaluations to identify features that improve the reliability and applicability of computational models in vaccine response prediction.
==== Body
pmc
