
==== Front
Comput Struct Biotechnol J
Comput Struct Biotechnol J
Computational and Structural Biotechnology Journal
2001-0370
Research Network of Computational and Structural Biotechnology

S2001-0370(24)00265-4
10.1016/j.csbj.2024.08.006
Short Communication
Utilizing large language models in infectious disease transmission modelling for public health preparedness
Kwok Kin On kkokwok@cuhk.edu.hk
abc⁎1
Huynh Tom d
Wei Wan In a
Wong Samuel Y.S. a
Riley Steven ef
Tang Arthur arthur.tang@rmit.edu.vn
d⁎⁎2
a JC School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
b Hong Kong Institute of Asia-Pacific Studies, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
c Department of Infectious Disease Epidemiology, School of Public Health, Imperial College London, London, United Kingdom
d School of Science, Engineering and Technology, RMIT University, Viet Nam
e MRC Centre for Global Infectious Disease Analysis and Jameel Institute, Imperial College London, London, United Kingdom
f School of Public Health, Imperial College London, Norfolk Place, London W2 1PG, United Kingdom
⁎ Correspondence to: Room 421, School of Public Health Building, Prince of Wales Hospital, Hong Kong Special Administrative Region of China. kkokwok@cuhk.edu.hk
⁎⁎ Correspondence to: 702 Nguyen Van Linh Blvd., District 7, Ho Chi Minh City, Viet Nam. arthur.tang@rmit.edu.vn
1 ORCID: 0000-0002-2804-5433

2 ORCID: 0000-0002-6655-6883

08 8 2024
12 2024
08 8 2024
23 32543257
19 4 2024
7 8 2024
7 8 2024
© 2024 Published by Elsevier B.V. on behalf of Research Network of Computational and Structural Biotechnology.
2024

https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Introduction

OpenAI's ChatGPT, a Large Language Model (LLM), is a powerful tool across domains, designed for text and code generation, fostering collaboration, especially in public health. Investigating the role of this advanced LLM chatbot in assisting public health practitioners in shaping disease transmission models to inform infection control strategies, marks a new era in infectious disease epidemiology research. This study used a case study to illustrate how ChatGPT collaborates with a public health practitioner in co-designing a mathematical transmission model.

Methods

Using natural conversation, the practitioner initiated a dialogue involving an iterative process of code generation, refinement, and debugging with ChatGPT to develop a model to fit 10 days of prevalence data to estimate two key epidemiological parameters: i) basic reproductive number (Ro) and ii) final epidemic size. Verification and validation processes are conducted to ensure the accuracy and functionality of the final model.

Results

ChatGPT developed a validated transmission model which replicated the epidemic curve and gave estimates of Ro of 4.19 (95 % CI: 4.13- 4.26) and a final epidemic size of 98.3 % of the population within 60 days. It highlighted the advantages of using maximum likelihood estimation with Poisson distribution over least squares method.

Conclusion

Integration of LLM in medical research accelerates model development, reducing technical barriers for health practitioners, democratizing access to advanced modeling and potentially enhancing pandemic preparedness globally, particularly in resource-constrained populations.

Keywords

Large language model
Generative artificial intelligence
Infectious diseases
Mathematical transmission modelling
Simulation and modelling
==== Body
pmc1 Introduction

From the 2003 Severe Acute Respiratory Syndrome epidemic (SARS) in Hong Kong to the 2009 influenza pandemic and recent continued clusters of Middle-East Respiratory Syndrome (MERS) in the Middle East and South Korea, mathematical modelling has become increasingly influential in informing infectious diseases transmission mitigation strategies in recent years [1], [2], [3]. The coronavirus disease 2019 pandemic highlighted the widespread adoption of data-driven intervention strategies and resources management informed by modelling results such as the adoption of non-pharmaceutical interventions and border control measures in many populations including United Kingdom [4], the United States [5], and Hong Kong [6], [7]. Chat Generative Pre-trained Transformer (ChatGPT), a large language model (LLM) powered chatbot developed by OpenAI, has generated significant interest in the medical research community [8], [9], [10].

With their sophisticated natural language processing capabilities, LLM-powered chatbots can serve as a comprehensive resource for infectious diseases for essential disease-related information [11], [12]. Aside from understanding and generating natural language, LLM-powered chatbots are also adept at creating, refining and debugging computer code spontaneously through natural conversational interaction [13]. The naturalistic interaction of LLM-powered chatbots accommodate researchers who are less competent in computer programming to construct transmission models with fewer technical barriers. Unlike traditional chatbots that rely on predefined rules and scripts, LLM-powered chatbots like ChatGPT employ advanced natural language processing, machine learning, and deep learning techniques to mimic human conversation based on information from its knowledge domains [14]. These technologies enable LLM-powered chatbot to handle complex queries, engage in context-aware conversations and maintain follow-up conversations.

What distinguishes LLM-powered chatbots from other AI-chatbots, such as Siri and Alexa, is its profound ability to conduct detailed and context-sensitive dialogues across various domains of knowledge. Trained on an extensive corpus of text that includes books, articles, and websites, ChatGPT can draw from a vast range of contexts, mirroring human conversational patterns more closely than ever before. This capability allows LLM-powered chatbots to provide nuanced responses that include relevant background information, use appropriate terminology, and engage in a way that is contextually appropriate for the conversation. As a result, the interactions with ChatGPT are not only more natural and human-like but also more engaging and effective in delivering information.

The adoption of LLMs holds transformative potential for public health science by expanding the pool of health practitioners and researchers who can efficiently implement these modelling methods. For epidemiologists and clinicians, especially those deeply familiar with specific diseases, LLM-powered chatbots offer assistance in building practical computational models for disease transmission. For experienced researchers in disease modeling, LLM-powered chatbots can accelerate the development process, enabling the rapid construction and comparison of diverse model scenarios. The ability to quickly iterate and refine models empowers researchers to respond more agilely to emerging health crises, optimizing interventions in real-time based on evolving data and insights. The integration of LLM-powered chatbot in modeling infectious diseases could revolutionize how we manage disease outbreaks globally.

Our study aims to investigate the potential of LLM-powered chatbots in enhancing the process of developing infectious diseases models for public health professionals. Specifically, we are interested in their role in resource-constrained settings where access to advanced analytical tools and expert consultation may be limited. By evaluating these aspects, we aim to provide comprehensive insights into the potential benefits and challenges of adopting LLM technologies in the field of infectious disease management.

2 Method

To assess ChatGPT's utility in developing disease transmission models, we conducted a case study focusing on the collaborative model design process between a human health practitioner and ChatGPT. The public health practitioner initiated a dialogue with ChatGPT, outlining the objective to develop a classic disease transmission model using the susceptible-exposed-infected-recovered (SEIR) framework. Using 10 days of prevalence data which exhibits expoential charcteristics, the health practitioner aimed to estimate two important epidemiological parameters: i) the basic reproductive number (Ro): the average number of secondary cases generated by an infectious individual in a totally susceptible population [15] with its 95 % confidence interval (CI) and ii) the final epidemic size [16] based on prevalence data from the first ten days of an outbreak. Through natural language conversation, the practitioner and ChatGPT engaged in an iterative process of code generation, refinement, and debugging to enhance accuracy. This process included clarifying the objectives, discussing the fitting methods, refining the code structure such as standard variable naming conventions, and exploring alternative estimation techniques. The generated codes underwent thorough validation with the prevalence data of influenza outbreak in British boarding School in 1978 [17] to ensure technical correctness, consistency with epidemiological expectations, and readability. This step was crucial to confirm the reliability of the model developed with ChatGPT's assistance.

3 Result

We have included four application scenarios of a case study (Supplementary Material) demonstrating how ChatGPT can be employed to assist public health practitioners and researchers in developing and validating disease transmission model. In these scenarios, the public health practitioner and ChatGPT collaborated in a progressive refinement process, with discussions focusing on understanding the existing code, seeking clarity on the model development, refining the code structure, and exploring alternative fitting techniques. In the initial scenario, ChatGPT successfully developed a SEIR transmission model and was then fitted to the prevalence data (Prompt 1.5), resulting in an estimated Ro of 4.18. Additionally, the model also predicted that by the end of the 60-day period, 9836 out of population size of 10,000 (approximately 98 %) will have been infected (Prompt 1.6). We further examine the epidemiological characteristic of the model using the code provided in Prompt 1.7 and adjusting the β values, where Ro=βγ. This allows us to observe that: when Ro > 1, it triggers the exponential growth; Ro = 1 yields a flat epidemic curve, and Ro < 1 results in a decline in the epidemic curve. Subsequently, we constructed plots to depict the time series of individuals in different disease states (Prompt 1.8). During the second scenario, the human collaborator and ChatGPT engaged in an iterative refinement process to clarify the details of the least squares method (LSE) in Scenario 1 along with the newly introduced approach by the human practitioner, involving the maximum likelihood estimation (MLE) with a Poisson distribution. ChatGPT highlighted the shortcomings of LSE and clarified the advantages of the latter approach over the former. The revised code based on the latter estimation approach provided a revised but similar R0 estimate of 4.19 from the Poisson MLE method (Prompt 2.7). A plot of the observed data and the fitted models employing both the least squares and Poisson MLE methods suggested that both fitting approaches demonstrated good ability to reproduce the prevalence data (Fig. 1B). In Scenario 3, we demonstrated how epidemiologists co-work with ChatGPT to construct 95 % CI for Ro (4.13- 4.26). In Scenario 4, the validation of the model constructed by ChatGPT indicated that our MLE estimate of R₀ at 3.78 (Prompt 4.2), based on British boarding school influenza outbreak prevalence data, aligned with the estimate reported in [18].Fig. 1 A. Daily case incidence (in log scale) under different values of Ro. The model exhibits important epidemiological characteristics: when Ro > 1, it triggers exponential growth; Ro = 1 yields a flat epidemic curve, and Ro < 1 results in a decline in the epidemic curve. B. Comparison of two fitting methods: Least Squares vs. Poisson Maximum Likelihood Estimation for reproducing observed prevalence data. Both fitting approaches demonstrated good ability to reproduce the prevalence data.

Fig. 1

4 Discussion

Our exploration of simple disease transmission model development with ChatGPT through natural conversation yielded results comparable with a small team of health practitioners and researchers with experience in these techniques working on a problem for several days. LLMs’ natural language interface allows public health practitioners and researchers with less experience in computer programming to construct transmission models with fewer technical barriers. In this collaborative process, LLMs act as co-pilots in assisting public health practitioners, epidemiologists and clinicians to swiftly construct functioning initial transmission models and potentially develop a wide range of model variants for experimentation, selection, and comparison. LLMs drastically reduce the time required for development of complex models that characterize heterogeneous social mixing patterns [19] or utilize individual-based approaches [20] , thereby potentially transforming the entire modelling workflow. The integration of LLMs into the public health sector plays a pivotal role in bolstering pandemic preparedness. Rapid response is crucial when facing potential outbreaks, and LLMs can significantly expedite the preliminary analysis and understanding of a novel pathogen's transmission dynamics. By providing instantaneous modeling support, these systems allow for real-time scenario analysis, facilitating faster and more informed decision-making. Also, the code displayed a high level of consistency with our expectations in terms of functionality, structure, and included valuable comments. It was well-organized and accompanied by explicit explanations of programming logic, ensuring good readability.

Like any new technology, LLMs have limitations. Rigorous verification of the technical correctness of the code and information produced by LLMs is critical, as bugs or logic errors may have inadvertently crept in. Also, while LLMs can provide a variety of technically correct designs to any request, human practitioners must be able to understand subtle differences between these variations and are ultimately responsible for making the final choices. While engaging in interactive conversations with LLMs and presenting follow-up questions and/or instructions is an effective approach to address uncertainties in the generated results, at this stage, human practitioners must also establish a line-by-line understanding of the results. Somewhat counter-intuitively, user expertise in understanding the nuances of alternate transmission models becomes even more important to effectively manage the rapid model development using LLMs.

The proliferation of LLMs reduces technological and financial barriers across a range of domains, facilitating the localization of modelling, particularly in low-income populations. This may have the potential to bridge the gap for countries and areas with limited resources, as it democratizes access to advanced modeling techniques, catering especially to healthcare facilities in low-income regions. This approach can also apply to mitigating nosocomial transmissions, an area often overlooked by researchers due to a scarcity of resources and a dearth of modelling expertise. With sufficient safeguards against model misspecification, this could potentially promote more equitable preparedness measures and responses across diverse populations, ultimately strengthening global resilience in the face of future pandemics [21] and prompting us to assess potential bias in LLM-independent advanced analytics and modeling.

Declaration of Generative AI and AI-assisted technologies in the writing process

No AI and AI-assisted technologies are involved in the writing process.

CRediT authorship contribution statement

Kin On Kwok: Writing – review & editing, Writing –original draft, Visualization, Validation, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Tom Huynh: Writing – review & editing, Investigation, Data curation. Wan In Wei: Writing – review & editing, Methodology, Investigation, Formal analysis. Samuel Yeung Shan Wong: Writing – review & editing, Supervision, Resources. Steven Riley: Writing –review & editing, Validation, Supervision, Methodology. Arthur Tang: Writing – review & editing, Visualization, Validation, Software, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization.

Declaration of Competing Interest

None.

Appendix A Supplementary material

Supplementary material

.

Acknowledgement

K.O. Kwok acknowledges support from 10.13039/501100005847 Health and Medical Research Fund (reference numbers: INF-CUHK-1 , 17160302 , 18170312 , 20210232 and COVID1903008), General Research Fund (reference numbers: 14112818 , 24104920 ), Wellcome Trust Fund (United Kingdom, 200861/Z/16/Z ), Group Research Scheme, Funding Allocation to Faculties by Research Committee and Strategic Partnership Award for Research Collaboration of The Chinese University of Hong Kong. A. Tang acknowledges the support from RMIT Vietnam Strategic Innovation Challenge.

Appendix A Supplementary data associated with this article can be found in the online version at doi:10.1016/j.csbj.2024.08.006.
==== Refs
References

1 Kwok K.O. Leung G.M. Lam W.Y. Riley S. Using models to identify routes of nosocomial infection: a large hospital outbreak of SARS in Hong Kong Proc Biol Sci 274 1610 2007 Mar 7 611 617 17254984
2 Kwok K.O. Leung G.M. Riley S. Modelling the proportion of influenza infections within households during pandemic and non-pandemic years PLoS One 6 7 2011 e22089
3 Lee J. Chowell G. Jung E. A dynamic compartmental model for the Middle East respiratory syndrome outbreak in the Republic of Korea: a retrospective analysis on control interventions and superspreading events J Theor Biol 408 2016 Nov 7 118 126 27521523
4 Chapter 5: Modelling. GOV.UK. https://www.gov.uk/government/publications/technical-report-on-the-covid-19-pandemic-in-the-uk/chapter-5-modelling (accessed Oct 9, 2023).
5 CDC. COVID-19 Forecasting and Mathematical Modeling. Centers for Disease Control and Prevention; 2023, published online Jun 27. https://www.cdc.gov/coronavirus/2019-ncov/science/forecasting/forecasting-math-modeling.html (accessed Oct 9, 2023).
6 Liang J.B. Yuan H.Y. Li K.K. Wei W.I. Wong S.Y.S. Tang A. Path to normality: assessing the level of social-distancing measures relaxation against antibody-resistant SARS-CoV-2 variants in a partially-vaccinated population Comput Struct Biotechnol J 20 2022 4052 4059 35935805
7 Kwok K.O. Wan In W. Huang Y. Wong A. Tang A. Wong S.Y.S. Estimation of early phase local-to-local transmissibility and importation hazard of Coronavirus Disease 2019 (COVID-19) epidemic under assorted containment measures in Hong Kong Travel Med Infect Dis 45 2022 Jan 102226 (Feb)
8 Wei W.I. Leung C.L.K. Tang A. McNeil E.B. Wong S.Y.S. Kwok K.O. Extracting symptoms from free-text responses using ChatGPT among COVID-19 cases in Hong Kong Clin. Microbiol. Infect. 30 1 2024 142.e1 142.e3 10.1016/j.cmi.2023.11.002 Jan; Epub 2023 Nov 8. PMID: 37949111
9 Li J. Dada A. Puladi B. Kleesiek J. Egger J. ChatGPT in healthcare: a taxonomy and systematic review Comput Methods Prog Biomed 245 2024 108013 10.1016/j.cmpb.2024.108013
10 Dave T. Athaluri S.A. Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations Front Artif Intell 6 2023 1169595 10.3389/frai.2023.1169595
11 Kwok K.O. Wei W.I. Tsoi M.T.F. Tang A. Chan M.W.H. Ip M. How can we transform travel medicine by leveraging on AI-powered search engines? J Travel Med 30 2023 taad0586
12 Liu J. Wang C. Liu S. Utility of ChatGPT in clinical practice J Med Internet Res 25 2023 e48568 10.2196/48568
13 Tam W. Huynh T. Tang A. Luong S. Khatri Y. Zhou W. Nursing education in the age of artificial intelligence powered Chatbots (AI-Chatbots): are we ready yet? Nurse Educ Today 129 2023 105917
14 Thirunavukarasu A.J. Ting D.S.J. Elangovan K. Gutierrez L. Tan T.F. Ting D.S.W. Large language models in medicine Nat Med 29 2023 1930 1940 37460753
15 Anderson R.M., Anderson B., May R.M. Infectious Diseases of Humans: Dynamics and Control. Oxford University Press, Oxford; 1992.
16 Barril C. Bliman P.A. Cuadrado S. Final size for epidemic models with asymptomatic transmission Bull Math Biol 85 2023 52 37156965
17 R.: Boarding school influenza data. https://search.r-project.org/CRAN/refmans/epimdr/html/flu.html.
18 Kalachev L. Landguth E.L. Graham J. Revisiting classical SIR modelling in light of the COVID-19 pandemic Infect Dis Model 8 1 2023 Mar 72 83 10.1016/j.idm.2022.12.002 Epub 2022 Dec 16. PMID: 36540893; PMCID: PMC9755423. 36540893
19 Kwok K.O. Cowling B. Wei V. Riley S. Read J.M. Temporal variation of human encounters and the number of locations in which they occur: a longitudinal study of Hong Kong residents J. R. Soc. Interface 15 138 2018 20170838 10.1098/rsif.2017.0838 Jan
20 Roche B. Duboz R. Individual-Based Models for Public Health Handb. Stat. 37 2017 347 365 10.1016/bs.host.2017.08.008 Epub 2017 Oct 10
21 WHO. Strengthening WHO Preparedness for and Response to Health Emergencies; 2023, published online Jan 5. https://apps.who.int/gb/ebwha/pdf_files/EB152/B152_12-en.pdf (accessed Aug 7, 2024).
