
==== Front
Neurol Res Pract
Neurol Res Pract
Neurological Research and Practice
2524-3489
BioMed Central London

39232852
338
10.1186/s42466-024-00338-z
Correspondence
Prognostic models in multiple sclerosis: progress and challenges in clinical integration
http://orcid.org/0000-0002-4386-1340
Havla Joachim joachim.havla@med.lmu.de

12
https://orcid.org/0000-0001-9325-6467
Reeve Kelly 3
https://orcid.org/0000-0002-9008-0141
On Begum Irmak 4
https://orcid.org/0000-0002-9955-8906
Mansmann Ulrich 4
https://orcid.org/0000-0003-3105-5840
Held Ulrike 3
1 grid.5252.0 0000 0004 1936 973X lnstitute of Clinical Neuroimmunology, LMU University Hospital, LMU Munich, Munich, Germany
2 grid.5252.0 0000 0004 1936 973X lnstitute of Clinical Neuroimmunology, Biomedical Center (BMC), Faculty of Medicine, LMU Munich, Munich, Germany
3 https://ror.org/02crff812 grid.7400.3 0000 0004 1937 0650 Epidemiology, Biostatistics and Prevention Institute, University of Zürich, Zurich, Switzerland
4 https://ror.org/05591te55 grid.5252.0 0000 0004 1936 973X Institute for Medical Information Processing, Biometry and Epidemiology, Faculty of Medicine, Ludwig-Maximilians-Universität München, Munich, Germany
5 9 2024
5 9 2024
2024
6 4422 3 2024
11 7 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
As a chronic inflammatory disease of the central nervous system, multiple sclerosis (MS) is of great individual health and socio-economic significance. To date, there is no prognostic model that is used in routine clinical care to predict the very heterogeneous course of the disease. Despite several research groups working on different prognostic models using traditional statistics, machine learning and/or artificial intelligence approaches, the use of published models in clinical decision making is limited because of poor model performance, lack of transferability and/or lack of validated models. To provide a systematic overview, we conducted a “Cochrane review” that assessed 75 published prediction models using relevant checklists (CHARMS, PROBAST, TRIPOD). We have summarized the relevant points from this analysis here so that the use of prognostic models for therapy decisions in clinical routine can be successful in the future.

Keywords

Multiple sclerosis
Prognostic model
Treatment decision
Validation
Personalized medicine
http://dx.doi.org/10.13039/501100006604 Bundesministerium für Bildung, Wissenschaft und Kultur Grant Numbers 01ZZ1603[A-D] and 01ZZ1804[A-H] (DIFUTURE) Mansmann Ulrich issue-copyright-statement© Deutsche Gesellschaft für Neurologie e.V. 2024
==== Body
pmcMain text

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system with great individual health and socio-economic relevance [1]. To date, there is no prognostic model that is used in routine clinical care to predict the very heterogeneous course of the disease. Such a model could aid in personalized therapy selection under the assumption that those at highest risk benefit most from treatment. In this way, over-, under- and/or expensive mistreatment risk could be minimized. MS care providers are fortunate to have multiple therapeutic options with different modes of action, which differ significantly in terms of effectiveness, but also in terms of the risk-benefit balance. There is evidence that early, highly effective therapy improves the long-term outcome of MS progression, but at the price of a higher therapy-associated burden [2]. With the help of good prognostic model research, we move toward solutions to enable personalized therapy on an individual level. Although several research groups are working on different prognostic models using traditional statistics, machine learning, and/or artificial intelligence approaches [3], the use of published models in clinical decision making has been limited, due to poor model performance, lack of transportability and/or lack of validated models.

In order to provide a systematic overview we conducted a “Cochrane Review” assessing 75 prediction models published between January 1996 and July 2021 using prognostic modeling relevant checklists for data extraction (CHARMS), risk of bias assessment (PROBAST), and completeness of reporting (TRIPOD) [3]. Evidence on the performance of a prediction model can only be combined when there is at least three external validations outside the model development process, preferably led by independent researchers [4]. However, no published model met this gold standard [3]. Of these 75 candidate models, only 12 were externally validated at all and only two of these multiple times (Fig. 1). No external validations were performed by independent researchers. In addition, the comparability of the models was limited. The prognostic models used heterogeneous outcomes with different definitions, such as disease progression (41%), conversion to secondary progressive MS (28%), conversion to definitive MS diagnosis (18%) or occurrence of relapses (8%). On the other hand, the rapid development with an increase in treatment options, the availability of markers and the diversification of the diagnostic criteria severely limits the comparability of different cohorts and models. Some models show only limited applicability in non-specialized treatment settings. Furthermore, 52% of the prognostic models lacked clear reporting or instructions to enable their validation in other cohorts.

Fig. 1 Summary of findings of [3]. Databases used for literature search: MEDLINE, EMBASE, Cochrane. 2 externally validated models: Manouchehrinia et al., 2019 [8] Bergamaschi et al., 2015 [9]

The following points would facilitate the translation of prognostic models for aiding treatment decisions in MS routine:

Prognostic model research should be undertaken by teams with expertise in MS and its treatment, in data collection process, in prediction algorithms, and all aware of the expectations of clinical prognostication and its reporting.

Independent external validation is a vital and ongoing process [5], especially in the dynamic MS domain with ever-changing disease definitions.

Model development publications clearly document the development and evaluation steps and guide the implementation following TRIPOD, including description of the intended time of model use and prediction horizon. The new TRIPOD + AI (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis + Artificial Intelligence) provides harmonized guidelines for the reporting of prediction model studies, regardless of whether regression models or machine learning methods were used [6].

To give an example of how such a model development could be methodologically implemented, a multicenter prospective cohort study was planned and conducted solely for the purpose of external validation of a prognostic score, ensuring compliance with the methodological guidelines [7].

By applying above mentioned recommendations, future prognostic models could overcome limitations and contribute to personalized prognostication in people with MS [3].

Acknowledgements

Not applicable.

Author contributions

JH: Writing – Original draft; All Authors: made substantial contributions to the conception or design of the work and the acquisition, analysis, or interpretation of data; revised the work critically for important intellectual content and approved the version to be published; agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project was partially funded by the German Federal Ministry of Education and Research (Grant Numbers 01ZZ1603[A-D] and 01ZZ1804[A-H] (DIFUTURE)). The funder had no role in the design of the project; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data availability

Not applicable.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

JH reports a grant for OCT research from the Friedrich-Baur-Stiftung Horizon, Roche and Merck, personal fees and nonfinancial support from Merck, Alexion, Novartis, Roche, Celgene, Biogen, Bayer and Horizon and nonfinancial support of the Sumaira-Foundation and Guthy-Jackson Charitable Foundation, all outside the submitted work. All other authors report no conflict of interest to this work.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Jakimovski D Bittner S Zivadinov R Morrow SA Benedict RH Zipp F Weinstock-Guttman B Multiple sclerosis Lancet 2024 403 10422 183 202 10.1016/S0140-6736(23)01473-3 37949093
Jakimovski, D., Bittner, S., Zivadinov, R., Morrow, S. A., Benedict, R. H., Zipp, F., & Weinstock-Guttman, B. (2024). Multiple sclerosis. Lancet, 403(10422), 183–202. 10.1016/S0140-6736(23)01473-3.37949093 10.1016/S0140-6736(23)01473-3
2. Bayas A Berthele A Hemmer B Warnke C Wildemann B Controversy on the treatment of multiple sclerosis and related disorders: Positional statement of the expert panel in charge of the 2021 DGN Guideline on diagnosis and treatment of multiple sclerosis, neuromyelitis optica spectrum diseases and MOG-IgG-associated disorders Neurol Res Pract 2021 3 1 45 10.1186/s42466-021-00139-8 34362474
Bayas, A., Berthele, A., Hemmer, B., Warnke, C., & Wildemann, B. (2021). Controversy on the treatment of multiple sclerosis and related disorders: Positional statement of the expert panel in charge of the 2021 DGN Guideline on diagnosis and treatment of multiple sclerosis, neuromyelitis optica spectrum diseases and MOG-IgG-associated disorders. Neurol Res Pract, 3(1), 45. 10.1186/s42466-021-00139-8.34362474 10.1186/s42466-021-00139-8
3. Reeve K On BI Havla J Burns J Gosteli-Peter MA Alabsawi A Alayash Z Gotschi A Seibold H Mansmann U Held U Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis Cochrane Database Systematic Review 2023 9 9 CD013606 10.1002/14651858.CD013606.pub2
Reeve, K., On, B. I., Havla, J., Burns, J., Gosteli-Peter, M. A., Alabsawi, A., Alayash, Z., Gotschi, A., Seibold, H., Mansmann, U., & Held, U. (2023). Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis. Cochrane Database Systematic Review, 9(9), CD013606. 10.1002/14651858.CD013606.pub2.10.1002/14651858.CD013606.pub2
4. Debray TP Damen JA Snell KI Ensor J Hooft L Reitsma JB Riley RD Moons KG A guide to systematic review and meta-analysis of prediction model performance Bmj 2017 356 i6460 10.1136/bmj.i6460 28057641
Debray, T. P., Damen, J. A., Snell, K. I., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. (2017). A guide to systematic review and meta-analysis of prediction model performance. Bmj, 356, i6460. 10.1136/bmj.i6460.28057641 10.1136/bmj.i6460
5. Van Calster B Steyerberg EW Wynants L van Smeden M There is no such thing as a validated prediction model Bmc Medicine 2023 21 1 70 10.1186/s12916-023-02779-w 36829188
Van Calster, B., Steyerberg, E. W., Wynants, L., & van Smeden, M. (2023). There is no such thing as a validated prediction model. Bmc Medicine, 21(1), 70. 10.1186/s12916-023-02779-w.36829188 10.1186/s12916-023-02779-w
6. Collins GS Moons KGM Dhiman P Riley RD Beam AL Van Calster B Ghassemi M Liu X Reitsma JB van Smeden M Boulesteix AL Camaradou JC Celi LA Denaxas S Denniston AK Glocker B Golub RM Harvey H Heinze G Hoffman MM Kengne AP Lam E Lee N Loder EW Maier-Hein L Mateen BA McCradden MD Oakden-Rayner L Ordish J Parnell R Rose S Singh K Wynants L Logullo P TRIPOD + AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods Bmj 2024 385 e078378 10.1136/bmj-2023-078378 38626948
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., Boulesteix, A. L., Camaradou, J. C., Celi, L. A., Denaxas, S., Denniston, A. K., Glocker, B., Golub, R. M., Harvey, H., Heinze, G., Hoffman, M. M., Kengne, A. P., Lam, E., Lee, N., Loder, E. W., Maier-Hein, L., Mateen, B. A., McCradden, M. D., Oakden-Rayner, L., Ordish, J., Parnell, R., Rose, S., Singh, K., Wynants, L., & Logullo, P. (2024). TRIPOD + AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. Bmj, 385, e078378. 10.1136/bmj-2023-078378.38626948 10.1136/bmj-2023-078378
7. Bayas, A., Mansmann, U., On, B. I., Hoffmann, V. S., Berthele, A., Muhlau, M., Kowarik, M. C., Krumbholz, M., Senel, M., Steuerwald, V., Naumann, M., Hartberger, J., Kerschensteiner, M., Oswald, E., Ruschil, C., Ziemann, U., Tumani, H., Vardakas, I., Albashiti, F., Kramer, F., Soto-Rey, I., Spengler, H., Mayer, G., Kestler, H. A., Kohlbacher, O., Hagedorn, M., Boeker, M., Kuhn, K., Buchka, S., Kohlmayer, F., Kirschke, J. S., Behrens, L., Zimmermann, H., Bender, B., Sollmann, N., Havla, J., Hemmer, B., & ProVal, M. S. s. g. (2024). Prospective study validating a multidimensional treatment decision score predicting the 24-month outcome in untreated patients with clinically isolated syndrome and early relapsing-remitting multiple sclerosis, the ProVal-MS study. Neurol Res Pract, 6(1), 15. 10.1186/s42466-024-00310-x.
8. Manouchehrinia A Zhu F Piani-Meier D Lange M Silva DG Carruthers R Glaser A Kingwell E Tremlett H Hillert J Predicting risk of secondary progression in multiple sclerosis: A nomogram Multiple Sclerosis (Houndmills, Basingstoke, England) 2019 25 8 1102 1112 10.1177/1352458518783667 29911467
Manouchehrinia, A., Zhu, F., Piani-Meier, D., Lange, M., Silva, D. G., Carruthers, R., Glaser, A., Kingwell, E., Tremlett, H., & Hillert, J. (2019). Predicting risk of secondary progression in multiple sclerosis: A nomogram. Multiple Sclerosis (Houndmills, Basingstoke, England), 25(8), 1102–1112. 10.1177/1352458518783667.29911467 10.1177/1352458518783667
9. Bergamaschi R Montomoli C Mallucci G Lugaresi A Izquierdo G Grand’Maison F Duquette P Shaygannejad V Alroughani R Grammond P Boz C Iuliano G Zwanikken C Petersen T Lechner-Scott J Hupperts R Butzkueven H Pucci E Oreja-Guevara C Cristiano E Amato P Havrdova MP Fernandez-Bolanos E Spelman R Trojano M BREMSO: A simple score to predict early the natural course of multiple sclerosis European Journal of Neurology 2015 22 6 981 989 10.1111/ene.12696 25808578
Bergamaschi, R., Montomoli, C., Mallucci, G., Lugaresi, A., Izquierdo, G., Grand’Maison, F., Duquette, P., Shaygannejad, V., Alroughani, R., Grammond, P., Boz, C., Iuliano, G., Zwanikken, C., Petersen, T., Lechner-Scott, J., Hupperts, R., Butzkueven, H., Pucci, E., Oreja-Guevara, C., Cristiano, E., Amato, P., Havrdova, M. P., Fernandez-Bolanos, E., Spelman, R., T., & Trojano, M. (2015). BREMSO: A simple score to predict early the natural course of multiple sclerosis. European Journal of Neurology, 22(6), 981–989. 10.1111/ene.12696.25808578 10.1111/ene.12696
