
==== Front
Res Sq
ResearchSquare
Research Square
2693-5015
American Journal Experts

10.21203/rs.3.rs-4868925/v1
10.21203/rs.3.rs-4868925
preprint
1
Article
Large language model non-compliance with FDA guidance for clinical decision support devices
Weissman Gary https://orcid.org/0000-0001-9588-3819

Mankowitz Toni
Kanter Genevieve https://orcid.org/0000-0002-3044-7829

09 9 2024
rs.3.rs-4868925https://creativecommons.org/licenses/by/4.0/ This work is licensed under a Creative Commons Attribution 4.0 International License, which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use.
https://www.researchsquare.com/article/rs-4868925/v1
nihpp-rs4868925v1.pdf
Abstract

Large language models (LLMs) show considerable promise for clinical decision support (CDS) but none is currently authorized by the Food and Drug Administration (FDA) as a CDS device. We evaluated whether two popular LLMs could be induced to provide unauthorized, device­like CDS, in violation of FDA’s requirements. We found that LLM output readily produced device­like decision support across a range of scenarios despite instructions to remain compliant with FDA guidelines.
==== Body
pmc
