
==== Front
Croat Med J
Croat Med J
CMJ
Croatian Medical Journal
0353-9504
1332-8166
Croatian Medical Schools

CroatMedJ_65_0399
10.3325/cmj.2024.65.399
In Reply
Is perfect prompting possible for chatbots?
In Reply to: Comparing ChatGPT’s ability between writing versus reviewing papers – then what?
Kadi Gültekin
Aslaner Mehmet Ali
Department of Emergency Medicine, Gazi University Faculty of Medicine, Ankara, Turkey
gultekinkadi@gazi.edu.tr

8 2024
65 4 399399
Copyright © 2024 by the Croatian Medical Journal. All rights reserved.
2024
https://creativecommons.org/licenses/by/2.5/ This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
==== Body
pmcWe are grateful to Dr Matsubara for his comments and have carefully considered the listed remarks. Although the author's concerns are justified, we would like to clarify some points.

First, our study question was whether AI can independently create a case report with an original title and other inputs used as prompts. Dr Matsubara is right about the importance of using specific input when evaluating ChatGPT's capabilities, and about entering a detailed prompt for a more specific output. In future experiments, different inputs may be tested. As can be seen from case studies in the literature, there has yet to be a perfect or standard prompt, and it remains unclear which prompt gives better output (1).

Second, the case reports generated by ChatGPT in our pilot studies contained made-up references. This happened even though we gave specific instructions to “Include in-text citations where appropriate for at least three references from real published original articles.” In other studies as well, AI created non-existent and irrelevant references (2). On the other hand, when in our study ChatGPT peer-reviewed a case report without a reference section, it rated the reference section as good. The report only scored poor in terms of writing quality. AI was not able to establish a relationship between manuscript sections (3).

Third, prompt engineering, a new concept, is developing faster every day (4). It is possible to develop personalized prompts for different research purposes and jobs. Just as the same output can be obtained with different prompts, different outputs can also be achieved with the same prompts. We want to highlight that AI is both a threat and an opportunity for academic publishing. We ethically and personally believe that AI can be used just to make writing easier, not to produce content.

Finally, we generated two outputs based on two different inputs to produce a case report: one is Kadi and Aslaner's prompt and the other is Matsubara's prompt (Supplemental Material 1(Supplementary Material)). Readers, decide!

Additional Material

Supplementary Material
==== Refs
References

1 Kıyak YS Emekli E ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review. Postgrad Med J 2024 Published online 06 06, 2024 10.1093/postmj/qgae065 38840505
2 Giray L ChatGPT references unveiled: Distinguishing the reliable from the fake. Internet Ref Serv Q 2024 28 9 18 10.1080/10875301.2023.2265369
3 Kadi G Aslaner MA Exploring ChatGPT’s abilities in medical article writing and peer review. Croat Med J 2024 65 93 10.3325/cmj.2024.65.93 38706235
4 Giray L Prompt engineering with ChatGPT: a guide for academic writers. Ann Biomed Eng 2023 51 2629 33 10.1007/s10439-023-03272-4 37284994
