
==== Front
J Educ Health Promot
J Educ Health Promot
JEHP
J Edu Health Promot
Journal of Education and Health Promotion
2277-9531
2319-6440
Wolters Kluwer - Medknow India

JEHP-13-172
10.4103/jehp.jehp_1033_23
Brief Report
Unleashing the power of AI: Assessing the reliability of ChatGPT in disseminating breast cancer awareness
Mheidly Nour
Department of Communication, University of Illinois Chicago, Chicago, Illinois, United States
Address for correspondence: Nour Mheidly, Department of Communication, University of Illinois Chicago, Chicago, Illinois - 60607, United States. E-mail: nmheid2@uic.edu
2024
05 7 2024
13 17216 7 2023
09 9 2023
Copyright: © 2024 Journal of Education and Health Promotion
2024
https://creativecommons.org/licenses/by-nc-sa/4.0/ This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
ChatGPT is a large language model that can initiate conversations with humans and respond to their questions. Due to its access to vast amounts of text data, it has the potential to offer health information. This study will explore the ability of ChatGPT to disseminate health information on breast cancer to draw predictions on its acceptance and utilization as a portal for breast cancer awareness. Through the Technology Acceptance Model that focuses on two main aspects, the ease of use and the usefulness, a qualitative comparative analysis was conducted to assess breast cancer information retrieved from ChatGPT and the Centers for Disease Control and Prevention (CDC) website. Common queries that patients with breast cancer and the public often ask were used for assessment. A checklist of the essential elements covered by each question was created. Four themes (definition, prevention, diagnostics, and therapeutics) were used for the coding process and truth tables were created to compare answers. Results showed that the design of ChatGPT renders it an easy platform to initiate conversation and obtain clarifications and explanations for ideas and questions instantaneously. ChatGPT provides adequate and correct information on breast cancer compared to the CDC website, collects information from multiple authentic resources, and provides better access to information. Nevertheless, ChatGPT lacks accountability, and the nature of its responses changes over time. Overall, ChatGPT is a promising medium for the dissemination of health information on breast cancer and an important tool for raising awareness and improving public health knowledge on the disease.

Breast cancer
ChatGPT
health awareness
health communication
new technologies
==== Body
pmcIntroduction

The integration of modern technological advancements into various aspects of daily life is a hallmark of our era. However, despite the technological fluidity and the benefits it affords,[1] the rapid emergence of new technologies often accompanies sentiments of uncertainty, hesitation, and perhaps fear. This historical pattern of skepticism has repeated itself with the advent of technologies such as the telegraph, printer, telephone, television, and the Internet.[2] People initially grappled with the complexities of these technologies and questioned their potential benefits, but over time, they grew more adept at utilizing these tools and ultimately embraced them.

AI technologies wield a transformative impact on societies, shaping social practices, enhancing human agency, and fostering social cohesion.[3] They have become deeply ingrained in daily activities, interactions, and relationships, extending their reach across diverse fields and disciplines, and constituting indispensable tools for optimal performance. Notably, within the healthcare sector, AI advancements have streamlined administrative tasks, enriched medical performance, and yielded improved public health outcomes in areas such as electronic health record systems, diagnostics, and precision medicine.[4]

AI technologies, furthermore, significantly contribute to the domain of health communication. ChatGPT, in particular, holds the potential to enhance patient care and experience by offering precise information and facilitating effective communication among stakeholders.[5] Its distinctive interface, user-friendliness, and conversational capabilities, coupled with access to an extensive repository of textual data, position it as a promising platform for the dissemination of health information.

In this study, a qualitative analysis is conducted to investigate the capacity of ChatGPT in disseminating health information, specifically related to breast cancer. ChatGPT, a substantial language model developed by OpenAI, is designed to provide comprehensive responses to user queries with the main goal of benefiting humanity.[6] Launched on November 30, 2022, ChatGPT underwent training utilizing Reinforcement Learning from Human Feedback (RLHF) and employed a novel dialogue format that facilitates interactive conversations between the model and users. However, similar to the introduction of any groundbreaking technology, the rise of ChatGPT was met with a wave of skepticism and uncertainty regarding its intentions, objectives, and utility. Primarily, concerns revolve around the potential for AI technologies to compromise the integrity of language, knowledge, and ethics.[7]

Previous research studies have delved into the application of ChatGPT in streamlining various medical tasks, assessed its knowledge base against that of medical professionals, and evaluated its performance in generating medical content.[8,9,10,11] Nevertheless, no empirical investigations have thus far probed the potential of ChatGPT as a tool for disseminating public health awareness on diseases and illnesses. This current study is important in the context of healthcare settings, as it aims to apply the Technology Acceptance Model to assess the acceptability of ChatGPT. TAM, originally formulated by Fred Davis in the 1980s, was conceived in response to the unwillingness of individuals in adopting emerging technologies.[12] This conceptual framework argues that for a technology to gain widespread acceptance, it must satisfy two primary criteria: perceived usefulness and perceived ease of use.[13,14] Perceived usefulness refers to an individual’s assessment of whether the technology will enhance their performance, while perceived ease of use pertains to the perception that utilizing the technology demands a minimal effort to navigate.[15]

In this study, our central focus will revolve around two central angles: the perceived usefulness of ChatGPT within the realm of public health awareness and the perceived ease of use associated with its utilization. Furthermore, this study will draw predictions about the utilization of ChatGPT as an instrument for disseminating knowledge and raising awareness of breast cancer.

Materials and Methods

Study design

A qualitative comparative analysis was conducted to assess responses from the Centers for Disease Control and Prevention (CDC) website and ChatGPT regarding breast cancer-related queries. Data collection relied on the CDC website as the primary source of authentic breast cancer information.[16] On February 18, 2023, the CDC search engine was employed to retrieve the “Basic Information About Breast Cancer” webpage. The selected CDC questions mirrored common inquiries from patients with breast cancer and the public. Responses encompassed comprehensive information on symptoms, risk factors, preventive measures, diagnostics, and treatment options for breast cancer.[16] Simultaneously, on February 18, 2023, a new ChatGPT account was established, and the same nine breast cancer questions available on the CDC’s website were posed to ChatGPT for assessment.

Data analysis

To start the analysis process, the CDC’s nine breast cancer questions were categorized under four themes as follows:

Definitions:

What is breast cancer?

What are the symptoms of breast cancer?

Prevention:

What are the risk factors of breast cancer?

What can I do to reduce my risk of breast cancer?

What is breast cancer screening?

Diagnostics:

What is a mammogram?

What does it mean to have dense breasts?

How is breast cancer diagnosed?

Therapeutics:

How is breast cancer treated?

Next, a checklist outlining the vital elements addressed in each question was established. For instance, the responses to the question “What is breast cancer?” included details on ductal carcinoma, invasive ductal carcinoma, and invasive lobular carcinoma, all of which were categorized under the theme “types of breast cancer.” These themes served as the basis for the coding process, resulting in the establishment of four distinct truth tables, one for each theme, as a tool for conducting comparative analysis.[17]

Qualitative comparative analysis, employed herein, is a method for examining qualitative data sources and constructing truth tables to explain emerging themes.[18] In this study, these truth tables facilitated comparisons between the CDC and ChatGPT and helped in determining whether ChatGPT serves as an effective tool for raising awareness about breast cancer and enhancing patient understanding and knowledge of the disease.

Results

The CDC website and the ChatGPT interface were both able to present information and answer the nine breast cancer questions.

Definitions

In the first question, “What is breast cancer?”, ChatGPT provided a definition that aligned with the CDC’s answer by explaining the site of breast cancer origin. Nevertheless, noteworthy distinctions emerged in their responses. While both ChatGPT and the CDC stated the common types of breast cancer, ChatGPT refrained from defining these types or describing their differences in terms of formation and progression. Unlike the CDC, ChatGPT delved into the mechanism of breast cancer metastasis and offered insights into its occurrence rates among both males and females. Furthermore, ChatGPT raised awareness regarding the importance of self-examination and regular screenings for early detection and the prospects for successful treatment.

In the second question, “What are the symptoms of breast cancer?”, both ChatGPT and the CDC offered identical lists of breast cancer symptoms: a lump in the breast, breast swelling, skin irritation, redness in the breast or nipple area, nipple inversion, nipple discharge, and breast pain. They both emphasized that experiencing these symptoms does not necessarily signify a cancer diagnosis, and both advised patients to seek consultation with a healthcare provider when encountering such symptoms. However, ChatGPT omitted one of the recognized breast cancer symptoms—changes in the size or shape of the breast—and did not explain what “normal breast” and “breast lumps” refer to [Table 1].

Table 1 Comparative assessment of the answers to the basic information about breast cancer definitions between the centers for disease control and prevention (CDC) and ChatGPT

Basic breast cancer questions	Themes covered	CDC	ChatGPT	
What is breast cancer?	Medical definition of breast cancer	✓	✓	
Location of breast cancer	✓	✓	
Breast composition	✓	-	
Occurrence frequency in males and females	-	✓	
Where can breast cancer spread?	✓	-	
Types of breast cancer	✓	✓	
Breast cancer prognosis	-	✓	
Screening and self-examination awareness	-	✓	
What are the symptoms of breast cancer?	Major symptoms	
   New lump in the breast or underarm	✓	✓	
   Swelling of the breast	✓	✓	
   Skin irritation	✓	✓	
   Redness in the nipple area	✓	✓	
   Nipple discharge	✓	✓	
   Nipple inversion	✓	✓	
   Change in size or shape of the breast	✓	-	
   Pain in the breast	✓	✓	
Having symptoms does not mean you have cancer	✓	✓	
What is a normal breast?	✓	-	
What do lumps in the breast mean?	✓	-	
Check with healthcare provider for any changes	✓	✓	
Awareness of mammograms and self-exams	-	✓	

Prevention

In addressing the third question, “What are the risk factors for breast cancer?”, ChatGPT’s response closely aligned with that of the CDC. Both sources underscored age, gender, personal cancer history, breast density, family history, physical activity, hormone usage, and alcohol consumption as recognized risk factors for breast cancer. Nevertheless, ChatGPT’s response failed to include reproductive history, prior exposure to radiation therapy, contact with diethylstilbestrol, being overweight, smoking, chemical exposures, and hormonal fluctuations stemming from night shift work among the risk factors. Furthermore, ChatGPT’s response lacked information regarding family history and genetic predispositions in assessing breast cancer risk.

In answering the fourth question, “What can I do to reduce my risk of breast cancer?”, both sources presented a similar set of strategies for mitigating breast cancer risk. However, ChatGPT introduced additional risk reduction measures, with advice to limit alcohol consumption, reduce smoking, undergo regular screenings, and consider certain medications such as raloxifene and tamoxifen. ChatGPT also noted that while these measures can reduce the risk, they do not guarantee the prevention of cancer, emphasizing the importance of early detection. Nonetheless, ChatGPT’s response did not emphasize on or mention consulting medical professionals in cases involving the use of hormones or contraception as a risk reduction strategy for breast cancer.

In addressing the fifth question, “What is breast cancer screening?”, the responses from ChatGPT and the CDC were highly compatible. Both responses began with a concise definition of breast cancer and subsequently enumerated the recognized breast cancer screening methods, which included mammograms, magnetic resonance imaging (MRI), breast self-awareness, and clinical breast exams. The CDC, however, introduced an additional subtitle “breast cancer recommendations,” which is generated by the United States Preventive Services Task Force (USPSTF). Furthermore, the CDC provided hyperlinks for further data about this organization [Table 2].

Table 2 Comparative assessment of the answers to the basic information about breast cancer prevention between the centers for disease control and prevention (CDC) and ChatGPT

Basic breast cancer questions	Elements and questions covered	CDC	ChatGPT	
What are the risk factors for breast cancer?	Risk factors	
   Age	✓	✓	
   Gender	✓	✓	
   Reproductive history	✓	-	
   Personal cancer history	✓	✓	
   Having dense breast	✓	✓	
   Family history of ovarian cancer	✓	✓	
   Previous exposure to radiation therapy	✓	-	
   Exposure to the drug diethylstilbestrol (DES)	✓	-	
   Being overweight	✓	-	
   Being physically active	✓	✓	
   Taking hormones	✓	✓	
   Drinking alcohol	✓	✓	
   Smoking	✓	-	
   Exposure to chemicals	✓	-	
   Changes in hormones due to night shift working	✓	-	
Who is at high risk of breast cancer?	✓	-	
What can I do to reduce my risk of breast cancer?	Major steps	
   Maintain a healthy weight	✓	✓	
   Be physically active	✓	✓	
   Limit alcohol consumption	-	✓	
   Do not smoke	-	✓	
   Talk to Dr. if you are taking hormone replacement therapy or oral contraceptives	✓	-	
   Breastfeed your children, if possible	✓	✓	
   Talk to healthcare provider about family history	✓	✓	
   Regular screenings	-	✓	
   Consider certain medications	-	✓	
Reducing risk does not mean you will not have cancer	-	✓	
Raising awareness of breast cancer for early detection	-	✓	
What is breast cancer screening?	Definition of breast cancer screening	✓	✓	
Breast cancer screening recommendations	✓	-	
Breast cancer tests	
   Mammogram	✓	✓	
   Magnetic resonance imaging (MRI)	✓	✓	
   Clinical breast exam	✓	✓	
   Breast self-awareness	✓	✓	
Benefits of screening	✓	✓	
Risks of screening	✓	✓	

Diagnostics

In addressing the sixth question, “What is a mammogram?”, ChatGPT’s definition of a mammogram aligned closely with the CDC’s definition. However, ChatGPT offered additional insights regarding the individuals who should undergo mammography and the recommended age groups for such screenings. Furthermore, ChatGPT highlighted the well-tolerated and generally safe nature of mammograms and emphasized the significance of consulting a healthcare practitioner for discussions on mammography, questions, and personalized breast cancer risk assessment based on individual risk factors. However, ChatGPT’s response lacked information on the experience of undergoing a mammogram. It also did not offer guidance for women considering mammograms, specify the expected timeline for receiving mammogram results, detail the course of action following normal or abnormal mammogram results, or provide information on where to obtain mammography services.

In response to the seventh question, “What does it mean to have dense breasts?”, ChatGPT’s definition of breast density aligned with the CDC’s definition. ChatGPT explained the significance of breast density, which was consistent with the CDC’s information. ChatGPT also highlighted the importance of seeking guidance from a healthcare provider and discussing preventive screening options based on an individual’s risk factors. However, ChatGPT’s response did not provide information regarding the anatomical components of the breast, explain the four categories of breast density, expand on why breast density holds clinical importance, specify which individuals are more likely to have dense breasts, or outline recommendations for individuals with dense breasts.

In response to the eighth question, “How is breast cancer diagnosed?”, both ChatGPT and the CDC compiled diagnostic procedures employed for cancer detection. ChatGPT added the importance of consulting a pathologist when dealing with advanced stages of cancer and acknowledged the emotional and hard nature of a breast cancer diagnosis, emphasizing the need for consultation with healthcare providers to ease this process. However, ChatGPT did not provide details regarding cancer staging [Table 3].

Table 3 Comparative assessment of the answers to the basic information about breast cancer diagnostics between the centers for disease control and prevention (CDC) and ChatGPT

Basic breast cancer questions	Elements and questions covered	CDC	ChatGPT	
What is a mammogram?	Definition of mammogram	✓	✓	
How is a mammogram done?	✓	✓	
Who performs mammograms usually?	-	✓	
Are mammograms safe?	-	✓	
What does having a mammogram feel like?	✓	-	
Tips for getting a mammogram	✓	-	
What happens if a mammogram is normal?	✓	-	
What happens if a mammogram is abnormal?	✓	-	
Where to get a mammogram?	✓	-	
Who to ask if you are having questions about mammograms?	✓	✓	
What does it mean to have dense breasts?	Definition of dense breast	✓	✓	
What are the parts of the breast?	✓	-	
Importance of breast density	✓	✓	
Who is more likely to have dense breasts?	✓	-	
What to do if you have dense breasts?	✓	-	
How is breast cancer diagnosed?	Diagnostic exams	
   Breast ultrasound	✓	✓	
   Mammogram	✓	✓	
   Breast MRI	✓	✓	
   Biopsies	✓	✓	
   Pathology	-	✓	
Cancer staging	✓	✓	
Raising awareness of the importance of working closely with a health provider	-	✓	
Hyperlink on staging	✓	-	

Therapeutics

In the final question, “How is breast cancer treated?”, ChatGPT and the CDC presented similar responses with slight differences. Both responses acknowledged the variability in breast cancer treatment based on factors such as cancer stage. Also, they both proposed surgery, chemotherapy, hormonal therapy, and radiation as viable treatment modalities. However, ChatGPT introduced targeted therapies as an additional treatment option, whereas the CDC suggested biological therapy. ChatGPT provided more comprehensive explanations compared to the CDC’s responses. For example, in explaining hormonal therapy, the CDC mentioned that it “blocks cancer cells from receiving the hormones they need to grow,” whereas ChatGPT provided a more detailed explanation, specifying that hormone therapy targets hormone receptor-positive breast cancer, which is stimulated by hormones like estrogen and progesterone. ChatGPT also elucidated that hormone therapy functions by either blocking the hormone’s effects or reducing estrogen levels in the body. Both ChatGPT and the CDC addressed complementary and alternative medicine in their responses. However, ChatGPT added “psychological support” as a vital intervention to aid patients with breast cancer. In contrast, the CDC’s response introduced an additional element regarding the roles of radiologists, oncologists, and surgeons in breast cancer patient care. The CDC further provided hyperlinks directing readers to clinical trials supported by the National Institutes of Health (NIH) and highlighted the importance of seeking a second opinion for breast cancer diagnosis and treatment [Table 4].

Table 4 Comparative assessment of the answers to the basic information about breast cancer therapeutics between the centers for disease control and prevention (CDC) and ChatGPT

Basic breast cancer questions	Elements and questions covered	CDC	ChatGPT	
How is breast cancer treated?	Treatments	
   Surgery	✓	✓	
   Chemotherapy	✓	✓	
   Hormonal therapy	✓	✓	
   Biological therapy	✓	-	
   Radiation	✓	✓	
   Targeted therapy	-	✓	
   Complementary therapy	✓	✓	
Counseling/support	-	✓	
Physician interventions	✓	-	
Clinical trails	✓	-	
Hyperlinks for clinical trails	✓	-	
Consulting a suitable physician for the available treatment depending on the cancer stage	✓	✓	
Getting a second opinion	✓	-	

Discussion

ChatGPT responses were compatible with the breast cancer information present on the CDC website. ChatGPT supplemented the information available on the CDC website by offering additional details regarding the prevalence of breast cancer in both males and females. Furthermore, it provided enhanced insights into risk reduction strategies, emphasizing the significance of limiting alcohol consumption, quitting smoking, and underscoring the importance of diagnostic examinations. In addition, ChatGPT raised awareness on the importance of self-examinations, regular screenings, and consultation with medical professionals in the context of breast cancer prevention and early detection. However, ChatGPT’s responses lacked comprehensive information about the definition and symptoms of breast cancer. It also missed certain risk factors and did not elaborate on breast cancer screening recommendations.

Regarding ease of use, the ChatGPT platform has a user-friendly design that is easy to navigate. All information is conveniently accessible within the same chat box or webpage, enabling patients to follow without any complex instructions. Moreover, the unique OpenAI feature of this interface allows patients to engage in interactive conversations with ChatGPT to obtain clarifications, explanations, and pose additional questions. In contrast, conventional websites are constrained by the limited amount of data they can present at any given time, which restricts patients’ access to information and their ability to engage in real-time communication or seek instant clarification.

Regarding its usefulness, ChatGPT exhibits the capability to amass information from a diverse array of sources. For instance, cancer information for this study were sourced from authoritative health institutions such as the American Cancer Society, the National Breast Cancer Foundation, and the Mayo Clinic, in addition to the CDC. This access to a wide spectrum of information awards ChatGPT with a distinct advantage over the CDC website and enhances the efficiency of information retrieval for users. Yet, reliance on websites for information access potentially diminishes ChatGPT’s accountability. In the event that the CDC website provides incorrect information, it bears the responsibility for rectification. Conversely, if ChatGPT provides wrongful information, the absence of a distinct entity to hold accountable raises pertinent ethical considerations. Furthermore, ChatGPT is a continually evolving AI technology, subject to continuous updates, which impacts the nature of its responses over time.

The inclusion of visual aids significantly enhances the utility and comprehensibility of a platform. ChatGPT currently lacks any visual elements and is restricted solely to text-based responses. It does not offer images, infographics, or videos to assist readers in visualizing the information being presented. In contrast, the CDC website integrates numerous visuals for nearly every topic. For instance, when addressing breast composition, they provide a detailed diagram illustrating the precise anatomical locations of various breast components, including the nipple, lobe, lobules, areola, and fat. Furthermore, they augment these visuals with hyperlinks that offer additional explanations. In the context of diagnostic procedures, the CDC supplements its content with actual images of patients undergoing MRI, thereby providing patients the opportunity to develop a clearer understanding and manage expectations regarding the subsequent steps in their diagnostic journey.

Conclusion

In summary, the ChatGPT interface exhibits promise as a medium for disseminating health information about breast cancer. It presents accurate and sufficient information on breast cancer, often comparable with the CDC website. Nevertheless, this study does not consider ChatGPT as a substitute for qualified medical professionals in the diagnosis or treatment of breast cancer.

ChatGPT’s role lies in its potential as a valuable communication tool for enhancing public understanding of health issues. It provides adequate health information, facilitates interactive dialogue with patients, explains complex medical concepts, simplifies medical information for better understanding, and delivers prompt responses to inquiries. However, ChatGPT may overlook important information, exhibit inconsistency in its answers, and lack visuals, content monitoring, and regulation [Figure 1]. Further research can explore its efficacy as a tool to bolster awareness of other diseases.

Figure 1 Opportunities vs. challenges of ChatGPT in medicine and health

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.
==== Refs
1 Bauman Z Liquid Modernity 2013 Polity Press
2 Standage T The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth Centuryʼs On-Line Pioneers 1999 Berkley Books
3 Floridi L Cowls J Beltrametti M Chatila R Chazerand P Dignum V AI4People-An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations Minds Mach (Dordr) 2018 28 689 707 30930541
4 Schwalbe N Wahl B Artificial intelligence and the future of global health Lancet 2020 395 1579 86 32416782
5 Santandreu-Calonge D Medina-Aguerrebere P Hultberg P Shah MA Can ChatGPT improve communication in hospitals? Profesional de la información 2023 32 e320219
6 OpenAI Introducing ChatGPT 2023 Available from: https://openai.com/blog/chatgpt [Last accessed on 2023 May 19]
7 Chomsky N Chomsky N Roberts I Watumull J Noam Chomsky: The false promise of ChatGPT, in The New York Times 2023
8 Patel SB Lam K ChatGPT: The future of discharge summaries? Lancet Digit Health 2023 5 e107 08. 36754724
9 Biswas S ChatGPT and the future of medical writing Radiology 2023 307 e223312 36728748
10 Rao A Kim J Kamineni M Pang M Lie W Succi MD Evaluating ChatGPT as an adjunct for radiologic decision-making medRxiv 2023 2023 02.02.23285399
11 Huh S Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination? A descriptive study J Educ Eval Health Prof 2023 20 1 36627845
12 Davis FD Perceived usefulness, perceived ease of use, and user acceptance of information technology MIS Quarterly 1989 13 319 40
13 Davis FD A technology acceptance model for empirically testing new end-user information systems: Theory and results 1985 Massachusetts Institute of Technology, Sloan School of Management
14 Lee Y Kozar KA Larsen KR The Technology acceptance model: Past, present, and future Communications of the Association for Information Systems 2003 12 50
15 Davis FD Bagozzi RP Warshaw PR User acceptance of computer technology: A comparison of two theoretical models Management Science 1989 35 982 1003
16 CDC Breast Cancer Basic Information 2022 Available from: https://www.cdc.gov/cancer/breast/basic_info/. [Last accessed on 2023 May 19]
17 Ragin CC The comparative method: Moving beyond qualitative and quantitative strategies 2014 University of California Press
18 Onwuegbuzie AJ Weinbaum RK A framework for using qualitative comparative analysis for the review of the literature The Qualitative Report 2017 22 359 72
