
==== Front
JMIR Form Res
JMIR Form Res
JFR
JMIR Formative Research
2561-326X
JMIR Publications Toronto, Canada

v8i1e59963
10.2196/59963
Original Paper
Original Paper
Development of an mHealth App by Experts for Queer Individuals’ Sexual-Reproductive Health Care Services and Needs: Nominal Group Technique Study
Mavragani Amaryllis
Rampilo Mashudu
Rasweswe Melitah
Seretlo Raikane James D4 (Nursing), Bcur I et A, MPH https://orcid.org/0000-0001-6459-9065
1Department of Public Health, School of Health Care Sciences Sefako Makgatho Health Sciences University Molotlegi St, Ga-Rankuwa Zone 1 Ga-Rankuwa Tshwane, 0208 South Africa 27 012 521 4111 Raikane.seretlo@smu.ac.za

Smuts Hanlie BSc, BSc (Hons), MSc, PhD 2https://orcid.org/0000-0001-7120-7787

Mokgatle Mathildah Mpata B Rad Diagnostics, B Rad Hons (Diagnostics), MPH, PhD 1https://orcid.org/0000-0001-5807-8212

1 Department of Public Health, School of Health Care Sciences Sefako Makgatho Health Sciences University Tshwane South Africa
2 Department of Informatics, Faculty of Engineering, Built Environment and Information Technology University of Pretoria Tshwane South Africa
Corresponding Author: Raikane James Seretlo Raikane.seretlo@smu.ac.za
2024
21 8 2024
8 e5996327 4 2024
26 5 2024
3 6 2024
17 6 2024
©Raikane James Seretlo, Hanlie Smuts, Mathildah Mpata Mokgatle. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.08.2024.
2024
https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

Background

Queer individuals continue to be marginalized in South Africa; they experience various health care challenges (eg, stigma, discrimination, prejudice, harassment, and humiliation), mental health issues (eg, suicide and depression), and an increased spread of HIV or AIDS and sexually transmitted illnesses (STIs; chlamydia, gonorrhea, and syphilis). Mobile health (mHealth) apps have the potential to resolve the health care deficits experienced by health care providers when managing queer individuals and by queer individuals when accessing sexual-reproductive health care services and needs, thus ensuring inclusivity and the promotion of health and well-being. Studies have proven that the nominal group technique (NGT) could be used to solve different social and health problems and develop innovative solutions. This technique ensures that different voices are represented during decision-making processes and leads to robust results.

Objective

This study aims to identify important contents to include in the development of an mHealth app for addressing the sexual-reproductive health care services and needs of queer individuals.

Methods

We invited a group of 13 experts from different fields, such as researchers, queer activists, sexual and reproductive health experts, private practicing health care providers, innovators, and private health care stakeholders, to take part in a face-to-face NGT. The NGT was conducted in the form of a workshop with 1 moderator, 2 research assistants, and 1 principal investigator. The workshop lasted approximately 2 hours 46 minutes and 55 seconds. We followed and applied 5 NGT steps in the workshop for experts to reach consensus. The main question that experts were expected to answer was as follows: Which content should be included in the mHealth app for addressing sexual-reproductive health care services and needs for queer individuals? This question was guided by user demographics and background, health education and information, privacy and security, accessibility and inclusivity, functionality and menu options, personalization and user engagement, service integration and partnerships, feedback and improvement, cultural sensitivity and ethical considerations, legal and regulatory compliance, and connectivity and data use.

Results

Overall, experts voted and ranked the following main icons: menu options (66 points), privacy and security (39 points), user engagement (27 points), information hub (26 points), user demographics (20 points), connectivity (16 points), service integration and partnerships (10 points), functionalities (10 points), and accessibility and inclusivity (7 points).

Conclusions

Conducting an NGT with experts from different fields, possessing vast skill sets, knowledge, and expertise, enabled us to obtain targeted data on the development of an mHealth app to address sexual-reproductive health care services and needs for queer individuals. This approach emphasized the usefulness of a multidisciplinary perspective to inform the development of our mHealth app and demonstrated the future need for continuity in using this approach for other digital health care innovations and interventions.

experts
mobile health app
mHealth app
digital health care
telemedicine
digital innovation
sexual-reproductive health care services and needs
lesbian, gay, bisexual, transgender, queer, intersex, asexual individuals, and related identities
LBQTQIA+
queer individuals
nominal group technique
mobile phone
==== Body
pmcIntroduction

Background

The literature shows that most countries are focusing on curbing the spread of sexually transmitted illnesses (STIs) [1-3] by promoting sexual and reproductive health care (SRH) among a vast number of patients. Various public health agencies and organizations, such as the World Health Organization (WHO) and the Joint United Nations Programme on HIV/AIDS, and others are supporting the prioritization of queer individuals’ health and well-being. For example, the WHO formulates guidelines, offers technical assistance, and undertakes research to assist countries in developing and strengthening inclusive health policies and structures that promote the health and well-being of all people, irrespective of sexual orientation, gender identity, expression, and sex characteristics [4]. In addition, Frescura et al [5] stated that United Nations Programme on HIV/AIDS has set additional ambitious targets for 2025, including 95% of all people who are HIV positive knowing their status, 95% of all individuals with HIV receiving prolonged antiretroviral medication, and 95% of every recipient of antiretroviral therapy having viral suppression. Furthermore, enhancing the health and well-being of all people is a central focus of the 2030 Agenda for Sustainable Development. This is evident in targets aimed at eradicating diseases such as AIDS [6]. As a result, countries are expected to develop policies to curb STIs.

In Africa, SRH is also primarily prioritized as part of universal health coverage; however, policies and legal hurdles, limiting gender norms, and gender-based inequities all pose challenges to delivering and accessing quality SRH services in low- or middle-income nations [7]. This situation exists regardless of objective 2 of the Africa Health Strategy 2016-2030, which calls for a reduction in morbidity and an end to preventable mortality from communicable diseases such as AIDS, ensuring equitable access to comprehensive and integrated SRH [8]. The South African National Lesbian, Gay, Bisexual, Transgender, and Intersex (LGBTI) HIV Plan, 2017 to 2022, aimed to provide HIV, STI, and tuberculosis services to LGBTI people through 5 interconnected packages: health, empowerment, human rights, psychosocial support, and assessment [9].

However, in South Africa, SRH services and needs (SRHSN) for queer individuals are either generalized or inadequately addressed. This is evidenced by the continuation of high rates of the spread of STIs among lesbian, gay, bisexual, transgender, questioning, intersex, asexual, and related identities (LGBTQIA+) individuals in the country [10-12]. In addition, several South African studies have shown that queer individuals continue to experience various STIs such as asymptomatic Neisseria gonorrhoeae, Chlamydia trachomatis [10,12], syphilis [11], and bacterial STI burden. Furthermore, Malefo et al [13] showed that the prevalence of asymptomatic STIs among men who have sex with men was high in South Africa, with discrete bacterial infections detected. According to the study by Khozah and Nunu [14], factors such as clinical settings, punishing laws, and the accessibility of services tailored to sexual and gender minority groups contribute to a generally low average use of SRH.

Several studies found a lack of skills, medical education, and knowledge among health care providers (HCPs) regarding health matters related to queer individuals [15-18]. However, in a South African study focused on motivational interviewing for reducing risky sexual behaviors, most of the participants stated that the sessions had a positive impact on the change and reduction of risky sexual behaviors, specifically by promoting the use of condoms and minimizing the number of multiple sexual partners [19]. Furthermore, skills and knowledge are not the only challenges in the health care sector; HCPs also continue to illtreat, stigmatize, prejudice, discriminate, harass, and judge queer individuals seeking health care services in South Africa, sub-Saharan African regions, and countries globally [20-26].

Unless these challenges are addressed, queer individuals’ health and well-being will continue to be at risk, resulting in the high prevalence and incidence of STIs [27]. South African queer individuals will continue to face extensive challenges, such as societal stigma; homophobic violence, specifically corrective rape; and high rates of STIs, particularly HIV or AIDS [28]. These social challenges hinder queer individuals from accessing and using SRHSN freely.

The State of Technology Use

In a ubiquitous computerized society, technology is considered a basic and crucial component of our lives [29]. Furthermore, the use of mobile health (mHealth) apps by HCPs globally has increased remarkably [30]. For patients and HCPs alike, mHealth could enhance access to information and also promote communication on healthy living and the management of diseases [31]. Mickan et al [32] highlighted that HCPs benefit from a variety of mobile devices and apps, with the most notable benefits being greater access to point-of-care tools, which have been shown to support better clinical decision-making and outcomes. Therefore, mHealth apps can centralize all of a patient’s medication-specific data, providing a much more logical process for teaching patients how to manage their illnesses and provide self-care [33]. The WHO supports the use of digital health interventions and introduced a guideline that emphasizes that health systems need to harness the heightened visibility and availability of information through the use of mobile phones, tablets, and computers to enhance people’s health and vital services [34]. In addition, the guideline stresses the importance of reaching out to populations considered vulnerable and guaranteeing that digital health care does not expose them to danger in any way [34].

The South African National Department of Health developed a web-based medium connecting the public health workforce to relevant, high-quality continuing professional development opportunities and resources [35]. This web-based platform provides outstanding health care to patients, addresses health care professionals’ need to enhance their abilities and competencies, and executes strategies to shape effective managers and health practitioners [35]. However, this web-based platform does not include details about how HCPs should address queer individuals SRHSN.

These discrete policies and innovative interventions do not stipulate what types of SRHSN HCPs should provide and render to queer individuals; therefore, this research study aimed to identify essential content for inclusion in an mHealth app to address the SRHSN of queer individuals. All the challenges queer individuals experience could be addressed by using innovative technologies such as mHealth apps.

Methods

Overview

To determine what content needs to be included in an mHealth app for addressing the SRHSN of queer individuals, we facilitated a face-to-face nominal group technique (NGT) expert workshop at a nongovernmental organization clinic in the Gauteng Province of South Africa. The NGT expert workshop was conducted in February 2024. This NGT expert workshop incorporated 2 design science research steps (suggestion and development) [36]. During the workshop, experts were guided through 4 main NGT stages, as discussed by Mullen et al [37] and designed by Delbecq et al [38].

Study Population and Recruitment

We sampled 13 experts who voluntarily agreed to participate in our workshop. The workshop criteria of the experts included persons with experience, skills, knowledge, and expertise in SRH, LGBTQI+, and digital innovation. The experts were identified by searching on social media platforms, such as Instagram, LinkedIn, university websites, and ResearchGate, using the search string “SRH, SRH researchers, digital innovation, queer, LGBTQIA+.” Recruitment of the experts occurred over approximately 3 months from December 2023 to February 2024 through email invitations to an initial sample of 22 potential experts; we followed up with calls and emails with those who did not respond. In total, 13 experts agreed to participate; each expert was sent an email with the workshop information and a Google Doc to provide their availability. On the date of the workshop, each participant was given a consent form before data collection commenced (refer to Table 1 for participant demographics). The study used convenience sampling; thus, participants who met the study criteria were easily accessible, resided within the Gauteng province, were available on the date set for the NGT workshop, and were voluntarily willing to participate in the NGT expert workshop [39].

Table 1 Experts’ demographic data.

Expert number	Age (y)	Sex	Preferred pronouns	Racea	Marital status	Focus area	
E1	31	Male	He or him	African	Single	Sexuality, LGBTQIA+b, masculinity, and gender	
E2	28	Female	She or her	African	Single	Innovative marketing	
E3	55	Female	She or her	African	Single	Sexually transmitted illnesses and key populations	
E4	39	Male	He or him	African	Single	HIV, tuberculosis, and key populations	
E5	35	Male	He or him	African	Married	Health communication and LGBTQIA+	
E6	29	Male	He or him	African	Single	Key population research	
E7	29	Female	She or her	African	Married	Primary health care	
E8	24	Male	He or him	African	Single	Data analytics	
E9	34	Female	She or her	African	Single	Psychosocial sciences	
E10	27	Female	She or Her	Coloured	Single	Communications	
E11	55	Male	He or him	White	Single	Sexual-reproductive health and LGBTQIA+	
E12	48	Male	He or him	African	Married	Obstetrics and gynecology and researcher	
E13	57	Female	She or her	African	Single	Sexual-reproductive health	
aRace was self-reported by the participants.

bLGBTQIA+: lesbian, gay, bisexual, transgender, queer, intersex, asexual, and related identities.

Table 1 presents the demographics of the experts who participated in the NGT workshop. Among the 13 experts, the majority were male (n=7, 54%) and the minority were female (n=6, 46%); in terms of race, 11 (85%) were African, 1 (8%) was White, and 1 (8%) was of the Coloured racial group. Of the 13 participants, 10 (77%) were single and 3 (23%) were married.

Instruments for the NGT

The principal investigator conferred with the prospective mobile app software developer before the NGT experts’ workshop and shared the research question to be posed during the workshop, namely, Which content should be included in the mHealth app to address sexual-reproductive health care services and needs for queer individuals? The following pillars helped answer the main research questions during the workshop: user demographics and background, health education and information, privacy and security, accessibility and inclusivity, functionality and menu options, personalization and user engagement, service integration and partnerships, feedback and improvement, cultural sensitivity and ethical considerations, legal and regulatory compliance, and connectivity and data use.

Our NGT expert workshop had 1 moderator who conducted the entire workshop following the study by Yahaya et al [40], thereby ensuring bias was avoided by the principal investigator, and assisted with the completion of the consent forms and the smooth running of the workshop. In addition, 2 research assistants participated to ensure that all the experts’ discussions and views were noted on the flipchart boards using markers. Finally, the principal investigator explained the purpose of the workshop, shared information on the pillars, handed out small cards containing written NGT steps whenever experts were moving to each step, and recorded the workshop session with a digital recorder.

Implementation of the NGT

We used a workshop program to facilitate the NGT expert workshop, thus achieving the purpose of the workshop effectively (program provided in Textbox 1). The workshop lasted approximately 2 hours 46 minutes and 55 seconds.

Nominal group technique program.

Time and activity

09:00 AM: Welcome

09:10 AM: Introductions

09:30 AM: Purpose and presentation of the findings

10:00 AM: Moderator preparation and orientation of experts

10:10 AM: Silent generation

10:30 AM: Tea break

11:00 AM: Round robin

11:30 AM: Clarification

12:30 PM: Voting and ranking

13:10 PM: A vote of thanks and conclusion

In this section, we present the program for the NGT expert workshop, including its duration and the activities conducted during the workshop. The workshop commenced at 09:00 AM with introductions and purpose and presentation of the findings; the first NGT step commenced at 10:10 AM, and the last step was completed at 12:30 PM, followed by a vote of thanks and closure at 13:10 PM.

The workshop moderator facilitated the workshop by first ensuring the experts understood the details in the consent form, guided them through the demographic data section, and answered all the questions raised by the experts up until the signing of the consent form. The attendees signed informed consent forms before the workshop commenced and after all questions from the experts regarding details about the consent form had been clarified. Thereafter, the principal investigator shared small cards describing the first step with all the experts, and the moderator explained what was written on the card, clarifying the activities expected of the experts. The first step was silent generation, whereby the experts were given sticky notes and pens to write down their ideas for answering the workshop or study question and aligning their answers to the guiding pillars. This step lasted 20 minutes, and the experts were not allowed to talk to each other until the time had elapsed. The experts were provided with differently colored sticky notes to group their ideas in silence. The second step, called the round robin, followed, during which the moderator explained what was expected of the experts. The information in the cards clearly indicated that the experts were allowed to share their previously written down ideas with the entire group, while other experts did not interject and critique during a presentation. Each expert had 10 minutes to present their ideas, and other experts were not allowed to add new ideas that might be influenced by the presenting expert. Experts were asked to answer the question, Which content should be included in the mHealth app to address sexual-reproductive health care services and needs for queer individuals? The question was supposed to be answered by all the experts, aligning their answers to the shared pillars. After each expert had presented, they handed their sticky notes to the research assistants and principal investigator, and all ideas were captured on a flipchart board using a black marker. A similar process ensued until all 13 experts had completed their presentations. The third NGT step was clarification, which allowed experts to discuss each other’s ideas and identify any similarities, meaning the experts were allowed to question each other and ask for clarification. The discussion session took approximately 1 hour. Finally, the voting and ranking step followed, where all experts were allowed to start looking at all the raw data of the ideas written on the flipchart boards and posted on the wall. The moderator continued to facilitate the session, and 1 research assistant, together with the principal investigator, used a red marker to start marking the grouping of ideas shared and suggested by all the experts. Discussions and arguments about which subthemes to be prioritize continued until the experts reached a consensus (findings presented in Table 2). This step lasted approximately 1.5 hours until a consensus was reached. The moderator continued maintaining order and ensured that all the experts were protected and given a fair chance to argue and vote. The experts were allowed to vote through a show of hands as each subheading was presented, and the moderator counted the votes. The experts voted for whichever contents they believed essential for inclusion in the mHealth app.

Table 2 Detailed summary of the final findings including pillars, headings, and subheading with their breakdown ranking and scoring.

Pillar	Headings and subheadings	Ranking and scoring	
User demographics	Recording of personal data for profile creation

Pronouns such as him, her, they, their, or them

Sexual identity such as male, female, binary, or nonbinary

Gender orientation such as gay, bisexual, transgender, and so on

Age

Ability to find locations and health care facilities within different areas

Specialized health care facilities

Nongovernmental facilities such as POP-INN

	11

09

	
Features	Contact support information

Web-based consultation

Artificial intelligence assistant or chat bot

Suggestion box

Call back and email request

Videos

Clips

Graphics

Share button

Share button for success stories

	11

11

10

09

09

08

08

	
Accessibility and inclusivity	Ensuring inclusivity and easy access by different people

Specifically, people with disabilities

	07

	
Connectivity	Ensuring free app access

Offline app access through sponsorships

Ensuring accommodative of people without smart phones

Use of USSIDa

	10

06

	
Privacy and security	Protection of the user’s data

Record keeping

Provide users online login details with passwords

User’s backup data

Agreements between users

Avail end user agreement

Ensure safeguard of the users

Ensure protection of the user’s login details

Change of passwords regularly

Passwords change reminders for the users

	10

10

09

08

02

	
Functionalities	Push notifications

Such as SMS reminders

	10

	
Service integration and partnerships	Health services

Health care providers

Laboratories

Medical aids

Private and public health care facilities (which will help with appointments and referrals)

	10

	
User engagement	Ensure referral to other health care providers

Option for referral to other health care providers

Frequently asked questions

Option for frequently asked questions and known facts

Support groups

Create option for support and peer groups

	10

09

08

	
Information hub	Health education

HIV (PrEPb and PEPc)

STId

Contraceptives

Abortions

Lubricants and condoms

Option of indication when to consult doctors when having complications

Gender affirming services

Hormonal services and information

Gender reassignment services and information

Lifestyle education

Sex toys use and cleanliness.

Dietary advice

	11

09

06

	
aUSSID: Unstructured Supplementary Service Data.

bPrEP: pre-exposure prophylaxis.

cPEP: postexposure prophylaxis.

dSTI: sexually transmitting illness.

Textbox 2 presents a detailed summary of the steps that guided the NGT expert workshop session. The steps included preparations and orientation by the moderator, silent generation, round robin, clarification, voting, and ranking.

Steps that guided the nominal group technique expert workshop.

Steps and details

Moderator preparation and orientation of experts: The moderator explained the consent form to the experts, guided them through the demographic section, answered all questions arising from the consent form, and oriented the experts on how the workshop would run. Shared information with the experts on the pillars that are to be used during the workshop to guide them answering the research question.

Silent generation: Experts were given sticky notes and pens and asked to answer the question, “Which SRHSN content should be included in the mHealth app to address SRHSN for queer individuals?” The question was supposed to be answered by all experts, aligning their answers to the shared pillars.

Round robin: The experts were given 10 minutes each to share the ideas and suggestions they had written on sticky notes. The experts were allowed to clarify their ideas briefly.

Clarification: The experts were allowed to discuss and ask each other questions based on the shared ideas and suggestions. Similar ideas were combined, and duplicates removed as the experts discussed their ideas.

Voting and ranking: The moderator asked the experts to examine the recorded ideas on the flipcharts, select their top 10, rank them according to their priority, and indicate which should be first and last in the mobile health app.

Data Analysis

The 2 research assistants collected the completed consent forms as well as all the ideas the NGT participants had written down on the sticky notes. The sticky notes were posted on the flipchart board. The principal investigator oriented the 2 research assistants on how the data needed to be classified around the guiding pillars. The principal investigator continued checking the work done by the research assistants to ensure that no idea was omitted in the flipchart recording (the generated ideas are shown in Figure 1). The second step after placing sticky notes on the flipchart was writing all the ideas using a black marker on different flipcharts with subheadings as themes (Figure 2). This enabled the experts to examine each idea generated and explain what it represented and vote and rank them accordingly (voting and ranking are shown in Figure 2). Finally, the principal investigator took down all the ranked and voted pillars, ideas, and subheadings, including the scores each had received (the findings are presented in Table 2).

Figure 2 depicts the ranking and voting on the flipcharts. These flipcharts were used during the ranking and voting by experts during steps 4 and 5 of NGT. Experts grouped similar ideas, ranked them according to priority, and voted for all contents that need to be included in the mHealth app per priority.

Figure 1 shows the initial generated ideas on sticky notes. These are the sticky notes collected from the experts after steps 1, 2, and 3 of NGT.

Figure 1 Generated ideas on sticky notes.

Figure 2 The process of ranking and voting on flipcharts.

Ethical Considerations

This study received ethics approval and clearance from the Sefako Makgatho Health Sciences Research Ethics Committee (SMUREC/H/291/2023:PG). All experts signed informed consent forms before the workshop commenced and were informed about their rights to withdraw from the workshop anytime they wished to without any consequences. The participants were allocated pseudonyms and numbers instead of their given names: E represented an expert, and 1 represented their allocated number; thus, all 13 experts had different numbers. All experts were reimbursed for transportation and fuel remuneration with 1000 ZAR (US $67; conversion rate 1 ZAR=US $0.067).

Results

Overview

In total, 13 experts participated in the NGT workshop meeting, and each expert represented different experiences, skills, and expertise. The 6 experts’ focus areas were in key populations and LGBTQIA+: 1 in innovative marketing, 1 in primary health care, 1 in data analytics, 1 in psychosocial science, 1 in SRH, 1 in communication, and 1 in obstetrics and gynecology and researcher (Table 1). Accordingly, the expert NGT workshop had a theoretical maximum ranking score of 13 for the pillars ranked, which comprised a total of 9 pillars with headings and subheadings. The experts focused on incorporating the pillars with similar and related functions. This was followed by identifying which headings resided under each pillar and then what information should be included as subheadings. Altogether, the experts voted and ranked the following pillars: menu options (66 points), privacy and security (39 points), user engagement (27 points), information hub (26 points), user demographics (20 points), connectivity (16 points), service integration and partnerships (10 points), functionalities (10 points), and accessibility and inclusivity (7 points; Table 2).

The content obtained through the NGT workshop is represented in Multimedia Appendix 1 by a tag cloud of themes to provide a high-level overview of the mHealth app, whereafter the specific themes and findings are discussed in detail. A tag cloud is a visual representation of text data in which the size of each word indicates its frequency or importance. Frequently used words appear larger and bolder, while less common words are smaller or lighter colored. Multimedia Appendix 1 presents the most prominent topics within the NGT workshop text. Data, information, services, support, privacy, and app are the most observably prominent themes.

Menu Options

The menu options pillar was voted and ranked highest for inclusion in the mHealth app, with an overall score of 66 points. The experts emphasized the importance of this feature containing contact support information for queer individuals, with a ranking score of 11, followed by the mHealth app having a web-based consultation feature, with a ranking score of 11. The experts further suggested that mHealth should have an artificial intelligence (AI) assistant or chatbot, which ranked 10th, and a suggestion box, call-back, and email request features as important menu options, which ranked ninth each. Finally, the experts expressed that an mHealth app should accommodate videos and allow queer individuals to share videos by having a share button, both of which were ranked eighth. They further clarified that videos should contain interesting clips regarding queer matters and allow queer individuals to share success stories, such as successful gender transitioning and familial acceptance of queer children.

Privacy and Security

The privacy and security pillar was the second top-voted and ranked pillar as experts emphasized the importance of the safety and security of queer individuals and their personal information when they used the mHealth app. Overall, this pillar received a score of 39 points, with an emphasis on ensuring the protection of the users’ data and safe record keeping, both with a ranking score of 10. In addition, the experts mentioned that these 2 headings should require passwords at log-in, and the mHealth app should have a backup database logging all the activities occurring on the mHealth app. The experts further voted for an mHealth app to contain an agreement section to ensure security, and this should provide an agreement between the users and the mHealth app developers before use, which scored 9 points. This was followed by ensuring users’ safeguarding with 8 points, where the experts highlighted that mHealth app developers should ensure the protection of users’ log-in details. The experts stressed the importance of the mHealth app having an option for allowing queer individuals to change their passwords regularly, with an embedded option of reminding them to change their passwords, achieving a ranking score of 2 points.

User Engagement

The user engagement pillar followed the privacy and security pillar with a total of 27 points. This pillar had 3 headings, which the experts emphasized should be included in an mHealth app: ensuring referrals to other health professionals ranked 10 points; the availability of frequently asked questions ranked 9 points; and the availability of support groups in the mHealth app, where queer individuals can engage with other members with similar experiences and challenges, ranked 8 points. Furthermore, the experts clarified the importance of creating options in the mHealth app for queer individuals to meet peer groups; making known facts available for queer individuals in an mHealth app; and selecting or being referred to other health professionals for second opinions, advanced medical services, and specific queer-related services.

Information Hub

The experts agreed on, ranked, and voted on the information hub after a long engagement and disagreement in terms of what this pillar should be called and what content it should include. An agreement was reached to call it an information hub, under which all health matters and topics apropos queer individuals would be placed. The information hub had an overall sum of 26 points. The experts emphasized the importance of including the health education information section, which had a ranking score of 11 points; followed by the gender-affirming services section, with 9 points; and lifestyle education, with 6 points. Moreover, a focus of the experts was that the health education section should include information such as HIV, postexposure prophylaxis, pre-exposure prophylaxis (PrEP), STIs, contraceptives, information on abortions, and where queer individuals could access and learn how to use lubricants and condoms and have the option of an indication of when to consult a physician when experiencing complications. Regarding gender affirmation, the experts stated that the information to be included should be about hormone and gender reassignment services. Finally, the experts highlighted the importance of information on sex toy use, their cleaning or care for hygiene purposes, and advice on healthy dietary practices.

User Demographics

This pillar ranked 20 points in total. The experts pointed out that this pillar should have the ability to record personal data for the profile creation of queer individuals, which should include aspects such as their pronouns (ie, him, her, they, their, or them), their sexual identity (eg, male, female, binary, or nonbinary), age, and their gender orientation (eg, gay, bisexual, and transgender). This pillar had a ranking score of 11 points. Again, the experts emphasized the importance of an mHealth app having the ability to assist queer individuals in finding locations and health care facilities in different areas, focusing on ensuring that the mHealth app includes information on specialized health care facilities and nongovernmental organizations focusing on queer matters, which scored 9 points.

Connectivity

The experts stressed the importance of how queer individuals should sign up for the mHealth app to access health care services and address the SRHSN of queer individuals, stating that an mHealth app should be free, which scored 10 points. The focus was ensuring that the mHealth app could be accessed offline (when users might not have data). Experts stated that the principal investigator should seek sponsorship of the mHealth app so that queer individuals could access it freely and whenever they are offline. Furthermore, the experts stated that the mHealth app should accommodate all queer individuals who do not have smartphones by providing the use of unstructured supplementary service data, with a ranking score of 6 points.

Service Integration and Partnerships

The experts stressed the importance of an mHealth app being integrated and partnered with different health care departments to address SRHSN for queer individuals as one of the essential elements for inclusion. This had an overall voting and ranking sum of 10 points. Experts further mentioned that the mHealth app should be linked with other health services, such as HCPs, laboratories, medical aids, and private and public health care facilities, which would help with appointments and referrals.

Functionalities

The experts discussed, voted, and ranked push notifications as functionalities that should be included in an mHealth app, with a ranking score of 10 points. The experts emphasized that this functionality would assist queer individuals with appointment reminders through SMS text messaging or emails, thereby improving their SRHSN.

Accessibility and Inclusivity

This pillar was the lowest voted and ranked, with a score of 7 points. The experts accentuated that the anticipated mHealth app should not be about queer individuals only but also their families and cisgender individuals to help minimize stigma and enhance the acceptance and normality of queer existence. In addition, the experts stated that an mHealth app should accommodate people living with disabilities.

Discussion

Principal Findings

This paper presents a comprehensive description of suggestions made by a group of experts regarding the content that should be included in the development of an mHealth app to address queer individuals’ SRHSN. The proposed suggestions were based on the NGT steps followed during the workshop to ensure queer individuals’ SRHSN are addressed. On the basis of the NGT experts’ workshop, the crucial contents that need to be included in an mHealth app are represented by 9 pillars.

Our study reports on the experts’ suggestion for the anticipated mHealth app to contain various menu options, such as a web-based consultation feature, to enhance remote care. This suggestion aligns with several studies that established the need for mHealth apps to show and facilitate web-based patient-HCP interaction, as seen in a study by Cao et al [41] demonstrating that mHealth apps improve the exchange of 2-way information between patients and HCPs. In addition, our study echoes the results of the study by Markossian et al [42] that mHealth apps have interactive menu options that enable patients to engage immediately with physicians or nurses when experiencing challenges.

In addition, the experts in our study suggested that mHealth should have an AI assistant or chatbot feature to enable the provision of help for queer individuals and HCPs. Several studies identified similar findings to our study, such as the importance of using AI chatbots [43], the ease of using chatbots in supporting caregivers of people living with dementia, and the patients themselves [44]. Furthermore, a study by Haque and Rubya [45] found that chatbots are personalized, have humanlike engagement, and are received positively by users; however, much interest is lost due to improper responses and assumptions about users’ personalities. Aggarwal et al [46] found that participants view AI chatbots as a nonjudgmental space for engaging with sensitive information. Rossouw and Smuts [47] described conventional AI chatbot capabilities, such as keyword recognition and using prewritten texts and discussions to imitate conversational interaction. The same study also showed that conversational AI might be used to create unscripted AI chatbots, allowing for rapid and effective engagements that provide more tailored value to consumers [47]. Moreover, our study finds an emphasis on an mHealth app being accommodative and interesting by containing visuals and videos. Other studies note the inclusion of data visualizations [48] and video games [49] in mHealth apps to improve patients’ HCP engagement.

Our study emphasizes that privacy and security should be highlighted and included in the anticipated mHealth app. The experts participating in our NGT workshop highlighted the importance of safeguarding queer individuals’ personal data. They emphasized that the mHealth app should include a user agreement, provide secure log-in details such as passwords, and allow users to change their passwords regularly. A study by Liu et al [50] supported the outcomes reported in our study by indicating that mHealth app developers in China should pay attention to improving privacy protection processes. In addition, Schroeder et al [51] stated that people are hesitant to use mHealth apps because they have concerns about the security and privacy of their medical information. Again, our experts’ suggestions in this regard are achieved when using mHealth apps, which was also stated in a study by Zhou et al [52] as a barrier to using an mHealth app. Furthermore, the participants indicated they would require various security menu options in mHealth apps, such as regular password updates, remote wiping, user consent, and access control [52]. Finally, regarding privacy and security, Rezaee et al [53] proposed a comprehensive criterion for advising app developers, researchers, and designers by stating that passwords should be stronger and changed regularly.

Our study shows that if mHealth does not have sufficient information, queer individuals will not use it. As a result, an information hub pillar with SRHSN information, such as HIV, postexposure prophylaxis, PrEP, STIs, contraceptives, and information on abortions, was suggested by experts in our study. A study by del Río-Lanza et al [54] supports our findings, indicating that insufficient information was a barrier that limited participants’ use of the mHealth app. Several studies have produced various innovative solutions and interventions that addressed SRHSN for queer individuals; for example, Songtaweesin et al [55] focused on youth-friendly services and a mobile phone app to promote adherence to PrEP among adolescent men who have sex with men and transgender women at risk of HIV. Garg et al [56] promoted an mHealth app for self-education on HIV prevention and service; Tanner et al [57] examined the lessons learned from implementing the weCare intervention. Thus, mHealth apps are valuable tools that can be used for educational purposes [58].

The findings from our NGT expert workshop demonstrated the importance of connectivity, whereby queer individuals should be able to access an mHealth app free of charge, whether web-based or offline. It is indeed important for mHealth apps to be free, as evidenced in the study by Amiri et al [59], who stated that a lack of data and internet connection is a barrier to using these apps fully. Furthermore, in a similar study by Keel et al [60], the participants explained that the problem was weak internet connectivity. They further stated that they needed more data and stronger internet connections and affirmed that a strong and stable internet connection is key to mHealth app use [60]. Moreover, Karthan et al [61] stipulated that offline use should be a requirement (although this depends on the type of mHealth app) and further stated that, in general, sensor implementation should not require an internet connection. In addition, Tshimula et al [62] stated that the use of unstructured supplementary service data for electronic health records could enhance HCPs in managing and accessing patient information and communicating with health care systems in regions with poor internet connectivity.

Our NGT expert workshop ascertained that the mHealth app should accommodate service integration, allow referrals between HCPs, and remind queer individuals of their appointments and treatments. Ogundaini et al [63] corroborated our findings by stating that mHealth apps are appropriate for enhancing HCPs to consult with patients at hospital clinics. The same study shows that with mHealth, physicians can request detailed patient data from colleagues before making clinical decisions [63]. Furthermore, Steyn et al [64] highlighted that the Vula app is a useful tool for referring patients between departments; thus, it has the potential to improve the immediate quality of care and ensure progressive arrangements of care. Chao et al [65] concluded that mHealth apps should be able to provide reminders and alerts to patients. In addition, participants in a study by Nguyen et al [66] suggested that an mHealth app should be improved by adding scheduled reminders on phones (instead of delivering them via email) and incorporating them into the device’s native calendar. Finally, participants in the study by Peng et al [67] found reminders useful for busy people who were likely to forget things or those who needed reminders to take a lot of medication on a specific day.

Strengths and Limitations

The development of an mHealth app to address SRHSN for queer individuals could be beneficial to such individuals, their family members, and HCPs to ensure and achieve inclusivity. Queer individuals would have access to SRHSN instantaneously by using the mHealth app through the phones in their pockets. Furthermore, HCPs’ skills and knowledge would be enhanced and strengthened, thus improving queer individuals’ well-being. Finally, queer individuals’ family members would attain an understanding of their loved ones without judgment or criticism but rather with openness and love through the knowledge they would receive from an mHealth app. We intended for our NGT experts’ workshop to have >25 participants, but 12 participants had to withdraw from the workshop due to work and family commitments. Considering that we had invited participants with significant work responsibilities and roles, we anticipated that this would be an obstacle. However, the experts who attended the workshop represented a vast array of specialties and expertise, as anticipated.

Further Development and Research

The next step in intervention design is to develop an mHealth prototype in collaboration with software developers and engineers. This would provide us with a prototype that can be tested and evaluated by queer individuals, HCPs, and the group of experts who attended the workshop. The feedback from the testing will facilitate improving the prototype and developing a fully functional mHealth app. Future research should focus on innovative solutions that would address health care deficiencies for populations considered vulnerable such as queer individuals. In addition, an NGT should be used continually, as this will improve the quality of the work and redirect the focus toward determining what content should be included in other mHealth apps to address health-related issues.

Conclusions

Using an NGT to advise and enhance the development of mHealth apps is crucial to improving health care services. The data provided by different experts and the discussions, arguments, and agreements helped gather information and feedback about what researchers should focus on. Our NGT expert workshop could serve as proof and evidence for scholars, developers, and designers who intend to create and develop innovative health care solutions to address the burden of diseases. In addition, the usefulness of a multidisciplinary approach and perspective to enhance the development of the mHealth app has shown that there is a need for the continued use of this method. Finally, an in-depth and proficient mHealth app is an approach to enhancing queer individuals’ SRHSN and well-being and ensuring equality and inclusivity.

The authors would like to thank all the experts who agreed to and took the time to participate in this study by sharing their knowledge, skills, and expertise. The authors acknowledge the nominal group technique (NGT) moderator, Professor Nombulelo Sepeng, and the 2 research assistants, Tinyiko Mkhabela and Kgaugelo Mmalegodi, for ensuring that everything ran smoothly during the meeting. The authors also appreciate the kindness and support of the POP-INN clinic manager, Pontsho Komane, for providing a venue to host the NGT meeting. The research study was sponsored by the National Research Funding for Research and Development Grants for new Generation of Academics Programme Scholars (NGAP23021680129) and subsidized by the Department of Higher Education and Training’s Staffing South Africa’s Universities Framework through the new Generation of Academics Programme. The authors acknowledge that Grammarly (Grammarly Inc) and QuillBot (Learneo Inc) were used to edit some of the grammar during preparation of manuscript to ensure fluency. The authors confirm that these 2 tools were used only for editing and paraphrasing but the preparation of the manuscript was mainly done by the authors without any assistance of artificial intelligence tools.

Multimedia Appendix 1 Tag cloud of the nominal group technique themes and contents.

Abbreviations

AI artificial intelligence

HCP health care provider

LGBTI lesbian, gay, bisexual, transgender, and intersex

LGBTQIA+ lesbian, gay, bisexual, transgender, questioning, intersex, asexual, and related identities

mHealth mobile health

NGT nominal group technique

PrEP pre-exposure prophylaxis

SRH sexual and reproductive health care

SRHSN sexual-reproductive health care services and needs

STI sexually transmitted illness

WHO World Health Organization

Data Availability

The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.

Conflicts of Interest: None declared.
==== Refs
1 Lewis DA HIV/sexually transmitted infection epidemiology, management and control in the IUSTI Africa region: focus on sub-Saharan Africa Sex Transm Infect 2011 12 87 Suppl 2 Suppl 2 ii10 3 10.1136/sextrans-2011-050178 22110143 sextrans-2011-050178 22110143
2 Mason E Tomlins L Lewis D Sexually transmissible infections: current approaches to management S Afr Gen Pract 2022 3 3 89 94 10.36303/sagp.2022.3.3.0132
3 Sharma M Rewari BB Aditama TY Turlapati P Dallabetta G Steen R Control of sexually transmitted infections and global elimination targets, South-East Asia region Bull World Health Organ 2021 99 4 304 11 10.2471/blt.20.254003 33953448
4 Improving LGBTIQ+ health and well-being with consideration for SOGIESC World Health Organization 2024-04-21 https://www.who.int/activities/improving-lgbtqi-health-and-well--being-with-consideration-for-sogiesc
5 Frescura L Godfrey-Faussett P Feizzadeh AA El-Sadr W Syarif O Ghys PD Achieving the 95 95 95 targets for all: a pathway to ending AIDS PLoS One 2022 8 4 17 8 e0272405 10.1371/journal.pone.0272405 35925943 PONE-D-22-08252 35925943
6 Bekker LG Alleyne G Baral S Cepeda J Daskalakis D Dowdy D Dybul M Eholie S Esom K Garnett G Grimsrud A Hakim J Havlir D Isbell MT Johnson L Kamarulzaman A Kasaie P Kazatchkine M Kilonzo N Klag M Klein M Lewin SR Luo C Makofane K Martin NK Mayer K Millett G Ntusi N Pace L Pike C Piot P Pozniak A Quinn TC Rockstroh J Ratevosian J Ryan O Sippel S Spire B Soucat A Starrs A Strathdee SA Thomson N Vella S Schechter M Vickerman P Weir B Beyrer C Advancing global health and strengthening the HIV response in the era of the Sustainable Development Goals: the International AIDS Society-Lancet Commission Lancet 2018 07 28 392 10144 312 58 10.1016/S0140-6736(18)31070-5 30032975 S0140-6736(18)31070-5 30032975
7 Ravindran TK Govender V Sexual and reproductive health services in universal health coverage: a review of recent evidence from low- and middle-income countries Sex Reprod Health Matters 2020 12 28 2 1779632 10.1080/26410397.2020.1779632 32530387 32530387
8 Africa health strategy 2016–2030 African Union 2024-04-21 https://au.int/sites/default/files/documents/24098-au_ahs_strategy_clean.pdf
9 South African national LGBTI HIV plan 2017/22 South African National AIDS Council 2017 2024-08-05 https://sanac.org.za/wp-content/uploads/2017/06/LGBTI-HIV-Plan-Final.pdf
10 Rebe K Lewis D Myer L de Swardt G Struthers H Kamkuemah M McIntyre J A cross sectional analysis of gonococcal and chlamydial infections among men-who-have-sex-with-men in Cape Town, South Africa PLoS One 2015 9 29 10 9 e0138315 10.1371/journal.pone.0138315 26418464 PONE-D-14-56199 26418464
11 Jones J Sanchez TH Dominguez K Bekker LG Phaswana-Mafuya N Baral SD McNaghten AD Kgatitswe LB Valencia R Yah CS Zahn R Siegler AJ Sullivan PS Sexually transmitted infection screening, prevalence and incidence among South African men and transgender women who have sex with men enrolled in a combination HIV prevention cohort study: the Sibanye Methods for Prevention Packages Programme (MP3) project J Int AIDS Soc 2020 10 23 Suppl 6 Suppl 6 e25594 10.1002/jia2.25594 33000886 33000886
12 Mashingaidze R Moodie Z Allen M Bekker LG Grove D Grunenberg N Huang Y Janes HE Lazarus EM Malahleha M Nchabeleng M Laher F Sexually transmitted infections amongst men who have sex with men (MSM) in South Africa PLOS Glob Public Health 2023 4 5 3 4 e0001782 10.1371/journal.pgph.0001782 37018240 PGPH-D-22-00629 37018240
13 Malefo MA Ayo-Yusuf O Mokgatle MM Risk factors for sexually transmitted infections among men who have sex with men Afr J Prim Health Care Fam Med 2023 10 17 15 1 e1 7 10.4102/phcfm.v15i1.4080 37916720
14 Khozah MY Nunu WN Sexual and gender minorities inclusion and uptake of sexual and reproductive health services: a scoping review of literature Am J Mens Health 2023 17 4 15579883231184078 10.1177/15579883231184078 37395415 37395415
15 Rowe D Ng YC O'Keefe L Crawford D Providers' attitudes and knowledge of lesbian, gay, bisexual, and transgender health Fed Pract 2017 11 34 11 28 34 30766240
16 Nowaskie DZ Sowinski JS Primary care providers' attitudes, practices, and knowledge in treating LGBTQ communities J Homosex 2019 66 13 1927 47 10.1080/00918369.2018.1519304 30265839 30265839
17 Yu H Flores DD Bonett S Bauermeister JA LGBTQ + cultural competency training for health professionals: a systematic review BMC Med Educ 2023 08 09 23 1 558 10.1186/s12909-023-04373-3 37559033 10.1186/s12909-023-04373-3 37559033
18 Seretlo RJ Mokgatle MM Practice, attitudes and views of right to access of sexual and reproductive health services by LGBTQI among primary health care nurses in Tshwane Afr J Prim Health Care Fam Med 2023 01 20 15 1 e1 9 10.4102/phcfm.v15i1.3790 36744461
19 Malefo MA Ayo-Yusuf OA Mokgatle MM A qualitative study of the benefits and utility of brief motivational interviewing to reduce sexually transmitted infections among men who have sex with men Behav Sci (Basel) 2023 08 04 13 8 654 10.3390/bs13080654 37622794 bs13080654 37622794
20 Müller A Scrambling for access: availability, accessibility, acceptability and quality of healthcare for lesbian, gay, bisexual and transgender people in South Africa BMC Int Health Hum Rights 2017 05 30 17 1 16 10.1186/s12914-017-0124-4 28558693 10.1186/s12914-017-0124-4 28558693
21 Logie CH Dias LV Jenkinson J Newman PA MacKenzie RK Mothopeng T Madau V Ranotsi A Nhlengethwa W Baral SD Exploring the potential of participatory theatre to reduce stigma and promote health equity for lesbian, gay, bisexual, and transgender (LGBT) people in Swaziland and Lesotho Health Educ Behav 2019 02 46 1 146 56 10.1177/1090198118760682 29589481 29589481
22 Luvuno ZP Mchunu G Ncama B Ngidi H Mashamba-Thompson T Evidence of interventions for improving healthcare access for lesbian, gay, bisexual and transgender people in South Africa: a scoping review Afr J Prim Health Care Fam Med 2019 06 12 11 1 e1 10 10.4102/phcfm.v11i1.1367 31296011
23 Ross MW Kashiha J Misedah L Mgopa LR Larsson M Agardh A Venkitachalam KK Addressing the healthcare needs of African men who have sex with men: barriers to healthcare and promoting HIV and STI treatment in sub-Saharan Africa East Afr J Health Sci 2021 06 08 3 1 59 77 10.37284/eajhs.3.1.339
24 Westwood S James J Hafford-Letchfield T ‘He’s a gay, he’s going to go to hell.': negative nurse attitudes towards LGBTQ people on a UK hospital ward: a single case study analysed in regulatory contexts Ethics Soc Welf 2023 06 29 17 4 387 402 10.1080/17496535.2023.2223786
25 Mwaniki SW Kaberia PM Mugo PM Palanee-Phillips T "What if I get sick, where shall I go?": a qualitative investigation of healthcare engagement among young gay and bisexual men in Nairobi, Kenya BMC Public Health 2024 01 02 24 1 52 10.1186/s12889-023-17555-x 38166989 10.1186/s12889-023-17555-x 38166989
26 Malefo MA Ayo-Yusuf OA Mokgatle MM Receptivity of referral letters and partner notification practices for sexually transmitted infections among a population of men who have sex with men in South Africa: a qualitative study Int J Innov Res Sci Stud 2024 01 7 2 420 8 10.53894/ijirss.v7i2.2646
27 Wong NS Kwan TH Chan DP Lui GC Lee SS Regular testing of HIV and sexually transmitted infections with self-collected samples from multiple anatomic sites to monitor sexual health in men who have sex with men: longitudinal study JMIR Form Res 2022 11 18 6 11 e40996 10.2196/40996 36399372 v6i11e40996 36399372
28 Abaver DT Cishe EN Violence, abuse and discrimination: key factors militating against control of HIV/AIDS among the LGBTI sector SAHARA J 2018 12 15 1 60 70 10.1080/17290376.2018.1492960 30025496 30025496
29 Raja R Nagasubramani PC Impact of modern technology in education J Appl Adv Res 2018 05 10 3 1 S33 5 10.21839/jaar.2018.v3is1.165
30 Kabanda S Rother HA Evaluating a South African mobile application for healthcare professionals to improve diagnosis and notification of pesticide poisonings BMC Med Inform Decis Mak 2019 03 11 19 1 40 10.1186/s12911-019-0791-2 30857525 10.1186/s12911-019-0791-2 30857525
31 Chetley A Davis J Trude B McConnell H Ramirez R Shields T Drury P Kumekawa J Louw J Fereday G Nyamai-Kisia C Improving health, connecting people: the role of ICTs in the health sector of developing countries: a framework paper InfoDev Program 2006 5 31 2024-04-21 https://asksource.info/pdf/framework2.pdf
32 Mickan S Tilson JK Atherton H Roberts NW Heneghan C Evidence of effectiveness of health care professionals using handheld computers: a scoping review of systematic reviews J Med Internet Res 2013 10 28 15 10 e212 10.2196/jmir.2530 24165786 v15i10e212 24165786
33 Dayer L Heldenbrand S Anderson P Gubbins PO Martin BC Smartphone medication adherence apps: potential benefits to patients and providers J Am Pharm Assoc (2003) 2013 53 2 172 81 10.1331/JAPhA.2013.12202 23571625 S1544-3191(15)30311-3 23571625
34 WHO releases first guideline on digital health interventions World Health Organization 2019 04 17 2024-04-08 https://www.who.int/news/item/17-04-2019-who-releases-first-guideline-on-digital-health-interventions
35 Development and management of South African national department of health knowledge hub platform NBA 2024-04-08 https://www.nba.co.za/project/development-and-management-south-african-national-department-health-knowledge-hub-platform
36 Vaishnavi V Kuechler B Design research in information systems/association for information systems Forschungsbericht 2011 2024-08-05 https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=Vaishnavi%2C+V%2C+Kuechler%2C+B.+Design+research+in+information+systems%2Fassociation+for+information+systems.+2005.+2011%2C+&btnG=
37 Mullen R Kydd A Fleming A McMillan L A practical guide to the systematic application of nominal group technique Nurse Res 2021 03 11 29 1 14 20 10.7748/nr.2021.e1777 33629547 e1777 33629547
38 Delbecq AL Van de Ven AH Gustafson DH Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes 1975 Glenview, IL Scott, Foresman
39 Etikan I Musa SA Alkassim RS Comparison of convenience sampling and purposive sampling Am J Theor Appl Stat 2016 5 1 1 4 10.11648/j.ajtas.20160501.11
40 Yahaya N Rasul MS Yasin RM Sulaiman M Nominal group technique application towards the formation of social skills and values elements for apprenticeship competency models J Tech Educ Train 2020 01 19 12 1 38 48
41 Cao J Zhang G Liu D The impact of using mHealth apps on improving public health satisfaction during the COVID-19 pandemic: a digital content value chain perspective Healthcare (Basel) 2022 03 04 10 3 479 10.3390/healthcare10030479 35326957 healthcare10030479 35326957
42 Markossian TW Boyda J Taylor J Etingen B Modave F Price R Kramer HJ A mobile app to support self-management of chronic kidney disease: development study JMIR Hum Factors 2021 12 15 8 4 e29197 10.2196/29197 34914614 v8i4e29197 34914614
43 Xue J Zhang B Zhao Y Zhang Q Zheng C Jiang J Li H Liu N Li Z Fu W Peng Y Logan J Zhang J Xiang X Evaluation of the current state of chatbots for digital health: scoping review J Med Internet Res 2023 12 19 25 e47217 10.2196/47217 38113097 v25i1e47217 38113097
44 Ruggiano N Brown EL Roberts L Framil Suarez CV Luo Y Hao Z Hristidis V Chatbots to support people with dementia and their caregivers: systematic review of functions and quality J Med Internet Res 2021 06 03 23 6 e25006 10.2196/25006 34081019 v23i6e25006 34081019
45 Haque MD Rubya S An overview of chatbot-based mobile mental health apps: insights from app description and user reviews JMIR Mhealth Uhealth 2023 05 22 11 e44838 10.2196/44838 37213181 v11i1e44838 37213181
46 Aggarwal A Tam CC Wu D Li X Qiao S Artificial intelligence-based chatbots for promoting health behavioral changes: systematic review J Med Internet Res 2023 02 24 25 e40789 10.2196/40789 36826990 v25i1e40789 36826990
47 Rossouw A Smuts H Key principles pertinent to user experience design for conversational user interfaces: a conceptual learning model Proceedings of the International Conference of Innovative Technologies and Learning 2023 ICITL 2023 August 28-30, 2023 Porto, Portugal 10.1007/978-3-031-40113-8_17
48 Subramaniam A Hensley E Stojancic R Vaughn J Shah N Careful considerations for mHealth app development: lessons learned from QuestExplore Mhealth 2022 7 20 8 24 10.21037/mhealth-21-51 35928515 mh-08-21-51 35928515
49 Peever N Vella K Johnson D Ploderer B Klarkowski M Mitchell J Understanding initial experiences with Mindmax, an mHealth app that draws on shared interests in sports and video games Proceedings of the 29th Australian Conference on Computer-Human Interaction 2017 OzCHI '17 November 28-December 1, 2017 Brisbane, Australia 10.1145/3152771.3156152
50 Liu P Wang L Wang F Evaluation of Chinese HIV mobile apps by researchers and patients with HIV: quality evaluation study JMIR Mhealth Uhealth 2024 01 26 12 e52573 10.2196/52573 38277215 v12i1e52573 38277215
51 Schroeder T Haug M Gewald H Data privacy concerns using mHealth apps and smart speakers: comparative interview study among mature adults JMIR Form Res 2022 06 13 6 6 e28025 10.2196/28025 35699993 v6i6e28025 35699993
52 Zhou L Bao J Watzlaf V Parmanto B Barriers to and facilitators of the use of mobile health apps from a security perspective: mixed-methods study JMIR Mhealth Uhealth 2019 04 16 7 4 e11223 10.2196/11223 30990458 v7i4e11223 30990458
53 Rezaee R Khashayar M Saeedinezhad S Nasiri M Zare S Critical criteria and countermeasures for mobile health developers to ensure mobile health privacy and security: mixed methods study JMIR Mhealth Uhealth 2023 03 02 11 e39055 10.2196/39055 36862494 v11i1e39055 36862494
54 del Río-Lanza AB Suárez-Vázquez A Suárez-Álvarez L Iglesias-Argüelles V Mobile health (mHealth): facilitators and barriers of the intention of use in patients with chronic illnesses J Commun Healthc 2020 06 12 13 2 138 46 10.1080/17538068.2020.1777513
55 Songtaweesin WN Kawichai S Phanuphak N Cressey TR Wongharn P Saisaengjan C Chinbunchorn T Janyam S Linjongrat D Puthanakit T Youth-friendly services and a mobile phone application to promote adherence to pre-exposure prophylaxis among adolescent men who have sex with men and transgender women at-risk for HIV in Thailand: a randomized control trial J Int AIDS Soc 2020 09 23 Suppl 5 Suppl 5 e25564 10.1002/jia2.25564 32869511 32869511
56 Garg PR Uppal L Mehra S Mehra D Mobile health app for self-learning on HIV prevention knowledge and services among a young Indonesian key population: cohort study JMIR Mhealth Uhealth 2020 09 08 8 9 e17646 10.2196/17646 32896831 v8i9e17646 32896831
57 Tanner AE Mann-Jackson L Song EY Alonzo J Schafer KR Ware S Horridge DN Garcia JM Bell J Hall EA Baker LS Rhodes SD Supporting health among young men who have sex with men and transgender women with HIV: lessons learned from implementing the weCare intervention Health Promot Pract 2020 09 21 5 755 63 10.1177/1524839920936241 32757841 32757841
58 Rowland SP Fitzgerald JE Holme T Powell J McGregor A What is the clinical value of mHealth for patients? NPJ Digit Med 2020 01 13 3 1 4 10.1038/s41746-019-0206-x 10.1038/s41746-019-0206-x 31970289 206 31970289
59 Amiri P Nadri H Bahaadinbeigy K Facilitators and barriers of mHealth interventions during the Covid-19 pandemic: systematic review BMC Health Serv Res 2023 10 28 23 1 1176 10.1186/s12913-023-10171-w 37898755 10.1186/s12913-023-10171-w 37898755
60 Keel S Schmid A Keller F Schoeb V Investigating the use of digital health tools in physiotherapy: facilitators and barriers Physiother Theory Pract 2023 07 03 39 7 1449 68 10.1080/09593985.2022.2042439 35293846 35293846
61 Karthan M Martin R Holl F Swoboda W Kestler HA Pryss R Schobel J Enhancing mHealth data collection applications with sensing capabilities Front Public Health 2022 9 15 10 926234 10.3389/fpubh.2022.926234 36187627 36187627
62 Tshimula JM Nkashama DK Kalala K Dialufuma MV Didier MM Kanda H Muabila JT Mayemba CN Redesigning electronic health record systems to support developing countries Proceedings of the 27th International Conference on Medical and Health Informatics 2023 ICMHI '23 May 12-14, 2023 Kyoto, Japan 10.1145/3608298.3608338
63 Ogundaini OO de la Harpe R McLean N Integration of mHealth information and communication technologies into the clinical settings of hospitals in sub-Saharan Africa: qualitative study JMIR Mhealth Uhealth 2021 10 13 9 10 e26358 10.2196/26358 34643540 v9i10e26358 34643540
64 Steyn L Mash RJ Hendricks G Use of the Vula app to refer patients in the West Coast district: a descriptive exploratory qualitative study S Afr Fam Pract (2004) 2022 04 25 64 1 e1 9 10.4102/safp.v64i1.5491 35532127
65 Chao SM Pan CK Wang ML Fang YW Chen SF Functionality and usability of mHealth apps in patients with peritoneal dialysis: a systematic review Healthcare (Basel) 2024 03 05 12 5 593 10.3390/healthcare12050593 38470704 healthcare12050593 38470704
66 Nguyen AD Frensham LJ Wong MX Meslin SM Martin P Lau AY Baysari MT Day RO mHealth app patient testing and review of educational materials designed for self-management of gout patients: descriptive qualitative studies JMIR Mhealth Uhealth 2018 10 15 6 10 e182 10.2196/mhealth.9811 30322835 v6i10e182 30322835
67 Peng W Kanthawala S Yuan S Hussain SA A qualitative study of user perceptions of mobile health apps BMC Public Health 2016 11 14 16 1 1158 10.1186/s12889-016-3808-0 27842533 10.1186/s12889-016-3808-0 27842533
