
==== Front
BMC Health Serv Res
BMC Health Serv Res
BMC Health Services Research
1472-6963
BioMed Central London

11597
10.1186/s12913-024-11597-6
Research
Facilitators and challenges of implementing a digital patient education programme for rheumatoid arthritis into clinical practice
https://orcid.org/0000-0002-3176-6207
Knudsen Line Raunsbaek linknuds@rm.dk

12
https://orcid.org/0000-0002-5809-3508
Lomborg Kirsten 34
https://orcid.org/0000-0003-2562-9174
Hauge Ellen-Margrethe 12
https://orcid.org/0000-0003-0103-4328
de Thurah Annette 12
1 https://ror.org/040r8fr65 grid.154185.c 0000 0004 0512 597X Department of Rheumatology, Aarhus University Hospital, Palle Juul- Jensens Boulevard 99, Aarhus, 8200 Denmark
2 https://ror.org/01aj84f44 grid.7048.b 0000 0001 1956 2722 Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
3 grid.419658.7 0000 0004 0646 7285 Steno Diabetes Center Copenhagen, Copenhagen, Denmark
4 https://ror.org/035b05819 grid.5254.6 0000 0001 0674 042X Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
20 9 2024
20 9 2024
2024
24 110412 6 2024
17 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Background

The integration of telehealth interventions into clinical practice is frequently delayed, hindering the full adoption. Previously, we developed a digital patient education (PE) programme for self-management in rheumatoid arthritis (RA). While the programme design considered crucial factors to ensure the likelihood of success in clinical practice, there is a need for a systematic evaluation of implementation perspectives. The purpose of this study was to explore perspectives crucial to implementation of a digital PE programme in clinical practice.

Methods

The non-adoption, abandonment, scale-up, spread and sustainability (NASSS) framework was used to evaluate the successes and challenges of implementing the digital PE programme. We included a data set consisting of qualitative focus group discussions involving study nurses, rheumatologists, and leaders from rheumatology departments. Data analysis was guided by a deductive content analysis approach. Further we included data from earlier studies pertaining to the programme’s implementation, comprising the programme development process, a randomized controlled trial evaluating the programme’s effectiveness, and a qualitative study exploring patients’ perspectives of the programme.

Results

Facilitators and challenges of importance to implementation of digital PE were identified. While a wide range of patients could benefit from using digital PE, future implementation should aim for an even broader group than those studied. Both patients and healthcare providers embraced the technology, and the fact that it did not require specific technical skills enhances its potential for success. However, offering digital PE should be based on individual assessments, and expanding its use will require organizational adjustments. An adaptable structure is needed to accommodate unforeseen care needs that may arise following the use of digital PE at home. There was indication of some reluctance among healthcare providers toward the programme shown by concerns about changing roles, which could impact the adoption of the program.

Conclusions

The design and ease of use of the technology, the program’s effectiveness, its availability, and the potential to release healthcare resources may encourage the implementation of digital patient education. Challenges associated with implementing this mode of care pertains to the condition and the patient population, user adoption of the technology, and the organization of patient education.

Trial registration

The study is registered by the Central Denmark Region Scientific Committee (no. 1-16-02-52-19).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-024-11597-6.

Keywords

Digital patient education
Self-management support
Telehealth
Implementation
NASSS framework
http://dx.doi.org/10.13039/501100018775 Danish Foundation TrygFonden 149582 149582 149582 149582 issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcBackground

Telehealth and digital interventions are increasingly used in rheumatology and have the potential to improve patient care by offering more efficient access to healthcare. However, the adoption of tele-health interventions into clinical practice is often hampered by late implementation [1–5]. Research has identified factors that may promote or hinder the adoption of various telehealth interventions in rheumatology [1, 3, 6–10]. To ensure successful implementation of telehealth and maximize the benefits of telehealth it is important to take advantage of its ability to offer increased flexibility, better access to healthcare, various means of communication, and greater patient participation. However, it is also essential to acknowledge the limitations or drawbacks, such as reduced face-to-face contact, potential technical difficulties, various levels of digital literacy, new ways of working, and limitations related to standardized features or content.

Only a few studies have focused specifically on the implementation process of digital interventions in rheumatology, although some studies do offer relevant insights into implementation. The European Alliance of Associations for Rheumatology (EULAR) highlights the importance of accessibility and usability of digital self-management applications (apps) in ensuring successful implementation and sustainability in rheumatic and musculoskeletal diseases (RMDs) [2]. A study on the development of telehealth interventions in RMD rehabilitation emphasizes the need for involving users in the development process and tailoring interventions to individual needs. Further, variations in health literacy should be considered to ensure usability, effectiveness, and implementation [5]. Another recent study found that only around 47% of rheumatologist and general practitioners felt confident in using telemedicine for communicating with patients, although they were positive about the potential for future implementation [11]. The study identified a need for training courses to improve telemedicine knowledge [11]. The same was found among physiotherapists delivering telephone-based exercise therapy for knee osteoarthritis [12].

Previously, we have developed a digital patient education (PE) programme for self-management in rheumatoid arthritis (RA), called ‘Know your rheumatoid arthritis’, and investigated both effectiveness in a randomized controlled trail (RCT) [13] and the patient’s perspective of using it [10]. The programme takes into account various factors that may increase the likelihood of success in clinical practice, including integration of evidence-based content, self-management and behaviour change theories, incorporation of features to support health literacy, and stakeholder involvement [14]. However, there is a need for a thorough and systematic evaluation of implementation perspectives to ensure integration of the programme into daily clinical practice.

In this study, we aimed to explore perspectives crucial to implementation of the digital PE programme in clinical practice.

The digital PE programme ‘Know your rheumatoid arthritis’

The programme was designed as a website, primarily optimised for computer or laptop use. It could be assessed on a tablet but was not optimised for smartphone use. In the RCT study [13], users were granted access to the programme by study nurses via email, and a private username and password were assigned during the system setup process in the Learning Management System (LMS) behind the programme. The programme could be used repeatedly free of charge. Both the digital PE programme and the LMS were external websites, independent of other hospital technologies such as the electronic medical record system. The LMS generated data regarding participants’ utilization of the programme. The programme comprised three modules. The first module was mandatory as it delivered essential disease-specific knowledge necessary for understanding RA, managing medical treatment properly, and responding to symptoms, flares, and potential side-effects. The two remaining modules were optional and provided additional insights into the disease, treatments, potential co-morbidities, examinations, and self-management strategies [13, 14].

Methods

Design

This study used the Non-Adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework [15–18], including the Complexity Assessment Tool (CAT) [19] as an evaluation framework to create a narrative outlining the successes, challenges, and failures of the digital PE programme. The NASSS framework addresses the challenges often faced by technological health interventions when it comes to implementation, including scaling up, spreading, and sustaining them in the long term. It consists of 7 domains covering: the illness or condition, technology, value proposition, adopters, organization, wider system, and embedding and adaptation over time. It provides an approach for identifying and analysing the complexities in technology projects to improve the chances of success and sustainability [16–18].

Inspired by the study of Abimbola et al. [15] we based our study on both a primary and a secondary dataset. The primary dataset consisted of qualitative focus group discussions with study nurses, rheumatologists, and leaders from the rheumatology departments in Denmark who had participated in the RCT evaluating the effectiveness of the digital PE programme. The secondary dataset consisted of data from the development process of the PE programme [14], the RCT [13], and the qualitative study exploring patients’ perspectives of the programme [10]. This study was reported according to the consolidated criteria for reporting qualitative research (COREQ) (Additional file 1) [20].

Primary dataset

Design, study population and recruitment

In March and May 2023, two qualitative focus group discussions were conducted, each comprising study nurses, rheumatologists and leaders. These discussions aimed to explore experiences with the digital PE programme and perspectives on its future implementation in clinical practice. Eligible participants were informed about the study by LRK during a meeting between the project group and the rheumatology departments from the RCT. Subsequently, they were contacted by email and provided with written information about the study. Verbal and signed written consent were obtained before inclusion. Participants were purposively selected based on criteria including involvement in the RCT to some extent and having interest in discussing future implementation of digital PE in clinical practice. The NASSS framework was applied, however our analysis primarily provided insights from the perspective of the intended adopters and the organization. Thus, not all domains of the NASSS framework were relevant as some are targeted e.g. technology developers, and business leads. Domain 3 (The value proposition) was evaluated using data from the secondary dataset, and thus not included in the focus group discussions as participants did not have insight into the costs of the technology. Finally, the current analysis did not include domain 7 (adoption over time), as the programme has only been utilized for a limited time in a research setting.

Data collection

To guide the focus group discussions, we created a questioning route inspired by the interview version of the NASSS-CAT [19], supplemented by studies addressing opportunities and barriers of using telehealth in rheumatology. Research questions according to the NASSS domains and examples of interview questions appear from Additional file 2. To ensure perspectives beyond this framework, open-ended questions that allowed participants to freely discuss their thoughts on the topic were also included. The initial step in developing the questioning route involved discussions with clinicians and patient research partners (PRPs). Then, LRK drafted the questions, and final agreement on questions was achieved through discussion with AdT and KL, ensuring that clinically relevant topics were covered.

The focus group discussions took place at Aarhus University Hospital, with LRK as the facilitator. LRK, experienced in qualitative studies, received assistance from AdT during the interviews, an experienced researcher and supervisor. A PRP was also present to observe and contribute with additional questions from the patient perspective. The discussions were recorded and verbatim transcribed into text by LRK.

Data analysis

To analyse the primary dataset, we used a deductive content analysis approach [21] structured by a categorization matrix according to the NASSS domains, and comprising the phases shown in Fig. 1. An example from the categorization matrix appears from Table 1. The analysis was organised and managed by LRK using NVivo 12 Pro [22]. Relevant features of NVivo were used to support the analysis process, including file and case classification, coding, memos, and visual features.

Fig. 1 Data analysis of the primary dataset. *To accommodate new concepts that did not align with the categorization matrix, additional categories were generated inspired by the principles of inductive content analysis; (1) Grouping data under appropriate headings; (2) Creating categories to describe, understand and generate knowledge (sub-categories, generic categories and main categories); (3) Abstraction, i.e. a general description of the research topic through categories; (4) Assigning content-characteristic words as names for the categories

Table 1 Example of categorization matrixa

Research question	Supplier / version	Functionality, usability and acceptability of users	Performance / dependability	Technical interdependencies	Competencies / different ways of working or interacting	Updates / maintenance	
What are the experiences of the existing technology?							
aAn example of the categorization matrix related to the technology domain of the NASSS framework. The complete matrix of all domains can be found in Additional file 3

Secondary dataset

Studies included in the secondary dataset were all part of the WebRA project and provided insights of relevance for future implementation. The studies are presented in Table 2. Selected secondary data were sorted into the categorization matrix. Both positive and negative findings were identified in a process performed by LRK, and continuously discussed with AdT and KL.

Table 2 Overview of secondary dataset

Paper	Aim	Selected data for analysis in this paper	Principal findings	
Knudsen et al. [14] (2021)	To describe the design and development process of a digital patient education (PE) programme for self-management in patients with rheumatoid arthritis (RA).	Data from feasibility test: Individual qualitative interviews with 10 patients following prototype testing of the digital PE programme.	Test persons found the programme feasible with clear and easily understandable content. They liked the pace and found coherence between graphics, speech, and tests. The animations and graphics were appreciated. The impression of the patient stories differed. Overall, they were satisfied with the user interface and design as well as the programme’s flexibility, which allowed for individual customization based on specific needs. Suggestions for improvements were incorporated when possible.	
Knudsen et al. [13] (2024)	To evaluate the effectiveness of digital PE integrated into standard healthcare practices in improving self-management among patients recently diagnosed with RA.	A total of 180 patients from five rheumatology clinics; 89 assigned to the intervention group receiving digital PE, and 91 patients assigned to usual care (face-to-face PE). Data on outcome measures and healthcare utilization compared to the control group. Data on use of the digital PE programme.	The analysis revealed a statistical significant improvement in self-efficacy favouring the intervention group at month 12. However, this difference was not considered to be clinically significant. High self-efficacy, disease and treatment knowledge, health literacy, and quality of life scores were consistently observed from the start with minor improvements over time in both groups. Out-patient clinic contacts were fewer in the intervention group compared to the control group.	
Knudsen et al. (2023) [10]	To explore patients’ perceptions of taking part in digital PE, and how this contributed to their self-management of RA.	Data from individual qualitative interviews with 20 patients who participated in the randomized controlled trial examining the effectiveness of the programme.	Participants generally had a positive perception of the programme, and highlighted advantages, such as flexibility, the possibility for repetition, the entertainment factor, the convenience of accessing if from familiar surroundings. However, a disadvantage was unmet relational support needs. Minor technical difficulties were identified, and the fact that the programme was supported by a website format was found inconvenient for some. For the majority, a need for insight into the condition motivated their active engagement with the programme. While the programme facilitated knowledge acquisition about RA, it was also clear that relational support from HCPs and a positive attitude toward living with RA were crucial for achieving self-management.	

Results

The focus groups included ten participants: eight females and two males. Five were nurses, two were rheumatologists, and three held leadership positions. Among the participants in leadership positions, some also had clinical roles, while others were primarily managers dedicated to leadership. On average the participants had 22.3 years of professional experience (range: 16 to 33 years), and 18 years (range: 9 to 33 years) in the field of rheumatology. The duration of the focus group discussion was approximately two hours.

The inductive analysis revealed an overarching theme termed ‘Future perspectives in digital patient education’. This theme was subsequently divided into three subthemes; ‘Transitioning from a website to an app’, ‘New modes of delivering patient education’, and ‘Forming relationship’. Themes of the content analysis and illustrative quotes from the focus groups appear from Table 3. Quotes from the secondary dataset (development study and qualitative study) are not included as these were previously published. Furthermore, Fig. 2 depicts the facilitators and challenges for implementation based on the NASSS domains. We refer to the primary dataset by the label ‘focus groups’, and studies in the secondary dataset are named ‘RCT’, ‘Development study’, and ‘Qualitative study’, respectively. To maintain participants’ anonymity, the perceptions of nurses, rheumatologist or leaders are gathered under the collective term ‘Healthcare providers’ (HCPs).

Table 3 Themes of the content analysis and examples of quotes supporting the analysis of data from the focus group discussions. (note to editorial office: table should be placed after section ‘Forming relationships’

NASSS domains and themesa	Illustrative quotes from healthcare providers in focus groups	
The condition	• “A generation gap exists within the group, but it is my impression that a major part of the older patients also seems comfortable with using computers, smartphones etc.”

• “I think this [digital PE programme] is a favourable solution for a range of patients. Despite age, it is my experience that they have been very receptive to using it. But some will be excluded, for example patients with language difficulties”.

• “Some patients aspire for the physical contact, and this may also be patients in their 20s or 40s”.

	
The technology	• “I don’t think this requires more of us [HCPs] than you can expect”.

• “If implementing a programme like this, we are entrusting patients with more responsibilities, which necessitates the need for an open-door policy to ensure timely care”.

	
The intended adopters	• “Often when such solutions are introduced, emotions like: “They no longer require my presence”, might arise… I believe it is very important to give words to these feelings and recognize that technology cannot and should not replace people in healthcare’ – if not doing so, it will be become difficult to succeed”.

• “I am of the opinion that the programme actually should do some replacements, but hopefully it will make it up for the deficiency of nurses within our healthcare system”.

• “The technology ought to complement human interactions rather than replace them, while taking over relevant tasks”.

• “I like the programme, but I would prefer to have both an in-person meeting and also utilize this [the programme] … Perhaps if I initiate the first interaction and observe that the patient acquired the app, I could provide some basic information, and the programme could be adequate for the remaining details… But it is of course a new way of working…’.

• “Encouraging patients to reach out to us if they have any uncertainties and conducting follow-ups ensures patients are not left on their own… I agree though, that certain patients might not be suitable for digital patient education… Therefore, we need to find out everyone’s capabilities and needs”.

	
The organization and the wider system	• “I think it is quite evident that we should embrace its usage, considering that this [digital advancements] is a part of our world – people know it [technology] and use it all the time”.	
Transitioning from a website to an appa	• “It could be a disadvantage that they cannot use it at their mobile phones”.

• “It would make a difference for me that I could use it at my phone, if I was a patient. My laptop is packed away – I actually never use it”.

• “I think this could be challenging…, I mean, which questions could be asked and answered through a chat functionality compared to telephone conversations, which allow for additional questions. I think the personal contact is important when asking questions”.

• “I don’t think it will be different from conversations in the phone”.

• “Some questions may be easy to reply to, and some may be more suitable for face-to-face or telephone follow-up”.

	
New modes of delivering patient educationa	• “Introducing patients to the programme could assist us in gaining a better understanding of the individual’s capabilities and support needs”.

• “I think this programme could serve as a working tool similar to the written material we use today, and thereby be a supplement to the crucial information [about the disease and treatment], which we should provide, and then show the programme’s opportunities to the patient”.

• “We have often discussed how to improve these conversations [Face-to-face PE] … It entails identifying the appropriate patients and tailoring the guidance. With digital PE, they [patients] could potentially be more prepared for a conversation aligned with their individual needs’.

• “I think we have a group which can navigate the programme independently, and then of course others may require additional support”.

	
Forming relationshipsa	• “In my experience, when I called patients for a follow-up regarding medication, they were pleased that I was the one making the call, because I had included them in the study… They knew me, and this familiarity by a name and a face, and having met before promotes confidence in relations… As a result, I think they would feel comfortable reaching out to me if needed. In contrast, I believe a lack of familiarity might discourage them from initiating such contact…”.

• “As HCPs, it’s our responsibility to be on track of the needs of our patients, enabling us to tailor our care and differentiate to their different needs”.

	
aThemes derived from the inductive analysis of primary data

Fig. 2 Facilitators and challenges for implementation of digital patient education

The condition

Both the primary and secondary data recognized the diversity within the RA patient group, reflecting varying needs along the patient pathway, including those related to PE. Based on HCPs experiences expressed in focus groups, patients’ age did not seem to hinder participation and use of the programme, but there could be a risk of not reaching all patients, e.g., due to literacy, language difficulties, or simply because some patients need the face-to-face contact. Baseline characteristics in the RCT [13] also reflected heterogeneity with respect to age, gender, educational levels, and employment statuses. Despite this sociodemographic diversity, the group demonstrated relatively high levels of self-efficacy, knowledge, health literacy, and quality of life, emphasizing the need for also considering non-participants when planning the implementation of the technology into clinical practice.

The qualitative study [10] provided data about the patient groups’ engagement with digital PE. An immediate need for information and knowledge about RA promoted the use of the programme among many patients. In contrast, mental distress following diagnosis could pose barriers to the utilization of digital PE. Lastly, patients shifted between an illness-focused and a wellness-focused perspective [23], influencing their approach to utilizing the programme. The target group of the digital PE programme was newly diagnosed patients. However, since RA is a chronic disease, ongoing adjustments and coping strategies in everyday life are necessary; certain information may be beneficial at the early stage and other parts of the programme may become more relevant later, for example if patients encounter new challenges or treatment changes.

The technology

Data from the development study and qualitative study showed that the technology was widely accepted in the patient group because of a simple and intuitive user interface, enabling easy navigation in using the programme. The design was also appreciated for its calming and appealing elements, including attractive colours, graphics and animations [14]. No specific skills or competencies seemed to be needed to use the programme. Minor technical difficulties included login errors or oversight of username and password.

In the qualitative study it was suggested to optimize the programme for smartphone and tablet use, as the limitation of the current version, including the login process, could potentially restrict accessibility and limit programme usage [10]. HCPs in focus groups likewise had the impression that patients did not encounter notable difficulties in using the programme, and they also found the LMS, that was used to set up individual patients for the programme, to be user-friendly and easy to navigate in. No challenges were reported regarding the lack of integration between the programme and other electronic systems in the hospital, and no specific competencies were required for HCPs to manage the system. The login process was, however, a drawback, as it prevented HCPs from having access to the patients’ user interface, and thereby provide help in starting the programme and helping them become familiar with its usage. In the qualitative study, it became evident that the lack of this element could pose a potential barrier for certain patients to engage in digital PE, particularly those who expressed hesitancy about using the programme, suggesting a need to incorporate a brief programme introduction [10].

Future implementation would necessitate modified work procedures. For instance, organizing digital PE and allowing space for face-to-face follow-up within the daily tasks of the organization may become more unpredictable if a substantial portion of the patients uses the technology. In contrast to the scheduled face-to-face PE sessions, it may necessitate a more flexible organizational structure to accommodate the potential needs arising during the use of digital PE.

Finally, updates and maintenance of the technology and content were discussed in the focus groups. The current version of the programme was developed by researchers at Aarhus University Hospital. To ensure a successful implementation of digital PE outside a research setting, it was suggested that a reorganization of this process by setting a steering group of relevant HCPs and leaders from different rheumatology departments, and technological staff could enhance involvement and ownership, potentially facilitating nationwide adoption of the programme through a collaborative approach.

The value propositions

The development and evaluation of the digital PE programme took place within a research setting, supported by external funding. Should there be a desire to integrate the programme in the organization in the future, it necessitates either securing additional funding or allocating resources from departmental budgets to cover associated expenses. This could potentially limit the implementation, as regards scaling up and spread of the intervention.

Findings from the RCT investigating the effectiveness of the digital PE programme in improving self-management among patients newly diagnosed with RA provided insights into the value of this programme to the patient and the healthcare system. Despite the minimal distinctions between the groups, the programme could be a valuable resource for both patients and the healthcare system. The findings showed that digital PE programme was superior in improving self-efficacy compared to usual care, even though the difference may not be clinically significant [13]. Both groups improved their knowledge, health literacy, and quality of life during the study. Additionally, individuals who received digital PE had fewer contacts with out-patient clinics, suggesting they might be more capable of handling challenges independently before seeking clinical assistance. If this strategy can be maintained, a programme like this holds the potential to contribute to resource allocation, however, further research is needed to determine the costs and benefits [13].

Insights from the qualitative study revealed that the programme’s significance to patients stemmed from the technology, allowing for convenient utilization while minimizing hospital visits. Some patients also appreciated the potential for revisiting and learning within the comfort of learning at home when appropriate. This was enhanced by the programme’s possibility of breaking down information to minor segments. Conversely, drawbacks were associated with diminished face-to-face interactions with HCPs, and occasional delays in programme initiation and responsibility of the patients for learning [10].

The intended adopters

As described within the technology domain, users of the programme and LMS found it easy to use. Nevertheless, insights from the focus groups revealed a duality in HCPs’ perceptions, encompassing both reluctance and endorsement of the technology. The reluctance or resistance seems to be linked to concerns about potentially being replaced by automated systems. Concerns regarding patient safety and the potential of overlooking patients’ needs due to the decrease in direct in-person interactions were also highlighted, followed by thoughts that some patients might be likely to decline the programme solely because it’s digital in nature.

In general, HCPs acknowledged the potential of the programme, suggesting that it could be an appropriate solution for numerous patients. The availability of information, the ability to supplement their own information, and the fact that the programme could convey more comprehensive information communicated in various forms compared to what can be covered during hospital appointments were emphasized.

Some HCPs considered the digital PE programme to be an “add on” to face-to-face PE and expressed discomfort due to the perception of relinquishing the “touch” and “sense” of the patient when not being physically present with the patient. HCPs also expressed that they gained trust in the information provided within the programme and saw it as a reliable source instead of patients relying on random internet pages.

The qualitative data showed that digital PE was well-received by patients as regards both the content, amount of information and the various ways it was presented. Patients regarded the programme as a valuable source of information. Most of them expressed appreciation for the opportunity to access it immediately after receiving their diagnosis, when their need for information was substantial, and the usage tended to decrease over time. Some patients found the solution inappropriate due to relational support needs and emotional distress following the diagnosis [10].

The organization and the wider system

Overall, HCPs viewed their organizations as prepared to adopt digital solutions like our programme, and the widespread acknowledgement of technologies in society acted as a facilitator for its acceptance and implementation, as this was perceived as an inherent part of everyday life. It was emphasized that any potential resistance should be recognized and considered in the process of implementation. No specific barriers to implementation within the organizations, except for economic concerns, were discussed, and HCPs held the belief that both colleagues and leaders would provide support. Nonetheless, the necessity for assistance during implementation was emphasized.

Future perspectives of digital patient education

Transitioning from a website to an app

The focus groups discussed the possibility of switching the platform into a mobile app solution, and in general, HCPs expressed a favourable response to this idea to enhance availability. Both the participants in the qualitative study [10] and the focus groups emphasized the importance of keeping a flexible organization of the programme’s content. Despite the standardized content of such a programme, this flexibility is crucial to accommodate individual needs and enables the selection of topics that are relevant to everyone’s specific situation. Data from the RCT showed that patients engaged with all the modules and topics within the programme, albeit to different extents, probably reflecting this individuality [13]. The current version of the programme lacks any interactive functionality with HCPs, but data from the qualitative study introduced the possibility for a chat function [10]. Opinions among HCPs in focus groups regarding this idea were diverse and not unanimous. This diversity of perspectives reflected concerns about the types of questions that could be effectively posed using this approach, and whether the quality of responses though a digital solution would be adequate, and for others this seemed to be straightforward. This could indicate a necessity for well-defined guidelines and alignment of objectives when considering the implementation of such a feature.

New modes of delivering patient education

An introduction to the digital PE programme could potentially help planning and tailoring care to the patients. Although, a risk of investing more time initially when introducing patients to the programme was present, HCPs regarded this time as well-spent, considering that some patients may not require any specific introduction. The HCPs noted that a reorganization of future PE and implementation of digital PE could potentially reduce the overall burden of PE, and this could lead to a transformation in the organization and mode of delivering PE. This could transform their professional role from teaching to guidance. For example, the current practice of face-to-face PE often implies a firm approach as nurses try to adhere closely to educational guidelines. However, with the implementation of digital PE, follow-up sessions could be provided as needed and customized according to the patient’s preferences and further needs. This shift could potentially make the task of PE more engaging and fulfilling for nurses and could strengthen the partnership between patients and HCPs if patients became more actively engaged.

Forming relationships

Data from the focus groups highlighted the importance of HCPs being able to establish a relationship with patients before feeling confident to “hand them over” to technology. They also believed that this was important for most patients. Because of the change in relational interactions, both rheumatologists and nurses also felt the necessity to ensure they had provided sufficient information about the disease and treatment, and had taken care of emotional responses, before introducing the technology. This initial interaction could serve as the basis for the decision about whether a patient should be offered digital PE. Hence, several HCPs emphasized the importance of evaluating individual needs related to PE and the need for self-management support when implementing digital PE.

Discussion

In this study we identified both the successes, facilitators, and challenges of a digital PE programme, which should be considered when implementing beyond a research setting.

While this programme holds promise for a wide range of patients, there is a concern that it may exclude individuals with health literacy and self-management difficulties. Patients and HCPs widely embraced the technology, and its usability did not require specific technical competencies. However, optimizing the programme for smartphones and tablets seems crucial for broader availability. Individual assessments should be conducted to cater varying support needs when enrolling patients into digital PE, and expanding its use necessitates organizational adjustments. Overall, HCPs endorsed digital PE, but there were mixed opinions with some expressing reluctance due to reduced face-to-face contact, including concerns such as fears of automation replacing their roles, potential neglect of patient’s needs, and safety compromises.

In the following, we delve into the main challenges of these findings in the context of future implementation.

RA is a well-defined disease with established treatment strategies and recognized management guidelines aimed at reducing disease progression and complications [24]. While RA primarily affects females and typically manifests around the sixties, it spans a broad age range [25], as also demonstrated in our RCT by a median age of 59.0, and an age span of 25 to 82 years [13]. Having clear management guidelines can enhance the likelihood of successfully implementing digital PE. However, the diversity in RA-patients age, the necessity of treatment adherence to ensure disease control, and the various approaches into using digital PE, as revealed in our qualitative study, add complexity to adopting the technology within this patient group. This induces a task of accommodating a diverse group of patients with varying needs in future implementation.

Despite our efforts to consider different health literacy levels during program development, we failed to include patients with diverse health literacy levels in the RCT. Moreover, we did not assess eHealth literacy competencies, which involve the ability to seek, find, understand, and evaluate health information from electronic sources to address health problems [26]. A Danish study examining eHealth literacy among individuals with endocrine and gastrointestinal diseases found that patients’ adoption and utilization of digital health services are more influenced by their eHealth literacy than by sociodemographic factors [27]. A systematic review also highlighted that issues related to these competencies are often overlooked in telehealth development for socially disadvantaged groups [28].

Thus, to enhance future implementation of digital PE, it may be advantageous to delve deeper into the realms of health literacy and eHealth literacy to ensure the successful integration of digital PE for a broader patient group than those included in our studies.

Although our studies provide valuable insights into the adoption of digital PE from both the perspectives of patients and HCP, a more detailed analysis of the factors contributing to technology acceptance or rejection may improve the implementation process. Assessing technology acceptance using established models, which consider factors like usefulness, ease of use, social influence, organizational support, and technical proficiency [29], could provide a comprehensive understanding of adoption dynamics.

While our studies indicate that digital PE is valued, easy to use, effective and warranted increases the likelihood of its success and adoption, our findings also indicate that integrating digital PE represents a shift in HCPs traditional practices, impacting perceptions of care quality and roles. Training in mastering the programmes use, gaining confidence in the technology, and integrating it in daily practice may help alleviate HCPs concerns. However, by the integration of technologies and artificial intelligence (AI) into healthcare, the models and methods of delivering care constantly undergo transformation. It is crucial to recognize the significance of HCPs roles in this process, including their central role in applying these technologies, working collaboratively with patients, and tailoring their use to meet individual patient needs, for example in a digital PE intervention like ours. Digital health interventions and AI can enhance healthcare and healthcare accessibility [30–32], but should not replace essential components like healthcare providers and leadership, and it is important to acknowledge that there are limitations to what digital health can address [30]. Guidance on AI usage also underscores the importance of responsible, effective, and safe use while ensuring inclusivity, with a core principle that humans should remain in control of health-care systems and medical decisions [32].

While the organizations capacity to implement digital PE was considered reasonable, potential challenges may emerge as this novel approach to PE will necessitate changes in work processes and a more adaptable organizational structure to provide various modes of PE tailored to individual needs and preferences. A study focusing on healthcare professionals’ perspectives regarding successful organizational changes in a continuously evolving healthcare system identified some key factors [33], which could be relevant in planning the implementation of digital PE. Influencing the change through early involvement or initiation, being prepared for change with clear and timely communication, and recognizing the value and benefits of the change for both healthcare professionals and patients were found to be crucial for the successful implementation of changes in healthcare [33].

Another crucial aspect to consider in implementing digital PE or similar digital health technologies is the potential shift of resources from one setting to another. While our RCT showed that patients receiving digital PE had fewer interactions with outpatient clinics across all forms of contact, other studies have reported mixed findings. For example, an internet-based self-management programme for RA and other rheumatic diseases showed no impact on healthcare utilisation [34], while another study found a significant decrease among RA patients accessing an online educational intervention [35], and yet another reported an increase in telephone contacts following remote monitoring of RA [36]. Therefore, it is also important to recognize whether the introduction of digital health technologies increases the need for telephone support or healthcare providers to support patients digitally and manage these technologies.

Strengths and limitations

This study exhibits a robust strength in its utilization of an extensive dataset derived from multiple studies. This dataset has yielded valuable insights into the implementation of digital PE, and although the secondary dataset was originally conducted for different purposes, its incorporation into this study ensured the inclusion of diverse perspectives.

Another strength is the application of the NASSS framework, which has facilitated a systematic examination of implementation perspectives by drawing our attention to both the potential successes and challenges associated with digital PE. The framework, including tools like the NASSS-CAT interview version [19], was instrumental in shaping our questioning route, ensuring the comprehensive coverage of all relevant domains in our focus groups.

We acknowledge that a limitation of this study could be the absence of a broader spectrum of stakeholders, e.g., technical staff, top leadership or health economists. The NASSS-CAT interview tool targets different groups of individuals involved in the project, and our analyses predominantly offer insights from the perspective of the intended adopters, i.e., patients, HCPs and leaders. Thus, certain domains may not have been fully explored in our focus groups meaning that a wider group of stakeholders could potentially have provided other insights.

Conclusions

In conclusion, we found that the design and ease of use of the technology, the program’s effectiveness, its availability as an ongoing resource for self-management support that can complement HCPs guidance, and the potential to release healthcare resources may encourage the implementation of digital patient education. In contrast, challenges associated with implementing this mode of care pertains to the condition and the patient population, user adoption of the technology, and the organization of patient education.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1.

Supplementary Material 2.

Supplementary Material 3.

Abbreviations

AI Artificial intelligence

EULAR European Alliance of Associations for Rheumatology

HCPs Healthcare providers

LMS Learning Management System

NASSS Non-adoption, abandonment, scale-up, spread and sustainability

NASSS-CAT Non- adoption, abandonment, scale-up, spread and sustainability complexity assessment tool

PE Patient education

PRPs Patient research partners

RMDs Rheumatic and musculoskeletal diseases

RCT Randomized controlled trail

RA Rheumatoid arthritis

WebRA Project acronym

Acknowledgements

We extend our gratitude to the healthcare providers and leaders who agreed to participate in this study, and generously shared their experiences and viewpoints regarding digital PE, and aspects of importance into future implementation. We also express our thanks to the patients who participated in other parts of the WebRA study for their contribution. Lastly, we appreciate the engagement of our patient research partners, Jette Fuhlendorff and Troels Overgaard Munch, and the steering group affiliated to the project, for their significant contributions to the execution of the present study.

Authors’ contributions

LRK: Conceptualization, Methodology, Investigation, Analysis, Writing – Original Draft, Visualization, Project administration. KL: Conceptualization, Methodology, Analysis, Writing – Review & Editing, Supervision. EMH: Conceptualization, Writing – Review & Editing. ATH: Conceptualization, Methodology, Analysis, Writing – Review & Editing, Supervision, Project administration. All authors read and approved the final manuscript.

Funding

The development of the digital PE programme was funded by a grant from Novo Nordisk foundation (grant number: NNF 18OC0052886). The remaining WebRA studies are funded by TrygFonden (grant number 149582). The funding sources had no role in any phases of the studies.

Availability of data and materials

Parts of the primary dataset are available in Table 3. However, the full datasets generated and analysed during the current study are not publicly available to ensure the anonymity of the participants (primary dataset) but are available in Danish from the corresponding author on reasonable request. For data availability from the secondary dataset we refer to the original articles.

Declarations

Ethics approval and consent to participate

The WebRA study was approved by the Central Denmark Region Scientific Committee (no. 1-16-02-52-19). As per Danish Law and the Central Denmark Region Committee of Health Research Ethics, this study does not require any further ethical approval, as interview studies that do not involve the inclusion of biological material are exempt from reporting [37]. Possible participants were provided with written and verbal information, and verbal and signed written consent were obtained before being included. This included details on voluntary participation, withdrawal, confidentiality, and anonymity.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. de Thurah A, Bosch P, Marques A, Meissner Y, Mukhtyar CB, Knitza J, et al. 2022 EULAR points to consider for remote care in rheumatic and musculoskeletal diseases. Ann Rheum Dis. 2022;8(1):e002290.
2. Najm A Nikiphorou E Kostine M Richez C Pauling JD Finckh A EULAR points to consider for the development, evaluation and implementation of mobile health applications aiding self-management in people living with rheumatic and musculoskeletal diseases RMD Open. 2019 5 2 e001014 10.1136/rmdopen-2019-001014 31565245
Najm A, Nikiphorou E, Kostine M, Richez C, Pauling JD, Finckh A, et al. EULAR points to consider for the development, evaluation and implementation of mobile health applications aiding self-management in people living with rheumatic and musculoskeletal diseases. RMD Open. 2019;5(2):e001014.31565245
3. Marques A, Bosch P, de Thurah A, Meissner Y, Falzon L, Mukhtyar C, et al. Effectiveness of remote care interventions: a systematic review informing the 2022 EULAR points to consider for remote care in rheumatic and musculoskeletal diseases. RMD Open. 2022;8(1):e002290.
4. Solomon DH Rudin RS Digital health technologies: opportunities and challenges in rheumatology Nat Rev Rheumatol 2020 16 9 525 35 10.1038/s41584-020-0461-x 32709998
Solomon DH, Rudin RS. Digital health technologies: opportunities and challenges in rheumatology. Nat Rev Rheumatol. 2020;16(9):525–35.32709998
5. de Thurah A Bremander A Primdahl J High-quality RMD rehabilitation and telehealth: evidence and clinical practice Best Pract Res Clin Rheumatol. 2020 34 2 101513 10.1016/j.berh.2020.101513 32307230
de Thurah A, Bremander A, Primdahl J. High-quality RMD rehabilitation and telehealth: evidence and clinical practice. Best Pract Res Clin Rheumatol. 2020;34(2):101513.32307230
6. Cronström A Dahlberg LE Nero H Ericson J Hammarlund CS I would never have done it if it hadn’t been digital’: a qualitative study on patients’ experiences of a digital management programme for hip and knee osteoarthritis in Sweden BMJ Open 2019 9 5 e028388 10.1136/bmjopen-2018-028388 31129601
Cronström A, Dahlberg LE, Nero H, Ericson J, Hammarlund CS. I would never have done it if it hadn’t been digital’: a qualitative study on patients’ experiences of a digital management programme for hip and knee osteoarthritis in Sweden. BMJ Open. 2019;9(5):e028388.31129601
7. Fernandes LG Devan H Fioratti I Kamper SJ Williams CM Saragiotto BT At my own pace, space, and place: a systematic review of qualitative studies of enablers and barriers to telehealth interventions for people with chronic pain Pain. 2022 163 2 e165 81 10.1097/j.pain.0000000000002364 34433776
Fernandes LG, Devan H, Fioratti I, Kamper SJ, Williams CM, Saragiotto BT. At my own pace, space, and place: a systematic review of qualitative studies of enablers and barriers to telehealth interventions for people with chronic pain. Pain. 2022;163(2):e165-81.34433776
8. Zuidema RM Van Dulmen S Nijhuis-van der Sanden MWG Fransen J Van Gaal BGI qLessons learned from patients with access to an online self-management enhancing program for RA patients: qualitative analysis of interviews alongside a randomized clinical trial Patient Educ Couns 2019 102 6 1170 7 10.1016/j.pec.2019.01.005 30638903
Zuidema RM, Van Dulmen S, Nijhuis-van der Sanden MWG, Fransen J, Van Gaal BGI. qLessons learned from patients with access to an online self-management enhancing program for RA patients: qualitative analysis of interviews alongside a randomized clinical trial. Patient Educ Couns. 2019;102(6):1170–7.30638903
9. Muehlensiepen F, Knitza J, Marquardt W, May S, Krusche M, Hueber A et al. Opportunities and barriers of Telemedicine in Rheumatology: a participatory, mixed-methods study. Int J Environ Res Public Health. 2021;18(24).
10. Knudsen LR Lomborg K Hauge EM Zangi HA de Thurah A The WebRA study: opportunities and challenges in digital patient education from the perspective of patients with rheumatoid arthritis: a qualitative study Patient Educ Couns. 2023 116 107969 10.1016/j.pec.2023.107969 37672918
Knudsen LR, Lomborg K, Hauge EM, Zangi HA, de Thurah A. The WebRA study: opportunities and challenges in digital patient education from the perspective of patients with rheumatoid arthritis: a qualitative study. Patient Educ Couns. 2023;116:107969.37672918
11. Muehlensiepen F Knitza J Marquardt W Engler J Hueber A Welcker M Acceptance of Telerheumatology by rheumatologists and General Practitioners in Germany: Nationwide cross-sectional survey study J Med Internet Res. 2021 23 3 e23742 10.2196/23742 33690147
Muehlensiepen F, Knitza J, Marquardt W, Engler J, Hueber A, Welcker M. Acceptance of Telerheumatology by rheumatologists and General Practitioners in Germany: Nationwide cross-sectional survey study. J Med Internet Res. 2021;23(3):e23742.33690147
12. Lawford BJ Delany C Bennell KL Hinman RS I was really pleasantly surprised: Firsthand Experience and shifts in physical therapist perceptions of telephone-delivered Exercise therapy for knee Osteoarthritis-A qualitative study Arthritis Care Res (Hoboken). 2019 71 4 545 57 10.1002/acr.23618 29885026
Lawford BJ, Delany C, Bennell KL, Hinman RS. I was really pleasantly surprised: Firsthand Experience and shifts in physical therapist perceptions of telephone-delivered Exercise therapy for knee Osteoarthritis-A qualitative study. Arthritis Care Res (Hoboken). 2019;71(4):545–57.29885026
13. Knudsen LR, Ndosi M, Hauge EM, Lomborg K, Dreyer L, Aaboe S, et al. Effectiveness of a novel digital patient education programme to support self-management of early rheumatoid arthritis: a randomised controlled trial. Rheumatology (Oxford). 2024;63(9):2547-56.
14. Knudsen LR Lomborg K de Thurah A Design and development of an e-learning patient education program for self-management support in patients with rheumatoid arthritis PEC Innov. 2022 1 100004 10.1016/j.pecinn.2021.100004 37364010
Knudsen LR, Lomborg K, de Thurah A. Design and development of an e-learning patient education program for self-management support in patients with rheumatoid arthritis. PEC Innov. 2022;1:100004.37364010
15. Abimbola S Patel B Peiris D Patel A Harris M Usherwood T Greenhalgh T The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme BMC Med 2019 17 1 233 10.1186/s12916-019-1463-x 31888718
Abimbola S, Patel B, Peiris D, Patel A, Harris M, Usherwood T, Greenhalgh T. The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme. BMC Med. 2019;17(1):233.31888718
16. Greenhalgh T Abimbola S The NASSS Framework - A synthesis of multiple theories of technology implementation Stud Health Technol Inf. 2019 263 193 204
Greenhalgh T, Abimbola S. The NASSS Framework - A synthesis of multiple theories of technology implementation. Stud Health Technol Inf. 2019;263:193–204.
17. Greenhalgh T Wherton J Papoutsi C Lynch J Hughes G A’Court C Beyond adoption: a New Framework for Theorizing and evaluating nonadoption, abandonment, and challenges to the Scale-Up, Spread, and sustainability of Health and Care technologies J Med Internet Res. 2017 19 11 e367 10.2196/jmir.8775 29092808
Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: a New Framework for Theorizing and evaluating nonadoption, abandonment, and challenges to the Scale-Up, Spread, and sustainability of Health and Care technologies. J Med Internet Res. 2017;19(11):e367.29092808
18. Greenhalgh T. How to improve success of technology projects in health and social care. Public Health Res Pract. 2018;28(3).
19. Greenhalgh T Maylor H Shaw S Wherton J Papoutsi C Betton V The NASSS-CAT tools for understanding, guiding, monitoring, and Researching Technology Implementation Projects in Health and Social Care: protocol for an evaluation study in real-world settings JMIR Res Protoc 2020 9 5 e16861 10.2196/16861 32401224
Greenhalgh T, Maylor H, Shaw S, Wherton J, Papoutsi C, Betton V, et al. The NASSS-CAT tools for understanding, guiding, monitoring, and Researching Technology Implementation Projects in Health and Social Care: protocol for an evaluation study in real-world settings. JMIR Res Protoc. 2020;9(5):e16861.32401224
20. Tong A Sainsbury P Craig J Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups Int J Qual Health Care. 2007 19 6 349 57 10.1093/intqhc/mzm042 17872937
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.17872937
21. Elo S Kyngäs H The qualitative content analysis process J Adv Nurs 2008 62 1 107 15 10.1111/j.1365-2648.2007.04569.x 18352969
Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.18352969
22. QSR International Pty Ltd. (2018) Nvivo (Version 12), Available from: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home.
23. Paterson BL The shifting perspectives model of chronic illness J Nurs Scholarsh. 2001 33 1 21 6 10.1111/j.1547-5069.2001.00021.x 11253576
Paterson BL. The shifting perspectives model of chronic illness. J Nurs Scholarsh. 2001;33(1):21–6.11253576
24. Aletaha D Smolen JS Diagnosis and management of rheumatoid arthritis: a review JAMA. 2018 320 13 1360 72 10.1001/jama.2018.13103 30285183
Aletaha D, Smolen JS. Diagnosis and management of rheumatoid arthritis: a review. JAMA. 2018;320(13):1360–72.30285183
25. Organization WH. Rheumatoid Arthritis 2023. Available from: https://www.who.int/news-room/fact-sheets/detail/rheumatoid-arthritis.
26. Norman CD Skinner HA eHealth literacy: Essential Skills for Consumer Health in a Networked World J Med Internet Res 2006 8 2 e9 10.2196/jmir.8.2.e9 16867972
Norman CD, Skinner HA. eHealth literacy: Essential Skills for Consumer Health in a Networked World. J Med Internet Res. 2006;8(2):e9.16867972
27. Holt KA Karnoe A Overgaard D Nielsen SE Kayser L Røder ME From G Differences in the Level of Electronic Health Literacy between Users and Nonusers of Digital Health Services: an exploratory survey of a Group of Medical outpatients Interact J Med Res 2019 8 2 e8423 10.2196/ijmr.8423 30950809
Holt KA, Karnoe A, Overgaard D, Nielsen SE, Kayser L, Røder ME, From G. Differences in the Level of Electronic Health Literacy between Users and Nonusers of Digital Health Services: an exploratory survey of a Group of Medical outpatients. Interact J Med Res. 2019;8(2):e8423.30950809
28. Cheng C Beauchamp A Elsworth GR Osborne RH Applying the Electronic Health Literacy Lens: Systematic Review of Electronic Health Interventions Targeted at socially disadvantaged groups J Med Internet Res. 2020 22 8 e18476 10.2196/18476 32788144
Cheng C, Beauchamp A, Elsworth GR, Osborne RH. Applying the Electronic Health Literacy Lens: Systematic Review of Electronic Health Interventions Targeted at socially disadvantaged groups. J Med Internet Res. 2020;22(8):e18476.32788144
29. Ammenwerth E Technology Acceptance Models in Health Informatics: TAM and UTAUT Stud Health Technol Inf 2019 263 64 71
Ammenwerth E. Technology Acceptance Models in Health Informatics: TAM and UTAUT. Stud Health Technol Inf. 2019;263:64–71.
30. World Health Organization. WHO Guideline. Recommendations on digital interventions for health system strenghening - Evidence and Recommendations. 2019. Available from: https://apps.who.int/iris/bitstream/handle/10665/311980/WHO-RHR-19.10-eng.pdf?sequence=1&isAllowed=y.
31. World Health Organization. Telemedicine - Opportunities and developments in Member States. 2010. Available from: https://iris.who.int/handle/10665/44497?locale-attribute=en&show=full.
32. World Health Organization. Ethics and governance of artificial intelligence for health. 2021. Availble from: https://iris.who.int/bitstream/handle/10665/341996/9789240029200-eng.pdf?sequence=1.
33. Nilsen P Seing I Ericsson C Birken SA Schildmeijer K Characteristics of successful changes in health care organizations: an interview study with physicians, registered nurses and assistant nurses BMC Health Serv Res 2020 20 1 147 10.1186/s12913-020-4999-8 32106847
Nilsen P, Seing I, Ericsson C, Birken SA, Schildmeijer K. Characteristics of successful changes in health care organizations: an interview study with physicians, registered nurses and assistant nurses. BMC Health Serv Res. 2020;20(1):147.32106847
34. Lorig KR Ritter PL Laurent DD Plant K The internet-based arthritis self-management program: a one-year randomized trial for patients with arthritis or fibromyalgia Arthritis Rheum 2008 59 7 1009 17 10.1002/art.23817 18576310
Lorig KR, Ritter PL, Laurent DD, Plant K. The internet-based arthritis self-management program: a one-year randomized trial for patients with arthritis or fibromyalgia. Arthritis Rheum. 2008;59(7):1009–17.18576310
35. Allam A Kostova Z Nakamoto K Schulz PJ The effect of social support features and gamification on a web-based intervention for rheumatoid arthritis patients: randomized controlled trial J Med Internet Res 2015 17 1 e14 10.2196/jmir.3510 25574939
Allam A, Kostova Z, Nakamoto K, Schulz PJ. The effect of social support features and gamification on a web-based intervention for rheumatoid arthritis patients: randomized controlled trial. J Med Internet Res. 2015;17(1):e14.25574939
36. Pers YM Valsecchi V Mura T Aouinti S Filippi N Marouen S A randomized prospective open-label controlled trial comparing the performance of a connected monitoring interface versus physical routine monitoring in patients with rheumatoid arthritis Rheumatology (Oxford). 2021 60 4 1659 68 10.1093/rheumatology/keaa462 33020846
Pers YM, Valsecchi V, Mura T, Aouinti S, Filippi N, Marouen S, et al. A randomized prospective open-label controlled trial comparing the performance of a connected monitoring interface versus physical routine monitoring in patients with rheumatoid arthritis. Rheumatology (Oxford). 2021;60(4):1659–68.33020846
37. Health MotIa. Bekendtgørelse af lov om videnskabsetisk behandling af sundhedsvidenskabelige forskningsprojekter, § 14, stk. 2. 2017. Available from: https://www.retsinformation.dk/eli/lta/2017/1083#P14.
