
==== Front
BMC Health Serv Res
BMC Health Serv Res
BMC Health Services Research
1472-6963
BioMed Central London

39285300
11583
10.1186/s12913-024-11583-y
Study Protocol
Identification of emergencies in the telephone queue and routing to a fast track (FAST): study protocol for a prospective, two-armed cohort study
Eichler Sarah seichler@zi.de

1
Herrmann Tobias 1
Weidlich-Wichmann Uta 2
Vissiennon Kodjo 1
Pollmann Thorsten 2
Weller Lisa 2
Pommerenke Christopher 1
Kroll Lars 1
Alix Nicolas 1
Dietsch Tanja 2
von Stillfried Dominik 1
Carnarius Sebastian 1
1 grid.439300.d Central Research Institute of Ambulatory Health Care in Germany, Berlin, Germany
2 aQua-Institute - Institute for Applied Quality Improvement and Research in Health Care, Göttingen, Germany
17 9 2024
17 9 2024
2024
24 107919 2 2024
12 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Background

In Germany, the telephone patient service 116,117 for callers with non-life-threatening health issues is available 24/7. Based on structured initial assessment, urgency and placement of suitable medical care offer have been offered since 2020. The service has been in increasing demand for several years: Depending on time and residence, this can result in longer waiting times.

Methods

Prospective, two-armed cohort study with two intervention groups and one control group, alternating between blinding and unblinding for employees of 116,117 regarding prioritization status. Two interventions based on automated voice dialogues (1: Simple self-rating tool, 2: Automated brief query of emergency symptoms). In case of high level of urgency, callers are prioritized. Validation of urgency and need for care is carried out routinely based on structured initial assessment.

Discussion

By creating and providing a largely reproducible documentation of the implemented solutions for a waiting queue management, the developed approach would be available for comparable projects in the German health care system or in the European context. This potentially leads to a reduction in the use of resources in the development of comparable technical solutions based on automated voice dialogs.

Trial registration

DRKS00031235, registered on 10th November 2023, https://drks.de/search/de/trial/DRKS00031235.

Keywords

Telephone patient service
Waiting queue management
SmED
Structural initial assessment
Fast track
issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcBackground

In Germany, due to the Appointment Service and Supply Act (Terminservice- und Versorgungsgesetz, TSVG) from the 1st of January 2020 on, the Associations of Statutory Health Insurance Physicians were obliged to be available 24/7 on the telephone number 116,117 for acute complaints and to provide callers a suitable offer of care after carrying out a structured initial assessment. Based on a guideline, issued by the National Association of Statutory Health Insurance Physicians (Kassenärztliche Bundesvereinigung, KBV), all 17 Associations of Statutory Health Insurance Physicians have introduced the software SmED (Strukturierte medizinische Ersteinschätzung in Deutschland, Strucured medical initial assessment in Germany) to support the decision-making of their specialist staff. With the help of the structured initial assessment, patients can be directed reliably to the right level of care at the right time.

From 2019 to 2020, the number of 116,117 calls temporarily rose sharply from 8 to 17 million – also due to an increased demand within the Covid-19 pandemic. Despite best possible planning, occupancy peaks mean that acute patients are sometimes confronted with longer waiting times at the service number. This is particularly problematic for people who potentially need urgent care. In Germany, callers can distinguish between the emergency number 112 and 116,117 for acute needs. Only a small proportion of about 3–5% [1] of callers on the 116,117 are classified as an obvious emergency according to initial assessment. Nonetheless, rescue coordination centres report that the number 112 is increasingly used by particularly worried callers in times of poorer accessibility of the number 116,117 due to long waiting times.

Initial studies indicate that an effective queue management is possible for similar service numbers. If urgent requests for help can be prioritized more quickly and precisely, this has a positive effect on the patient safety of the 116,117’s mediation function. If this also reduces the use of 112 by urgent acute cases that do not require emergency care, the efficiency of 112 and emergency care can also be improved. So far, it is unclear which method can be used to detect possible emergencies and very urgent acute cases in the telephone queue of the 116,117 as sensitively and specifically as possible to prioritize them for a structured initial assessment.

Danish studies show that a simple self-assessment (emergency button, self-rated urgency, degree of worry) can provide valuable clues to the urgency of the medical complaint and that abuse was negligible in the study context [2–6]. Based on previous experience from other European countries, we want to clarify whether efficiency advantages can also be expected in the German context through a simple self-assessment, e.g. by asking whether the concern is particularly urgent [7], or whether this creates incentives for abuse. Further, it will be examined whether the use of an automated short query of selected emergency symptoms or urgency characteristics can be used to identify potentially urgent cases more sensitively or specifically and thus to respond more specifically to the concerns of those seeking help.

Hence, the aim of the project is to create and implement suitable (technical) methods for the identification and prioritization of callers with potentially very urgent treatment needs in a telephone queue. With this project, we want to improve patient safety in 116,117 and the efficiency of acute and emergency care. In addition, the mediation function for those seeking help with urgently needed emergency or acute care is to be improved and the use of 112 by acute cases is to be reduced.

Methods/Design

The aim of the study is the implementation and evaluation of a telephone queue management for prioritizing urgent cases by investigating the following hypotheses:The interventions identify a high number of cases with very urgent treatment needs (high sensitivity) and lead to a low proportion of incorrectly prioritized calls (high specificity). [Primary Outcomes]

Waiting times for very urgent cases can be reduced with the interventions when compared to the control group.

The cancellation rate of calls and thus also the use of the emergency number 112 can be reduced with the interventions when compared to the control group.

The queue management tools used in the intervention groups are easy to use for callers.

Based on routine data, the external validity of the disposition decisions is confirmed.

The automated brief query of emergency symptoms (intervention 2) is superior to the simple self-rating tool (intervention 1) regarding the investigated parameters (hypotheses 1–5).

The study is conducted in a stepped wedge design. Accordingly, it is a cluster partially randomized study. The individual telephone centers of the Associations of Statutory Health Insurance Physicians are regarded as clusters. Allocation to the intervention or control group is not randomized to ensure comparability of clusters (e.g. according to the number of calls).

A distinction is made between two intervention groups and a control group, whereby all clusters will start in the control group and switch gradually to one of the intervention groups during intervention phase.

In the first testing period, the interventions are tested without prioritization. After this, the interventions in period two (nine months) including prioritization of urgent calls (fast track) start.

Study setting

The trial is conducted in 9 out of 17 Associations of Statutory Health Insurance Physicians in Germany. The interventions are implemented into the waiting queues of their telephone patient services 116,117.

Eligibility criteria

The inclusion and exclusion criteria for participants are defined as follows:

Inclusion criteria

Callers of the telephone patient service 116,117 calling because of urgent medical complaints.

18 years and older.

All sexes.

Knowledge of the German language.

Exclusion criteria

Callers of the telephone patient service 116,117 under 18 years of age.

Callers using the service for different reasons (e.g. booking appointments in non-urgent cases).

Intervention description

One of the two interventions is implemented in each telephone patient service 116,117 of the participating Associations of Statutory Health Insurance Physicians for 9 months. The intervention is switched on, if the waiting time within the waiting queue of three minutes or more is expected. Both interventions are based on automated voice dialogues. Intervention 1 is a simple self-rating tool with one statement, where the caller defines its urgency/severity by pressing one single button. Intervention 2 is an automated brief query of emergency symptoms (six questions/statements with a possibility to be prioritized after each symptom). The validation of the urgency and the need of care is routinely carried out by employees of 116,117 using a structured initial assessment.

Participants calling the 116,117 of the participating Associations of Statutory Health Insurance Physicians, where the interventions are not implemented yet according to the stepped wedge design in order to ensure comparability of the data.

Outcomes

The test quality of the intervention tools (Intervention 1, Intervention 2) will be used as primary outcomes by investigating sensitivity (true-positive rate) and specificity (true-negative rate). The golden standard for urgency is the decision of the dispatchers after the structured initial assessment. Sensitivity is calculated as the quotient of true-positive decisions (i.e. urgent calls classified correctly by the intervention) and the sum of true-positive and false-negative decisions (i.e. the sum of urgent calls classified by the dispatchers). Specificity is defined as the quotient of true-negative decisions (i.e. all non-urgent calls classified correctly by the intervention) and the sum of true-negative and false-positive decisions (i.e. the sum of non-urgent calls classified by the dispatchers).

Three secondary outcomes will be examined: waiting time, cancellation rate and usability.

The waiting time and the cancellation rate will be calculated depending on data availability of the respective Associations of Statutory Health Insurance Physicians for both intervention groups and the control group, and the usability for both intervention groups accordingly. The waiting time is defined as the average total waiting time of all callers (per group) until a caller can talk to a dispatcher. The cancellation rate is calculated as the sum of all abandoned calls (i.e. calls that ended before a caller was able to talk to a dispatcher) divided by the total number of calls (per group).

The usability is examined as a part of the survey of the 116,117 callers. Usability is understood as “degree to which a person believes that using [the intervention tool] will be free of effort” [8]. Callers of the 116,117 are asked to rate the speed of the intervention, content comprehension, and the preference for voice input on a five-point Likert scale from 1 “disagree” to 5 “strongly agree”.

Sample size

The calculation of the necessary sample size is based on hypothesis 6 (superiority of the automated short query of emergency symptoms compared to the simple self-rating instrument), because it requires the largest sample.

Overall, the participating Associations of Statutory Health Insurance Physicians receive 2.5 million calls per year (extrapolation of the National Association of Statutory Health Insurance Physicians call data). The callers should be evenly distributed among the three groups. With a power of 80% and a significance level of 5%, it is possible to detect whether the sensitivity is intervention 2 is at least 1% point better than intervention 1 (hypothesis 6); multiple testing [9] results in a sample of N = 470,933 calls needed. In the smallest participating Association of Statutory Health Insurance Physicians, 12,586 calls are expected.

Thus, there could still be a difference in the sensitivity of the (effect size) of 6.1% points. This sample size is also sufficient to improve the test quality in real operation (hypothesis 1) and to reduce the waiting time (hypothesis 2) to verify.

Recruitment

All persons (meeting the inclusion criteria) calling the 116,117 for an emergency assessment in a participating Association of Statutory Health Insurance Physicians region, where the intervention is implemented (see study setting), are requested to participate. Once the current expected waiting time for callers exceeds the threshold of three minutes, callers will have the opportunity to participate in the study and go through one of the two interventions. If they consent to participate in the study, the callers will automatically go through the respective intervention.

For the subsequent survey of the 116,117 callers, a computer generated sample (n = 24,000), which is stratified disproportionately according to the characteristics of the groups (intervention 1/intervention 2/control group) and the prioritization within the interventions (prioritized/not prioritized), is drawn from all callers. Within the survey, the selected persons will also be asked to consent to the transfer of their routine data, if they are insured by the three health insurances involved in the project.

Recruitment begins on the 1st April 2024 and ends on the 31st December 2024.

Allocation

Based on the stepped wedge design:Participation in intervention◦ Full collection in intervention groups: all callers in a participating Association of Statutory Health Insurance Physicians with intervention who meet the inclusion criteria and agree to participate.

◦ Full collection in control group: all callers in a control Association of Statutory Health Insurance Physicians who meet the inclusion criteria.

Survey (24,000 questionnaires):◦ Sampling with disproportionate stratification according to the following characteristics:▪ Group affiliation (Intervention 1/Intervention 2/Control)

▪ Prioritization within the intervention (not prioritized/prioritized)

Analysis of routine data◦ Subsample survey participants: Questionnaire response from persons of the health insurance involved in the project (written consent)

Blinding

The dispatchers will be blinded in the way whether callers are prioritized or non-prioritized.

Data collection and management

There are three data segments: first, the data that is routinely collected during the 116,117-call by the individual telephone centers of the Associations of Statutory Health Insurance Physicians. Second, the data collected within the patient survey and third, the data collected by health insurance companies.

Data collected during the 116,117 call will be available for all participating callers. Data of the patient survey and health insurance will only be collected for a subset of callers. More precisely, for the second period of the intervention phase (April 2024 to December 2024), names and address data as well as information on (non-)participation in the intervention of all 116,117 callers will be sent to the project’s trust center DMATC (Data Management and Trust Center GmbH) by all Associations of Statutory Health Insurance Physicians in the following month. This is the basis for the 116,117 callers survey, which will be sent to a subset of the callers.

The survey primarily collects information on usability - but also on compliance with the recommendations. While answering the questionnaire, participants can fill out a declaration of consent that their health insurance company (if they are insured at one of the three participating health insurances) will make their data on the utilization of emergency care available to the aQua Institute as evaluator. The data collection and evaluation project were reviewed by the responsible supervisory authorities in Germany.

The Associations of Statutory Health Insurance Physicians generate a monthly data set of all 116,117 callers. A project ID is assigned to every case. The Associations of Statutory Health Insurance Physicians submit the data set to the trust office (DMATC) via a secure transmission channel, which draws a sample from all data sets per month. All selected persons will receive a questionnaire by post with study information and a declaration of consent regarding the routine data of the health insurances. The survey participants return the declaration of consent to the trust office, which documents and stores the consents digitally. The questionnaire itself will be sent to the evaluation centre by the study participants. After receiving the documents, the project ID and the questionnaire will be digitized by the evaluation office. After the intervention phase, the Associations of Statutory Health Insurance Physicians provide data from all callers meeting the selection criteria to the evaluation centre once. The trust office sends the assignment list to the respective health insurance via a state-of-the-art secure channel. The health insurances provide the project ID and their routine data to the data collection point via a secure and encrypted method, which checks the data and, after approval, creates a general data record. Afterwards, this pseudonymised data set is transferred to the evaluation office. After data processing, the 116,117 data, survey data and routine data are stored for ten years in accordance with German Law (§ 75 para. 4 sentence 6, SGB X) and Good Practice of Secondary Data Analysis [10].

The Associations of Statutory Health Insurance Physicians assign a study pseudonym (FAST-ID) to all callers meeting the selection criteria, which allows the data from the various sources to be linked. For this purpose, a separate identifier for this project is defined and placed as a prefix at the beginning of the FAST-ID for each participating Association of Statutory Health Insurance Physicians. The case number itself will be assigned in numerical sequence and will be unique for each case throughout the duration of the project.

All callers are thus assigned a unique identifier, which is stored together with the address data and transmitted to the trust office and stored there for a specified period. Study participants can obtain the deletion of their data by stating their study pseudonym.

Statistical methods

Initially, a descriptive analysis of all outcomes will be conducted by calculating frequencies, point estimators, and measures of variation, where appropriate stratified by trial arms or other group variables. For the two primary outcomes, sensitivity and specificity, two-sample tests for binomial proportions (chi-square tests) will be calculated to examine whether the two interventions differ from each other significantly.

The secondary outcomes waiting time and cancellation rate will be compared using a one-way analysis of variance (ANOVA) with the levels intervention 1, intervention 2 and control group. In case of a significant effect, t-tests for independent samples will be calculated for pairwise differences. In addition and depending on statistical requirements (e.g. distributional assumptions), suitable methods like linear regression or analysis of covariance (ANCOVA) are used to adjust for possible confounding variables that could influence the waiting time and cancellation rate (region, day of week, time of day etc.). In terms of the secondary outcome usability, the related questionnaire item will be analyzed by using a non-parametric test (e.g. chi-square-test) to check for independence between intervention group 1 and 2. All mentioned tests will be performed at the significance level of 5%, corrected for multiple testing if necessary.

The routine data from the three participating health insurances will be used to investigate the course of treatment after calling 116,117 exploratively and thus the external validity of the disposition decision. For this purpose, data from the treatment level (patient transport, hospital, outpatient treatment) as well as documented diagnoses are used and linked to other data sources. In this way, insights into the coherence between the 116,117 callers in the context of queue management, initial assessment of the dispatchers and follow-up care are possible. Other sources are data of the 116,117 calls (including date, length, disposition decision, health information, prioritization in the fast track), as well as data from a follow-up survey of the callers (satisfaction with the call and, if applicable, the fast track routing).

To test hypothesis 3, aggregated anonymized call numbers from local emergency services are used for the exploratory and descriptive analysis of spill-over effects to the number 112 due to waiting times in the service number 116,117.

If data from certain sections are missing (116117 data, data from the follow-up survey, health insurance routine data), the persons are excluded from the respective analysis, but are included in all other analyses for which this section is not relevant. If single variables are missing, the person will be excluded from the analysis in the respective analysis; there will be no data imputation.

Oversight and monitoring

The Central Research Institute of Ambulatory Health Care in Germany is the consortium leader of the project with its 13 partners (nine Associations of Statutory Health Insurance Physicians, three health insurances and aQua institute as the evaluating partner) and functions as the coordinating centre and trial steering committee. The whole project team will meet every six months. Further, there are weekly meetings with the data monitoring committee. The additional responsibilities lie with the constitution and constant project management for the whole project duration, as well as creating the proposals for the relevant supervisory authorities and ethics committee together with the aQua institute. The Central Research Institute is also responsible for the conception and identification of the intervention tools, as well as the implementation of the technical solutions (voicebot) and the integration of the tools into the Associations of Statutory Health Insurance Physicians’ queue management. Further, the aQua institute is supported in the evaluation and interpretation of the results.

The aQua Institute is responsible for the conception and implementation of the evaluation. For this purpose, the aQua Institute develops survey instruments and conducts the survey with 116,117 callers about their experiences with the interventions. In addition, the aQua-Institute analyses routine data from the health insurances and data from the 116,117 with information from the survey of 116,117 callers, who were linked to each other in advance via a study pseudonym.

DMATC is a wholly owned subsidiary of the aQua Institute and acts as an independent trust office in the project. In addition to the sampling for the survey and the postal dispatch of the questionnaires to 116,117 callers, the DMATC receives the consent for the transmission and processing of the routine data, creates separate assignment lists and makes them available to the participating health insurance companies.

Results will be presented at national and international health care congresses and published in journals.

Discussion

With this study, we create and evaluate reliable prioritization possibilities of acute and emergency patients in potentially urgent care in telephone queues, which can be applied beyond the 116,117 in other areas of the national and international healthcare system.

The use of 116,117 in Germany is increasing and the initial telephone contact with the care system could become common practice in the future. In addition, a large number of telemedical care models are to be expected, in which access to care is not primarily provided by a trained specialist, but via telephone services or digital access. Capacity bottlenecks might occur on a regular basis. In these cases, experiences with how potentially urgent concerns can be identified quickly and reliably and prioritized in the case of a non-specific population of people seeking help has been made in the Danish studies so far [2–6].

The FAST project aims to collect, evaluate and report experiences using 116,117 service as example, so that it can be routinely used in the 116,117 context and can also be transferred to other telephone hotlines or telemedical services in the healthcare sector. Examples include the hotlines of health insurances, Associations of Statutory Health Insurance Physicians, hospitals or the public health service. Studies from Denmark show that there may also be an international need to draw on experience from the project. In addition, using these tools in waiting areas of real or telemedical care services could be possible. This potentially leads to a reduction in the use of resources in the development of comparable technical solutions based on automated voice dialogs.

Abbreviations

ANCOVA Analysis of covariance

ANOVA Analysis of variance

DRKS German Clinical Trials Register (Deutsches Register klinischer Studien)

DMATC Data Management and Trust Center GmbH

G BA-Federal Joint Committee (Gemeinsamer Bundesausschuss)

KBV National Association of Statutory Health Insurance Physicians (Kassenärztliche Bundesvereinigung)

SmED Structured medical initial assessment (Strukturierte medizinische Ersteinschätzung in Deutschland)

TSVG Appointment Service and Supply Act (Terminservice-und Versorgungsgesetz)

Acknowledgements

We acknowledge the support of the Federal Joint Committee (G-BA) as well as the callers who applied the study.

Authors’ contributions

DvS, SC, TH, TP, LK designed the study. SE, UW-W, KV, CP, LW, TD and NA contributed significantly to the study design. SE and TH wrote the first draft of the manuscript and managed the co-ordination of the study with SC. TH was the lead trial methodologist. SC, LK, TH, UW-W, KV, TP, LW, TD, CP and NA revised the manuscript and wrote the final draft. All authors read and approved the final version of the manuscript.

Funding

Financial funding by the Federal Joint Committee (G-BA), Germany: 01VSF22028. The funder plays no part in the study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication.

Availability of data and materials

The datasets used and/or analyzed during the current study will be available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The study and protocol are developed according to the Declaration of Helsinki guidelines and Good Clinical Practice (International Conference of Harmonization). Informed consent will be obtained from the participants. Informed consent and participation are asked by the voicebot via telephone; the patients also have the possibility to reject participation. Callers are asked to participate in the intervention and potential non-statement is counted as “no”. Since callers are potential acute and emergency cases, we refrain from a question on data evaluation, because a further delay after the conclusion of the telephone call due to extensive clarification would be disproportionate and ethically unjustifiable.

The study protocol was approved by the Ethics Committee of the Berlin Medical Association (Eth.-32/23, 09th November 2023) and registered at the German Clinical Trials Register (DRKS00031235, 10th November 2023); https://drks.de/search/de/trial/DRKS00031235.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Sarah Eichler and Tobias Herrmann contributed equally to this work.
==== Refs
References

1. https:/. /smed.ziapp.de. Accessed 13 Feb 2024.
2. Ebert JF Huibers L Christensen B Lippert FK Christensen MB Do callers to out-of-hours care misuse an option to jump the phone queue? Scand J Prim Health Care 2019 37 2 207 17 31070507
Ebert JF, Huibers L, Christensen B, Lippert FK, Christensen MB. Do callers to out-of-hours care misuse an option to jump the phone queue? Scand J Prim Health Care. 2019;37(2):207–17.31070507
3. Ebert JF Huibers L Christensen B Lippert FK Christensen MB Giving callers the option to bypass the telephone waiting line in out-of-hours services: a comparative intervention study Scand J Prim Health Care 2019 37 1 120 7 30712448
Ebert JF, Huibers L, Christensen B, Lippert FK, Christensen MB. Giving callers the option to bypass the telephone waiting line in out-of-hours services: a comparative intervention study. Scand J Prim Health Care. 2019;37(1):120–7.30712448
4. Gamst-Jensen H Frischknecht Christensen E Lippert F Folke F Egerod I Huibers L Self-rated worry is associated with hospital admission in out-of-hours telephone triage - a prospective cohort study Scand J Trauma Resusc Emerg Med 2020 28 1 53 32522240
Gamst-Jensen H, Frischknecht Christensen E, Lippert F, Folke F, Egerod I, Huibers L, et al. Self-rated worry is associated with hospital admission in out-of-hours telephone triage - a prospective cohort study. Scand J Trauma Resusc Emerg Med. 2020;28(1):53.32522240
5. Gamst-Jensen H Frishknecht Christensen E Lippert F Folke F Egerod I Brabrand M Impact of caller’s degree-of-worry on triage response in out-of-hours telephone consultations: a randomized controlled trial Scand J Trauma Resusc Emerg Med 2019 27 1 44 30975160
Gamst-Jensen H, Frishknecht Christensen E, Lippert F, Folke F, Egerod I, Brabrand M, et al. Impact of caller’s degree-of-worry on triage response in out-of-hours telephone consultations: a randomized controlled trial. Scand J Trauma Resusc Emerg Med. 2019;27(1):44.30975160
6. Gamst-Jensen H Huibers L Pedersen K Christensen EF Ersboll AK Lippert FK Self-rated worry in acute care telephone triage: a mixed-methods study Br J Gen Pract 2018 68 668 e197 203 29440015
Gamst-Jensen H, Huibers L, Pedersen K, Christensen EF, Ersboll AK, Lippert FK, et al. Self-rated worry in acute care telephone triage: a mixed-methods study. Br J Gen Pract. 2018;68(668):e197–203.29440015
7. Ebert JF Huibers L Christensen B Collatz Christensen H Christensen MB Does an emergency access button increase the patients’ satisfaction and feeling of safety with the out-of-hours health services? A randomised controlled trial in Denmark BMJ Open 2020 10 9 e030267 32998912
Ebert JF, Huibers L, Christensen B, Collatz Christensen H, Christensen MB. Does an emergency access button increase the patients’ satisfaction and feeling of safety with the out-of-hours health services? A randomised controlled trial in Denmark. BMJ Open. 2020;10(9):e030267.32998912
8. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319-39.
9. Vickerstaff V Omar RZ Ambler G Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes BMC Med Res Methodol 2019 19 1 129 31226934
Vickerstaff V, Omar RZ, Ambler G. Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes. BMC Med Res Methodol. 2019;19(1):129.31226934
10. Swart E Gothe H Geyer S Jaunzeme J Maier B Grobe TG [Good Practice of Secondary Data Analysis (GPS): guidelines and recommendations] Gesundheitswesen 2015 77 2 120 6 25622207
Swart E, Gothe H, Geyer S, Jaunzeme J, Maier B, Grobe TG, et al. [Good Practice of Secondary Data Analysis (GPS): guidelines and recommendations]. Gesundheitswesen. 2015;77(2):120–6.25622207
