
==== Front
BMC Med Inform Decis Mak
BMC Med Inform Decis Mak
BMC Medical Informatics and Decision Making
1472-6947
BioMed Central London

2667
10.1186/s12911-024-02667-0
Research
Differences in changes of data completeness after the implementation of an electronic medical record in three surgical departments of a German hospital–a longitudinal comparative document analysis
http://orcid.org/0000-0001-5527-9238
Wurster Florian florian.wurster@uni-koeln.de

1
http://orcid.org/0000-0003-1986-7512
Herrmann Christin 1
http://orcid.org/0000-0002-9826-3575
Beckmann Marina 1
http://orcid.org/0000-0002-2548-5986
Cecon-Stabel Natalia 2
http://orcid.org/0000-0003-3390-1377
Dittmer Kerstin 1
http://orcid.org/0000-0001-6175-7670
Hansen Till 1
http://orcid.org/0000-0003-3424-2588
Jaschke Julia 3
http://orcid.org/0000-0002-3451-7847
Köberlein-Neu Juliane 3
http://orcid.org/0000-0002-9301-8829
Okumu Mi-Ran 1
http://orcid.org/0000-0001-9154-6575
Pfaff Holger 1
http://orcid.org/0000-0002-5383-4549
Rusniok Carsten 1
http://orcid.org/0000-0003-3479-9474
Karbach Ute 1
1 grid.6190.e 0000 0000 8580 3777 Faculty of Human Sciences, Faculty of Medicine and University Hospital Cologne, Institute of Medical Sociology, Health Services Research, and Rehabilitation Science, Chair of Quality Development and Evaluation in Rehabilitation, University of Cologne, Eupener Str. 129, 50933 Cologne, Germany
2 https://ror.org/024z2rq82 grid.411327.2 0000 0001 2176 9917 Medical Faculty, Unit of Child Health Services Research, Clinic of General Pediatrics, Neonatology and Pediatric Cardiology, University Hospital Düsseldorf, Heinrich-Heine University Düsseldorf, Moorenstr. 5, 40225 Düsseldorf, Germany
3 https://ror.org/00613ak93 grid.7787.f 0000 0001 2364 5811 Center for Health Economics and Health Services Research, University of Wuppertal, Rainer-Gruenter-Str. 21, 42119 Wuppertal, Germany
16 9 2024
16 9 2024
2024
24 2586 6 2024
5 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by/4.0/ Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Purpose

The European health data space promises an efficient environment for research and policy-making. However, this data space is dependent on high data quality. The implementation of electronic medical record systems has a positive impact on data quality, but improvements are not consistent across empirical studies. This study aims to analyze differences in the changes of data quality and to discuss these against distinct stages of the electronic medical record’s adoption process.

Methods

Paper-based and electronic medical records from three surgical departments were compared, assessing changes in data quality after the implementation of an electronic medical record system. Data quality was operationalized as completeness of documentation. Ten information that must be documented in both record types (e.g. vital signs) were coded as 1 if they were documented, otherwise as 0. Chi-Square-Tests were used to compare percentage completeness of these ten information and t-tests to compare mean completeness per record type.

Results

A total of N = 659 records were analyzed. Overall, the average completeness improved in the electronic medical record, with a change from 6.02 (SD = 1.88) to 7.2 (SD = 1.77). At the information level, eight information improved, one deteriorated and one remained unchanged. At the level of departments, changes in data quality show expected differences.

Conclusion

The study provides evidence that improvements in data quality could depend on the process how the electronic medical record is adopted in the affected department. Research is needed to further improve data quality through implementing new electronical medical record systems or updating existing ones.

Keywords

Electronic medical record
Secondary use
Data quality
Documentation
Completeness
Clinical adoption meta model
Hospital
Health Services research
Implementation
Germany
Universität zu Köln (1017)Open Access funding enabled and organized by Projekt DEAL.

issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcIntroduction

A digitized healthcare system offers various benefits to all stakeholders involved. There is evidence showing facilitation of working tasks for physicians and nurses [1], the reduction of costs for health care facilities and insurances [2] and an improved quality of care for patients [3]. The corresponding transformation from an analogue to a digital healthcare system has not only changed the way in which the delivery of care or interprofessional collaboration is organized [4]. The increasing digitalization has also a significant impact on how clinical information is assessed and managed. Hospitals are now recording the majority of their documentational tasks through digital platforms such as hospital information systems (HIS) or electronic medical records (EMR). These platforms allow information to be accessible to pre-defined authorized persons independent of time and place [5].

This development has led to an almost exponential growth in the amount of data available in near real time [6]. Even though this data serves the primary purpose of clinical practice and is essential for safety and quality of care [7], the secondary usage of such data sets opens up chances of delivering data-based personalized care [8] and assessing real world data for health services research purposes [9]. This is where the European health data space comes in. As a digital infrastructure, it aims to ensure the secure and interoperable collection of health data from across the European Union. At the patient level, individual disease predictions or optimal treatment options could be determined e.g. with regard to mental and cognitive disorders [10] or oncological care [11]. At the population level, complex causal relationships of pathogenic factors could be identified [12] and issues such as the inadequate care of vulnerable groups be tackled [13]. To be able to do this and avoid possible bias, the underlying datasets must be of high quality.

Data quality is defined as the data’s ability “to be used effectively, economically and rapidly” [14], or in other words, only “data that is fit for use” is sufficient data [15]. According to this wide definition, data quality is a multidimensional construct and there are numerous frameworks, depicting several dimensions of data quality for a wide variety of purposes [16–18]. Exemplary, for the context of health services research, the data’s conformance, completeness and plausibility are named as the most important dimensions of data quality [19], while for the clinical context, the completeness, legibility, accuracy and meaning are postulated as most important [20].

Completeness of documentation

Of the different dimensions of data quality, completeness can be seen as one of the most important one, since incompleteness is assumed as the most significant threat to data usability [21]. It poses a major risk to utilizing the described data for secondary purposes and leads to uncertainty when planning personalized care or conducting research [22]. An example of this is a decision support system whose functionality was examined with regard to its susceptibility to incomplete data. Run with insufficient completeness scores, the system made incorrect or even unsafe recommendations in up to 77% of the cases [23]. EMRs represent one of the largest sources of the digital data described [24]. It is therefore only consistent that completeness has been the most frequently investigated dimension of data quality regarding the documentation in EMRs since their emergence [22] and is still today [25]. If the concept of completeness is applied to EMRs, various conceptualizations of the completeness of these records emerge [26] (Table 1).

Table 1 Conceptualizations of Medical records completeness

Concept	Definition	
Documentation

Completeness

	Completeness is achieved when a record includes all clinical observations that were made during an encounter.	
Breadth

Completeness

	Completeness is achieved when a record includes all relevant data types for secondary use scenarios.	
Density

Completeness

	Completeness is achieved when a record includes a specified number of data points of one information over time.	
Predictive

Completeness

	Completeness is achieved when a record includes sufficient information for training models to predict future events.	

However, the reasons for incompleteness of these data are not fully understood yet [27] and explanatory approaches mention a variety of factors, e.g. of social or technical nature [15]. What has been identified in this matter is that the implementation of new EMR systems can have an influence on completeness [28]. The implementation of a new EMR system is even stated as one of the most important factors for increasing completeness [29]. However, improvements in completeness are not consistent across different empirical studies [28].

Implementation & adoption

This leads to the field of implementation science, in which it is well known that an “one size fits all” approach does not work when implementing digital interventions in healthcare [30]. It is therefore crucial to consider the context in which the implementation takes place [31]. Ignoring the context can even lead to adverse effects of the intervention intended to be beneficial [32] as also described for the implementation of EMR [28], where context can be interpreted e.g. as a technological, financial, cultural or organizational context [33]. However, even within the same context, different changes in data quality are observed [28]. Any differences in changing data quality across studies do therefore not arise exclusively from varying contexts, but could also be influenced by differing adoption processes at the respective departments. In the matter of EMRs, the adoption process is not a linear process with a certain endpoint (as usual). It can be rather seen as an iterative cycle, since EMRs are subject to a constant transformation in order to adapt technological progress (e.g. linking the EMR to a patient portal [34]) or reflect societal demands (e.g. enabling shared clinical notes [35]). This underlines the variety of possibilities to further improve data quality. But to do this, a more comprehensive understanding of influencing factors is required [36].

Research question

The study presented was part of the overarching eCoCo case study, which is described in detail by Beckmann et al. [37]. The main research question of the eCoCo case study was about possible effects of the EMR on social aspects of interprofessional collaboration and clinical workflows. The present study represents an independent work package within the eCoCo case study with a focus on possible changes in data quality. The completeness of documentation in paper-based medical records (PMR) (pre-intervention) is compared with those in electronic medical records (EMR) (post-intervention) across three surgical departments, aiming to analyze possible differences in the changes of data quality even within a similar context and to discuss possible differences in the changes of data quality against the background of distinct stages of the adoption process.

Methods

All analyses are based on anonymized data, provided by the departments’ hospital archive, complying with German and European data protection regulations. This includes redacting any patient’s socio-demographics from the records, as well as any information regarding the documenting individual. The eCoCo case study was approved by the ethics committee of the medical faculty of the university of cologne (20-1349) and is listed in the German national registry for clinical trials (DRKS00023343).

Subject of investigation

The study object for the present study was the fever chart document, which is a central part of every inpatient medical record. The fever chart contains basic information about the patient, like the body temperature levels and other vital signs or the experienced pain. The fever chart was selected as specific use case for the study presented because it is considered to be filled out in every inpatient medical record [38]. The hospital is replacing the printed fever chart document by an EMR system (Meona©) in which the fever chart has retained an almost identical format, implying that the information is documented in the same way in both record types (e.g. indicating the pain on a scale of 0–10). Since the various information can be documented in a different frequency, depending on the clinical relevance up to several times a day, the breadth of completeness was chosen as the most appropriate concept of the records completeness. As described, the breadth of completeness means that completeness is achieved when a record includes all relevant data types for secondary use scenarios (Table 1). For the present study’s analysis, all information that has to be recorded as an initial assessment at the day of admission and is documented in both record types was therefore collected for the statistical analysis. This approach means that the completeness of the following ten items was analyzed: Admission Diagnosis, Blood Pressure, Body Temperature, Code Status, Diet, Excretions, Height, Pain, Pulse, Weight. This selection mechanism, based on the parallel elements in the record types to be compared, is described as best practice [22] and has been proven in empirical studies [39]. As the data is stored in the archives after the patients have been discharged, any subsequent documentation is consistently assessed in all records, even if the documentation was put in after the day of admission.

Setting & record sampling

Three surgical departments were included in the study, implying comparable contexts (at least more comparable than comparing surgical with conservative specialities). The three surgical departments included in the study are each representing individual clinics within one large academic teaching hospital in Germany (orthopaedic surgery, cardiac surgery and abdominal surgery). By trying to equate the context, presumed differences in the change in data quality are more likely to be explained by the department-specific adoption processes, rather than organizational context. With regard to the sampling of the analyzed records, records from all three departments were collected, each department contributing both paper-based and electronical records. The records were taken from a three-week period before the implementation process started and a three-week period after the implementation process started, corresponding to all inpatients treated within those periods and departments. German hospitals are required by law to publish publicly available quality reports. Due to the complete anonymization of the individual medical records included, a classic sample description is not possible. The quality reports are therefore used to describe the study sample at department level and include information e.g. on the most frequently treated ICD-codes or the number of beds or working staff. The quality reports describe each department in its entirety on a mean annual basis.

Data analysis

For each record included, all of the ten items were coded as 1 if they were documented on the day of admission, otherwise they were coded as 0. This resulted in a percentage completeness of each item per record type, e.g. that blood pressure was documented in 80% of the paper-based records. In addition, this approach gives a mean completeness per record type, for example that an average of 6 of the 10 items were documented in the electronic records. The percentage completeness of the items in both record types was compared using chi square tests. The mean completeness of the two record types was compared using unpaired t-test. The calculations were made for both the department’s internal pre-post comparison as well as the pooled overall comparison above the three departments. A P-value < 0.05 was considered statistically significant for all calculations. The datasets of the three departments were collected in MS Excel and analyzed in SPSS 29 in December 2023.

Results

Data base

The completeness of a total of Ntotal=659 records was assessed. Of these, the documentation of Npaper=248 PMRs and Neletronic=411 EMRs was analyzed. The detailed distribution of the records by record type, period and department is shown in Table 2.

Table 2 Record distribution

	Orthopaedic Surgery	Abdominal Surgery	Cardiac Surgery	Overall	
Record Type	PMR	EMR	PMR	EMR	PMR	EMR	PMR	EMR	
Period	11/2020	08/2022	05/2019	05/2020	07/2020	07/2021	05/2019–11/2020	05/2020–08/2022	
N	44	136	95	139	109	136	248	411	
Go Live	May 2021	June 2019	August 2020	June 2019 – May 2021	

For the departments of cardiac and abdominal surgery, the beginning of the implementation process of the EMR took place right after the period in which the PMRs were collected, with the data collection period for the EMRs being 12 months after the beginning of implementation process. For the department of orthopaedic surgery, the Go Live as well as the second data collection period were delayed due to the COVID-19 pandemic, so that the data collection period for the PMRs was 6 months before the beginning of the implementation process and the data collection period for the EMRs was 15 months after the beginning of the implementation process.

Setting

Due to the lack of personal data in the analyzed records, the quality reports of the departments are used as a rough description of the study sample. The following information are reported: the departments bed capacity and number of inpatient cases, the total number of physicians and number of nursing staff. Moreover, the top 5 most treated diagnoses (ICD-10) are presented for each department and period.

The numbers of inpatient cases vary from a minimum of N = 1787 treated inpatients in the paper-based period on the cardiac department to a maximum of N = 2957 treated inpatients in the electronic period on the orthopaedic department, which does not correspond to the departments beds capacity which also peaks in the electronic period on the orthopaedic department (N = 97) but shows its minimum in the paper-based period on the abdominal department (N = 58). Table 3 offers an overview of assessed characteristics for all three departments per period.

Table 3 Departments characteristics

	Orthopaedic Surgery	Abdominal Surgery	Cardiac Surgery	
Paper
2020	Electronic
2022	Paper
2019	Electronic
2020	Paper
2020	Electronic
2021	
Inpatient cases / year	2912	2957	1984	1867	1787	1844	
Beds capacity	94	97	58	62	74	74	
Top 5 Diagnoses (ICD) Treated	T84, M54, S42, S52, M48	S52, M54, S42, T84, M48	C16, C15, C78, C25, N18	C16, C15, C78, K80, N18	I35, I21, I20, I25, C34	I35, I20, I21, I25, I71	
Number of physicians	32,06	32,17	23,61	23,10	40,66	44,49	
Number of nursing staff	74,50	62,21	43,72	51,99	100,99	116,45	

Change of mean completeness

When looking at the mean completeness scores, which describe the average number of documented items out of the total of ten possible items across all records per record type, the t-tests show statistical significance regarding improvements in all three departments, as well as a pooled improvement over all three departments for the EMR. The overall mean score of completeness was 6.02 (SD = 1.88) for the PMR type, rising up to 7.2 (SD = 1.77) for the EMR type. The lowest mean completeness score was seen in the PMR type on the abdominal department (m = 5.91, SD = 2.14) which also showed the highest mean completeness score in the EMR type (m = 7.3, SD = 1.95), implying the highest difference of 1.39. Vice versa the orthopaedical department showed the highest mean completeness score in the PMR type (m = 6.25, SD = 2.12), but the lowest mean completeness score in the EMR type (m = 7.13, SD = 2.0), implying the lowest difference of 0.88. All mean completeness scores and differences are stated in detail in Table 4.

Table 4 Change of Mean completeness scores per record type

	Orthopaedic Surgery	Abdominal Surgery	Cardiac Surgery	Overall	
Mean Score

0–10

	PMR	EMR	P	PMR	EMR	P	PMR	EMR	P	PMR	EMR	P	
N = 44	N = 136	N = 95	N = 139	N = 109	N = 136	N = 248	N = 411	
6.25	7.13	0.014	5.91	7.30	< 0.001	6.04	7.18	< 0.001	6.02	7.20	< 0.001	
Difference	0.88	1.39	1.14	1.18	

Change of percentage completeness

When looking at the changes of percentage completeness of the ten analyzed items, the pooled analysis shows mixed results. Overall, eight of the ten analyzed items exhibited an improvement in the EMR, specifically the documentation of the items Blood Pressure, Body Temperature, Code Status, Diet, Excretions, Height, Pulse. Conversely, one item showed a decline (Admission Diagnosis) and one item remained unaltered (Pain). Differentiated to the three departments, the results show an even more inconsistent picture in some cases. On the one hand, percentage completeness of the item admission diagnosis has consistently decreased in all three departments. On the other hand, the items diet, height and weight have consistently increased in all three departments. All other changes do not show the same direction over all three departments. At department level, only three items have improved in orthopaedic surgery, while further three have deteriorated and four remain unchanged, meaning that the overall picture for orthopaedics surgery remains balanced. In abdominal surgery as a counterexample, all items improved, except for the admission diagnosis which -as already stated- decreased in all departments and excretions, which remained unchanged. This includes that also the documentation of pain improved, which deteriorated in the other two departments and remained unchanged at the overall level. All values for the changes of percentage completeness per item are shown in Table 5.

Table 5 Change of percentage completeness per Item*

Discussion

The results of the case study show that the implementation of an EMR altered the completeness of documentation within the departments studied. All departments experienced significant improvements in the mean completeness scores, but as expected, with differences across the three departments. In addition, the changes of percentage completeness of the ten individual items show a highly differentiated picture. On the one hand, there are consistent changes of percentage completeness of certain information in all three departments. As an example, percentage completeness of the item diet increased in all three departments, while the item admission diagnosis decreased in all three departments. On the other hand, there are also some divergent changes with improvements in one department and deterioration in another, like the item pain increasing in abdominal surgery, but decreasing in orthopaedic and cardiac surgery. Based on the selection of comparable departments (all surgical disciplines), it can be assumed that the differences in changes are due less to a deviating fundamental context than due to the department-specific adoption process of the EMR.

Clinical adoption meta model as a frame of reference

The clinical adoption meta model (CAMM) serves as a framework for categorizing this department-specific adoption process [40]. The CAMM functions as a generic model for describing the adoption process of digital innovations in healthcare [40]. It is made up of the four interrelated dimensions of availability, system use, clinical behavior and outcomes (Fig. 1). In the first step, the EMR must be available. This includes not only the Go Life, but also possible access rights that need to be distributed or mandatory training for end users to be completed. Following the availability, the system use dimension is introduced, which is accompanied by the quantitative usage metrics, such as the number of logins. The third dimension, clinical / health behavior, describes potential changes in behavior or processes and if routines are adjusted to the new EMR. Within the final dimension, potential outcomes like the completeness scores become visible. Due to the comparable context, it is therefore the first three dimensions of CAMM (availability, system use, clinical/health behavior) that are relevant to explain the differences in the changes in completeness across the three departments.

Fig. 1 The four dimensions of the clinical adoption meta model (CAMM)

Availability

In the matter of availability, hardware-related issues are empirically proven which could impede the data entry processes, including poor battery capacity, challenges in docking computer tablets, and malfunctioning touch screens [41], . Moreover, Topaz et al. [42] highlighted a lack of user training, which may weaken data completeness. Although insufficient training or hardware are often cited as a contributor to incomplete documentation, it is noteworthy that both, hospital’s mandatory user training and the hardware used, were consistently the same across the three departments under investigation in this study. The same applies to the EMRs user interface, which can have an influence on the completeness (e.g. with mandatory fields or templates) [43], but is the same across the departments. Therefore, the CAMM’s dimension availability probably plays a subordinate role in explaining the differences in the changes in data quality, shown in this study.

System use

Similarities across the three departments like the consistent user training, consistent hardware and consistent user interface can also be observed with regard to the CAMM’s dimension system use. On the level of percentage completeness, all three departments have experienced significant declines in the completeness of diagnosis documentation. It should be noted at this point that at the time of the PMR, the diagnosis was entered in an already digital HIS. During the period of the paper-based record, the diagnosis was manually transferred from the digital HIS to the still paper-based pre-print. However, now that both, HIS and EMR are digital, they are run as data silos without an interoperable interface. However, the diagnosis is still being documented in the HIS but is no longer, neither manually nor automatically, transferred into the EMR. This fact makes it clear that both, before and after the beginning of the implementation process of the EMR, a diagnosis was available for each individual patient. Strudwick et al. [44] have already described the same fact that clinical staff experience too heavy workload to document the same information in different places. Here, technical options would be desirable that clearly exist to fill several fields which contain the same information within one system or even across competing systems with just one entry. This is an important point, since although the staff can access the diagnosis in the HIS just as easily as in the EMR, but overall the data quality of the EMR data is more incomplete as a result. However, inadequate system use has consistently led to a deterioration in data quality in all three clinics, so the CAMM’s dimension system use is probably also not explaining the differences in the changes in data quality, shown in this study.

Clinical / Health Behavior.

In view of the fact that the CAMM’s dimensions availability and system use must be regarded as largely the same across the three different departments, the clinical / health behavior appears to have a significant influence. However, this dimension can be subdivided into staff and patients, but with the EMR being only used by staff, solely the aspect of clinical behavior remains relevant. In this matter, clinical behavior can be reflected as the usage of advanced functions of the EMR. An example for such an advanced function would be the connection of intelligent devices to the EMR, like the wireless and automatic transfer of data from blood pressure monitoring systems into the EMR. In this context, there is evidence that due to problems with the wireless transfer, information is still often entered manually and accordingly clinical behavior is not adapted to the advanced or smart options of the EMR [41]. In addition, the staff’s clinical behavior can be strongly influenced by the attributes of the treated patient. In this matter, information could be missing, since they are just not seen as relevant to document. An example is a patient whose diet was not documented. The missing information could also be interpreted that if the diet is not explicitly documented, no special diets need to be considered. This is a known mechanism called informative missingness [45]. The informative missingness mechanism is underlined by the fact that people who are seriously ill have a mean complete documentation above average, because much more information is relevant for those patients and therefore cannot be left blank with the idea of informational missingness [46]. At last, emergencies tend to be less complete documented, logically due to the time pressure under which these patients are treated and their treatment is documented [47]. As clinicians have to decide whether documenting information is worth the time [48], a deep understanding of the importance of complete documentation should be promoted. In this respect, the advanced functions of the EMR should always also be aimed at saving time in the documentation process.

Limitations

For the presented article, limitations must be stated regarding (1) the interfering impact of the Covid-19 pandemic and (2) the underlying data basis.

(1) As depicted in Table 2, the records distribution cannot be assumed to be equal. Especially the period of the orthopaedic department in which the PMR were collected was affected by the Covid-19 pandemic, when planned interventions were suspended. This might explain the considerable difference between the Npaper (N = 44) and the Neletronic (N = 136) for the orthopaedic department. In terms of the impact of the Covid-19 pandemic on data quality, however, it became apparent that the infectious situation has potentially increased the completeness of the documentation. On the one hand, information was collected as completely as possible in order to avoid additional contact for adding missing information afterwards [49], and also because this information had to contribute to political decision-making in this uncertain situation and complete datasets therefore been requested [50]. However, since the other departments and times were unaffected by the restrictions of the Covid-19 pandemic but also have lower Npaper than Neletronic, it is conceivable that the PMR type itself has an influence on the number of available records. This is conceivable due to the fact that the PMR could actually not have been found in the department’s archives, whereas the EMRs were of course available for all persons treated. Especially since the number of treated patients remained roughly the same (Table 3), this might explain the differences in analyzed records.

(2) As described in the methods section, the analyses were based on fully de-identified data in accordance with national data protection regulations. As a result, no explicit study sample description was possible, which was countered by describing the annual patient population at department level. Nevertheless, the results can only be interpreted to a limited extent with regard to the possible influence of the actual patients whose treatment was documented, or with regard to the actual person who carried out the documentation. In this regard, there is no information about the comparability of the compared groups of documenting persons (e.g. due to normal staff turnover). In addition, the data only represents the patients from a three-week period. Longer periods could provide more meaningful results here. Future research projects should aim to analyze datasets that contain individual characteristics of the patients and the documenting staff in order to be able to calculate mixed effects models or other more advanced statistical models. In addition, department-specific empirical data at the level of all respective dimensions of the CAMM model could help to better understand the changes in data quality.

Conclusion & implications

The European health data space and big data analyzability is heavily dependent on sufficient data quality. The completeness of the data is one of the most important dimensions in this regard. This study underlines the importance of implementing or adapting electronic medical records to improve the completeness of this data. At the same time, it strengthens the evidence that the empirically demonstrated improvements might be dependent on the departments’ adoption process. Future research should explicitly address the clinical behavior of the documenting individuals and the technical possibilities to reduce the staff’s documentation burden. Moreover, sufficient datasets should be aimed at to apply more complex statistical models, which should be linked to qualitative data to conclusively understand the dependence on the adoption process. Clinical practice should consider this potential dependency in the inevitable adaptation of their EMRs and conduct the corresponding needs analyses of documenting staff. Furthermore, existing data silos should be dismantled with a focus on interoperability.

Acknowledgements

Not applicable.

Author contributions

FW and UK conceptualized the study. FW and CH interpreted the data, being supervised by UK. FW wrote the first draft, which was reviewed and edited by all authors.

Funding

The work is funded by the German Federal Ministry of Education and Research with grant number 01GP1906B.

Open Access funding enabled and organized by Projekt DEAL.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

The eCoCo case study was approved by the ethics committee of the medical faculty of the university of cologne (20-1349) and is listed in the German national registry for clinical trials (DRKS00023343). Consent to participate was not required, as the analysis was based on completely anonymous secondary data in accordance with German and European data protection regulations (General Data Protection Regulation (EU) 2016/679), which was confirmed by the aforementioned ethics committee of the medical faculty of the university of cologne.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Schopf TR Nedrebø B Hufthammer KO Daphu IK Lærum H How well is the electronic health record supporting the clinical tasks of hospital physicians? A survey of physicians at three Norwegian hospitals BMC Health Serv Res 2019 19 1 934 10.1186/s12913-019-4763-0 31801518
Schopf TR, Nedrebø B, Hufthammer KO, Daphu IK, Lærum H. How well is the electronic health record supporting the clinical tasks of hospital physicians? A survey of physicians at three Norwegian hospitals. BMC Health Serv Res. 2019;19(1):934. 10.1186/s12913-019-4763-0.31801518 10.1186/s12913-019-4763-0
2. Kraus S Schiavone F Pluzhnikova A Invernizzi AC Digital transformation in healthcare: analyzing the current state-of-research J Bus Res 2021 123 557 67 10.1016/j.jbusres.2020.10.030
Kraus S, Schiavone F, Pluzhnikova A, Invernizzi AC. Digital transformation in healthcare: analyzing the current state-of-research. J Bus Res. 2021;123:557–67. 10.1016/j.jbusres.2020.10.030.10.1016/j.jbusres.2020.10.030
3. Campanella P Lovato E Marone C Fallacara L Mancuso A Ricciardi W Specchia ML The impact of electronic health records on healthcare quality: a systematic review and meta-analysis Eur J Public Health 2016 26 1 60 4 10.1093/eurpub/ckv122 26136462
Campanella P, Lovato E, Marone C, Fallacara L, Mancuso A, Ricciardi W, Specchia ML. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. Eur J Public Health. 2016;26(1):60–4. 10.1093/eurpub/ckv122.26136462 10.1093/eurpub/ckv122
4. Vos JFJ Boonstra A Kooistra A Seelen M van Offenbeek M The influence of electronic health record use on collaboration among medical specialties BMC Health Serv Res 2020 20 1 676 10.1186/s12913-020-05542-6 32698807
Vos JFJ, Boonstra A, Kooistra A, Seelen M, van Offenbeek M. The influence of electronic health record use on collaboration among medical specialties. BMC Health Serv Res. 2020;20(1):676. 10.1186/s12913-020-05542-6.32698807 10.1186/s12913-020-05542-6
5. Rajaei O Khayami SR Rezaei MS Smart hospital definition: academic and industrial perspective Int J Med Inf 2023 182 105304 10.1016/j.ijmedinf.2023.105304
Rajaei O, Khayami SR, Rezaei MS. Smart hospital definition: academic and industrial perspective. Int J Med Inf. 2023;182:105304. 10.1016/j.ijmedinf.2023.105304.10.1016/j.ijmedinf.2023.105304
6. Al-Sahab B Leviton A Loddenkemper T Paneth N Zhang B Biases in Electronic Health Records Data for Generating Real-World evidence: an overview J Healthc Inf Res 2023 10.1007/s41666-023-00153-2
Al-Sahab B, Leviton A, Loddenkemper T, Paneth N, Zhang B. Biases in Electronic Health Records Data for Generating Real-World evidence: an overview. J Healthc Inf Res. 2023. 10.1007/s41666-023-00153-2.10.1007/s41666-023-00153-2
7. Liaw ST Rahimi A Ray P Taggart J Dennis S de Lusignan S Jalaludin B Yeo AET Talaei-Khoei A Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature Int J Med Inf 2013 82 1 10 24 10.1016/j.ijmedinf.2012.10.001
Liaw ST, Rahimi A, Ray P, Taggart J, Dennis S, de Lusignan S, Jalaludin B, Yeo AET, Talaei-Khoei A. Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature. Int J Med Inf. 2013;82(1):10–24. 10.1016/j.ijmedinf.2012.10.001.10.1016/j.ijmedinf.2012.10.001
8. Chawla NV Davis DA Bringing big data to personalized healthcare: a patient-centered framework J GEN INTERN MED 2013 28 Suppl 3 S660 5 10.1007/s11606-013-2455-8 23797912
Chawla NV, Davis DA. Bringing big data to personalized healthcare: a patient-centered framework. J GEN INTERN MED. 2013;28(Suppl 3):S660–5. 10.1007/s11606-013-2455-8.23797912 10.1007/s11606-013-2455-8
9. Zhou H Silverman G Niu Z Silverman J Evans R Austin R Zhang R Extracting complementary and Integrative Health Approaches in Electronic Health Records J Healthc Inf Res 2023 7 3 277 90 10.1007/s41666-023-00137-2
Zhou H, Silverman G, Niu Z, Silverman J, Evans R, Austin R, Zhang R. Extracting complementary and Integrative Health Approaches in Electronic Health Records. J Healthc Inf Res. 2023;7(3):277–90. 10.1007/s41666-023-00137-2.10.1007/s41666-023-00137-2
10. Maclagan LC Abdalla M Harris DA Stukel TA Chen B Candido E Swartz RH Iaboni A Jaakkimainen RL Bronskill SE Can patients with dementia be identified in primary Care Electronic Medical records using Natural Language Processing? J Healthc Inf Res 2023 7 1 42 58 10.1007/s41666-023-00125-6
Maclagan LC, Abdalla M, Harris DA, Stukel TA, Chen B, Candido E, Swartz RH, Iaboni A, Jaakkimainen RL, Bronskill SE. Can patients with dementia be identified in primary Care Electronic Medical records using Natural Language Processing? J Healthc Inf Res. 2023;7(1):42–58. 10.1007/s41666-023-00125-6.10.1007/s41666-023-00125-6
11. Ballarini NM Rosenkranz GK Jaki T König F Posch M Subgroup identification in clinical trials via the predicted individual treatment effect PLoS ONE 2018 13 10 e0205971 10.1371/journal.pone.0205971 30335831
Ballarini NM, Rosenkranz GK, Jaki T, König F, Posch M. Subgroup identification in clinical trials via the predicted individual treatment effect. PLoS ONE. 2018;13(10):e0205971. 10.1371/journal.pone.0205971.30335831 10.1371/journal.pone.0205971
12. Shen X Ma S Vemuri P Castro MR Caraballo PJ Simon GJ A novel method for causal structure discovery from EHR data and its application to type-2 diabetes mellitus Sci Rep 2021 11 1 21025 10.1038/s41598-021-99990-7 34697394
Shen X, Ma S, Vemuri P, Castro MR, Caraballo PJ, Simon GJ. A novel method for causal structure discovery from EHR data and its application to type-2 diabetes mellitus. Sci Rep. 2021;11(1):21025. 10.1038/s41598-021-99990-7.34697394 10.1038/s41598-021-99990-7
13. Glaser J. It’s time for a new kind of electronic health record. Harvard Business Review; 2020.
14. Karr AF Sanil AP Banks DL Data quality: a statistical perspective Stat Methodol 2006 3 2 137 73 10.1016/j.stamet.2005.08.005
Karr AF, Sanil AP, Banks DL. Data quality: a statistical perspective. Stat Methodol. 2006;3(2):137–73. 10.1016/j.stamet.2005.08.005.10.1016/j.stamet.2005.08.005
15. Strong DM Lee YW Wang RY Data quality in context Commun ACM 1997 40 5 103 10 10.1145/253769.253804
Strong DM, Lee YW, Wang RY. Data quality in context. Commun ACM. 1997;40(5):103–10. 10.1145/253769.253804.10.1145/253769.253804
16. Cai L Zhu Y 2015 The Challenges of Data Quality and Data Quality Assessment in the Big Data Era 10.5334/dsj-2015-002
Cai L, Zhu Y. CODATA. 2015;14(0):2. 10.5334/dsj-2015-002. The Challenges of Data Quality and Data Quality Assessment in the Big Data Era.10.5334/dsj-2015-002
17. Pipino LL Lee YW Wang RY Data quality assessment Commun ACM 2002 45 4 211 8 10.1145/505248.506010
Pipino LL, Lee YW, Wang RY. Data quality assessment. Commun ACM. 2002;45(4):211–8. 10.1145/505248.506010.10.1145/505248.506010
18. Kahn MG Callahan TJ Barnard J Bauck AE Brown J Davidson BN Estiri H Goerg C Holve E Johnson SG Liaw S-T Hamilton-Lopez M Meeker D Ong TC Ryan P Shang N Weiskopf NG Weng C Zozus MN Schilling L 2016 A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data 10.13063/2327-9214.1244
Kahn MG, Callahan TJ, Barnard J, Bauck AE, Brown J, Davidson BN, Estiri H, Goerg C, Holve E, Johnson SG, Liaw S-T, Hamilton-Lopez M, Meeker D, Ong TC, Ryan P, Shang N, Weiskopf NG, Weng C, Zozus MN, Schilling L. EGEMS (Wash DC). 2016;4(1):1244. 10.13063/2327-9214.1244. A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data.10.13063/2327-9214.1244
19. Lee K Weiskopf N Pathak J A Framework for Data Quality Assessment in Clinical Research Datasets AMIA Annu Symp Proc 2017 2017 1080 9 29854176
Lee K, Weiskopf N, Pathak J. A Framework for Data Quality Assessment in Clinical Research Datasets. AMIA Annu Symp Proc. 2017;2017:1080–9.29854176
20. Dick RS Steen EB Detmer DE The computer-based patient record. An essential technology for health care 1997 Washington, D.C National Academy
Dick RS, Steen EB, Detmer DE. The computer-based patient record. An essential technology for health care. Washington, D.C.: National Academy; 1997.
21. Nobles AL, Vilankar K, Wu H, Barnes LE. (2015) Evaluation of data quality of multisite electronic health record data for secondary analysis 2015 IEEE International Conference on Big Data (Big Data). IEEE.
22. Weiskopf NG Weng C Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research J Am Med Inf Assoc 2013 20 1 144 51 10.1136/amiajnl-2011-000681
Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inf Assoc. 2013;20(1):144–51. 10.1136/amiajnl-2011-000681.10.1136/amiajnl-2011-000681
23. Berner ES Kasiraman RK Yu F Ray MN Houston TK Data quality in the outpatient setting: impact on clinical decision support systems AMIA Annu Symp Proc 2005 2005 41 5 16778998
Berner ES, Kasiraman RK, Yu F, Ray MN, Houston TK. Data quality in the outpatient setting: impact on clinical decision support systems. AMIA Annu Symp Proc. 2005;2005:41–5.16778998
24. Hecht J The future of electronic health records Nature 2019 573 7775 S114 6 10.1038/d41586-019-02876-y 31554991
Hecht J. The future of electronic health records. Nature. 2019;573(7775):S114–6. 10.1038/d41586-019-02876-y.31554991 10.1038/d41586-019-02876-y
25. Lewis AE Weiskopf N Abrams ZB Foraker R Lai AM Payne PRO Gupta A Electronic health record data quality assessment and tools: a systematic review J Am Med Inf Assoc 2023 30 10 1730 40 10.1093/jamia/ocad120
Lewis AE, Weiskopf N, Abrams ZB, Foraker R, Lai AM, Payne PRO, Gupta A. Electronic health record data quality assessment and tools: a systematic review. J Am Med Inf Assoc. 2023;30(10):1730–40. 10.1093/jamia/ocad120.10.1093/jamia/ocad120
26. Weiskopf NG Hripcsak G Swaminathan S Weng C Defining and measuring completeness of electronic health records for secondary use J Biomed Inf 2013 46 5 830 6 10.1016/j.jbi.2013.06.010
Weiskopf NG, Hripcsak G, Swaminathan S, Weng C. Defining and measuring completeness of electronic health records for secondary use. J Biomed Inf. 2013;46(5):830–6. 10.1016/j.jbi.2013.06.010.10.1016/j.jbi.2013.06.010
27. Stevenson JE Israelsson J Petersson G Bath PA Factors influencing the quality of vital sign data in electronic health records: a qualitative study J Clin Nurs 2018 27 5–6 1276 86 10.1111/jocn.14174 29149483
Stevenson JE, Israelsson J, Petersson G, Bath PA. Factors influencing the quality of vital sign data in electronic health records: a qualitative study. J Clin Nurs. 2018;27(5–6):1276–86. 10.1111/jocn.14174.29149483 10.1111/jocn.14174
28. Wurster F Fütterer G Beckmann M Dittmer K Jaschke J Köberlein-Neu J Okumu M-R Rusniok C Pfaff H Karbach U The Analyzation of Change in Documentation due to the Introduction of Electronic Patient Records in Hospitals-A systematic review J Med Syst 2022 46 8 54 10.1007/s10916-022-01840-0 35781136
Wurster F, Fütterer G, Beckmann M, Dittmer K, Jaschke J, Köberlein-Neu J, Okumu M-R, Rusniok C, Pfaff H, Karbach U. The Analyzation of Change in Documentation due to the Introduction of Electronic Patient Records in Hospitals-A systematic review. J Med Syst. 2022;46(8):54. 10.1007/s10916-022-01840-0.35781136 10.1007/s10916-022-01840-0
29. Wiebe N Otero Varela L Niven DJ Ronksley PE Iragorri N Quan H Evaluation of interventions to improve inpatient hospital documentation within electronic health records: a systematic review J Am Med Inf Assoc 2019 26 11 1389 400 10.1093/jamia/ocz081
Wiebe N, Otero Varela L, Niven DJ, Ronksley PE, Iragorri N, Quan H. Evaluation of interventions to improve inpatient hospital documentation within electronic health records: a systematic review. J Am Med Inf Assoc. 2019;26(11):1389–400. 10.1093/jamia/ocz081.10.1093/jamia/ocz081
30. Abbott PA Foster J Marin HF Dykes PC Complexity and the science of implementation in health IT–knowledge gaps and future visions Int J Med Inf 2014 83 7 e12 22 10.1016/j.ijmedinf.2013.10.009
Abbott PA, Foster J, Marin HF, Dykes PC. Complexity and the science of implementation in health IT–knowledge gaps and future visions. Int J Med Inf. 2014;83(7):e12–22. 10.1016/j.ijmedinf.2013.10.009.10.1016/j.ijmedinf.2013.10.009
31. Söling S Demirer I Köberlein-Neu J Hower KI Müller BS Pfaff H Karbach U Complex implementation mechanisms in primary care: do physicians’ beliefs about the effectiveness of innovation play a mediating role? Applying a realist inquiry and structural equation modeling approach in a formative evaluation study BMC Prim Care 2023 24 1 131 10.1186/s12875-023-02081-x 37369994
Söling S, Demirer I, Köberlein-Neu J, Hower KI, Müller BS, Pfaff H, Karbach U. Complex implementation mechanisms in primary care: do physicians’ beliefs about the effectiveness of innovation play a mediating role? Applying a realist inquiry and structural equation modeling approach in a formative evaluation study. BMC Prim Care. 2023;24(1):131. 10.1186/s12875-023-02081-x.37369994 10.1186/s12875-023-02081-x
32. Ash JS Berg M Coiera E Some unintended consequences of information technology in health care: the nature of patient care information system-related errors J Am Med Inf Assoc 2004 11 2 104 12 10.1197/jamia.M1471
Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inf Assoc. 2004;11(2):104–12. 10.1197/jamia.M1471.10.1197/jamia.M1471
33. Cucciniello M Lapsley I Nasi G Pagliari C Understanding key factors affecting electronic medical record implementation: a sociotechnical approach BMC Health Serv Res 2015 15 268 10.1186/s12913-015-0928-7 26184405
Cucciniello M, Lapsley I, Nasi G, Pagliari C. Understanding key factors affecting electronic medical record implementation: a sociotechnical approach. BMC Health Serv Res. 2015;15:268. 10.1186/s12913-015-0928-7.26184405 10.1186/s12913-015-0928-7
34. Wurster F Di Gion P Goldberg N Hautsch V Hefter K Herrmann C Langebartels G Pfaff H Karbach U Roger’s diffusion of innovations theory and the adoption of a patient portal’s digital anamnesis collection tool: study protocol for the MAiBest project Implement Sci Commun 2024 5 1 74 10.1186/s43058-024-00614-8 39010236
Wurster F, Di Gion P, Goldberg N, Hautsch V, Hefter K, Herrmann C, Langebartels G, Pfaff H, Karbach U. Roger’s diffusion of innovations theory and the adoption of a patient portal’s digital anamnesis collection tool: study protocol for the MAiBest project. Implement Sci Commun. 2024;5(1):74. 10.1186/s43058-024-00614-8.39010236 10.1186/s43058-024-00614-8
35. Meier-Diedrich E Davidge G Hägglund M Kharko A Lyckblad C McMillan B Blease C Schwarz J Changes in Documentation due to Patient Access to Electronic Health records: protocol for a scoping review JMIR Res Protoc 2023 12 e46722 10.2196/46722 37639298
Meier-Diedrich E, Davidge G, Hägglund M, Kharko A, Lyckblad C, McMillan B, Blease C, Schwarz J. Changes in Documentation due to Patient Access to Electronic Health records: protocol for a scoping review. JMIR Res Protoc. 2023;12:e46722. 10.2196/46722.37639298 10.2196/46722
36. Kaipio J Kuusisto A Hyppönen H Heponiemi T Lääveri T Physicians’ and nurses’ experiences on EHR usability: comparison between the professional groups by employment sector and system brand Int J Med Inf 2020 134 104018 10.1016/j.ijmedinf.2019.104018
Kaipio J, Kuusisto A, Hyppönen H, Heponiemi T, Lääveri T. Physicians’ and nurses’ experiences on EHR usability: comparison between the professional groups by employment sector and system brand. Int J Med Inf. 2020;134:104018. 10.1016/j.ijmedinf.2019.104018.10.1016/j.ijmedinf.2019.104018
37. Beckmann M Dittmer K Jaschke J Karbach U Köberlein-Neu J Nocon M Rusniok C Wurster F Pfaff H Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol BMC Health Serv Res 2021 21 1 377 10.1186/s12913-021-06377-5 33892703
Beckmann M, Dittmer K, Jaschke J, Karbach U, Köberlein-Neu J, Nocon M, Rusniok C, Wurster F, Pfaff H. Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol. BMC Health Serv Res. 2021;21(1):377. 10.1186/s12913-021-06377-5.33892703 10.1186/s12913-021-06377-5
38. The Royal Australian College of General Practitioners. (2020) Standards for general practices. 5th edn., East Melbourne, Vic.
39. Wurster F Beckmann M Cecon-Stabel N Dittmer K Hansen TJ Jaschke J Köberlein-Neu J Okumu M-R Rusniok C Pfaff H Karbach U The implementation of an Electronic Medical Record in a German hospital and the change in completeness of documentation: longitudinal document analysis JMIR Med Inf 2024 12 e47761 10.2196/47761
Wurster F, Beckmann M, Cecon-Stabel N, Dittmer K, Hansen TJ, Jaschke J, Köberlein-Neu J, Okumu M-R, Rusniok C, Pfaff H, Karbach U. The implementation of an Electronic Medical Record in a German hospital and the change in completeness of documentation: longitudinal document analysis. JMIR Med Inf. 2024;12:e47761. 10.2196/47761.10.2196/47761
40. Price M Lau F The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health information systems BMC Med Inf Decis Mak 2014 14 43 10.1186/1472-6947-14-43
Price M, Lau F. The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health information systems. BMC Med Inf Decis Mak. 2014;14:43. 10.1186/1472-6947-14-43.10.1186/1472-6947-14-43
41. Kjær J Milling L Wittrock D Nielsen LB Mikkelsen S The data quality and applicability of a Danish prehospital electronic health record: a mixed-methods study PLoS ONE 2023 18 10 e0293577 10.1371/journal.pone.0293577 37883522
Kjær J, Milling L, Wittrock D, Nielsen LB, Mikkelsen S. The data quality and applicability of a Danish prehospital electronic health record: a mixed-methods study. PLoS ONE. 2023;18(10):e0293577. 10.1371/journal.pone.0293577.37883522 10.1371/journal.pone.0293577
42. Topaz M Ronquillo C Peltonen L-M Pruinelli L Sarmiento RF Badger MK Ali S Lewis A Georgsson M Jeon E Tayaben JL Kuo C-H Islam T Sommer J Jung H Eler GJ Alhuwail D Lee Y-L Nurse informaticians Report low satisfaction and multi-level concerns with Electronic Health Records: results from an International Survey AMIA Annu Symp Proc 2016 2016 2016 25 28269961
Topaz M, Ronquillo C, Peltonen L-M, Pruinelli L, Sarmiento RF, Badger MK, Ali S, Lewis A, Georgsson M, Jeon E, Tayaben JL, Kuo C-H, Islam T, Sommer J, Jung H, Eler GJ, Alhuwail D, Lee Y-L. Nurse informaticians Report low satisfaction and multi-level concerns with Electronic Health Records: results from an International Survey. AMIA Annu Symp Proc. 2016;2016:2016–25.28269961
43. Madandola OO Bjarnadottir RI Yao Y Ansell M Dos Santos F Cho H Dunn Lopez K Macieira TGR Keenan GM The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review J Am Med Inf Assoc 2023 31 1 240 55 10.1093/jamia/ocad188
Madandola OO, Bjarnadottir RI, Yao Y, Ansell M, Dos Santos F, Cho H, Dunn Lopez K, Macieira TGR, Keenan GM. The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review. J Am Med Inf Assoc. 2023;31(1):240–55. 10.1093/jamia/ocad188.10.1093/jamia/ocad188
44. Strudwick G McGillis Hall L Nagle L Trbovich P Acute care nurses’ perceptions of electronic health record use: a mixed method study Nurs Open 2018 5 4 491 500 10.1002/nop2.157 30338094
Strudwick G, McGillis Hall L, Nagle L, Trbovich P. Acute care nurses’ perceptions of electronic health record use: a mixed method study. Nurs Open. 2018;5(4):491–500. 10.1002/nop2.157.30338094 10.1002/nop2.157
45. Bots SH Groenwold RHH Dekkers OM Using electronic health record data for clinical research: a quick guide Eur J Endocrinol 2022 186 4 E1 6 10.1530/eje-21-1088 35157608
Bots SH, Groenwold RHH, Dekkers OM. Using electronic health record data for clinical research: a quick guide. Eur J Endocrinol. 2022;186(4):E1–6. 10.1530/eje-21-1088.35157608 10.1530/eje-21-1088
46. Rusanov A Weiskopf NG Wang S Weng C Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research BMC Med Inf Decis Mak 2014 14 51 10.1186/1472-6947-14-51
Rusanov A, Weiskopf NG, Wang S, Weng C. Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research. BMC Med Inf Decis Mak. 2014;14:51. 10.1186/1472-6947-14-51.10.1186/1472-6947-14-51
47. Jensen FB, Ladefoged KT, Lindskou TA, Søvsø MB, Christensen EF, Teli M. Understanding the Effect of Electronic Prehospital Medical Records in ambulances: a qualitative observational study in a Prehospital setting. Int J Environ Res Public Health. 2021;18(5). 10.3390/ijerph18052330.
48. Wassell M Murray JL Kumar C Verspoor K Butler-Henderson K Understanding Clinician EHR Data Quality for Reuse in Predictive Modelling Stud Health Technol Inf 2024 310 169 73 10.3233/SHTI230949
Wassell M, Murray JL, Kumar C, Verspoor K, Butler-Henderson K. Understanding Clinician EHR Data Quality for Reuse in Predictive Modelling. Stud Health Technol Inf. 2024;310:169–73. 10.3233/SHTI230949.10.3233/SHTI230949
49. Curtis CA Nguyen MU Rathnasekara GK Manderson RJ Chong MY Malawaraarachchi JK Song Z Kanumuri P Potenzi BJ Lim AKH Impact of electronic medical records and COVID-19 on adult goals-of-care document completion and revision in hospitalised general medicine patients Intern Med J 2022 52 5 755 62 10.1111/imj.15543 34580964
Curtis CA, Nguyen MU, Rathnasekara GK, Manderson RJ, Chong MY, Malawaraarachchi JK, Song Z, Kanumuri P, Potenzi BJ, Lim AKH. Impact of electronic medical records and COVID-19 on adult goals-of-care document completion and revision in hospitalised general medicine patients. Intern Med J. 2022;52(5):755–62. 10.1111/imj.15543.34580964 10.1111/imj.15543
50. Lee P, Abernethy A, Shaywitz D, Gundlapalli AV, Weinstein J, Doraiswamy PM, Schulman K, Madhavan S. Digital Health COVID-19 Impact Assessment: lessons learned and compelling needs. NAM Perspect. 2022;2022. 10.31478/202201c.
