
==== Front
Eur J Neurol
Eur J Neurol
10.1111/(ISSN)1468-1331
ENE
European Journal of Neurology
1351-5101
1468-1331
John Wiley and Sons Inc. Hoboken

39045891
10.1111/ene.16418
ENE16418
EJoN-24-0663.R2
Original Article
Dementia and Cognitive Disorders
Diagnostic accuracy of the Brief Assessment of Impaired Cognition case‐finding instrument in a general practice setting and comparison with other widely used brief cognitive tests—a cross‐validation study
Cross‐validation of BASIC in a GP setting
Jørgensen et al.
Jørgensen Kasper https://orcid.org/0000-0002-1395-5143
1 niels.kasper.joergensen@regionh.dk

Nielsen T. Rune 1 2
Nielsen Ann 1
Oxbøll Anne‐Britt 1
Gerner Sofie D. 3
Waldorff Frans B. https://orcid.org/0000-0002-7859-633X
3
Waldemar Gunhild 1 4
1 Danish Dementia Research Centre, Department of Neurology Copenhagen University Hospital‐Rigshospitalet Copenhagen Denmark
2 Department of Psychology University of Copenhagen Copenhagen Denmark
3 Section of General Practice, Department of Public Health University of Copenhagen Copenhagen Denmark
4 Department of Clinical Medicine University of Copenhagen Copenhagen Denmark
* Correspondence
Kasper Jørgensen, Danish Dementia Research Centre, Department of Neurology, Copenhagen University Hospital‐ Rigshospitalet, Copenhagen, Denmark.
Email: niels.kasper.joergensen@regionh.dk

24 7 2024
10 2024
31 10 10.1111/ene.v31.10 e1641804 7 2024
08 4 2024
09 7 2024
© 2024 The Author(s). European Journal of Neurology published by John Wiley & Sons Ltd on behalf of European Academy of Neurology.
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the terms of the http://creativecommons.org/licenses/by-nc-nd/4.0/ License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made.

Abstract

Background and purpose

The aim of this study was to examine the discriminative validity of the Brief Assessment of Impaired Cognition (BASIC) case‐finding instrument in a general practice (GP) setting and compare it with other widely used brief cognitive instruments.

Methods

Patients aged ≥70 years were prospectively recruited from 14 Danish GP clinics. Participants were classified as having either normal cognition (n = 154) or cognitive impairment (n = 101) based on neuropsychological test performance, reported instrumental activities of daily living, and concern regarding memory decline. Comparisons involved the Mini‐Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), the Rowland Universal Dementia Assessment Scale (RUDAS), the Mini‐Cog, the 6‐item Clock Drawing Test (CDT‐6) and the BASIC Questionnaire (BASIC‐Q).

Results

BASIC demonstrated good overall classification accuracy with an area under the receiver operating characteristic curve (AUC) of 0.88 (95% confidence interval [CI] 0.84–0.92), a sensitivity of 0.72 (95% CI 0.62–0.80) and a specificity of 0.86 (95% CI 0.79–0.91). Pairwise comparisons of the AUCs of BASIC, MMSE, MoCA and RUDAS produced non‐significant results, but BASIC had significantly higher classification accuracy than Mini‐Cog, BASIC‐Q and CDT‐6. Depending on the pretest probability of cognitive impairment, the positive predictive validity of BASIC varied from 0.83 to 0.36, and the negative predictive validity from 0.97 to 0.76.

Conclusions

BASIC demonstrated good discriminative validity in a GP setting. The classification accuracy of BASIC is equivalent to more complex, time‐consuming instruments, such as the MMSE, MoCA and RUDAS, and higher than very brief instruments, such as the CDT‐6, Mini‐Cog and BASIC‐Q.

cognitive dysfunction
dementia
general practice
memory disorders
mental status and dementia tests
Ministeriet Sundhed Forebyggelse 10.13039/501100007037 1604063 source-schema-version-number2.0
cover-dateOctober 2024
details-of-publishers-convertorConverter:WILEY_ML3GV2_TO_JATSPMC version:6.4.8 mode:remove_FC converted:20.09.2024
Jørgensen K , Nielsen TR , Nielsen A , et al. Diagnostic accuracy of the Brief Assessment of Impaired Cognition case‐finding instrument in a general practice setting and comparison with other widely used brief cognitive tests—a cross‐validation study. Eur J Neurol. 2024;31 :e16418. doi:10.1111/ene.16418
==== Body
pmcINTRODUCTION

Timely diagnosis of dementia facilitates access to non‐pharmacological interventions, counselling and social community services [1]. General practice (GP) physicians play a key role in the identification of cognitive impairment as they perform initial assessments of patients with cognitive concerns and have a pivotal gatekeeper function for referral to specialist services, including memory clinics. Brief cognitive tests are an essential element in case‐finding of cognitive impairment in the GP setting.

The Brief Assessment of Impaired Cognition (BASIC) was developed as a case‐finding instrument mainly considered for diverse patient populations in the GP setting [2]. It was inspired by existing instruments [3, 4, 5], includes items from validated questionnaires [6, 7] and was developed from a cross‐cultural perspective incorporating tasks with low cultural bias [8]. Subsequently English, French, Italian, Spanish, Chinese, Persian and Japanese versions have been developed without need for cultural adaptations [9]. BASIC has high test–retest reliability [9, 10] and the effect of socio‐demographic variables on BASIC performance is too limited to necessitate adjustment of cutoff scores [2]. BASIC was initially validated in a memory clinic setting demonstrating excellent discriminative validity for dementia, good discriminative validity for mild cognitive impairment (MCI) and superior accuracy for staging of dementia compared to the Mini‐Mental State Examination (MMSE) [10, 11]. As BASIC was developed and validated in the same sample, there was a risk of overfitting the model for selection of scale items to the specific sample, possibly inflating the discriminative validity results.

The aim of this study was to further examine the discriminative validity of BASIC and perform head‐to‐head comparisons with the accuracy of other widely used brief cognitive instruments.

METHODS

As the study design, study activities and classification of participants were previously described in detail [12], only main methodological points are summarized.

Study participants

Patients in 14 GP clinics located in four of the five administrative regions in Denmark were consecutively recruited from October 2021 to October 2022. According to a pre‐study power calculation [13], assuming the prevalence of dementia in a Danish GP setting to be 6%–7% [14], the aim was to include 524 participants. However, as the proportion of participants with cognitive impairment during the inclusion period was substantially higher, the power estimation was adjusted to less than 100 participants. Patients ≥70 years of age consulting their GP physician for any reason were invited to participate. Further inclusion criteria were (i) being fluent in Danish and (ii) having a close relative agreeing to complete the informant report in BASIC and the Functional Assessment Questionnaire (FAQ) [15]. Exclusion criteria were (i) a known diagnosis of dementia or (ii) a condition that, judged by the physician, would affect the patient's ability to complete the comprehensive cognitive assessment.

The study was carried out in accordance with the Code of Ethics of the World Medical Association for experiments involving humans and approved by the Danish Data Protection Agency (P‐2020‐685). Ethical approval by the institutional review board was not required as the intervention was too limited to qualify as a health science research project as defined in the committee act. Written informed consent was obtained from all participants.

Participants were initially assessed with the BASIC by their GP physician or nurse. Within 2 weeks, a research assistant conducted a structured telephone interview with questions on socio‐demographic status, health and lifestyle. Level of education was categorized into six groups using a slightly simplified version of the Danish classification system of educational level, DISCED‐15 [16]. Within a month after recruitment, a research assistant administered a comprehensive assessment battery including the Repeatable Battery for Assessment of Neuropsychological Status (RBANS) [17], MMSE [18], the 15‐item Geriatric Depression Scale (GDS‐15) [19], Montreal Cognitive Assessment (MoCA) [20] and Rowland Universal Dementia Assessment Scale (RUDAS) [21]. The order of administration was fixed. Research assistants and researchers involved in telephone interviews and in administration, scoring or evaluation of the results of RBANS and brief cognitive instruments were blinded to BASIC results.

Brief cognitive instruments

The contents of the brief cognitive instruments are summarized in Table 1.

TABLE 1 Characteristics and contents of brief cognitive instruments.

Instrument	Duration, app.	Items	Score range	Year and references	
Administered by GP medical doctor or nurse	
BASIC	5 min	Self‐report (3 questions), verbal fluency (supermarket items), category cued memory test (4 pictures), informant report (3 questions)	0–25	2019 [2]	
BASIC‐Q	<5 min	Self‐report (3 questions), orientation in time and own age, informant report (3 questions)	0–20	2020 [24]	
Administered by research assistant	
MMSE	10 min	Temporal and spatial orientation, immediate recall (3 words), serial subtraction, delayed recall (3 words), naming, verbal repetition, verbal comprehension, writing, reading, figure copy (overlapping pentagons)	0–30	1975 [18]	
MoCA	10–15 min	Brief Trail Making B, Clock Drawing Test, Naming, Digit Span, auditory motor attention, serial subtraction, verbal repetition, verbal fluency (f‐words), similarities, delayed recall (5 words), temporal and spatial orientation	0–30	2005 [20]	
RUDAS	10 min	Visuospatial orientation (body parts), praxis (motor sequence), figure copy (cube), problem‐solving, delayed recall (4 words), verbal fluency (animals)	0–30	2004 [21]	
Derived subsequently	
CDT‐6	<5 min	Clock drawing test (hands pointing to designated minute and hour, number sequence correct, numerals inside circle, ‘12’ placed correctly)	0–6	2015 [22]	
Mini‐Cog	<5 min	Clock drawing test, delayed recall (3 words)	0–5	2000 [23]	
Note: CDT‐6 was derived by rescoring the MoCA clock drawing test. Mini‐Cog was derived by rescoring the MoCA clock drawing test and combining it with the MMSE delayed recall (3 words) item.

Abbreviations: BASIC, Brief Assessment of Impaired Cognition; BASIC‐Q, Brief Assessment of Impaired Cognition Questionnaire; CDT‐6, 6‐item Clock Drawing Test; MMSE, Mini‐Mental State Examination; MoCA, Montreal Cognitive Assessment; RUDAS, Rowland Universal Dementia Assessment Scale.

The 6‐item Clock Drawing Test (CDT‐6) [22] and the Mini‐Cog [23] were not administered as separate tests but were subsequently derived by rescoring the clock drawing test included in the MoCA and combining it with the MMSE delayed recall (three words) item. Regarding memory test components, BASIC includes the Category Cued Memory Test (CCMT) whereas MMSE, MoCA, RUDAS and Mini‐Cog include delayed recall of words. BASIC and the Brief Assessment of Impaired Cognition Questionnaire (BASIC‐Q) [24] have a substantial item overlap as both instruments include the components self‐report and informant report.

Classification of participants

Based on an algorithm including cognitive performance (RBANS total index score), level of caregiver‐reported independence in instrumental activities of daily living (IADL) as measured by the FAQ, and concern regarding memory decline at inclusion, participants were categorized as follows (Figure S1). Normal cognition: Normal cognition was an RBANS total score ≥76 (regardless of FAQ score or memory concern). The RBANS cutoff score of 75/76 is equivalent to the fifth percentile often used in clinical settings as an indicator of impaired performance [25, 26].

Impaired cognition: Impairment in cognitive performance (an RBANS total score ≤75) was chosen as the main criterion, and functional impairment (a FAQ score ≥6) and/or reported concern regarding memory decline at inclusion as supporting criteria. The FAQ cutoff score of 5/6 was derived from a validation study including MCI and mild Alzheimer's disease participants [27]. Clinical diagnoses could not be established, but participants classified as cognitively impaired were referred to their GP for further evaluation.

Non‐relevant cognitive dysfunction: This concept of patients without ‘genuine cognitive deterioration, but rather a low premorbid cognitive level, low educational level, lack of motivation or sensory deficit’ [28] was adopted to describe participants not classified by the algorithm. This residual group scored below the normal range on RBANS (total score ≤75) but were without substantial reported functional decline (FAQ score ≤5) or memory concerns. This group had significantly less education than the cognitively impaired group.

Two secondary analyses were performed. First, to examine the classification accuracy of the brief cognitive instruments in patients with different levels of cognitive impairment, the cognitively impaired group was split ad hoc based on their RBANS scores to obtain two subgroups of roughly the same size. Participants with RBANS scores in the range of ≥58 to ≤75 were labelled ‘very mild’ cognitive impairment and participants with RBANS scores in the range of 40 to ≤57 were labelled ‘mild to moderate’ cognitive impairment.

Second, to further explore the classification accuracy of the brief cognitive instruments, the areas under the receiver operating characteristic (ROC) curve (AUCs) were compared for the memory test components of BASIC, MMSE, MoCA and RUDAS.

Data analysis

The significance of group differences in pairwise comparisons was determined using the independent‐samples t test with approximately normally distributed continuous variables and the independent‐samples median test with non‐normally distributed continuous variables. The significance of group differences in gender distribution was determined with the Pearson χ 2 test. Effect sizes were calculated as Hedges' g [29]. p < 0.05 (two‐tailed) was considered significant. Associations between brief cognitive instruments and memory test components were assessed by the Pearson product–moment correlation coefficient.

Discriminative validity was assessed by calculating sensitivity, specificity, and positive and negative likelihood ratios (LRs) using cognitive status classification (normal vs. impaired cognition) as reference standard. Optimal cutoff scores for balancing sensitivity and specificity were determined using Youden's J [30]. ROC curves were constructed and the AUCs were compared in a series of pairwise comparisons, using a paired‐sample design. Discriminative validity statistics were interpreted as follows: ≥0.90 ‘excellent’, ≥0.80–0.89 ‘good’, ≥0.70–0.79 ‘fair’, <0.70 ‘poor’ [31]. Predictive validity estimates for selected pretest probabilities of cognitive impairment were calculated according to Bayes' classical theorem [32]. Effects of socio‐demographic variables on BASIC performance were examined by linear regression analysis with plots of residuals as model control. An online clinical research calculator was used to calculate confidence intervals (CI) for sensitivity, specificity and LRs [33]. All other analyses were performed with SPSS statistical software (v29.0.1.0, SPSS, Chicago, IL, USA).

RESULTS

Of 299 patients initially screened with BASIC, 24 withdrew consent, leaving 275 patients eligible for inclusion. After exclusion of 20 patients with non‐relevant cognitive dysfunction [28], 255 participants were included in the analyses (Figure S1). Eleven participants were unable to complete the full RBANS but were classified as cognitively impaired as they had an FAQ ≥6, MMSE ≤20, and concern regarding memory decline. Characteristics of the included participants are presented in Table 2.

TABLE 2 Participant characteristics.

	Normal cognition (n = 154)	Cognitive impairment (n = 101)	p	
Mean (SD) (range)	Mean (SD) (range)	
Age (years)	76.5 (5.25) (70–91)	79.3 (5.10) (70–92)	<0.001	
Sex (female/male), n (%)	88 (57%)/66 (43%)	48 (48%)/53 (53%)	0.132	
Education (DISCED)	4.1 (1.34) (1–6)	3.4 (1.06) (1–6)	<0.001	
FAQ	0 (0–19) a	6 (0–22) a	<0.001	
GDS‐15	1 (0–11) a	3 (0–13) a	<0.001	
MMSE	28.7 (1.39) (24–30)	24.9 (3.83) (12–30)	<0.001	
RBANS total score	96.7 (13.42) (76–143)	57.9 (11.74) (40–75)	<0.001	
Abbreviations: DISCED, Statistics Denmark's Classification of Education; FAQ, Functional Assessment Questionnaire; GDS‐15, 15‐item Geriatric Depression Scale; MMSE, Mini‐Mental State Examination; RBANS, Repeatable Battery for Assessment of Neuropsychological Status.

a FAQ and GDS reported as median and range.

The normal cognition group (n = 154) was significantly younger, better educated, had better IADL (FAQ) and fewer affective symptoms (GDS‐15) than the cognitively impaired group (n = 101), but there was no significant difference in gender distribution.

Discriminative validity of BASIC

Significant differences with large effect sizes were found between the two groups on BASIC performance (t[253] = 12.03, p < 0.001, g = 1.69) and the components of BASIC: self‐report (t[253] = 5.95, p < 0.001, g = 0.76), supermarket fluency (t[253] = 8.88, p < 0.001, g = 1.25), CCMT (t[253] = 5.97, p < 0.001, g = 0.91) and informant report (t[253] = 10.31, p < 0.001, g = 1.32) (Table 3).

TABLE 3 Classification performance of BASIC and its components for participants with cognitive impairment (n = 101) versus normal cognition (n = 154).

Component (score range)	Normal cognition	Cognitive impairment	Effect size (g)	AUC (95% CI)	p	
BASIC (0–25)	21.9 (2.41)	16.7 (3.91)	1.69	0.88 (0.84–0.92)	<0.0005	
Self‐report (0–6)	4.6 (1.26)	3.5 (1.49)	0.76	0.70 (0.63–0.76)	0.002	
Supermarket fluency (0–5)	4.6 (0.78)	3.3 (1.30)	1.25	0.78 (0.72–0.84)	<0.0005	
Category cued memory test (0–8)	7.9 (0.45)	6.8 (1.69)	0.91	0.70 (0.63–0.77)	<0.0005	
Informant report (0–6)	4.9 (1.35)	3.0 (1.56)	1.32	0.82 (0.77–0.87)	<0.0005	
Abbreviations: AUC, area under the receiver operating characteristic curve; BASIC, Brief Assessment of Impaired Cognition; CI, confidence interval.

Applying the AUC as a general measure of classification accuracy, BASIC had good accuracy (AUC 0.88, 95% CI 0.84–0.92), and its components had fair to good accuracy (AUCs from 0.70 to 0.82), thereby all contributing to the discriminative validity. The optimal cutoff score for differentiating between cognitive impairment and normal cognition was 19/20 (Table 4).

TABLE 4 Classification accuracy of BASIC for cognitive impairment at different cutoff scores.

Cutoff	Sensitivity (95% CI)	Specificity (95% CI)	LR+ (95% CI)	LR− (95% CI)	
17/18	0.60 (0.50–0.70)	0.95 (0.90–0.98)	11.63 (5.81–23.25)	0.42 (0.33–0.53)	
18/19	0.66 (0.56–0.75)	0.92 (0.86–0.96)	8.51 (4.86–14.92)	0.37 (0.28–0.48)	
19/20 a	0.72 (0.62–0.80)	0.86 (0.79–0.91)	5.06 (3.37–7.59)	0.32 (0.24–0.44)	
20/21	0.80 (0.71–0.87)	0.77 (0.69–0.83)	3.43 (2.54–4.64)	0.26 (0.17–0.38)	
21/22	0.90 (0.82–0.95)	0.61 (0.53–0.69)	2.31 (1.88–2.85)	0.16 (0.09–0.29)	
Abbreviations: CI, confidence interval; LR+, positive likelihood ratio; LR−, negative likelihood ratio.

a Optimal cutoff score.

At this cutoff BASIC had good specificity (0.86), fair sensitivity (0.72), an LR+ of 5.06, indicating that the probability of a positive score (≤19) is five times higher in the cognitively impaired group than in the normal cognition group, and an LR– of 0.32, indicating that the probability of a negative score (≥20) is three times lower in the cognitively impaired group. In the present sample, with a relatively high prevalence of cognitive impairment (40%), the predictive validity of BASIC was fair to good with a positive predictive validity (PPV) of 0.77 (95% CI 0.67–0.85) and a negative predictive validity (NPV) of 0.83 (95% CI 0.76–0.88). As predictive validity is affected by the pretest probability of cognitive impairment, predictive validity estimates were further calculated for three selected pretest probabilities (Table 5).

TABLE 5 Predictive validity estimates for BASIC at selected pretest probabilities of cognitive impairment.

Cutoff	10% pretest probability	25% pretest probability	50% pretest probability	
PPV	NPV	PPV	NPV	PPV	NPV	
17/18	0.56	0.96	0.79	0.88	0.92	0.71	
18/19	0.49	0.96	0.74	0.89	0.89	0.73	
19/20 a	0.36	0.97	0.63	0.90	0.83	0.76	
20/21	0.28	0.97	0.53	0.92	0.77	0.79	
21/22	0.20	0.98	0.44	0.95	0.70	0.86	
Abbreviations: BASIC, Brief Assessment of Impaired Cognition; NPV, negative predictive validity; PPV, positive predictive validity.

a Optimal cutoff score.

The PPV was good (0.83) in a setting with a high (50%) pretest probability of cognitive impairment, attenuated (0.63) in a setting with a medium (25%) pretest probability, and poor (0.36) in a setting with a low (10%) pretest probability. The NPV was high (0.90–0.97) in settings with medium or low pretest probability and acceptable (0.76) in a setting with high pretest probability of cognitive impairment.

Age and gender, but not education, had significant, but numerically small, effects on BASIC performance explaining 7% of the variance in the normal cognition group. Women outperformed men by 0.92 points and the effect of age was −0.8 point per 10 years of life. To explore the effect of socio‐demographic adjustment on classification accuracy, age‐ and gender‐adjusted BASIC scores were computed for the full sample. This adjustment had no significant effect on overall classification accuracy (AUC difference 0.01, z = 1.94, p = 0.053).

Comparative classification accuracy of brief cognitive instruments

Significant correlations were found between all instruments (Table S1). The highest correlations were between BASIC and BASIC‐Q, and between MMSE and MoCA, whereas the correlations between CDT‐6 and the other instruments were low. The classification accuracy statistics of the instruments are reported in Table 6 and illustrated in Figure 1.

TABLE 6 Classification accuracy of brief cognitive instruments for cognitive impairment versus normal cognition.

Instrument (score range)	Optimal cutoff	Sensitivity (95% CI)	Specificity (95% CI)	AUC (95% CI)	p	
BASIC (0–25)	19/20	0.72 (0.62–0.80)	0.86 (0.79–0.91)	0.88 (0.84–0.92) a	<0.0005	
BASIC‐Q (0–20)	16/17	0.80 (0.72–0.87)	0.71 (0.63–0.78)	0.84 (0.79–0.89) a	<0.0005	
MMSE (0–30)	27/28	0.71 (0.61–0.80)	0.87 (0.80–0.92)	0.85 (0.80–0.90) a	<0.0005	
MoCA (0–30)	23/24	0.76 (0.64–0.85)	0.79 (0.72–0.85)	0.85 (0.80–0.90) b	<0.0005	
Mini‐Cog (0–5)	3/4	0.58 (0.45–0.69)	0.93 (0.88–0.97)	0.78 (0.71–0.85) c	<0.0005	
CDT‐6 (0–6)	5/6	0.59 (0.47–0.70)	0.63 (0.55–0.71)	0.63 (0.56–0.70) c	0.001	
RUDAS (0–30)	25/26	0.63 (0.49–0.74)	0.82 (0.75–0.88)	0.82 (0.75–0.88) d	<0.0005	
Abbreviations: AUC, area under the receiver operating characteristic curve; BASIC, Brief Assessment of Impaired Cognition; BASIC‐Q, Brief Assessment of Impaired Cognition Questionnaire; CDT‐6, 6‐item Clock Drawing Test; CI, confidence interval; MMSE, Mini‐Mental State Examination; MoCA, Montreal Cognitive Assessment; RUDAS, Rowland Universal Dementia Assessment Scale.

a Cognitive impairment (n = 101) vs. normal cognition (n = 154).

b Cognitive impairment (n = 71) vs. normal cognition (n = 150).

c Cognitive impairment (n = 73) vs. normal cognition (n = 150).

d Cognitive impairment (n = 64) vs. normal cognition (n = 137).

FIGURE 1 Receiver operating characteristics (ROCs) of the brief cognitive instruments. BASIC, Brief Assessment of Impaired Cognition; BASIC‐Q, Brief Assessment of Impaired Cognition Questionnaire; CDT‐6, 6‐item Clock Drawing Test; MMSE, Mini‐Mental State Examination; MoCA, Montreal Cognitive Assessment; RUDAS, Rowland Universal Dementia Assessment Scale.

BASIC had the highest overall accuracy (AUC 0.88) followed by MMSE, MoCA, BASIC‐Q and RUDAS (AUCs from 0.85 to 0.82). Mini‐Cog had fair accuracy (AUC 0.78, 95% CI 0.71–0.85), whereas the accuracy of the CDT‐6 was poor (AUC 0.63, 95% CI 0.56–0.70). Pairwise comparisons of BASIC, MMSE, MoCA and RUDAS produced non‐significant results, indicating that the accuracy of these instruments is roughly equivalent in this sample (Table S2). BASIC had significantly higher classification accuracy than the briefer instruments BASIC‐Q (z = 2.79, p = 0.005) and Mini‐Cog (z = 2.19, p = 0.028). The CDT‐6 had significantly lower accuracy than all other instruments (p < 0.0005). All other pairwise comparisons were non‐significant (p > 0.05).

Classification accuracy of brief cognitive instruments in ‘very mild’ and ‘mild to moderate’ cognitive impairment

The comparative classification accuracy analyses were repeated after splitting the cognitively impaired sample into two halves based on RBANS performance (Figure 2; Table S3).

FIGURE 2 Boxplot of BASIC score distribution across three participant groups defined by cognitive status: normal cognition (n = 154), very mild cognitive impairment (n = 49), mild to moderate cognitive impairment (n = 52). The dotted horizontal line represents the optimal cutoff score (19/20) for differentiating between normal cognition and cognitive impairment.

In the ‘very mild’ cognitive impairment versus normal cognition comparison condition all instruments had lower sensitivities and AUCs than in the full sample comparison (Table S4; Figure S3). The overall classification accuracy of MMSE, Mini‐Cog and RUDAS was reduced to a ‘fair’ level (AUCs 0.72 to 0.77) in this comparison. The sensitivity of the MMSE decreased from 0.71 to 0.55 and the AUC decreased from 0.85 to 0.75. In the ‘mild to moderate’ cognitive impairment versus normal cognition comparison condition the accuracy of BASIC, MMSE and MoCA was excellent (AUCs ≥0.90), whereas the accuracy of BASIC‐Q, RUDAS and Mini‐Cog was good (AUCs from 0.87 to 0.88) (Table S5; Figure S4).

Comparative classification accuracy of the memory test components of brief cognitive instruments

Significant correlations were found between all memory tests with the highest correlation (r = 0.460) between BASIC CCMT and RUDAS delayed recall (Table S6). Again, classification accuracy was affected by the comparison condition. In the all cognitive impairment versus normal cognition condition, classification accuracy was fair for all memory test components with AUCs from 0.70 to 0.79 (Table S7). BASIC CCMT had higher specificity than the other memory test components across the comparison conditions (as the normal cognition group is the same across comparisons, specificity values are redundant). In the very mild impairment versus normal cognition BASIC CCMT had very low sensitivity (0.29) and poor overall accuracy (AUC 0.58, 95% CI 0.49–0.68) and pairwise AUC comparisons indicated that BASIC CCMT had significantly lower accuracy than MoCA and RUDAS delayed recall, and marginally significantly lower accuracy than MMSE delayed recall (p = 0.050) (Table S8). In the mild to moderate cognitive impairment versus normal cognition condition, all memory test components had good accuracy (AUCs from 0.80 to 0.84) and pairwise AUC comparisons revealed no significant differences.

DISCUSSION

BASIC is a brief case‐finding instrument combining cognitive tests and questionnaire items. It is free of copyright restrictions for clinical use and non‐commercial research. In this cross‐validation study, the discriminative validity of BASIC in a GP sample was further examined, head‐to‐head comparisons with other widely used brief cognitive instruments were performed and the instruments' memory test components were compared. BASIC had good overall classification accuracy for detection of cognitive impairment in a GP setting, with good specificity and fair sensitivity. As previously reported, the optimal cutoff score was 19/20 and informant report seemed to contribute more to the validity of BASIC than the other components of the instrument [2, 10, 11]. The classification accuracy of BASIC in the present study is lower than in the primary validation study [2, 11] possibly due to overfitting of the item selection model in the primary validation. This is indirectly supported by the fact that, whilst the classification accuracy of BASIC was reduced in the present cross‐validation study, the accuracy of MMSE remained the same. Classification accuracy may also be influenced by differences in reference standards (expert clinical diagnosis versus RBANS performance) or study sample differences.

In the present sample with a relatively high prevalence of cognitive impairment (40%) BASIC had fair to good predictive validity. In settings with lower pretest probability, the PPV may be poor, as illustrated by the presented estimates (Table 5). In a low pretest probability setting, a more conservative cutoff score of 18/19 or even 17/18 may be considered to counteract attenuation of the PPV. Although a lower cutoff score will reduce the number of identified cases whilst increasing the probability that these cases are truly impaired, the PPV was suboptimal, and in a low pretest probability setting a substantial proportion of patients with a score below cutoff may not be truly cognitively impaired. Pretest probability of cognitive impairment was focused on, rather than prevalence, as the use of case‐finding instruments in GP settings is prompted by a suspicion of cognitive impairment, increasing pretest probability over prevalence—contrary to a screening situation where prevalence determines pretest probability.

Assuming that the classification accuracy of a cognitive instrument is associated with its administration time, MMSE, RUDAS (approximately 10 min each) and MoCA (approximately 15 min) were expected to have higher classification accuracy than BASIC (approximately 5 min). But in this sample the overall classification accuracy of these instruments was roughly equivalent with AUCs from 0.88 to 0.82 (despite some variation in sensitivity and specificity), and pairwise comparisons of the AUCs revealed no significant differences. A previous study comparing the MMSE, MoCA and RUDAS, amongst other tests, found similar classification accuracies (AUCs 0.86–0.89), and pairwise comparisons of AUCs were also non‐significant [34]. In the present study, BASIC had significantly higher accuracy than the even shorter instruments Mini‐Cog and BASIC‐Q, but no significant differences were found between Mini‐Cog, BASIC‐Q, MMSE, MoCA or RUDAS. This was unexpected, considering that both the Mini‐Cog and BASIC‐Q are very brief and simple instruments that can be administered in a few minutes. The CDT‐6, however, demonstrated significantly lower accuracy than all other instruments, with specificity and a sensitivity close to chance level. Consequently, our advice is against using the CDT‐6 as a stand‐alone case‐finding instrument.

Supplementary analyses found that all instruments had relatively poorer AUCs and sensitivity for very mild cognitive impairment (defined ad hoc by splitting the cognitively impaired group in halves) and relatively better AUCs and sensitivity for mild to moderate impairment. In the very mild cognitive impairment comparison condition, MoCA, which was designed for identification of MCI [20], did not perform significantly better than other instruments except for the CDT‐6. In the mild to moderate cognitive impairment comparison, all instruments except for the CDT‐6 had good to excellent classification accuracy.

The memory test components of BASIC, MMSE, MoCA and RUDAS all had fair classification accuracy in the all cognitive impairment comparison versus normal cognition condition, and good accuracy in the mild to moderate cognitive impairment comparison. BASIC CCMT had higher specificity than the other memory tests across the comparison conditions, but very poor sensitivity in the very mild impairment comparison condition. The optimal cutoff score for CCMT was 7/8 indicating a ceiling effect, whereas the optimal cutoff for MoCA delayed recall was 1/2 indicating a floor effect. The CCMT is inspired by the picture‐based memory impairment screen [5] and the memory impairment screen (MIS) [35], both of which were previously reported to have excellent classification accuracy for dementia versus normal cognition. It has been proposed that the use of ‘controlled learning’, applying the same category cues at acquisition and retrieval, may optimize encoding specificity and possibly improve the discriminative validity of a memory test [36, 37]. This proposition was not immediately supported by the results of the present study, but comparison with previous validation studies reveals important sampling differences as they included participants with substantially more pronounced cognitive impairment [5, 35]. A previous study found that the MIS had significantly higher classification accuracy than the memory test components of MMSE and MoCA, whereas the AUCs of the MIS and the RUDAS were identical [34]. Again, sampling differences may explain some of the discrepancies between the previous and the present study. To sum up, the results across the studies indicate that memory tests using category cuing may have relatively higher accuracy than traditional memory tests for case‐finding of dementia, but BASIC CCMT with only four items has a ceiling effect.

Amongst the strengths of the present study is the fact that the study population was recruited from GP clinics in four out of five administrative regions in Denmark, representing both urban and rural residents. Also, head‐to‐head comparisons of BASIC to other widely used brief cognitive instruments could be performed.

Amongst the limitations of the study are (i) the reference standard for classification of cognitive status. Participants did not receive a clinical diagnostic evaluation by medical dementia specialists (including, for example, brain imaging or biomarker status) and were not assigned an aetiological diagnosis, when relevant. Instead, they were classified by an algorithm combining neuropsychological test performance (RBANS), reported IADL functioning and memory concerns. The underlying causes of eventual cognitive impairment remained unknown. (ii) Selection bias was another limitation. The GP physicians included a relatively high proportion of participants with memory concerns, reflected in the 40% cognitive impairment prevalence, whilst also excluding patients who could not complete the comprehensive assessment battery. The exclusion of 20 patients with assumed non‐relevant cognitive dysfunction may have further added to the selection bias. (iii) The administration of tests was not counterbalanced, and performance, for example on memory subtests, may be influenced by fatigue and item interference.

CONCLUSIONS

BASIC appears to be a time‐saving, easy to use and an accurate case‐finding instrument that facilitates early detection of cognitive impairment in GP settings and other settings. BASIC should only be used if cognitive impairment is suspected as its predictive validity is limited when the pretest probability of cognitive impairment is low. Head‐to‐head comparisons of BASIC with MMSE, MoCA and RUDAS revealed roughly equivalent classification accuracy.

AUTHOR CONTRIBUTIONS

Kasper Jørgensen: Conceptualization; methodology; writing – original draft; formal analysis. T. Rune Nielsen: Conceptualization; methodology; formal analysis; writing – original draft. Ann Nielsen: Project administration; funding acquisition; conceptualization. Anne‐Britt Oxbøll: Investigation. Sofie D. Gerner: Investigation; formal analysis. Frans B. Waldorff: Conceptualization; supervision; resources. Gunhild Waldemar: Conceptualization; funding acquisition; supervision.

FUNDING INFORMATION

Ministeriet Sundhed Forebyggelse, Grant/Award Number 1604063. The study funder had no role in study design, collection, analysis or interpretation of data, writing of the manuscript, or the decision to submit for publication.

CONFLICT OF INTEREST STATEMENT

None.

Supporting information

Data S1.

ACKNOWLEDGEMENTS

All study participants are sincerely thanked for dedicating their time and effort to the project and the general practice staff at the 14 involved clinics for recruiting and examining the study participants. The Danish Ministry of Health is also thanked for study funding.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
==== Refs
REFERENCES

1 Woods RT , Moniz‐Cook E , Iliffe S , et al. Dementia: issues in early recognition and intervention in primary care. J R Soc Med. 2003;96 (7 ):320‐324. doi:10.1177/014107680309600703 12835442
2 Jorgensen K , Nielsen TR , Nielsen A , et al. Brief assessment of impaired cognition (BASIC)—validation of a new dementia case‐finding instrument integrating cognitive assessment with patient and informant report. Int J Geriatr Psychiatry. 2019;34 (11 ):1724‐1733. doi:10.1002/gps.5188 31389089
3 Brodaty H , Pond D , Kemp NM , et al. The GPCOG: a new screening test for dementia designed for general practice. J Am Geriatr Soc. 2002;50 (3 ):530‐534. doi:10.1046/j.1532-5415.2002.50122.x 11943052
4 Ehrensperger MM , Taylor KI , Berres M , et al. BrainCheck—a very brief tool to detect incipient cognitive decline: optimized case‐finding combining patient‐ and informant‐based data. Alzheimers Res Ther. 2014;6 (9 ):69. doi:10.1186/s13195-014-0069-y 25422675
5 Verghese J , Noone ML , Johnson B , et al. Picture‐based memory impairment screen for dementia. J Am Geriatr Soc. 2012;60 (11 ):2116‐2120. doi:10.1111/j.1532-5415.2012.04191.x 23039180
6 Amariglio RE , Donohue MC , Marshall GA , et al. Tracking early decline in cognitive function in older individuals at risk for Alzheimer disease dementia: the Alzheimer's disease cooperative study cognitive function instrument. JAMA Neurol. 2015;72 (4 ):446‐454. doi:10.1001/jamaneurol.2014.3375 25706191
7 Jorm AF , Korten AE . Assessment of cognitive decline in the elderly by informant interview. Br J Psychiatry. 1988;152 :209‐213. doi:10.1192/bjp.152.2.209 3167337
8 Matias‐Guiu JA , Delgado‐Alvarez A . Novel cognitive screening tests to address new clinical priorities and cultural diversity. Eur J Neurol. 2023;30 (4 ):799‐801. doi:10.1111/ene.15653 36445003
9 Yuan XF , Zhang Y , Zhang Y , et al. Reliability and validity of the brief assessment of impaired cognition (Chinese version) for stroke patients. Appl Neuropsychol Adult. 2023;30 (1 ):27‐33. doi:10.1080/23279095.2021.1908287 33879014
10 Jorgensen K , Nielsen TR , Nielsen A , Waldorff FB , Waldemar G . Validation of the brief assessment of impaired cognition and the brief assessment of impaired cognition questionnaire for identification of mild cognitive impairment in a memory clinic setting. Int J Geriatr Psychiatry. 2020;35 (8 ):907‐915. doi:10.1002/gps.5312 32291812
11 Jorgensen K , Nielsen TR , Nielsen A , et al. Validity of the brief assessment of impaired cognition case‐finding instrument for identification of dementia subgroups and staging of dementia. Eur J Neurol. 2023;30 (3 ):578‐586. doi:10.1111/ene.15636 36380695
12 Oxboll AB , Jorgensen K , Nielsen TR , et al. Diagnostic accuracy of BASIC‐Q for detection of cognitive impairment in a primary care setting—a cross‐validation study. BMC Geriatr. 2024;24 (1 ):53. doi:10.1186/s12877-024-04675-1 38212693
13 Buderer NM . Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med. 1996;3 (9 ):895‐900. doi:10.1111/j.1553-2712.1996.tb03538.x 8870764
14 Waldorff FB , Siersma V , Vogel A , Waldemar G . Subjective memory complaints in general practice predicts future dementia: a 4‐year follow‐up study. Int J Geriatr Psychiatry. 2012;27 (11 ):1180‐1188. doi:10.1002/gps.3765 22253004
15 Pfeffer RI , Kurosaki TT , Harrah CH Jr , Chance JM , Filos S . Measurement of functional activities in older adults in the community. J Gerontol. 1982;37 (3 ):323‐329. doi:10.1093/geronj/37.3.323 7069156
16 Classification on education (DISCED‐15) . Current educations, V1:2023. 2024 Accessed 02‐14‐2024. https://www.dst.dk/en/Statistik/dokumentation/nomenklaturer/disced15‐audd
17 Randolph C . Repeatable Battery for the Assessment of Neuropsychological Status ‐ Manual. Danish Version. [Repeatable Battery for the Assessment of Neuropsychological Status ‐ Vejledning. Dansk version]. NCS Pearson; 2013.
18 Folstein MF , Folstein SE , McHugh PR . ‘Mini‐mental state’. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12 (3 ):189‐198. doi:10.1016/0022-3956(75)90026-6 1202204
19 Yesavage JA , Brink TL , Rose TL , et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res. 1982;17 (1 ):37‐49. doi:10.1016/0022-3956(82)90033-4 7183759
20 Nasreddine ZS , Phillips NA , Bedirian V , et al. The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53 (4 ):695‐699. doi:10.1111/j.1532-5415.2005.53221.x 15817019
21 Storey JE , Rowland JT , Basic D , Conforti DA , Dickson HG . The Rowland universal dementia assessment scale (RUDAS): a multicultural cognitive assessment scale. Int Psychogeriatr. 2004;16 (1 ):13‐31. doi:10.1017/s1041610204000043 15190994
22 Jorgensen K , Kristensen MK , Waldemar G , Vogel A . The six‐item clock drawing test—reliability and validity in mild Alzheimer's disease. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2015;22 (3 ):301‐311. doi:10.1080/13825585.2014.932325 24974730
23 Borson S , Scanlan J , Brush M , Vitaliano P , Dokmak A . The mini‐cog: a cognitive ‘vital signs’ measure for dementia screening in multi‐lingual elderly. Int J Geriatr Psychiatry. 2000;15 (11 ):1021‐1027.11113982
24 Jorgensen K , Nielsen TR , Nielsen A , Waldorff FB , Waldemar G . Brief assessment of impaired cognition questionnaire (BASIC‐Q)—development and validation of a new tool for identification of cognitive impairment in community settings. Int J Geriatr Psychiatry. 2020;35 (7 ):693‐701. doi:10.1002/gps.5286 32100328
25 Sylvestre G , Mauny F , Lavaux M , et al. The RAPID neuropsychological test battery for subjects aged 50–89: from norms to cognitive profile. Eur Neurol. 2017;78 (5–6 ):247‐256. doi:10.1159/000481090 28942453
26 Gunner JH , Miele AS , Lynch JK , McCaffrey RJ . Performance of non‐neurological older adults on the Wisconsin card sorting test and the Stroop color–word test: normal variability or cognitive impairment? Arch Clin Neuropsychol. 2012;27 (4 ):398‐405. doi:10.1093/arclin/acs047 22591916
27 Teng E , Becker BW , Woo E , Knopman DS , Cummings JL , Lu PH . Utility of the functional activities questionnaire for distinguishing mild cognitive impairment from very mild Alzheimer disease. Alzheimer Dis Assoc Disord. 2010;24 (4 ):348‐353. doi:10.1097/WAD.0b013e3181e2fc84 20592580
28 Cruz‐Orduna I , Bellon JM , Torrero P , et al. Detecting MCI and dementia in primary care: effectiveness of the MMS, the FAQ and the IQCODE [corrected]. Fam Pract. 2012;29 (4 ):401‐406. doi:10.1093/fampra/cmr114 22121012
29 Hedges LV . Distribution theory for Glass's estimator of effect size and related estimators. J Educ Stat. 1981;6 (2 ):107‐208.
30 Youden WJ . Index for rating diagnostic tests. Cancer. 1950;3 (1 ):32‐35.15405679
31 Muller MP , Tomlinson G , Marrie TJ , et al. Can routine laboratory tests discriminate between severe acute respiratory syndrome and other causes of community‐acquired pneumonia? Clin Infect Dis. 2005;40 (8 ):1079‐1086. doi:10.1086/428577 15791504
32 Crawford JR , Garthwaite PH , Betkowska K . Bayes' theorem and diagnostic tests in neuropsychology: interval estimates for post‐test probabilities. Clin Neuropsychol. 2009;23 (4 ):624‐644. doi:10.1080/13854040802524229 19235634
33 Vassarstats . 2024. Accessed 02‐13‐2024. http://vassarstats.net/
34 Matias‐Guiu JA , Valles‐Salgado M , Rognoni T , Hamre‐Gil F , Moreno‐Ramos T , Matias‐Guiu J . Comparative diagnostic accuracy of the ACE‐III, MIS, MMSE, MoCA, and RUDAS for screening of Alzheimer disease. Dement Geriatr Cogn Disord. 2017;43 (5–6 ):237‐246. doi:10.1159/000469658 28384640
35 Buschke H , Kuslansky G , Katz M , et al. Screening for dementia with the memory impairment screen. Neurology. 1999;52 (2 ):231‐238. doi:10.1212/wnl.52.2.231 9932936
36 Buschke H , Sliwinski M , Kuslansky G , Lipton RB . Aging, encoding specificity, and memory change in the double memory test. J Int Neuropsychol Soc. 1995;1 (5 ):483‐493. doi:10.1017/s1355617700000576 9375233
37 Buschke H , Sliwinski MJ , Kuslansky G , Lipton RB . Diagnosis of early dementia by the double memory test: encoding specificity improves diagnostic sensitivity and specificity. Neurology. 1997;48 (4 ):989‐997. doi:10.1212/wnl.48.4.989 9109889
