
==== Front
eClinicalMedicine
EClinicalMedicine
eClinicalMedicine
2589-5370
Elsevier

S2589-5370(24)00392-4
10.1016/j.eclinm.2024.102813
102813
Articles
Asthma diagnosis: a comparison of established diagnostic guidelines in adults with respiratory symptoms
Simpson Andrew J. a
Drake Sarah b
Healy Laura b
Wang Ran bc
Bennett Miriam c
Wardman Hannah c
Durrington Hannah bc
Fowler Stephen J. bc
Murray Clare S. bc
Simpson Angela angela.simpson@manchester.ac.uk
bc∗
a School of Sport, Exercise and Rehabilitation Sciences, University of Hull, United Kingdom
b Manchester University NHS Foundation Trust, United Kingdom
c Division of Immunology, Immunity to Infection and Respiratory Medicine at the University of Manchester, School of Biological Sciences, The University of Manchester, Manchester Academic Health Science Centre, United Kingdom
∗ Corresponding author. Division of Immunology, Immunity to Infection and Respiratory Medicine at the University of Manchester, School of Biological Sciences, The University of Manchester, M13 9PL, United Kingdom. angela.simpson@manchester.ac.uk
05 9 2024
10 2024
05 9 2024
76 10281318 6 2024
2 8 2024
19 8 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Summary

Background

Considerable variability exists between asthma diagnostic guidelines. We tested the performance characteristics of the European Respiratory Society (ERS), the National Institute for Health and Care Excellence (NICE) and the Global Initiative for Asthma (GINA) guidelines for the diagnosis of asthma in adults.

Methods

In this prospective observational study (ISRCTN—11676160, May 2019–June 2022), participants referred from primary care with clinician-suspected asthma underwent comprehensive investigation including: spirometry, bronchodilator reversibility, fractional exhaled nitric oxide, peak expiratory flow variability, bronchial challenge testing with methacholine and mannitol, and responsiveness to inhaled corticosteroid therapy. Results were reviewed by a panel of asthma specialists to determine asthma diagnosis (reference standard) and compared to each diagnostic test and the ERS, NICE and GINA diagnostic algorithms (index tests). The sensitivity, specificity, positive predictive and negative predictive values were calculated.

Findings

One hundred and forty adults were enrolled and 118 given a definitive diagnostic outcome [75 female; mean (SD) age 36 (12) years; 70 (59%) with asthma] and included in the analysis. Sensitivity of individual tests was poor (15–62%), but they provided good specificity at the most stringent thresholds (range: 88–100%). The sensitivity/specificity of ERS, NICE and GINA was 81/85%, 41/100% and 47/100%, respectively. Concordance between guidelines was only moderate (Cohen’s Kappa 0.45–0.51).

Interpretation

Current guidelines for the diagnosis of asthma in adults provide either excellent specificity but low sensitivity (GINA and NICE) or only reasonable sensitivity and specificity (ERS). All guidelines therefore have limitations with regards to their clinical application; new guidelines are needed but should be tested prospectively before roll out.

Funding

This work was supported by the 10.13039/100014653 Manchester NIHR Biomedical Research Centre (BRC) (grant no. BRC-1215-20007 , and NIHR203308 ), 10.13039/501100000362 Asthma UK /Innovate (grant no. AUK-PG-2018-406 ), GSK ID 212474 and North West Lung Centre Charity.

Keywords

Asthma
Diagnosis
Guidelines
==== Body
pmc Research in context

Evidence before this study

There are a growing number of national and international guidelines for the diagnosis of asthma, with European Respiratory Society (ERS) guidelines recently added to those produced by the National Institute of Health and Care Excellence (NICE, UK) and the Global Initiative for Asthma (GINA). In an analysis of preliminary data done to assess the performance of non-aerosol generating procedures during the COVID-19 pandemic, we found that GINA (2020) and NICE performed with good specificity but poor sensitivity. We have performed a systematic review of the literature (across EMBASE, Medline and PubMed), including manuscripts whereby the primary focus of the work was on how ERS/GINA/NICE guidelines perform diagnostically in adults with suspected asthma, excluding opinion and review articles. Search terms in brief: “asthma” ADJ2 “diagnos∗”) AND (“performance characteristic∗” OR “diagnos∗criteria” OR “guideline” OR “algorithm” OR “flow chart”) AND (“GINA” OR “ERS” OR “NICE”); limited to publication since 2017 (publication of the NICE guideline, the oldest of interest). No language limitations were used. The search returned 179 (non-duplicate) titles and abstracts, of which two were assessed as being possibly relevant, but published only in abstract form (conference proceedings) and with insufficient methodological detail available to assess the robustness of the reference standard or tests performed. We therefore aimed to test the performance of such guidelines in practice, that is their ability to accurately diagnose asthma.

Added value of this study

We have reported the key performance characteristics of three major guidelines—ERS, NICE and GINA—as well as the characteristics of key tests that make up the diagnostic pathways such as spirometry, bronchodilator reversibility, peak flow variability, fractional exhaled nitric oxide, and bronchial challenge, at the varied cut-offs used across the guidelines. In 140 adults with suspected (but untreated) asthma both NICE and GINA performed with high (100%) specificity but relatively poor (less than 50%) sensitivity; ERS performed with a more balanced sensitivity/specificity (81%/85%) but still resulted in 1-in-6 patients being misdiagnosed. We present how each diagnostic test performs, which could be used to inform new diagnostic pathways.

Implications of all the available evidence

Our data allows clinicians to compare the effectiveness of current major guidelines for the diagnosis of asthma and influence how they utilise these in their current practice. Our findings suggest that all three pathways require significant refinement and subsequent testing to reduce misdiagnosis.

Introduction

Asthma affects around 300 million people worldwide, and in the UK 12% of the population have been diagnosed with the disease.1 Asthma is characterised by symptoms of wheeze, chest tightness, breathlessness and cough, typically associated with variable airflow obstruction and inflammation. There are however no gold standard diagnostic criteria for asthma, likely reflecting the heterogenous nature of the condition which encompasses different phenotypes (reflecting different underlying pathophysiology), particularly in adults.2 Given this phenotypic heterogeneity, it is inevitable that there are multiple pathways to an asthma diagnosis, incorporating different features measured with different tests.

Asthma misdiagnosis is common, reported by Aaron et al.3 in up to one third of subjects investigated within five years of initial diagnosis, more than half of whom had not undergone any objective testing at the time of diagnosis. Asthma symptoms are non-specific,4 so it is important to complete confirmatory diagnostic tests to minimise costs and risks associated with unnecessary treatment, and to offer the opportunity to make the correct diagnosis. These should be done prior to starting inhaled corticosteroids (ICS), as treatment can change results.5

The most frequently used diagnostic pathways for asthma are produced by the European Respiratory Society (ERS; Guidelines for the Diagnosis of Asthma in Adults 2022),6 the National Institute for Health and Care Excellence (NICE; Asthma: diagnosis, monitoring and chronic asthma management 2017)4 and the Global Initiative for Asthma (GINA; Global strategy for asthma management and prevention 2023).5 There were methodological differences in the development of the guidelines.7 However, these pathways show considerable variability in the utility of different tests, their thresholds, and the order in which they are conducted. None of these pathways has been tested prospectively in symptomatic adults not taking ICS.

The aim of this study was therefore to test the performance characteristics of the ERS, NICE, and GINA asthma diagnostic algorithms, in adults with symptoms in keeping with asthma but not on ICS, in the setting of our Rapid Access Diagnostics in Asthma research clinic (RADicA).

Methods

Study design

The RADicA clinic (ISRCTN 11676160) prospectively evaluated adults (and children, not included in this report) with clinician-suspected asthma (symptoms of wheeze, chest tightness, cough, and/or breathlessness), in Manchester, UK.

Participants

Potential participants were referred between May 2019 and June 2022 primarily by general practitioners in Greater Manchester, having presented with symptoms in keeping with asthma. Exclusion criteria: age >70 years; oral corticosteroids within 4 weeks; ICS within 2 weeks (this was subsequently amended to 4 weeks and none of the adults recruited had ICS within 4 weeks of the first visit); antibiotics within 2 weeks; greater than 10 pack year smoking history; other significant lung disease. At recruitment, a structured clinical history and physical examination were completed; participants with a suspected alternative diagnosis more likely than asthma were withdrawn. Remaining participants completed a series of diagnostic tests over a two-week period before starting ICS (Flixotide Accuhaler, 250 mcg twice daily) for a period of 10 ± 4 weeks8 after which time diagnostic tests were repeated (Supplementary Table E1). The study protocol was approved by the Local Research Ethics Committee (18/NW/0777); participants gave written informed consent.

Diagnostic testing

Spirometry (Forced Expiratory Volume in 1 s [FEV1] and Forced Vital Capacity [FVC]) were measured (JAEGER Vyntus PNEUMO, Vyaire Medical, Basingstoke, UK) according to American Thoracic Society/European Respiratory Society (ATS/ERS) guidelines (full details in Supplementary Appendix).9 For bronchodilator reversibility (BDR), spirometry was repeated 15 min following 400 μg of inhaled salbutamol. BDR was defined as FEV1 increase >12% and >200 mL, unless otherwise stated. Peak expiratory flow variability (PEFv) was recorded using an eMini Wright digital flow meter (Clement Clarke Ltd, Harlow, UK) twice daily (morning and evening) over a 2-week period. Fractional exhaled nitric oxide (FeNO) was measured according to manufacturer’s instructions (NIOX VERO, Circassia, Oxford, UK) and international recommendations,10 at a standardised flow rate (50 mL/s). Methacholine bronchial challenge test (BCTmeth) was performed (Vyntus APS Nebulizer system with integrated dosimeter; Vyaire Medical), with a 4-dose protocol (cumulative dose: 0.015, 0.060, 0.240, 0.960 mg) using single concentration methacholine (16 mg/mL, Stockport Pharmaceuticals, Stockport, UK), according to international guidelines.11 The dose provoking a 20% fall in FEV1 was recorded (PD20); PD20 ≤ 0.2 mg was defined as positive challenge.11 Mannitol bronchial challenge test (BCTmann) was conducted in accordance with the manufacturer’s instructions (Aridol, Pharmaxis) and international guidelines.12 Following baseline spirometry, participants inhaled doubling doses of mannitol starting from 5 mg, until a reduction in FEV1 of 15% was recorded or the maximum cumulative dose (635 mg) administered. The provocative dose causing a reduction in FEV1 of 15% (PD15) was derived from the dose–response curve. Skin prick tests (SPT) were performed to a panel of inhalant allergens. Asthma control was recorded using the Asthma Control Questionnaire (ACQ).13 Blood eosinophils were measured (Sysmex Ltd, Milton Keynes, UK).

Index tests

The ERS, NICE and GINA diagnostic algorithms were investigated as index tests against the reference standard. Where a test result was missing, this was assigned a negative result so the algorithm could be completed. In addition, analysis has been completed where those with missing tests are excluded if it is not possible to complete the algorithm (shown in the Supplementary Appendix).

The ERS algorithm (2022)6 proposes four pathways that lead to an asthma diagnosis (Supplementary Fig. E1). (1) FEV1/FVC <75% and BDR with FEV1 reversibility >12% and 200 mL; (2) FeNO >50 ppb; (3) PEFv >20%; (4) BCTmeth PD20 < 0.2 mg. Each participant’s test results were put through the four pathways in series, until a diagnosis of asthma was confirmed or ruled out.

NICE guideline NG804 comprises 15 possible pathways (Supplementary Fig. E2). Data from each participant were put through the sequential algorithm using the following cut-offs: spirometry, FEV1 < 70% predicted (or LLN, shown in Supplementary Appendix); BDR ≥12% and ≥200 mL; FeNO, ≥40 ppb, or ≤ 39 and ≥ 25 ppb; PEFv, >20%; BCTmeth, PD20 ≤ 0.2 mg. Those who reached a diagnosis of asthma (pathways 1–6) were recorded as NICE-defined asthma. In a separate analysis (shown in Supplementary Appendix) we also included those categorised as inconclusive (Suspect Asthma and review after treatment, pathways 7–9) with ‘NICE-defined asthma’. All others, who followed one of six possible alternate pathways (pathways 10–15), were diagnosed as NICE-“not asthma”, as the recommendation here is to “Consider alternative diagnoses or referral for a second opinion”, and not to start treatment.4

GINA diagnostic guidance5 (Supplementary Fig. E3) states that asthma should only be diagnosed in patients with obstructed spirometry (FEV1/FVC ratio < LLN at any visit) AND evidence of variable airflow from at least one of the following: (1) BDR ≥12% and ≥200 mL (2) PEFv >10%, (3) increase in FEV1 >12% and 200 mL after 4 weeks of ICS treatment, (4) positive challenge test (BCTmeth PD20 < 0.2 mg and/or BCTmann PD15 < 635 mg or exercise challenge test (not available in RADicA), or (5) variation of FEV1 > 12% and 200 mL between pre-treatment visits. Remaining participants were recorded as “not asthma”.

Reference standard: asthma diagnosis by expert panel objective evidence review (EPOER)

The reference standard against which the published diagnostic algorithms were tested for accuracy was the expert panel objective evidence review (EPOER). All evidence, including history, physical examination, ACQ, and all test results before and after ICS (Supplementary Table E1), was reviewed by at least three physicians (a minimum of two senior asthma physicians) with a diagnosis reached by consensus after open discussion. Index test data were therefore available to the assessors of the reference standard. Not all participants completed all aspects of the study, but all evaluable data were assessed including raw data (such as flow volume loops, dose–response curves, peak flow diaries), to take account of uncertainty and inherent biological variability.14 EPOER assigned a diagnosis of “asthma” or “not asthma”; where this was not possible, participants were assigned “possible asthma” or “insufficient evidence” and excluded from further analysis.

Statistical analysis

Participant characteristics and results were tested for normality; box plots showing the distribution of key lung function measures are shown in Supplementary Fig. E4. Results from the individual diagnostics tests were dichotomised as positive or negative, according to the thresholds from each diagnostic guideline. Dichotomised variables were run through each respective diagnostic algorithm to establish a guideline-specific asthma diagnosis (index tests). The principal analysis was descriptive. Each individual diagnostic test, and each index test were compared to the reference standard using cross tabulation. The sensitivity, specificity, positive predictive value and negative predictive value were calculated. Index tests were compared to each other using Cohen’s kappa statistic. Confidence intervals for sensitivity and specificity are exact Clopper-Pearson confidence intervals. Data analysis was supported with SPSS (SPSS 25, IBM, NY), with significance set a P < 0.05.

Role of the funding source

The funders of the study had no role in study design, data collection, data analysis, data interpretation or writing of the article.

Results

Participant characteristics

One hundred and forty adults with symptoms in keeping with asthma were enrolled into the RADicA study; two were excluded as they had another diagnosis more likely than asthma (Supplementary Fig. E5). The remaining 138 underwent expert panel evaluation of all available data (EPOER, reference standard) at the end of each participant’s involvement in the study; 17 with insufficient evidence and three with possible asthma were excluded from further analysis. Of the 118 adults remaining, 70 (59%) had asthma and 48 (41%) did not have asthma.

Participant characteristics are presented in Table 1. Most reported symptoms for several years (median 3 years, range 2 months to over 40 years). All four symptoms of asthma were common in both groups. Approximately one quarter had previously tried ICS, but for most this treatment had been discontinued for over 2 years, and all had discontinued at least 1 month before enrolment.Table 1 Participant characteristics, for all cases and expert panel objective evidence review (EPOER)-defined “asthma” and “not asthma”.

	All cases N = 118	Asthma N = 70	Not asthma N = 48	
Clinical and demographic features:	
 Age, mean (SD) years	36 (12)	34 (11)	38 (13)	
 Gender, n (%) females	75 (64)	40 (57)	35 (73)	
 Current or ex-smokers, n (%)	40 (35)	21 (30)	19 (40)	
 Pack years, median (IQR)	0.0 (0.0–1.0)	0.0 (0.0–1.0)	0.00 (0.0–1.0)	
 BMI, mean (SD) kg/m2	28 (5)	28 (5)	28 (6)	
 Wheeze, n (%)	97 (82)	64 (91)	33 (69)	
 Cough without a cold, n (%)	94 (80)	56 (80)	38 (79)	
 Breathlessness, n (%)	103 (87)	62 (89)	41 (85)	
 Chest tightness, n (%)	96 (81)	61 (87)	35 (73)	
 Duration symptoms, median (IQR) years	3.0 (1.1–7.6)	2.4 (1.0–7.8)	3.4 (1.7-7.4)	
 Current salbutamol use, n (%)	78 (66)	54 (77)	24 (50)	
 Previous ICS use	31 (26)	18 (26)	13 (27)	
 Number of months off ICS for those that had used (n = 31), median (IQR)	24 (4–168)	96 (6–228)	12 (2–42)	
Physiological data:	
 FEV1, mean (SD) L	3.28 (0.94)	3.23 (0.99)	3.36 (0.87)	
 FEV1, mean (SD) %predicted	94 (16)	89 (17)	101 (14)	
 FVC, mean (SD) L	4.28 (1.24)	4.37 (1.32)	4.14 (1.12)	
 FVC, mean (SD) % predicted	101 (15)	100 (15)	103 (15)	
 FEV1/FVC ratio, mean (SD) %	77 (8)	74 (9)	82 (6)	
 BDR, median (IQR) %	6.8 (3.1–12.0)	9.8 (5.7–14.6)	3.7 (1.6-6.2)	
 BCTmeth PD20 (n = 98), median (IQR) ug	1920 (75–1920)	83 (26–53)	1920 (1920–1920)	
 FeNO (n = 118), median (IQR) ppb	25 (13–72)	57 (23–92)	15 (11–22)	
 PEFv (n = 96), median (IQR) %a	9.0 (4.3–12.8)	11.1 (6.7–16.4)	5.6 (3.4–10.3)	
 Eos (n = 114), median (IQR) ×109 cells/L	0.19 (0.1–0.33)	0.29 (0.16–0.51)	0.11 (0.08–0.20)	
 Eos, (n = 114) n (%) >0.4 × 109 cells/L	23 (20)	22 (33)	1 (2)	
 Sensitised (n = 115), n (%) ≥1 SPT allergen positive	75 (65)	52 (75)	23 (50)	
 Auscultated wheeze, n (%)	12 (10%)	12 (17%)	0 (0%)	
BMI, body mass index; FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; BDR, bronchodilator reversibility; BCTmeth, methacholine bronchial challenge test; FeNO, fractional exhaled nitric oxide; PEFv, peak expiratory flow variability; Eos, eosinophil levels; SPT, skin prick test.

a Minimum of 5 days of PEF recordings required for valid test.

The performance of individual tests used in asthma diagnosis is presented in Supplementary Table E2, with thresholds used for each guideline presented separately. Although the specificity for obstructive spirometry (96% if <70% or LLN), BDR (100%), and PEFv at 20% (97%) was high, making them good rule in tests, they were positive in less than half of those with asthma, dropping as low as 15% for PEFv. Bronchial challenge tests were also highly specific but were still only positive in approximately two thirds of subjects subsequently diagnosed with asthma. Although generally more sensitive (51%), FeNO had a poorer specificity for asthma diagnosis, reaching only 88% at the higher threshold of 50 ppb.

ERS guideline

Of 118 participants, 23 (19%) were diagnosed with asthma following ERS pathway 1 (FEV1/FVC <75% and BDR), all of whom had EPOER asthma (Table 2). A further 27 were diagnosed with asthma following ERS pathway 2 (FeNO >50 ppb), of whom 21 (78%) had EPOER asthma. The inclusion of PEFv >20% (ERS-defined asthma pathway 3) only identified an additional three participants as having asthma, two of whom also had EPOER-defined asthma. From the remaining 65 participants, a positive methacholine challenge (ERS pathway 4) identified asthma in 11 cases, all in agreement with EPOER. The remaining 54 (46%) did not have asthma according to the ERS pathway; however, 13 (24%) of these participants were given a diagnosis of EPOER asthma. For illustrative purposes, test results for these 13 subjects are presented in Supplementary Table E3. When missing values were not assigned a negative value, results are shown in Supplementary Table E4.Table 2 ERS pathways for asthma diagnosis and concordance to expert panel objective evidence review (EPOER).

ERS pathway	FEV1/FVC <75%	BDR FEV1 ≥12% and 200 mL	FeNO >50 ppb	PEFv >20%a	BCTmeth PD20 < 0.200 mg	Positive on thispathway, n	Reference standard concordance, n (%)	
Asthma pathway 1 (n = 118)	✓	✓	NA	NA	NA	23	23 (100%)	
Asthma pathway 2 (n = 95)	✗ or ✓	NA or ✗	✓	NA	NA	27	21 (78%)	
Asthma pathway 3 (n = 68)	✗ or ✓	NA or ✗	✗	✓	NA	3	2 (67%)	
Asthma pathway 4 (n = 65)	✗ or ✓	NA or ✗	✗	✗	✓	11	11 (100%)	
ERS defined asthma (n = 118)	Any of the above pathways					64	57 (89%)	
ERS defined not asthma	None of the above pathways					54	41 (76%)	
Pathways were followed sequentially, such that once asthma had been diagnosed the individual was removed from the pool of participants to be tested on the next path; consequently numbers tested on subsequent pathways 2–4 reduces.

✗ indicates test was done and result was negative, ✓ indicates test was positive, NA indicates test is not included in this pathway and so the result was not considered.

Abbreviations: FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; BDR, bronchodilator reversibility; FeNO, fractional exhaled nitric oxide; PEFv, peak expiratory flow variability; BCTmeth, methacholine bronchial challenge test; PD, provoking dose.

a Minimum of 5 days of PEF recordings required for valid test.

NICE guideline

The number of participants following each of the 15 possible pathways together with concordance with reference standard, EPOER is shown in Table 3. Three pathways were not followed by any participants (pathways 1, 5 and 7). Asthma Pathways 2,3,4 and 6 were followed by 29 participants, all of whom also received an EPOER asthma diagnosis. Five participants followed a “suspect asthma” inconclusive pathway (pathways 7–9) all of whom had asthma confirmed by EPOER. The remaining 84 participants followed pathways 10–15, where asthma is not diagnosed, and treatment not started. Here, results were much less concordant with the EPOER diagnosis. Pathway 10 (no airflow obstruction, FeNO <40, PEFv <20%, BCT and BDR not done) was by far the most commonly followed pathway (n = 60) but had a concordance with EPOER of only 63%. Of the 60 following this pathway, 22 were found to have asthma by EPOER including 10 with a positive BCT and 5 with BDR (tests not included in pathway 10), making this pathway the one resulting in the highest number of missed diagnoses. When missing values were not assigned a negative value, results are shown in Supplementary Table E5. When obstructed spirometry was defined as FEV1/FVC < LLN results are shown in Supplementary Table E6.Table 3 NICE guideline pathways for asthma diagnosis and concordance to expert panel objective evidence review (EPOER).

NICE pathway	FEV1/FVC <70%	BDR FEV1 ≥12% and 200 mL	FeNO ≥40 ppb	FeNO ≤39 ≥ 25 ppb	PEFv ≥20%a	BCTmeth <200ug	Following pathway, n	Reference standard concordance, n (%)	
Asthma 1	✗	NA	✗	NA	✓	✓	0		
Asthma 2	✗	NA	✓	NA	✗	✓	13	13 (100%)	
Asthma 3	✗	NA	✓	NA	✓	NA	3	3 (100%)	
Asthma 4	✓	✗	✗	✓	✗	✓	1	1 (100%)	
Asthma 5	✓	✓	✗	NA	✓	NA	0		
Asthma 6	✓	✓	✓	NA	NA	NA	12	12 (100%)	
Suspect asthma/inconclusive 7	✓	✗	✗	✓	✓	NA	0		
Suspect asthma inconclusive 8	✓	✗	✓	NA	NA	NA	4	4 (100%)	
Suspect asthma/inconclusive 9	✓	✓	✗	✓b	✗	NA	1	1 (100%)	
Alternative diagnosis 10	✗	NA	✗	NA	✗	NA	60	38 (63%)	
Alternative diagnosis 11	✗	NA	✗	NA	✓	✗	3	1 (33%)	
Alternative diagnosis 12	✗	NA	✓	NA	✗	✗	16	7 (44%)	
Alternative diagnosis 13	✓	✗	✗	✗	NA	NA	1	1 (100%)	
Alternative diagnosis 14	✓	✗	✗	✓	✗	✗	1	1 (100%)	
Alternative diagnosis 15	✓	✓	✗	✗b	✗	NA	3	0 (0%)	
NICE-defined asthma (pathways 1–6)							29	29 (100%)	
NICE-defined asthma or suspect asthma/inconclusive (pathways 1–9)							34	34 (100%)	
✗ indicates test was done and result was negative, ✓ indicates test was positive, NA indicates test is not included in this pathway and so the result was not considered. Reference standard concordance—findings are concordant if both NICE and EPOER assigned asthma (pathways 1–9), or both assigned not asthma (pathways 10–15).

FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; BDR, bronchodilator reversibility; BCTmeth, methacholine bronchial challenge test; FeNO, fractional exhaled nitric oxide; PEFv, peak expiratory flow variability.

a Minimum of 5 days of PEF recordings required for valid test.

b Indicates test is not performed in the sequence of the table, FeNO ≤ 39 ≥ 25 ppb in this instance is considered after PEFv.

GINA

Expiratory airflow obstruction (FEV1/FVC < LLN at any visit) is a requirement for each of the five pathways that lead to a diagnosis of asthma (Table 4). This test was negative in 82 subjects, 37 of whom had EPOER asthma, and 42 had at least one other positive test (23 BCT positive, 26 had PEFv >10%), but in the absence of expiratory airflow obstruction this was not sufficient to achieve an asthma diagnosis.Table 4 GINA guideline pathways for asthma diagnosis and concordance to expert panel objective evidence review (EPOER).

GINA pathway	FEV1/FVC < LLN	BDR FEV1 > 12% and 200 mL	PEFv ≥10%∗	BCTmeth PD20 < 200ug AND/OR BCTmann PD15 < 635 mg	FEV1 variability >12%	ICS response FEV1 > 12% and 200 mL	Following this pathway, n	Reference standard concordance, n (%)	
Not asthma, not obstructed (n = 118)	✗	NA	NA	NA	NA	NA	82	45 (55%)	
Not asthma, obstructed but other tests all negative (n = 36)	✓	✗	✗	✗	✗	✗	4	3 (75%)	
Asthma pathway 1 (n = 32)	✓	✓	NA	NA	NA	NA	22	22 (100%)	
Asthma pathway 2 (n = 10)	✓	✗	✓	NA	NA	NA	7	7 (100%)	
Asthma pathway 3 (n = 3)	✓	✗	✗	✓	NA	NA	3	3 (100%)	
Asthma pathway 4 (n = 0)	✓	✗	✗	✗	✓	NA	0	–	
Asthma pathway 5 (n = 0)	✓	✗	✗	✗	✗	✓	0	–	
Any asthma pathway (n = 118)	✓	Any asthma pathway					32	32 (100%)	
Pathways were followed sequentially, such that once asthma had been diagnosed the individual was removed from the pool of participants to be tested on the next path; consequently, numbers tested on subsequent pathways reduces. ✗ indicates test was done and result was negative, ✓indicates test was positive, NA indicates test is not included in this pathway and so the result was not considered.

FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; LLN, lower limit of normal; BDR, bronchodilator reversibility; PEFv, peak expiratory flow variability; BCTmeth, methacholine bronchial challenge test; FeNO, fractional exhaled nitric oxide; ICS, inhaled corticosteroid.

Of the remaining 36 participants with measurable airflow obstruction, taking the tests in sequence (and stopping testing once an asthma diagnosis was achieved), 22 had asthma diagnosed using BDR, a further seven with PEFv and a further three with BCT. All 32 had EPOER asthma. Pathways 4 and 5 were not needed, as all participants had already achieved an asthma diagnosis via another pathway. When missing values were not assigned a negative value, results are shown in Supplementary Table E7.

Overall performance of the guidelines compared to EPOER asthma

ERS-defined asthma had a moderate sensitivity (81%, 95% CI 70–90%), resulting in 13 (11% of the whole population) having a diagnosis of asthma missed (Supplementary Table E8). The specificity was also moderate (85%, 95% CI 72–94%), leading to seven subjects being diagnosed with asthma who did not have asthma by EPOER. Overall, 1 in 6 (17%) of the symptomatic adults being investigated for asthma were misdiagnosed by this guideline.

NICE-defined asthma had perfect specificity (100% 95% CI 93–100%) but poor sensitivity (41%, 95% CI 30–59%). Including participants who followed the “suspect asthma” pathway increased the sensitivity to 49% (95% CI 36–91%) and the specificity remained unchanged at 100% (95% CI 93–100%) (Table 3). Overall, 36 (31%) of the symptomatic adults being investigated for asthma were misdiagnosed by this guideline.

GINA-defined asthma had perfect specificity (100%, 95% CI 93–100%), but poor sensitivity (47%, 95% CI 35–59%), resulting in the missed diagnosis of asthma in 38 (53%) individuals. Overall, 38 (32%) of the symptomatic adults being investigated for asthma were misdiagnosed by this guideline.

When missing tests were not treated as negative, up to 14% of the subjects (1 in 7) were unable to complete the guidelines to confirm or refute asthma (Supplementary Tables E4–E6) but this had little effect on overall sensitivity and specificity of each guideline (Supplementary Table E9).

Concordance between the three published guidelines

Of the 118 subjects, 53 (45%) did not have asthma by any of the three measures, and only 21 (18%) were always diagnosed with asthma (by all three guidelines) (Fig. 1). More than one third of the study subjects (37%) could be given a different diagnosis depending on which guideline was followed. Even though sensitivity and specificity for GINA and NICE were very similar, it is clear that concordance between their results was only moderate (<0.51, Supplementary Table E10); of the 45 diagnosed with asthma by either guideline, only 21 were in common. A Venn diagram including EPOER shows similar findings (Supplementary Fig. E6) and results are also presented as an alluvial plot (Supplementary Fig. E7).Fig. 1 Venn diagram showing concordance of, ERS, NICE and GINA guidelines for the diagnosis of asthma in 118 adults with respiratory symptoms.

Discussion

The heterogenous nature of asthma makes it unlikely that there will be a single diagnostic test for the condition, hence the need for diagnostic guidance and algorithms. Current guidance for diagnosing asthma also shows significant heterogeneity both in the tests included, the order of these tests and diagnostic thresholds recommended. Further, there is little evidence that the performance or feasibility of published guidance has been tested prior to implementation. We have evaluated three sets of guidance (ERS, NICE and GINA) within a population of adults with symptoms suggestive of asthma who underwent extensive diagnostic tests before and after starting treatment with ICS. Guidelines from NICE and GINA provided excellent specificity but lacked sensitivity for the diagnosis of asthma, thus affording excellent utility for ruling-in asthma, but limited ability to rule it out. In contrast, ERS guidelines provided reasonable sensitivity and specificity, but resulted in misdiagnosis (both under- or over-diagnosis) in 1 in 6 people with suggestive symptoms and so left a problematic degree of uncertainty. Importantly, and irrespective of the gold standard used in this study (EPOER), concordance between the three published guidelines being tested was only moderate (≤0.5), so that even where sensitivity and specificity were similar, different individuals were diagnosed with asthma. These original observations have significant implications for the diagnosis, and therefore management, of asthma.

Any investigation of diagnostic accuracy requires a reference standard, which should be the gold-standard or current standard.15 As there is no widely accepted gold standard for asthma diagnosis and the current standard(s) are the diagnostic guidelines assessed herein, this is a limitation of our study. We used a panel of experts (including a minimum of two senior asthma physicians) to confirm or refute the diagnosis by reviewing all objective evidence collected as part of the RADiCA study (EPOER is described in Supplementary Appendix, with examples of discordant cases in Supplementary Table E3). We emphasise that EPOER is not an algorithm but represents a (subjective) overall assessment of multiple (objective) pieces of data, and consequently may be seen as a weakness. Furthermore, we recognise that in clinical practice a physician, would not have the benefit of this depth of information nor the luxury of at least 2 colleagues with decades of experience with whom to discuss the case. However, we were unable to devise a more robust reference standard than to capitalise on all of the wide array of clinical data available and the panel’s collective but varied experience, and ultimately consider this to be a strength of the study. Individual tests used for asthma diagnosis provided poor sensitivity but good specificity against EPOER asthma, in line with previous reports. Our finding of modest sensitivity of BCTmeth (62%) was lower than previously reported6 reflecting the nature of EPOER where absolute values (rather than dichotomised results) were reviewed, so results just below the threshold could be evaluated (i.e., nine of those with EPOER asthma had a PD20 just below the BCTmeth threshold [0.2–0.7 mg], in combination with other results supportive of an asthma diagnosis), including patients’ treatment response.

Other limitations of our study include exclusion of those aged over 70 years and of smokers with a >10 pack year history. This was done to avoid diagnostic confusion with those with COPD, so as to enable the guidelines being tested to perform as well as possible. This will however mean that any future guideline developed from this dataset will have reduced application to older people or heavy smokers. One in four of the study subjects had previously used ICS and could not be described as steroid naïve. However, they were at least 4 weeks off ICS, which exceeds the ‘washout period’ typically recommended in clinical trials, and most had been off ICS for more than 2 years; therefore it is unlikely that this would have changed the test results and would not affect the performance of the guidelines. Also, we were unable to arrange long term follow up those who did not receive an EPOER diagnosis of asthma, to confirm what diagnosis they were eventually given.

There are considerable differences in the three guidelines assessed, in the tests included, (e.g., FeNO not in GINA), thresholds set (e.g., different FEV1/FVC ratios) and the number of positive tests required for diagnosis [ERS requires one or two (depending on the tests), GINA requires two (one of which must be obstructed spirometry), and NICE requires two or more (depending on the tests)]. The requirement from GINA that airflow obstruction (FEV1/FVC < LLN at any time) is recorded in addition to one other positive test reduced the sensitivity of these guidelines; obstruction was never documented in nearly half of people with asthma (47%). NICE provides a more complex algorithm, including 15 possible pathways. All participants that followed a ‘suspect asthma and review diagnosis after treatment’ pathway had a diagnosis of asthma, indicating that the “suspect asthma” pathways might be better-defined as ‘diagnose asthma’. In our study, if those on a ‘suspect asthma and treat’ pathway were included as asthma, NICE performance was very similar to GINA, with 100% specificity but limited sensitivity (∼50%). Our analysis identified that a large proportion of people where a diagnosis of asthma was missed by NICE followed a pathway which did not include tests for BDR or BCTmeth (Pathway 10). Changing the algorithm in this regard would improve its sensitivity. For ERS, although the overall proportion of subjects with an incorrect diagnosis was the smallest (17%), this guideline both under and over diagnosed asthma. The low specificity arose mostly from the pathways which allowed an asthma diagnosis with FeNO >50 ppb as the only positive test.

Evidence suggests that the majority of adults diagnosed with asthma have not undergone lung function testing at time of diagnosis.3 Furthermore, it has been shown that preventer treatment could be safely withdrawn in one third of adults with a recent diagnosis of asthma. Taken together this indicates that lack of objective testing at the time of diagnosis is resulting in overdiagnosis, and overtreatment, indicating a need for guidance on diagnostic testing. Our findings however demonstrate that current guidelines generally lack sensitivity, potentially resulting in underdiagnosis in up to half of those with asthma.

As most patients first present to their primary care physician, we propose that it would be most helpful if diagnostic algorithms started with tests available in a primary care setting with a high specificity for asthma. This would allow asthma to be confidently ‘ruled-in’ in a proportion of patients, who can then start treatment. Thus, a smaller pool of symptomatic patients can be referred to secondary care for more complex tests and an objective assessment of response to treatment, in a holistic way more akin to the EPOER. Significant modification would need to be made to all algorithms to achieve this objective.

A strength of this research is that the population included only patients with symptoms in-keeping with asthma who were not taking ICS, as this treatment can modify test results.5,6 Although completing tests by following diagnostic algorithms could delay the start of treatment (mean time to diagnosis in the NICE feasibility study was 53 days compared to 35 days previously),16 we were interested to observe that most of our participants reported having had symptoms for a median of 3 years prior to recruitment to the study. We fully support the view that wherever possible diagnostic tests should be completed before the commencement of treatment, and treatment only started before this when deemed clinically necessary.5

The European Asthma Research and Innovation Partnership (EARIP) identified the need to develop tools for improving asthma diagnosis in their 15 research priorities.17 Our analysis supports this proposal, as current practices provide either excellent specificity but very poor sensitivity (GINA and NICE) or only reasonable sensitivity and specificity (ERS), and concordance between these guidelines is moderate, resulting in more than one third of patients receiving a different diagnosis depending on which guideline was used. All guidelines therefore have significant limitations with regards to their function in clinical practice and further research is required to inform the next iteration of asthma diagnostic guidelines.

Contributors

CM, SJF and AS conceived and planned the study. SD, LH, RW, MB collected data. AS and AJS accessed and verified the data and conducted the statistical analysis. AS and AJS wrote the manuscript. All authors provided critical feedback and helped shape the research, analysis, and manuscript.

Data sharing statement

The authors will consider all reasonable requests for deidentified data, following approval by the study sponsors. Proposals should be directed to research.sponsor@mft.nhs.uk. To gain access, data requestors will need to sign a data access agreement.

Declaration of interests

HD participates in a data safety and monitoring board (HEAL-COVID). CM has received lecture fees from Sanofi and GSK and a travel grant from Sanofi. MB has received travel grants form by North West Lung Foundation and European Respiratory Society. None of the other authors had any interest to declare.

Appendix ASupplementary data

Supplementary Figs. E1–E7 and Tables E1–E10

Acknowledgements

Funding: This work was supported by the 10.13039/100014653 Manchester NIHR Biomedical Research Centre (BRC) (grant no. BRC-1215-20007 , and NIHR203308 ), 10.13039/501100000362 Asthma UK /Innovate (grant no. AUK-PG-2018-406 ), GSK ID 212474 and North West Lung Centre Charity. We would like to thank the participants for taking part in the study and the RADicA Study Team for their support. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Appendix A Supplementary data related to this article can be found at https://doi.org/10.1016/j.eclinm.2024.102813.
==== Refs
References

1 NICE Clinical Knowledge Summaries site (CKS) What is the prevalence of asthma? 2023 https://cks.nice.org.uk/topics/asthma/background-information/prevalence/
2 Haldar P. Pavord I.D. Shaw D.E. Cluster analysis and clinical asthma phenotypes Am J Respir Crit Care Med 178 3 2008 218 224 10.1164/rccm.200711-1754OC 18480428
3 Aaron S.D. Vandemheen K.L. FitzGerald J.M. Reevaluation of diagnosis in adults with physician-diagnosed asthma JAMA 317 3 2017 269 279 10.1001/jama.2016.19627 28114551
4 National Institute for Health and Care Excellence Asthma: diagnosis, monitoring and chronic asthma management [NG80] 2017 https://www.nice.org.uk/guidance/ng80
5 Global Initiative for Asthma Global strategy for asthma management and prevention, 2023 ginaasthma.org 2023
6 Louis R. Satia I. Ojanguren I. European Respiratory Society guidelines for the diagnosis of asthma in adults Eur Respir J 60 3 2022 2101585 10.1183/13993003.01585-2021
7 Gupta S. Paolucci E. Kaplan A. Boulet L. Contemporaneous international asthma guidelines present differing recommendations: an analysis Can Respir J 2016 2016 3085065 10.1155/2016/3085065
8 Szefler S.J. Boushey H.A. Pearlman D.S. Time to onset of effect of fluticasone propionate in patients with asthma J Allergy Clin Immunol 103 5 1999 780 788 10.1016/S0091-6749(99)70420-3 10329810
9 Miller M.R. Hankinson J. Brusasco V. Standardisation of spirometry Eur Respir J 26 2 2005 319 338 10.1183/09031936.05.00034805 16055882
10 Dweik R.A. Boggs P.B. Erzurum S.C. An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FeNO) for clinical applications Am J Respir Crit Care Med 184 5 2011 602 615 10.1164/rccm.9120-11ST 21885636
11 Coates A.L. Wanger J. Cockcroft D.W. ERS technical standard on bronchial challenge testing: general considerations and performance of methacholine challenge tests Eur Respir J 49 5 2017 1601526 10.1183/13993003.01526-2016
12 Hallstrand T.S. Leuppi J.D. Joos G. ERS technical standard on bronchial challenge testing: pathophysiology and methodology of indirect airway challenge testing Eur Respir J 52 5 2018 1801033 10.1183/13993003.01033-2018
13 Juniper E.F. O'Byrne P.M. Guyatt G.H. Ferrie P.J. King D. Development and validation of a questionnaire to measure asthma control Eur Respir J 14 1999 902 907 10573240
14 Stanojevic S. Kaminsky D.A. Miller M.R. ERS/ATS technical standard on interpretive strategies for routine lung function tests Eur Respir J 60 1 2022 2101499 10.1183/13993003.01499-2021
15 BMJ Best Practice Diagnostic test studies: assessment and critical appraisal https://bestpractice.bmj.com/info/toolkit/learn-ebm/diagnostic-test-studies-assessment-and-critical-appraisal/
16 National Clinical Guideline Centre Asthma: diagnosis, monitoring and chronic asthma management Appendices A - R 2017 https://www.nice.org.uk/guidance/ng80/evidence
17 Masefield S. Edwards J. Hansen K. The future of asthma research and development: a roadmap from the European Asthma Research and Innovation Partnership (EARIP) Euro Respir J 49 5 2017 1602295 10.1183/13993003.02295-2016
