
==== Front
Glob Epidemiol
Glob Epidemiol
Global Epidemiology
2590-1133
Elsevier

S2590-1133(24)00026-9
10.1016/j.gloepi.2024.100160
100160
Research Paper
Challenging unverified assumptions in causal claims: Do gas stoves increase risk of pediatric asthma?
Cox, Jr. Louis Anthony tcoxdenver@aol.com
⁎
Cox Associates, Entanglement, University of Colorado at Denver, Denver, Colorado, USA
⁎ Corresponding author. tcoxdenver@aol.com
21 8 2024
12 2024
21 8 2024
8 1001602 6 2024
16 8 2024
17 8 2024
© 2024 The Author
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
The use of unverified models for risk estimates and policy recommendations can be highly misleading, as their predictions may not reflect real-world health impacts. For example, a recent article states that NO2 from gas stoves “likely causes ∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone” annually in the United States. This explicitly causal claim, which is contrary to several methodology and review articles published in this journal, among others, reflects both (a) An unverified modeling assumption that pediatric asthma burden is approximately proportional to NO2; and (b) An unverified causal assumption that the assumed proportionality between exposure and response is causal. The article is devoid of any causal analysis showing that these assumptions are likely to be true. It does not show that reducing NO2 exposure from gas stoves would reduce pediatric asthma risk. Its key references report no significant associations – let alone causation – between NO2 and pediatric asthma. Thus, the underlying data suggests that the number of pediatric asthma cases caused by gas stoves in the United States is indistinguishable from zero. This highlights the need to rigorously validate modeling assumptions and causal claims in public health risk assessments to ensure scientifically sound foundations for policy decisions.

Keywords

Gas stoves
Asthma
NO2
Causality
==== Body
pmcIntroduction

In May of 2024, headlines in many news outlets, from the Los Angeles Times (“Gas stoves may contribute to early deaths and childhood asthma, new Stanford study finds”) [15] to NBC News (“Gas and propane stoves linked to 50,000 cases of childhood asthma, study finds”) [16], announced the results of a just-published study by Kashtan et al. [1]. The abstract of the study states that “Gas and propane stoves increase long-term NO2 exposure 4.0 parts per billion volume on average across the United States, 75% of the World Health Organization's exposure guideline. This increased exposure likely causes ∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone.” This is a startling causal claim. The paper presents no causal analyses to support it. It does not cite any references to such causal analyses. Indeed, the rest of the paper makes no reference to causal methods, causal analyses, or causal inferences. It deals solely with associations and associational concepts [2] such as relative risk (RR) and quantities derived from RR, such as population attributable fractions (PAFs).

As many epidemiologists know well, and notwithstanding frequent erroneous interpretations to the contrary [2,3], PAFs are simple functions of relative risks and exposure prevalence [4], estimating an associational relationship in a population. Hence, they are associational rather than causal and they do not in general reveal how changing exposure would change risk [2]. Despite the suggestive name, saying that a health risk is “attributable” to an exposure in this sense does not imply that the exposure is a cause of the risk, or that reducing the exposure would reduce the risk [3,5]. PAFs do not consider or contain the essential mechanistic information needed to determine whether, or by how much, reducing exposure would reduce risk; indeed, PAFs can be as high as 100% for each of many factors that are correlated with risk, even if reducing or eliminating them would have no effect on risk [5]. Thus, the expressly causal claim in the abstract of Kashtan et al. that “exposure likely causes ∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone,” is immaculately conceived: it arrives without benefit of prior causal models, data, or analyses.

To this claim, therefore, we add the following crucial caveats:• Neither the Kashtan et al. paper nor its references analyze, let alone establish, causality. They do not show that reducing NO2 exposures from gas stoves would have any effect on pediatric asthma risk. The paper and its references provide no empirical basis for suggesting a causal relationship or justifying the use of causal language such as “asthma incidence due to long-term NO2 exposure” or “gas and propane stoves in the United States are responsible for 200 [95% CI: −20, 410] thousand current cases of pediatric asthma” (emphases added). These claims and the explicitly causal claim in the abstract are not warranted, but arise from the too-common practice of unjustifiably conflating association (in this case, based on unverified modeling assumptions, as discussed below) and causation [6]. Key references cited by Kashtan et al. make clear that they, too, only address association. For example, Lin et al. [7] state that “We extracted the association between indoor NO2 (and gas cooking) and childhood asthma and wheeze from population studies published up to 31 March 2013” (emphasis added).

• The associations studied may not exist. Kashtan et al. cite the meta-analysis by Lin et al. (7) for estimating measures of association (PAFs) between NO2 and pediatric asthma. But Lin et al. explicitly state that “The summary odds ratio from random effects meta-analysis for asthma and gas cooking exposure was 1.32 [95% confidential interval (CI) 1.18-1.48], and for a 15-ppb increase in NO2 it was 1.09 (95% CI 0.91-1.31).” Thus, they report no significant statistical association specifically between NO2 and pediatric asthma; moreover, any associations, significant or not, are not necessarily causal. By contrast, the first paragraph in Kashtan et al. cites Lin et al. as supporting their claim that “Long-term exposure (averaged over a year) to NO2 has been linked to increased incidence and exacerbation of pediatric asthma.” This does not make clear that the claimed “link” (i.e., association) is not significantly different from no effect. Lin et al. [7] further state that “Residual confounding by (unmeasured) factors that are associated with gas cooking might be another explanation for our finding of an association between asthma and gas cooking, but not with indoor NO2.” Although they do not believe this is likely, they do not formally test for effects of residual confounding or omitted confounders. Thus, even if they had found a significant association between pediatric asthma and indoor NO2, it would not be clear why it existed.

• Likewise, a 2024 meta-analyses [8], on which Kashtan et al. rely for their claimed association between gas stoves and pediatric asthma, also reported no significant association between cooking and heating with gas and pediatric asthma: “In addition, a small increased risk of asthma in children was not significant (OR 1.09, 0.99-1.19; p = 0.071) and no significant associations were found for adult asthma… [although] a significant decreased risk of bronchitis was observed (OR 0.87, 0.81-0.93; p < 0.0001)” [8]. Athough this 2024 article by Puzzolo et al. [8] addresses substantially the same research questions as earlier articles such as Lin et al. (2013) [7], among others, it also includes several more recent studies. It finds no association between cooking with gas and asthma, even though Kashtan et al. [1] cite it as if it supported such an association, writing that “Combining data from the 2020 RECS with the 2024 meta-analysis of Puzzolo et al. of the association between pediatric asthma and gas stoves, we estimate that gas and propane stoves in the United States are responsible for 200 [95% CI: −20, 410] thousand current cases of pediatric asthma.” That the 95% confidence interval includes zero reflects the absence of a clear statistically significant effect. Similarly, for population attributable risks, Kashtan et al. show a pediatric asthma PAF for long-term NO2 from gas stoves of 0.91 with a 95% uncertainty interval from −1.33% to 3.0%. This range suggests that the three null hypotheses that NO2 exposure (a) decreases pediatric asthma risk (by up to 1.3%); (b) increases pediatric asthma risk (by up to 3%); or (c) leaves pediatric asthma risk unchanged are all consistent with the data. In short, there is no clear evidence of an effect. Thus, the estimated associations between gas cooking or NO2 and pediatric asthma that Kashtan et al. use to predict “∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone” and “200 [95% CI: −20, 410] thousand current cases of pediatric asthma” may well not exist. This absence of a significant NO2-asthma association in data is replaced in Kashtan et al.'s analysis by an unverified modeling assumption that a significant association is present even if the data don't show it; this is discussed next.

• Kashtan et al. simply assume a mathematical model that implies that burden of adverse health effects is approximately proportional to exposure, with a constant of proportionality that is independent of exposure level. They do not justify this strong assumption. Their “finding” that NO2 exposure from gas stoves increases pediatric asthma risk is driven by this modeling assumption, rather than an empirical discovery. As discussed later, the mathematical model they use implies (or assumes) a positive exposure-response association even in cases where the data show a negative exposure-response association. Reporting a positive burden of asthma or mortality attributed to exposure in this context is little more than a tautology, based on assuming what is to be proved: it is a “finding” that is guaranteed by the model specification, independent of the data, as long as both exposure estimates and burden estimates are non-negative (and at least some are positive).

• The assumed model was not tested or empirically validated. Kashtan et al. do not mention model validation or discuss the empirical validity of their model predictions. No evidence is offered that they have any predictive validity.

• Thus, the specific causal claim that “exposure likely causes ∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone” is unsupported by the data and analyses presented. No evidence has been presented that exposure causes any cases of pediatric asthma, let alone 50,000 per year.

The following sections explain technical aspects of these caveats and comment on their implications.

Results

No evidence that NO 2 emissions from gas stoves increase pediatric asthma risk.

Kashtan et al. state that their “Results are based on measurements and assumptions throughout the modeling chain.” As just emphasized, perhaps the most important of these assumptions is that the number of adverse health effects per year, such as pediatric asthma cases or adult deaths, that is “attributable to” NO2 from gas stoves is assumed to be approximately directly proportional to NO2 exposure, even at low exposure concentrations. The following passage makes this clear (bearing in mind that 1 - e-x ≈ x for small values of x):

“Consistent with prior epidemiological work assessing the influence of long-term NO2 exposure on respiratory diseases…, we assumed a log-linear concentration-response function and calculated health outcome burdens as.(7) Burden=IncgΣnPn×Wn×1−e−βΔcn

where values are summed over all n = 7632 RECS [Residential Energy Consumption Survey] residence types with gas or propane ranges or cooktops in the RECS database, Burden is the number of adverse health outcomes (death or pediatric asthma) attributable to NO2 from stoves, Incg is the current incidence rate of the adverse health outcome in question in the geography in question, Pn is the number of people living in the nth household, Wn is the number of households the nth RECS household represents in the U.S. housing stock, β is the concentration response factor (calculated as ln(RR)/Δc), which is assumed to be constant, and Δcn is the median year-averaged gas stove-attributable NO2 exposure in the nth residence.” (1).

The author's eq. (7) can be written more simply as follows:(1) Burden≈kxExposure

by defining the constant k as k = IncgΣnPn × Wn × β and approximating (1 − e−βΔcn) by βΔcn. This linear no-threshold (LNT) equation represents an assumption, not an empirical finding. Moreover, it is an assumption that is not validated: the authors simply assume that burden is approximately proportional to exposure, with a constant of proportionality, k, that is the same for all levels of exposure. They later report that burden increases with exposure as if this were a finding, but in fact it is a necessary logical implication of applying the assumed model (1) to data with positive values of Burden and Exposure.

How might a risk analyst avoid such strong modeling assumptions? A less tendentious analysis would replace (1) with the more general model Burden ≈ intercept + k x Exposure, thus allowing for the possibility that unmeasured confounders, model specification errors, measurement and classification errors, coincident historical trends in exposure and response, and other modeling challenges might yield a non-zero intercept for the estimated relationship, even if the intercept would be zero in the absence of these realistic complexities; and then let the data determine whether the intercept term is significantly different from 0 and, if so, whether k is still significantly different from 0. It would also analyze the residuals to assess whether a linear model is consistent with the data or whether modifications, such as thresholds and curvature, are necessary in order to adequately describe real-world data. These model diagnostics and validation steps are omitted from the discussion in Kashtan et al. and its references, which do not provide any information about validation of the assumed model.

Even if model (1) were shown to describe available data, crucially, as many epidemiology articles and textbooks explain, the number of deaths or illnesses “attributable” to exposure (“Burden”) from this equation is still an association-based quantity derived from exposure prevalence and RR [4]. It has no necessary relationship to the number of deaths or illnesses caused by exposure, or preventable by reducing exposure, although the conflation of association and causation is a common mistake in many applied epidemiological studies [2,3,5]. To illustrate just how strong the assumption in eq. (1) is, suppose that the estimated values of Burden and Exposure are independent random variables (e.g., each independently uniformly distributed between 0 and some maximum value, which without loss of generality, we may normalize to be 1). Then fitting model (1) to any large data set with non-negative (and some positive) values of Exposure and Burden using ordinary least-squares regression guarantee a positive estimate of k. In this case, assuming model (1) (or, equivalently, the authors' eq. (7)) assures that a positive burden will be attributed to exposure, even if burden is statistically independent of exposure level. Indeed, even if exposure actually reduces burden, eq. (1) still predicts that burden increases with exposure. For example, if the true empirical relationship between estimated values of Exposure and Burden is Burden = 1 - Exposure, and the data consist of 50 exposure-response studies with estimated Exposure = 0.25 and estimated Burden = 0.75 and another 50 studies with estimated Exposure = 0.5 and estimated Burden = 0.5 then fitting model (1) to these data using linear regression would give an estimated value for the slope coefficient k of 1.5 with a 95% confidence interval from 1.24 to 1.56. Thus, the unwarranted modeling assumption that burden is approximately proportional to exposure suffices to give positive estimates of burden attributed to exposure even though the data show lower levels of burden for higher levels of exposure. The modeling assumption trumps the data. Using such a mathematical straightjacket to reach conclusions does not provide a sound basis for risk estimation or policy recommendations. More flexible, less assumption-dependent models (e.g., generalized additive models) can and should be used instead to avoid artificially constraining the conclusions that can be reached.

Discussion

No likely non-zero effect has been established.

Kashtan et al. provide a section on “Uncertainty” that focuses solely on sampling variability, without addressing or quantifying uncertainty about the assumed model, eq. (1). It assumes “a normal distribution of RRs reported for asthma.” This, too, is a remarkably strong assumption that is not appropriate for the specific, important purpose of testing whether there is any effect of exposure on response. The normal distribution is continuous, so it cannot be used to assess a discrete (non-zero) probability that RR = 1, i.e., that there is no effect. That would require augmenting the normal distribution with a discrete probability component. Likewise, the “Uncertainty” section does not quantify the probability that correctly accounting for confounders would entirely explain away the statistical association between estimated NO2 exposures and estimated rates of pediatric asthma (or of adult mortality). The paper mentions in passing (under Table 1) that “estimates of pediatric asthma attributable to stoves do not fully account for confounding variables and could be too high,” but it presents no quantitative uncertainty analysis to address whether the correct value is zero with high probability.

Other researchers have helped to fill this gap. For example, Li et al. [9] caution that “We conclude that the [gas stove-pediatric asthma] epidemiology literature is limited by high heterogeneity and low study quality and, therefore, it does not provide sufficient evidence regarding causal relationships between gas cooking or indoor NO2 and asthma or wheeze. We caution against over-interpreting the quantitative evidence synthesis estimates from meta-analyses of these studies. … An exposure-response relationship has not been well-characterized. No study has examined whether or how asthma risk or severity would change after reducing indoor NO2 exposure…. Taken together, we conclude that the evidence does not support causality.” Likewise, Atkinson et al. [10], on whom Kashtan et al. rely for mortality risk estimates, caution that “Our study confirms the need for continued caution in respect of causality particularly since the revised meta-analyses suggest HRs close to one [no effect], with the possibility of further attenuation if meta-analyses are restricted to studies with individual measures of BMI and smoking. The substantial heterogeneity between study results also weakens the argument for causality.”

The use of unverified modeling assumptions such as the linear no-threshold (LNT) model to generate estimated health “burdens” that are “attributed” to exposures based on associational formulas, and then misrepresented to policy makers and the media as being caused by exposure, and preventable by reducing exposure, is far from unique to Kashtan et al. Similar low-exposure LNT models have previously been applied by the US EPA to fine particulate matter [11] and by the US FDA to animal antibiotics [12]. An unverified LNT assumption can override data showing little or no (or a low-dose negative) exposure-response association and replace it with a positive association, even at low exposure levels, that projects significant public health benefits from further reducing exposure; this happens when the assumption of no threshold constrains the left end of the exposure-response curve to go through (0,0) but the exposure and response data points have positive values and yet a negative slope [11,12]. This practice may help to create impressive-looking benefits estimates for proposed regulations, but it is deceptive. The assumed projected benefits may very well not exist. In the absence of model validation, analyses based on eq. (1) give no valid reason to believe that either the harms attributed to exposures or the projected health benefits from reducing exposures are real.

Conclusions

Assumption-driven risk models, risk assessments, and predictions of health benefits from regulation may succeed in achieving tighter regulations for a variety of (possibly harmless) exposures in the short run, especially if policy-makers, media, and members of the public mistakenly assume that investigators have followed traditional scientific method by testing and validating their modeling assumptions and model-based predictions against data. If confidently predicted benefits fail to materialize after assumption-driven policies are implemented (e.g., Henneman et al. [13] note for air pollution that “Multiple studies in the accountability field have found it difficult to attribute significant improvements in air quality or public health attributable to air quality regulations”; and Sproston et al. [14] note that bans of animal antibiotics to prevent increases in antibiotic-resistant human infections were followed by “a continued and sustained increase in many countries” in antibiotic-resistant human infections), the credibility of unverified modeling assumptions and mathematical models as a basis for public policy recommendations may wane. For adverse human health effects currently attributed to gas stoves, it is perhaps not premature to point out that assumption-driven causal claims and recommendations such as those advocated by Kashtan et al. have no known validity and that they disagree with empirical studies finding no significant association between NO2 from gas stoves and childhood asthma [[6], [7], [8], [9]]. They provide no sound scientific basis for estimating or managing risks. Attempts to systematically review available evidence without pre-supposing or advocating particular polices have concluded that there is no evidence of empirically demonstrated or highly likely real-world adverse health effects caused by gas stove emissions (including pediatric asthma and adult mortality), despite decades of research attempting to find such evidence.

Risk management of gas stoves is currently a highly contentious and politicized topic in the United States. Against this heated background, it may be worth reflecting that sometimes absence of evidence, after considerable effort and diligent search, is, if not precisely evidence of absence (since it is hard to prove a negative), at least evidence of the absence of an effect large enough to be detected or to be statistically significant. This provides a sharp contrast to the assumption-driven assurance by Kashtan et al. that gas stove exposure likely “causes” thousands of cases of pediatric asthma per year. A more scientifically supportable (empirically grounded) conclusion, based on the data that Kashtan et al. rely upon, is that the number of adverse health effects per year that have been shown to be caused, or at least to be highly likely to be caused, by NO2 exposure from gas stoves is not detectably different from zero.

CRediT authorship contribution statement

Louis Anthony Cox, Jr.: Writing – review & editing, Writing – original draft, Methodology, Conceptualization.

Declaration of competing interest

I declare no competing interests that could influence the work presented in this manuscript.

Funding for this work was provided by Cox Associates, LLC. Cox Associates has reasonable expectation of receiving funds in the future from the American Gas Association (AGA) to cover part of the cost of writing up these comments and/or defraying open access charges. Cox Associates has also previously received support from the 10.13039/100000199 United States Department of Agriculture (USDA) for research on statistical methods for causal analysis of observational and interventional data. The author has previously received support from the USEPA for a part of the work related to duties related to NAAQS reviews. The research questions, methods, and conclusions are solely those of the author and do not necessarily reflect the views of any funding organizations.

Acknowledgments

This work was supported by Cox Associates, LLC. Cox Associates received funding from the American Gas Association (AGA) to cover part of the cost of preparing this paper and has previously received support from the 10.13039/100000199 United States Department of Agriculture (USDA) for research on statistical methods for causal analysis of observational and interventional data. The research questions asked, methods used, and conclusions reached are solely those of the author and do not necessarily reflect the views of any funding organizations.
==== Refs
References

1 Kashtan Y. Nicholson M. Finnegan C.J. Ouyang Z. Garg A. Lebel E.D. Nitrogen dioxide exposure, health outcomes, and associated demographic disparities due to gas and propane combustion by U.S. stoves Sci. Adv. 10 2024 eadm8680 38701214
2 Pearl J. Causal inference in statistics: an overview Stat Surv 3 2009 96 146
3 Greenland S. Concepts and pitfalls in measuring and interpreting attributable fractions, prevented fractions, and causation probabilities Ann. Epidemiol. 25 2015 155 161 25498918
4 Lin C.K. Chen S.T. Estimation and application of population attributable fraction in ecological studies Environ. Health 18 2019 52 31196209
5 Cox L.A. Jr. Thinking about causation: a thought experiment with dominos Glob. Epidemiol. 3 2021 100064
6 Li W. Goodman J.E. Long C. Population attributable fraction of gas cooking and childhood asthma: what was missed? Glob. Epidemiol. 7 2024 100141
7 Lin W. Brunekreef B. Gehring U. Meta-analysis of the effects of indoor nitrogen dioxide and gas cooking on asthma and wheeze in children Int. J. Epidemiol. 42 2013 1724 1737 23962958
8 Puzzolo E. Fleeman N. Lorenzetti F. Rubinstein F. Li Y. Xing R. Estimated health effects from domestic use of gaseous fuels for cooking and heating in high-income, middle-income, and low-income countries: a systematic review and meta-analyses Lancet Respir. Med. 12 2024 281 293 38310914
9 Li W. Long C. Fan T. Anneser E. Chien J. Goodman J.E. Gas cooking and respiratory outcomes in children: a systematic review Glob Epidemiol 5 2023 100107
10 Atkinson R.W. Butland B.K. Anderson H.R. Maynard R.L. Long-term concentrations of nitrogen dioxide and mortality: a meta-analysis of cohort studies Epidemiology 29 2018 460 472 29746370
11 Cox L.A. Jr. Improving interventional causal predictions in regulatory risk assessment Crit. Rev. Toxicol. 53 2023 311 325 37489873
12 Cox L.A. Jr. Some limitations of a proposed linear model for antimicrobial risk management Risk Anal. 25 2005 1327 1332 16506967
13 Henneman L.R. Liu C. Mulholland J.A. Russell A.G. Evaluating the effectiveness of air quality regulations: a review of accountability studies and frameworks J. Air Waste Manage. Assoc. 67 2017 144 172
14 Sproston E.L. Wimalarathna H.M.L. Sheppard S.K. Trends in fluoroquinolone resistance in Campylobacter Microb Genom 4 2018 e000198
15 Briscoe T. Gas stoves may contribute to early deaths and childhood asthma, new Stanford study finds Los Angeles Times 2024 May 22. https://www.latimes.com/environment/story/2024-05-22/stanford-study-gas-stoves-early-deaths-childhood-asthma#:∼:text=They%20also%20found%20some%20gas,breaths%20than%20adults%2C%20Kashtan%20said
16 Bendix A. Gas and propane stoves linked to 50,000 cases of childhood asthma, study finds. Stanford researchers measured real-world exposures to nitrogen dioxide from gas stoves in home kitchens across the U.S https://www.nbcnews.com/health/kids-health/gas-stoves-linked-childhood-asthma-study-rcna150241 2024
