Recently much has been said and written about gas stoves causing childhood asthma, justifying politicians moving to ban residential gas stoves. We reviewed the epidemiology studies used to support the ban in a recently published study. We demonstrated that the studies were unreliable and there is no basis for the claims made. This essay sums up our findings.

NO2 and asthma

Nitrogen dioxide (NO2), which is not carbon-based, is the emission at issue for gas stoves. Allergens are the basic issue in asthma. Allergens are carbon-based and are mostly protein substances. For example, pollen and plant parts; biological fragments shed from furry animals, rodents, cockroaches, dust mites; and fungal detritus can cause allergenic asthma reactions. A recent American Thoracic Society report notes, “It is unclear whether direct effects of NO2 … explain the causal link with asthma.” If NO2 were a cause of asthma, they would know.

Image: Gas stove by wirestock.

Thus, epidemiology studies in the literature are not founded on evidence that NO2 causes asthma. They are just studies looking for statistical associations of NO2 with asthma. Association does not prove causation, and the people arguing to ban gas stoves know it. However, their analytical methods increase the chance they can show a statistical association.

NO2-asthma epidemiology

We evaluated a meta-analysis of indoor NO2 and gas cooking effects on childhood asthma. We used novel statistical methods (search space analysis and p-value plotting) to evaluate the reliability of a research claim in the meta-analysis. We found that the claim is not well founded in good evidence.

“Search space analysis” allowed us to see whether excessive hypotheses testing (called multiple testing bias) occurs in gas stove-childhood asthma studies. Multiple testing occurs when researchers use their data set to test many statistical hypotheses. This presents an opportunity for fishing for the results you want.

We used “p-value plotting” to examine whether statistical results from gas stove-childhood asthma studies (p-values) used in the meta-analysis support either no association (randomness) or a statistically significant association.

If p-values follow an approximate 45-degree line in the plot, the studies support randomness (no effect). In essence, the plot shows the studies are worthless. If p-values approximately follow a line with a flat/shallow slope, where most (the majority) of p-values are small (< 0.05), then the studies provide evidence for a real association.

Results of our search space analysis

Our study counted the number of statistical hypotheses tested in 14 of 27 epidemiology studies used for meta-analysis. Summary statistics for the 14 studies are shown below:

One-in-twenty (5%) of hypothesis tests can be a statistically significant but false-positive error. A study with a median count of 15,360 hypothesis tests can have 5% × 15,360 = 768. Under the latter circumstances, how do you tell if a result presented in a study is real or just a false-positive error?

Results of our p-value plotting analysis

You can see in the chart below a p-value plot for 13 test statistics used for meta-analysis of a gas stove-childhood asthma association. The p-values are aligned at approximately 45 degrees. This is evidence of randomness (i.e., no association between gas stove NO2 and childhood asthma). Boom! There’s no proof that gas stove emissions cause asthma.

Conclusions

The people pushing to ban gas stoves rely on studies that have flawed methodologies, making their results meaningless.

1. Research on NO2 has not established that it or any other emission from gas stoves causes childhood asthma.

2. Questionable statistical methods (excessive hypotheses testing) appear common in NO2-childhood asthma epidemiology studies. These studies are made unreliable and invalid because of unknown false positive errors.

3. The p-value plot for a gas stove-childhood asthma association supports randomness. This is not proof of a toxic causal effect from gas stoves.

The published studies we evaluated, which are supposed to be reliable, were not. They have methods that facilitate the researchers’ pre-existing biases. The public can consider the gas stove−childhood asthma claim both unproven and wrong. There is no reliable evidence to justify a public policy as extreme as banning gas stoves.

Warren Kindzierski has a Ph.D. in environmental engineering; Stan Young has a Ph.D. in Statistics and Genetics; John Dunn has an M.D. and a J.D.