The evaluation team compared households that purchased improved stoves to those using traditional stoves using a propensity score matching (PSM) design.

Goal:

Determine the impact of Millennium Challenge Account (MCA) stoves on reducing air pollution (carbon monoxide and fine particulate matter), reducing stove fuel expenditures, and improving health related to air quality problems.

Activities:

  • Conducted an evaluability assessment, which led to a final design documented in the Evaluation Design Report.
  • Established a field laboratory. Procured  air quality sampling equipment. Recruited and trained local technicians.
  • Completed pilot data collection to calibrate air quality measurement methods and sample sizes and to improve survey instruments.
  • Completed three phases of evaluation data collection during the winter of 2012-2013. The following methods were used:
    • Panel data from three surveys of 1,057 randomly sampled households provided data on fuel consumption and stove usage patterns over the course of the winter, demographic and economic characteristics, dwelling characteristics, and stove perceptions.
    • Electronic stove use monitors (SUMs) recorded fueling event and temperature data, used to triangulate household survey findings.
    • Household stove emissions and indoor air quality measurements of PM2.5 (fine particulate matter capable of causing various health problems), carbon monoxide, and other pollutants were obtained from a random subsample of 143 dwellings throughout the winter
    • Ambient air quality modeling estimated changes in stove contributions to ambient PM2.5 levels in light of MCA stove sales
    • Engaged an independent local firm to complete extensive quality control verification on survey data. This confirmed a very low 0.2% error rate.
  • Shared report findings through dissemination workshops in Ulaanbaatar and Washington, DC to encourage utilization of evaluation results. More than 60 representatives of parliament, national and city government agencies, academic institutions, private sector stove distributors, international donors, the media, and the U.S. Embassy attended the Ulaanbaatar workshop. Local television and print news distributed findings  to Ulaanbaatar residents.

Results:

Air pollution reductions:

  • MCA stoves significantly lowered emissions (65% reduction in PM2.5, 16% reduction in CO) compared to traditional stoves in real-world home settings.
  • MCA stoves estimated to have led to 30% reduction of ambient air pollution (PM2.5 concentrations) from residential heating stoves.

Improved health:

  • In year of evaluation, subsidy program estimated to have contributed to ~10% reduction in air pollution related disease outcomes and $3.9 million savings in health care costs.

Coal consumption:

  • The EEP stove subsidy program did not achieve significant reductions in daily coal consumption or fuel expenditures under typical usage conditions.
  • Very low compliance with MCA stove operation instructions may have contributed to lack of reduced coal consumption. Significant reductions in coal use were observed when households used MCA stoves according to instructions.
  • Households using MCA stoves enjoyed significantly higher indoor temperatures, suggesting that users may be sacrificing fuel economy for comfort.

Related resources:

All reports, presentation videos, questionnaires, and data are available on MCC’s website

Contact: projects@socialimpact.com