Repository navigation
Conversation
Refs mlco2#503. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dilipna
force-pushed
the
refresh-global-energy-mix
branch
from
October 28, 2026 00:46
e5205a3 to
dd9b899
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Refreshes the country carbon intensities in
global_energy_mix.jsonfrom Our World in Data's maintained energy dataset (Ember's electricity data and the Energy Institute's Statistical Review), and replaces the manual notebook run with a script.codecarbon/data/private_infra/update_global_energy_mix.py: rebuilds the file from OWID'sowid_energy.csv(default: OWID's catalog URL, or a CSV file or URL given as argument). It keeps each country's latest year with a carbon intensity, refuses to run if a country would disappear, and writes a carbon intensity that is not positive asnull(CodeCarbon then computes it from the electricity mix), and prints the dataset's sha256 plus a short list of entries to check (empty mix, data older than two years, a year going backwards).codecarbon/data/private_infra/global_energy_mix.json: regenerated. Same 213 countries and 19 keys, values only, except that Central African Republic now has anullcarbon intensity instead of 0.0 (see below). 210 countries move to a newer year: 90 now use 2025 and 105 use 2024.tests/test_update_global_energy_mix.py: new tests for the script, including that the committed file rebuilds exactly from the script's logic.tests/test_emissions.py: the France and Canada tests hardcoded the old values; they now read the expected value from the data, so a refresh does not break them.docs/explanation/methodology.md: describes the source (Ember via OWID, lifecycle basis) and the refresh command, and corrects China's national figure (582 → 525).No logic in
emissions.pychanges.Related Issue
Refs #503. Part of #947 (process for updating reference data).
Motivation and Context
The country values came from 2022 (130 countries) and 2023 (78), and refreshing them meant re-running a notebook by hand. What changes (carbon intensity in gCO₂eq/kWh, year in brackets):
Across the 209 countries that gain a newer year and had a non-zero value, the median change is -2% and 40% of them move by more than 10%. Regional datasets (US states, Canadian provinces, Nordic zones, Chinese provinces) still take precedence where they apply; these are the national fallback values.
Source. The notebook reads OWID's GitHub copy of the dataset, which OWID now marks as "no longer updated" in favour of its catalog. I compared the two: on the 5,691 rows they share, the carbon intensity is identical, the country names are identical, and OWID publishes a column mapping. The script reads the catalog.
Central African Republic. The source reports a carbon intensity of 0.0 for CAF for every year from 2013 to 2023. Following your answer on #503 ("try with
nullinstead of 0.0"), the script writes a non-positive value asnull, so CAF falls back to the intensity computed from its electricity mix. The existing handling of an explicit numeric zero (#1441) is not touched. The docs paragraph describes this rule.Other things you may want to decide
Dataset used:
owid_energy.csv(9,081,888 bytes, served 2026-09-30), sha2569d8efc4a9b1cc2ffdfa7f7f6f1e171b2922e0be2d97c0edf2982e874357ed6de. Running the script on it rebuilds the committed JSON byte for byte.How Has This Been Tested?
tests/test_update_global_energy_mix.py(20 tests): latest year with a carbon intensity, aggregates and countries without an intensity left out, missing sources, rounding and integer years, missing columns, each of the four checks, reading a file and a URL, the committed file rebuilding exactly, every country having a positive total (the viz code divides by it), and the command line (including refusing to drop a country). Each check was also confirmed to fail when the script is deliberately broken.bcb446f2): 823 passed, 3 skipped. After rebasing on current master, on Windows (Python 3.12,tests/test_viz_data.pyexcluded becausedashis not installed): 824 passed, 2 failed, 32 skipped. The 2 failures (test_expired_cache_entry_is_refetched,test_task_energy_with_live_update_interference) are the Windows/NVIDIA test issues from Unit tests on Windows: 13 failures besides the CLI ones in #1459 (one also on any machine with an NVIDIA GPU) #1460 and do not involve this change.tests/test_emissions.pyandtests/test_zero_carbon_intensity.pyalso pass on Python 3.13 and 3.14 (pandas 3.0.5).Types of changes
AI Usage Disclosure
Checklist:
🤖 Generated with Claude Code