Apply PUE and WUE for the year the usage was incurred - #271
Merged
Conversation
AWS publishes PUE and WUE per year and the figures move noticeably from one year to the next, so applying a single blanket value skews any report that spans more than the most recent year. aws-pue-wue.csv gains a Year column and now carries the 2022-2025 figures. They come from the tables on the AWS sustainability page rather than the downloadable aws-wue-pue.csv, which stopped being updated after 2024 and so has neither the 2025 figures nor the restatements of earlier years. That brings in the regions AWS started reporting on since: London, Paris, Tel Aviv, Bangkok and Osaka, plus PUE for Sydney and Melbourne. The global average is included as a GLOBAL row and used as a tier below the geography averages, so an unrecognised region gets the published global figure for its year instead of the hardcoded default. PWUE picks the row for the usage year, falling back to the closest year published for that region - which covers regions reported on recently, WUE that only exists from 2024, and rows with no usable date. The year comes from whichever of ChargePeriodStart, line_item_usage_start_date, Date or BILLING_PERIOD the report carries, all read optionally so the module keeps working across providers and report formats without requiring a new column. Microsoft does not break its figures down by year, so azure-pue-wue.csv takes the same schema with the year left empty, meaning the row applies to every year.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #249
AWS publishes PUE and WUE per year, and the figures move noticeably from one year to the next —
eu-central-1went 1.32 → 1.33 → 1.35 → 1.24 across 2022-2025, and its WUE went from 0.01 in 2024 to 0.17 in 2025. Applying a single blanket value skews any report spanning more than the latest year.What changed
aws-pue-wue.csvis nowGeography,RegionID,Year,PUE,WUEwith one row per region per year, 2022-2025.The figures come from the tables on the AWS sustainability page rather than the downloadable
aws-wue-pue.csv, which was last updated in December 2025 and stops at 2024. Using the page picks up:eu-south-22024 is 1.07 on the page, 1.09 in the stale CSV);The one thing the page does not carry is the global WUE for 2022 and 2023; those two values are taken from the downloadable CSV, which is noted in the file header.
PWUEkeys its lookups by region and year:eu-.+,(me|il)-.+,(us|ca|mx)-.+, …)GLOBALrowdefaultWithin a tier it takes the row for the usage year, falling back to the closest year published for that region. That fallback is what covers recently-reported regions, WUE that only exists from 2024 onwards, and rows with no usable date.
Reading the year: new
RowColumn.getYear(), probingChargePeriodStart→line_item_usage_start_date→Date→BILLING_PERIOD. All four are read optionally, so the module keeps working across AWS/Azure and NATIVE/FOCUS without adding a required column tocolumnsNeeded(). It handles Spark timestamps and dates as well as the string forms CSV exports use.LINE_ITEM_USAGE_START_DATE/_END_DATEmoved from the AWS FOCUS bridge intoCURColumnso the shared module can reach them.azure-pue-wue.csvtakes the same schema with the year left empty — meaning the row applies to every year — since Microsoft does not break its figures down by year.Notes for reviewers
defaultconfig key is now a genuine last resort, reachable only for Azure, whose CSV has noGLOBALrow.HashMap<Pattern, Double>to an ordered list.Patternhas identity equality, so once a key carries several years the map would have split one regex across several entries; the list also makes the match order deterministic.il-central-1used to have a standalone row carrying the Middle East average. It now has its own 2025 figure and falls back to the(me|il)-.+geography average for earlier years.RowColumnTestis new rather than an addition toColumnTest, becauseColumnTest,CURColumnTestandAzureColumnTestare written against JUnit 4 and the project only ships the Jupiter engine — Surefire skips all three silently. Worth fixing, but separately from this change.mvn test: 362 tests, all passing.