Estimate LLM inference impacts from Azure Foundry token meters

docs/src/modules.md

@@ -43,7 +43,7 @@ flowchart LR
 | [Serverless](#serverless) | AWS | `operational_energy_kwh` | [Tailpipe](https://tailpipe.ai/methodology/serverless-explained/) |
 | [Accelerators](#accelerators) | AWS | `operational_energy_kwh` | [Cloud Carbon Footprint](https://www.cloudcarbonfootprint.org/) |
 | [Compute — Boavizta](#compute-boavizta) | AWS, Azure | `operational_energy_kwh`, `embodied_emissions_co2eq_g`, `embodied_adp_sbeq_g` | [BoaviztAPI](https://doc.api.boavizta.org/) |
-| [LLM inference — EcoLogits](#llm-inference-ecologits) | AWS | `operational_energy_kwh`, `embodied_emissions_co2eq_g` | [EcoLogits](https://ecologits.ai/) |
+| [LLM inference — EcoLogits](#llm-inference-ecologits) | AWS, Azure | `operational_energy_kwh`, `embodied_emissions_co2eq_g` | [EcoLogits](https://ecologits.ai/) |
 | [PWUE](#pwue) | AWS, Azure | `power_usage_effectiveness`, `water_usage_effectiveness` | Provider-published data |
 | [AverageCarbonIntensity](#averagecarbonintensity) | AWS, Azure | `carbon_intensity` | [Ember](https://ember-energy.org/) |
 | [OperationalEmissions](#operationalemissions) | AWS, Azure | `operational_emissions_co2eq_g` | — |
@@ -194,21 +194,34 @@ Each provider has two variants:
 
 ### LLM inference — EcoLogits
 
-Estimates the energy consumption and embodied emissions of LLM inference on **AWS Bedrock**,
-based on static per-model coefficients derived from the [EcoLogits](https://ecologits.ai/)
-project. Like `BoaviztAPIstatic`, a static data file bundled in the JAR is loaded at
-initialisation time; the module then matches Bedrock CUR rows to per-model coefficients.
+Estimates the energy consumption and embodied emissions of LLM inference on **AWS Bedrock**
+and **Azure AI Foundry** (initially the Azure OpenAI models), based on static per-model
+coefficients derived from the [EcoLogits](https://ecologits.ai/) project. Like `BoaviztAPIstatic`, a static data file
+bundled in the JAR is loaded at initialisation time; the modules then match billing rows to
+per-model coefficients.
 
-The module parses the `line_item_usage_type` field (format:
+**BedrockEcoLogits** parses the `line_item_usage_type` field (format:
 `{REGION}-{ModelKey}-{input|output}-tokens[-batch]`) to extract both the model key and the
 token type, then normalises the token count from `pricing_unit` (handling real-world values
 such as `1K tokens` or `1M tokens`). Only output-token rows are scored — the EcoLogits
 methodology attributes ~all generation cost to the autoregressive output phase, so
 input-token rows are skipped.
 
+**AzureFoundryTokenEcoLogits** does the same for token meters billed under the `Azure OpenAI` (or
+newer `Foundry Models`) category: the model label and token direction are extracted from
+`MeterName` (e.g. `GPT 5 outpt Glbl 1M Tokens`) and the token count is read from `Quantity`
+(`ConsumedQuantity` in FOCUS reports). Both hold the number of tokens consumed: the `1K`/`1M`
+unit in `UnitOfMeasure` only describes the pricing block and does not scale the quantity —
+Microsoft's FOCUS conversion defines `ConsumedQuantity = Quantity` and
+`ContractedCost = UnitPrice × Quantity / x_PricingBlockSize`. Provisioned throughput (PTU),
+hourly hosting and fine-tuning meters are not token-based and are not covered. The other
+Foundry model families (Mistral, Cohere, Llama, ...) bill through the same kind of token
+meters and only need `mapping.csv` entries verified against real exports — the module
+initially ships with Azure OpenAI mappings.
+
 | | |
 |---|---|
-| **Class** | `com.digitalpebble.spruce.modules.ecologits.BedrockEcoLogits` |
+| **Classes** | `com.digitalpebble.spruce.modules.ecologits.BedrockEcoLogits`<br>`com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits` |
 | **Writes** | `operational_energy_kwh`, `embodied_emissions_co2eq_g` |
 
 !!! note "Batch size assumption"

src/main/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogits.java

@@ -0,0 +1,177 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package com.digitalpebble.spruce.modules.ecologits;
+
+import com.digitalpebble.spruce.AzureColumn;
+import com.digitalpebble.spruce.AzureFOCUSColumn;
+import com.digitalpebble.spruce.Column;
+import com.digitalpebble.spruce.EnrichmentModule;
+import com.digitalpebble.spruce.FOCUSColumn;
+import com.digitalpebble.spruce.ReportFormat;
+import com.digitalpebble.spruce.RowColumn;
+import org.apache.spark.sql.Row;
+
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+
+import static com.digitalpebble.spruce.SpruceColumn.EMBODIED_EMISSIONS;
+import static com.digitalpebble.spruce.SpruceColumn.ENERGY_USED;
+
+/**
+ * Enrichment module estimating energy consumption and embodied emissions
+ * for LLM inference billed through Azure AI Foundry token meters.
+ * <p>
+ * Token usage is billed through meters under the {@code Azure OpenAI} (or, for
+ * newer meters, {@code Foundry Models}) category. The module extracts the model
+ * label and the token direction from {@code MeterName} (e.g.
+ * {@code "GPT 5 outpt Glbl 1M Tokens"}), maps the label to the matching
+ * {@link EcoLogits} coefficients via {@code ecologits/mapping.csv}, and applies
+ * them to the consumed token count: {@code Quantity} in native cost details
+ * exports, {@code ConsumedQuantity} in FOCUS exports. Both hold the number of
+ * tokens consumed — the 1K/1M unit in {@code UnitOfMeasure} only describes the
+ * pricing block ({@code ContractedCost = UnitPrice × Quantity / x_PricingBlockSize}
+ * in Microsoft's FOCUS conversion rules) and must not scale the quantity.
+ * <p>
+ * Coefficients only describe output tokens; input-token rows are ignored
+ * (the EcoLogits methodology attributes ~all generation cost to output tokens).
+ * Provisioned throughput (PTU), hourly hosting and fine-tuning meters are not
+ * token-based and are therefore not covered. Other Foundry model families
+ * (Mistral, Cohere, Llama, ...) bill through the same kind of token meters and
+ * only need entries in {@code mapping.csv} verified against real exports —
+ * initially the module ships with Azure OpenAI mappings. Reasoning and
+ * embedding token meters carry no input/output marker and are skipped too,
+ * which slightly underestimates the impacts of reasoning models.
+ */
+public class AzureFoundryTokenEcoLogits implements EnrichmentModule {
+
+    private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(AzureFoundryTokenEcoLogits.class);
+
+    private static final Set<String> METER_CATEGORIES = Set.of("Azure OpenAI", "Foundry Models");
+
+    // Direction markers observed in Azure OpenAI meter names ("5.1 codex opt Gl 1M Tokens"
+    // pairs with "5.1 codex inp Gl 1M Tokens", so "opt" is an output marker).
+    private static final Set<String> INPUT_MARKERS = Set.of("inp", "inpt", "input");
+    private static final Set<String> OUTPUT_MARKERS = Set.of("outp", "outpt", "out", "opt", "output");
+
+    // Deployment/pricing qualifiers that may precede the direction marker and are not
+    // part of the model label (batch pricing, cached tokens, short/long context, ...).
+    private static final Set<String> QUALIFIERS = Set.of("batch", "cchd", "cd", "wr", "pp", "shortco", "longco");
+
+    private EcoLogits impacts;
+
+    protected RowColumn meterCategory = AzureColumn.METER_CATEGORY;
+    protected RowColumn meterName = AzureColumn.METER_NAME;
+    protected RowColumn quantity = AzureColumn.QUANTITY;
+
+    @Override
+    public void bindReportFormat(ReportFormat reportFormat) {
+        if (reportFormat == ReportFormat.FOCUS) {
+            meterCategory = AzureFOCUSColumn.X_SKU_METER_CATEGORY;
+            meterName = AzureFOCUSColumn.X_SKU_METER_NAME;
+            quantity = FOCUSColumn.CONSUMED_QUANTITY;
+        } else {
+            meterCategory = AzureColumn.METER_CATEGORY;
+            meterName = AzureColumn.METER_NAME;
+            quantity = AzureColumn.QUANTITY;
+        }
+    }
+
+    @Override
+    public void init(Map<String, Object> params) {
+        if (impacts == null) {
+            impacts = new EcoLogits();
+            impacts.load();
+        }
+    }
+
+    /** Test hook: inject a pre-built EcoLogits instance before {@link #init(Map)}. */
+    void setEcoLogits(EcoLogits impacts) {
+        this.impacts = impacts;
+    }
+
+    @Override
+    public Column[] columnsNeeded() {
+        return new Column[]{meterCategory, meterName, quantity};
+    }
+
+    @Override
+    public Column[] columnsAdded() {
+        return new Column[]{ENERGY_USED, EMBODIED_EMISSIONS};
+    }
+
+    @Override
+    public void enrich(Row row, Map<Column, Object> enrichedValues) {
+        String category = this.meterCategory.getString(row);
+        if (category == null || !METER_CATEGORIES.contains(category)) {
+            return;
+        }
+
+        String[] parsed = parseMeterName(this.meterName.getString(row));
+        if (parsed == null || "input".equals(parsed[1])) {
+            return;
+        }
+
+        EcoLogits.ModelImpacts modelImpacts = impacts.getImpacts(parsed[0]);
+        if (modelImpacts == null) {
+            return;
+        }
+
+        if (this.quantity.isNullAt(row)) {
+            return;
+        }
+        double totalTokens = this.quantity.getDouble(row);
+        if (totalTokens <= 0) {
+            return;
+        }
+
+        double per1k = totalTokens / 1_000.0;
+        double energyKwh = per1k * modelImpacts.getEnergyKwhPer1kOutputTokens();
+        double embodiedEmissions = per1k * modelImpacts.getGwpEmbodiedGPer1kOutputTokens();
+
+        enrichedValues.put(ENERGY_USED, energyKwh);
+        enrichedValues.put(EMBODIED_EMISSIONS, embodiedEmissions);
+
+        LOG.debug("Azure OpenAI model={} outputTokens={} energy_kwh={} embodied_g={}",
+                parsed[0], totalTokens, energyKwh, embodiedEmissions);
+    }
+
+    /**
+     * Parses an Azure OpenAI token meter name such as {@code "GPT 5 outpt Glbl 1M Tokens"}
+     * or {@code "5 mini pp Inp Gl 1M Tokens"}: the words before the first input/output
+     * marker form the model label (minus pricing qualifiers like {@code Batch} or
+     * {@code cchd}), lowercased so it can be looked up in {@code mapping.csv}.
+     *
+     * @return [modelLabel, "input"|"output"], or {@code null} if the meter is not a
+     * recognisable token meter
+     */
+    static String[] parseMeterName(String meterName) {
+        if (meterName == null || meterName.isBlank()) {
+            return null;
+        }
+        String[] tokens = meterName.trim().split("\\s+");
+        if (!"tokens".equalsIgnoreCase(tokens[tokens.length - 1])) {
+            return null;
+        }
+
+        StringBuilder label = new StringBuilder();
+        for (String token : tokens) {
+            String lower = token.toLowerCase(Locale.ROOT);
+            if (INPUT_MARKERS.contains(lower) || OUTPUT_MARKERS.contains(lower)) {
+                if (label.isEmpty()) {
+                    return null;
+                }
+                String direction = INPUT_MARKERS.contains(lower) ? "input" : "output";
+                return new String[]{label.toString(), direction};
+            }
+            if (!QUALIFIERS.contains(lower)) {
+                if (!label.isEmpty()) {
+                    label.append(' ');
+                }
+                label.append(lower);
+            }
+        }
+        // no direction marker found
+        return null;
+    }
+}

src/main/resources/default-config-azure-focus.json

@@ -23,6 +23,9 @@
     {
       "className": "com.digitalpebble.spruce.modules.boavizta.azure.BoaviztAPIstatic"
     },
+    {
+      "className": "com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits"
+    },
     {
       "className": "com.digitalpebble.spruce.modules.PWUE",
       "config": {

src/main/resources/default-config-azure.json

@@ -23,6 +23,9 @@
     {
       "className": "com.digitalpebble.spruce.modules.boavizta.azure.BoaviztAPIstatic"
     },
+    {
+      "className": "com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits"
+    },
     {
       "className": "com.digitalpebble.spruce.modules.PWUE",
       "config": {

src/main/resources/ecologits/mapping.csv

@@ -8,3 +8,16 @@ label,provider,model_name
 Mistral7B,huggingface_hub,mistralai/Mistral-7B-v0.3
 MistralLarge,mistralai,mistral-large-latest
 Mixtral8x7B,huggingface_hub,mistralai/Mixtral-8x7B-Instruct-v0.1
+# Azure OpenAI meter labels: lowercased words of MeterName before the input/output
+# marker, minus pricing qualifiers (see AzureFoundryTokenEcoLogits#parseMeterName).
+# Example: "GPT 5 outpt Glbl 1M Tokens" → label: gpt 5
+gpt 5,openai,gpt-5
+5,openai,gpt-5
+5 mini,openai,gpt-5-mini
+5 nano,openai,gpt-5-nano
+5.1,openai,gpt-5.1
+5.1 codex,openai,gpt-5.1-codex
+5.4,openai,gpt-5.4
+gpt 4.1,openai,gpt-4.1
+o1,openai,o1
+o1 1217,openai,o1-2024-12-17

src/test/java/com/digitalpebble/spruce/AzureFoundryTokenEndToEndTest.java

@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package com.digitalpebble.spruce;
+
+import org.apache.spark.sql.Dataset;
+import org.apache.spark.sql.Encoder;
+import org.apache.spark.sql.Row;
+import org.apache.spark.sql.SparkSession;
+import org.apache.spark.sql.catalyst.encoders.RowEncoder;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+
+import static org.apache.spark.sql.functions.lit;
+import static org.junit.jupiter.api.Assertions.*;
+
+/**
+ * Runs minimal Azure OpenAI billing exports — one native, one FOCUS — through the real
+ * default configurations and the {@link EnrichmentPipeline}, as {@link SparkJob} does.
+ * The key invariant: the same inference expressed in both report formats must yield the
+ * same estimated impacts.
+ **/
+public class AzureFoundryTokenEndToEndTest {
+
+    private static SparkSession spark;
+
+    private static final String ENERGY = SpruceColumn.ENERGY_USED.getLabel();
+
+    @BeforeAll
+    static void startSpark() {
+        spark = SparkSession.builder()
+                .appName("AzureFoundryTokenEndToEndTest")
+                .master("local[1]")
+                .config("spark.ui.enabled", "false")
+                .getOrCreate();
+    }
+
+    @AfterAll
+    static void stopSpark() {
+        spark.stop();
+    }
+
+    /** Replicates the SparkJob steps: read, normalise, add module columns, run the pipeline. */
+    private List<Row> enrich(String resource, ReportFormat reportFormat) throws Exception {
+        String path = getClass().getResource(resource).getPath();
+        Dataset<Row> dataframe = spark.read().option("header", "true").option("inferSchema", "true")
+                .option("quote", "\"")
+                .option("escape", "\"").csv(path);
+        dataframe = SparkJob.normalizeAzureColumns(dataframe, reportFormat);
+
+        Config config = Config.loadDefault(Provider.AZURE, reportFormat);
+        for (EnrichmentModule module : config.getModules()) {
+            for (Column c : module.columnsNeeded()) {
+                assertFalse(dataframe.schema().getFieldIndex(c.getLabel()).isEmpty(),
+                        "Fixture " + resource + " misses column '" + c.getLabel()
+                                + "' needed by " + module.getClass().getSimpleName());
+            }
+            for (Column c : module.columnsAdded()) {
+                dataframe = dataframe.withColumn(c.getLabel(), lit(null).cast(c.getType()));
+            }
+        }
+
+        Encoder<Row> encoder = RowEncoder.encoderFor(dataframe.schema());
+        return dataframe.mapPartitions(new EnrichmentPipeline(config), encoder).collectAsList();
+    }
+
+    private static Double energy(Row row) {
+        int index = row.fieldIndex(ENERGY);
+        return row.isNullAt(index) ? null : row.getDouble(index);
+    }
+
+    @Test
+    void ecologitsRunsBeforeFactorAndImpactModules() throws Exception {
+        for (ReportFormat format : new ReportFormat[]{ReportFormat.NATIVE, ReportFormat.FOCUS}) {
+            List<String> names = Config.loadDefault(Provider.AZURE, format).getModules().stream()
+                    .map(m -> m.getClass().getSimpleName()).toList();
+            int ecologits = names.indexOf("AzureFoundryTokenEcoLogits");
+            assertTrue(ecologits >= 0, "AzureFoundryTokenEcoLogits missing from " + format + " config");
+            for (String downstream : new String[]{"PWUE", "Water", "OperationalEmissions"}) {
+                assertTrue(ecologits < names.indexOf(downstream),
+                        "AzureFoundryTokenEcoLogits must run before " + downstream + " (" + format + ")");
+            }
+        }
+    }
+
+    @Test
+    void enrichesNativeExport() throws Exception {
+        List<Row> rows = enrich("/azure/native-openai.csv", ReportFormat.NATIVE);
+        assertEquals(6, rows.size());
+
+        // 1,000,000 output tokens of gpt-5 (Quantity is the consumed token count; the "1M"
+        // UnitOfMeasure is only the pricing block): energy matches the bundled coefficients
+        com.digitalpebble.spruce.modules.ecologits.EcoLogits impacts = new com.digitalpebble.spruce.modules.ecologits.EcoLogits();
+        impacts.load();
+        double expected = 1_000.0 * impacts.getImpacts("gpt 5").getEnergyKwhPer1kOutputTokens();
+        assertEquals(expected, energy(rows.get(0)), 1e-12);
+
+        assertNull(energy(rows.get(1)), "input tokens must not be estimated");
+        assertNull(energy(rows.get(2)), "unmapped model must not be estimated");
+        assertNull(energy(rows.get(3)), "non-token meter must not be estimated");
+        // Quantity is the token count whatever the pricing block (here a meter priced per 1K)
+        double expected41 = 2.0 * impacts.getImpacts("gpt 4.1").getEnergyKwhPer1kOutputTokens();
+        assertEquals(expected41, energy(rows.get(4)), 1e-12);
+        assertNull(energy(rows.get(5)), "non-usage charge must not be enriched");
+    }
+
+    @Test
+    void focusExportMatchesNativeExport() throws Exception {
+        List<Row> nativeRows = enrich("/azure/native-openai.csv", ReportFormat.NATIVE);
+        List<Row> focusRows = enrich("/azure/focus-openai.csv", ReportFormat.FOCUS);
+        assertEquals(2, focusRows.size());
+
+        // same inference (1M output tokens of gpt-5), same impacts in both formats
+        assertNotNull(energy(focusRows.get(0)));
+        assertEquals(energy(nativeRows.get(0)), energy(focusRows.get(0)), 1e-12);
+
+        assertNull(energy(focusRows.get(1)), "input tokens must not be estimated");
+    }
+}