diff --git a/docs/src/modules.md b/docs/src/modules.md index d5d87aaa..4c7aa487 100644 --- a/docs/src/modules.md +++ b/docs/src/modules.md @@ -43,7 +43,7 @@ flowchart LR | [Serverless](#serverless) | AWS | `operational_energy_kwh` | [Tailpipe](https://tailpipe.ai/methodology/serverless-explained/) | | [Accelerators](#accelerators) | AWS | `operational_energy_kwh` | [Cloud Carbon Footprint](https://www.cloudcarbonfootprint.org/) | | [Compute — Boavizta](#compute-boavizta) | AWS, Azure | `operational_energy_kwh`, `embodied_emissions_co2eq_g`, `embodied_adp_sbeq_g` | [BoaviztAPI](https://doc.api.boavizta.org/) | -| [LLM inference — EcoLogits](#llm-inference-ecologits) | AWS | `operational_energy_kwh`, `embodied_emissions_co2eq_g` | [EcoLogits](https://ecologits.ai/) | +| [LLM inference — EcoLogits](#llm-inference-ecologits) | AWS, Azure | `operational_energy_kwh`, `embodied_emissions_co2eq_g` | [EcoLogits](https://ecologits.ai/) | | [PWUE](#pwue) | AWS, Azure | `power_usage_effectiveness`, `water_usage_effectiveness` | Provider-published data | | [AverageCarbonIntensity](#averagecarbonintensity) | AWS, Azure | `carbon_intensity` | [Ember](https://ember-energy.org/) | | [OperationalEmissions](#operationalemissions) | AWS, Azure | `operational_emissions_co2eq_g` | — | @@ -194,21 +194,34 @@ Each provider has two variants: ### LLM inference — EcoLogits -Estimates the energy consumption and embodied emissions of LLM inference on **AWS Bedrock**, -based on static per-model coefficients derived from the [EcoLogits](https://ecologits.ai/) -project. Like `BoaviztAPIstatic`, a static data file bundled in the JAR is loaded at -initialisation time; the module then matches Bedrock CUR rows to per-model coefficients. +Estimates the energy consumption and embodied emissions of LLM inference on **AWS Bedrock** +and **Azure AI Foundry** (initially the Azure OpenAI models), based on static per-model +coefficients derived from the [EcoLogits](https://ecologits.ai/) project. Like `BoaviztAPIstatic`, a static data file +bundled in the JAR is loaded at initialisation time; the modules then match billing rows to +per-model coefficients. -The module parses the `line_item_usage_type` field (format: +**BedrockEcoLogits** parses the `line_item_usage_type` field (format: `{REGION}-{ModelKey}-{input|output}-tokens[-batch]`) to extract both the model key and the token type, then normalises the token count from `pricing_unit` (handling real-world values such as `1K tokens` or `1M tokens`). Only output-token rows are scored — the EcoLogits methodology attributes ~all generation cost to the autoregressive output phase, so input-token rows are skipped. +**AzureFoundryTokenEcoLogits** does the same for token meters billed under the `Azure OpenAI` (or +newer `Foundry Models`) category: the model label and token direction are extracted from +`MeterName` (e.g. `GPT 5 outpt Glbl 1M Tokens`) and the token count is read from `Quantity` +(`ConsumedQuantity` in FOCUS reports). Both hold the number of tokens consumed: the `1K`/`1M` +unit in `UnitOfMeasure` only describes the pricing block and does not scale the quantity — +Microsoft's FOCUS conversion defines `ConsumedQuantity = Quantity` and +`ContractedCost = UnitPrice × Quantity / x_PricingBlockSize`. Provisioned throughput (PTU), +hourly hosting and fine-tuning meters are not token-based and are not covered. The other +Foundry model families (Mistral, Cohere, Llama, ...) bill through the same kind of token +meters and only need `mapping.csv` entries verified against real exports — the module +initially ships with Azure OpenAI mappings. + | | | |---|---| -| **Class** | `com.digitalpebble.spruce.modules.ecologits.BedrockEcoLogits` | +| **Classes** | `com.digitalpebble.spruce.modules.ecologits.BedrockEcoLogits`
`com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits` | | **Writes** | `operational_energy_kwh`, `embodied_emissions_co2eq_g` | !!! note "Batch size assumption" diff --git a/src/main/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogits.java b/src/main/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogits.java new file mode 100644 index 00000000..72243cbc --- /dev/null +++ b/src/main/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogits.java @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: Apache-2.0 + +package com.digitalpebble.spruce.modules.ecologits; + +import com.digitalpebble.spruce.AzureColumn; +import com.digitalpebble.spruce.AzureFOCUSColumn; +import com.digitalpebble.spruce.Column; +import com.digitalpebble.spruce.EnrichmentModule; +import com.digitalpebble.spruce.FOCUSColumn; +import com.digitalpebble.spruce.ReportFormat; +import com.digitalpebble.spruce.RowColumn; +import org.apache.spark.sql.Row; + +import java.util.Locale; +import java.util.Map; +import java.util.Set; + +import static com.digitalpebble.spruce.SpruceColumn.EMBODIED_EMISSIONS; +import static com.digitalpebble.spruce.SpruceColumn.ENERGY_USED; + +/** + * Enrichment module estimating energy consumption and embodied emissions + * for LLM inference billed through Azure AI Foundry token meters. + *

+ * Token usage is billed through meters under the {@code Azure OpenAI} (or, for + * newer meters, {@code Foundry Models}) category. The module extracts the model + * label and the token direction from {@code MeterName} (e.g. + * {@code "GPT 5 outpt Glbl 1M Tokens"}), maps the label to the matching + * {@link EcoLogits} coefficients via {@code ecologits/mapping.csv}, and applies + * them to the consumed token count: {@code Quantity} in native cost details + * exports, {@code ConsumedQuantity} in FOCUS exports. Both hold the number of + * tokens consumed — the 1K/1M unit in {@code UnitOfMeasure} only describes the + * pricing block ({@code ContractedCost = UnitPrice × Quantity / x_PricingBlockSize} + * in Microsoft's FOCUS conversion rules) and must not scale the quantity. + *

+ * Coefficients only describe output tokens; input-token rows are ignored + * (the EcoLogits methodology attributes ~all generation cost to output tokens). + * Provisioned throughput (PTU), hourly hosting and fine-tuning meters are not + * token-based and are therefore not covered. Other Foundry model families + * (Mistral, Cohere, Llama, ...) bill through the same kind of token meters and + * only need entries in {@code mapping.csv} verified against real exports — + * initially the module ships with Azure OpenAI mappings. Reasoning and + * embedding token meters carry no input/output marker and are skipped too, + * which slightly underestimates the impacts of reasoning models. + */ +public class AzureFoundryTokenEcoLogits implements EnrichmentModule { + + private static final org.slf4j.Logger LOG = org.slf4j.LoggerFactory.getLogger(AzureFoundryTokenEcoLogits.class); + + private static final Set METER_CATEGORIES = Set.of("Azure OpenAI", "Foundry Models"); + + // Direction markers observed in Azure OpenAI meter names ("5.1 codex opt Gl 1M Tokens" + // pairs with "5.1 codex inp Gl 1M Tokens", so "opt" is an output marker). + private static final Set INPUT_MARKERS = Set.of("inp", "inpt", "input"); + private static final Set OUTPUT_MARKERS = Set.of("outp", "outpt", "out", "opt", "output"); + + // Deployment/pricing qualifiers that may precede the direction marker and are not + // part of the model label (batch pricing, cached tokens, short/long context, ...). + private static final Set QUALIFIERS = Set.of("batch", "cchd", "cd", "wr", "pp", "shortco", "longco"); + + private EcoLogits impacts; + + protected RowColumn meterCategory = AzureColumn.METER_CATEGORY; + protected RowColumn meterName = AzureColumn.METER_NAME; + protected RowColumn quantity = AzureColumn.QUANTITY; + + @Override + public void bindReportFormat(ReportFormat reportFormat) { + if (reportFormat == ReportFormat.FOCUS) { + meterCategory = AzureFOCUSColumn.X_SKU_METER_CATEGORY; + meterName = AzureFOCUSColumn.X_SKU_METER_NAME; + quantity = FOCUSColumn.CONSUMED_QUANTITY; + } else { + meterCategory = AzureColumn.METER_CATEGORY; + meterName = AzureColumn.METER_NAME; + quantity = AzureColumn.QUANTITY; + } + } + + @Override + public void init(Map params) { + if (impacts == null) { + impacts = new EcoLogits(); + impacts.load(); + } + } + + /** Test hook: inject a pre-built EcoLogits instance before {@link #init(Map)}. */ + void setEcoLogits(EcoLogits impacts) { + this.impacts = impacts; + } + + @Override + public Column[] columnsNeeded() { + return new Column[]{meterCategory, meterName, quantity}; + } + + @Override + public Column[] columnsAdded() { + return new Column[]{ENERGY_USED, EMBODIED_EMISSIONS}; + } + + @Override + public void enrich(Row row, Map enrichedValues) { + String category = this.meterCategory.getString(row); + if (category == null || !METER_CATEGORIES.contains(category)) { + return; + } + + String[] parsed = parseMeterName(this.meterName.getString(row)); + if (parsed == null || "input".equals(parsed[1])) { + return; + } + + EcoLogits.ModelImpacts modelImpacts = impacts.getImpacts(parsed[0]); + if (modelImpacts == null) { + return; + } + + if (this.quantity.isNullAt(row)) { + return; + } + double totalTokens = this.quantity.getDouble(row); + if (totalTokens <= 0) { + return; + } + + double per1k = totalTokens / 1_000.0; + double energyKwh = per1k * modelImpacts.getEnergyKwhPer1kOutputTokens(); + double embodiedEmissions = per1k * modelImpacts.getGwpEmbodiedGPer1kOutputTokens(); + + enrichedValues.put(ENERGY_USED, energyKwh); + enrichedValues.put(EMBODIED_EMISSIONS, embodiedEmissions); + + LOG.debug("Azure OpenAI model={} outputTokens={} energy_kwh={} embodied_g={}", + parsed[0], totalTokens, energyKwh, embodiedEmissions); + } + + /** + * Parses an Azure OpenAI token meter name such as {@code "GPT 5 outpt Glbl 1M Tokens"} + * or {@code "5 mini pp Inp Gl 1M Tokens"}: the words before the first input/output + * marker form the model label (minus pricing qualifiers like {@code Batch} or + * {@code cchd}), lowercased so it can be looked up in {@code mapping.csv}. + * + * @return [modelLabel, "input"|"output"], or {@code null} if the meter is not a + * recognisable token meter + */ + static String[] parseMeterName(String meterName) { + if (meterName == null || meterName.isBlank()) { + return null; + } + String[] tokens = meterName.trim().split("\\s+"); + if (!"tokens".equalsIgnoreCase(tokens[tokens.length - 1])) { + return null; + } + + StringBuilder label = new StringBuilder(); + for (String token : tokens) { + String lower = token.toLowerCase(Locale.ROOT); + if (INPUT_MARKERS.contains(lower) || OUTPUT_MARKERS.contains(lower)) { + if (label.isEmpty()) { + return null; + } + String direction = INPUT_MARKERS.contains(lower) ? "input" : "output"; + return new String[]{label.toString(), direction}; + } + if (!QUALIFIERS.contains(lower)) { + if (!label.isEmpty()) { + label.append(' '); + } + label.append(lower); + } + } + // no direction marker found + return null; + } +} diff --git a/src/main/resources/default-config-azure-focus.json b/src/main/resources/default-config-azure-focus.json index c6dd1d19..415d0ba9 100644 --- a/src/main/resources/default-config-azure-focus.json +++ b/src/main/resources/default-config-azure-focus.json @@ -23,6 +23,9 @@ { "className": "com.digitalpebble.spruce.modules.boavizta.azure.BoaviztAPIstatic" }, + { + "className": "com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits" + }, { "className": "com.digitalpebble.spruce.modules.PWUE", "config": { diff --git a/src/main/resources/default-config-azure.json b/src/main/resources/default-config-azure.json index ec0cbeab..287ed81d 100644 --- a/src/main/resources/default-config-azure.json +++ b/src/main/resources/default-config-azure.json @@ -23,6 +23,9 @@ { "className": "com.digitalpebble.spruce.modules.boavizta.azure.BoaviztAPIstatic" }, + { + "className": "com.digitalpebble.spruce.modules.ecologits.AzureFoundryTokenEcoLogits" + }, { "className": "com.digitalpebble.spruce.modules.PWUE", "config": { diff --git a/src/main/resources/ecologits/mapping.csv b/src/main/resources/ecologits/mapping.csv index 54c9f273..7d2c3813 100644 --- a/src/main/resources/ecologits/mapping.csv +++ b/src/main/resources/ecologits/mapping.csv @@ -8,3 +8,16 @@ label,provider,model_name Mistral7B,huggingface_hub,mistralai/Mistral-7B-v0.3 MistralLarge,mistralai,mistral-large-latest Mixtral8x7B,huggingface_hub,mistralai/Mixtral-8x7B-Instruct-v0.1 +# Azure OpenAI meter labels: lowercased words of MeterName before the input/output +# marker, minus pricing qualifiers (see AzureFoundryTokenEcoLogits#parseMeterName). +# Example: "GPT 5 outpt Glbl 1M Tokens" → label: gpt 5 +gpt 5,openai,gpt-5 +5,openai,gpt-5 +5 mini,openai,gpt-5-mini +5 nano,openai,gpt-5-nano +5.1,openai,gpt-5.1 +5.1 codex,openai,gpt-5.1-codex +5.4,openai,gpt-5.4 +gpt 4.1,openai,gpt-4.1 +o1,openai,o1 +o1 1217,openai,o1-2024-12-17 diff --git a/src/test/java/com/digitalpebble/spruce/AzureFoundryTokenEndToEndTest.java b/src/test/java/com/digitalpebble/spruce/AzureFoundryTokenEndToEndTest.java new file mode 100644 index 00000000..e899d7ee --- /dev/null +++ b/src/test/java/com/digitalpebble/spruce/AzureFoundryTokenEndToEndTest.java @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: Apache-2.0 + +package com.digitalpebble.spruce; + +import org.apache.spark.sql.Dataset; +import org.apache.spark.sql.Encoder; +import org.apache.spark.sql.Row; +import org.apache.spark.sql.SparkSession; +import org.apache.spark.sql.catalyst.encoders.RowEncoder; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.List; + +import static org.apache.spark.sql.functions.lit; +import static org.junit.jupiter.api.Assertions.*; + +/** + * Runs minimal Azure OpenAI billing exports — one native, one FOCUS — through the real + * default configurations and the {@link EnrichmentPipeline}, as {@link SparkJob} does. + * The key invariant: the same inference expressed in both report formats must yield the + * same estimated impacts. + **/ +public class AzureFoundryTokenEndToEndTest { + + private static SparkSession spark; + + private static final String ENERGY = SpruceColumn.ENERGY_USED.getLabel(); + + @BeforeAll + static void startSpark() { + spark = SparkSession.builder() + .appName("AzureFoundryTokenEndToEndTest") + .master("local[1]") + .config("spark.ui.enabled", "false") + .getOrCreate(); + } + + @AfterAll + static void stopSpark() { + spark.stop(); + } + + /** Replicates the SparkJob steps: read, normalise, add module columns, run the pipeline. */ + private List enrich(String resource, ReportFormat reportFormat) throws Exception { + String path = getClass().getResource(resource).getPath(); + Dataset dataframe = spark.read().option("header", "true").option("inferSchema", "true") + .option("quote", "\"") + .option("escape", "\"").csv(path); + dataframe = SparkJob.normalizeAzureColumns(dataframe, reportFormat); + + Config config = Config.loadDefault(Provider.AZURE, reportFormat); + for (EnrichmentModule module : config.getModules()) { + for (Column c : module.columnsNeeded()) { + assertFalse(dataframe.schema().getFieldIndex(c.getLabel()).isEmpty(), + "Fixture " + resource + " misses column '" + c.getLabel() + + "' needed by " + module.getClass().getSimpleName()); + } + for (Column c : module.columnsAdded()) { + dataframe = dataframe.withColumn(c.getLabel(), lit(null).cast(c.getType())); + } + } + + Encoder encoder = RowEncoder.encoderFor(dataframe.schema()); + return dataframe.mapPartitions(new EnrichmentPipeline(config), encoder).collectAsList(); + } + + private static Double energy(Row row) { + int index = row.fieldIndex(ENERGY); + return row.isNullAt(index) ? null : row.getDouble(index); + } + + @Test + void ecologitsRunsBeforeFactorAndImpactModules() throws Exception { + for (ReportFormat format : new ReportFormat[]{ReportFormat.NATIVE, ReportFormat.FOCUS}) { + List names = Config.loadDefault(Provider.AZURE, format).getModules().stream() + .map(m -> m.getClass().getSimpleName()).toList(); + int ecologits = names.indexOf("AzureFoundryTokenEcoLogits"); + assertTrue(ecologits >= 0, "AzureFoundryTokenEcoLogits missing from " + format + " config"); + for (String downstream : new String[]{"PWUE", "Water", "OperationalEmissions"}) { + assertTrue(ecologits < names.indexOf(downstream), + "AzureFoundryTokenEcoLogits must run before " + downstream + " (" + format + ")"); + } + } + } + + @Test + void enrichesNativeExport() throws Exception { + List rows = enrich("/azure/native-openai.csv", ReportFormat.NATIVE); + assertEquals(6, rows.size()); + + // 1,000,000 output tokens of gpt-5 (Quantity is the consumed token count; the "1M" + // UnitOfMeasure is only the pricing block): energy matches the bundled coefficients + com.digitalpebble.spruce.modules.ecologits.EcoLogits impacts = new com.digitalpebble.spruce.modules.ecologits.EcoLogits(); + impacts.load(); + double expected = 1_000.0 * impacts.getImpacts("gpt 5").getEnergyKwhPer1kOutputTokens(); + assertEquals(expected, energy(rows.get(0)), 1e-12); + + assertNull(energy(rows.get(1)), "input tokens must not be estimated"); + assertNull(energy(rows.get(2)), "unmapped model must not be estimated"); + assertNull(energy(rows.get(3)), "non-token meter must not be estimated"); + // Quantity is the token count whatever the pricing block (here a meter priced per 1K) + double expected41 = 2.0 * impacts.getImpacts("gpt 4.1").getEnergyKwhPer1kOutputTokens(); + assertEquals(expected41, energy(rows.get(4)), 1e-12); + assertNull(energy(rows.get(5)), "non-usage charge must not be enriched"); + } + + @Test + void focusExportMatchesNativeExport() throws Exception { + List nativeRows = enrich("/azure/native-openai.csv", ReportFormat.NATIVE); + List focusRows = enrich("/azure/focus-openai.csv", ReportFormat.FOCUS); + assertEquals(2, focusRows.size()); + + // same inference (1M output tokens of gpt-5), same impacts in both formats + assertNotNull(energy(focusRows.get(0))); + assertEquals(energy(nativeRows.get(0)), energy(focusRows.get(0)), 1e-12); + + assertNull(energy(focusRows.get(1)), "input tokens must not be estimated"); + } +} diff --git a/src/test/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogitsTest.java b/src/test/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogitsTest.java new file mode 100644 index 00000000..2cd243d0 --- /dev/null +++ b/src/test/java/com/digitalpebble/spruce/modules/ecologits/AzureFoundryTokenEcoLogitsTest.java @@ -0,0 +1,237 @@ +// SPDX-License-Identifier: Apache-2.0 + +package com.digitalpebble.spruce.modules.ecologits; + +import com.digitalpebble.spruce.*; +import org.apache.spark.sql.Row; +import org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema; +import org.apache.spark.sql.types.StructType; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Stream; + +import static com.digitalpebble.spruce.SpruceColumn.EMBODIED_EMISSIONS; +import static com.digitalpebble.spruce.SpruceColumn.ENERGY_USED; +import static org.junit.jupiter.api.Assertions.*; + +public class AzureFoundryTokenEcoLogitsTest { + + private AzureFoundryTokenEcoLogits module; + private StructType schema; + + private static final String TEST_MAPPING = "ecologits-test/mapping.csv"; + private static final String TEST_COEFFICIENTS = "ecologits-test/coefficients.csv"; + + // Coefficients in test-coefficients.csv: 1e-3 kWh and 5e-4 kg (=0.5 g) per 1k output tokens. + private static final double OUTPUT_ENERGY_PER_1K = 1.0e-3; + private static final double OUTPUT_EMBODIED_G_PER_1K = 0.5; + + @BeforeEach + void setUp() { + module = new AzureFoundryTokenEcoLogits(); + schema = Utils.getSchema(module); + EcoLogits impacts = new EcoLogits(TEST_MAPPING, TEST_COEFFICIENTS); + impacts.load(); + module.setEcoLogits(impacts); + module.init(new HashMap<>()); + } + + /** + * Creates a {@link Row} matching the schema produced by {@link Utils#getSchema(EnrichmentModule)} + * for either binding. + * + *

Schema order: meter category, meter name, quantity, {@code ENERGY_USED}, + * {@code EMBODIED_EMISSIONS} + */ + static Row createRow(StructType schema, String meterCategory, String meterName, Double quantity) { + Object[] values = new Object[5]; + values[0] = meterCategory; + values[1] = meterName; + values[2] = quantity; + values[3] = null; + values[4] = null; + return new GenericRowWithSchema(values, schema); + } + + @Test + void testColumnsNeeded() { + Column[] needed = module.columnsNeeded(); + assertEquals(3, needed.length); + assertEquals(AzureColumn.METER_CATEGORY, needed[0]); + assertEquals(AzureColumn.METER_NAME, needed[1]); + assertEquals(AzureColumn.QUANTITY, needed[2]); + } + + @Test + void testColumnsAdded() { + Column[] added = module.columnsAdded(); + assertEquals(2, added.length); + assertEquals(ENERGY_USED, added[0]); + assertEquals(EMBODIED_EMISSIONS, added[1]); + } + + @ParameterizedTest + @MethodSource("nullValueTestCases") + void testProcessWithNullValues(String meterCategory, String meterName, Double quantity) { + Row row = createRow(schema, meterCategory, meterName, quantity); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertTrue(enriched.isEmpty(), "Should skip rows with invalid null values"); + } + + static Stream nullValueTestCases() { + return Stream.of( + Arguments.of(null, "GPT 5 outpt Glbl 1M Tokens", 1.0), + Arguments.of("Azure OpenAI", null, 1.0), + Arguments.of("Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", null) + ); + } + + @ParameterizedTest + @MethodSource("unsupportedValueTestCases") + void testProcessWithUnsupportedValues(String meterCategory, String meterName, Double quantity) { + Row row = createRow(schema, meterCategory, meterName, quantity); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertTrue(enriched.isEmpty(), "Should skip unsupported categories or unknown models"); + } + + static Stream unsupportedValueTestCases() { + return Stream.of( + Arguments.of("Virtual Machines", "GPT 5 outpt Glbl 1M Tokens", 1.0), + Arguments.of("Storage", "GPT 5 outpt Glbl 1M Tokens", 1.0), + Arguments.of("Azure OpenAI", "UnknownModel outpt Glbl 1M Tokens", 1.0), + Arguments.of("Azure OpenAI", "Code-Interpreter-global Session", 1.0), + Arguments.of("Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", 0.0), + Arguments.of("Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", -5.0) + ); + } + + @Test + void testEnrichesOutputTokens() { + // Quantity is the consumed token count: 1,000,000 tokens = 1000 × 1k tokens. + // UnitOfMeasure ("1M") only describes the pricing block and must not scale it. + Row row = createRow(schema, "Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", 1_000_000.0); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertNotNull(enriched.get(ENERGY_USED)); + assertEquals(1000.0 * OUTPUT_ENERGY_PER_1K, ENERGY_USED.getDouble(enriched), 1e-12); + assertEquals(1000.0 * OUTPUT_EMBODIED_G_PER_1K, EMBODIED_EMISSIONS.getDouble(enriched), 1e-9); + } + + @Test + void testAcceptsFoundryModelsCategory() { + Row row = createRow(schema, "Foundry Models", "GPT 5 outpt Glbl 1M Tokens", 1.0); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertNotNull(enriched.get(ENERGY_USED)); + } + + @Test + void testSkipsInputTokens() { + // EcoLogits attributes ~all generation cost to output tokens; input rows are ignored. + Row row = createRow(schema, "Azure OpenAI", "5 mini pp Inp Gl 1M Tokens", 1.0); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertTrue(enriched.isEmpty()); + } + + @Test + void testQuantityIsTokenCountRegardlessOfPricingBlock() { + // A meter priced per 1K tokens still reports the consumed token count in Quantity + Row row = createRow(schema, "Azure OpenAI", "GPT 5 Outp regnl Tokens", 2_000.0); + Map enriched = new HashMap<>(); + module.enrich(row, enriched); + + assertEquals(2.0 * OUTPUT_ENERGY_PER_1K, ENERGY_USED.getDouble(enriched), 1e-12); + } + + private AzureFoundryTokenEcoLogits focusModule() { + AzureFoundryTokenEcoLogits focusModule = new AzureFoundryTokenEcoLogits(); + focusModule.bindReportFormat(ReportFormat.FOCUS); + EcoLogits impacts = new EcoLogits(TEST_MAPPING, TEST_COEFFICIENTS); + impacts.load(); + focusModule.setEcoLogits(impacts); + focusModule.init(new HashMap<>()); + return focusModule; + } + + @Test + void testFOCUSBindingColumns() { + AzureFoundryTokenEcoLogits focusModule = focusModule(); + assertEquals(3, focusModule.columnsNeeded().length); + assertEquals(AzureFOCUSColumn.X_SKU_METER_CATEGORY, focusModule.columnsNeeded()[0]); + assertEquals(AzureFOCUSColumn.X_SKU_METER_NAME, focusModule.columnsNeeded()[1]); + assertEquals(FOCUSColumn.CONSUMED_QUANTITY, focusModule.columnsNeeded()[2]); + } + + @Test + void testNativeAndFOCUSAgreeOnSameInference() { + // The same inference expressed in both report formats must yield the same impacts. + // Microsoft's FOCUS conversion defines ConsumedQuantity = Quantity for usage rows, so + // both columns hold the consumed token count (1,000,000 tokens here). + Row nativeRow = createRow(schema, "Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", 1_000_000.0); + Map nativeEnriched = new HashMap<>(); + module.enrich(nativeRow, nativeEnriched); + + AzureFoundryTokenEcoLogits focusModule = focusModule(); + Row focusRow = createRow(Utils.getSchema(focusModule), + "Azure OpenAI", "GPT 5 outpt Glbl 1M Tokens", 1_000_000.0); + Map focusEnriched = new HashMap<>(); + focusModule.enrich(focusRow, focusEnriched); + + assertNotNull(nativeEnriched.get(ENERGY_USED)); + assertNotNull(focusEnriched.get(ENERGY_USED)); + assertEquals(ENERGY_USED.getDouble(nativeEnriched), + ENERGY_USED.getDouble(focusEnriched), 1e-12); + assertEquals(EMBODIED_EMISSIONS.getDouble(nativeEnriched), + EMBODIED_EMISSIONS.getDouble(focusEnriched), 1e-9); + } + + @ParameterizedTest + @MethodSource("parseMeterNameCases") + void testParseMeterName(String meterName, String expectedLabel, String expectedDirection) { + String[] result = AzureFoundryTokenEcoLogits.parseMeterName(meterName); + if (expectedLabel == null) { + assertNull(result); + } else { + assertNotNull(result, "Expected a match for: " + meterName); + assertEquals(expectedLabel, result[0]); + assertEquals(expectedDirection, result[1]); + } + } + + // Meter names taken from the Azure Retail Prices API for Azure OpenAI / Foundry Models. + static Stream parseMeterNameCases() { + return Stream.of( + Arguments.of("GPT 5 outpt Glbl 1M Tokens", "gpt 5", "output"), + Arguments.of("5.1 codex opt Gl 1M Tokens", "5.1 codex", "output"), + Arguments.of("5 mini pp Inp Gl 1M Tokens", "5 mini", "input"), + Arguments.of("gpt 4.1 Inp regnl Tokens", "gpt 4.1", "input"), + Arguments.of("gpt-4o-rt-txt-1217 Outp glbl Tokens", "gpt-4o-rt-txt-1217", "output"), + Arguments.of("5.6 terra ShortCo Cd Inp PP Gl 1M Tokens", "5.6 terra", "input"), + Arguments.of("5.4 opt Dz 1M Tokens", "5.4", "output"), + Arguments.of("o1 1217 Outp Data Zone Tokens", "o1 1217", "output"), + Arguments.of("gpt rt aud 0828 cchd Inp glbl Tokens", "gpt rt aud 0828", "input"), + Arguments.of("5.4 pro Batch inp Dz 1M Tokens", "5.4 pro", "input"), + // no "Tokens" → not a token meter + Arguments.of("Code-Interpreter-global Session", null, null), + // no input/output marker + Arguments.of("gpt image 1 generations", null, null), + Arguments.of(null, null, null), + Arguments.of("", null, null) + ); + } + +} diff --git a/src/test/resources/azure/focus-openai.csv b/src/test/resources/azure/focus-openai.csv new file mode 100644 index 00000000..24c8928a --- /dev/null +++ b/src/test/resources/azure/focus-openai.csv @@ -0,0 +1,3 @@ +ChargeCategory,RegionId,x_SkuMeterCategory,x_SkuMeterSubcategory,x_SkuMeterName,x_PricingUnitDescription,ConsumedQuantity,BilledCost +Usage,swedencentral,Azure OpenAI,gpt-5,GPT 5 outpt Glbl 1M Tokens,1M Tokens,1000000,22.0 +Usage,swedencentral,Azure OpenAI,gpt-5,5 mini pp Inp Gl 1M Tokens,1M Tokens,3000000,1.5 diff --git a/src/test/resources/azure/native-openai.csv b/src/test/resources/azure/native-openai.csv new file mode 100644 index 00000000..01a4e914 --- /dev/null +++ b/src/test/resources/azure/native-openai.csv @@ -0,0 +1,7 @@ +Date,ChargeType,MeterCategory,MeterSubCategory,MeterName,UnitOfMeasure,Quantity,CostInBillingCurrency,SubscriptionId,ResourceLocation,Tags +2026-08-01,Usage,Azure OpenAI,gpt-5,GPT 5 outpt Glbl 1M Tokens,1M,1000000,22.0,sub-1,swedencentral, +2026-08-01,Usage,Azure OpenAI,gpt-5,5 mini pp Inp Gl 1M Tokens,1M,3000000,1.5,sub-1,swedencentral, +2026-08-01,Usage,Azure OpenAI,gpt image,gpt img 1.5 out img DZ 1M Tokens,1M,2000000,4.0,sub-1,swedencentral, +2026-08-01,Usage,Azure OpenAI,Code Interpreter,Code-Interpreter-global Session,1,5,0.15,sub-1,swedencentral, +2026-08-01,Usage,Azure OpenAI,gpt-4.1,gpt 4.1 Outp regnl Tokens,1K,2000,0.02,sub-1,swedencentral, +2026-08-01,Purchase,Azure OpenAI,gpt-5,GPT 5 outpt Glbl 1M Tokens,1M,7000000,70.0,sub-1,swedencentral, diff --git a/src/test/resources/ecologits-test/mapping.csv b/src/test/resources/ecologits-test/mapping.csv index ab663fc0..044dbe82 100644 --- a/src/test/resources/ecologits-test/mapping.csv +++ b/src/test/resources/ecologits-test/mapping.csv @@ -2,3 +2,6 @@ # Key is the model key extracted from line_item_usage_type (e.g. EUN1-Claude-output-tokens → Claude) label,provider,model_name Claude,test-provider,test-claude +# Azure meter labels (AzureFoundryTokenEcoLogitsTest) - lowercased words extracted from MeterName +gpt 5,test-provider,test-claude +5 mini,test-provider,test-claude