Skip to content

Support fractional GPUs in the AWS Accelerators module - #265

Merged
jnioche merged 1 commit into
mainfrom
claude/aws-accelerators-fractional-gpu-784da1
Aug 24, 2026
Merged

Support fractional GPUs in the AWS Accelerators module#265
jnioche merged 1 commit into
mainfrom
claude/aws-accelerators-fractional-gpu-784da1

Conversation

@jnioche

@jnioche jnioche commented Aug 23, 2026

Copy link
Copy Markdown
Member

What

Lets the CCF Accelerators module handle instance types that get a fraction of a physical GPU, and adds a script to check the bundled GPU data against what EC2 actually offers.

Code — every numeric value read from the configuration now goes through Number instead of a cast to Integer, which accepts both the Integer and the Double Jackson produces for 1 and 0.125:

  • quantity is a double, so a fractional share scales the wattage through the existing amount * energy_used * quantity / 1000;
  • min/max watts are double too. This fixes a latent ClassCastException: GPU_INFO already holds "max": 76.5 for NVIDIA_TESLA_P4 and would have thrown the moment an instance type referenced it;
  • gpu_utilisation_percent is a double read as a Number, matching the pattern in OperationalEmissions, so "gpu_utilisation_percent": 12.5 is now valid config.

Data — added the five fractional sizes AWS offers, all NVIDIA L4:

instance type quantity
g6f.large, g6f.xlarge 0.125
g6f.2xlarge 0.25
g6f.4xlarge, gr6f.4xlarge 0.5

Taken from the AWS G6f spec table and confirmed against the GpuPartitionSize returned by ec2 describe-instance-types. The URL is recorded in the file's sources.

Scriptscripts/compare_gpu_accelerators_aws.py pages through aws ec2 describe-instance-types, keeps everything with a GpuInfo, and reports instance types missing from accelerators.json (with whether their GPU model already has wattage data), ones the queried regions no longer offer, GPU count mismatches (float-tolerant), GPU model mismatches, and undefined or unused GPU_INFO entries. --region is repeatable since instance types are region-specific; --strict exits 1 on real gaps. Stdlib only, same shape as the existing compare_gpu_accelerators_boavizta.py.

Notes for the reviewer

Scaling a full GPU's min-max range linearly by the partition size means a g6f.large is modelled as drawing an eighth of an idle L4 (1 W). Hardware-partitioned GPUs don't idle down proportionally — the card's floor is shared across partitions — so fractional sizes are under-estimated at low utilisation. Improving that needs a per-partition floor in GPU_INFO, which is a modelling decision beyond this PR.

Running the new script against us-east-1 surfaces gaps that are not addressed here, since each needs wattage figures sourced first:

  • the whole g7 family (NVIDIA RTX PRO 4500), g7e (RTX PRO Server 6000) and p6-b300.48xlarge (B300) are missing, and none of those three models has a GPU_INFO entry;
  • g5g.* is mapped to NVIDIA_T4 but AWS reports T4g — inherited from CCF, and whether the power profile differs enough to deserve its own entry is a judgement call;
  • NVIDIA_TESLA_P100 and NVIDIA_TESLA_P4 are defined but referenced by no instance type.

Full test suite passes (322 tests). AcceleratorsTest covers the fractional sizes in both CUR and FOCUS form, plus a decimal gpu_utilisation_percent.

Instance types can expose a share of a physical GPU rather than a whole
one: g6f.large gets an eighth of an NVIDIA L4. The module read the GPU
quantity as an Integer, so such an entry in accelerators.json would fail
with a ClassCastException.

Every numeric value read from the configuration now goes through Number,
which accepts both the Integer and the Double that Jackson produces. That
also fixes a latent ClassCastException on the min/max wattage: GPU_INFO
already holds "max": 76.5 for NVIDIA_TESLA_P4, which would have blown up
as soon as an instance type referenced it. gpu_utilisation_percent
becomes a double too, so a decimal rate can be configured.

Adds the five fractional sizes AWS offers (g6f.large, g6f.xlarge,
g6f.2xlarge, g6f.4xlarge, gr6f.4xlarge) with their L4 shares, confirmed
against the GpuPartitionSize reported by ec2 describe-instance-types.

Also adds scripts/compare_gpu_accelerators_aws.py, which checks
accelerators.json against the instance types EC2 actually offers so that
new GPU generations do not silently go without an estimate: it reports
missing instance types, ones no longer offered, GPU count and model
disagreements, and unused or undefined GPU_INFO entries.
@jnioche jnioche added this to the 1.3 milestone Aug 23, 2026
@jnioche jnioche added enhancement New feature or request AWS issues specific to AWS labels Aug 23, 2026
@jnioche
jnioche merged commit 6240fd9 into main Aug 24, 2026
3 checks passed
@jnioche
jnioche deleted the claude/aws-accelerators-fractional-gpu-784da1 branch August 24, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AWS issues specific to AWS enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant