Support fractional GPUs in the AWS Accelerators module - #265
Merged
Conversation
Instance types can expose a share of a physical GPU rather than a whole one: g6f.large gets an eighth of an NVIDIA L4. The module read the GPU quantity as an Integer, so such an entry in accelerators.json would fail with a ClassCastException. Every numeric value read from the configuration now goes through Number, which accepts both the Integer and the Double that Jackson produces. That also fixes a latent ClassCastException on the min/max wattage: GPU_INFO already holds "max": 76.5 for NVIDIA_TESLA_P4, which would have blown up as soon as an instance type referenced it. gpu_utilisation_percent becomes a double too, so a decimal rate can be configured. Adds the five fractional sizes AWS offers (g6f.large, g6f.xlarge, g6f.2xlarge, g6f.4xlarge, gr6f.4xlarge) with their L4 shares, confirmed against the GpuPartitionSize reported by ec2 describe-instance-types. Also adds scripts/compare_gpu_accelerators_aws.py, which checks accelerators.json against the instance types EC2 actually offers so that new GPU generations do not silently go without an estimate: it reports missing instance types, ones no longer offered, GPU count and model disagreements, and unused or undefined GPU_INFO entries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Lets the CCF
Acceleratorsmodule handle instance types that get a fraction of a physical GPU, and adds a script to check the bundled GPU data against what EC2 actually offers.Code — every numeric value read from the configuration now goes through
Numberinstead of a cast toInteger, which accepts both theIntegerand theDoubleJackson produces for1and0.125:quantityis adouble, so a fractional share scales the wattage through the existingamount * energy_used * quantity / 1000;min/maxwatts aredoubletoo. This fixes a latentClassCastException:GPU_INFOalready holds"max": 76.5forNVIDIA_TESLA_P4and would have thrown the moment an instance type referenced it;gpu_utilisation_percentis adoubleread as aNumber, matching the pattern inOperationalEmissions, so"gpu_utilisation_percent": 12.5is now valid config.Data — added the five fractional sizes AWS offers, all NVIDIA L4:
g6f.large,g6f.xlargeg6f.2xlargeg6f.4xlarge,gr6f.4xlargeTaken from the AWS G6f spec table and confirmed against the
GpuPartitionSizereturned byec2 describe-instance-types. The URL is recorded in the file'ssources.Script —
scripts/compare_gpu_accelerators_aws.pypages throughaws ec2 describe-instance-types, keeps everything with aGpuInfo, and reports instance types missing fromaccelerators.json(with whether their GPU model already has wattage data), ones the queried regions no longer offer, GPU count mismatches (float-tolerant), GPU model mismatches, and undefined or unusedGPU_INFOentries.--regionis repeatable since instance types are region-specific;--strictexits 1 on real gaps. Stdlib only, same shape as the existingcompare_gpu_accelerators_boavizta.py.Notes for the reviewer
Scaling a full GPU's min-max range linearly by the partition size means a
g6f.largeis modelled as drawing an eighth of an idle L4 (1 W). Hardware-partitioned GPUs don't idle down proportionally — the card's floor is shared across partitions — so fractional sizes are under-estimated at low utilisation. Improving that needs a per-partition floor inGPU_INFO, which is a modelling decision beyond this PR.Running the new script against
us-east-1surfaces gaps that are not addressed here, since each needs wattage figures sourced first:g7family (NVIDIA RTX PRO 4500),g7e(RTX PRO Server 6000) andp6-b300.48xlarge(B300) are missing, and none of those three models has aGPU_INFOentry;g5g.*is mapped toNVIDIA_T4but AWS reportsT4g— inherited from CCF, and whether the power profile differs enough to deserve its own entry is a judgement call;NVIDIA_TESLA_P100andNVIDIA_TESLA_P4are defined but referenced by no instance type.Full test suite passes (322 tests).
AcceleratorsTestcovers the fractional sizes in both CUR and FOCUS form, plus a decimalgpu_utilisation_percent.