Add get-capacity: GPU capacity matrix across host CUDA versions

.changeset/get-capacity-tool.md

@@ -0,0 +1,5 @@
+---
+'@runpod/mcp-server': minor
+---
+
+Add `get-capacity`: GPU capacity across host CUDA versions as one matrix call. Agents picking an endpoint's `allowedCudaVersions`/`minCudaVersion` (or diagnosing a capacity-starved endpoint) previously had to reverse-engineer per-version stock by calling the GraphQL `gpuTypes.lowestPrice` query once per CUDA version — an undocumented idiom nobody discovers organically. The default mode returns, per GPU type, overall stock plus AVAILABLE/UNAVAILABLE per host-reported CUDA version in a single credential-free catalog query; passing `cudaVersions` deep-probes those versions instead, returning graded stock (High/Medium/Low/Out) and the lowest on-demand price per version. Nothing is hidden by default — out-of-stock versions appear as explicit `Out` cells (set `includeUnavailable: false` to drop no-stock GPUs), and a transiently failing probe reports a per-version `probeErrors` entry instead of failing the whole call. Works on both v1 and v2 APIs — both use the public GraphQL catalog, since the v2 REST catalog's CUDA support is a `minCudaVersion` floor filter on availability, not the exact per-version breakdown, graded stock, or per-version pricing this tool returns.

CLAUDE.md

@@ -22,7 +22,7 @@ Prefer using paragraphs to bullet points unless directly asked. When using bulle
 
 ## Architecture
 
-The server communicates with two separate Runpod API backends. The REST API at `https://rest.runpod.io/v1` handles all authenticated CRUD operations for Pods, endpoints, templates, network volumes, and container registry auths. It requires a `RUNPOD_API_KEY` environment variable. The GraphQL API at `https://api.runpod.io/graphql` is reached two ways. The public, unauthenticated path serves read-only discovery queries — GPU types, data centers, the Hub catalog, Public Endpoints. The authenticated path (`graphqlAuthed`, API key as a Bearer token) serves the handful of write operations that have no REST equivalent: `deploy-hub-repo` and `set-endpoint-gpus`, both of which call the `saveEndpoint` mutation. The two resolve their host from separate env vars on purpose — see `RUNPOD_PUBLIC_GRAPHQL_URL` and `RUNPOD_AUTHED_GRAPHQL_URL` below.
+The server communicates with two separate Runpod API backends. The REST API at `https://rest.runpod.io/v1` handles all authenticated CRUD operations for Pods, endpoints, templates, network volumes, and container registry auths. It requires a `RUNPOD_API_KEY` environment variable. The GraphQL API at `https://api.runpod.io/graphql` is reached two ways. The public, unauthenticated path serves read-only discovery queries — GPU types, GPU capacity by CUDA version, data centers, the Hub catalog, Public Endpoints. The authenticated path (`graphqlAuthed`, API key as a Bearer token) serves the handful of write operations that have no REST equivalent: `deploy-hub-repo` and `set-endpoint-gpus`, both of which call the `saveEndpoint` mutation. The two resolve their host from separate env vars on purpose — see `RUNPOD_PUBLIC_GRAPHQL_URL` and `RUNPOD_AUTHED_GRAPHQL_URL` below.
 
 The source is split by responsibility:
 
@@ -39,7 +39,7 @@ The hosted HTTP path (`api/index.ts` + `src/http.ts`) reads these, all optional
 - `RUNPOD_GRAPHQL_URL`: flash auth backend for the OAuth flow (default `https://api.runpod.io/graphql`). Also the host the hosted credential pre-flight verifies against, so unlike the guest flash-auth mutations it now receives the caller's bearer token — point it only at a host you trust with that.
 - `CONSOLE_BASE_URL`: console that hosts the sign-in handoff page (default `https://console.runpod.io`).
 - `RUNPOD_REST_API_URL` / `RUNPOD_SERVERLESS_API_URL`: override the REST and Serverless API hosts (e.g. for a dev API key).
-- `RUNPOD_PUBLIC_GRAPHQL_URL`: override the public discovery GraphQL host used by `list-gpu-types`/`list-data-centers` (default `https://api.runpod.io/graphql`). Never carries a credential — safe to point at a stub.
+- `RUNPOD_PUBLIC_GRAPHQL_URL`: override the public discovery GraphQL host used by `list-gpu-types`, `list-data-centers`, `get-capacity`, `list-hub-repos`, and `list-public-endpoints` (default `https://api.runpod.io/graphql`). Never carries a credential — safe to point at a stub, though note `get-capacity` has no REST fallback on either API version.
 - `RUNPOD_AUTHED_GRAPHQL_URL`: override the GraphQL host for **authenticated** operations with no REST equivalent — `deploy-hub-repo` and `set-endpoint-gpus` (default `https://api.runpod.io/graphql`). These send the caller's API key as a Bearer token, so only point this at a host you trust with it; on the hosted server that key is a per-user OAuth-minted one.
 - `RUNPOD_API_KEY_NAME`: name for the minted key (default `runpod-mcp`; set to `""` to omit for a backend without the `apiKeyName` argument).
 - `MCP_VERBOSE_LOGS`: set to `true` to log OAuth request ids (live auth codes) for debugging.

docs/context.md

@@ -34,7 +34,7 @@ All optional, with production-safe defaults:
 - `RUNPOD_GRAPHQL_URL`: flash auth backend for the OAuth flow (default `https://api.runpod.io/graphql`). Also the host the hosted credential pre-flight verifies against, so unlike the guest flash-auth mutations it now receives the caller's bearer token — point it only at a host you trust with that.
 - `CONSOLE_BASE_URL`: console hosting the sign-in handoff page (default `https://console.runpod.io`).
 - `RUNPOD_REST_API_URL` / `RUNPOD_SERVERLESS_API_URL`: override the REST and Serverless API hosts.
-- `RUNPOD_PUBLIC_GRAPHQL_URL`: override the public discovery GraphQL host used by `list-gpu-types`/`list-data-centers`. Never carries a credential.
+- `RUNPOD_PUBLIC_GRAPHQL_URL`: override the public discovery GraphQL host used by `list-gpu-types`, `list-data-centers`, `get-capacity`, `list-hub-repos`, and `list-public-endpoints`. Never carries a credential.
 - `RUNPOD_AUTHED_GRAPHQL_URL`: override the GraphQL host for authenticated, no-REST-equivalent operations (`deploy-hub-repo`, `set-endpoint-gpus`). Sends the caller's API key — point it only at a trusted host.
 - `RUNPOD_API_KEY_NAME`: name for the minted key (default `runpod-mcp`; `""` to omit).
 - `MCP_VERBOSE_LOGS`: `true` to log OAuth request ids (live auth codes) for debugging.

pnpm-workspace.yaml

@@ -0,0 +1,2 @@
+allowBuilds:
+  esbuild: true

src/tools/catalog.ts

@@ -348,6 +348,278 @@ export function registerCatalogTools(server: McpServer, rt: ToolRuntime): void {
     }
   );
 
+  // Get capacity (GPU × host-CUDA availability). Public GraphQL on both API
+  // versions — the v2 REST catalog's only CUDA dimension is a minCudaVersion
+  // floor filter on availability (see MinCudaVersionFilter in the vendored
+  // spec); the exact per-version breakdown, per-version graded stock, and
+  // per-version pricing this tool returns are GraphQL-only, so unlike the
+  // other catalog tools this one never branches on backendFor.
+  server.tool(
+    'get-capacity',
+    "GPU capacity across host CUDA versions, as a matrix. Use this to choose an endpoint's allowedCudaVersions/minCudaVersion (or diagnose/widen a capacity-starved one) and to distinguish capacity problems from compatibility problems. Default mode is one call returning, per GPU type, overall stock plus AVAILABLE/UNAVAILABLE per host-reported CUDA version. Pass cudaVersions to deep-probe instead: one stock lookup per listed version, returning graded stock (High/Medium/Low/Out) and the lowest on-demand price per version; a probe that fails transiently reports a per-version error instead of failing the call. Nothing is hidden by default — set includeUnavailable:false to drop GPUs with no stock on any listed version. Credential-free public catalog data; works on both v1 and v2 APIs. Stock is live, so page cursors can shift between calls. Note: the matrix reflects host-reported versions, but endpoint allowedCudaVersions only accepts values from the platform enum — check create-endpoint/update-endpoint for the accepted list.",
+    {
+      ...listPaginationParams,
+      cudaVersions: z
+        .array(z.string().regex(/^\d{1,2}\.\d{1,2}$/))
+        .min(1)
+        .max(12)
+        .optional()
+        .describe(
+          'Deep-probe these host CUDA versions (e.g. ["12.8", "13.0"], max 12): one stock lookup per version returns graded stock and price. Omit for the single-call AVAILABLE/UNAVAILABLE matrix across all versions the fleet currently reports.'
+        ),
+      includeUnavailable: z
+        .boolean()
+        .optional()
+        .describe(
+          'Out-of-stock GPUs are included by default (explicit "Out" cells / all-UNAVAILABLE rows, sorted last). Set false to hide GPUs with no stock on any listed version.'
+        ),
+      gpuTypeIds: z
+        .array(z.string())
+        .optional()
+        .describe(
+          "Filter to GPU types matching any of these ids or display names (case-insensitive substring, e.g. ['NVIDIA GeForce RTX 4090'] or ['4090', 'H200'])."
+        ),
+      gpuCount: z
+        .number()
+        .int()
+        .min(1)
+        .max(8)
+        .optional()
+        .describe('GPUs per worker/pod to check stock for (default 1).'),
+      secureCloudOnly: z
+        .boolean()
+        .optional()
+        .describe('Restrict the stock lookup to Secure Cloud hosts.'),
+    },
+    { title: 'Get GPU capacity by CUDA version', ...READ_ONLY },
+    async (params) => {
+      // The public GraphQL path takes no variables, so arguments are inlined.
+      // Every inlined value is re-validated here at runtime (not just in zod)
+      // because direct handler calls can bypass schema validation: gpuCount is
+      // coerced to an int in [1,8], cudaVersions is regex-checked below, and
+      // secureCloud is a literal.
+      const gpuCount = Math.min(
+        8,
+        Math.max(1, Math.floor(Number(params.gpuCount)) || 1)
+      );
+      const secureArg = params.secureCloudOnly ? ', secureCloud: true' : '';
+
+      interface CapacityGpu {
+        id: string;
+        displayName: string;
+        memoryInGb: number;
+        secureCloud: boolean;
+        communityCloud: boolean;
+        lowestPrice?: {
+          stockStatus: string | null;
+          uninterruptablePrice: number | null;
+          gpuTypeCudaVersions?: Array<{
+            cudaVersion: string;
+            availability: string;
+          }> | null;
+        } | null;
+      }
+      interface CapacityResponse {
+        gpuTypes: CapacityGpu[];
+      }
+
+      // Blank/non-string filter entries are ignored (an all-blank list means
+      // "no filter", same as omitting it) — a zod-bypassed [""] must not
+      // silently match the whole catalog as if it were a real term.
+      const filterTerms = (params.gpuTypeIds ?? [])
+        .filter((t) => typeof t === 'string' && t.trim().length > 0)
+        .map((t) => t.toLowerCase());
+      const matchesFilter = (gpu: CapacityGpu) => {
+        if (filterTerms.length === 0) return true;
+        const id = gpu.id.toLowerCase();
+        const name = gpu.displayName.toLowerCase();
+        return filterTerms.some(
+          (term) => id.includes(term) || name.includes(term)
+        );
+      };
+
+      const stockPriority: Record<string, number> = {
+        High: 3,
+        Medium: 2,
+        Low: 1,
+      };
+
+      // Matrix mode: one query, per-version availability from the fleet.
+      if (!params.cudaVersions?.length) {
+        const data = await graphql<CapacityResponse>(`
+          query {
+            gpuTypes {
+              id
+              displayName
+              memoryInGb
+              secureCloud
+              communityCloud
+              lowestPrice(input: { gpuCount: ${gpuCount}${secureArg} }) {
+                stockStatus
+                uninterruptablePrice
+                gpuTypeCudaVersions {
+                  cudaVersion
+                  availability
+                }
+              }
+            }
+          }
+        `);
+        let rows = data.gpuTypes
+          .filter((gpu) => gpu.id !== 'unknown' && matchesFilter(gpu))
+          .map((gpu) => {
+            const cuda: Record<string, string> = {};
+            for (const c of gpu.lowestPrice?.gpuTypeCudaVersions ?? []) {
+              cuda[c.cudaVersion] = c.availability;
+            }
+            return {
+              id: gpu.id,
+              displayName: gpu.displayName,
+              memoryGb: gpu.memoryInGb,
+              secureCloud: gpu.secureCloud,
+              communityCloud: gpu.communityCloud,
+              stockStatus: gpu.lowestPrice?.stockStatus || 'unavailable',
+              pricePerHr: gpu.lowestPrice?.uninterruptablePrice ?? null,
+              cudaVersions: cuda,
+            };
+          });
+        const availableCount = (r: (typeof rows)[number]) =>
+          Object.values(r.cudaVersions).filter((a) => a === 'AVAILABLE').length;
+        if (params.includeUnavailable === false)
+          rows = rows.filter((r) => availableCount(r) > 0);
+        rows.sort((a, b) => {
+          const diff = availableCount(b) - availableCount(a);
+          if (diff !== 0) return diff;
+          const stockDiff =
+            (stockPriority[b.stockStatus] || 0) -
+            (stockPriority[a.stockStatus] || 0);
+          if (stockDiff !== 0) return stockDiff;
+          return b.memoryGb - a.memoryGb;
+        });
+        return capListResult(rows, {
+          limit: params.limit,
+          cursor: params.cursor,
+        });
+      }
+
+      // Probe mode: one stock lookup per requested version, merged per GPU.
+      // Zod caps at 12, but the cap is re-enforced here because direct handler
+      // calls (tests, other transports) can bypass schema validation.
+      const versions = [...new Set(params.cudaVersions)].slice(0, 12);
+      const invalid = versions.filter((v) => !/^\d{1,2}\.\d{1,2}$/.test(v));
+      if (invalid.length > 0) {
+        return jsonReply({
+          error: `Invalid CUDA version format: ${invalid.join(', ')}. Use "major.minor" strings like "12.8".`,
+          status: 400,
+        });
+      }
+      // allSettled, not all: a transient failure (429/5xx) on one probe must
+      // not discard the other versions' results — the same best-effort stance
+      // as get-job-status's worker fan-out. Failed versions are reported in a
+      // probeErrors sibling field instead of failing the call.
+      const perVersion = await Promise.allSettled(
+        versions.map((v) =>
+          graphql<CapacityResponse>(`
+            query {
+              gpuTypes {
+                id
+                displayName
+                memoryInGb
+                secureCloud
+                communityCloud
+                lowestPrice(input: { gpuCount: ${gpuCount}, allowedCudaVersions: ["${v}"]${secureArg} }) {
+                  stockStatus
+                  uninterruptablePrice
+                }
+              }
+            }
+          `)
+        )
+      );
+
+      const probeErrors: Record<string, string> = {};
+      const byId = new Map<
+        string,
+        {
+          id: string;
+          displayName: string;
+          memoryGb: number;
+          secureCloud: boolean;
+          communityCloud: boolean;
+          cudaVersions: Record<
+            string,
+            { stock: string; pricePerHr: number | null }
+          >;
+        }
+      >();
+      versions.forEach((v, i) => {
+        const settled = perVersion[i];
+        if (settled.status === 'rejected') {
+          probeErrors[v] =
+            settled.reason instanceof Error
+              ? settled.reason.message
+              : String(settled.reason);
+          return;
+        }
+        for (const gpu of settled.value.gpuTypes) {
+          if (gpu.id === 'unknown' || !matchesFilter(gpu)) continue;
+          let row = byId.get(gpu.id);
+          if (!row) {
+            row = {
+              id: gpu.id,
+              displayName: gpu.displayName,
+              memoryGb: gpu.memoryInGb,
+              secureCloud: gpu.secureCloud,
+              communityCloud: gpu.communityCloud,
+              cudaVersions: {},
+            };
+            byId.set(gpu.id, row);
+          }
+          const stock = gpu.lowestPrice?.stockStatus;
+          // No stockStatus means no hosts match this version at all — an
+          // explicit "Out" cell, so starvation is visible rather than an
+          // absence the agent has to infer (a capacity-diagnosis tool must
+          // not hide the empty cells it exists to reveal).
+          row.cudaVersions[v] =
+            !stock || stock === 'Out'
+              ? { stock: 'Out', pricePerHr: null }
+              : {
+                  stock,
+                  pricePerHr: gpu.lowestPrice?.uninterruptablePrice ?? null,
+                };
+        }
+      });
+
+      let rows = [...byId.values()];
+      const inStockCount = (r: (typeof rows)[number]) =>
+        Object.values(r.cudaVersions).filter((c) => c.stock !== 'Out').length;
+      if (params.includeUnavailable === false)
+        rows = rows.filter((r) => inStockCount(r) > 0);
+      rows.sort((a, b) => {
+        const diff = inStockCount(b) - inStockCount(a);
+        if (diff !== 0) return diff;
+        const best = (r: (typeof rows)[number]) =>
+          Math.max(
+            0,
+            ...Object.values(r.cudaVersions).map(
+              (c) => stockPriority[c.stock] || 0
+            )
+          );
+        const bestDiff = best(b) - best(a);
+        if (bestDiff !== 0) return bestDiff;
+        return b.memoryGb - a.memoryGb;
+      });
+      return capListResult(
+        rows,
+        { limit: params.limit, cursor: params.cursor },
+        {
+          probedCudaVersions: versions,
+          ...(Object.keys(probeErrors).length > 0 ? { probeErrors } : {}),
+        }
+      );
+    }
+  );
+
   // Get Data Center by id (v2-only — GET /v2/catalog/datacenters/{id})
   server.tool(
     'get-data-center',