fix: bound every outbound request with a client-side deadline

.changeset/per-request-timeouts.md

@@ -0,0 +1,15 @@
+---
+'@runpod/mcp-server': minor
+---
+
+Give every outbound request a client-side deadline — every tool call, the OAuth handshake on the hosted server, and the install wizard's key check. node-fetch applies no timeout of its own, so a Runpod host that accepted the connection and then went silent — a wedged worker, a load balancer holding the socket — left a tool call pending forever, and on the hosted server that ended as a bare 504 when Vercel reaped the function at its 60s limit.
+
+Requests now abort after 30 seconds with a named `RequestTimeoutError` naming the API that went quiet, the deadline it was given, and what to do next. `runsync-endpoint` is the one call that legitimately asks the server to hold a connection open, so it derives its deadline from the `wait` it requested (the server's own 90-second default when `wait` is omitted) rather than being truncated. Successful tool output is unchanged.
+
+On the hosted transport the cap is a single budget for the whole invocation rather than a fresh allowance per request, because several tools make more than one call — `get-job-status` adds a queued-job diagnosis, `deploy-hub-repo` and `set-endpoint-gpus` read before they write, `update-endpoint` reads the current scaler before patching it — and two full deadlines back to back outlived the platform even with each one bounded. Each request is clamped to what is left, so a stall anywhere in a handler surfaces as the named error instead of a 504. The queued-job diagnosis, which only decorates a status that is already in hand, is bounded at 5 seconds so it cannot spend a budget the reply itself needs.
+
+`stream-job` bounds each poll by the wait the server was asked to hold rather than by its whole budget. A deadline set to the budget meant one wedged socket consumed the entire run in a single attempt — 45 seconds on the hosted server, five minutes on stdio — so the loop's retry path never ran and a stall returned nothing at all. Each poll now gets the hold plus a round trip, which both clears a reply already in flight and leaves the budget room to reconnect; a run that ends on the budget reports `pollingTimedOut` with the last error rather than discarding it.
+
+The hosted OAuth routes are bounded too. `/token` polls the flash backend for an approval, and one silent socket there hung the whole handshake until the platform reaped it — the worst place for a blank error, since the user has no credential yet to retry with. Each backend call now has its own 10-second deadline (override with `MCP_FLASH_TIMEOUT_MS`), never exceeding what is left of a 45-second poll budget, and a stall is reported as a named error naming the operation, the host and the deadline. The `runpod-mcp install` wizard's key verification gets the same treatment, so it can no longer sit at "Verifying…" indefinitely.
+
+A timed-out GraphQL read is also no longer described as a possible write. The advice keys off the HTTP method and GraphQL is always POST on the wire, so `list-gpu-types` timing out used to tell the agent the call may have landed and to "check with the matching list-/get- tool first" — which is the tool that just failed. Only actual mutations carry that warning now.

CLAUDE.md

@@ -43,6 +43,7 @@ The hosted HTTP path (`api/index.ts` + `src/http.ts`) reads these, all optional
 - `RUNPOD_AUTHED_GRAPHQL_URL`: override the GraphQL host for **authenticated** operations with no REST equivalent — `deploy-hub-repo` and `set-endpoint-gpus` (default `https://api.runpod.io/graphql`). These send the caller's API key as a Bearer token, so only point this at a host you trust with it; on the hosted server that key is a per-user OAuth-minted one.
 - `RUNPOD_API_KEY_NAME`: name for the minted key (default `runpod-mcp`; set to `""` to omit for a backend without the `apiKeyName` argument).
 - `MCP_VERBOSE_LOGS`: set to `true` to log OAuth request ids (live auth codes) for debugging.
+- `MCP_FLASH_TIMEOUT_MS`: deadline in milliseconds for one call to the flash auth backend during the OAuth flow (default `10000`). Raise it for a slow backend; a value that is not a positive number is ignored, so the deadline cannot be disabled by a typo. The `/token` poll additionally caps itself at 45 seconds total, below the function's `maxDuration`.
 - `MCP_SKIP_CREDENTIAL_CHECK`: set to the exact string `true` to disable the hosted pre-flight credential verification (dead bearers then surface as tool-level 401 errors instead of an HTTP 401 re-auth signal). Use this if the pre-flight itself is ever causing outages. Note the pre-flight ALSO self-disables when a REST/Serverless host is overridden without a matching `RUNPOD_GRAPHQL_URL`, since it would otherwise validate the key against the wrong environment and reject every request.
 
 The build produces `dist/stdio.*`, `dist/http.*`, and `dist/tools.*`. Because `package.json` has `"type": "module"`, always use `dist/stdio.mjs` when running the built local server with `node`.

api/index.ts

@@ -94,6 +94,61 @@ function sleep(ms: number): Promise<void> {
   return new Promise((resolve) => setTimeout(resolve, ms));
 }
 
+// node-fetch applies no timeout of its own, and this file runs under
+// vercel.json's 60s maxDuration for api/index.ts. A flash backend that accepts
+// the connection and goes quiet would otherwise hang the OAuth handshake until
+// the platform reaps it — a blank 504 on the one flow the user cannot retry
+// their way out of, since they have no credential yet. Tool requests get the
+// same treatment through createHttpClient (src/_shared/http.ts); these calls
+// build their own request, so they are bounded here.
+// Exported so the parsing test asserts the real fallback, not a copy of it.
+export const DEFAULT_FLASH_GRAPHQL_TIMEOUT_MS = 10_000;
+// Whole-poll budget for /token, below maxDuration so the OAuth error response
+// is serialized rather than reaped. 20 attempts spaced 2s apart can otherwise
+// reach the platform limit on latency alone, with or without a wedged socket.
+// Exported for the test that checks it against vercel.json — the same
+// cross-file invariant HTTP_LONG_POLL_BUDGET_MS is pinned by, and the same
+// silent failure if maxDuration moves and this does not.
+export const TOKEN_POLL_BUDGET_MS = 45_000;
+const TOKEN_POLL_INTERVAL_MS = 2_000;
+// Budget below which /token stops polling instead of squeezing in one more
+// read. Reading an APPROVED request consumes the code atomically upstream, so a
+// read we abandon mid-flight can burn it — the user's retry then gets "already
+// used" rather than a key. Stopping answers authorization_pending, which is
+// retryable and consumes nothing, so it is the better trade for the last few
+// seconds of a budget.
+const MIN_TOKEN_POLL_REMAINDER_MS = 5_000;
+// AbortSignal.timeout takes a uint32: a fractional or out-of-range delay throws
+// ERR_OUT_OF_RANGE synchronously, which would 500 BOTH OAuth routes and make
+// signing in impossible. Just above int32 it does not throw at all — it warns
+// and fires immediately, turning a dial someone raised into a 1ms deadline.
+const MAX_FLASH_TIMEOUT_MS = 2_147_483_647;
+
+/**
+ * Deadline for one flash-backend call. `MCP_FLASH_TIMEOUT_MS` overrides it —
+ * an ops dial for a slow backend, and how the timeout tests reach this path in
+ * milliseconds instead of ten real seconds. Anything that is not a positive
+ * integer inside the timer's range is ignored, so no value of this variable can
+ * disable the deadline or break the routes that depend on it. Exported so the
+ * test for that promise calls the real parser.
+ */
+export function getFlashTimeoutMs(): number {
+  const override = Number(process.env.MCP_FLASH_TIMEOUT_MS);
+  return Number.isSafeInteger(override) &&
+    override > 0 &&
+    override <= MAX_FLASH_TIMEOUT_MS
+    ? override
+    : DEFAULT_FLASH_GRAPHQL_TIMEOUT_MS;
+}
+
+// Deadline for one /token poll: the per-call deadline, never past what is left
+// of the whole poll. The sibling of streamPollTimeoutMs in src/tools/jobs.ts.
+// No floor is needed — the loop below will not start a read at all once the
+// budget is down to MIN_TOKEN_POLL_REMAINDER_MS.
+function tokenPollDeadlineMs(remainingMs: number): number {
+  return Math.min(getFlashTimeoutMs(), remainingMs);
+}
+
 /**
  * Name for the minted Runpod API key, shown in the user's dashboard. Defaults
  * to "runpod-mcp" so keys minted through this server are identifiable and
@@ -120,15 +175,34 @@ interface FlashAuthRequestStatus {
  * non-JSON responses (e.g. an SST live-debug notice when the dev session is
  * down) as a clear error instead of a cryptic JSON parse failure.
  */
-async function flashGraphql<T>(query: string, field: string): Promise<T> {
+async function flashGraphql<T>(
+  query: string,
+  field: string,
+  requestedTimeoutMs?: number
+): Promise<T> {
   const url = getRunpodGraphqlUrl();
-  const response = await fetch(url, {
-    method: 'POST',
-    headers: { 'Content-Type': 'application/json' },
-    body: JSON.stringify({ query }),
-  });
+  const timeoutMs = requestedTimeoutMs ?? getFlashTimeoutMs();
+  // Covers the body drain too: a backend can send headers and then stall.
+  const signal = AbortSignal.timeout(timeoutMs);
+  let response: Awaited<ReturnType<typeof fetch>>;
+  let text: string;
+  try {
+    response = await fetch(url, {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ query }),
+      signal,
+    });
+    text = await response.text();
+  } catch (error) {
+    // `signal.aborted`, not the error name — node-fetch and undici label an
+    // abort differently, and a transport failure must keep its own message.
+    if (!signal.aborted) throw error;
+    throw new Error(
+      `${field} got no response from ${url} after ${timeoutMs}ms — the Runpod API may be unavailable. Start the sign-in again.`
+    );
+  }
 
-  const text = await response.text();
   let result: { data?: Record<string, T>; errors?: Array<{ message: string }> };
   try {
     result = JSON.parse(text);
@@ -175,12 +249,16 @@ async function createFlashAuthRequest(codeChallenge: string): Promise<string> {
  * Read the current status of a flash auth request (guest query). Once the user
  * approves it in the console, the backend mints and returns a Runpod API key.
  */
-async function getFlashAuthStatus(id: string): Promise<FlashAuthRequestStatus> {
+async function getFlashAuthStatus(
+  id: string,
+  timeoutMs?: number
+): Promise<FlashAuthRequestStatus> {
   return flashGraphql<FlashAuthRequestStatus>(
     `query { flashAuthRequestStatus(flashAuthRequestId: ${JSON.stringify(
       id
     )}) { id status apiKey codeChallenge codeChallengeMethod } }`,
-    'flashAuthRequestStatus'
+    'flashAuthRequestStatus',
+    timeoutMs
   );
 }
 
@@ -447,8 +525,27 @@ async function handleToken(
     // once (backend: model/src/flash/authRequests.ts), so this poll cannot hand
     // the same key out twice.
     const maxAttempts = 20;
-    for (let attempt = 0; attempt < maxAttempts; attempt++) {
-      const status = await getFlashAuthStatus(code);
+    const startedAt = Date.now();
+    const remainingBudgetMs = () =>
+      TOKEN_POLL_BUDGET_MS - (Date.now() - startedAt);
+    // Stops on whichever comes first: the attempt count, or a budget too thin
+    // to cover a read we would rather not start (MIN_TOKEN_POLL_REMAINDER_MS).
+    // Both fall through to the authorization_pending answer below.
+    for (
+      let attempt = 0;
+      attempt < maxAttempts &&
+      remainingBudgetMs() >= MIN_TOKEN_POLL_REMAINDER_MS;
+      attempt++
+    ) {
+      // Deliberately NOT wrapped in try/continue: an APPROVED read consumes
+      // the code atomically upstream, so a read that failed on our side may
+      // already have minted and burned the key. Retrying would then read
+      // CONSUMED-with-no-key and report "already used" — a worse answer than
+      // the honest failure. One failed read ends the poll.
+      const status = await getFlashAuthStatus(
+        code,
+        tokenPollDeadlineMs(remainingBudgetMs())
+      );
       console.log('oauth_token_poll', {
         attempt,
         status: status.status,
@@ -503,8 +600,10 @@ async function handleToken(
         return;
       }
 
-      // PENDING — wait and retry.
-      if (attempt < maxAttempts - 1) await sleep(2000);
+      // PENDING — wait and retry. Always the full interval: skipping the sleep
+      // near the end of the budget bought the user no extra approval time and
+      // just re-read the backend back to back.
+      if (attempt < maxAttempts - 1) await sleep(TOKEN_POLL_INTERVAL_MS);
     }
 
     tokenError(