Skip to content

feat(auth): add --experimental-auth-token-file for short-lived OAuth2 access tokens#4871

Open
asdiamond wants to merge 1 commit into
GoogleCloudPlatform:masterfrom
asdiamond:experimental-auth-token-file
Open

feat(auth): add --experimental-auth-token-file for short-lived OAuth2 access tokens#4871
asdiamond wants to merge 1 commit into
GoogleCloudPlatform:masterfrom
asdiamond:experimental-auth-token-file

Conversation

@asdiamond

Copy link
Copy Markdown

Fixes #4686

What

Adds --experimental-auth-token-file <path> (config: gcs-auth.experimental-auth-token-file), a flag that points at a file containing an OAuth2 token response in JSON format, used directly as the GCS credential:

echo '{"access_token":"ya29...","expires_in":3600,"token_type":"Bearer"}' > /tmp/gcs-token
gcsfuse --experimental-auth-token-file=/tmp/gcs-token --only-dir=user-prefix bucket /mnt/gcs

This targets short-lived sandbox environments (Firecracker microVM providers like e2b, Modal, Daytona) where an orchestrator mints a downscoped, prefix-scoped access token (e.g. via STS Credential Access Boundaries) and hands it to gcsfuse inside the sandbox — without shipping a long-lived SA key into an environment executing untrusted code, and without running a localhost HTTP server just to satisfy --token-url.

Design decisions (per maintainer guidance on #4686)

  • Flag name: --experimental-auth-token-file (@trinadhkotturu preferred auth-token-file, @anushka567 asked for the experimental- prefix for external contributions).
  • Fail at mount, not at GCS: the token file is read eagerly when the token source is constructed, so mounting fails immediately if the token is missing, malformed, empty, or expired (@trinadhkotturu's recommendation).
  • File shape: the OAuth2 token response JSON, consistent with what --token-url endpoints return (@anushka567's recommendation). The relative expires_in field is anchored to the file read time; an absolute expiry field (RFC 3339), when present, takes precedence.
  • Refresh: the source is wrapped in oauth2.ReuseTokenSource, so the file is re-read when the cached token expires. An external process may therefore refresh the credential by rewriting the file; if the file isn't refreshed, requests fail with an "invalid or expired token" error rather than hitting GCS with a stale credential.
  • Mutual exclusion: combining the new flag with --key-file or --token-url is rejected at config validation time, to fail early instead of silently picking one. Happy to relax this to a documented precedence order if you'd prefer.

Changes

  • cfg/params.yaml + regenerated cfg/config.go (via go generate): new param, type resolvedPath.
  • internal/auth/file_token_source.go: new fileTokenSource + NewTokenSourceFromTokenFile, mirroring the existing proxyTokenSource structure.
  • internal/storage/storageutil/auth_client_option.go: token-file branch in GetClientAuthOptionsAndToken (google-lib-auth flow).
  • internal/auth/auth.go / internal/storage/storageutil/client.go: token-file support in the legacy GetTokenSource flow; also replaced the deprecated golang.org/x/net/context import with context in client.go to satisfy the govet inline check on the touched line.
  • cfg/validate.go: isValidGcsAuthConfig mutual-exclusion check.
  • Tests: internal/auth/file_token_source_test.go (valid/expired/no-expiry/malformed/missing/empty-token/re-read cases), storageutil/auth_client_option_test.go, cfg/validate_test.go, and a happy-path + conflict cases in cmd/root_test.go, per the dev guide.

Testing

  • go test ./cmd/... ./cfg/... ./internal/auth/... ./internal/storage/storageutil/... passes on linux/amd64 (Docker, golang:1.26).
  • go test -run=PATTERN_THAT_DOES_NOT_MATCH_ANYTHING ./... (buildTest) compiles cleanly on linux/amd64.
  • golangci-lint run -E=unused --new-from-rev=master reports 0 issues on the touched packages.

🤖 Generated with Claude Code

@asdiamond
asdiamond requested a review from a team as a code owner July 12, 2026 05:59
@asdiamond
asdiamond requested a review from charith87 July 12, 2026 05:59
@google-cla

google-cla Bot commented Jul 12, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an experimental feature to authenticate with GCS using an OAuth2 token file via the experimental-auth-token-file flag. It includes the necessary configuration, validation to prevent concurrent usage with other authentication methods, and a custom oauth2.TokenSource implementation that reads and parses the token from a JSON file. However, a critical compilation error was identified in internal/auth/file_token_source.go because oauth2.Token does not contain an ExpiresIn field. A review comment suggests using an auxiliary struct to correctly unmarshal and map the JSON fields.

Comment thread internal/auth/file_token_source.go
@asdiamond

Copy link
Copy Markdown
Author

@googlebot I signed it!

Adds an experimental flag that points at a file containing an OAuth2
token response in JSON format ({"access_token": ..., "expires_in": ...,
"token_type": "Bearer"}), to be used directly as the GCS credential.

This supports short-lived sandbox environments where an orchestrator
mints a downscoped access token (e.g. via STS Credential Access
Boundaries) and hands it to gcsfuse, without shipping a long-lived
service account key into the sandbox or running a localhost HTTP
server just to satisfy --token-url.

Behavior:
- The token file is read eagerly, so mounting fails if the token is
  missing, malformed, or expired.
- The relative expires_in field is anchored to the read time; an
  absolute expiry field takes precedence when present.
- The file is re-read when the token expires, so an external process
  may refresh the credential by rewriting the file.
- The flag is rejected when combined with key-file or token-url.

Fixes GoogleCloudPlatform#4686
@asdiamond
asdiamond force-pushed the experimental-auth-token-file branch from 95c684f to e0618ec Compare July 12, 2026 06:41
@charith87
charith87 requested review from anushka567 and removed request for charith87 July 13, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: --auth-token-file for short-lived OAuth2 access tokens

1 participant