Add default S3 auth provider to the CLP connector to support IRSA and AWS EC2 instance credentials

[junhaoliao]

Description

The CLP connector's S3 authentication currently only supports the CLP_PACKAGE auth provider
(ClpPackageS3AuthProvider), which requires explicit static credentials (clp.s3-access-key-id,
clp.s3-secret-access-key). This prevents use of the AWS SDK's default credential provider chain,
which supports:

• IRSA (IAM Roles for Service Accounts) on EKS via AWS_WEB_IDENTITY_TOKEN_FILE / AWS_ROLE_ARN
• EC2 instance profiles / instance metadata
• Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• AWS config/credential files

Add a new ClpS3AuthProviderBase implementation so that the CLP connector can resolve S3
credentials through the AWS SDK's DefaultAWSCredentialsProviderChain, analogous to how Velox's
Hive connector handles hive.s3.use-instance-credentials=true.

Related reading: https://prestodb.io/docs/current/connector/hive.html#s3-credentials

Current behavior

In ClpConfig.cpp, the auth provider is resolved via stringToS3AuthProvider(), which only
recognizes CLP_PACKAGE:

ClpConfig::S3AuthProvider stringToS3AuthProvider(const std::string& strValue) {
  auto upperValue = boost::algorithm::to_upper_copy(strValue);
  VELOX_CHECK(!upperValue.empty());
  if (upperValue == "CLP_PACKAGE") {
    return ClpConfig::S3AuthProvider::kClpPackage;
  }
  VELOX_UNSUPPORTED("Unsupported s3 auth provider type: {}.", strValue);
}

ClpPackageS3AuthProvider::exportAuthEnvironmentVariables() has hard checks that accessKeyId
and secretAccessKey are non-empty, so there is no fallback to other credential sources.

Use case

When deploying CLP on EKS with S3-backed archives, users configure IRSA so that pods can access S3
without managing static AWS credentials. CLP's own compression/query workers already support this
via a "default" authentication type that uses the AWS SDK's default credential chain. However,
Presto workers using the CLP connector cannot use IRSA because the connector requires explicit
credentials, forcing users to either:

• Manage and rotate static AWS credentials for the Presto workers separately, or
• Forgo Presto-based querying when using IRSA

Proposed implementation

1. Add a new enum value kDefault to ClpConfig::S3AuthProvider and recognize "DEFAULT" in
   stringToS3AuthProvider().

2. Implement a new class (e.g., ClpDefaultS3AuthProvider) that extends ClpS3AuthProviderBase:

   • In exportAuthEnvironmentVariables(): use the AWS C++ SDK's
     Aws::Auth::DefaultAWSCredentialsProviderChain to resolve credentials, then export
     AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optionally AWS_SESSION_TOKEN as
     environment variables for CLP-S.
   • In constructS3Url(): construct the URL using the configured endpoint/region and bucket, same
     as ClpPackageS3AuthProvider.

3. Wire it up in ClpConfig::ClpConfig():

   case S3AuthProvider::kDefault:
     s3AuthProvider_ = std::make_shared<ClpDefaultS3AuthProvider>(config_);
     break;

4. Configuration: when using the default provider, only clp.s3-end-point (or region) and
   clp.s3-bucket would be required. clp.s3-access-key-id and clp.s3-secret-access-key would
   be optional/unused.

Note on credential expiry

IRSA temporary credentials expire (typically after 1 hour). The current architecture calls
exportAuthEnvironmentVariables() once in the ClpConfig constructor. For the DEFAULT provider,
the call site may need to be moved to before each split/query batch so that credentials are
re-resolved from the chain. The AWS SDK's DefaultAWSCredentialsProviderChain internally handles
token refresh, so repeated calls are cheap.

A more robust long-term solution would be to update CLP-S's NetworkReader / S3PresignedUrlV4
(in the CLP repo) to use the AWS SDK directly for signing instead of relying on exported env vars.
That would eliminate the credential-expiry concern entirely but is a separate, larger change.