Add guided AWS self-hosting installer - #432
Closed
dhempler wants to merge 9 commits into
Closed
Conversation
dhempler
force-pushed
the
codex/aws-self-hosting-installer-pr
branch
from
July 24, 2026 15:28
61b5740 to
24699e5
Compare
dhempler
marked this pull request as ready for review
July 24, 2026 15:32
- Workflow: replace npm ci / npm cache with corepack+yarn (repo is Yarn Berry; npm ci failed on every run) - upload-backend-artifact: return the S3 object VersionId so redeploys pin LambdaCodeS3ObjectVersion and Lambda code updates are not silently skipped - publish-lambda-layer: accept --content-bucket/--content-key/ --content-object-version (the flags deploy-aws and deploy-backend already pass) as an S3 alternative to --source-file - backend-api.yaml: add DependsOn (DatabaseInstanceWriter, MigrationLogGroup) to MigrationRunner so migrations cannot fire before the DB instance or log group exist - validate-aws-deploy: fix undefined frontendStackName reference (ReferenceError on the split-stack publish-only path) - deploy-aws: treat --run-api-migrations=false / --run-bootstrap-admin=false as disabled (string "false" was truthy) - IAM samples: scope role management, PassRole, and Secrets Manager access to project-prefixed ARNs; split CreateServiceLinkedRole into its own correctly conditioned statement; scope the OIDC role's secret read, SSM writes, rds-data, and layer publish to project resources Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From a three-part usability audit (novice doc walkthrough, wizard UX review, prerequisite-burden inventory) aimed at letting a non-technical church admin complete the install: - Yarn/Corepack: doctor now checks for yarn and suggests `corepack enable`; start-here quick checks and local-runtime cover yarn with an explicit warning against `npm install -g yarn` (previously yarn was required by every command but never checked or explained) - Cost and time: start-here's "What Costs Money?" now leads with a dollar estimate (~$80-90/mo prod-only, itemized) plus a 2-4 hour time expectation and a billing-alarm suggestion - Solo-admin framing: "Who Does What" and aws-account-access now say plainly that in a small church all three roles are usually the same person and the operator can run the IAM commands themselves - infrastructure/README.md gets a banner routing novices to start-here.md (previously never linked) - Tool install links for Node, Git, GitHub CLI, and AWS CLI - first-rollout-checklist defines $DEPLOY_REPO/$DEPLOY_ENV_DIR before using them; deployment-repository clone example uses HTTPS instead of SSH; root README dev steps use yarn (npm install is rejected by preinstall) - installer:customer-values generates a temporary admin password on Enter (unambiguous alphabet, xxxx-xxxx-xxxx) instead of requiring the user to invent one - installer:doctor reads region/account-id/repo from customer-values (not just env vars) and its next steps point at the interview instead of advising env var exports Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second tier of the non-technical-installer work. The guided runner can now drive the install from scaffold to report without ejecting the user into copy-paste shell commands: - installer:run executes yarn commands (previously it only recognized npm/cp, so every generated next-command fell back to "run this manually"); failures now surface a plain-English hint (AWS sign-in, GitHub sign-in, missing tool, network) via a shared explainFailure helper instead of raw stderr, and crashes no longer print stack traces - installer:aws-roles --apply=true creates the GitHub OIDC provider and both IAM roles with the operator's own AWS sign-in (idempotent: refreshes trust policy if a role exists), writes apply-result.json evidence, and the state machine gains an "AWS IAM roles created" step gated behind the runner's y/N approval; the admin-handoff document remains the alternative for orgs with a separate AWS admin - new installer:commit syncs the private deployment repo: initializes git, creates the private GitHub repository if missing, stages ONLY the safe file allowlist, refuses to run if the .gitignore protecting secrets is absent, commits, and pushes; the state machine gains a "Private repository synced" step so manual git ceremony disappears - adopt-frontend-origin archives the stale deployment summary and deploy dispatch evidence, so the runner automatically walks the second prod pass (commit -> redeploy -> observe) instead of leaving login broken until the user rereads the docs - environment-setup-wizard announces itself as an advanced tool and points to installer:run; customer-values failures keep answers and say so instead of dumping a stack trace - docs updated to match: no manual git commands on the guided path, IAM self-service framing, automatic second-deploy explanation - smoke walkthrough scenario extended to cover the IAM apply step, both private-repo sync points, and the post-adopt redeploy loop Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third tier of the non-technical-installer work: - backend-api.yaml: the CreateNatGateway=false mode gains Polly and execute-api interface endpoints (the latter only when the WebSocket API is enabled), so text-to-speech and socket pushes work without NAT. The NAT default stays "true" deliberately: outbound email via the SES API and third-party integrations (payment gateways, Mautic, YouTube) have no VPC-endpoint equivalent and would silently break. The trade-off is now documented instead of hidden. - New infrastructure/environments/operations.md day-2 runbook: routine updates, version pinning (pin both repos to a release tag instead of dual `main`), Aurora backup/restore guidance, secret rotation with the applied-at-deploy-time caveat, billing alarms, final-snapshot cleanup after resets, NAT cost trade-off, and where logs live. Linked from start-here (TOC, update section, cost section) and the environments README. - installer:browser-smoke gains --mode=auto|browser|http. Auto falls back to plain HTTP checks (home page + login page reachable and app-shaped) when Playwright or its browser is not installed, so the smoke step no longer forces a Chromium download onto the operator's machine; browser mode reports how to fix the missing browser instead of a raw error. Evidence records which mode ran. Verified against a local HTTP server (success and closed-port failure paths). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes the round-1 security review findings: - ManagedAssetBucket is no longer world-readable. Public access is fully blocked and uploaded assets are served through a new CloudFront distribution with Origin Access Control (same pattern as the frontend site); CONTENT_ROOT / CONTENT_ROOT_URL env vars and the ContentRootUrl stack output now point at the distribution domain, and the bucket's CORS rule is scoped to the CorsOrigin parameter. New AssetDistributionId output. Externally supplied AssetBucketName deployments are unchanged. - API Gateway CORS now honors the CorsOrigin parameter instead of a hardcoded "*" (the parameter default remains "*", tightened automatically when the installer adopts the frontend origin). - sync-legacy-ssm-parameters no longer leaks secrets: parameter values go to AWS via a 0600 temp --cli-input-json file instead of argv, the echoed command hides the value, JSON/text output lists parameter names only, --output=json actually syncs instead of silently returning early, and connection-string credentials are URL-encoded. Sample and smoke contract updated to assert values never appear. - sync-app-config-secret passes the secret to the AWS CLI as --secret-string file://<path> instead of putting the full secret JSON on the process argv. - deploy-aws/deploy-backend/deploy-full-stack mask password and secret-string values in their echoed command lines. - operations.md gains a security-posture section, including the honest limitation that secrets reach Lambda as env vars resolved at deploy time (runtime fetching would require upstream Api changes) and what that implies for AWS account access. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reviewer-driven diet. The PR shipped three overlapping ways to deploy the same stack; this commits to the single golden path the guided installer uses (split-stack via GitHub Actions) and removes the parallel alternatives: - Remove the full-stack deployment mode: deploy-full-stack.mjs, full-stack.yaml, its 8 output/parameter samples, and the package.json entry. validate:aws-deploy now defaults to --mode=split-stack and exits with a clear message if --mode=full-stack is requested. - Remove the "direct" API migration runner (run-api-migrations.mjs and its sample): migrations run through the RDS Data API everywhere. deploy-backend/deploy-aws default to data-api and explain the removal if direct is requested; run:api-migrations now maps to the data-api script. - Remove tools superseded by the guided installer: environment-setup-wizard.mjs (replaced by installer:customer-values) and launch-staging.mjs (replaced by installer:run). - Docs: deployment-workbook.md removed; infrastructure/README.md rewritten from a 1,186-line manual-flow reference into a ~55-line architecture overview that defers to start-here.md and operations.md. - Smoke suite updated: ~32 scenarios and 19 helper functions for the removed tools deleted (916 lines), replaced by 4 scenarios asserting the new removal errors; script/template/sample check lists updated. Full suite passes. Net: -5,200 lines, -15 files versus the previous branch state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
scripts/smoke-aws-tooling.mjs (7,943 lines) and the 40 example output fixtures it checks against now live on the codex/aws-tooling-smoke-harness branch, to be opened as a follow-up PR once the installer PR lands. This keeps the reviewable surface of the main PR to the product itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From the adversarially verified code review of this branch: - backend-api.yaml declares LambdaCodeS3ObjectVersion and MigrationCodeS3ObjectVersion and wires them into every Lambda Code block, so the deploy scripts' existing versionId plumbing now both passes CloudFormation validation and actually forces function updates when a new zip lands under the same S3 key - OIDC deploy policy grants s3:GetObjectVersion (publish-layer-version with S3ObjectVersion reads the versioned object with the caller's credentials) - deploy-aws getBooleanArgString honors space-separated values, so `--run-api-migrations false` no longer runs migrations - sync-legacy-ssm --output=json defaults back to dry-run, restoring the side-effect-free preview contract; pass --dry-run=false to sync while emitting JSON (deploy-backend's invocation never passes json, so real deploys still sync) - removed the execute-api VPC endpoint: API Gateway VPC endpoints do not support the WebSocket @connections API, so it could never work; docs now list WebSocket pushes among the NAT-required features - staging deploy-split-stack.sh defaults API_MIGRATION_RUNNER=data-api; validate-aws-deploy defaults to data-api, rejects "direct" with the same removal message as the deploy scripts, and drops the deleted direct-runner checks - installer-commit scans `git diff --cached` after staging and aborts (unstaging first) if any staged path looks like a secret, instead of trusting a five-path list; allowlist moved to installer-common as the shared DEPLOY_REPO_SAFE_PATHS - installer-run spawns yarn/npm through a shell on Windows (cmd shims cannot be spawned directly), quoting args with spaces - installer-start's synced check watches only the safe-path allowlist, so stray files like .DS_Store no longer wedge the runner in a commit loop; the scaffold .gitignore also ignores .DS_Store/Thumbs.db Verified: full smoke harness green against this tree (fixture for the api-migrations validation scenario updated on the harness branch for the data-api default). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
Closing this out — the AWS self-hosting installer is being pulled into its own standalone repo so it can be maintained independently and evolve without needing to land here first. Thanks for the review time so far; will link the new repo if it's useful to point to from here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
yarn installer:runentrypoint that walks a non-technical operator from empty AWS account to a verified, logged-in B1Admin install (evidence-file state machine, interactive customer interview, approval gates before anything state-changing)infrastructure/environments/start-here.md(setup) andoperations.md(updates, backups, secrets, cost control, including a ~$80-90/mo default-stack estimate)Scope notes for reviewers
scripts/smoke-aws-tooling.mjs, ~8k lines, plus the 40 example output fixtures it checks) lives in follow-up PR Add AWS tooling smoke harness dhempler/B1Admin-source-fork#1 (branchcodex/aws-tooling-smoke-harness, stacked on this branch). It was branched from this PR's exact code state and passes against it.Testing
node scripts/smoke-aws-tooling.mjs— every script exercised against stubbedaws/ghbinaries; passes clean. Run it with invalid AWS credentials in the environment so its few live-CLI scenarios cannot touch a real account.node --checkacross all scripts; CloudFormation templates parse;installer:commitsecret-guard and HTTP smoke fallback verified against local fixtures.This is opened as a draft for maintainer review before it is considered ready to merge.