From 78e87590ac7b6abc026bdab29ae8e1f997d86446 Mon Sep 17 00:00:00 2001 From: Patrice Chalin Date: Sun, 31 May 2026 14:14:08 -0400 Subject: [PATCH] Update repo-reorg task docs after monorepo cleanup --- tasks/0.16/repo-reorg/README.md | 21 ++-- tasks/0.16/repo-reorg/spike-notes.md | 28 ++++- .../repo-reorg/theme-only-folder.execution.md | 82 ++++++------- .../0.16/repo-reorg/theme-only-folder.plan.md | 114 +++++++++--------- 4 files changed, 132 insertions(+), 113 deletions(-) diff --git a/tasks/0.16/repo-reorg/README.md b/tasks/0.16/repo-reorg/README.md index 9c8924295..a0acc1714 100644 --- a/tasks/0.16/repo-reorg/README.md +++ b/tasks/0.16/repo-reorg/README.md @@ -36,17 +36,17 @@ specific status decision matters. ## Status at a glance -Updated 2026-05-27. Per-phase detail (with exit criteria) lives in the -[execution plan][exec]. The canonical TOF move is now on `main` since [#2641][]. +Updated 2026-05-31. Per-phase detail (with exit criteria) lives in the +[execution plan][exec]. The canonical TOF move is on `main` since [#2641][]. -| Phase | Status | -| ------------------------------------ | --------------------------------------------------------------- | -| 0 — structural move | Done — landed on task branch, and `main` | -| 1 — `docsy.dev` consumes TOF | Done — local + Netlify deploy preview green ([#2640][]) | -| 2 — local smoke tests (CI emulation) | Done — local smoke passes; CI smoke matrix green ([#2640][]) | -| 3 — GitHub CI | Done — run-hugo + install:all; full CI matrix green ([#2640][]) | -| 4 — `docsy-example` | Done — final update merged via [#458][] | -| 5 — docs and release notes | Next | +| Phase | Status | +| -------------------------------------- | ------------------------------------------------------------ | +| 0 — structural move | Done — landed on `main` via [#2641][] | +| 1 — `docsy.dev` consumes TOF | Done — local + Netlify deploy preview green ([#2640][]) | +| 2 — local smoke tests (CI emulation) | Done — local smoke passes; CI smoke matrix green ([#2640][]) | +| 3 — GitHub CI | Done — workspace install; full CI matrix green | +| 4 — `docsy-example` + monorepo cleanup | Done — final example update + package/workspace mini-cycle | +| 5 — docs and release notes | Next | Next concrete steps, in order: @@ -55,6 +55,7 @@ Next concrete steps, in order: - (b) merge the task branch to `main` via [#2641][] — **done**. - (c) final `docsy-example` config + test against Docsy from `main` — done via [google/docsy-example#458][#458]. + - (d) monorepo package/workspace cleanup — **done**. 2. **Phase 5 (next):** update the get-started "clone" docs for the new non-module setup procedure recorded in [spike-notes][spike] Phase 2, plus changelog/blog and release-process notes for the nested module tag diff --git a/tasks/0.16/repo-reorg/spike-notes.md b/tasks/0.16/repo-reorg/spike-notes.md index 56aba5b5d..acc3cdba4 100644 --- a/tasks/0.16/repo-reorg/spike-notes.md +++ b/tasks/0.16/repo-reorg/spike-notes.md @@ -87,15 +87,35 @@ Setup commands after TOF: ```sh cd themes git clone -b https://github.com/google/docsy -(cd docsy/theme && npm install) -(cd docsy && node scripts/mkdirp-hugo-mod.js ..) +(cd docsy && npm run postinstall) cd .. npm install --save-dev autoprefixer postcss-cli ``` For a Git submodule install, keep the existing submodule workflow but run the -same `docsy/theme` npm install and `mkdirp-hugo-mod.js` commands after updating -the submodule. +same `docsy` postinstall after updating the submodule. + +## Current monorepo design after the Phase 4 mini-cycle + +This section supersedes the earlier spike narrative where it conflicts. + +- Root `package.json` is private and declares the two workspaces: `docsy.dev` + and `theme`. +- Root `package.json` has no theme runtime dependencies. It owns repo-wide + maintainer tooling and the GitHub-NPM package contract through `files`. +- The GitHub-NPM package surface is intentionally small: `theme/` plus + `scripts/mkdirp-hugo-mod.js`, excluding `theme/node_modules` and + `theme/package-lock.json`. +- `theme/package.json` owns the theme runtime dependencies and + `theme/package-lock.json`. +- Root `postinstall` runs `install:theme-deps` and `_mkdir:hugo-mod`. +- `docsy.dev/package.json` owns website/dev tooling, including `hugo-extended`, + and its `postinstall` delegates to the root `install:theme-deps` script. +- Netlify builds from the repo root with `npm run -C docsy.dev build:preview` or + `build:production`; `docsy.dev/netlify.toml` sets `HUGO_THEME=repo/theme`. +- CI and local smoke use plain `npx hugo` after `npm install`; the temporary + `scripts/run-hugo.mjs` helper was removed. +- `scripts/npm-pack.test.mjs` is the fast guard for the GitHub-NPM tarball. ## Repo state at start of spike diff --git a/tasks/0.16/repo-reorg/theme-only-folder.execution.md b/tasks/0.16/repo-reorg/theme-only-folder.execution.md index dfeac4da6..91571ff01 100644 --- a/tasks/0.16/repo-reorg/theme-only-folder.execution.md +++ b/tasks/0.16/repo-reorg/theme-only-folder.execution.md @@ -59,9 +59,8 @@ small stack) so the move is reviewable as a unit. `theme/` (`mkdirp-hugo-mod.js`, `_gen-chroma-style.sh`, `refresh-sass-variables.pl`, `scrollspy-patch/*`, `getHugoModules/`, `make-site.sh`). Update root `package.json` script globs accordingly. -- Add `theme/package.json` as a private mirror of root's theme deps, plus - `scripts/sync-theme-deps.mjs` and the hook wiring (`_prepare`, - `postupdate:dep`, `postupdate:packages`). +- Add `theme/package.json` as the canonical owner of theme runtime deps + (`bootstrap`, `@fortawesome/fontawesome-free`). - Confirm `_prepare` (and downstream `ci:prepare`) succeeds against the new paths and `_diff:check` is clean. @@ -78,16 +77,15 @@ Make Docsy's primary end-to-end test (the website) build against the new layout. This is the most informative single check. - Update `docsy.dev` config to use the new theme path (`theme: [docsy/theme]`). -- Add the `_install-theme-deps` postinstall to `docsy.dev/package.json` so theme - runtime deps land in `docsy.dev/node_modules/` (consumer-cwd lookup). +- Add a `docsy.dev` postinstall hand-off so local website installs also install + `theme/` runtime deps. - Build `docsy.dev` locally. Iterate on TOF if anything in the build chain surprises. - Record the exact before/after config change and any maintainer-workflow observations in `tasks/0.16/repo-reorg/spike-notes.md`. -- Confirm Netlify deploy previews work from the task branch - (`docsy.dev/netlify.toml`: `npm run install:all && npm run build:preview`, and - the production variant; `docsy.dev`'s `install:all` installs the repo root - then `docsy.dev`). +- Confirm Netlify deploy previews work from the task branch using + `docsy.dev/netlify.toml` (`npm run -C docsy.dev build:preview`, and + `build:production` in production). Exit criterion: `docsy.dev` builds locally and on Netlify against the task branch. @@ -95,9 +93,9 @@ branch. Status (2026-05-27): **exit criterion met.** `docsy.dev` builds from `theme/` with the one-line `theme: [docsy/theme]` consumer config change and no symlinks anywhere (223 EN + 218 FR pages), and the Netlify deploy preview is green on -[#2640][]. Netlify uses `npm run install:all && npm run build:preview` (and -`build:production` in production); `docsy.dev`'s `install:all` installs the repo -root then `docsy.dev` (workspaces are empty). +[#2640][]. The final Netlify command is `npm run -C docsy.dev build:preview` +(and `build:production` in production), with `HUGO_THEME=repo/theme` set for the +Netlify clone shape. ### Phase 2: local smoke tests (CI emulation) @@ -116,14 +114,10 @@ _same_ Hugo-resolution mechanism CI uses, and the spike-notes matrix is complete with exact one-line edits. A local-only pass that relies on an ad-hoc `HUGO` override does not count. -Status (2026-05-27): **exit criterion met.** All three install modes build with -the same Hugo-resolution mechanism CI uses (`run-hugo.mjs`, no `HUGO` override), -local smoke tests pass on `main`, and the CI smoke matrix is green ([#2640][]). -Earlier local-only runs had masked a gap — they set `HUGO` explicitly, while -CI's `npx hugo` found nothing once `docsy.dev` stopped being a workspace (so -`hugo-extended` was no longer hoisted to the repo root). Closing that gap is -exactly what `run-hugo.mjs` + `install:all` do (Phase 3). The non-module-clone -setup-step follow-up for Phase 5 still stands. +Status (2026-05-31): **exit criterion met.** All three install modes build with +the same Hugo-resolution mechanism CI uses (`npx hugo` after the workspace +install), local smoke tests pass on `main`, and the CI smoke matrix is green +([#2640][]). The non-module-clone setup-step follow-up for Phase 5 still stands. [spike-notes]: ./spike-notes.md @@ -132,21 +126,11 @@ setup-step follow-up for Phase 5 still stands. Lift Phase 2 into Actions on the same branch. Paired with Phase 2 (see Working principles). -- **Resolve Hugo availability on the runner.** Root cause: `docsy.dev` is no - longer an npm workspace, so root `npm install` no longer hoists - `hugo-extended` into the repo-root `node_modules/.bin/` where the smoke test's - `npx hugo` resolved it. **Decided: option E, made cross-OS** — - `scripts/run-hugo.mjs` reuses `docsy.dev`'s installed `hugo-extended` (version - single-sourced via `config.hugo_version`) and is now the `make-site.sh` - default; verified locally for NPM + HUGO_MODULE. See [spike-notes][] Phase 3. -- **Make `hugo-extended` available to the smoke job** (done): added an - `install:all` root script (`npm run docsy.dev-install && npm install`) and - switched `smoke.yaml`'s "Setup workspace" step to `npm run install:all`, so - `docsy.dev`'s `hugo-extended` is installed where `run-hugo.mjs` looks. - (`docsy.dev-install` runs first so `test.yaml`'s - `install:all -- --omit=optional` lands the flag on the final root - `npm install`.) Chose `install:all` over a lean targeted install for - consistency + onboarding value. +- Restore a real npm workspace shape: repo root declares `docsy.dev` and `theme` + as workspaces. +- Use plain `npm install --omit=optional` in CI. This installs workspace + tooling, including `docsy.dev`'s `hugo-extended`, so `npx hugo` works without + a custom runner script. - Update `.github/workflows/smoke.yaml` and `.github/workflows/test.yaml` for the new paths (done). - Push the branch and watch the Windows + Ubuntu matrix go green (done). @@ -155,11 +139,10 @@ Exit criterion: full CI matrix green on the task branch (which also closes Phase 2). **Decision gate to merge to `main`.** If everything above held, the canonical move could land. -Status (2026-05-27): **exit criterion met — CI matrix green ([#2640][]).** Both -`test` (build; ubuntu + windows) and `smoke` (new-site NPM + HUGO_MODULE; ubuntu - -- windows) pass, and the Netlify deploy preview is green. The merge gate is - satisfied; the canonical move then landed on `main` via [#2641][]. +Status (2026-05-31): **exit criterion met — CI matrix green ([#2640][]).** Both +`test` (build; ubuntu + windows) and `smoke` (new-site NPM + HUGO_MODULE; +ubuntu + windows) pass, and the Netlify deploy preview is green. The merge gate +is satisfied; the canonical move then landed on `main` via [#2641][]. ### Phase 4: `docsy-example` @@ -175,12 +158,22 @@ TOF layout. Three parts, in order: `docsy-example` import path / `theme:` value to the released `google/docsy` layout and confirm a clean build (and its own smoke checks) against `main`. Done via [google/docsy-example#458][#458]. +- **(d) Monorepo cleanup mini-cycle (done):** settle the final package/workspace + design: + - root `package.json` is private, declares `docsy.dev` and `theme` workspaces, + and uses `files` as the GitHub-NPM package contract; + - `theme/package.json` owns runtime deps and `theme/package-lock.json`; + - `docsy.dev/package.json` owns website/dev tooling and delegates theme dep + installs to the root `install:theme-deps` script; + - `docsy.dev/netlify.toml` builds with `npm run -C docsy.dev ...`; + - `scripts/npm-pack.test.mjs` guards the packed GitHub-NPM surface; + - the local clone smoke case runs the root `postinstall` in `themes/docsy`. Exit criterion: `docsy-example` builds against released Docsy from `main` and passes its own checks. -Status (2026-05-27): **exit criterion met.** (a) done; (b) done via [#2641][]; -(c) done via [google/docsy-example#458][#458]. +Status (2026-05-31): **exit criterion met.** (a) done; (b) done via [#2641][]; +(c) done via [google/docsy-example#458][#458]; (d) done. ### Phase 5: docs and release notes @@ -207,8 +200,8 @@ covers what the CI smoke matrix runs (NPM, HUGO_MODULE) **plus** the non-module clone, which CI does not exercise. Target repo/branch via `--repo` / `--branch` (defaults: `google/docsy`, `main`; the `test:smoke` npm script also passes `--branch main`). During TOF rollout on a task branch: -`npm run test:smoke -- --branch task/repo-reorg-2026-05`. Prereq: -`npm run install:all` (for `hugo-extended`). Deliberately kept out of +`npm run test:smoke -- --branch task/repo-reorg-2026-05`. Prereq: `npm install` +(for workspace tooling, including `hugo-extended`). Deliberately kept out of `test:tooling` / CI `ci:post` (slow, network-bound). **Later (not started):** @@ -238,6 +231,7 @@ canonical list of deferred work. - Spike notes: `tasks/0.16/repo-reorg/spike-notes.md`, grown through Phases 1–3. - The main TOF move has merged to `main` via [#2641][]. - Phase 4c landed via [google/docsy-example#458][#458]. +- The Phase 4 monorepo cleanup mini-cycle is done. - Phase 5 lands as follow-up PRs against `main`. [#2617]: https://github.com/google/docsy/issues/2617 diff --git a/tasks/0.16/repo-reorg/theme-only-folder.plan.md b/tasks/0.16/repo-reorg/theme-only-folder.plan.md index 9b7005c2c..0ddc2bdfa 100644 --- a/tasks/0.16/repo-reorg/theme-only-folder.plan.md +++ b/tasks/0.16/repo-reorg/theme-only-folder.plan.md @@ -51,17 +51,17 @@ install mode. It trades a small documented migration for a clean, predictable install shape. Mapping the four [Motivation](#motivation) workarounds to this PR: -| Workaround | This PR | -| ----------------------------------------- | ------------------- | -| `../../node_modules/*` escape mounts | **Fixed** | -| `postinstall` mkdirp helper | Kept; deferred | -| `_prepare` scrollspy + vendored SCSS | Kept; deferred | -| Root `package.json` mixes theme + tooling | Partially; deferred | - -"Partially" because `theme/package.json` now declares the theme runtime deps (as -a synced mirror), giving us a clean boundary to push the rest of the cleanup -through. The thin-shim refactor that fully separates the manifests is -[scoped out](#tof-repo-layout) and lives in a follow-on plan. +| Workaround | This PR | +| ----------------------------------------- | ----------------------------------------------- | +| `../../node_modules/*` escape mounts | **Fixed** | +| `postinstall` mkdirp helper | Kept, but scoped to the minimal install surface | +| `_prepare` scrollspy + vendored SCSS | Kept as maintainer-only generated artifacts | +| Root `package.json` mixes theme + tooling | **Fixed for runtime deps** | + +The final Phase 4 mini-cycle moved the repo to a cleaner monorepo shape: +`theme/package.json` owns theme runtime dependencies, the repo root is private +workspace orchestration, and the GitHub-NPM install surface is constrained by +the root `files` list. ## TOF repo layout @@ -77,26 +77,28 @@ through. The thin-shim refactor that fully separates the manifests is │ ├── theme.toml # canonical │ ├── go.mod # module path: github.com/google/docsy/theme │ ├── go.sum -│ └── package.json # private mirror of theme runtime deps (see below) +│ └── package.json # canonical theme runtime deps ├── docsy.dev/ # website; its own node_modules for the site build -├── scripts/ # maintainer scripts (scrollspy patch, sync-theme-deps, …) +├── scripts/ # maintainer scripts and install helper +├── tests/ # repo-level smoke tests ├── tasks/ -└── package.json # repo root: canonical for theme runtime deps + repo-wide tooling +└── package.json # private workspace root + GitHub-NPM files contract ``` Key properties: - `theme/` is what Hugo module resolves to. GitHub-NPM still ships the full repo - (see below). -- The repo root `package.json` is the **single source of truth** for theme - runtime dependency versions and continues to host repo-wide maintainer tooling - (Prettier, markdownlint, cSpell French dict). The `theme/` `package.json` is a - small **private mirror** so that file/tarball installs of `theme/` (notably - `docsy.dev`'s postinstall — see [Maintainer workflow](#maintainer-workflow)) - see the right versions. -- The mirror is kept in sync by `scripts/sync-theme-deps.mjs` (run by - `_prepare`, `postupdate:dep`, and `postupdate:packages`). Maintainers should - not edit `theme/package.json`'s `dependencies` by hand. + package, but the root `files` list limits the packed/installable surface to + `theme/` plus `scripts/mkdirp-hugo-mod.js`. +- The repo root `package.json` is private and owns workspace orchestration plus + repo-wide maintainer tooling (Prettier, markdownlint, cSpell French dict). It + declares workspaces for `docsy.dev` and `theme`. +- `theme/package.json` is the **single source of truth** for theme runtime + dependency versions (`bootstrap`, `@fortawesome/fontawesome-free`). It is + private because Docsy is not publishing a separate NPM registry package in + 0.16. +- `theme/package-lock.json` is checked in so theme runtime installs are stable + and separate from repo/website tooling. - Theme-only configuration moved to `theme/`: `hugo.yaml`, `theme.toml`, `go.mod`/`go.sum`. The `theme/go.mod` module path becomes `github.com/google/docsy/theme`. @@ -110,9 +112,6 @@ Key properties: What this plan **does not** change (deferred to a follow-on plan): -- The repo root keeps `devDependencies` for repo-wide maintainer tooling and - keeps `bootstrap` + `@fortawesome/fontawesome-free` as runtime deps. There is - no `files` whitelist yet and no thin-shim refactor. - There is no dedicated `_dev/` (or similar) maintainer-orchestration folder yet. Maintainers continue to work from the repo root. - NPM-registry publication remains future work; the immediate consumer surface @@ -188,18 +187,27 @@ release) are run, see the [execution plan][exec]. ## Package boundary -Each `package.json` carries a clearly-scoped set of dependencies. The three -manifests are: +Each `package.json` carries a clearly-scoped set of dependencies: -| File | Role | Notable contents | -| -------------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | -| `package.json` (repo root) | Canonical theme deps + repo-wide maintainer tools | `bootstrap`, `@fortawesome/fontawesome-free`; `prettier`, `markdownlint-cli2`, markdownlint rule plugins, `@cspell/dict-fr-fr`; `postinstall` | -| `theme/package.json` | Private mirror of theme runtime deps | Same `dependencies` as root (kept in sync by `sync-theme-deps.mjs`); `"private": true` | -| `docsy.dev/package.json` | Website's site-build / site-tooling | `autoprefixer`, `postcss-cli`, `cross-env`, `rtlcss`, `afdocs`, `netlify-cli`, `npm-check-updates`, `hugo-extended`; `_install-theme-deps` hook | +- `package.json` (repo root): private workspace root, repo dev tools, `files` + list for GitHub-NPM, and install helpers. +- `theme/package.json`: canonical theme runtime package with `bootstrap` and + `@fortawesome/fontawesome-free`; no dev dependencies. +- `docsy.dev/package.json`: website build and site tooling such as + `hugo-extended`, `autoprefixer`, `postcss-cli`, `cross-env`, `rtlcss`, + `afdocs`, `netlify-cli`, and `npm-check-updates`. Keep docs-site, CI, release, formatting, link-checking, and test-only dependencies out of `theme/package.json`. +The root `files` list is part of the GitHub-NPM contract. It includes: + +- `theme` +- `scripts/mkdirp-hugo-mod.js` +- explicit exclusions for `theme/node_modules` and `theme/package-lock.json` + +The fast `scripts/npm-pack.test.mjs` test guards that package surface. + ## Maintainer workflow The lead maintainer (and other contributors) coordinate theme and website @@ -209,39 +217,35 @@ theme changes, so coordinated edits across both happen daily. Orchestration commands continue to run from the repo root: ```sh -npm run install:all # repo root deps + docsy.dev (incl. hugo-extended) -npm run build # builds docsy.dev against theme/ -npm run serve # docsy.dev dev server with live reload -npm run check # repo-wide format + markdown checks -npm run fix # auto-fix -npm run test:smoke # local NPM / Hugo-module / non-module-clone smoke +npm install # workspace install; also installs theme deps +npm run build # builds docsy.dev against theme/ +npm run serve # docsy.dev dev server with live reload +npm run check # repo-wide format + markdown checks +npm run fix # auto-fix +npm run test:smoke # local NPM / Hugo-module / non-module-clone smoke ``` -`docsy.dev`'s `postinstall` runs `_install-theme-deps` (see -`docsy.dev/package.json`), which creates `theme/`'s declared runtime deps into -`docsy.dev/node_modules/` so Hugo's consumer-cwd lookup finds them during the -build. The mechanics (`--install-links` flag and the cleanup line) are recorded -in [spike-notes][]. - -[spike-notes]: ./spike-notes.md +Root `postinstall` runs `install:theme-deps` and `_mkdir:hugo-mod`. The same +theme-dependency install is also exposed from `docsy.dev`'s `postinstall`, so a +site-only install still refreshes `theme/node_modules` for the local website +build. ## Tooling versions Each shared tool has exactly one declaration in the repo: - `hugo-extended` lives in `docsy.dev/devDependencies`. The `hugo` binary lands - at `docsy.dev/node_modules/.bin/hugo` after `npm run docsy.dev-install` and is - invoked by `docsy.dev`'s `_hugo` scripts. Bumping the Hugo version is a single - edit (see `scripts/set-hugo-version.mjs`). + through the workspace install and is invoked by `docsy.dev`'s `_hugo` scripts + and by `npx hugo` in smoke tests. Bumping the Hugo version is a single edit + (see `scripts/set-hugo-version.mjs`). - `prettier`, `markdownlint-cli2`, the markdownlint rule plugins, and `@cspell/dict-fr-fr` live in the **repo-root** `devDependencies` because they are run from the repo root. - Site-build tools (`autoprefixer`, `postcss-cli`, `cross-env`, `rtlcss`) and site-tooling (`afdocs`, `netlify-cli`, `npm-check-updates`) live in `docsy.dev/devDependencies` because that is where they are used. -- Theme runtime deps (`bootstrap`, `@fortawesome/fontawesome-free`) live in the - repo-root `dependencies` and are mirrored into `theme/package.json` by - `scripts/sync-theme-deps.mjs`. +- Theme runtime deps (`bootstrap`, `@fortawesome/fontawesome-free`) live in + `theme/package.json`. ## Test boundary @@ -262,8 +266,8 @@ toolchain. - A new site can use Docsy through the GitHub/NPM install path with the documented one-line config change. - Non-module theme usage works with the documented one-line config change. -- `theme/package.json` contains only theme runtime dependencies (mirrored from - root) and is marked `"private": true`. No other lifecycle scripts. +- `theme/package.json` contains only theme runtime dependencies and package + update scripts, and is marked `"private": true`. - All generated theme assets (vendored Bootstrap SCSS, scrollspy patch output, chroma styles, `go.sum`) are committed under `theme/`. - CI and smoke tests pass against the new layout on Windows and Ubuntu.