Skip to content

fix: harden transform caches and playground package boundaries#980

Merged
samchon merged 34 commits into
masterfrom
fix/cache-and-playground-boundaries
Jul 23, 2026
Merged

fix: harden transform caches and playground package boundaries#980
samchon merged 34 commits into
masterfrom
fix/cache-and-playground-boundaries

Conversation

@samchon

@samchon samchon commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Root cause

The reported Bun runtime delay combines two independent costs.

A checker-backed plugin such as Typia still needs one coherent project compile. On an empty ttsc plugin cache, ttsc also compiles the Go source plugin once. That persistent build can be moved out of application startup with ttsc prepare.

The minute-scale warm growth was not required compiler work. The Unplugin transform cache revalidated the complete project before returning every first module from a result that had already compiled the whole project. Exact public ttsc@0.20.0, @ttsc/unplugin@0.20.0, and typia@13.2.0 controls grew from 4.56 seconds at 100 modules to 103.74 seconds at 800 modules. The 800-module run made 732,246 synchronous reads. The first-party Banner plugin reproduced the same class, so this is a ttsc/Unplugin defect rather than a Typia defect.

Changes

  • Treat one real bundler build, and one immutable Bun runtime setup, as one transform generation. Each module's first delivery compares its supplied source in constant time; repeated delivery and lifecycle-less hosts retain complete project and graph-input validation.
  • Preserve authoritative newer generations across rejected, mismatching, and superseded cache races.
  • Keep graph inputs reached through symlinks and Windows junctions in Unplugin and Metro external-input snapshots.
  • Match Bun 1.3.14 loader composition and BuildConfig.files ownership, including runtime pass-through, build start, NUL virtual ids, in-memory files, separator and drive-letter normalization, component case, relative/absolute spelling, dot segments, and cwd changes.
  • Complete the Playground CommonJS resolver's Node-compatible exports, array, target, legacy directory, npm-alias self-reference, and nearest-scope boundaries.
  • Authenticate npm tarballs with the strongest supported SRI digest or legacy shasum, enforce compressed and expanded byte budgets, validate tar framing, and confine every selected path below one safe consistent archive root.
  • Preserve cancellation through fetch, metadata JSON, response handoff, stream, bodyless fallback, digest, decompression, metadata caller, and archive caller boundaries while disposing unused responses and readers.
  • Document the cache lifecycle, cold plugin preparation, archive trust limits, and CommonJS evaluator boundary.

Measured acceptance

With a warm Typia source-plugin cache, the corrected Bun runtime control completed in 10.802 seconds at 400 modules and 33.892 seconds at 800 modules. This removes the retained per-module project revalidation while preserving the one required checker-backed project compile.

With exact 0.20.0, a warm Go build cache, and an empty ttsc plugin cache, ttsc prepare took 47.10 seconds; the subsequent one-module Bun start took 1.37 seconds without rebuilding the plugin binary.

Verification

  • complete @ttsc/test-unplugin suite;
  • complete @ttsc/test-metro suite, 59 features;
  • complete @ttsc/test-playground suite, 35 features;
  • repository-wide pnpm test:typecheck;
  • affected package builds;
  • exact Bun 1.3.14 real-build path and loader controls;
  • Node 22 package-resolution controls;
  • live authenticated typia@13.2.0 and @types/node@26.1.1 archive controls;
  • public Playground package checks for 0.19.0 and 0.20.1;
  • all 34 changed feature documentation/export contracts;
  • changed-file Prettier and git diff --check;
  • five full Overall Self-Review rounds, ending clean; and
  • one renewed discovery cycle with no additional sound candidate in the declared cache, adapter, resolver, and archive surface.

The root pnpm test umbrella completed project layout, generated-flag validation, current-platform builds, Go tests, and repository typecheck locally before its serial lint corpus exceeded the 20-minute command transport window. The same suites are split across the pull-request checks.

On the final head, 51 checks pass. The remaining nestia failure is a confirmed external baseline failure: ttsc master run 30012034569, Nestia PR #1594 with published ttsc@0.20.0, and this pull request all fail at the same Nestia openapi_v2 diagnostic. Nestia always supplies a default server description even when targeting Swagger 2.0, while @typia/utils@13.2.0 correctly reports that the field cannot be represented. This compatibility regression is tracked as samchon/nestia#1595; it is neither introduced nor affected by this pull request.

Typia ownership

No Typia issue is warranted by the evidence. Banner reproduces the dominant warm defect, Typia output and integrity controls pass, and the plugin binary cache remains stable after preparation. The separate Nestia openapi_v2 failure also originates in the document Nestia supplies at its Swagger 2.0 boundary, not in ttsc or Typia transformation. If a later isolated Typia-specific failure is demonstrated, it should be reported separately with that reproduction rather than attached to this cache defect.

Issues

Closes #252
Closes #969
Closes #975
Closes #976
Closes #977
Closes #978
Closes #979

#670 is already fixed by #699. The first release containing that correction is v0.19.0; this pull request adds adjacent resolver and archive hardening but does not claim the original fix.

samchon added 2 commits July 24, 2026 03:34
Close #975: fix(unplugin): track compiler inputs reached through filesystem links

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review for d404d5f: reviewed the complete fc09401..d404d5f diff and the linked-input classification consumers in Unplugin and Metro. The lstat-based predicate matches the non-following project walk for directory links, file links, junctions, broken links, missing paths, and regular files. The external-input hash and Metro snapshot paths preserve the linked spelling as intended. No actionable findings; git diff --check passes.

samchon added 2 commits July 24, 2026 03:53
Close #252: validate a cached generation once per module lifecycle instead of rehashing the project before every first delivery.

Close #969: remove the per-module full-project validation cost behind the reported Bun startup regression.

Close #978: preserve and retry a newer cache generation when stale input validation finishes late.
Restrict first-delivery cache shortcuts to hosts that signal a build start, and retry any generation superseded while a caller was awaiting it.

Close #976: let Bun loaders fall through on excluded or unchanged modules and clear bundler generations through onStart.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review for 55124be found two valid high-severity gaps, both accepted and remediated in 90a3749. First-delivery O(1) reuse is now enabled only after an explicit build-start signal; Metro, Turbopack, Bun runtime, and other lifecycle-less caches perform complete validation on every hit. Second, every awaited cache generation is identity-checked before validation or selection, so a matching waiter retries a newer authoritative generation instead of returning superseded output. Dedicated regressions cover an unrelated input changing before a lazy module's first request and concurrent matching/mismatching waiters. Focused tests and the exact Bun 1.3.14 composition control pass.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review of 55124be5b..90a374971

Two high-severity findings were reproduced and accepted.

  1. The persistent-cache correction restored a complete project validation on every first Bun runtime module delivery because Bun.plugin() exposes no onStart. That reintroduced the O(N×P) path behind #969.
  2. Vite serve calls buildStart once for the development plugin container rather than once per HMR edit. Treating it as a generation boundary allowed a previously emitted but unserved module to cross a later edit without validation.

Remediation:

  • 65c5f0e16 defines one Bun.plugin() setup as one process-scoped module-loading session, while continuing to clear every Bun.build() generation through onStart.
  • Vite serve now explicitly resets to persistent validation mode; production and real per-build hosts retain constant-time first delivery.
  • Dedicated Bun runtime and real Vite serve regressions reproduce both lifecycle shapes.
  • 24b064e19 removed a review-only global I/O monkey-patch and aligned the canonical website guide, source comments, and numbered test contracts with the implemented lifecycle.

Focused adapter, persistent-freshness, race, typecheck, and real Bun 1.3.14 controls pass after remediation. No evidence in this range identifies a Typia implementation defect.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review of 65c5f0e16..66a0a4b4b

Six Node-compatibility gaps were reproduced against Node v22.21.0 and accepted:

  • empty and null-exhausted exports arrays incorrectly reached an outer default;
  • alias self-reference crossed a nearer package scope;
  • exports: null incorrectly suppressed legacy resolution;
  • URL-style targets were looked up as raw pack keys instead of normalized pathnames;
  • nested dot-prefixed target keys were treated as fatal configuration errors; and
  • missing main targets and JSON directory indexes did not complete legacy fallback.

d89ef2fb2 remediates the full class. It preserves the last meaningful array decision, stops at the nearest manifest, treats null exports as absent, resolves target URL pathnames with one decoding step and mount confinement, leaves inactive nested keys unresolved, and completes root/main-directory JSON fallback.

Four new feature scenarios cover the reported matrix, and the complete create_sandbox_require suite passes. Existing protections for valid-but-missing selected targets, mixed top-level maps, numeric conditions, wildcard specificity, encoded separators, and evaluation-cache eviction remain intact.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review of 66a0a4b4b..65c5f0e16

The cache-session implementation had no reproduced functional defect, but the review found three correctness-of-change issues:

  • the canonical website guide still described Bun runtime as a persistent validating cache;
  • the performance regression used a forbidden global fs.readFileSync monkey-patch; and
  • the two new feature tests lacked the required numbered scenario summary.

24b064e19 resolves all three. The guide and source comments now state the process-scoped Bun runtime contract and Vite serve's persistent contract. The Bun test observes the contract through generation behavior rather than replacing a global API, and both feature files carry the required three-step scenarios.

The affected Bun/Vite tests and Unplugin typecheck pass after remediation.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review of 65c5f0e16..d89ef2fb2

Two additional Node-compatibility findings were reproduced and accepted.

  1. URL/path conversion failures such as encoded separators, malformed percent escapes, and a selected package-directory target were classified as skippable invalid targets. In an exports array this executed a later fallback, whereas Node selects the first target and then reports a load error.
  2. Legacy CommonJS directory resolution remained asymmetric: relative directories and bare package subdirectories did not consistently load package.json#main or index.json.

7146bd08e remediates both findings. Invalid package-target selection is now distinct from post-selection module-specifier/path loading, so arrays cannot fall through after a target was selected. A shared legacy file-or-directory resolver now serves package roots, relative requests, and bare subpaths with consistent main and JSON-index behavior.

The expanded resolver suite passes, including exact Node 22 controls for encoded separators, ./ array targets, malformed escapes, relative JSON directories, nested manifests, and bare subdirectory manifests.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Individual Self-Review of 24b064e19..3afc77ae7

Three functional findings and two coverage/convention findings were accepted.

  • High: literal package/ confinement rejected official DefinitelyTyped tarballs, including @types/node, @types/lodash, and @types/react.
  • High: pending stream reads and WebCrypto digests did not reject promptly on abort.
  • Medium: a Content-Length preflight rejection did not cancel the response body.
  • Safe GNU long-name compatibility lacked a positive regression.
  • Six feature-test comments lacked numbered scenario summaries.

9b83bf6ae remediates the complete set:

  • extraction establishes one safe, consistent top-level root and strips it, preserving normal package/ archives and current @types/* name-root archives while rejecting mixed roots and unsafe segments;
  • stream reads race the abort signal and request cancellation, WebCrypto results are fenced by an abort race, and oversized declared bodies are cancelled;
  • safe GNU long-name, official-root, pending-abort, cancellation, and documentation-contract cases are pinned; and
  • public guidance now describes a consistent archive root rather than promising literal package/.

Live registry controls pass for authenticated typia@13.2.0 and @types/node@26.1.1; the latter unpacks 89 selected files with its manifest and index.d.ts intact. The complete Playground feature suite passes after remediation.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Self-Review, round 1

Reviewed the complete PR surface from 5a63e0 through the current cache, Bun, Playground resolver, archive, test, and documentation changes. This round found four actionable boundary defects.

  1. Bun's BuildConfig.files matcher resolved relative callback paths against the callback-time cwd and described separator behavior too strictly. A real Bun 1.3.14 control failed after process.chdir(). Commit 477e13cab captures the setup cwd, accepts Bun's normalized path spelling, and pins memory-only, override, relative, and cwd-change cases against the real runtime.
  2. The first-delivery performance regression replaced global fs.readFileSync, contrary to the repository's test-isolation rule. 477e13cab now mutates a real plugin input after the first compile and proves that the build-scoped generation is not revalidated or rerun for later first module deliveries.
  3. Metro and Bun were still grouped together as persistent caches in source/test prose. 477e13cab separates Metro/Turbopack's every-hit validation from Bun runtime's immutable setup session and aligns the package documentation.
  4. A response implementation with body === null could stall forever in arrayBuffer() after cancellation. 9b58898f4 puts the fallback behind the same abort race as stream and digest work. Its Individual Self-Review then reproduced a narrower fetch-completion race where arrayBuffer() could start after the signal was already aborted; f8ecec5b0 checks cancellation before invoking the fallback and asserts the post-abort call count remains zero.

Focused builds and regressions pass after each remediation. Round 1 is not a clean review because it produced findings; a fresh full-surface round follows.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Self-Review, round 2

The fresh second pass was not clean. It found two further classes at public adapter boundaries, and the mandatory per-commit reviews widened both before acceptance.

  1. Bun file-map identity was still broader than Bun 1.3.14. deb245f99 first stopped Windows component-case folding, but its review proved that Bun also preserves relative versus absolute spelling and ./.. segments; only separators and drive-letter case are normalized. The over-broad matcher could silently yield a real disk input and skip a required ttsc transform. 7282649e5 now compares exactly that Bun identity. Unit controls distinguish early ownership from a no-op transform through the options provider, and real Bun builds retain memory-only, override, relative, separator, cwd-change, and absolute cases while using disk for case-only, relative/absolute, and dot-segment aliases.
  2. Cancellation stopped stream, digest, decompression, and bodyless fallback waits, but the public custom-fetch seam could still stall in the fetch Promise or metadata JSON. b03aece07 added those abort races. Its review then reproduced an orphan rejected Promise when fetch synchronously aborted, uncancelled responses that fulfilled after losing the race, and tests that aborted before the intended inner phase had started. f54a700b2 makes the abort helper own a producer before evaluating the changed signal state, disposes late Response bodies, preserves the exact abort reason, and uses explicit phase latches for metadata JSON, stream reads, bodyless reads, and fetch handoff. A late rejection after synchronous abort is also exercised under Node's unhandled-rejection policy.

The complete Playground suite, all Bun adapter cases, the exact Bun 1.3.14 matrix, and targeted package builds pass after remediation. Round 2 remains a findings round; a new full-surface review starts from the remediated head.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Self-Review round 3 is not clean.

I re-read the complete 54-file change surface after the Bun and response-handoff remediations, including the Unplugin lifecycle/race paths, Metro linked-input ownership, Playground archive and package-resolution boundaries, public options, documentation, and all 33 changed feature-test contracts.

Finding:

  • Medium: installPlaygroundDependencies can lose the exact abort reason at its archive-stage error boundary. Its catch block currently checks signal.aborted and rethrows the stage error. If an archive operation rejects and the signal aborts before the caller's catch continuation runs, the public install promise rejects with that older stage error even though cancellation is now authoritative. A deterministic reproduction used a successful custom tarball response whose headers.get() queued controller.abort(reason) and then threw. The result was signal.aborted === true, but the promise rejected with the header error rather than signal.reason. The catch must call throwIfAborted(options.signal) before wrapping a non-abort stage error, and a microtask-handoff regression must pin exact reason identity.

The prior remediation commits are independently clean: 2c8933bc9 preserves aborts across the nested response handoff and disposes unused bodies, 8ec5346b4 completes the actual Bun 1.3.14 Windows path-identity matrix, and 99d385047 restores strict typing without weakening the integrity mutation. Changed-file Prettier, git diff --check, and the 33 feature documentation/export contracts pass. The current CI head has no failed checks, but this review cannot be marked clean until the public install boundary is corrected and a fresh Overall Self-Review finds nothing further.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Self-Review round 4 is not clean.

The archive-stage remediation in bed99558d correctly restores exact cancellation at the public install catch, but its mandatory Individual Self-Review found the symmetric metadata boundary immediately above it.

A real Response with a controlled json() Promise reproduced the gap deterministically. After fetchNpmMetadata() had selected Error("metadata JSON failed"), the test queued controller.abort(reason) between the internal rejection handler and installPlaygroundDependencies() resuming. On bed99558d, the public promise observed signal.aborted === true but still rejected with the stale metadata error rather than signal.reason.

Commit 62c026436 remediates that boundary by catching metadata resolution, calling throwIfAborted(options.signal), and rethrowing only a still-authoritative non-abort error. The same feature now pins both metadata and archive handoffs with exact reason identity. All four async archive-install boundaries are fenced: metadata, download, integrity verification, and unpacking.

The Playground package build, repository-wide test:typecheck, the focused handoff regression, all 35 Playground features, changed-file Prettier, feature-contract scan, and git diff --check pass. This remains a findings round because the metadata gap was discovered here. A completely fresh full-surface review follows from 62c026436.

@samchon samchon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Self-Review round 5: clean.

This pass restarted from 62c026436 and re-read the complete 55-file change surface rather than checking only the last remediation. I revalidated:

  • Unplugin generation ownership, first-delivery cost, persistent validation, stale-generation identity guards, volatile outputs, watch derivation, linked inputs, and Vite/Bun/Metro/Turbopack lifecycle distinctions;
  • Bun 1.3.14 loader composition and its exact BuildConfig.files identity rules for separators, drive-letter case, component case, relative/absolute spelling, dot segments, cwd changes, and NUL virtual ids;
  • Playground package exports selection versus loading, array blocking, target confinement, legacy directory fallbacks, alias self-reference, and nearest package scope;
  • npm metadata, response, stream, digest, decompression, tar-record, root-confinement, byte-budget, error-context, and cancellation handoffs, including the exact reason at metadata and archive caller boundaries;
  • public option types and package/website guidance; and
  • every changed feature file's single export, Verifies summary, rationale, and numbered scenario contract.

The final metadata remediation has a clean Individual Self-Review. Its controlled Promise order is deterministic, late rejection is handled, optional 404 behavior is unchanged, and non-abort metadata errors retain their original identity. The archive boundary continues to preserve package/version context and original causes for non-abort failures.

Verification on the final head:

  • complete @ttsc/test-unplugin suite: pass;
  • complete @ttsc/test-metro suite, 59 features: pass;
  • complete @ttsc/test-playground suite, 35 features: pass;
  • repository-wide pnpm test:typecheck: pass;
  • affected package builds: pass;
  • all 34 changed feature contracts: pass;
  • changed-file Prettier and git diff --check: pass;
  • exact Bun 1.3.14 behavior controls and the warm Typia/Banner performance controls remain consistent with the implementation.

No actionable correctness, reliability, security, compatibility, documentation, or test-integrity finding remains. The evidence still identifies the dominant #969 defect in ttsc/Unplugin, not Typia.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment