Skip to content

typia generate derives filesystem identity from a lossy fs.Stats.ino, so distinct paths can collide #2269

Description

@samchon

Problem

TypiaGenerateWizard builds every filesystem identity key from fs.Stats.ino used as a JavaScript number. On NTFS the 64-bit file ID routinely exceeds Number.MAX_SAFE_INTEGER, where a double's spacing is 8 or more, so two distinct directories or files can round to the same key.

The keys feed two different guards, and a collision has a different consequence in each:

  • Traversal dedup (visitedDirectories, visitedFiles): a distinct directory or file looks already-visited and is silently skipped. Its sources are never transformed and no diagnostic is emitted — silent source loss.
  • Overwrite protection (ensureTargetFiles): a legitimate output looks like one of the inputs and generation is refused with output file would overwrite input file through a physical file alias. Loud and safe, but a false rejection.

The dedup direction is the severe one. The overwrite guard cannot lose data through this cause, because one file has one ino and therefore one rounded key; rounding cannot make the same file yield two different keys.

Evidence

packages/typia/src/executable/TypiaGenerateWizard.ts:

function fileIdentityKey(stat: fs.Stats, realpath: string): string {
  return stat.ino === 0
    ? `path:${path.normalize(realpath)}`
    : `inode:${stat.dev}:${stat.ino}`;
}

Callers: line 455 and line 482 (ensureTargetFiles input set and output check), line 878 (visited directories), line 946 (visited files).

Measurement, Windows 11 / NTFS, 4000 directories created in one temporary parent. For each, statSync(d).ino (number) was compared with statSync(d, { bigint: true }).ino:

  • 1879 of 4000 directory IDs exceeded 2^53 and are therefore not exactly representable.
  • Observed ID 41658296553217950 sits between 2^55 and 2^56, where the double spacing is 8.
  • 0 collisions were observed at this sample size, so this is a proved representability defect and an unproved-in-the-wild collision, not an observed data loss.

Why collisions are plausible rather than merely arithmetically possible: an NTFS file ID is (sequenceNumber << 48) | mftRecordIndex. Two entries sharing a sequence number with MFT record indices within 8 of each other round to one double. Adjacent MFT records are exactly what a directory tree created in one pass produces.

stat.ino === 0 already has a path fallback, so the code acknowledges that inode identity is not always available; it does not yet acknowledge that it is not always exact.

Consequence surface

  • typia generate in directory mode on NTFS, and on any filesystem whose IDs exceed 2^53 or are otherwise not exactly representable.
  • Silent skip of a source file or an entire subtree, producing an output tree that is incomplete with a zero exit code.
  • Spurious would overwrite input file refusal, blocking a legitimate generation.
  • Not addressed by the directory identity work considered in typia generate follows directory symlinks without physical traversal guards #2082: an additional key combined with .some(...) can only make the guard skip more, so it cannot repair a false positive.

Invariant

Filesystem identity must be exact. Two distinct filesystem objects must never share an identity key, and one object must always yield the same key within a run.

Mechanism is the implementer's to determine and to validate. fs.promises.stat(p, { bigint: true }) removes the representability loss, but it is not by itself a complete answer to filesystem identity here: it does nothing for filesystems that report ino === 0, and the existing path: fallback is itself inexact on win32, where fs.realpathSync does not case-canonicalize, so ...\dir and ...\DIR produce two different fallback keys for one directory while FileSystemIdentity in the same module already owns a caseSensitive policy for that question.

Acceptance and verification

  • Two distinct directories whose 64-bit IDs differ by less than the double spacing at their magnitude receive distinct keys, and neither is skipped.
  • One directory reached through several spellings — relative, absolute, trailing separator, differing case on a case-insensitive volume — receives one key, and the intended alias behaviour is unchanged.
  • A filesystem reporting ino === 0 still terminates and still refuses to overwrite an input.
  • Existing cycle, external-link, output-alias, and repeated-alias regressions in tests/test-typia-generate stay green.
  • The regression must fail when the identity derivation is reverted while the test is retained.

Coordination

Found while implementing #2082 and deliberately kept out of that change, which fixed a locale-dependent traversal order and did not touch identity derivation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions