🐛 Preserve matched target identifier for case-sensitive cross-references

[coretl]

Fixes #2988.

The bug

resolveReferenceLinksTransform looks up link targets by their verbatim identifier before falling back to the normalized (lowercased) form:

const target = opts.state.getTarget(identifier) ?? opts.state.getTarget(reference?.identifier);

...but then unconditionally overwrites the resulting cross-reference with the normalized identifier:

xref.identifier = reference.identifier; // always the lowercased form

So when two case-distinct targets exist, a link to the exact-case one finds the right target and then silently re-points at its lowercase sibling. With a plugin directive registering sample.Match (a class) and sample.match (a function) as targets, both [](#sample.Match) and [](#sample.match) end up as crossReferences to sample.match. resolveUnlinkedCitations has the same pattern a few lines below.

The fix

Keep the identifier of the target that actually matched, falling back to the normalized identifier as before:

xref.identifier = target?.node.identifier ?? reference.identifier;

Ordinary prose labels are unaffected: their stored identifier is the normalized identifier (normalizeLabel at creation time), so the two expressions are identical for every target that exists today. Only targets registered with intentionally case-sensitive identifiers (which today can only come from plugins setting identifier directly on nodes) change behaviour.

Changes

• packages/myst-transforms/src/enumerate.ts: use the matched target's identifier in resolveReferenceLinksTransform and resolveUnlinkedCitations.
• packages/myst-transforms/src/enumerate.spec.ts: regression test with two case-distinct targets; fails without the fix (both resolve to sample.match), passes with it. All myst-transforms tests pass.
• changeset (patch).

Independent CI reproduction

https://github.com/coretl/mystmd-repro-repo runs a full end-to-end reproducer (plugin + project + AST assertion) as a matrix: released 1.10.1 and upstream main asserting the bug, this branch asserting the fix: https://github.com/coretl/mystmd-repro-repo/actions

Context

This is the local-reference sibling of the intersphinx $-anchor lowercasing bug (#1758, #2955), just fixed in continuous-foundation/intersphinx#6 (released as intersphinx 1.1.0). Together they make case-sensitive references correct end-to-end, which matters for Python API docs where re.match (function) and re.Match (class) are different objects; see the discussion in #1259. We (Diamond Light Source / bluesky) hit both while migrating ophyd-async's API docs from Sphinx to MyST.

AI Disclaimer

The code and this description were generated with AI and reviewed/tweaked by the author.

[changeset-bot]

🦋 Changeset detected

Latest commit: bb6f7dd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
myst-transforms	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[choldgraf]

This seems good to me - thanks for adding a simple test for this as well. Given that it brings this behavior in line with another part of the ref transformer, it feels safe to merge to me. IMO this is also closer to what one would expect given that we haven't documented labels as case insensitive

[rowanc1]

@fwkoch can you take a look?

[stefanv]

Our logic is pretty yucky here. Wondering if we should update the getTarget API a bit to do case-insensitive matching + return of the found target label. That way the logic can become:

target, reference.identifier = opts.state.getTarget(identifier);

or similar.

What do you think @fwkoch?

[choldgraf]

I do agree with this as well 🙂 is the author is up for a little clean up I'm happy to iterate via this PR.

I think that we want is case sensitive matching though, right?

[coretl]

I've had a go at folding the normalization into getTarget. Changing the signature of getTarget seemed too big a change for me to make without understanding the codebase better. Is that any cleaner?

[fwkoch]

Thanks for this @coretl! This is a subtle bug I agree we should fix. From my perspective, the point of normalization is two-fold: (1) making non-standard characters in identifier safe for reuse in other contexts and (2) allowing user-friendly coercion for minor mismatches rather than hard failures.

This PR addresses (2) in a nice way - it allows careful authors to set up robust case-sensitive xref/target pairs, but it still falls back to the "user-friendly" normailze-if-there-is-not-a-case-sensitive-match behavior.

However, it opens up (1) to more risk I think - if an author sets up matching xref/target pairs with non-standard characters, it won't be normalized at all and may cause issues in other contexts where these characters break the xrefs.

I'm wondering if your fix should only address case mismatch... That would cover your sample.Match example. (It's also worth pointing out - in other contexts - web, typst, tex - cross-references are all case-sensitive. So supporting case but not special characters could make sense.) If we are going this route, it would probably change the implementation a bit, where normalizeLabel (and probably createHtmlId) no longer includes toLowerCase() and we move the lower fallback to getTarget (so we never check against raw identifier; instead we check against normalized-but-not-lowered value then normalized-and-lowered value).

Does that track and make sense...? (Happy to discuss further if you think I'm missing the mark 😅)

[coretl]

Thanks for the reply @fwkoch. I'll try to answer it, but I work mainly in low level python, so a PR in TypeScript about front end concepts is stretching my knowledge somewhat! I leaned heavily on AI to try and explain the details to me, but I may have missed some important context.

Were you suggesting that prose labels should also become case sensitive as @choldgraf suggested in #2988 (comment)?

If so, then wouldn't that break the html anchors of every existing mystmd site that happened to use uppercase characters in the label text? Is there a use case for having case preserving anchors generated from prose references?

The only reason I ask is that it doesn't actually seem required for this change, as long as the producer assigns a valid identifier, it is used without lowercasing (this PR stops the reference resolver forcing it to lowercase), which fixes the issue for any plugin which wants to make case sensitive references.

More details (AI generated)

Why this PR doesn't need prose labels to change

A preset identifier is already honored verbatim. Every normalizeLabel call in the pipeline derives an identifier from a label (or the :label: option), and that is the only path that lowercases. A node whose identifier is set directly is stored in state.targets verbatim by addTarget, never re-lowercased. So a plugin that assigns a valid exact-case identifier (e.g. sample.Match py:class vs sample.match py:function) gets two distinct targets today.

The only real bug was resolution. resolveReferenceLinksTransform / resolveUnlinkedCitations found the correct case-sensitive target via getTarget, but then re-stamped the resulting crossReference with the normalized (lowercased) identifier, so #sample.Match silently re-pointed at sample.match. This PR keeps the matched target's own identifier. If we agree that "a preset identifier is honored verbatim" is the intended contract, I'm happy to formalize it with a test (two case-distinct targets registering verbatim and keeping their html_id, i.e. the exact fields writeObjectsInv reads) so a future refactor can't regress it.

The export side is already case-preserving. writeObjectsInv keys each std:label row by the target's identifier and locates it by html_id; intersphinx setEntry stores explicit names and anchors verbatim (it only ever lowercased the $ shorthand, which the writer doesn't emit, and that $ case is fixed in the latest intersphinx release for the consume side). So the two objects come out as distinct exact-case rows.

Why making prose labels case sensitive would break existing anchors

Prose identifiers are lowercased at creation, by normalizeLabel and createHtmlId. A heading ## My Section currently produces anchor #my-section. Dropping the toLowerCase() (the choldgraf/fwkoch idea) would make it #My-Section, so:

• Every existing site with uppercase in a heading or label gets different anchor slugs (#my-section -> #My-Section), breaking external deep-links and any hand-written #anchor links.
• createHtmlId's character class would also need widening to [A-Za-z0-9-], or uppercase letters get replaced with -.
• getTarget would need a lowercased fallback index (not just targets[normalized.toLowerCase()]) so old-style #my-section links to ## My Section still resolve once stored keys are mixed-case.
• Duplicate-identifier detection changes: ## Foo and ## foo would stop colliding.

Is there a use case for case-preserving prose anchors?

There's a reasonable one: web, typst, and tex cross-references are all case sensitive, and Python's own docs serve re.Match and re.match as distinct anchors. So it's defensible as a deliberate, separate change. It just isn't required to let a plugin produce case-sensitive references, and it's a breaking change to existing sites, so it seems better kept out of this fix.

[choldgraf]

I just pushed a few small improvements to this branch:

• added a two-pass system so that we first search all pages for exact matches, then do another pass for normalized matches if we don't find an exact match. That way we can prefer exact matches across the whole site, even if a normalized match exists somewhere (that's probably relevant for things like #Classname
• made the function for exact matching more explicit
• added some docstrings and tried to comment a little bit more so it's clearer how the logic works

I feel like this is safe to merge - it seems like we don't want to change the HTML labels since that'd be a bigger user-facing shift, and since HTML is already getting its normalized labels via something other than the identifier, that behavior won't change.