chore(deps): pin docling-ibm-models compound list-marker branch#3849
chore(deps): pin docling-ibm-models compound list-marker branch#3849DanielNg0729 wants to merge 2 commits into
Conversation
Pin docling-ibm-models to the branch that adds compound/hierarchical list markers (9a., 3.a., 1.2.1, 5.1) to ListItemMarkerProcessor, via [tool.uv.sources]. Only the docling-ibm-models source changes in uv.lock. Follow-up to the docling-ibm-models change; temporary git pin until it is released, then it becomes a normal version bump. Affected reference data (2203.01017v2, wiki_duck.html) to be regenerated on CI. Signed-off-by: Daniel Nguyen <danielnguyenh07@gmail.com>
|
✅ DCO Check Passed Thanks @DanielNg0729, all your commits are properly signed off. 🎉 |
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@DanielNg0729 I see your concerns with reproduction of ground-truth data on Windows. I will try to run your fork soon on a well supported OS. |
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
|
@DanielNg0729 updated the test GT in this PR. Please ignore the noise:
What is left is only one actual change in Conclusion: The changes on docling-ibm-models seem to do their job and no regressions are apparent. |
Depends on / follow-up to docling-project/docling-ibm-models#169 (and supersedes the
docling-side approach in #3815).
What this does
Pins
docling-ibm-modelsto the branch that adds compound / hierarchical list-marker patterns(
9a.,3.a.,1.2.1,5.1…) toListItemMarkerProcessor, via[tool.uv.sources]:uv.lockupdates accordingly (only thedocling-ibm-modelssource changes; no other dependencymoves). This is a temporary pin until the ibm-models change is released, after which it becomes a
normal version bump.
Ground-truth impact (measured)
The marker processor runs on every backend via
document_converter, so I measured the exacteffect by running the patched processor over the committed ground-truth documents (released vs.
patched processor, drift-free). It changes 2 documents / 15 list items — all previously had an
empty
markerwith the marker fused into the text:tests/data/pdf/groundtruth/2203.01017v2(5 items) — the algorithm list:markermarker""3.a.""9a.""9b.""9c.""9d.tests/data/html/groundtruth/wiki_duck.html(10 items) — Wikipedia section entries:5.1 Feeding,5.2 Breeding,5.3 Communication,5.4 Predators,6.1 Hunting,6.2 Domestication,6.3 Heraldry,6.4 Cultural references,8.1 Citations,8.2 Sources—each moves the dotted-decimal number from the text into
marker, withenumerated=True.In every case
enumeratedflipsFalse → Trueand the marker is removed from the item text.Reference-data regeneration
The reference
.json/.md/.doctagsfor those two documents need regenerating(
DOCLING_GEN_TEST_DATA=1). This was not done in this PR because it must run on a Linux/CIenvironment to match the committed ground truth. A local (Windows) regen is unusable: it rewrites
~117 files that are dominated by environment artifacts unrelated to this change — CRLF line
endings,
/→\path separators in HTML links, and a"orientation": "rot_0"field added by thecurrent docling-core — and the large PDF (
2203.01017v2) runs out of memory before it can beregenerated locally. Please regenerate on CI (or advise and I'll do it in a Linux environment).
Checklist: