fix(serializer): serialize table footnotes in Markdown and HTML#675
Open
Kavishkagaya wants to merge 1 commit into
Open
fix(serializer): serialize table footnotes in Markdown and HTML#675Kavishkagaya wants to merge 1 commit into
Kavishkagaya wants to merge 1 commit into
Conversation
…ing-project#496) Since footnote handling was delegated to the parent FloatingItem, table footnotes were silently dropped from Markdown and HTML output (and from chunking, which relies on serialization). The `serialize_footnotes` hook existed on the doc serializer but was never invoked by the table serializers, so the footnotes stored on `TableItem.footnotes` never reached the output. - MarkdownTableSerializer: emit the table's footnotes after the table body, mirroring the existing caption handling. - HTMLTableSerializer: emit the footnotes after the closing </table>. - HTMLDocSerializer: add a serialize_footnotes override so footnotes are HTML-escaped and wrapped in <div class="footnote"> (the base method returns raw text, matching the existing serialize_captions override). - Add a regression test asserting table footnotes survive both exports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Kavishka Rambukwella <90164480+Kavishkagaya@users.noreply.github.com>
Contributor
|
✅ DCO Check Passed Thanks @Kavishkagaya, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Author
|
Aware of the related #545 and #569 targeting the same issue — opening this as a smaller, focused alternative:
Happy to fold this into either of the other PRs if the maintainers prefer to consolidate. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #496
Problem
Table footnotes are silently dropped from Markdown and HTML output. The footnotes are correctly stored on
TableItem.footnotesin the document model (they appear in the JSON), but never make it into the serialized text — so a table renders with dangling*/**markers whose definitions have vanished. As #496 notes, this also affects chunking, which relies on serialization.Root cause
After footnote handling was delegated to the parent
FloatingItem, theserialize_footnoteshook was added to the doc serializer — but the table serializers were never wired to call it. They callserialize_captions(so captions appear) but notserialize_footnotes, so the method was effectively dead code and the footnotes had nowhere to go.Fix
MarkdownTableSerializer— emit the table's footnotes after the table body, mirroring the existing caption handling.HTMLTableSerializer— emit the footnotes after the closing</table>.HTMLDocSerializer— add aserialize_footnotesoverride so footnotes are HTML-escaped and wrapped in<div class="footnote">, matching the existingserialize_captionsoverride (the base method returns raw text, which is unsafe for HTML).Evidence
Using the
table_parsing_test.pdfattached to #496 (digital PDF, standard pipeline). The table has two footnotes assigned; before the fix both are missing from the Markdown, after the fix both are present:Before (
export_to_markdown()):After:
HTML output, after the fix, now emits (escaped):
Testing
test_table_footnotes_serialized_to_markdown_and_html) that builds an in-memoryDoclingDocumentwith a table + two footnotes and asserts they survive bothexport_to_markdown()andexport_to_html().test_serialization.py,test_serialization_doctag.py,test_latex_serialization.py,test_azure_serializer.py) and chunking (test_hierarchical_chunker.py).ruff checkandruff format --checkclean.