Skip to content

feat(meta): add DataPointMention for quantitative data points#679

Open
ceberam wants to merge 1 commit into
mainfrom
dev/entity-data-point
Open

feat(meta): add DataPointMention for quantitative data points#679
ceberam wants to merge 1 commit into
mainfrom
dev/entity-data-point

Conversation

@ceberam

@ceberam ceberam commented Jul 8, 2026

Copy link
Copy Markdown
Member

Summary

Extends the document metadata model with first-class support for quantitative
data point mentions — values such as "$4B", "3.0%", or "between 10 and 20 °C" that appear in running text and carry machine-readable numeric meaning.

What's new

DataPointMention

A new subclass of EntityMention that adds a structured numeric breakdown
alongside the inherited surface-form fields (text, orig, label,
charspan):

Field Type Description
value float Numeric value as written, before scale application
unit str Dimensional unit ("USD", "°C", "%")
scale DataPointScale Magnitude multiplier ("billion", "k", …)
normalized_value float value * scale_factor in base units
range_end float Upper bound for range expressions
display_dp int Decimal places as written ("3%" → 0, "3.0%" → 1)
precision DataPointPrecision Epistemic qualifier ("exact", "approximate", "range_low", …)
direction DataPointDirection Change direction for delta values ("increase", "decrease", "neutral")

Two helper members are provided:

  • scale_factor — property returning the numeric multiplier for the stored scale value
  • compute_normalized_value() — derives value * scale_factor on demand, useful when fields are set after construction

Public Literal type aliases

DataPointScale, DataPointPrecision, and DataPointDirection are exported
as public Literal type aliases. They serve as the field annotations (giving
Pydantic automatic validation and IDE completion) and as a documented API
surface consumers can inspect at runtime via typing.get_args(...).

Integration with EntitiesMetaField

DataPointMention objects are stored in the existing entities metadata
field — no new top-level field is introduced. EntitiesMetaField.mentions
is widened to list[Union[DataPointMention, EntityMention]] so that plain
named entities and data points can coexist in the same list and survive
JSON roundtrips with full type fidelity.

Design notes

  • DataPointMention is a non-breaking extension of EntityMention: all
    numeric fields are optional, so existing entity mentions are unaffected.
  • The scale field is a closed Literal matching the keys of the internal
    scale_factor map — storing an unrecognised scale string is rejected at
    construction time rather than silently making compute_normalized_value()
    return None.
  • display_dp and precision are orthogonal: the former captures how many
    decimal places the author wrote, the latter captures their epistemic hedge
    ("roughly 3%"precision="approximate", display_dp=0).

@ceberam
ceberam requested a review from PeterStaar-IBM July 8, 2026 15:22
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

@ceberam
ceberam requested a review from vagenas July 8, 2026 15:22
@mergify

mergify Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require two reviewer for test updates 👀 reviews
🟢 Enforce conventional commit

🔴 Require two reviewer for test updates

Waiting for

  • #approved-reviews-by >= 2
This rule is failing.

When test data is updated, we require two reviewers

  • #approved-reviews-by >= 2

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.22222% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling_core/types/doc/common/meta.py 97.22% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ed in text

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
@ceberam
ceberam force-pushed the dev/entity-data-point branch from 6087c26 to 09aee82 Compare July 9, 2026 13:50
"""Valid values for DataPointMention.scale. Each value maps to a known numeric multiplier."""


class DataPointMention(EntityMention):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if everything is optional, can we potentially have completely empty fields?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants