Skip to content

osi converter: preserve N dimensions over one column across the round-trip (#222)#229

Merged
ralfbecher merged 3 commits into
mainfrom
fix/osi-n-dims-per-column
Jul 16, 2026
Merged

osi converter: preserve N dimensions over one column across the round-trip (#222)#229
ralfbecher merged 3 commits into
mainfrom
fix/osi-n-dims-per-column

Conversation

@ralfbecher

Copy link
Copy Markdown
Collaborator

Closes #222. Builds on #220's extension hook.

Problem

OBML allows N dimensions over one column (grain variants, role-playing via via); OSI is field-centric (one dimension per field). The converter used to break after the first matching dimension on export and build one dimension per field on import, so a second dimension on the same column was dropped silently - both names lost, zero warnings on either side.

Repro (from the issue):

dimensions:
  Order Day:   {dataObject: Orders, column: date, resultType: date, timeGrain: day}
  Order Month: {dataObject: Orders, column: date, resultType: date, timeGrain: month}

OBML -> OSI -> OBML used to yield a single order_date dimension.

Fix

  • Export (obml_to_osi): the first matching dimension stays the primary carried on the field; any extras are serialized into an obml_extra_dimensions list extension (name + resultType + timeGrain + format + description + owner + via), and a fidelity warning is emitted (warn, do not raise - conversion still succeeds).
  • Import (osi_to_obml): rebuild each preserved descriptor as its own dimension. The collision-key logic is factored into _insert_dimension and reused for the primary and every extra.

Now the same repro round-trips to all three dimensions with their distinct grains, and the export warns rather than silently dropping.

Robustness

Descriptors are opaque foreign-modifiable data (validate_osi does not inspect extension payloads), so each is shape-guarded - must be a dict with a non-empty string name, string props only. A malformed payload (not a list, non-dict items, non-string names) is skipped, never crashes on an unhashable key. This is the same class of guard #228 added for obml_dimension_name, applied here from the start.

Tests

TestMultipleDimensionsPerColumn: all dimensions survive with distinct grains and the same physical column; export warns (not silent); parametrized malformed-payload cases never crash.

177 package + convert-integration tests pass; ruff clean; no new mypy errors (package's pre-existing typing debt unchanged, outside CI's mypy src/ scope).

Upstream note

Whether OSI should natively represent multiple dimensions over one field (cf. LookML dimension_group, Cube granularities) remains an Ossie spec question, tracked in the issue. This PR is the converter-side warn + preserve, independent of that outcome.

Refs #201, #202, #220.

…-trip (#222)

OBML allows N dimensions over one column (grain variants, role-playing via
`via`); OSI is field-centric (one dimension per field). The converter used to
`break` after the first matching dimension on export and build one dimension per
field on import, so a second dimension on the same column was dropped silently -
both names lost, zero warnings.

- Export (obml_to_osi): the first matching dimension stays the primary carried
  on the field; any extras are serialized into an `obml_extra_dimensions` list
  extension (name + resultType + timeGrain + format + description + owner + via),
  and a fidelity warning is emitted (warn, do not raise).
- Import (osi_to_obml): rebuild each preserved descriptor as its own dimension.
  The collision-key logic is factored into `_insert_dimension` and reused for
  the primary and every extra. Descriptors are opaque foreign-modifiable data,
  so each is shape-guarded (must be a dict with a non-empty string name; string
  props only) - a malformed payload is skipped, never crashes on an unhashable
  key.

Tests: TestMultipleDimensionsPerColumn - all dimensions survive with distinct
grains, export warns (not silent), and malformed extra-dimension payloads never
crash.

Closes #222. Refs #201, #202, #220.
The obml_extra_dimensions descriptor only carried the scalar props (resultType,
timeGrain, format, description, owner, via), so a non-primary dimension's own
synonyms and vendor customExtensions were dropped on the round-trip.

Include both in the descriptor on export and restore them on import, with the
same shape guards as the rest of the opaque extension data: synonyms kept only
as non-empty strings, customExtensions only as dict entries. Malformed values
are filtered, never crash.

Tests: extra dimension's synonyms + customExtensions survive the round-trip;
malformed synonyms/customExtensions payloads are filtered without crashing.

Refs #222.
The primary (single) dimension on a column lost its own synonyms entirely and
its customExtensions moved to the column (OSI has no dimension entity, so they
surfaced as field foreign extensions and re-imported onto the column). Now the
extras preserved these but the primary did not - inconsistent.

Stash the primary dimension's synonyms and customExtensions in authoritative
obml_dimension_synonyms / obml_dimension_custom_extensions extension keys and
restore them to the dimension on import, with the same shape guards (string
synonyms, dict extensions). The existing foreign-extension emission stays, so
the dimension's vendor extensions remain visible to other OSI tools on the
field; that column-side surfacing is pre-existing and unchanged.

Test: the primary dimension's synonyms + customExtensions survive the round
trip. Full package suite green (the field-foreign-extension export test still
passes).

Refs #222.
@ralfbecher
ralfbecher merged commit 7222403 into main Jul 16, 2026
6 checks passed
@ralfbecher
ralfbecher deleted the fix/osi-n-dims-per-column branch July 16, 2026 17:27
ralfbecher pushed a commit to ralfbecher/OSI that referenced this pull request Jul 17, 2026
Mirrors two further canonical converter fixes on top of the round-trip
robustness work already in this PR:

- ralforion/orionbelt-semantic-layer#228: restore the OBML dimension name
  across the round trip. The OSI field name is the physical column code, so a
  round trip renamed every dimension to its code and could trip the collision
  fallback for names unique in the source. The name is now preserved in an
  obml_dimension_name extension and restored on import, making the collision
  fallback genuinely foreign-OSI only. Non-string values are ignored so a
  foreign payload cannot crash the converter on an unhashable key.

- ralforion/orionbelt-semantic-layer#229: preserve N dimensions over one
  column. OBML allows many dimensions on one column (grain variants, role
  playing); OSI is field-centric. Extras are now preserved in an
  obml_extra_dimensions list extension with a fidelity warning (warn, not
  raise) and rebuilt on import. Both primary and extra dimensions carry their
  own synonyms and vendor extensions, with shape guards on the opaque
  extension data.

177 converter tests pass; ruff clean.
ralfbecher added a commit that referenced this pull request Jul 19, 2026
Patch release: converter round-trip fixes (#228, #229), convert-boundary input validation (#226), and the OBML label removal/rename (#221, #227).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

osi converter: silent fidelity loss when multiple OBML dimensions map to one column

1 participant