Skip to content

{source-materialize}-postgres: support JSONB#4400

Open
mdibaiee wants to merge 3 commits into
mainfrom
mahdi/postgres-jsonb
Open

{source-materialize}-postgres: support JSONB#4400
mdibaiee wants to merge 3 commits into
mainfrom
mahdi/postgres-jsonb

Conversation

@mdibaiee
Copy link
Copy Markdown
Member

@mdibaiee mdibaiee commented May 11, 2026

Description:

  • Use a custom contentMediaType: "application/vnd.postgresql.jsonb+json" annotation in JSONSchema to mark an object as originating from a postgresql JSONB column type
  • In materializations, use JSONB for creating such a column
  • Add automatic column migrations from JSON to JSONB and vice versa

I have identified customers that would be affected by this and I am coordinating with our support team to make sure we can transition them to this new change without breaking their pipelines down the line.

Workflow steps:

(How does one use this feature, and how has it changed)

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

(anything that might help someone review this PR)

@mdibaiee mdibaiee force-pushed the mahdi/postgres-jsonb branch from cdb342e to cad0b04 Compare May 11, 2026 15:53
@mdibaiee mdibaiee requested review from a team, Alex-Bair and willdonnelly May 11, 2026 16:41
@mdibaiee mdibaiee linked an issue May 12, 2026 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@danielnelson danielnelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both json and jsonb are encoded in the collection in the same way, so it doesn't seem like the media type is describing anything about the actual format of the data. It also see that contentMediaType is described only for strings.

I feel like we potentially could need to invent media types for every possible source type? I'm also imagining things like a request for mirroring aspects such as string collation. We do already have the information in the description and while it's not super machine readable we could try parsing.

Or perhaps we would be better served with a new custom annotation for source type?

Comment thread source-postgres/.snapshots/TestDiscoveryComplex Outdated
Comment thread materialize-postgres/sqlgen.go Outdated
@mdibaiee
Copy link
Copy Markdown
Member Author

@danielnelson that was a great point I had overlooked. I had a chat with data plane folks and they were happy to have contentMediaType for all types, so I'm working on a PR for that estuary/flow#2945

@mdibaiee mdibaiee force-pushed the mahdi/postgres-jsonb branch 5 times, most recently from 441d34b to 2be7ce9 Compare May 21, 2026 13:02
@mdibaiee mdibaiee force-pushed the mahdi/postgres-jsonb branch from 2be7ce9 to f5ed74a Compare May 21, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSONB mapping from source-postgres to materialize-postgres

3 participants