Skip to content

feat(parquet): Support nested 2D lists#864

Open
gregfurman wants to merge 1 commit into
warpstreamlabs:mainfrom
gregfurman:add/parquet/list-of-list
Open

feat(parquet): Support nested 2D lists#864
gregfurman wants to merge 1 commit into
warpstreamlabs:mainfrom
gregfurman:add/parquet/list-of-list

Conversation

@gregfurman

Copy link
Copy Markdown
Collaborator

Motivation

Adds support for nested 2D lists in the parquet processors.

Fixes #360

Changes

  • Updates parquet-go to v0.28.0, which includes support for 2D nested types.
  • Ensures nested schema structs are created with the new parquet-element tag.
  • Unskips TestParquetDecodeListFormatEdgeCases as support has been added.

@gregfurman
gregfurman requested a review from jem-davies as a code owner May 24, 2026 14:27
return baseType, nil
}

func isListOfLists(field *service.ParsedConfig) bool {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would we need to document that we can't handle more than 2D lists?
Would it be possible to be able to handle 3D+ lists?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Support nested list attributes in parquet encoder

2 participants