Skip to content

Streaming engine occasionally ignores maintain_order on right/full joins #29362

Description

@kdn36

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

a = pl.LazyFrame({"ak": [1]})
b = pl.LazyFrame({"bk": [1, 2, 3, 4, 5], "bv": [10, 20, 30, 40, 50]})

q = a.join(b, left_on="ak", right_on="bk", how="right", maintain_order="left_right")

print(q.collect(engine="in-memory"))  # ok
print(q.collect(engine="streaming"))  # sometimes not ok

run as

POLARS_MAX_THREADS=2 python tt.py

Log output

shape: (5, 2)
┌─────┬─────┐
│ bk  ┆ bv  │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 10  │
│ 2   ┆ 20  │
│ 3   ┆ 30  │
│ 4   ┆ 40  │
│ 5   ┆ 50  │
└─────┴─────┘
shape: (5, 2)
┌─────┬─────┐
│ bk  ┆ bv  │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 10  │
│ 3   ┆ 30  │
│ 2   ┆ 20  │
│ 5   ┆ 50  │
│ 4   ┆ 40  │
└─────┴─────┘

Issue description

maintain_order not respected in streaming engine

Expected behavior

results match

Installed versions

Reproduces on 1.44.2 and latest main

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-streamingRelated to the streaming enginebugSomething isn't workingneeds triageAwaiting prioritization by a maintainerpythonRelated to Python Polars

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions