Skip to content

Explicit joins in Relation.join()#3868

Open
burnash wants to merge 28 commits into
develfrom
feat/3747-explicit-joins
Open

Explicit joins in Relation.join()#3868
burnash wants to merge 28 commits into
develfrom
feat/3747-explicit-joins

Conversation

@burnash
Copy link
Copy Markdown
Collaborator

@burnash burnash commented Apr 15, 2026

Closes #3747

@burnash burnash changed the base branch from feat/3403-relation-join to devel April 15, 2026 12:55
@burnash burnash changed the base branch from devel to feat/3403-relation-join April 15, 2026 12:55
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 15, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
docs 5f01e4e Commit Preview URL

Branch Preview URL
Jun 02 2026, 04:10 PM

@burnash burnash changed the base branch from feat/3403-relation-join to devel April 15, 2026 15:41
Comment thread dlt/destinations/sql_client.py
Comment thread dlt/dataset/relation.py Outdated
f"'{target_dataset.dataset_name}' vs '{self._dataset.dataset_name}'"
)
# cross-dataset filesystem not supported
if isinstance(self.sql_client, WithSchemas):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a good followup ticket. if we use duckdb ATTACH we will be able to join dataset on separate physical locations. ie. joining lance and HF tables will be possible

Comment thread dlt/dataset/relation.py Outdated

# physical destination check
if target_dataset is not self._dataset:
if not self._dataset.is_same_physical_destination(target_dataset):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

our physical destination check is currently half implemented:
#3758

currently FYI but we can add this to our estimations

Comment thread dlt/dataset/relation.py Outdated
@burnash burnash force-pushed the feat/3747-explicit-joins branch from 0c6c278 to 390d40a Compare April 16, 2026 11:38
@blacksmith-sh

This comment has been minimized.

@burnash burnash force-pushed the feat/3747-explicit-joins branch from 38633e2 to 4aff685 Compare May 27, 2026 09:01
@burnash burnash force-pushed the feat/3747-explicit-joins branch from 4aff685 to 07baafa Compare May 27, 2026 12:39
@blacksmith-sh

This comment has been minimized.

@blacksmith-sh

This comment has been minimized.

@burnash burnash force-pushed the feat/3747-explicit-joins branch from 1c86bdf to d4378f7 Compare May 28, 2026 13:57
@blacksmith-sh

This comment has been minimized.

@burnash burnash changed the title [WIP] Explicit joins in Relation.join() Explicit joins in Relation.join() Jun 2, 2026
@burnash burnash added the enhancement New feature or request label Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(feat) implement explicit joins with local and foreign datasets

2 participants