OBSL-1013: ship table discovery in soda-core (soda data-source discover)#2773
Draft
mivds wants to merge 12 commits into
Draft
OBSL-1013: ship table discovery in soda-core (soda data-source discover)#2773mivds wants to merge 12 commits into
mivds wants to merge 12 commits into
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ect DQN via dialect hooks Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… DQNs Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nmatch Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jun 30, 2026
…handler-level test (OBSL-1013) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The handler discovers with unscoped prefixes, which doesn't surface the test table on databricks/sparkdf shared metastores. The test verifies the handler's connection open/close lifecycle, which is dialect-independent, so one dialect suffices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First-party table discovery in soda-core (OBSL-1013, part of the v3→v4 observability port OBSL-1003). Adds a
soda data-source discovercommand that posts a DQN-onlysodaCoreInsertScanResultspayload (version: "4") for the BE's v4DiscoveryScanHandlerModule.DatasetIdentifier.from_object(...)— builds a dialect-correct DQN from a discoveredFullyQualifiedObjectNameusing the dialect'sget_database_prefix_index()/get_schema_prefix_index()hooks (verified across all dialects: postgres[db,schema], duckdb[schema](catalog dropped), BigQuery[project,dataset], oracle/db2/sparkdf[schema]). No hand-rolled prefixes.DiscoveryRun.execute(...)— discovers within a scope, filters__soda_temp+ include/exclude (pushed down toMetadataTablesQuery's server-side SQLLIKEfilters), maps each object to its DQN.discovery_payload— DQN-only v4 envelope, posted viaSodaCloud._execute_command.soda data-source discoverCLI command +handle_discover_data_sourcehandler (ExitCodesemantics).Design/plan:
docs/plans/2026-06-30-obsl-1013-discovery-implementation-plan.md.Notes for reviewers
discover_v4_data_source) and the OBSL-1026 golden.prefixes=[](discover all visible); per-dialect scope refinement (e.g. excluding postgres system schemas, BigQuery project/dataset scoping) is a documented follow-up.Test Plan
from_objectacross dialect shapes;DiscoveryRunfiltering/mapping; payload shape; CLI arg mapping (7 tests)soda-tests/tests/integration/test_discovery.pyagainst postgres viaMockSodaCloud— DQN-only metadata,version == "4", test table's DQN presenttest_cli.py(21) pass; pre-commit (isort/black/autoflake) cleanDiscoveryScanHandlerModule(parity-time)🤖 Generated with Claude Code