Skip to content

Fixes #27443: add OCI Autonomous Database support for Oracle connector#27508

Open
hassaansaleem28 wants to merge 46 commits intoopen-metadata:mainfrom
hassaansaleem28:issue-27443-oci-autonomous
Open

Fixes #27443: add OCI Autonomous Database support for Oracle connector#27508
hassaansaleem28 wants to merge 46 commits intoopen-metadata:mainfrom
hassaansaleem28:issue-27443-oci-autonomous

Conversation

@hassaansaleem28
Copy link
Copy Markdown
Contributor

@hassaansaleem28 hassaansaleem28 commented Apr 18, 2026

Describe your changes:

Fixes #27443

I worked on OCI Autonomous Oracle connector support because the existing Oracle flow required manual host-level wallet and TNS setup, which prevented a complete UI-driven configuration experience.

Screencast.from.2026-04-20.20-20-59.webm

Why migration scripts are not needed:

This change updates connector configuration schema and ingestion runtime behavior only.
It does not introduce database table/entity schema changes in OpenMetadata persistence layers.

Type of change:

  • New feature

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes #27443: add OCI Autonomous Database support for Oracle connector
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
  • The issue properly describes why the new feature is needed, what's the goal, and how we are building it. Any discussion
    or decision-making process is reflected in the issue.
  • I have updated the documentation.
  • I have added tests around the new logic.

Summary by Gitar

  • Oracle connector maintenance:
    • Reverted type hint for _wallet_temp_dir from Optional[str] back to str | None to comply with ruff UP045.
  • Configuration updates:
    • Updated createIngestionPipeline.ts, ingestionPipeline.ts, and workflow.ts to support new OCI Autonomous Database connector parameters.
  • Test infrastructure:
    • Added new configuration schemas and definitions to testSuitePipeline.ts for automated ingestion workflow validation.

This will update automatically on new commits.

…ctor

Signed-off-by: Muhammad Hassaan Saleem <iamhassaans@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Signed-off-by: Muhammad Hassaan Saleem <iamhassaans@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@hassaansaleem28 hassaansaleem28 marked this pull request as ready for review April 18, 2026 08:19
@hassaansaleem28 hassaansaleem28 requested review from a team as code owners April 18, 2026 08:19
Copilot AI review requested due to automatic review settings April 18, 2026 08:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Oracle Autonomous Database (OCI) support to the Oracle connector to enable a wallet-based, UI-driven configuration path (no host-level TNS/WALLET provisioning required), spanning UI docs, connector schema, ingestion runtime, and unit tests.

Changes:

  • Extend Oracle connection JSON schema with an OracleAutonomousConnection option (tnsAlias + walletPath/walletContent + optional walletPassword).
  • Update Oracle ingestion connection logic to accept wallet-based Autonomous config, including safe zip extraction and temp-dir lifecycle handling.
  • Add unit tests for Autonomous URL building, connection-argument setup, walletContent extraction reuse, and Zip Slip rejection.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
openmetadata-ui/src/main/resources/ui/public/locales/en-US/Database/Oracle.md Documents the new Autonomous connection mode and wallet fields.
openmetadata-spec/src/main/resources/json/schema/entity/services/connections/database/oracleConnection.json Adds OracleAutonomousConnection to the connector config schema and updates the connection type description.
ingestion/src/metadata/ingestion/source/database/oracle/connection.py Implements wallet-based Autonomous connection handling (args wiring + wallet extraction + reuse/cleanup).
ingestion/tests/unit/test_source_connection.py Adds unit tests covering Autonomous URL/args behavior and wallet zip safety checks.

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
…tion.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 18, 2026 08:26
…tion.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
…tion.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 19, 2026 01:30
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Signed-off-by: hassaansaleem28 <iamhassaans@gmail.com>
Copilot AI review requested due to automatic review settings April 19, 2026 01:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated 2 comments.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

hassaansaleem28 and others added 2 commits May 4, 2026 11:46
A recent automated suggestion replaced `str | None` with `Optional[str]`
on the wallet temp-dir attribute. Ruff's UP007/UP045 rule (enabled in
pyproject.toml without an `ignore`) treats Optional as legacy syntax
and fails py-checkstyle on this file. Switch back to the union form
to satisfy CI; behaviour is identical at runtime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated 2 comments.

Comment thread ingestion/tests/unit/test_source_connection.py
Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated 2 comments.

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated 1 comment.

Comment thread ingestion/src/metadata/ingestion/source/database/oracle/connection.py Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated no new comments.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

hassaansaleem28 and others added 2 commits May 6, 2026 12:58
The Optional[str] form re-appeared on this attribute and again tripped
ruff's UP045 (use-pep604-annotation) rule, which is enabled in
ingestion/pyproject.toml without an ignore. Other Optional uses in the
file are on the Pydantic-touching test_connection signature and carry
explicit `# noqa: UP045` markers; this attribute has no such reason
to keep the legacy form, so use the union syntax.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated no new comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated no new comments.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 15 changed files in this pull request and generated no new comments.

@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented May 7, 2026

Code Review ✅ Approved 7 resolved / 7 findings

Adds OCI Autonomous Database support to the Oracle connector, enabling UI-driven configuration. Resolved multiple security and logic issues including zip-slip vulnerabilities, unreliable temp directory cleanup, and insecure directory creation.

✅ 7 resolved
Security: Zip extraction vulnerable to zip-slip path traversal

📄 ingestion/src/metadata/ingestion/source/database/oracle/connection.py:109-118
The _extract_wallet_content method calls zip_ref.extractall(self._wallet_temp_dir) without validating that extracted file paths stay within the target directory. A malicious base64-encoded wallet zip could contain entries like ../../etc/crontab that write outside the temp directory (zip-slip attack). Since walletContent is user-supplied input from the UI, this is a realistic attack vector.

Additionally, there is no size limit on the decoded content, so a zip bomb could exhaust disk space.

Edge Case: Temp directory cleanup relies on __del__, which is unreliable

📄 ingestion/src/metadata/ingestion/source/database/oracle/connection.py:79-80 📄 ingestion/src/metadata/ingestion/source/database/oracle/connection.py:106-118
__del__ is not guaranteed to be called promptly (or at all) by CPython's garbage collector, especially in the presence of reference cycles or interpreter shutdown. If _get_client raises an exception after the wallet is extracted, the temp directory may linger on disk indefinitely.

Since BaseConnection doesn't use a context-manager pattern, consider adding explicit cleanup in exception paths or using atexit.register as a safety net.

Bug: Repeated _get_client calls duplicate wallet connection args

📄 ingestion/src/metadata/ingestion/source/database/oracle/connection.py:159
_configure_autonomous_connection_arguments is called at the start of every _get_client() invocation. If _get_client is called more than once (e.g., on reconnect), and walletContent is provided, a new temp directory is created each time while the old one is cleaned up—but the connectionArguments are mutated on service_connection every time. This is likely harmless but wasteful; more importantly, if walletPath (not walletContent) is used, the arguments are appended redundantly on each call.

Bug: list_all_entities for Table missing fields=["columns"]

📄 ingestion/src/metadata/ingestion/source/database/trino/lineage.py:181-188 📄 ingestion/src/metadata/ingestion/source/database/trino/lineage.py:121-128
The new _get_case_insensitive_cross_database_table method (line 181) calls self.metadata.list_all_entities(entity=Table, params={...}) without specifying fields=["columns"]. The OpenMetadata API does not return columns by default, so table.columns will be None/empty.

This means check_same_table (lines 121-124) will always hit the not table1.columns and not table2.columns early return and match purely by table name, making the column comparison logic dead code. While schema-scoped name matching reduces false positives, the column validation was clearly intended as an extra safety check and won't function without requesting the columns field.

Bug: Duplicate host assignment; second line accesses wrong model field

📄 ingestion/src/metadata/ingestion/source/database/oracle/connection.py:320-321
In the OracleAutonomousConnection branch of get_connection_dict, line 321 immediately overwrites line 320 and accesses oracleTNSConnection — a field that does not exist on OracleAutonomousConnection (it belongs to OracleTNSConnection). This will raise an AttributeError at runtime whenever get_connection_dict() is called for an autonomous connection.

Line 320 (autonomous_connection.tnsAlias) was the correct value. Line 321 appears to be an accidental leftover from a copy-paste or merge.

...and 2 more resolved from earlier reviews

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 7, 2026

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 7, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OCI Autonomous Database Support

4 participants