MINOR: add more logs in pbi for lineage

[harshsoni2024]

Describe your changes:

I worked on adding more logs on lineage process to debug

Type of change:

• Bug fix
• Improvement
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

High-level design:

N/A — small change.

Tests:

Use cases covered

Unit tests

Backend integration tests

Ingestion integration tests

Playwright (UI) tests

Manual testing performed

UI screen recording / screenshots:

Not applicable.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• My PR is linked to a GitHub issue via Fixes #<issue-number> above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
• For UI changes: I attached a screen recording and/or screenshots above.
• I have added tests (unit / integration / Playwright as applicable) and listed them above.

---

Summary by Gitar

• Logging improvements:
  • Added comprehensive debug logs throughout PowerbiSource to trace lineage creation processes for dashboards, reports, and datamodels.
  • Included logs for missing entity checks and data extraction states to facilitate easier debugging.
• Code refinements:
  • Applied # noqa: RUF010 to various debug log statements to maintain linting compliance.
  • Corrected type hint for _fetch_report_from_workspace return type from Dataset to PowerBIReport.

_{This will update automatically on new commits.}

[Copilot]

Pull request overview

This PR adds additional debug-level logging around PowerBI lineage creation to make it easier to troubleshoot missing entities (dashboard/report/datamodel) and missing dataset IDs during lineage processing.

Changes:

• Added granular logger.debug statements in report↔dashboard lineage creation (including per-tile report linkage and missing-entity cases).
• Added debug logs in datamodel↔report lineage creation, including datasetId presence vs. datasource API extraction path.
• Corrected _fetch_report_from_workspace return type annotation to Optional[PowerBIReport].

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

ingestion/src/metadata/ingestion/source/dashboard/powerbi/metadata.py:765

• The return type annotation is inconsistent with the yielded values. This method yields _get_add_lineage_request(...), which produces Either[AddLineageRequest] (or None), not Either[CreateDashboardRequest]. Please update the return type to Iterable[Either[AddLineageRequest]] (and ensure you never yield None).

    def create_datamodel_report_lineage(
        self,
        db_service_prefix: Optional[str],  # noqa: UP045
        dashboard_details: PowerBIReport,
    ) -> Iterable[Either[CreateDashboardRequest]]:
        """

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

🔴 Playwright Results — 4 failure(s), 20 flaky

✅ 4004 passed · ❌ 4 failed · 🟡 20 flaky · ⏭️ 86 skipped

Shard	Passed	Failed	Flaky	Skipped
🟡 Shard 1	297	0	2	4
🟡 Shard 2	749	0	6	8
🔴 Shard 3	750	2	7	7
🔴 Shard 4	788	2	0	18
🟡 Shard 5	686	0	1	41
🟡 Shard 6	734	0	4	8

Genuine Failures (failed on all attempts)

❌ Features/Tasks/TaskNavigation.spec.ts › clicking task notification while on entity task tab refreshes the task list (shard 3)

Error: �[2mexpect(�[22m�[31mlocator�[39m�[2m).�[22mtoBeVisible�[2m(�[22m�[2m)�[22m failed

Locator: locator('.notification-box').locator('li.ant-list-item.notification-dropdown-list-btn').first()
Expected: visible
Timeout: 15000ms
Error: element(s) not found

Call log:
�[2m  - Expect "toBeVisible" with timeout 15000ms�[22m
�[2m  - waiting for locator('.notification-box').locator('li.ant-list-item.notification-dropdown-list-btn').first()�[22m

❌ Features/Tasks/TaskNavigation.spec.ts › two sessions: admin on Columns tab creates task, assignee sees refresh on notification click (shard 3)

Error: �[2mexpect(�[22m�[31mlocator�[39m�[2m).�[22mtoBeVisible�[2m(�[22m�[2m)�[22m failed

Locator: locator('.notification-box').locator('li.ant-list-item.notification-dropdown-list-btn').first()
Expected: visible
Timeout: 15000ms
Error: element(s) not found

Call log:
�[2m  - Expect "toBeVisible" with timeout 15000ms�[22m
�[2m  - waiting for locator('.notification-box').locator('li.ant-list-item.notification-dropdown-list-btn').first()�[22m

❌ Pages/DataContracts.spec.ts › Contract Status badge should be visible on condition if Contract Tab is present/hidden by Persona (shard 4)

TimeoutError: page.reload: Timeout 60000ms exceeded.
Call log:
�[2m  - waiting for navigation until "load"�[22m
�[2m    - navigated to "http://localhost:8585/topic/pw-messaging-service-a1b1ecbc.pw-topic-entity-class-63392930/contract"�[22m

❌ Pages/DataContracts.spec.ts › Contract Status badge should be visible on condition if Contract Tab is present/hidden by Persona (shard 4)

�[31m"beforeAll" hook timeout of 60000ms exceeded.�[39m

🟡 20 flaky test(s) (passed on retry)

• Pages/AuditLogs.spec.ts › should apply both User and EntityType filters simultaneously (shard 1, 1 retry)
• Flow/SearchRBAC.spec.ts › User without permission (shard 1, 1 retry)
• Features/ActivityAPI.spec.ts › Activity event shows the actor who made the change (shard 2, 1 retry)
• Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
• Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts › Admin: Complete export-import-validate flow (shard 2, 1 retry)
• Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts › EditAll User: Complete export-import-validate flow (shard 2, 1 retry)
• Features/DataQuality/TestCaseResultPermissions.spec.ts › User with TEST_CASE.EDIT_ALL can see edit action on test case (shard 2, 1 retry)
• Features/Glossary/GlossaryP3Tests.spec.ts › should handle multiple rapid API calls (shard 2, 1 retry)
• Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
• Features/Table.spec.ts › Table pagination with sorting should works (shard 3, 1 retry)
• Flow/ConditionalPermissions.spec.ts › User with owner permission can only view owned Api Services (shard 3, 2 retries)
• Flow/ConditionalPermissions.spec.ts › User with owner permission can only view owned Pipeline Services (shard 3, 2 retries)
• Flow/ConditionalPermissions.spec.ts › User with owner permission can only view owned Database (shard 3, 2 retries)
• Flow/ObservabilityAlerts.spec.ts › Alert operations for a user with and without permissions (shard 3, 2 retries)
• Flow/PersonaFlow.spec.ts › Set default persona for team should work properly (shard 3, 1 retry)
• Pages/EntityDataConsumer.spec.ts › Tier Add, Update and Remove (shard 5, 1 retry)
• Pages/Glossary.spec.ts › Drag and Drop Glossary Term (shard 6, 1 retry)
• Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
• Pages/ServiceEntity.spec.ts › Tag Add, Update and Remove (shard 6, 1 retry)
• Pages/ServiceEntity.spec.ts › Inactive Announcement create & delete (shard 6, 1 retry)

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

[Copilot]

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

[gitar-bot]

Code Review ✅ Approved

Adds supplementary logging to the PBI lineage process to improve debugging visibility. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[sonarqubecloud]

Quality Gate failed for 'open-metadata-ingestion'

Failed conditions
13.9% Coverage on New Code (required ≥ 20%)

See analysis details on SonarQube Cloud