diff --git a/.claude/skills/dax-optimization/SKILL.md b/.claude/skills/dax-optimization/SKILL.md new file mode 100644 index 000000000..08288854e --- /dev/null +++ b/.claude/skills/dax-optimization/SKILL.md @@ -0,0 +1,208 @@ +--- +name: dax-optimization +description: Methodology and an executable rule catalog for diagnosing and optimizing DAX query performance. Use this when analyzing a slow DAX query, interpreting trace timings / DAX query plans, reducing column cardinality, or extending the performance-analysis rules used by the interactive DAX test widget (sempy_labs.semantic_model.test). +--- + +# DAX Optimization + +This skill explains how to determine optimization techniques for a DAX query and +its components, and documents the **executable rule catalog** +(`dax_optimization_rules.json`) that powers the *Performance analysis* tab of the +interactive DAX test widget. + +The runtime rule engine lives in +`src/sempy_labs/semantic_model/_dax_optimization.py` and loads its rules from +`src/sempy_labs/semantic_model/_dax_optimization_rules.json` (the canonical copy +that is packaged and executed). The JSON in this skill folder is the same schema +and is the human-facing reference; keep the two in sync when adding rules. + +## When to Use This Skill + +- Diagnosing why a DAX query is slow. +- Interpreting Formula Engine (FE) vs Storage Engine (SE) timings from a trace. +- Reading a DAX query plan (logical/physical) for `CallbackDataID`, spools, scans. +- Deciding whether high column cardinality is the bottleneck. +- Adding, editing, or reviewing the performance-analysis rules. + +--- + +## The Inputs + +The analysis is computed from up to six artifacts. Each rule declares which +artifacts it `requires`; a rule is skipped if any required artifact is missing. + +| Input | Source | What it provides | +|-------|--------|------------------| +| **DAX query** | The editor text **plus the DAX expressions of every measure the query transitively depends on** | Syntax-level patterns (IFERROR, FILTER over a full table, nested iterators, raw `/` division). The query usually references a measure only by name, so the syntax rules also scan the DAX of dependent measures (resolved from model metadata) to catch issues that live inside those measures. | +| **Model metadata** | TOM (`connect_semantic_model`) | Tables, columns, measures, relationships, data types. | +| **Query dependencies** | `INFO.CALCDEPENDENCY` | The exact tables/columns the query references. | +| **Trace details** | Server-side trace | `QueryEnd`, `VertiPaqSEQueryEnd`, cache matches → total/FE/SE duration, CPU, SE query count, parallelism. | +| **DAX query plan** | Trace `DAXQueryPlan` events | Logical/physical plan text → `CallbackDataID`, `Spool`, scan operators. | +| **Vertipaq Analyzer** | `vertipaq_analyzer(...)` | Column cardinality, size, encoding, data types. **Only used when the cardinalities of the `Data` columns are not all `1`** — otherwise there is nothing meaningful to analyze and Vertipaq rules are skipped. | + +--- + +## Optimization Methodology + +Work top-down, from the cheapest signal to the most detailed. + +### 1. Establish the engine balance (FE vs SE) + +The Storage Engine (VertiPaq) is multi-threaded and fast; the Formula Engine is +single-threaded. From the trace: + +- `Total Duration = QueryEnd.Duration` +- `SE Duration = sum of VertiPaqSEQueryEnd.Duration` (excluding `Internal` subqueries) +- `FE Duration = Total − SE` + +Then: +- **SE-bound** (`SE% ≥ 70%`): the query spends its time scanning data → attack + **data volume and cardinality** (rule `SE_BOUND`). +- **FE-bound** (`FE% ≥ 70%`): the query spends its time in single-threaded logic + → push work to the SE, remove callbacks, reduce materialization (rule + `FE_BOUND`). + +### 2. Look for `CallbackDataID` (the #1 red flag) + +`CallbackDataID` in the **physical plan** means the SE had to call back into the +FE mid-scan. It disables VertiPaq optimizations and is usually caused by: + +- `IF` / `IFERROR` / `ISERROR` / error handling inside an iterator, +- division by `/` (wrap in `DIVIDE`), +- rounding / date arithmetic / conditional logic evaluated row-by-row. + +Rules: `CALLBACK_DATA_ID`, `USES_IFERROR`, `DIVISION_WITHOUT_DIVIDE`. + +### 3. Count and size the Storage Engine queries + +- **Many SE queries** (`≥ 10`) usually means fusion failed — simplify filter + context and use variables to compute base values once (`MANY_SE_QUERIES`). +- **A single slow scan** (`≥ 50 ms`) points at a specific large/high-cardinality + table — inspect its xmSQL in the plan (`SLOW_SE_SCAN`). +- **Low parallelism** (`SE CPU / SE Duration < 1.2x` over a meaningful SE + duration) means scans are effectively single-threaded (`LOW_SE_PARALLELISM`). + +### 4. Inspect the query plan for materialization + +Large/many **spools** materialize intermediate results in the FE and cost memory +and time (`LARGE_SPOOL`). Reduce them with variables and earlier filtering. + +### 5. Reduce cardinality (Vertipaq) + +Cardinality drives dictionary size, scan cost, and `DISTINCTCOUNT`/join cost. +Focus on **columns the query actually references**: + +- **High-cardinality columns** (`≥ 1,000,000` unique values): split datetime + into date+time, bucket/round numerics, drop unused keys (`HIGH_CARDINALITY_COLUMN`). +- **High-cardinality floating point** columns are especially expensive — convert + to fixed decimal/integer or round (`FLOAT_HIGH_CARDINALITY_COLUMN`). + +### 6. Simplify the DAX itself + +- **Nested iterators** multiply row evaluations (`NESTED_ITERATORS`). +- **Many iterators** (`≥ 5`) raise the chance of row-by-row work (`MANY_ITERATORS`). +- **FILTER over a whole table to evaluate a measure** (e.g. `FILTER(Sales, + [Total Qty] > 100)`) tests the measure on every row of the table — iterate the + smallest grouping instead, e.g. `FILTER(VALUES(Sales[OrderId]), [Total Qty] > + 100)` (`FILTER_FULL_TABLE`). This rule fires only when the FILTER predicate + references a **measure**. +- **FILTER wrapping a column predicate** (e.g. `FILTER(Customer, + Customer[Category] = "A")`) materializes the whole table for a condition over + its columns — rewrite as `KEEPFILTERS(Customer[Category] = "A")` + (`FILTER_COLUMN_USE_KEEPFILTERS`). This rule fires when the FILTER predicate + references **columns** (and no measure). Measure vs. column is resolved from + model metadata when available, otherwise inferred from whether the bracket + reference is table-qualified. +- **Many referenced columns** (`≥ 15`) widen datacaches — project only what's + needed (`MANY_REFERENCED_COLUMNS`). + +### 7. Diagnostics + +If the trace or plan wasn't captured, the engine emits an informational finding +(`NO_TRACE_CAPTURED`, `NO_QUERY_PLAN_CAPTURED`) telling the user to run the query +first so the full analysis can be produced. + +--- + +## The Rules JSON Schema + +Each entry in `rules` is one rule: + +```jsonc +{ + "id": "CALLBACK_DATA_ID", // stable identifier + "title": "…", // short headline + "category": "Query plan", // grouping label + "severity": "high|medium|low|info", + "requires": ["query_plan"], // artifacts that must be present + "kind": "scalar|for_each", // evaluation mode + "condition": { … }, // scalar rules: evaluated against metrics + "collection": "high_cardinality_columns", // for_each rules: list to iterate + "where": { … }, // for_each rules: per-item filter + "max_findings": 8, // for_each rules: cap on emitted findings + "message": "… {placeholder} …", // templated; {tokens} filled from context + "recommendation": "…", + "references": ["https://…"] +} +``` + +### Conditions + +A condition is a tree of: + +- **Leaf** — `{"metric": "se_pct", "op": ">=", "value": 0.7}` for scalar rules, + or `{"field": "cardinality", "op": ">=", "value": 1000000}` inside a + `for_each` `where`. +- **Composite** — `{"all": [ … ]}`, `{"any": [ … ]}`, `{"not": { … }}`. + +Operators: `>`, `>=`, `<`, `<=`, `==`, `!=`, `contains`, `not_contains`, +`regex`, `in`, `not_in`. Unknown operators / type errors evaluate to `false`, +so a malformed rule can never crash the analysis. + +### Available metrics (scalar context) + +`has_query`, `query_length`, `iterator_count`, `nested_iterator`, +`uses_iferror`, `uses_divide_function`, `uses_division_operator`, +`filter_full_table_count`, `filter_column_predicate_count`, `cold_cache`, +`has_trace`, `total_duration_ms`, +`se_duration_ms`, `fe_duration_ms`, `cpu_time_ms`, `se_pct`, `fe_pct`, +`se_query_count`, `se_internal_count`, `se_cache_match_count`, `se_cpu_ms`, +`se_parallelism`, `has_query_plan`, `callback_dataid_count`, +`encode_callback_count`, `spool_count`, `referenced_column_count`, +`referenced_table_count`, `has_dependencies`, `vertipaq_available`, +`vertipaq_skipped_trivial`, `max_data_column_cardinality`, +`high_cardinality_data_column_count`. Display helpers: `se_pct_display`, +`fe_pct_display`, `se_parallelism_display`. + +### Available collections (`for_each` context) + +| Collection | Item fields | +|------------|-------------| +| `high_cardinality_columns` | `table`, `column`, `cardinality`, `cardinality_display`, `data_type`, `is_floating_point`, `data_size`, `encoding` | +| `slow_se_queries` | `subclass`, `duration`, `cpu` | + +`message` placeholders for `for_each` rules can reference any item field as well +as any scalar metric. + +--- + +## Adding or Editing a Rule + +1. Add the rule object to **both** JSON copies (package + this skill folder). +2. If the rule needs a new metric or collection, add it in + `build_context()` in `_dax_optimization.py`. +3. Keep `severity` honest: reserve `high` for things that clearly dominate + runtime (e.g. `CallbackDataID`, `IFERROR`). +4. Provide an actionable `recommendation` and at least one authoritative + `reference` (SQLBI or Microsoft Learn). +5. Validate: `python -c "import json,sys; json.load(open('src/sempy_labs/semantic_model/_dax_optimization_rules.json'))"`. + +--- + +## References + +- SQLBI — Understanding DAX query plans: https://www.sqlbi.com/articles/understanding-dax-query-plans/ +- SQLBI — Optimizing high-cardinality columns in VertiPaq: https://www.sqlbi.com/articles/optimizing-high-cardinality-columns-in-vertipaq/ +- SQLBI — Error handling in DAX measures: https://www.sqlbi.com/articles/error-handling-in-dax-measures/ +- Microsoft Learn — DIVIDE function: https://learn.microsoft.com/dax/divide-function-dax +- Microsoft Learn — Data reduction techniques for import modeling: https://learn.microsoft.com/power-bi/guidance/import-modeling-data-reduction diff --git a/.claude/skills/dax-optimization/dax_optimization_rules.json b/.claude/skills/dax-optimization/dax_optimization_rules.json new file mode 100644 index 000000000..310c9d0d3 --- /dev/null +++ b/.claude/skills/dax-optimization/dax_optimization_rules.json @@ -0,0 +1,325 @@ +{ + "schema_version": 1, + "description": "Declarative DAX performance optimization rules executed by sempy_labs.semantic_model._dax_optimization. Each rule is evaluated against a flat 'metrics' dictionary (scalar rules) or a named 'collection' of items (for_each rules) derived from the DAX query, semantic model metadata (TOM), query dependencies, trace details, the DAX query plan and (optionally) Vertipaq Analyzer statistics. See the dax-optimization skill for the methodology behind these rules.", + "severity_order": ["high", "medium", "low", "info"], + "rules": [ + { + "id": "SE_BOUND", + "title": "Query is Storage Engine bound", + "category": "Engine balance", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "total_duration_ms", "op": ">=", "value": 50}, + {"metric": "se_pct", "op": ">=", "value": 0.7} + ] + }, + "message": "The Storage Engine accounts for {se_pct_display} of the {total_duration_ms} ms total duration. The query spends most of its time scanning data.", + "recommendation": "Reduce the amount of data scanned by the Storage Engine: lower column cardinality, filter earlier, avoid scanning high-cardinality columns, and ensure relationships allow VertiPaq to prune partitions. Confirm the slow xmSQL scans in the trace correspond to large or high-cardinality tables.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/", + "https://learn.microsoft.com/power-bi/guidance/dax-variables" + ] + }, + { + "id": "FE_BOUND", + "title": "Query is Formula Engine bound", + "category": "Engine balance", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "total_duration_ms", "op": ">=", "value": 50}, + {"metric": "fe_pct", "op": ">=", "value": 0.7} + ] + }, + "message": "The Formula Engine accounts for {fe_pct_display} of the {total_duration_ms} ms total duration. The query spends most of its time in single-threaded Formula Engine work.", + "recommendation": "The Formula Engine is single-threaded and cannot be parallelized. Push more work to the Storage Engine: replace row-by-row logic with set-based DAX, avoid forcing CallbackDataID (e.g. IF/IFERROR inside iterators), reduce large intermediate materializations, and prefer aggregations that the Storage Engine can resolve directly.", + "references": [ + "https://www.sqlbi.com/articles/optimizing-the-use-of-variables-in-dax/", + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "MANY_SE_QUERIES", + "title": "High number of Storage Engine queries", + "category": "Storage Engine", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "metric": "se_query_count", "op": ">=", "value": 10 + }, + "message": "The query generated {se_query_count} Storage Engine queries (excluding internal sub-queries). A large number of separate scans usually means the engine could not fuse them into fewer, larger scans.", + "recommendation": "Aim for fewer Storage Engine queries. Multiple scans of the same table often indicate complex filter context transitions, non-foldable logic, or iterators producing one SE query per row group. Simplify measure logic, use variables to compute a base value once, and avoid patterns that defeat VertiPaq fusion.", + "references": [ + "https://www.sqlbi.com/articles/introducing-vertipaq-fusion/" + ] + }, + { + "id": "CALLBACK_DATA_ID", + "title": "CallbackDataID detected in the physical query plan", + "category": "Query plan", + "severity": "high", + "requires": ["query_plan"], + "kind": "scalar", + "condition": { + "metric": "callback_dataid_count", "op": ">=", "value": 1 + }, + "message": "The physical query plan contains {callback_dataid_count} CallbackDataID operation(s). CallbackDataID means the Storage Engine had to call back into the Formula Engine during a scan, which is slow and disables many VertiPaq optimizations.", + "recommendation": "Remove the constructs that force CallbackDataID. Common causes: IF / IFERROR / DIVIDE-with-alternate / complex conditional logic, error handling, date/time arithmetic, and rounding inside an iterator that runs in the Storage Engine. Move conditional logic out of the row context, precompute values with variables, or use calculated columns where appropriate.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/", + "https://www.sqlbi.com/tv/callbackdataid-in-dax/" + ] + }, + { + "id": "LOW_SE_PARALLELISM", + "title": "Low Storage Engine parallelism", + "category": "Storage Engine", + "severity": "low", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "se_duration_ms", "op": ">=", "value": 100}, + {"metric": "se_parallelism", "op": "<", "value": 1.2} + ] + }, + "message": "Storage Engine parallelism is {se_parallelism_display} (SE CPU {se_cpu_ms} ms over {se_duration_ms} ms of SE duration). The Storage Engine scans are running mostly single-threaded.", + "recommendation": "Low parallelism limits the benefit of VertiPaq's multi-threaded engine. This often happens with many small sequential scans or scans serialized behind Formula Engine callbacks. Reduce CallbackDataID, consolidate scans, and ensure tables are large enough to benefit from segment-level parallelism.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "LARGE_SPOOL", + "title": "Large intermediate spools in the physical plan", + "category": "Query plan", + "severity": "medium", + "requires": ["query_plan"], + "kind": "scalar", + "condition": { + "metric": "spool_count", "op": ">=", "value": 4 + }, + "message": "The physical query plan contains {spool_count} Spool operations. Spools materialize intermediate results in the Formula Engine and can consume significant memory and time when they are large.", + "recommendation": "Reduce the number and size of materialized intermediate results. Use variables to avoid recomputing the same sub-expression, filter before joining, and avoid producing large datacaches that the Formula Engine must iterate over.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "USES_IFERROR", + "title": "IFERROR / ISERROR forces row-by-row evaluation", + "category": "DAX syntax", + "severity": "high", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "uses_iferror", "op": "==", "value": true + }, + "message": "The query uses IFERROR or ISERROR. These functions force the engine to evaluate the wrapped expression row by row and almost always introduce CallbackDataID, preventing Storage Engine optimization.", + "recommendation": "Avoid IFERROR/ISERROR for performance-sensitive logic. For division use DIVIDE(numerator, denominator) which safely handles divide-by-zero without error trapping. Validate inputs with explicit conditions instead of catching errors.", + "references": [ + "https://www.sqlbi.com/articles/error-handling-in-dax-measures/", + "https://learn.microsoft.com/dax/divide-function-dax" + ] + }, + { + "id": "DIVISION_WITHOUT_DIVIDE", + "title": "Division operator used without DIVIDE", + "category": "DAX syntax", + "severity": "low", + "requires": ["query"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "uses_division_operator", "op": "==", "value": true}, + {"metric": "uses_divide_function", "op": "==", "value": false} + ] + }, + "message": "The query uses the '/' division operator but does not use the DIVIDE function. A raw '/' can raise errors on divide-by-zero, which then tend to be wrapped in IFERROR and slow the query.", + "recommendation": "Replace 'a / b' with DIVIDE(a, b). DIVIDE returns BLANK (or a supplied alternate) on divide-by-zero without the overhead of error handling.", + "references": [ + "https://learn.microsoft.com/dax/divide-function-dax" + ] + }, + { + "id": "FILTER_FULL_TABLE", + "title": "FILTER over an entire table to evaluate a measure", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "filter_full_table_count", "op": ">=", "value": 1 + }, + "message": "The query iterates a full table with FILTER to evaluate a measure {filter_full_table_count} time(s) (e.g. FILTER(Sales, [Total Qty] > 100)). Testing a measure for every row of an entire table forces a full scan and row-by-row Formula Engine evaluation of the measure.", + "recommendation": "Iterate the smallest set needed instead of the whole table. Filter on the distinct values that drive the measure (e.g. FILTER(VALUES(Sales[OrderId]), [Total Qty] > 100)), or restructure the logic so the measure is evaluated once per group rather than per row. When the predicate is a simple column comparison, use KEEPFILTERS(Table[Column] = value) instead of FILTER.", + "references": [ + "https://www.sqlbi.com/articles/filter-arguments-in-calculate/", + "https://learn.microsoft.com/dax/best-practices/dax-avoid-converting-blank" + ] + }, + { + "id": "FILTER_COLUMN_USE_KEEPFILTERS", + "title": "FILTER over a table for a simple column predicate", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "filter_column_predicate_count", "op": ">=", "value": 1 + }, + "message": "The query wraps a simple column predicate in FILTER(Table, ...) {filter_column_predicate_count} time(s) (e.g. CALCULATE([Qty], FILTER(Customer, Customer[Category] = \"A\"))). This materializes the entire table just to apply a single boolean condition.", + "recommendation": "Replace FILTER(Table, Table[Column] = value) with KEEPFILTERS(Table[Column] = value) so the engine pushes the predicate down to the Storage Engine instead of iterating the whole table. For example, rewrite CALCULATE([Qty], FILTER(Customer, Customer[Category] = \"A\")) as CALCULATE([Qty], KEEPFILTERS(Customer[Category] = \"A\")). A bare column predicate inside CALCULATE (without KEEPFILTERS) is also faster but replaces, rather than intersects, the existing filter context.", + "references": [ + "https://www.sqlbi.com/articles/filter-arguments-in-calculate/", + "https://www.sqlbi.com/articles/using-keepfilters-in-dax/" + ] + }, + { + "id": "NESTED_ITERATORS", + "title": "Nested iterators over potentially large tables", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "nested_iterator", "op": "==", "value": true + }, + "message": "The query contains nested iterator functions (e.g. SUMX inside SUMX). Nested iteration multiplies the number of rows evaluated and is a common cause of slow, Formula-Engine-bound queries.", + "recommendation": "Flatten nested iterators where possible. Compute inner aggregations once with variables, push aggregation into the Storage Engine with simple SUMX/AVERAGEX over a single table, and avoid iterating one large table inside another.", + "references": [ + "https://www.sqlbi.com/articles/nested-iterators-in-dax/" + ] + }, + { + "id": "MANY_ITERATORS", + "title": "Many iterator functions in the query", + "category": "DAX syntax", + "severity": "low", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "iterator_count", "op": ">=", "value": 5 + }, + "message": "The query uses {iterator_count} iterator functions (SUMX, AVERAGEX, FILTER, etc.). A high count of iterators increases the chance of row-by-row Formula Engine work.", + "recommendation": "Review each iterator and replace those that can be expressed as set-based aggregations. Where an iterator only sums a single column, a simple SUM over the column is faster and Storage-Engine friendly.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "MANY_REFERENCED_COLUMNS", + "title": "Query references a large number of columns", + "category": "Model design", + "severity": "low", + "requires": ["dependencies"], + "kind": "scalar", + "condition": { + "metric": "referenced_column_count", "op": ">=", "value": 15 + }, + "message": "The query depends on {referenced_column_count} columns across {referenced_table_count} table(s). Wide queries scan more dictionaries and produce larger datacaches.", + "recommendation": "Confirm every referenced column is required. Removing unused columns from the query (and from SUMMARIZECOLUMNS/SELECTCOLUMNS projections) reduces the data the Storage Engine must materialize.", + "references": [ + "https://www.sqlbi.com/articles/using-summarizecolumns-and-addmissingitems/" + ] + }, + { + "id": "HIGH_CARDINALITY_COLUMN", + "title": "High-cardinality column referenced by the query", + "category": "Cardinality", + "severity": "medium", + "requires": ["vertipaq"], + "kind": "for_each", + "collection": "high_cardinality_columns", + "max_findings": 8, + "where": { + "field": "cardinality", "op": ">=", "value": 1000000 + }, + "message": "Column '{table}'[{column}] has {cardinality_display} unique values ({data_type}). High-cardinality columns produce large dictionaries and slow scans, joins, and DISTINCTCOUNT operations.", + "recommendation": "Reduce cardinality where possible: split a datetime column into separate date and time columns, round or bucket numeric values that do not need full precision, remove unused high-cardinality keys, and avoid grouping or DISTINCTCOUNT on the highest-cardinality columns in hot queries.", + "references": [ + "https://www.sqlbi.com/articles/optimizing-high-cardinality-columns-in-vertipaq/", + "https://learn.microsoft.com/power-bi/guidance/import-modeling-data-reduction" + ] + }, + { + "id": "FLOAT_HIGH_CARDINALITY_COLUMN", + "title": "High-cardinality floating point column", + "category": "Cardinality", + "severity": "medium", + "requires": ["vertipaq"], + "kind": "for_each", + "collection": "high_cardinality_columns", + "max_findings": 8, + "where": { + "all": [ + {"field": "cardinality", "op": ">=", "value": 100000}, + {"field": "is_floating_point", "op": "==", "value": true} + ] + }, + "message": "Floating point column '{table}'[{column}] has {cardinality_display} unique values. Floating point (Double) columns with high cardinality are expensive to store and scan and can cause subtle rounding in grouping.", + "recommendation": "If the column does not need full floating point precision, convert it to a fixed decimal (Currency) or integer, or round it to fewer decimal places to dramatically reduce cardinality and size.", + "references": [ + "https://www.sqlbi.com/articles/data-types-in-dax-and-data-modeling/" + ] + }, + { + "id": "SLOW_SE_SCAN", + "title": "Slow Storage Engine scan", + "category": "Storage Engine", + "severity": "medium", + "requires": ["trace"], + "kind": "for_each", + "collection": "slow_se_queries", + "max_findings": 5, + "where": { + "field": "duration", "op": ">=", "value": 50 + }, + "message": "A Storage Engine scan ran for {duration} ms (CPU {cpu} ms). This single scan is a significant contributor to the total duration.", + "recommendation": "Inspect the xmSQL text of this scan in the DAX query plan. Long scans usually target large or high-cardinality tables, apply complex filters, or include a CallbackDataID. Reduce the rows scanned, simplify the filter, or remove the callback.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "NO_TRACE_CAPTURED", + "title": "No trace details were captured", + "category": "Diagnostics", + "severity": "info", + "requires": [], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "has_trace", "op": "==", "value": false}, + {"metric": "has_query", "op": "==", "value": true} + ] + }, + "message": "No trace details were captured for the current query, so engine-timing rules could not be evaluated.", + "recommendation": "Run the DAX query first (so trace events and the query plan are captured), then generate the performance analysis again for complete results.", + "references": [] + }, + { + "id": "NO_QUERY_PLAN_CAPTURED", + "title": "No DAX query plan was captured", + "category": "Diagnostics", + "severity": "info", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "has_query_plan", "op": "==", "value": false}, + {"metric": "has_trace", "op": "==", "value": true} + ] + }, + "message": "Trace details are available but no DAX query plan (logical/physical) was captured, so plan-based rules (CallbackDataID, spools) could not be evaluated.", + "recommendation": "Re-run the query to capture the query plan. CallbackDataID and spool analysis require the physical plan.", + "references": [] + } + ] +} diff --git a/.claude/skills/query-builder-dax/SKILL.md b/.claude/skills/query-builder-dax/SKILL.md new file mode 100644 index 000000000..d8e5dd15b --- /dev/null +++ b/.claude/skills/query-builder-dax/SKILL.md @@ -0,0 +1,161 @@ +--- +name: query-builder-dax +description: Guide for the DAX query shape produced by the Query Builder in the DAX test widget. Use this when modifying how the Query Builder generates DAX, or when generating an EVALUATE query from columns, measures, filters and sorting. +--- + +# SKILL.md — Query Builder DAX Generation + +## Purpose + +Document the canonical DAX query structure produced by the Query Builder in +the DAX performance test widget (`_build_summarize_dax` in +`src/sempy_labs/semantic_model/_test_dax.py`). + +The pattern follows the well-known "Using DAX as a Query Language" approach +(Michael Kovalsky, Elegant BI: +), built on +`SUMMARIZECOLUMNS`. + +Use this skill when changing how Query Builder state (columns, measures, +filters, sorting) is converted to DAX, so the generated query stays valid and +consistent. + +--- + +## The canonical query shape + +```dax +EVALUATE +SUMMARIZECOLUMNS( + 'Geography'[Area], + 'Geography'[Country], + 'Product'[Product], + FILTER(KEEPFILTERS(VALUES('Product'[Product Category])), 'Product'[Product Category] = "Bicycles"), + "Revenue", [Revenue] +) +ORDER BY 'Geography'[Area], 'Geography'[Country] +``` + +`SUMMARIZECOLUMNS` implies the aggregation (the SQL `GROUP BY` is not needed) +and the model relationships handle the joins. + +--- + +## Element order (STRICT — wrong order errors out) + +Inside `SUMMARIZECOLUMNS(...)` the elements MUST appear in this order: + +| # | Element | Where it goes | Syntax | +|---|---------|---------------|--------| +| 1 | **Attributes (columns)** | First | `'Table'[Column]` — one per group-by column, left to right | +| 2 | **Filters (on columns)** | After all attributes | `FILTER(KEEPFILTERS(VALUES('Table'[Column])), )` — one per filter, any order | +| 3 | **Measures** | After all filters | `"Display Name", [Measure]` — display name in quotes, measure in brackets | + +Then, OUTSIDE `SUMMARIZECOLUMNS`: + +| Element | Where it goes | Syntax | +|---------|---------------|--------| +| **Sorting** | Final clause, after the table expression | `ORDER BY 'Table'[Column] [ASC\|DESC], ...` (defaults to ascending) | + +--- + +## Columns (attributes) + +- Each group-by column from the builder's *Columns & Measures* pane becomes an + attribute, in pane order (top to bottom = left to right in the result). +- Always fully qualify: `'Table'[Column]`. Escape `'` → `''` in the table name + and `]` → `]]` in the column name. + +## Measures + +- Each measure from the *Columns & Measures* pane becomes a measure element. +- Syntax is `"Display Name", [Measure]`. The display name only renames the + output column; it does not rename the model measure. +- Custom/test measures can be added with a `DEFINE MEASURE ... ` block placed + **before** `EVALUATE`, then referenced in the measures section. (The Query + Builder currently only references existing model measures.) + +## Filters + +Filters come from the builder's *Filters* pane. Split by object kind: + +- **Column filters** go **inside** `SUMMARIZECOLUMNS`, after the attributes, + each as: + ```dax + FILTER(KEEPFILTERS(VALUES('Table'[Column])), ) + ``` + `KEEPFILTERS(VALUES(...))` preserves any existing filter context on that + column while applying the new condition. + +- **Measure filters** cannot live inside `SUMMARIZECOLUMNS` (measures are not + yet projected at that point). Wrap the whole table instead: + ```dax + FILTER( + SUMMARIZECOLUMNS( ... ), + [Measure] > 100 + ) + ``` + Multiple measure predicates are combined with `&&`. + +### Predicate operators + +`_qb_build_predicate` maps a filter item (`ref`, `kind`, `data_type`, `op`, +`value`, `value2`) to a boolean predicate: + +| Filter family | Operators → DAX | +|---------------|-----------------| +| text | `eq` `=`, `ne` `<>`, `contains` `CONTAINSSTRING(ref, "v")`, `startswith` `LEFT(ref, n) = "v"` | +| numeric / datetime / measure | `eq` `=`, `ne` `<>`, `gt` `>`, `ge` `>=`, `lt` `<`, `le` `<=`, `between` `ref >= lo && ref <= hi` | +| boolean | `istrue` `ref = TRUE()`, `isfalse` `ref = FALSE()` | +| any | `blank` `ISBLANK(ref)`, `notblank` `NOT ISBLANK(ref)` | + +Value literals: +- numeric → bare number (quoted string if not numeric), +- datetime → `DATE(y, m, d)` when the value matches `YYYY-MM-DD`, else a quoted + string, +- text → quoted string (`"` escaped as `""`). + +## Sorting (ORDER BY) + +- The `ORDER BY` clause is the final part of the query, placed **after** the + (possibly `FILTER`-wrapped) `SUMMARIZECOLUMNS` table expression. +- It is driven by the builder's *Order By* pane, which mirrors the + columns/measures from the *Columns & Measures* pane. Each Order By item has: + - a **toggle** (on/off, default **off**) — only enabled items appear in + `ORDER BY`; + - a **direction** icon — A-Z = ascending (`ASC`), Z-A = descending (`DESC`); + - independent **reorder** support (the pane order = clause order). +- The serialized state carries an `order_by` list of items + (`ref`, `kind`, `name`, `table`, `enabled`, `dir`). `_build_summarize_dax` + emits `ORDER BY ASC|DESC, ...` for the enabled items, in pane order. +- Measures can be used in `ORDER BY` because each measure is projected as a + column in the result. +- If the `order_by` key is **absent** (legacy state), the builder falls back to + ordering by all attribute columns ascending. If present but no item is + enabled, no `ORDER BY` is emitted. + +--- + +## Edge cases + +- **Measures only (no columns):** emit `SUMMARIZECOLUMNS("Name", [Measure], ...)` + with no attributes and no `ORDER BY`. Column filters still apply inside; + measure filters still wrap the outside. +- **Nothing usable:** return an empty string (no columns and no measures). +- **TOPN:** to cap rows, wrap the table in `TOPN(n, )` (respects the + `ORDER BY`). Not currently emitted by the Query Builder. + +--- + +## Keep in sync + +The JS front-end (`onBuildClick`) serializes builder state to JSON with +snake_case keys; the Python helpers read them. When adding operators or field +metadata, update BOTH: + +- JS `QB_OPS` / chip serialization in `_test_dax.py` (widget_js), +- Python `_qb_build_predicate` / `_classify_filter_type` / `_build_summarize_dax`. + +Field objects use keys: `kind`, `table`, `name`, `data_type`, `ref`. +Filter objects additionally: `op`, `value`, `value2`. +Order By objects additionally: `enabled` (bool), `dir` (`"asc"`/`"desc"`). diff --git a/.claude/skills/ui-styling/SKILL.md b/.claude/skills/ui-styling/SKILL.md index 8679979b0..c0f9aa204 100644 --- a/.claude/skills/ui-styling/SKILL.md +++ b/.claude/skills/ui-styling/SKILL.md @@ -39,11 +39,16 @@ Semantic Link Labs has exactly two supported patterns for interactive UI tools. | Export | Purpose | |--------|---------| -| `ICONS` | Dict of monochrome SVG icons. All use `stroke="currentColor"` / `fill="currentColor"` so they adapt to light and dark themes automatically. Keys include tabular-object icons (`table`, `column`, `column_chunk`, `measure`, `hierarchy`, `partition`, `relationship`) and UI icons (`sun`, `moon`, `search`, `plus`, `caret_right`). | -| `LIGHT_THEME_VARS`, `DARK_THEME_VARS` | CSS custom-property blocks defining the Apple-inspired light and dark palettes. Always reference colors via these `--ui-*` tokens, never hard-coded hex values. | +| `ICONS` | Dict of monochrome SVG icons. All use `stroke="currentColor"` / `fill="currentColor"` so they adapt to light and dark themes automatically. Keys include tabular-object icons (`table`, `calculation_group`, `column`, `column_chunk`, `measure`, `hierarchy`, `calculation_item`, `partition`, `relationship`), tree/navigation icons (`caret_right`, `folder`, `level`), and UI/action icons (`sun`, `moon`, `search`, `plus`, `play`, `stop`, `refresh`, `swap`, `sort_asc`, `sort_desc`, `panel_collapse`, `panel_expand`, `builder`, `close`, `fullscreen`, `fullscreen_exit`). | +| `LIGHT_THEME_VARS`, `DARK_THEME_VARS` | CSS custom-property blocks defining the Apple-inspired light and dark palettes. Always reference colors via these `--ui-*` tokens, never hard-coded hex values. Includes semantic tokens for hover backgrounds (`--ui-bg-hover`), on-accent text (`--ui-on-accent`), and destructive/error states (`--ui-danger*`). | +| `SYNTAX_HIGHLIGHT_VARS` | Theme-independent `--ui-syntax-*` token block for colorizing DAX/code in an editor. Inject once into the widget's base scope (it is the same in light and dark). | | `HEADER_CSS`, `scoped_header_css(root_selector)` | Standard widget header styles (title + dataset/workspace subtitle + theme toggle button). `scoped_header_css` prefixes every rule with the root selector so the styles win against notebook host CSS (e.g. Jupyter's `.jp-RenderedHTMLCommon button`). | -| `render_header_html(title, dataset_name, workspace_name, theme_btn_id, dark_mode)` | Renders the standard header markup. | +| `render_header_html(title, dataset_name, workspace_name, theme_btn_id, dark_mode, fullscreen_btn_id)` | Renders the standard header markup. Pass `fullscreen_btn_id` to include a full-screen toggle button next to the theme toggle. | | `theme_toggle_script(btn_id, root_selector, dark_class)` | Returns a `\n" + ) + + +# --------------------------------------------------------------------------- +# Rendering a self-contained HTML string as an anywidget +# --------------------------------------------------------------------------- +# ESM for a minimal anywidget that hosts a pre-built HTML string (styles + +# markup + + + """ + display(HTML(html)) + + +@log +def capture_report_pages( + report: str | UUID, + workspace: Optional[str | UUID] = None, + visible: bool = True, + timeout: int = 120, + quiet_period: int = 10, +) -> pd.DataFrame: + """ + Opens a Power BI report (in view mode) within the notebook, cycles through + each page of the report, and captures the trace logs of the DAX queries + generated by the report's visuals. + + Before the report is embedded, a server-side trace is started against the + semantic model behind the report (using the same trace events as + :func:`sempy_labs.semantic_model.test`). The report is then embedded using + the `Power BI JavaScript client + `_ and each + page is activated in turn. A page is only advanced once the report signals + that the current page has fully finished rendering its visuals (rather than + waiting a fixed amount of time), so that each page's visuals render and + execute their DAX queries against the semantic model. The trace logs of + those DAX queries are then returned. + + Service Principal Authentication is supported (see `here `_ for examples). + + Parameters + ---------- + report : str | uuid.UUID + Name or ID of the Power BI report. + workspace : str | uuid.UUID, default=None + The name or ID of the Fabric workspace. + Defaults to None which resolves to the workspace of the attached lakehouse + or if no lakehouse attached, resolves to the workspace of the notebook. + visible : bool, default=True + If True, the embedded report is displayed within the notebook while the + pages are cycled through. If False, the report is embedded and cycled + through in the background without being shown to the user. + timeout : int, default=120 + The maximum number of seconds to keep the trace open while the report + renders and cycles through its pages. + quiet_period : int, default=10 + The trace is stopped early once no new trace events have been captured + for this many seconds (after at least one event has been captured), + indicating the report has finished rendering all of its pages. + + Returns + ------- + pandas.DataFrame + A pandas dataframe of the captured trace events generated by the report's + visuals while cycling through each page. + """ + + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + report_name, report_id = resolve_item_name_and_id( + item=report, type="Report", workspace=workspace_id + ) + """ + {"DatasetId":"974a1e22-6f9c-410b-be9c-45fc53c78cfc","Sources":[{"ReportId":"9be18c55-2e33-4d4e-a88b-c8846c5dfc24","VisualId":"ae7fd3c0d2e25807e685","HostProperties":{"ConsumptionMethod":"Power BI Web App","UserSession":"ca7b502c-a2d8-4a84-92fb-37d7032cf8e2"}}]} + """ + + # Retrieve the report's embed URL and underlying semantic model id. + report_df = list_reports_base(report=report_id, workspace=workspace_id) + if report_df.empty: + raise ValueError( + f"{icons.red_dot} The '{report_name}' report was not found within the '{workspace_name}' workspace." + ) + + embed_url = report_df["Embed Url"].iloc[0] + dataset_id = report_df["Dataset Id"].iloc[0] + dataset_workspace_id = report_df["Dataset Workspace Id"].iloc[0] + + if not embed_url or not dataset_id: + raise ValueError( + f"{icons.red_dot} The '{report_name}' report within the '{workspace_name}' workspace cannot be embedded (missing embed URL or semantic model)." + ) + + # Generate an embed token for the report. + access_token = generate_embed_token( + dataset_ids=[dataset_id], + report_ids=[report_id], + ) + + # Start a server-side trace against the semantic model behind the report + # (using the same trace events as sempy_labs.semantic_model.test), then + # embed the report and cycle through its pages so the visuals execute their + # DAX queries while the trace is running. + df = pd.DataFrame() + with fabric.create_trace_connection( + dataset=dataset_id, workspace=dataset_workspace_id + ) as trace_connection: + with trace_connection.create_trace(_TEST_EVENT_SCHEMA) as trace: + trace.start() + + embed_report_cycle_pages( + embed_url=embed_url, + access_token=access_token, + visible=visible, + ) + + # Poll the trace while the report renders in the browser. Stop once + # no new events have arrived for ``quiet_period`` seconds (the + # report has finished cycling through its pages) or once the overall + # ``timeout`` is reached. + start_time = time.time() + last_count = 0 + last_change_time = start_time + while time.time() - start_time < timeout: + time.sleep(1) + logs = _get_trace_logs(trace) + current_count = 0 if logs is None else len(logs) + if current_count != last_count: + last_count = current_count + last_change_time = time.time() + elif ( + last_count > 0 + and time.time() - last_change_time >= quiet_period + ): + break + + try: + stopped = trace.stop() + if stopped is not None and not stopped.empty: + df = stopped + else: + logs = _get_trace_logs(trace) + if logs is not None: + df = logs + except Exception: + logs = _get_trace_logs(trace) + if logs is not None: + df = logs + + return df diff --git a/src/sempy_labs/report/_download_report.py b/src/sempy_labs/report/_download_report.py index b13fbb9e3..55abdbab4 100644 --- a/src/sempy_labs/report/_download_report.py +++ b/src/sempy_labs/report/_download_report.py @@ -48,8 +48,8 @@ def download_report( f"{icons.red_dot} A lakehouse must be attached to the notebook." ) - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (lakehouse_name, lakehouse_id) = resolve_lakehouse_name_and_id() + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + lakehouse_name, lakehouse_id = resolve_lakehouse_name_and_id() lakehouse_workspace = resolve_workspace_name() dfI = fabric.list_items(workspace=workspace_id) diff --git a/src/sempy_labs/report/_endorsement.py b/src/sempy_labs/report/_endorsement.py index 5afcacdab..5d9f99f8e 100644 --- a/src/sempy_labs/report/_endorsement.py +++ b/src/sempy_labs/report/_endorsement.py @@ -30,8 +30,8 @@ def set_endorsement( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (report_name, report_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + report_name, report_id = resolve_item_name_and_id( item=report, type="Report", workspace=workspace_id ) diff --git a/src/sempy_labs/report/_export_report.py b/src/sempy_labs/report/_export_report.py index 35ad9f75a..1ab197052 100644 --- a/src/sempy_labs/report/_export_report.py +++ b/src/sempy_labs/report/_export_report.py @@ -64,7 +64,7 @@ def export_report( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) if isinstance(page_name, str): page_name = [page_name] diff --git a/src/sempy_labs/report/_generate_report.py b/src/sempy_labs/report/_generate_report.py index b58c8ddbf..5fb7e547a 100644 --- a/src/sempy_labs/report/_generate_report.py +++ b/src/sempy_labs/report/_generate_report.py @@ -49,8 +49,8 @@ def create_report_from_reportjson( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (dataset_name, dataset_id) = resolve_dataset_name_and_id(dataset, workspace_id) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + dataset_name, dataset_id = resolve_dataset_name_and_id(dataset, workspace_id) dfI = fabric.list_items(workspace=workspace, type="Report") dfI_rpt = dfI[dfI["Display Name"] == report] @@ -146,7 +146,7 @@ def update_report_from_reportjson( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) report_id = resolve_item_id(item=report, type="Report", workspace=workspace) # Get the existing PBIR file @@ -247,7 +247,7 @@ def create_model_bpa_report( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (dataset_workspace_name, dataset_workspace_id) = resolve_workspace_name_and_id( + dataset_workspace_name, dataset_workspace_id = resolve_workspace_name_and_id( dataset_workspace ) @@ -333,7 +333,7 @@ def _create_report( from sempy_labs.report import report_rebind - (report_workspace_name, report_workspace_id) = resolve_workspace_name_and_id( + report_workspace_name, report_workspace_id = resolve_workspace_name_and_id( workspace=report_workspace ) @@ -388,8 +388,8 @@ def _get_report( report: str | UUID, workspace: Optional[str | UUID] = None ) -> pd.DataFrame: - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (report_name, report_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + report_name, report_id = resolve_item_name_and_id( item=report, type="Report", workspace=workspace ) diff --git a/src/sempy_labs/report/_paginated.py b/src/sempy_labs/report/_paginated.py index 7d1a087b2..f560ed5a6 100644 --- a/src/sempy_labs/report/_paginated.py +++ b/src/sempy_labs/report/_paginated.py @@ -49,7 +49,7 @@ def get_report_datasources( } df = _create_dataframe(columns=columns) - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) df_reports = list_reports(workspace=workspace_id) if _is_valid_uuid(report): df_filt = df_reports[ diff --git a/src/sempy_labs/report/_report_bpa.py b/src/sempy_labs/report/_report_bpa.py index 95af5bf64..b413cab5f 100644 --- a/src/sempy_labs/report/_report_bpa.py +++ b/src/sempy_labs/report/_report_bpa.py @@ -109,7 +109,7 @@ def execute_rule(row): for scope in scopes: # common fields for each scope - (df, violation_cols_or_func) = scope_to_dataframe[scope] + df, violation_cols_or_func = scope_to_dataframe[scope] # execute rule and subset df df_violations = df[row["Expression"](df)] @@ -200,8 +200,8 @@ def execute_rule(row): max_run_id = _get_column_aggregate(table_name=delta_table_name) runId = max_run_id + 1 - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (report_name, report_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + report_name, report_id = resolve_item_name_and_id( item=report, type="Report", workspace=workspace_id ) diff --git a/src/sempy_labs/report/_report_functions.py b/src/sempy_labs/report/_report_functions.py index b0ce55e6b..4f133fcf3 100644 --- a/src/sempy_labs/report/_report_functions.py +++ b/src/sempy_labs/report/_report_functions.py @@ -55,8 +55,8 @@ def get_report_json( The report.json file for a given Power BI report. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (report_name, report_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + report_name, report_id = resolve_item_name_and_id( item=report, type="Report", workspace=workspace_id ) @@ -112,7 +112,7 @@ def report_dependency_tree(workspace: Optional[str | UUID] = None): or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) dfR = fabric.list_reports(workspace=workspace_id) dfD = fabric.list_datasets(workspace=workspace_id) @@ -188,7 +188,7 @@ def clone_report( Defaults to None which resolves to the semantic model used by the initial report. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) dfI = fabric.list_items(workspace=workspace_id, type="Report") dfI_filt = dfI[(dfI["Display Name"] == report)] @@ -254,7 +254,7 @@ def launch_report(report: str, workspace: Optional[str | UUID] = None): from sempy_labs import resolve_report_id - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) report_id = resolve_report_id(report, workspace_id) report = Report(group_id=workspace_id, report_id=report_id) @@ -281,7 +281,7 @@ def list_report_pages(report: str, workspace: Optional[str | UUID] = None): A pandas dataframe showing the pages within a Power BI report and their properties. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) df = pd.DataFrame( columns=["Page ID", "Page Name", "Hidden", "Width", "Height", "Visual Count"] @@ -349,7 +349,7 @@ def list_report_visuals(report: str, workspace: Optional[str | UUID] = None): A pandas dataframe showing the visuals within a Power BI report and their properties. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) reportJson = get_report_json(report=report, workspace=workspace_id) @@ -403,7 +403,7 @@ def list_report_bookmarks(report: str, workspace: Optional[str | UUID] = None): A pandas dataframe showing the bookmarks within a Power BI report and their properties. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) df = pd.DataFrame( columns=[ @@ -492,7 +492,7 @@ def translate_report_titles( """ from synapse.ml.services import Translate - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) if isinstance(languages, str): languages = [languages] diff --git a/src/sempy_labs/report/_report_helper.py b/src/sempy_labs/report/_report_helper.py index 58be85b84..1c460cf32 100644 --- a/src/sempy_labs/report/_report_helper.py +++ b/src/sempy_labs/report/_report_helper.py @@ -1,7 +1,6 @@ import requests import sempy_labs._icons as icons - vis_type_mapping = { "barChart": "Bar chart", "columnChart": "Column chart", diff --git a/src/sempy_labs/report/_report_list_functions.py b/src/sempy_labs/report/_report_list_functions.py index f1f1c0b5c..78cedc2c6 100644 --- a/src/sempy_labs/report/_report_list_functions.py +++ b/src/sempy_labs/report/_report_list_functions.py @@ -35,8 +35,8 @@ def list_unused_objects_in_reports( # TODO: what about relationships/RLS? - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (dataset_name, dataset_id) = resolve_dataset_name_and_id(dataset, workspace_id) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + dataset_name, dataset_id = resolve_dataset_name_and_id(dataset, workspace_id) fabric.refresh_tom_cache(workspace=workspace) @@ -83,8 +83,8 @@ def _list_all_report_semantic_model_objects( A pandas dataframe. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (dataset_name, dataset_id) = resolve_dataset_name_and_id(dataset, workspace_id) + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + dataset_name, dataset_id = resolve_dataset_name_and_id(dataset, workspace_id) dfR = list_reports_using_semantic_model(dataset=dataset_id, workspace=workspace_id) dfs = [] diff --git a/src/sempy_labs/report/_report_rebind.py b/src/sempy_labs/report/_report_rebind.py index 674d965e6..29e170701 100644 --- a/src/sempy_labs/report/_report_rebind.py +++ b/src/sempy_labs/report/_report_rebind.py @@ -42,24 +42,24 @@ def report_rebind( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (report_workspace_name, report_workspace_id) = resolve_workspace_name_and_id( + report_workspace_name, report_workspace_id = resolve_workspace_name_and_id( report_workspace ) if dataset_workspace is None: dataset_workspace = report_workspace_name - (dataset_workspace_name, dataset_workspace_id) = resolve_workspace_name_and_id( + dataset_workspace_name, dataset_workspace_id = resolve_workspace_name_and_id( dataset_workspace ) if isinstance(report, str): report = [report] for rpt in report: - (report_name, report_id) = resolve_item_name_and_id( + report_name, report_id = resolve_item_name_and_id( item=rpt, type="Report", workspace=report_workspace_id ) - (dataset_name, dataset_id) = resolve_item_name_and_id( + dataset_name, dataset_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=dataset_workspace ) @@ -110,7 +110,7 @@ def report_rebind_all( the new semantic model. """ - (dataset_name, dataset_id) = resolve_item_name_and_id( + dataset_name, dataset_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=dataset_workspace ) new_dataset_id = resolve_item_id( @@ -133,7 +133,7 @@ def report_rebind_all( & (dfR["Dataset Workspace Id"] == dataset_workspace_id) ] if dfR_filt.empty: - (wksp_name, _) = resolve_workspace_name_and_id(workspace=w) + wksp_name, _ = resolve_workspace_name_and_id(workspace=w) print( f"{icons.info} No reports found for the '{dataset_name}' semantic model within the '{wksp_name}' workspace." ) diff --git a/src/sempy_labs/report/_reportwrapper.py b/src/sempy_labs/report/_reportwrapper.py index 380a5713b..7aa2d843b 100644 --- a/src/sempy_labs/report/_reportwrapper.py +++ b/src/sempy_labs/report/_reportwrapper.py @@ -90,10 +90,10 @@ def __init__( readonly: bool = True, show_diffs: bool = True, ): - (self._workspace_name, self._workspace_id) = resolve_workspace_name_and_id( + self._workspace_name, self._workspace_id = resolve_workspace_name_and_id( workspace ) - (self._report_name, self._report_id) = resolve_item_name_and_id( + self._report_name, self._report_id = resolve_item_name_and_id( item=report, type="Report", workspace=self._workspace_id ) self._readonly = readonly @@ -515,7 +515,7 @@ def _resolve_page_name_and_display_name( The page name and display name. """ - (_, page_id, page_name) = self.__resolve_page_name_and_display_name_file_path( + _, page_id, page_name = self.__resolve_page_name_and_display_name_file_path( page, return_error=return_error, ) @@ -537,8 +537,8 @@ def resolve_page_name(self, page_display_name: str) -> str: The page name. """ - (path, page_id, page_name) = ( - self.__resolve_page_name_and_display_name_file_path(page_display_name) + path, page_id, page_name = self.__resolve_page_name_and_display_name_file_path( + page_display_name ) return page_id @@ -557,8 +557,8 @@ def resolve_page_display_name(self, page_name: str) -> str: The page display name. """ - (path, page_id, page_name) = ( - self.__resolve_page_name_and_display_name_file_path(page_name) + path, page_id, page_name = self.__resolve_page_name_and_display_name_file_path( + page_name ) return page_name @@ -1657,7 +1657,7 @@ def list_semantic_model_objects(self, extended: bool = False) -> pd.DataFrame: ) if extended: - (dataset_id, dataset_name, dataset_workspace_id, dataset_workspace_name) = ( + dataset_id, dataset_name, dataset_workspace_id, dataset_workspace_name = ( resolve_dataset_from_report( report=self._report_id, workspace=self._workspace_id ) @@ -1777,7 +1777,7 @@ def list_bookmarks(self) -> pd.DataFrame: apply_only_to_target_visuals = payload.get("options", {}).get( "applyOnlyToTargetVisuals", False ) - (page_id, page_display) = self._resolve_page_name_and_display_name( + page_id, page_display = self._resolve_page_name_and_display_name( page=rpt_page_id, return_error=False ) @@ -2059,9 +2059,7 @@ def set_active_page(self, page_name: str): """ self._ensure_pbir() - (page_id, page_display_name) = self._resolve_page_name_and_display_name( - page_name - ) + page_id, page_display_name = self._resolve_page_name_and_display_name(page_name) self.set_json( file_path=self._pages_file_path, json_path="$.activePageName", @@ -2106,7 +2104,7 @@ def set_page_type(self, page_name: str, page_type: str): f"{icons.red_dot} Invalid page_type parameter. Valid options: ['Tooltip', 'Letter', '4:3', '16:9']." ) - (file_path, page_id, page_display_name) = ( + file_path, page_id, page_display_name = ( self.__resolve_page_name_and_display_name_file_path(page_name) ) @@ -2133,7 +2131,7 @@ def set_page_visibility(self, page_name: str, hidden: bool): If set to False, makes the report page visible. """ self._ensure_pbir() - (file_path, page_id, page_display_name) = ( + file_path, page_id, page_display_name = ( self.__resolve_page_name_and_display_name_file_path(page_name) ) @@ -2687,7 +2685,7 @@ def _add_visual(self, page: str, payload: dict | bytes, generate_id: bool = True visual_file_copy["name"] = visual_id else: visual_id = visual_file_copy.get("name") - (page_file_path, page_id, page_name) = ( + page_file_path, page_id, page_name = ( self.__resolve_page_name_and_display_name_file_path(page) ) visual_file_path = helper.generate_visual_file_path(page_file_path, visual_id) @@ -2707,7 +2705,7 @@ def _add_new_visual( type = helper.resolve_visual_type(type) visual_id = generate_hex() - (page_file_path, page_id, page_name) = ( + page_file_path, page_id, page_name = ( self.__resolve_page_name_and_display_name_file_path(page) ) visual_file_path = helper.generate_visual_file_path(page_file_path, visual_id) diff --git a/src/sempy_labs/report/_save_report.py b/src/sempy_labs/report/_save_report.py index ddd14fec8..e40a5983f 100644 --- a/src/sempy_labs/report/_save_report.py +++ b/src/sempy_labs/report/_save_report.py @@ -52,10 +52,10 @@ def save_report_as_pbip( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (report_workspace_name, report_workspace_id) = resolve_workspace_name_and_id( + report_workspace_name, report_workspace_id = resolve_workspace_name_and_id( workspace ) - (report_name, report_id) = resolve_item_name_and_id( + report_name, report_id = resolve_item_name_and_id( item=report, type="Report", workspace=workspace ) indent = 2 diff --git a/src/sempy_labs/semantic_model/_caching.py b/src/sempy_labs/semantic_model/_caching.py index de317f828..f59fff440 100644 --- a/src/sempy_labs/semantic_model/_caching.py +++ b/src/sempy_labs/semantic_model/_caching.py @@ -31,8 +31,8 @@ def enable_query_caching( Set to True to enable query caching, or False to disable it. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (item_name, item_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + item_name, item_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=workspace_id ) model_id = get_model_id(item_id=item_id) diff --git a/src/sempy_labs/semantic_model/_copilot.py b/src/sempy_labs/semantic_model/_copilot.py index 67be89e10..068b945e1 100644 --- a/src/sempy_labs/semantic_model/_copilot.py +++ b/src/sempy_labs/semantic_model/_copilot.py @@ -31,8 +31,8 @@ def approved_for_copilot( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (item_name, item_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + item_name, item_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=workspace_id ) payload = {"preppedForCopilot": approved_for_copilot, "isReadOnly": False} @@ -70,8 +70,8 @@ def set_endorsement( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (item_name, item_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + item_name, item_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=workspace_id ) id = get_model_id(item_id=item_id) @@ -123,8 +123,8 @@ def make_discoverable( or if no lakehouse attached, resolves to the workspace of the notebook. """ - (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace) - (item_name, item_id) = resolve_item_name_and_id( + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + item_name, item_id = resolve_item_name_and_id( item=dataset, type="SemanticModel", workspace=workspace_id ) id = get_model_id(item_id=item_id) diff --git a/src/sempy_labs/semantic_model/_dax_optimization.py b/src/sempy_labs/semantic_model/_dax_optimization.py new file mode 100644 index 000000000..6079fbe85 --- /dev/null +++ b/src/sempy_labs/semantic_model/_dax_optimization.py @@ -0,0 +1,841 @@ +"""DAX performance optimization rule engine. + +This module evaluates a set of declarative optimization rules (defined in +``_dax_optimization_rules.json``) against the artifacts produced by the +interactive DAX test widget: + +* the DAX query text, +* the semantic model metadata (from TOM), +* the query dependencies (referenced tables/columns), +* the trace details (Formula/Storage Engine timings and events), +* the DAX query plan (logical/physical), and +* (optionally) Vertipaq Analyzer statistics -- only used when the column + cardinalities for ``Data`` columns are not all ``1`` (i.e. there is + something meaningful to analyze). + +The rules are intentionally data-driven so the catalog can be reviewed and +extended without changing the evaluation code. See the ``dax-optimization`` +skill for the methodology behind each rule. +""" + +from __future__ import annotations + +import json +import os +import re +from typing import Any, Optional + +import pandas as pd + +_RULES_FILE = os.path.join(os.path.dirname(__file__), "_dax_optimization_rules.json") + +# Iterator (row-by-row) DAX functions. Used to flag nested/expensive iteration. +_ITERATOR_FUNCTIONS = [ + "SUMX", + "AVERAGEX", + "MINX", + "MAXX", + "COUNTX", + "COUNTAX", + "PRODUCTX", + "CONCATENATEX", + "RANKX", + "MEDIANX", + "GEOMEANX", + "FILTER", + "ADDCOLUMNS", + "GENERATE", + "GENERATEALL", + "TOPN", +] + + +def _load_rules() -> dict: + """Load the optimization rules catalog from the packaged JSON file. + + Returns an empty (but well-formed) catalog if the file cannot be read so + the caller can degrade gracefully rather than raising in the UI.""" + + try: + with open(_RULES_FILE, "r", encoding="utf-8") as f: + data = json.load(f) + if not isinstance(data, dict) or "rules" not in data: + return {"rules": [], "severity_order": []} + return data + except Exception: + return {"rules": [], "severity_order": []} + + +# --------------------------------------------------------------------------- # +# Condition evaluation +# --------------------------------------------------------------------------- # +def _compare(actual: Any, op: str, expected: Any) -> bool: + """Evaluate a single comparison ``actual expected`` defensively. + + Unknown operators and type errors evaluate to ``False`` so a malformed + rule can never crash the analysis.""" + + try: + if op in (">", ">=", "<", "<="): + a = float(actual) + b = float(expected) + if op == ">": + return a > b + if op == ">=": + return a >= b + if op == "<": + return a < b + return a <= b + if op == "==": + return actual == expected + if op == "!=": + return actual != expected + if op == "contains": + return str(expected).lower() in str(actual).lower() + if op == "not_contains": + return str(expected).lower() not in str(actual).lower() + if op == "regex": + return re.search(str(expected), str(actual), re.IGNORECASE) is not None + if op == "in": + return actual in (expected or []) + if op == "not_in": + return actual not in (expected or []) + except Exception: + return False + return False + + +def _eval_condition(cond: Any, ctx: dict) -> bool: + """Recursively evaluate a condition tree against a context dict. + + Supports composites ``all`` / ``any`` / ``not`` and leaf comparisons that + reference either a ``metric`` (scalar rules) or a ``field`` (per-item + ``for_each`` rules).""" + + if not isinstance(cond, dict): + return False + if "all" in cond: + return all(_eval_condition(c, ctx) for c in cond["all"]) + if "any" in cond: + return any(_eval_condition(c, ctx) for c in cond["any"]) + if "not" in cond: + return not _eval_condition(cond["not"], ctx) + key = cond.get("metric", cond.get("field")) + if key is None: + return False + return _compare(ctx.get(key), cond.get("op", "=="), cond.get("value")) + + +def _render(template: str, ctx: dict) -> str: + """Render ``{placeholder}`` tokens in ``template`` from ``ctx``. + + Missing placeholders are left as-is rather than raising, keeping the + message readable even if a rule references a value that wasn't computed.""" + + def _sub(m: "re.Match") -> str: + name = m.group(1) + if name in ctx and ctx[name] is not None: + return str(ctx[name]) + return m.group(0) + + try: + return re.sub(r"\{([a-zA-Z0-9_]+)\}", _sub, template or "") + except Exception: + return template or "" + + +# --------------------------------------------------------------------------- # +# Query text heuristics +# --------------------------------------------------------------------------- # +def _strip_strings_and_comments(dax: str) -> str: + """Return the DAX text with string literals and comments removed so that + keyword/operator heuristics don't match inside strings or comments.""" + + if not dax: + return "" + # Remove block comments, line comments, then double-quoted strings. + no_block = re.sub(r"/\*.*?\*/", " ", dax, flags=re.DOTALL) + no_line = re.sub(r"//[^\n]*", " ", no_block) + no_str = re.sub(r'"(?:[^"]|"")*"', '""', no_line) + return no_str + + +def _count_iterators(dax_clean: str) -> int: + total = 0 + for fn in _ITERATOR_FUNCTIONS: + total += len(re.findall(rf"\b{fn}\s*\(", dax_clean, re.IGNORECASE)) + return total + + +def _has_nested_iterator(dax_clean: str) -> bool: + """Detect whether any iterator function is invoked inside the argument + list of another iterator function (parenthesis-depth aware).""" + + upper = dax_clean.upper() + iter_pattern = re.compile( + r"\b(" + "|".join(_ITERATOR_FUNCTIONS) + r")\s*\(", re.IGNORECASE + ) + # Stack of paren depths at which an iterator call was opened. + open_iter_depths: list = [] + depth = 0 + i = 0 + n = len(upper) + while i < n: + ch = upper[i] + m = iter_pattern.match(upper, i) + if m: + # An iterator opens here. If we're already inside another + # iterator's parentheses, this is a nested iterator. + if open_iter_depths: + return True + # Advance to the '(' and record the depth it opens at. + paren_pos = m.end() - 1 + depth += 1 + open_iter_depths.append(depth) + i = paren_pos + 1 + continue + if ch == "(": + depth += 1 + elif ch == ")": + if open_iter_depths and depth == open_iter_depths[-1]: + open_iter_depths.pop() + depth = max(depth - 1, 0) + i += 1 + return False + + +def _classify_filter_predicate( + predicate: str, measure_names: Optional[set] +) -> Optional[str]: + """Classify the predicate (second argument) of a ``FILTER(
, ...)`` + call as ``"measure"`` or ``"column"`` (or ``None`` when it references + neither). + + A predicate is classified as ``"measure"`` when it references a measure + (resolved from ``measure_names`` when model metadata is available, otherwise + inferred from an unqualified ``[...]`` reference). Otherwise, when it + references a column it is classified as ``"column"``. Measure references take + precedence, so a FILTER that evaluates a measure is never reported as a + simple column predicate.""" + + measure_names = measure_names or set() + has_measure = False + has_column = False + for mm in re.finditer(r"\[([^\]]+)\]", predicate): + name = mm.group(1) + start = mm.start() + prev = predicate[start - 1] if start > 0 else "" + qualified = bool(re.match(r"[A-Za-z0-9_')\]]", prev)) + if measure_names: + if name.lower() in measure_names: + has_measure = True + else: + has_column = True + else: + if qualified: + has_column = True + else: + has_measure = True + if has_measure: + return "measure" + if has_column: + return "column" + return None + + +def _filter_table_predicate_kinds( + dax_clean: str, measure_names: Optional[set] = None +) -> list: + """Return the classification (``"measure"`` / ``"column"``) of each + ``FILTER(, )`` call in ``dax_clean``. + + Only FILTER calls whose first argument is a whole-table reference (a bare or + quoted table name) and that have exactly two arguments are considered; other + FILTER usages are ignored.""" + + measure_names = measure_names or set() + kinds: list = [] + for m in re.finditer(r"\bFILTER\s*\(", dax_clean, re.IGNORECASE): + open_idx = m.end() - 1 + close_idx = _match_paren(dax_clean, open_idx) + if close_idx == -1: + continue + inner = dax_clean[open_idx + 1 : close_idx] + args = _split_top_level_args(inner) + if len(args) != 2: + continue + arg1 = args[0].strip() + arg2 = args[1].strip() + is_table = bool( + re.fullmatch(r"'(?:[^']|'')+'", arg1) + or re.fullmatch(r"[A-Za-z_][A-Za-z0-9_ ]*", arg1) + ) + if not is_table: + continue + kind = _classify_filter_predicate(arg2, measure_names) + if kind: + kinds.append(kind) + return kinds + + +def _count_filter_full_table( + dax_clean: str, measure_names: Optional[set] = None +) -> int: + """Count ``FILTER(, )`` occurrences whose predicate + evaluates a **measure** (e.g. ``FILTER(Sales, [Total Qty] > 100)``). + + Iterating an entire table only to test a measure forces a full scan and + row-by-row Formula Engine evaluation of that measure. FILTER calls whose + predicate is a simple column comparison are reported separately by + :func:`_count_filter_column_predicate` instead.""" + + return sum( + 1 + for k in _filter_table_predicate_kinds(dax_clean, measure_names) + if k == "measure" + ) + + +def _match_paren(s: str, open_idx: int) -> int: + """Return the index of the ``)`` matching the ``(`` at ``open_idx`` in + ``s``, or ``-1`` if it is unbalanced.""" + + depth = 0 + for i in range(open_idx, len(s)): + c = s[i] + if c == "(": + depth += 1 + elif c == ")": + depth -= 1 + if depth == 0: + return i + return -1 + + +def _split_top_level_args(s: str) -> list: + """Split a comma-separated argument list on top-level (depth-0) commas.""" + + args: list = [] + depth = 0 + cur: list = [] + for c in s: + if c == "(": + depth += 1 + cur.append(c) + elif c == ")": + depth -= 1 + cur.append(c) + elif c == "," and depth == 0: + args.append("".join(cur)) + cur = [] + else: + cur.append(c) + args.append("".join(cur)) + return args + + +def _count_filter_column_predicate( + dax_clean: str, measure_names: Optional[set] = None +) -> int: + """Count ``FILTER(, )`` occurrences that + could be rewritten with ``KEEPFILTERS()``. + + Matches the inefficient pattern ``CALCULATE([M], FILTER(Customer, + Customer[Category] = "A"))`` where the FILTER iterates an entire table only + to apply a boolean predicate over its columns. The faster equivalent is + ``CALCULATE([M], KEEPFILTERS(Customer[Category] = "A"))``, which lets the + engine push the predicate down instead of materializing the whole table. + FILTER calls whose predicate evaluates a measure are reported by + :func:`_count_filter_full_table` instead.""" + + return sum( + 1 + for k in _filter_table_predicate_kinds(dax_clean, measure_names) + if k == "column" + ) + + +def _build_measure_expr_map(model_tree: Optional[list]) -> dict: + """Map ``measure name (lowercased) -> DAX expression`` from the model tree. + + Used to expand the analysis to the DAX of dependent measures, so the + query-text heuristics also inspect logic that lives inside referenced + measures rather than only the outer query text.""" + + out: dict = {} + for table in model_tree or []: + for m in (table.get("measures") or []) if isinstance(table, dict) else []: + name = str(m.get("name", "") or "") + expr = str(m.get("expression", "") or "") + if name: + out[name.lower()] = expr + return out + + +def _extract_bracket_names(text: str) -> set: + """Return the set of names inside ``[...]`` references in a DAX text (both + measure references like ``[Sales]`` and column references like + ``Table[Column]`` yield their bracketed name).""" + + return set(re.findall(r"\[([^\]]+)\]", text or "")) + + +def _collect_dependent_measure_expressions( + dax_query: str, model_tree: Optional[list] +) -> str: + """Return the concatenated DAX expressions of every measure transitively + referenced by ``dax_query``. + + Starting from the measures referenced directly in the query, this walks the + measure dependency graph (a measure's expression may reference further + measures) and gathers each expression exactly once. The result lets the + syntax-level rules (e.g. ``FILTER_COLUMN_USE_KEEPFILTERS``, ``USES_IFERROR``) + detect issues that live inside dependent measures used by the query rather + than in the query text itself. Returns an empty string when no model tree is + available or the query references no measures.""" + + measure_map = _build_measure_expr_map(model_tree) + if not measure_map: + return "" + + seen: set = set() + collected: list = [] + # Seed with measures referenced directly by the query text. + queue = [ + name + for name in _extract_bracket_names(_strip_strings_and_comments(dax_query or "")) + if name.lower() in measure_map + ] + while queue: + name = queue.pop() + key = name.lower() + if key in seen: + continue + seen.add(key) + expr = measure_map.get(key, "") + if not expr: + continue + collected.append(expr) + # Expand any further measures referenced inside this expression. + for ref in _extract_bracket_names(_strip_strings_and_comments(expr)): + if ref.lower() in measure_map and ref.lower() not in seen: + queue.append(ref) + return "\n".join(collected) + + +# --------------------------------------------------------------------------- # +# Metric / collection construction +# --------------------------------------------------------------------------- # +def _vertipaq_columns_df(vertipaq: Optional[dict]) -> Optional[pd.DataFrame]: + """Return the Vertipaq 'Columns' dataframe from the analyzer result dict, + tolerating either the 'Columns' or 'Column' key.""" + + if not isinstance(vertipaq, dict): + return None + for key in ("Columns", "Column"): + df = vertipaq.get(key) + if isinstance(df, pd.DataFrame) and not df.empty: + return df + return None + + +def _is_floating_point(data_type: str) -> bool: + dt = str(data_type or "").lower() + return "double" in dt or "float" in dt + + +def build_context( + *, + dax_query: str = "", + trace_rows: Optional[list] = None, + total_duration_ms: int = 0, + fe_duration_ms: int = 0, + se_duration_ms: int = 0, + cpu_time_ms: int = 0, + query_plan_rows: Optional[list] = None, + dependency_columns: Optional[list] = None, + model_tree: Optional[list] = None, + vertipaq: Optional[dict] = None, + cold_cache: bool = True, +) -> dict: + """Build the ``(metrics, collections)`` evaluation context from the raw + widget artifacts. + + Returns a dict with ``"metrics"`` (flat dict for scalar rules) and + ``"collections"`` (named lists of item dicts for ``for_each`` rules), plus + a small ``"inputs"`` summary describing which artifacts were available. + """ + + trace_rows = trace_rows or [] + query_plan_rows = query_plan_rows or [] + dependency_columns = dependency_columns or [] + model_tree = model_tree or [] + + dax_clean = _strip_strings_and_comments(dax_query or "") + + # Extend the syntax-level analysis to the DAX of dependent measures used by + # the query, so rules such as FILTER_COLUMN_USE_KEEPFILTERS / USES_IFERROR + # also flag problems that live inside referenced measures (the query itself + # often only references a measure by name). Falls back to the query text + # alone when no model tree is available. + measure_expr_text = _collect_dependent_measure_expressions(dax_query, model_tree) + analysis_clean = ( + dax_clean + if not measure_expr_text + else dax_clean + "\n" + _strip_strings_and_comments(measure_expr_text) + ) + # Known measure names (lowercased) used to tell whether a FILTER predicate + # evaluates a measure or a column. + measure_names = set(_build_measure_expr_map(model_tree).keys()) + + # ---- Query text metrics ---- + iterator_count = _count_iterators(analysis_clean) + metrics: dict = { + "has_query": bool((dax_query or "").strip()), + "query_length": len(dax_query or ""), + "iterator_count": iterator_count, + "nested_iterator": _has_nested_iterator(analysis_clean), + "uses_iferror": bool( + re.search(r"\b(IFERROR|ISERROR)\s*\(", analysis_clean, re.IGNORECASE) + ), + "uses_divide_function": bool( + re.search(r"\bDIVIDE\s*\(", analysis_clean, re.IGNORECASE) + ), + "uses_division_operator": bool(re.search(r"(? 0 else 0.0, + "fe_pct": (fe / total) if total > 0 else 0.0, + "se_query_count": len(se_non_internal), + "se_internal_count": len(se_internal), + "se_cache_match_count": len(cache_matches), + "se_cpu_ms": se_cpu_ms, + "se_parallelism": (se_cpu_ms / se) if se > 0 else 0.0, + } + ) + + # ---- Query plan metrics ---- + physical_text = "\n".join( + str(r.get("text", "") or "") + for r in query_plan_rows + if str(r.get("plan_type", "")).lower() == "physical" + ) + if not physical_text: + # Fall back to all plan text if plan_type wasn't classified. + physical_text = "\n".join(str(r.get("text", "") or "") for r in query_plan_rows) + metrics.update( + { + "has_query_plan": bool(query_plan_rows), + "callback_dataid_count": len( + re.findall(r"CallbackDataID", physical_text, re.IGNORECASE) + ), + "encode_callback_count": len( + re.findall(r"EncodeCallback", physical_text, re.IGNORECASE) + ), + "spool_count": len(re.findall(r"\bSpool\b", physical_text, re.IGNORECASE)), + } + ) + + # ---- Dependency metrics ---- + ref_pairs = set() + for c in dependency_columns: + t = str(c.get("table", "") or "") + col = str(c.get("column", "") or "") + if t or col: + ref_pairs.add((t, col)) + metrics["referenced_column_count"] = len(ref_pairs) + metrics["referenced_table_count"] = len({t for (t, _) in ref_pairs}) + metrics["has_dependencies"] = bool(ref_pairs) + + # ---- Vertipaq metrics & collections ---- + # Vertipaq stats are only used when the Data columns are not all + # cardinality 1 (i.e. there is something meaningful to analyze). + collections: dict = {"slow_se_queries": [], "high_cardinality_columns": []} + vertipaq_available = False + vertipaq_skipped_trivial = False + max_data_card = 0 + high_card_count = 0 + cols_df = _vertipaq_columns_df(vertipaq) + if cols_df is not None: + df = cols_df + type_col = "Type" if "Type" in df.columns else None + card_col = "Cardinality" if "Cardinality" in df.columns else None + if card_col: + data_df = df + if type_col: + data_df = df[df[type_col].astype(str).str.lower() == "data"] + cardinalities = pd.to_numeric(data_df[card_col], errors="coerce").fillna(0) + max_data_card = int(cardinalities.max()) if len(cardinalities) else 0 + all_ones = len(cardinalities) > 0 and bool((cardinalities <= 1).all()) + if len(cardinalities) == 0 or all_ones: + # Nothing meaningful to analyze -> skip vertipaq rules. + vertipaq_skipped_trivial = True + else: + vertipaq_available = True + # Focus on columns referenced by the query when dependency + # info is available; otherwise consider all data columns. + items = [] + for _, row in data_df.iterrows(): + t = str(row.get("Table Name", "") or "") + col = str(row.get("Column Name", "") or "") + if ref_pairs and (t, col) not in ref_pairs: + continue + try: + card = int( + pd.to_numeric(row.get(card_col), errors="coerce") or 0 + ) + except (TypeError, ValueError): + card = 0 + dtype = str(row.get("Data Type", "") or "") + items.append( + { + "table": t, + "column": col, + "cardinality": card, + "cardinality_display": f"{card:,}", + "data_type": dtype, + "is_floating_point": _is_floating_point(dtype), + "data_size": int( + pd.to_numeric(row.get("Data Size"), errors="coerce") + or 0 + ), + "encoding": str(row.get("Encoding", "") or ""), + } + ) + if card >= 1000000: + high_card_count += 1 + items.sort(key=lambda x: x["cardinality"], reverse=True) + collections["high_cardinality_columns"] = items + metrics.update( + { + "vertipaq_available": vertipaq_available, + "vertipaq_skipped_trivial": vertipaq_skipped_trivial, + "max_data_column_cardinality": max_data_card, + "high_cardinality_data_column_count": high_card_count, + } + ) + + # ---- Slow SE query collection ---- + slow = [] + for r in se_non_internal: + try: + dur = int(r.get("duration", 0) or 0) + except (TypeError, ValueError): + dur = 0 + try: + cpu = int(r.get("cpu", 0) or 0) + except (TypeError, ValueError): + cpu = 0 + slow.append( + { + "subclass": str(r.get("event_subclass", "") or ""), + "duration": dur, + "cpu": cpu, + } + ) + slow.sort(key=lambda x: x["duration"], reverse=True) + collections["slow_se_queries"] = slow[:10] + + # ---- Display-friendly derived values for message templates ---- + metrics["se_pct_display"] = f"{metrics['se_pct'] * 100:.0f}%" + metrics["fe_pct_display"] = f"{metrics['fe_pct'] * 100:.0f}%" + metrics["se_parallelism_display"] = f"{metrics['se_parallelism']:.1f}x" + + inputs = { + "query": metrics["has_query"], + "trace": has_trace, + "query_plan": metrics["has_query_plan"], + "dependencies": metrics["has_dependencies"], + "vertipaq": vertipaq_available, + "model": bool(model_tree), + } + + return {"metrics": metrics, "collections": collections, "inputs": inputs} + + +# --------------------------------------------------------------------------- # +# Rule evaluation +# --------------------------------------------------------------------------- # +def _requirements_met(rule: dict, inputs: dict) -> bool: + """Return True only if every artifact listed in the rule's ``requires`` is + available, so rules don't fire (or report misleading 'no issue') on + missing data. The two diagnostics rules deliberately have permissive + requirements and handle availability in their conditions.""" + + for req in rule.get("requires", []) or []: + if not inputs.get(req, False): + return False + return True + + +def evaluate_rules(context: dict, rules_catalog: Optional[dict] = None) -> list: + """Evaluate all rules against a context produced by :func:`build_context`. + + Returns a list of finding dicts ``{id, title, category, severity, message, + recommendation, references, evidence}`` ordered by severity. + """ + + catalog = rules_catalog or _load_rules() + rules = catalog.get("rules", []) + severity_order = catalog.get("severity_order", ["high", "medium", "low", "info"]) + metrics = context.get("metrics", {}) + collections = context.get("collections", {}) + inputs = context.get("inputs", {}) + + findings: list = [] + for rule in rules: + if not _requirements_met(rule, inputs): + continue + kind = rule.get("kind", "scalar") + if kind == "for_each": + coll = collections.get(rule.get("collection", ""), []) + where = rule.get("where") + max_findings = int(rule.get("max_findings", 5)) + matched = 0 + for item in coll: + if where is not None and not _eval_condition(where, item): + continue + ctx = dict(metrics) + ctx.update(item) + findings.append( + { + "id": rule.get("id", ""), + "title": rule.get("title", ""), + "category": rule.get("category", ""), + "severity": rule.get("severity", "info"), + "message": _render(rule.get("message", ""), ctx), + "recommendation": rule.get("recommendation", ""), + "references": rule.get("references", []), + "evidence": item, + } + ) + matched += 1 + if matched >= max_findings: + break + else: + cond = rule.get("condition") + if cond is not None and not _eval_condition(cond, metrics): + continue + if cond is None: + continue + findings.append( + { + "id": rule.get("id", ""), + "title": rule.get("title", ""), + "category": rule.get("category", ""), + "severity": rule.get("severity", "info"), + "message": _render(rule.get("message", ""), metrics), + "recommendation": rule.get("recommendation", ""), + "references": rule.get("references", []), + "evidence": None, + } + ) + + def _sev_key(f: dict) -> int: + try: + return severity_order.index(f.get("severity", "info")) + except ValueError: + return len(severity_order) + + findings.sort(key=_sev_key) + return findings + + +def analyze_dax_performance( + *, + dax_query: str = "", + trace_rows: Optional[list] = None, + total_duration_ms: int = 0, + fe_duration_ms: int = 0, + se_duration_ms: int = 0, + cpu_time_ms: int = 0, + query_plan_rows: Optional[list] = None, + dependency_columns: Optional[list] = None, + model_tree: Optional[list] = None, + vertipaq: Optional[dict] = None, + cold_cache: bool = True, +) -> dict: + """Run the full DAX performance analysis. + + Builds the evaluation context from the supplied artifacts, evaluates the + rule catalog, and returns a result dict with ``findings``, a ``summary`` + (severity counts + engine balance), the computed ``metrics`` and the + ``inputs`` availability map. This is the single entry point used by the + interactive DAX test widget. + """ + + context = build_context( + dax_query=dax_query, + trace_rows=trace_rows, + total_duration_ms=total_duration_ms, + fe_duration_ms=fe_duration_ms, + se_duration_ms=se_duration_ms, + cpu_time_ms=cpu_time_ms, + query_plan_rows=query_plan_rows, + dependency_columns=dependency_columns, + model_tree=model_tree, + vertipaq=vertipaq, + cold_cache=cold_cache, + ) + findings = evaluate_rules(context) + metrics = context["metrics"] + + severity_counts: dict = {} + for f in findings: + sev = f.get("severity", "info") + severity_counts[sev] = severity_counts.get(sev, 0) + 1 + + summary = { + "total_findings": len(findings), + "severity_counts": severity_counts, + "total_duration_ms": metrics.get("total_duration_ms", 0), + "fe_duration_ms": metrics.get("fe_duration_ms", 0), + "se_duration_ms": metrics.get("se_duration_ms", 0), + "fe_pct": metrics.get("fe_pct", 0.0), + "se_pct": metrics.get("se_pct", 0.0), + } + + return { + "findings": findings, + "summary": summary, + "metrics": metrics, + "inputs": context["inputs"], + } diff --git a/src/sempy_labs/semantic_model/_dax_optimization_rules.json b/src/sempy_labs/semantic_model/_dax_optimization_rules.json new file mode 100644 index 000000000..310c9d0d3 --- /dev/null +++ b/src/sempy_labs/semantic_model/_dax_optimization_rules.json @@ -0,0 +1,325 @@ +{ + "schema_version": 1, + "description": "Declarative DAX performance optimization rules executed by sempy_labs.semantic_model._dax_optimization. Each rule is evaluated against a flat 'metrics' dictionary (scalar rules) or a named 'collection' of items (for_each rules) derived from the DAX query, semantic model metadata (TOM), query dependencies, trace details, the DAX query plan and (optionally) Vertipaq Analyzer statistics. See the dax-optimization skill for the methodology behind these rules.", + "severity_order": ["high", "medium", "low", "info"], + "rules": [ + { + "id": "SE_BOUND", + "title": "Query is Storage Engine bound", + "category": "Engine balance", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "total_duration_ms", "op": ">=", "value": 50}, + {"metric": "se_pct", "op": ">=", "value": 0.7} + ] + }, + "message": "The Storage Engine accounts for {se_pct_display} of the {total_duration_ms} ms total duration. The query spends most of its time scanning data.", + "recommendation": "Reduce the amount of data scanned by the Storage Engine: lower column cardinality, filter earlier, avoid scanning high-cardinality columns, and ensure relationships allow VertiPaq to prune partitions. Confirm the slow xmSQL scans in the trace correspond to large or high-cardinality tables.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/", + "https://learn.microsoft.com/power-bi/guidance/dax-variables" + ] + }, + { + "id": "FE_BOUND", + "title": "Query is Formula Engine bound", + "category": "Engine balance", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "total_duration_ms", "op": ">=", "value": 50}, + {"metric": "fe_pct", "op": ">=", "value": 0.7} + ] + }, + "message": "The Formula Engine accounts for {fe_pct_display} of the {total_duration_ms} ms total duration. The query spends most of its time in single-threaded Formula Engine work.", + "recommendation": "The Formula Engine is single-threaded and cannot be parallelized. Push more work to the Storage Engine: replace row-by-row logic with set-based DAX, avoid forcing CallbackDataID (e.g. IF/IFERROR inside iterators), reduce large intermediate materializations, and prefer aggregations that the Storage Engine can resolve directly.", + "references": [ + "https://www.sqlbi.com/articles/optimizing-the-use-of-variables-in-dax/", + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "MANY_SE_QUERIES", + "title": "High number of Storage Engine queries", + "category": "Storage Engine", + "severity": "medium", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "metric": "se_query_count", "op": ">=", "value": 10 + }, + "message": "The query generated {se_query_count} Storage Engine queries (excluding internal sub-queries). A large number of separate scans usually means the engine could not fuse them into fewer, larger scans.", + "recommendation": "Aim for fewer Storage Engine queries. Multiple scans of the same table often indicate complex filter context transitions, non-foldable logic, or iterators producing one SE query per row group. Simplify measure logic, use variables to compute a base value once, and avoid patterns that defeat VertiPaq fusion.", + "references": [ + "https://www.sqlbi.com/articles/introducing-vertipaq-fusion/" + ] + }, + { + "id": "CALLBACK_DATA_ID", + "title": "CallbackDataID detected in the physical query plan", + "category": "Query plan", + "severity": "high", + "requires": ["query_plan"], + "kind": "scalar", + "condition": { + "metric": "callback_dataid_count", "op": ">=", "value": 1 + }, + "message": "The physical query plan contains {callback_dataid_count} CallbackDataID operation(s). CallbackDataID means the Storage Engine had to call back into the Formula Engine during a scan, which is slow and disables many VertiPaq optimizations.", + "recommendation": "Remove the constructs that force CallbackDataID. Common causes: IF / IFERROR / DIVIDE-with-alternate / complex conditional logic, error handling, date/time arithmetic, and rounding inside an iterator that runs in the Storage Engine. Move conditional logic out of the row context, precompute values with variables, or use calculated columns where appropriate.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/", + "https://www.sqlbi.com/tv/callbackdataid-in-dax/" + ] + }, + { + "id": "LOW_SE_PARALLELISM", + "title": "Low Storage Engine parallelism", + "category": "Storage Engine", + "severity": "low", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "se_duration_ms", "op": ">=", "value": 100}, + {"metric": "se_parallelism", "op": "<", "value": 1.2} + ] + }, + "message": "Storage Engine parallelism is {se_parallelism_display} (SE CPU {se_cpu_ms} ms over {se_duration_ms} ms of SE duration). The Storage Engine scans are running mostly single-threaded.", + "recommendation": "Low parallelism limits the benefit of VertiPaq's multi-threaded engine. This often happens with many small sequential scans or scans serialized behind Formula Engine callbacks. Reduce CallbackDataID, consolidate scans, and ensure tables are large enough to benefit from segment-level parallelism.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "LARGE_SPOOL", + "title": "Large intermediate spools in the physical plan", + "category": "Query plan", + "severity": "medium", + "requires": ["query_plan"], + "kind": "scalar", + "condition": { + "metric": "spool_count", "op": ">=", "value": 4 + }, + "message": "The physical query plan contains {spool_count} Spool operations. Spools materialize intermediate results in the Formula Engine and can consume significant memory and time when they are large.", + "recommendation": "Reduce the number and size of materialized intermediate results. Use variables to avoid recomputing the same sub-expression, filter before joining, and avoid producing large datacaches that the Formula Engine must iterate over.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "USES_IFERROR", + "title": "IFERROR / ISERROR forces row-by-row evaluation", + "category": "DAX syntax", + "severity": "high", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "uses_iferror", "op": "==", "value": true + }, + "message": "The query uses IFERROR or ISERROR. These functions force the engine to evaluate the wrapped expression row by row and almost always introduce CallbackDataID, preventing Storage Engine optimization.", + "recommendation": "Avoid IFERROR/ISERROR for performance-sensitive logic. For division use DIVIDE(numerator, denominator) which safely handles divide-by-zero without error trapping. Validate inputs with explicit conditions instead of catching errors.", + "references": [ + "https://www.sqlbi.com/articles/error-handling-in-dax-measures/", + "https://learn.microsoft.com/dax/divide-function-dax" + ] + }, + { + "id": "DIVISION_WITHOUT_DIVIDE", + "title": "Division operator used without DIVIDE", + "category": "DAX syntax", + "severity": "low", + "requires": ["query"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "uses_division_operator", "op": "==", "value": true}, + {"metric": "uses_divide_function", "op": "==", "value": false} + ] + }, + "message": "The query uses the '/' division operator but does not use the DIVIDE function. A raw '/' can raise errors on divide-by-zero, which then tend to be wrapped in IFERROR and slow the query.", + "recommendation": "Replace 'a / b' with DIVIDE(a, b). DIVIDE returns BLANK (or a supplied alternate) on divide-by-zero without the overhead of error handling.", + "references": [ + "https://learn.microsoft.com/dax/divide-function-dax" + ] + }, + { + "id": "FILTER_FULL_TABLE", + "title": "FILTER over an entire table to evaluate a measure", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "filter_full_table_count", "op": ">=", "value": 1 + }, + "message": "The query iterates a full table with FILTER to evaluate a measure {filter_full_table_count} time(s) (e.g. FILTER(Sales, [Total Qty] > 100)). Testing a measure for every row of an entire table forces a full scan and row-by-row Formula Engine evaluation of the measure.", + "recommendation": "Iterate the smallest set needed instead of the whole table. Filter on the distinct values that drive the measure (e.g. FILTER(VALUES(Sales[OrderId]), [Total Qty] > 100)), or restructure the logic so the measure is evaluated once per group rather than per row. When the predicate is a simple column comparison, use KEEPFILTERS(Table[Column] = value) instead of FILTER.", + "references": [ + "https://www.sqlbi.com/articles/filter-arguments-in-calculate/", + "https://learn.microsoft.com/dax/best-practices/dax-avoid-converting-blank" + ] + }, + { + "id": "FILTER_COLUMN_USE_KEEPFILTERS", + "title": "FILTER over a table for a simple column predicate", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "filter_column_predicate_count", "op": ">=", "value": 1 + }, + "message": "The query wraps a simple column predicate in FILTER(Table, ...) {filter_column_predicate_count} time(s) (e.g. CALCULATE([Qty], FILTER(Customer, Customer[Category] = \"A\"))). This materializes the entire table just to apply a single boolean condition.", + "recommendation": "Replace FILTER(Table, Table[Column] = value) with KEEPFILTERS(Table[Column] = value) so the engine pushes the predicate down to the Storage Engine instead of iterating the whole table. For example, rewrite CALCULATE([Qty], FILTER(Customer, Customer[Category] = \"A\")) as CALCULATE([Qty], KEEPFILTERS(Customer[Category] = \"A\")). A bare column predicate inside CALCULATE (without KEEPFILTERS) is also faster but replaces, rather than intersects, the existing filter context.", + "references": [ + "https://www.sqlbi.com/articles/filter-arguments-in-calculate/", + "https://www.sqlbi.com/articles/using-keepfilters-in-dax/" + ] + }, + { + "id": "NESTED_ITERATORS", + "title": "Nested iterators over potentially large tables", + "category": "DAX syntax", + "severity": "medium", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "nested_iterator", "op": "==", "value": true + }, + "message": "The query contains nested iterator functions (e.g. SUMX inside SUMX). Nested iteration multiplies the number of rows evaluated and is a common cause of slow, Formula-Engine-bound queries.", + "recommendation": "Flatten nested iterators where possible. Compute inner aggregations once with variables, push aggregation into the Storage Engine with simple SUMX/AVERAGEX over a single table, and avoid iterating one large table inside another.", + "references": [ + "https://www.sqlbi.com/articles/nested-iterators-in-dax/" + ] + }, + { + "id": "MANY_ITERATORS", + "title": "Many iterator functions in the query", + "category": "DAX syntax", + "severity": "low", + "requires": ["query"], + "kind": "scalar", + "condition": { + "metric": "iterator_count", "op": ">=", "value": 5 + }, + "message": "The query uses {iterator_count} iterator functions (SUMX, AVERAGEX, FILTER, etc.). A high count of iterators increases the chance of row-by-row Formula Engine work.", + "recommendation": "Review each iterator and replace those that can be expressed as set-based aggregations. Where an iterator only sums a single column, a simple SUM over the column is faster and Storage-Engine friendly.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "MANY_REFERENCED_COLUMNS", + "title": "Query references a large number of columns", + "category": "Model design", + "severity": "low", + "requires": ["dependencies"], + "kind": "scalar", + "condition": { + "metric": "referenced_column_count", "op": ">=", "value": 15 + }, + "message": "The query depends on {referenced_column_count} columns across {referenced_table_count} table(s). Wide queries scan more dictionaries and produce larger datacaches.", + "recommendation": "Confirm every referenced column is required. Removing unused columns from the query (and from SUMMARIZECOLUMNS/SELECTCOLUMNS projections) reduces the data the Storage Engine must materialize.", + "references": [ + "https://www.sqlbi.com/articles/using-summarizecolumns-and-addmissingitems/" + ] + }, + { + "id": "HIGH_CARDINALITY_COLUMN", + "title": "High-cardinality column referenced by the query", + "category": "Cardinality", + "severity": "medium", + "requires": ["vertipaq"], + "kind": "for_each", + "collection": "high_cardinality_columns", + "max_findings": 8, + "where": { + "field": "cardinality", "op": ">=", "value": 1000000 + }, + "message": "Column '{table}'[{column}] has {cardinality_display} unique values ({data_type}). High-cardinality columns produce large dictionaries and slow scans, joins, and DISTINCTCOUNT operations.", + "recommendation": "Reduce cardinality where possible: split a datetime column into separate date and time columns, round or bucket numeric values that do not need full precision, remove unused high-cardinality keys, and avoid grouping or DISTINCTCOUNT on the highest-cardinality columns in hot queries.", + "references": [ + "https://www.sqlbi.com/articles/optimizing-high-cardinality-columns-in-vertipaq/", + "https://learn.microsoft.com/power-bi/guidance/import-modeling-data-reduction" + ] + }, + { + "id": "FLOAT_HIGH_CARDINALITY_COLUMN", + "title": "High-cardinality floating point column", + "category": "Cardinality", + "severity": "medium", + "requires": ["vertipaq"], + "kind": "for_each", + "collection": "high_cardinality_columns", + "max_findings": 8, + "where": { + "all": [ + {"field": "cardinality", "op": ">=", "value": 100000}, + {"field": "is_floating_point", "op": "==", "value": true} + ] + }, + "message": "Floating point column '{table}'[{column}] has {cardinality_display} unique values. Floating point (Double) columns with high cardinality are expensive to store and scan and can cause subtle rounding in grouping.", + "recommendation": "If the column does not need full floating point precision, convert it to a fixed decimal (Currency) or integer, or round it to fewer decimal places to dramatically reduce cardinality and size.", + "references": [ + "https://www.sqlbi.com/articles/data-types-in-dax-and-data-modeling/" + ] + }, + { + "id": "SLOW_SE_SCAN", + "title": "Slow Storage Engine scan", + "category": "Storage Engine", + "severity": "medium", + "requires": ["trace"], + "kind": "for_each", + "collection": "slow_se_queries", + "max_findings": 5, + "where": { + "field": "duration", "op": ">=", "value": 50 + }, + "message": "A Storage Engine scan ran for {duration} ms (CPU {cpu} ms). This single scan is a significant contributor to the total duration.", + "recommendation": "Inspect the xmSQL text of this scan in the DAX query plan. Long scans usually target large or high-cardinality tables, apply complex filters, or include a CallbackDataID. Reduce the rows scanned, simplify the filter, or remove the callback.", + "references": [ + "https://www.sqlbi.com/articles/understanding-dax-query-plans/" + ] + }, + { + "id": "NO_TRACE_CAPTURED", + "title": "No trace details were captured", + "category": "Diagnostics", + "severity": "info", + "requires": [], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "has_trace", "op": "==", "value": false}, + {"metric": "has_query", "op": "==", "value": true} + ] + }, + "message": "No trace details were captured for the current query, so engine-timing rules could not be evaluated.", + "recommendation": "Run the DAX query first (so trace events and the query plan are captured), then generate the performance analysis again for complete results.", + "references": [] + }, + { + "id": "NO_QUERY_PLAN_CAPTURED", + "title": "No DAX query plan was captured", + "category": "Diagnostics", + "severity": "info", + "requires": ["trace"], + "kind": "scalar", + "condition": { + "all": [ + {"metric": "has_query_plan", "op": "==", "value": false}, + {"metric": "has_trace", "op": "==", "value": true} + ] + }, + "message": "Trace details are available but no DAX query plan (logical/physical) was captured, so plan-based rules (CallbackDataID, spools) could not be evaluated.", + "recommendation": "Re-run the query to capture the query plan. CallbackDataID and spool analysis require the physical plan.", + "references": [] + } + ] +} diff --git a/src/sempy_labs/semantic_model/_dax_perf.py b/src/sempy_labs/semantic_model/_dax_perf.py new file mode 100644 index 000000000..bce73d67a --- /dev/null +++ b/src/sempy_labs/semantic_model/_dax_perf.py @@ -0,0 +1,12543 @@ +import sempy.fabric as fabric +import pandas as pd +from sempy_labs._helper_functions import ( + resolve_item_name_and_id, +) +from typing import Optional, Tuple +from sempy._utils._log import log +from uuid import UUID +import time +import warnings +import re +import json +from datetime import datetime, timezone + + +def _get_trace_logs(trace) -> Optional[pd.DataFrame]: + """Return ``trace.get_trace_logs()`` while suppressing the noisy + ``"No trace logs have been recorded..."`` ``UserWarning`` that sempy emits + when the call happens before the engine has flushed any events. The polling + loops in this module call ``get_trace_logs`` repeatedly (often before logs + exist), so this warning is expected and not actionable for the user.""" + + with warnings.catch_warnings(): + warnings.filterwarnings( + "ignore", + message="No trace logs have been recorded", + category=UserWarning, + ) + return trace.get_trace_logs() + + +@log +def dax_perf_optimizer( + dataset: Optional[str | UUID] = None, + dax_string: str = "", + workspace: Optional[str | UUID] = None, + clear_cache: bool = True, + visualize: bool = True, + effective_user_name: Optional[str] = None, + role: Optional[str] = None, + dark_mode: bool = False, +) -> pd.DataFrame: + """ + Runs a DAX query against a semantic model while capturing a server-side + trace, and computes high-level performance statistics (Total Duration, + Formula Engine Duration, Storage Engine Duration, and CPU time) using + the same conventions as `DAX Studio `_. + + Parameters + ---------- + dataset : str | uuid.UUID, default=None + Name or ID of the semantic model. Optional when ``visualize=True``: + if not provided, the interactive widget lets you choose a workspace + and a semantic model within it before running a query. Required when + ``visualize=False``. + dax_string : str, default="" + The DAX query to execute. May be left empty when ``visualize=True`` + (you can type the query directly in the widget). Required when + ``visualize=False``. + workspace : str | uuid.UUID, default=None + The Fabric workspace name or ID. + Defaults to None which resolves to the workspace of the attached lakehouse + or if no lakehouse attached, resolves to the workspace of the notebook. + clear_cache : bool, default=True + If True, clears the dataset cache before running the query so the + run reflects a cold-cache state. + visualize : bool, default=True + If True, displays an interactive widget showing the high-level + timings (Duration, FE, SE, CPU), a per-event details table, and an + editable DAX editor with a Run button to re-execute the query. + effective_user_name : str, default=None + If set, runs the query impersonating this user (passed as the + ``effective_user_name`` parameter of ``fabric.evaluate_dax``). Use + this for user impersonation. Cannot be used together with ``role``. + role : str, default=None + If set, runs the query impersonating this security role (passed as + the ``role`` parameter of ``fabric.evaluate_dax``). Use this for + role impersonation. Cannot be used together with + ``effective_user_name``. + dark_mode : bool, default=False + If True, the interactive widget is initially displayed using its + dark color theme. Only applies when ``visualize=True``. + + Returns + ------- + pandas.DataFrame + A pandas dataframe of the captured trace events, including the + ``Event Class``, ``Event Subclass``, ``Duration`` and ``Cpu Time`` + for each event. + """ + + from sempy_labs._helper_functions import resolve_workspace_name_and_id + + if effective_user_name and role: + raise ValueError( + "Cannot use both 'effective_user_name' (user impersonation) and " + "'role' (role impersonation) at the same time. Specify at most " + "one of them." + ) + + if not visualize: + if dataset is None: + raise ValueError( + "The 'dataset' parameter is required when 'visualize=False'." + ) + if not (dax_string and dax_string.strip()): + raise ValueError( + "The 'dax_string' parameter is required when 'visualize=False'." + ) + + df = pd.DataFrame() + result_df = pd.DataFrame() + total_duration = fe_duration = se_duration = cpu_time = 0 + dataset_name = None + dataset_id = None + workspace_name = None + workspace_id = None + + if dataset is not None: + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + dataset_name, dataset_id = resolve_item_name_and_id( + item=dataset, type="SemanticModel", workspace=workspace_id + ) + if dax_string and dax_string.strip(): + ( + df, + total_duration, + fe_duration, + se_duration, + cpu_time, + result_df, + ) = _run_dax_trace( + dataset_id=dataset_id, + workspace_id=workspace_id, + dax_string=dax_string, + clear_cache=clear_cache, + effective_user_name=effective_user_name, + role=role, + ) + elif workspace is not None: + # No dataset chosen yet, but a workspace was provided: resolve it so + # the widget's model picker can pre-select that workspace. + workspace_name, workspace_id = resolve_workspace_name_and_id(workspace) + + if visualize: + _visualize_dax_test( + df=df, + total_duration=total_duration, + fe_duration=fe_duration, + se_duration=se_duration, + cpu_time=cpu_time, + dax_string=dax_string, + dataset_id=dataset_id, + workspace_id=workspace_id, + dataset_name=( + (str(dataset_name) if dataset_name else str(dataset)) + if dataset is not None + else None + ), + workspace_name=workspace_name, + clear_cache=clear_cache, + result_df=result_df, + effective_user_name=effective_user_name, + role=role, + dark_mode=dark_mode, + ) + + return df + + +# Trace event schema captured by :func:`test` / :func:`_run_dax_trace`. +_TEST_EVENT_SCHEMA: dict = { + "QueryBegin": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "NTUserName", + "TextData", + "StartTime", + "ApplicationName", + "ApplicationContext", + "RequestID", + ], + "QueryEnd": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "NTUserName", + "TextData", + "StartTime", + "EndTime", + "Duration", + "CpuTime", + "Success", + "ApplicationName", + "ApplicationContext", + "RequestID", + ], + "VertiPaqSEQueryBegin": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "NTUserName", + "TextData", + "StartTime", + "ApplicationName", + "ApplicationContext", + "RequestID", + ], + "VertiPaqSEQueryEnd": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "NTUserName", + "TextData", + "StartTime", + "EndTime", + "Duration", + "CpuTime", + "Success", + "ApplicationName", + "ApplicationContext", + "RequestID", + ], + "VertiPaqSEQueryCacheMatch": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "NTUserName", + "TextData", + "RequestID", + ], + "DirectQueryEnd": [ + "EventClass", + "TextData", + "StartTime", + "EndTime", + "Duration", + "CpuTime", + "RequestID", + ], + "DAXQueryPlan": [ + "EventClass", + "EventSubclass", + "CurrentTime", + "TextData", + "ApplicationName", + "ApplicationContext", + "RequestID", + ], + "ExecutionMetrics": [ + "EventClass", + #"EventSubclass", + #"CurrentTime", + "TextData", + "ApplicationName", + "ApplicationContext", + "RequestID", + ] +} + + +def _normalize_dax_text(text) -> str: + """Normalize a DAX query string for matching a trace event's TextData to + the query typed in the DAX query pane (collapse all whitespace runs to a + single space and strip).""" + + if text is None: + return "" + return re.sub(r"\s+", " ", str(text)).strip() + + +def _trace_col(df: pd.DataFrame, *names: str) -> Optional[str]: + """Return the first of ``names`` that is a column of ``df`` (the trace + column naming differs between the space-delimited logs and the raw schema, + e.g. ``"Event Class"`` vs ``"EventClass"``).""" + + for n in names: + if n in df.columns: + return n + return None + + +def _resolve_query_request_id(new_logs: pd.DataFrame, dax_string: str) -> Optional[str]: + """Find the RequestID of the ``QueryBegin`` event that corresponds to the + DAX query executed from the query pane. + + The interactive widget runs several auxiliary DAX queries against the same + model/trace (Vertipaq Analyzer, query dependencies, performance analysis). + Those events accumulate in the long-running trace between user runs, so + slicing the logs by row count alone is not enough to isolate the user's + query. Instead we match the executed query text against the ``QueryBegin`` + ``TextData`` (preferring events from the SemPy application) and capture + that event's RequestID; every event for the user's query shares that same + RequestID. + + Returns the RequestID as a string, or ``None`` if it cannot be determined + (in which case the caller keeps the unfiltered rows).""" + + if new_logs is None or new_logs.empty: + return None + ec = _trace_col(new_logs, "Event Class", "EventClass") + td = _trace_col(new_logs, "Text Data", "TextData") + req_col = _trace_col(new_logs, "Request ID", "RequestID") + app = _trace_col(new_logs, "Application Name", "ApplicationName") + if ec is None or td is None or req_col is None: + return None + + qb = new_logs[new_logs[ec] == "QueryBegin"] + if qb.empty: + return None + + # Prefer QueryBegin events emitted by the SemPy application (the client used + # to run the query pane's query); fall back to all QueryBegin events. + candidates = qb + if app is not None: + sem = qb[qb[app].astype(str).str.contains("SemPy", case=False, na=False)] + if not sem.empty: + candidates = sem + + target = _normalize_dax_text(dax_string) + matches = candidates[candidates[td].astype(str).map(_normalize_dax_text) == target] + if matches.empty: + # No exact text match (e.g. the engine reformatted the text): fall back + # to the most recent non-warm-up QueryBegin among the candidates. + matches = candidates[~candidates[td].astype(str).str.startswith("EVALUATE {1}")] + if matches.empty: + return None + + val = matches[req_col].iloc[-1] + if pd.isna(val): + return None + return str(val).strip() + + +def _filter_logs_to_request_id( + df: pd.DataFrame, request_id: Optional[str] +) -> pd.DataFrame: + """Return only the rows of ``df`` whose RequestID matches ``request_id``. + If ``request_id`` is ``None`` or no RequestID column is present, ``df`` is + returned unchanged.""" + + if df is None or df.empty or request_id is None: + return df + req_col = _trace_col(df, "Request ID", "RequestID") + if req_col is None: + return df + + def _norm(v) -> str: + if pd.isna(v): + return "" + return str(v).strip() + + return df[df[req_col].map(_norm) == str(request_id).strip()].reset_index(drop=True) + + +def _execute_and_capture( + trace, + dataset_id: str, + workspace_id: str, + dax_string: str, + effective_user_name: Optional[str], + role: Optional[str], + baseline_count: int, + run_warmup: bool = True, + wait_for_optional_events: bool = True, +) -> Tuple[pd.DataFrame, pd.DataFrame, int]: + """Run the real DAX query (optionally preceded by a one-time warm-up + evaluation) against an already-started ``trace`` and capture the trace + rows produced by this execution. + + The trace is *not* stopped: events are read live via + :meth:`Trace.get_trace_logs`, which lets a single long-running trace serve + many queries. ``baseline_count`` is the number of log rows already consumed + by previous executions; only rows beyond it belong to this query. + + ``run_warmup`` controls whether the throwaway ``EVALUATE {1}`` warm-up + query is executed first. This is only needed for the very first query run + against a semantic model (to prime caches/connection); subsequent queries + against the same model skip it. + + ``wait_for_optional_events`` controls whether capture blocks for late DAX + query-plan events after the essential ``QueryEnd`` event arrives. The + interactive widget disables this wait and back-fills optional artifacts in + the background so timing stats can be displayed immediately. + + Returns + ------- + tuple + ``(result_df, new_logs, new_total_count)`` where ``new_logs`` are the + trace rows for this execution (rows after ``baseline_count``) and + ``new_total_count`` is the total log row count after this execution + (the next baseline). + """ + + if run_warmup: + # Warm-up evaluation; filtered out of results below. Only run for the + # first query against the model. + fabric.evaluate_dax( + dataset=dataset_id, + workspace=workspace_id, + dax_string="EVALUATE {1}", + ) + # Run the actual DAX query. + result_df = fabric.evaluate_dax( + dataset=dataset_id, + workspace=workspace_id, + dax_string=dax_string, + effective_user_name=effective_user_name, + role=role, + ) + # Wait for the trace to flush this query's events. ``fabric.evaluate_dax`` + # has already returned, which means the engine has finished the query and + # *will* emit a QueryEnd event plus the query's DAX query plan events — we + # only need to wait for the trace buffer to flush them. Events frequently + # arrive across several separate flushes (and the logical/physical plan + # rows usually land *after* QueryEnd), so capture happens in two phases: + # 1. wait for the real query's QueryEnd, then + # 2. wait for the DAX query plan rows to stop growing. + # The common case breaks out of each phase in a fraction of a second; the + # generous caps only matter on a slow capacity / slow trace flush, where + # the previous short caps (2s / 5s) silently dropped the query plan and + # zeroed the trace timings. + logs: Optional[pd.DataFrame] = None + + def _new_rows(_logs: Optional[pd.DataFrame]) -> pd.DataFrame: + if _logs is None: + return pd.DataFrame() + if len(_logs) > baseline_count: + return _logs.iloc[baseline_count:] + return _logs.iloc[0:0] + + # Phase 1: wait for the real query's QueryEnd event (not the warm-up + # ``EVALUATE {1}``). Since the query has already completed, this is + # essentially guaranteed to arrive; break the instant it does. + qe_seen = False + _qe_deadline = time.monotonic() + 30.0 + _first_qe_poll = True + while time.monotonic() < _qe_deadline: + if _first_qe_poll: + _first_qe_poll = False + else: + time.sleep(0.05) + try: + _l = _get_trace_logs(trace) + except Exception: + continue + if _l is not None and not _l.empty: + logs = _l + _new = _new_rows(logs) + _ec = "Event Class" if "Event Class" in _new.columns else "EventClass" + _td = "Text Data" if "Text Data" in _new.columns else "TextData" + if _ec not in _new.columns: + continue + _qe = _new[_new[_ec] == "QueryEnd"] + if _qe.empty: + continue + if _td in _new.columns: + # Break once a QueryEnd for the real query (not the warm-up + # EVALUATE {1}) has been captured. + _real = _qe[~_qe[_td].astype(str).str.startswith("EVALUATE {1}")] + if not _real.empty: + qe_seen = True + break + elif len(_qe) >= (2 if run_warmup else 1): + # No text available: warm-up + real query == 2 QueryEnd, or just + # the real query when the warm-up was skipped. + qe_seen = True + break + + # Phase 2: wait for the DAX Query Plan events. The engine serializes these + # during query cleanup and almost always flushes them into the trace buffer + # *after* the QueryEnd captured above (often in a separate flush a second or + # more later). A normal ``EVALUATE`` query emits exactly two plan events — a + # logical plan and a physical plan — which can arrive in separate flushes, + # so this phase keeps polling until BOTH have been captured (the fast path), + # falling back to a "row count held steady" heuristic for the rare query + # whose plan shape differs. + # + # Because essentially every real DAX query produces a plan, the loop biases + # heavily toward waiting: it keeps polling for a generous window after + # QueryEnd rather than declaring "no plan" the moment the buffer is briefly + # quiet. Only a query that truly emits no plan (e.g. a trivial constant + # evaluation) pays the longer wait, and that is rare in practice. + def _plan_types(_n: pd.DataFrame, _ecol: str) -> set: + if _ecol not in _n.columns or "Event Subclass" not in _n.columns: + return set() + _pl = _n[_n[_ecol] == "DAXQueryPlan"] + _types: set = set() + for _sc in _pl["Event Subclass"].astype(str): + _low = _sc.lower() + if "physical" in _low or _sc.strip() == "2": + _types.add("physical") + elif "logical" in _low or _sc.strip() == "1": + _types.add("logical") + else: + _types.add(_sc) + return _types + + # The inline wait below blocks until the DAX query plan has actually been + # captured: because essentially every real ``EVALUATE`` query emits both a + # logical and a physical plan, the loop keeps polling the trace until BOTH + # have arrived rather than giving up early and relying on the background + # back-fill. The plan rows are serialized during query cleanup and often + # flush into the trace buffer a second or more after QueryEnd (sometimes in + # several separate flushes), so a generous deadline is used. Only a query + # that genuinely produces no plan (e.g. a trivial constant evaluation) waits + # out the longer no-plan grace period below before giving up. + _plan_deadline = time.monotonic() + (30.0 if wait_for_optional_events else 0.0) + _plan_count = -1 + _stable_since: Optional[float] = None + _qe_at: Optional[float] = time.monotonic() if qe_seen else None + while time.monotonic() < _plan_deadline: + try: + _l = _get_trace_logs(trace) + except Exception: + _l = None + if _l is not None and not _l.empty: + logs = _l + _new = _new_rows(logs) + _ec = "Event Class" if "Event Class" in _new.columns else "EventClass" + if not qe_seen and _ec in _new.columns: + # QueryEnd may still be in flight if phase 1 timed out; keep an eye + # out for it so the trace timings are not lost. + _qe = _new[_new[_ec] == "QueryEnd"] + if not _qe.empty: + qe_seen = True + _qe_at = time.monotonic() + _cur = int((_new[_ec] == "DAXQueryPlan").sum()) if _ec in _new.columns else 0 + # Fast path: both a logical and a physical plan have been captured — + # this is the complete plan for a normal query, so stop immediately. + if {"logical", "physical"}.issubset(_plan_types(_new, _ec)): + break + if _cur != _plan_count: + # New plan row(s) arrived; reset the stability timer so we keep + # waiting for any further plan events still being flushed. + _plan_count = _cur + _stable_since = time.monotonic() + elif _stable_since is not None and (time.monotonic() - _stable_since >= 2.0): + # Plan count held steady. Only stop early if at least one plan row + # was captured AND it has been quiet for a couple of seconds past + # QueryEnd (a partial plan that is not going to be completed). If no + # plan row has arrived yet, keep waiting for the full deadline so a + # late-flushing plan is never missed. + if _cur > 0 and _qe_at is not None and (time.monotonic() - _qe_at >= 2.0): + break + time.sleep(0.1) + + if logs is None: + try: + logs = _get_trace_logs(trace) + except Exception: + logs = pd.DataFrame() + if logs is None: + logs = pd.DataFrame() + + if len(logs) > baseline_count: + new_logs = logs.iloc[baseline_count:].reset_index(drop=True) + else: + new_logs = logs.iloc[0:0].reset_index(drop=True) + # Restrict the captured rows to the request that ran the query pane's query + # so that events from auxiliary queries (Vertipaq Analyzer, dependencies, + # performance analysis) accumulated in the long-running trace are excluded. + _req_id = _resolve_query_request_id(new_logs, dax_string) + if _req_id is not None: + new_logs = _filter_logs_to_request_id(new_logs, _req_id) + return result_df, new_logs, len(logs) + + +def _compute_trace_stats( + df: pd.DataFrame, +) -> Tuple[pd.DataFrame, int, int, int, int]: + """Filter trace rows and compute DAX Studio style aggregate stats. + + Returns + ------- + tuple + ``(df, total_duration, fe_duration, se_duration, cpu_time)`` + """ + + if df is None: + df = pd.DataFrame() + # Drop events from other sessions / warm-up evaluation. + if "Application Name" in df.columns: + df = df[~df["Application Name"].isin(["PowerBI", "PowerBIEIM"])] + if "Text Data" in df.columns: + df = df[~df["Text Data"].astype(str).str.startswith("EVALUATE {1}")] + df = df.reset_index(drop=True) + + if "Event Class" not in df.columns: + return df, 0, 0, 0, 0 + + # Compute aggregate stats using DAX Studio conventions: + # Total Duration = QueryEnd.Duration + # SE Duration = sum of VertiPaqSEQueryEnd Duration, EXCLUDING + # internal sub-queries (EventSubclass contains + # "Internal") so we do not double-count time that + # is already rolled up into the parent scan. + # FE Duration = Total Duration - SE Duration + # CPU = QueryEnd.CpuTime + qe = df[df["Event Class"] == "QueryEnd"] + total_duration = int(qe["Duration"].iloc[-1]) if not qe.empty else 0 + cpu_time = int(qe["Cpu Time"].iloc[-1]) if not qe.empty else 0 + + se_events = df[df["Event Class"] == "VertiPaqSEQueryEnd"] + if not se_events.empty: + not_internal = ( + ~se_events["Event Subclass"] + .astype(str) + .str.contains("Internal", case=False, na=False) + ) + se_duration = int(se_events.loc[not_internal, "Duration"].sum()) + else: + se_duration = 0 + fe_duration = max(total_duration - se_duration, 0) + + return df, total_duration, fe_duration, se_duration, cpu_time + + +def _captured_queries_from_df(df: pd.DataFrame) -> list: + """Build one timing record per report-issued ``QueryEnd`` trace event.""" + + if df is None or df.empty: + return [] + event_col = _trace_col(df, "Event Class", "EventClass") + text_col = _trace_col(df, "Text Data", "TextData") + request_col = _trace_col(df, "Request ID", "RequestID") + duration_col = _trace_col(df, "Duration") + cpu_col = _trace_col(df, "Cpu Time", "CpuTime") + subclass_col = _trace_col(df, "Event Subclass", "EventSubclass") + if event_col is None or text_col is None: + return [] + + def _integer(row, column: Optional[str]) -> int: + if column is None: + return 0 + try: + value = row.get(column, 0) + return 0 if pd.isna(value) else int(value) + except (TypeError, ValueError): + return 0 + + def _text(row, column: Optional[str]) -> str: + if column is None: + return "" + value = row.get(column, "") + return "" if pd.isna(value) else str(value).strip() + + storage_by_request: dict[str, int] = {} + if request_col is not None: + storage_events = df[df[event_col] == "VertiPaqSEQueryEnd"] + if subclass_col is not None and not storage_events.empty: + storage_events = storage_events[ + ~storage_events[subclass_col] + .astype(str) + .str.contains("Internal", case=False, na=False) + ] + for _, row in storage_events.iterrows(): + request_id = _text(row, request_col) + if request_id: + storage_by_request[request_id] = storage_by_request.get( + request_id, 0 + ) + _integer(row, duration_col) + + queries = [] + for _, row in df[df[event_col] == "QueryEnd"].iterrows(): + query = _text(row, text_col) + if not query or query.startswith("EVALUATE {1}"): + continue + request_id = _text(row, request_col) + total = _integer(row, duration_col) + storage = storage_by_request.get(request_id, 0) + queries.append( + { + "dax_query": query, + "duration": total, + "cpu": _integer(row, cpu_col), + "fe_duration": max(total - storage, 0), + "se_duration": storage, + } + ) + return queries + + +def _run_dax_trace( + dataset_id: str, + workspace_id: str, + dax_string: str, + clear_cache: bool, + effective_user_name: Optional[str] = None, + role: Optional[str] = None, +) -> Tuple[pd.DataFrame, int, int, int, int, pd.DataFrame]: + """Run a DAX query with a one-shot server-side trace and compute DAX + Studio style aggregate stats. + + A fresh trace is created, started, used for this single query, and then + stopped. This is used for ``visualize=False`` and the initial query of the + interactive widget. The widget itself uses a single long-running trace for + subsequent queries (see ``_visualize_dax_test``). + + Returns + ------- + tuple + ``(df, total_duration, fe_duration, se_duration, cpu_time, result_df)`` + """ + from sempy_labs._clear_cache import clear_cache as _clear_cache_fn + + if clear_cache: + _clear_cache_fn(dataset=dataset_id, workspace=workspace_id) + + result_df: pd.DataFrame = pd.DataFrame() + df = pd.DataFrame() + with fabric.create_trace_connection( + dataset=dataset_id, workspace=workspace_id + ) as trace_connection: + with trace_connection.create_trace(_TEST_EVENT_SCHEMA) as trace: + trace.start() + result_df, df, _ = _execute_and_capture( + trace, + dataset_id, + workspace_id, + dax_string, + effective_user_name, + role, + 0, + ) + # Stop the one-shot trace; prefer its authoritative logs. + try: + stopped = trace.stop() + if stopped is not None and not stopped.empty: + df = stopped + except Exception: + pass + + df, total_duration, fe_duration, se_duration, cpu_time = _compute_trace_stats(df) + + return df, total_duration, fe_duration, se_duration, cpu_time, result_df + + +def _trace_rows_from_df(df: pd.DataFrame) -> list: + """Convert the captured trace dataframe to a list of plain-dict rows + suitable for serialization to the front-end.""" + + if df is None or df.empty: + return [] + event_col = _trace_col(df, "Event Class", "EventClass") + subclass_col = _trace_col(df, "Event Subclass", "EventSubclass") + duration_col = _trace_col(df, "Duration") + cpu_col = _trace_col(df, "Cpu Time", "CpuTime") + text_col = _trace_col(df, "Text Data", "TextData") + if event_col is None: + return [] + detail_classes = { + "VertiPaqSEQueryEnd", + "VertiPaqSEQueryCacheMatch", + "DirectQueryEnd", + } + rows_df = df[df[event_col].isin(detail_classes)] + out = [] + for _, row in rows_df.iterrows(): + event = str(row.get(event_col, "") or "") + subclass = row.get(subclass_col, "") if subclass_col else "" + subclass_v = "" if not pd.notna(subclass) else str(subclass) + if subclass_v == "VertiPaqScanInternal": + continue + dur = row.get(duration_col, 0) if duration_col else 0 + cpu = row.get(cpu_col, 0) if cpu_col else 0 + text = row.get(text_col, "") if text_col else "" + text_v = "" if not pd.notna(text) else str(text) + estimate = re.search( + r"Estimated size \(volume, marshalling bytes\):\s*(\d+),\s*(\d+)", + text_v, + ) + try: + dur_v = int(dur) if pd.notna(dur) else 0 + except (TypeError, ValueError): + dur_v = 0 + try: + cpu_v = int(cpu) if pd.notna(cpu) else 0 + except (TypeError, ValueError): + cpu_v = 0 + out.append( + { + "event_class": event, + "event_subclass": subclass_v, + "duration": dur_v, + "cpu": cpu_v, + "rows": int(estimate.group(1)) if estimate else None, + "kb": int(estimate.group(2)) / 1024 if estimate else None, + "text": text_v, + } + ) + return out + + +def _query_plan_rows_from_df(df: pd.DataFrame) -> list: + """Convert the captured trace dataframe to a list of DAX Query Plan rows + (``{plan_type, event_subclass, text}``) suitable for serialization to the + front-end. ``plan_type`` is ``"Logical"`` or ``"Physical"`` based on the + event subclass (e.g. ``DAXVertiPaqLogicalPlan`` / + ``DAXVertiPaqPhysicalPlan``).""" + + if df is None or df.empty or "Event Class" not in df.columns: + return [] + rows_df = df[df["Event Class"] == "DAXQueryPlan"] + out = [] + for _, row in rows_df.iterrows(): + sc = str(row.get("Event Subclass", "") or "") + text = row.get("Text Data", "") + text = "" if not pd.notna(text) else str(text) + low = sc.lower() + if "physical" in low or sc.strip() == "2": + plan_type = "Physical" + elif "logical" in low or sc.strip() == "1": + plan_type = "Logical" + else: + plan_type = sc or "Unknown" + out.append( + { + "plan_type": plan_type, + "event_subclass": sc, + "text": text, + } + ) + return out + + +# Subset of the ``ExecutionMetrics`` TextData JSON surfaced in the widget's +# Execution Metrics tab, in display order: (json_key, label). +_EXECUTION_METRIC_FIELDS: list = [ + ("vertipaqJobCpuTimeMs", "VertiPaq Job CPU Time (ms)"), + ("queryProcessingCpuTimeMs", "Query Processing CPU Time (ms)"), + ("totalCpuTimeMs", "Total CPU Time (ms)"), + ("executionDelayMs", "Execution Delay (ms)"), + ("approximatePeakMemConsumptionKB", "Approximate Peak Memory Consumption (KB)"), + ("directQueryTotalRows", "DirectQuery Total Rows"), +] + + +def _execution_metrics_from_df(df: pd.DataFrame) -> list: + """Convert the captured trace dataframe to a list of execution metric rows + (``{key, label, value}``) suitable for serialization to the front-end. + + The ``ExecutionMetrics`` trace event carries a JSON document in its + ``TextData``; only the fields in :data:`_EXECUTION_METRIC_FIELDS` are + surfaced. Returns an empty list if no ``ExecutionMetrics`` event was + captured or its TextData cannot be parsed.""" + + if df is None or df.empty or "Event Class" not in df.columns: + return [] + rows_df = df[df["Event Class"] == "ExecutionMetrics"] + if rows_df.empty: + return [] + # Use the most recent ExecutionMetrics event for this query. + text = rows_df.iloc[-1].get("Text Data", "") + if not pd.notna(text): + return [] + try: + data = json.loads(str(text)) + except (ValueError, TypeError): + return [] + if not isinstance(data, dict): + return [] + out = [] + for key, label in _EXECUTION_METRIC_FIELDS: + if key not in data: + continue + val = data.get(key) + try: + val_v = int(val) if val is not None else 0 + except (TypeError, ValueError): + try: + val_v = float(val) + except (TypeError, ValueError): + val_v = 0 + out.append({"key": key, "label": label, "value": val_v}) + return out + + +def _execution_metrics_dict(metric_rows: list) -> dict: + """Convert execution metric rows to the dictionary shown in history.""" + + return { + str(row.get("label") or row.get("key") or ""): row.get("value") + for row in metric_rows or [] + if row.get("label") or row.get("key") + } + + +def _result_payload_from_df(df: pd.DataFrame, max_rows: int = 5000) -> dict: + """Convert a query result dataframe to a payload of ``{columns, rows, + total_rows, truncated}`` for the front-end.""" + + if df is None or not hasattr(df, "columns"): + return {"columns": [], "rows": [], "total_rows": 0, "truncated": False} + columns = [str(c) for c in df.columns] + total_rows = int(len(df)) + truncated = total_rows > max_rows + view = df.head(max_rows) if truncated else df + rows: list = [] + for _, r in view.iterrows(): + row: list = [] + for v in r.tolist(): + if v is None: + row.append(None) + elif pd.isna(v): + row.append(None) + else: + row.append(v if isinstance(v, (int, float, bool, str)) else str(v)) + rows.append(row) + return { + "columns": columns, + "rows": rows, + "total_rows": total_rows, + "truncated": truncated, + } + + +def _prepare_embedded_vertipaq_dataframe(name: str, df: pd.DataFrame) -> pd.DataFrame: + """Filter Vertipaq Analyzer data for the embedded performance tool view.""" + + view = df.copy() + if name == "Columns": + if "Type" in view.columns: + view = view[ + view["Type"].astype("string").str.casefold() != "rownumber" + ] + view = view.drop(columns=["Source Column"], errors="ignore") + elif name == "Partitions": + has_direct_lake = "Mode" in view.columns and view["Mode"].astype( + "string" + ).str.casefold().eq("directlake").any() + if not has_direct_lake: + view = view.drop( + columns=[ + "Direct Lake Type", + "Source Name", + "Source Type", + "Source Workspace", + "Source Schema Name", + "Source Table Name", + ], + errors="ignore", + ) + return view.reset_index(drop=True) + + +def _collect_model_tree(dataset_id: str, workspace_id: str) -> list: + """Collect a lightweight metadata tree of the semantic model for the + sidebar (tables → columns / measures / hierarchies).""" + + try: + from sempy_labs.tom import connect_semantic_model + except Exception: + return [] + + try: + with connect_semantic_model( + dataset=dataset_id, workspace=workspace_id, readonly=True + ) as tom: + return _build_model_tree(tom) + except Exception: + return [] + + +def _build_model_tree(tom) -> list: + """Build the sidebar metadata tree from an already-open TOM connection.""" + + tree: list = [] + try: + for table in tom.model.Tables: + tname = str(table.Name) + columns = sorted( + ( + { + "name": str(c.Name), + "hidden": bool(getattr(c, "IsHidden", False)), + "data_type": str(getattr(c, "DataType", "") or ""), + "description": str(getattr(c, "Description", "") or ""), + "display_folder": str(getattr(c, "DisplayFolder", "") or ""), + } + for c in table.Columns + if str(getattr(c, "Type", "")) != "RowNumber" + ), + key=lambda x: x["name"].lower(), + ) + measures = sorted( + ( + { + "name": str(m.Name), + "hidden": bool(getattr(m, "IsHidden", False)), + "description": str(getattr(m, "Description", "") or ""), + "display_folder": str(getattr(m, "DisplayFolder", "") or ""), + "expression": str(getattr(m, "Expression", "") or ""), + } + for m in table.Measures + ), + key=lambda x: x["name"].lower(), + ) + hierarchies = sorted( + ( + { + "name": str(h.Name), + "hidden": bool(getattr(h, "IsHidden", False)), + "description": str(getattr(h, "Description", "") or ""), + "display_folder": str(getattr(h, "DisplayFolder", "") or ""), + "levels": [ + { + "name": str(lvl.Name), + "description": str( + getattr(lvl, "Description", "") or "" + ), + } + for lvl in sorted( + h.Levels, + key=lambda lvl: getattr(lvl, "Ordinal", 0), + ) + ], + } + for h in table.Hierarchies + ), + key=lambda x: x["name"].lower(), + ) + is_calc_group = getattr(table, "CalculationGroup", None) is not None + calculation_items: list = [] + if is_calc_group: + calculation_items = sorted( + ( + { + "name": str(ci.Name), + "description": str(getattr(ci, "Description", "") or ""), + } + for ci in table.CalculationGroup.CalculationItems + ), + key=lambda x: x["name"].lower(), + ) + tree.append( + { + "name": tname, + "hidden": bool(getattr(table, "IsHidden", False)), + "description": str(getattr(table, "Description", "") or ""), + "calculation_group": bool(is_calc_group), + "calculation_items": calculation_items, + "columns": columns, + "measures": measures, + "hierarchies": hierarchies, + } + ) + except Exception: + return [] + tree.sort(key=lambda t: t["name"].lower()) + return tree + + +def _build_relationship_lookup(tom) -> dict: + """Build a mapping of relationship name -> a human-readable description of + the columns it joins (``'FromTable'[FromColumn] → 'ToTable'[ToColumn]``), + using an already-open TOM connection. Inactive relationships are flagged. + """ + + lookup: dict = {} + try: + for rel in tom.model.Relationships: + try: + detail = ( + f"'{rel.FromTable.Name}'[{rel.FromColumn.Name}]" + f" → '{rel.ToTable.Name}'[{rel.ToColumn.Name}]" + ) + if not bool(getattr(rel, "IsActive", True)): + detail += " (inactive)" + lookup[str(rel.Name)] = detail + except Exception: + continue + except Exception: + return {} + return lookup + + +def _build_relationship_columns(tom) -> dict: + """Build a mapping of relationship name -> the list of ``(table, column)`` + tuples it joins (the From and To columns), using an already-open TOM + connection. Used to include relationship columns in the unique list of + referenced columns.""" + + lookup: dict = {} + try: + for rel in tom.model.Relationships: + try: + lookup[str(rel.Name)] = [ + (str(rel.FromTable.Name), str(rel.FromColumn.Name)), + (str(rel.ToTable.Name), str(rel.ToColumn.Name)), + ] + except Exception: + continue + except Exception: + return {} + return lookup + + +def _build_rownumber_columns(tom) -> set: + """Build a set of ``(table_name, column_name)`` tuples for every column + whose TOM ``ColumnType`` is ``RowNumber``, using an already-open TOM + connection. These internal columns are excluded from the query + dependencies output.""" + + rownumber: set = set() + try: + for table in tom.model.Tables: + tname = str(table.Name) + for c in table.Columns: + try: + if str(getattr(c, "Type", "")) == "RowNumber": + rownumber.add((tname, str(c.Name))) + except Exception: + continue + except Exception: + return set() + return rownumber + + +def _build_dependency_tree( + rows: list, rel_lookup: dict, model_label: str, rownumber_cols: Optional[set] = None +) -> list: + """Organize flat ``INFO.CALCDEPENDENCY`` rows into a hierarchical tree. + + The tree has a single ``Model`` root whose children are a ``Tables`` group + (each referenced table, with its referenced columns / measures / + hierarchies grouped beneath it) and a ``Relationships`` group (each + referenced relationship, labeled with the columns it joins, looked up via + TOM in ``rel_lookup``). Columns whose TOM ``ColumnType`` is ``RowNumber`` + (provided in ``rownumber_cols`` as ``(table, column)`` tuples) are omitted. + """ + + rownumber_cols = rownumber_cols or set() + + def _classify(obj_type: str) -> str: + t = (obj_type or "").upper() + if "RELATIONSHIP" in t: + return "relationship" + if "MEASURE" in t: + return "measure" + if "HIERARCHY" in t: + return "hierarchy" + if "COLUMN" in t: + return "column" + if "CALC_GROUP" in t or "CALCULATION_GROUP" in t: + return "calc_group" + if t == "TABLE": + return "table" + return "other" + + tables: dict = {} + relationships: list = [] + seen_rel: set = set() + + def _ensure_table(name: str) -> dict: + if name not in tables: + tables[name] = { + "columns": [], + "measures": [], + "hierarchies": [], + "other": [], + } + return tables[name] + + def _add(bucket: list, value: str) -> None: + if value and value not in bucket: + bucket.append(value) + + for r in rows: + kind = _classify(r.get("object_type")) + table = (r.get("table") or "").strip() + obj = (r.get("object") or "").strip() + if kind == "relationship": + if obj and obj not in seen_rel: + seen_rel.add(obj) + relationships.append(obj) + elif kind == "measure": + if table: + _add(_ensure_table(table)["measures"], obj) + elif kind == "column": + if table and (table, obj) not in rownumber_cols: + _add(_ensure_table(table)["columns"], obj) + elif kind == "hierarchy": + if table: + _add(_ensure_table(table)["hierarchies"], obj) + elif kind == "table": + _ensure_table(obj or table) + else: + if table: + _add(_ensure_table(table)["other"], obj) + + def _leaf_group(label: str, kind: str, names: list) -> dict: + return { + "label": label, + "kind": "group", + "children": [ + {"label": n, "kind": kind} for n in sorted(names, key=str.lower) + ], + } + + table_nodes = [] + for tname in sorted(tables, key=str.lower): + tdata = tables[tname] + tchildren = [] + if tdata["columns"]: + tchildren.append(_leaf_group("Columns", "column", tdata["columns"])) + if tdata["measures"]: + tchildren.append(_leaf_group("Measures", "measure", tdata["measures"])) + if tdata["hierarchies"]: + tchildren.append( + _leaf_group("Hierarchies", "hierarchy", tdata["hierarchies"]) + ) + if tdata["other"]: + tchildren.append(_leaf_group("Other", "column", tdata["other"])) + table_nodes.append({"label": tname, "kind": "table", "children": tchildren}) + + children = [] + if table_nodes: + children.append({"label": "Tables", "kind": "group", "children": table_nodes}) + if relationships: + rel_nodes = [] + for rname in relationships: + detail = rel_lookup.get(rname) + rel_nodes.append({"label": detail or rname, "kind": "relationship"}) + rel_nodes.sort(key=lambda n: n["label"].lower()) + children.append( + {"label": "Relationships", "kind": "group", "children": rel_nodes} + ) + + if not children: + return [] + return [ + { + "label": model_label or "Model", + "kind": "model", + "children": children, + } + ] + + +def _build_dependency_columns( + rows: list, + rel_columns: Optional[dict] = None, + rownumber_cols: Optional[set] = None, +) -> list: + """Build a unique, sorted list of the columns referenced by the query. + + Each entry is a ``{"table": ..., "column": ...}`` dict. Columns directly + referenced by the query (``REFERENCED_OBJECT_TYPE`` containing ``COLUMN``) + as well as the columns participating in any referenced relationship (looked + up via TOM in ``rel_columns`` keyed by relationship name) are included. + Columns whose TOM ``ColumnType`` is ``RowNumber`` (provided in + ``rownumber_cols`` as ``(table, column)`` tuples) are omitted. + """ + + rel_columns = rel_columns or {} + rownumber_cols = rownumber_cols or set() + seen: set = set() + out: list = [] + + def _add(table: str, column: str) -> None: + table = (table or "").strip() + column = (column or "").strip() + if not table or not column: + return + key = (table, column) + if key in rownumber_cols or key in seen: + return + seen.add(key) + out.append({"table": table, "column": column}) + + for r in rows: + otype = (r.get("object_type") or "").upper() + if "RELATIONSHIP" in otype: + obj = (r.get("object") or "").strip() + for tbl, col in rel_columns.get(obj, []): + _add(tbl, col) + elif "COLUMN" in otype: + _add(r.get("table"), r.get("object")) + + out.sort(key=lambda x: (x["table"].lower(), x["column"].lower())) + return out + + +def _build_model_roles(tom) -> list: + """Build the sorted list of security role names from an already-open TOM + connection.""" + + try: + roles = [str(r.Name) for r in tom.model.Roles] + except Exception: + return [] + roles.sort(key=lambda x: x.lower()) + return roles + + +def _collect_model_metadata(dataset_id: str, workspace_id: str) -> tuple: + """Collect both the sidebar metadata tree and the security role names in a + single TOM connection (avoids opening the XMLA connection twice). + + Returns + ------- + tuple + ``(tree, roles)``. Either may be an empty list if metadata cannot be + read. + """ + + try: + from sempy_labs.tom import connect_semantic_model + except Exception: + return [], [] + + try: + with connect_semantic_model( + dataset=dataset_id, workspace=workspace_id, readonly=True + ) as tom: + return _build_model_tree(tom), _build_model_roles(tom) + except Exception: + return [], [] + + +def _list_reports_for_capture(dataset_id: str, workspace_id: str) -> list: + """List embeddable reports in ``workspace_id`` bound to ``dataset_id``.""" + + from sempy_labs.report._items import list_reports_base + + reports = list_reports_base(workspace=workspace_id) + if reports is None or reports.empty: + return [] + matches = reports[ + (reports["Dataset Id"].astype(str) == str(dataset_id)) + & (reports["Dataset Workspace Id"].astype(str) == str(workspace_id)) + ] + result = [ + { + "id": str(row["Report Id"]), + "name": str(row["Report Name"]), + "embed_url": ( + "" if pd.isna(row["Embed Url"]) else str(row["Embed Url"]) + ), + } + for _, row in matches.iterrows() + ] + result.sort(key=lambda item: item["name"].lower()) + return result + + +def _collect_model_roles(dataset_id: str, workspace_id: str) -> list: + """Collect the security role names defined in the semantic model, sorted + alphabetically (case-insensitive). Returns an empty list if the model has + no roles or the metadata cannot be read.""" + + try: + from sempy_labs.tom import connect_semantic_model + except Exception: + return [] + + roles: list = [] + try: + with connect_semantic_model( + dataset=dataset_id, workspace=workspace_id, readonly=True + ) as tom: + roles = [str(r.Name) for r in tom.model.Roles] + except Exception: + return [] + roles.sort(key=lambda x: x.lower()) + return roles + + +def _list_workspaces_for_picker() -> list: + """Return a list of ``{"id", "name"}`` dicts for the workspaces the user + can access, sorted alphabetically. Used by the interactive widget's model + picker when no ``dataset`` is supplied to :func:`test`.""" + + out: list = [] + try: + dfW = fabric.list_workspaces() + except Exception: + return [] + for _, r in dfW.iterrows(): + out.append({"id": str(r["Id"]), "name": str(r["Name"])}) + out.sort(key=lambda x: x["name"].lower()) + return out + + +def _list_datasets_for_picker(workspace_id: str) -> list: + """Return a list of ``{"id", "name"}`` dicts for the semantic models in + the given workspace, sorted alphabetically. Used by the interactive + widget's model picker.""" + + out: list = [] + try: + dfD = fabric.list_datasets(workspace=workspace_id, mode="rest") + except Exception: + return [] + for _, r in dfD.iterrows(): + out.append({"id": str(r["Dataset Id"]), "name": str(r["Dataset Name"])}) + out.sort(key=lambda x: x["name"].lower()) + return out + + +def _classify_filter_type(kind: str, data_type: str) -> str: + """Classify a query-builder field into a filter family used to pick the + available filter operators: ``measure``, ``numeric``, ``datetime``, + ``boolean`` or ``text``.""" + + if kind == "measure": + return "measure" + dt = (data_type or "").strip().lower() + if dt in ("int64", "double", "decimal", "currency", "int", "integer"): + return "numeric" + if dt in ("datetime", "date", "time"): + return "datetime" + if dt in ("boolean", "bool"): + return "boolean" + return "text" + + +def _qb_quote_str(value: str) -> str: + """Return a DAX string literal for ``value`` (double quotes escaped).""" + + return '"' + str(value).replace('"', '""') + '"' + + +def _qb_numeric_literal(value: str) -> str: + """Return a numeric DAX literal for ``value`` or a quoted string if the + value is not numeric.""" + + raw = str(value).strip() + try: + float(raw) + return raw + except ValueError: + return _qb_quote_str(raw) + + +def _qb_value_literal(filter_type: str, value: str) -> str: + """Return a DAX literal for a filter value based on its filter family.""" + + import re as _re + + raw = str(value).strip() + if filter_type in ("numeric", "measure"): + return _qb_numeric_literal(raw) + if filter_type == "datetime": + m = _re.match(r"^(\d{4})-(\d{1,2})-(\d{1,2})$", raw) + if m: + return f"DATE({int(m.group(1))}, {int(m.group(2))}, " f"{int(m.group(3))})" + return _qb_quote_str(raw) + return _qb_quote_str(raw) + + +def _qb_build_predicate(item: dict) -> Optional[str]: + """Build a single DAX boolean predicate for a query-builder filter item. + Returns None if the operator/value combination is unusable.""" + + ref = item.get("ref") or "" + if not ref: + return None + ftype = _classify_filter_type(item.get("kind", "column"), item.get("data_type", "")) + op = (item.get("op") or "").strip() + value = item.get("value", "") + value2 = item.get("value2", "") + + if op == "blank": + return f"ISBLANK({ref})" + if op == "notblank": + return f"NOT ISBLANK({ref})" + if op == "istrue": + return f"{ref} = TRUE()" + if op == "isfalse": + return f"{ref} = FALSE()" + + if op in ("in", "notin"): + items = [v.strip() for v in str(value).split(",")] + items = [v for v in items if v != ""] + if not items: + return None + literals = ", ".join(_qb_value_literal(ftype, v) for v in items) + predicate = f"{ref} IN {{{literals}}}" + return f"NOT({predicate})" if op == "notin" else predicate + + if ftype == "text": + if op == "contains": + return f"CONTAINSSTRING({ref}, {_qb_quote_str(value)})" + if op == "startswith": + length = len(str(value)) + return f"LEFT({ref}, {length}) = {_qb_quote_str(value)}" + if op == "eq": + return f"{ref} = {_qb_quote_str(value)}" + if op == "ne": + return f"{ref} <> {_qb_quote_str(value)}" + return None + + symbol = { + "eq": "=", + "ne": "<>", + "gt": ">", + "ge": ">=", + "lt": "<", + "le": "<=", + }.get(op) + if op == "between": + lo = _qb_value_literal(ftype, value) + hi = _qb_value_literal(ftype, value2) + return f"{ref} >= {lo} && {ref} <= {hi}" + if symbol is not None: + return f"{ref} {symbol} {_qb_value_literal(ftype, value)}" + return None + + +def _build_summarize_dax(state: dict, dataset_id: str, workspace_id: str) -> str: + """Build an ``EVALUATE SUMMARIZECOLUMNS(...)`` DAX statement from a + query-builder state (a dict with ``fields`` and ``filters`` lists). + + The generated query follows the canonical "DAX as a query language" + pattern (see ``.claude/skills/sql_to_dax`` / the Query Builder SKILL): + + 1. ``EVALUATE`` + 2. ``SUMMARIZECOLUMNS(`` with elements in strict order: + a. attributes (group-by columns), + b. filters (column filters via ``FILTER(KEEPFILTERS(VALUES(col)), ...)``), + c. measures (``"Name", [Measure]``), + 3. measure filters wrap the table via ``FILTER(SUMMARIZECOLUMNS(...), ...)``, + 4. ``ORDER BY`` the enabled Order By items (``ASC``/``DESC``), or all + attribute columns ascending when no ``order_by`` is supplied. + + Returns an empty string if there is nothing usable to build. + """ + + fields = state.get("fields") or [] + filters = state.get("filters") or [] + + group_cols = [f for f in fields if f.get("kind") == "column"] + measures = [f for f in fields if f.get("kind") == "measure"] + if not group_cols and not measures: + return "" + + def _tbl_ref(name: str) -> str: + return "'" + str(name).replace("'", "''") + "'" + + def _mea_ref(name: str) -> str: + return "[" + str(name).replace("]", "]]") + "]" + + def _col_ref(item: dict) -> str: + ref = item.get("ref") + if ref: + return str(ref) + return _tbl_ref(item.get("table")) + _mea_ref(item.get("name")) + + # Split filters into column predicates (applied inline within + # SUMMARIZECOLUMNS) and measure predicates (applied via an outer FILTER + # over the summarized table, since measures are projected as columns). + col_filters: list = [] + meas_preds: list = [] + for item in filters: + pred = _qb_build_predicate(item) + if not pred: + continue + if item.get("kind") == "measure": + meas_preds.append(pred) + else: + col_filters.append(f"FILTER(KEEPFILTERS(VALUES({_col_ref(item)})), {pred})") + + # SUMMARIZECOLUMNS elements: attributes, then column filters, then + # measures (in that exact order, as required by the engine). + parts: list = [] + for c in group_cols: + parts.append(_col_ref(c)) + parts.extend(col_filters) + for m in measures: + parts.append(f'{_qb_quote_str(m.get("name"))}, {_mea_ref(m.get("name"))}') + + inner = "SUMMARIZECOLUMNS(" + ", ".join(parts) + ")" + + # Measure-based filters cannot live inside SUMMARIZECOLUMNS; wrap the + # whole table in a FILTER referencing the measure columns. + if meas_preds: + inner = "FILTER(" + inner + ", " + " && ".join(meas_preds) + ")" + + dax = "EVALUATE\n" + inner + + # ORDER BY clause. When the builder supplies an explicit "order_by" list + # (from the Order By pane), only the enabled items contribute, each with + # its chosen direction (ASC for A-Z, DESC for Z-A), in pane order. When + # no "order_by" key is present (backward compatibility), fall back to + # ordering by all attribute columns ascending. + order_by = state.get("order_by") + if order_by is None: + if group_cols: + order_cols = ", ".join(_col_ref(c) for c in group_cols) + dax += "\nORDER BY " + order_cols + else: + order_parts: list = [] + for item in order_by: + if not item.get("enabled"): + continue + ref = _col_ref(item) + if not ref: + continue + direction = "DESC" if str(item.get("dir", "")).lower() == "desc" else "ASC" + order_parts.append(f"{ref} {direction}") + if order_parts: + dax += "\nORDER BY " + ", ".join(order_parts) + + return dax + + +def _classify_dax_spans(dax_expression: str) -> list: + """Classify a DAX expression into a flat list of ``{text, kind}`` spans + using the project's DAX parser/tokenizer. + + Spans cover the full string including inter-token whitespace (which has + ``kind = ""``) so the front-end can faithfully reproduce the input + layout while applying syntax colors. + """ + + if not dax_expression: + return [] + try: + from sempy_labs.dax._format import _classify_tokens + + classified = _classify_tokens(dax_expression) + except Exception: + return [{"text": dax_expression, "kind": ""}] + + spans: list = [] + cursor = 0 + for token, kind in classified: + if token.position > cursor: + spans.append({"text": dax_expression[cursor : token.position], "kind": ""}) + spans.append({"text": token.text, "kind": kind or ""}) + cursor = token.position + len(token.text) + if cursor < len(dax_expression): + spans.append({"text": dax_expression[cursor:], "kind": ""}) + return spans + + +def _clean_monitoring_query(value: str) -> str: + """Return the Workspace Monitoring query exactly as displayed in the UI.""" + + return re.sub(r"\s*\[WaitTime:[^\]]*\]\s*$", "", value, flags=re.IGNORECASE).rstrip() + + +def _monitoring_dax_spans(value: str) -> list: + """Classify a Workspace Monitoring query when it is DAX.""" + + query = _clean_monitoring_query(value) + if not re.match(r"^\s*(?:EVALUATE|DEFINE)\b", query, flags=re.IGNORECASE): + return [] + return _classify_dax_spans(query) + + +_FALLBACK_SEARCH_SELECT_CSS = r""" +.slls-ss { position: relative; display: flex; width: 100%; } +.slls-ss-btn { + display: flex; align-items: center; gap: 8px; width: 100%; min-width: 0; + border: 1px solid var(--ui-border-strong); border-radius: 8px; + padding: 8px 10px; background: var(--ui-bg); color: var(--ui-text); + font: inherit; cursor: pointer; +} +.slls-ss-value { flex: 1 1 auto; overflow: hidden; text-align: left; text-overflow: ellipsis; white-space: nowrap; } +.slls-ss-caret { color: var(--ui-text-tertiary); font-size: 11px; } +.slls-ss-panel { + display: none; position: absolute; top: calc(100% + 5px); left: 0; right: 0; + z-index: 30; padding: 6px; border: 1px solid var(--ui-border-strong); + border-radius: 10px; background: var(--ui-bg); box-shadow: var(--ui-shadow-md); +} +.slls-ss-open .slls-ss-panel { display: block; } +.slls-ss-search { + width: 100%; margin-bottom: 5px; border: 1px solid var(--ui-border-strong); + border-radius: 8px; padding: 7px 9px; background: var(--ui-bg-secondary); + color: var(--ui-text); font: inherit; font-size: 13px; +} +.slls-ss-search:focus { outline: none; border-color: var(--ui-accent); } +.slls-ss-list { max-height: 240px; overflow-y: auto; } +.slls-ss-opt { + display: block; width: 100%; border: 0; border-radius: 7px; + padding: 7px 10px; background: transparent; color: var(--ui-text); + font: inherit; font-size: 13px; text-align: left; cursor: pointer; + overflow: hidden; text-overflow: ellipsis; white-space: nowrap; +} +.slls-ss-opt:hover { background: var(--ui-surface-2); } +.slls-ss-opt.slls-ss-selected { color: var(--ui-accent); font-weight: 500; } +.slls-ss-empty { padding: 9px 10px; color: var(--ui-text-tertiary); font-size: 12.5px; } +.slls-ss-disabled { opacity: 0.6; } +""" + +_FALLBACK_SEARCH_SELECT_JS = r""" +function createSearchSelect(config) { + const cfg = config || {}; + const wrap = document.createElement("div"); + wrap.className = "slls-ss"; + const btn = document.createElement("button"); + btn.type = "button"; + btn.className = "slls-ss-btn"; + btn.setAttribute("aria-haspopup", "listbox"); + btn.setAttribute("aria-expanded", "false"); + if (cfg.ariaLabel) btn.setAttribute("aria-label", cfg.ariaLabel); + const valueLabel = document.createElement("span"); + valueLabel.className = "slls-ss-value"; + const caret = document.createElement("span"); + caret.className = "slls-ss-caret"; + caret.textContent = "\u25be"; + btn.appendChild(valueLabel); + btn.appendChild(caret); + wrap.appendChild(btn); + + const panel = document.createElement("div"); + panel.className = "slls-ss-panel"; + const search = document.createElement("input"); + search.type = "search"; + search.className = "slls-ss-search"; + search.placeholder = cfg.searchPlaceholder || "Search\u2026"; + panel.appendChild(search); + const list = document.createElement("div"); + list.className = "slls-ss-list"; + list.setAttribute("role", "listbox"); + panel.appendChild(list); + wrap.appendChild(panel); + + let options = []; + let value = ""; + let emptyLabel = cfg.emptyLabel || "No items"; + let disabled = false; + function close() { + wrap.classList.remove("slls-ss-open"); + btn.setAttribute("aria-expanded", "false"); + } + function renderValue() { + const selected = options.find(option => String(option.value) === String(value)); + valueLabel.textContent = selected ? selected.label : (cfg.placeholder || "Select\u2026"); + btn.disabled = disabled; + wrap.classList.toggle("slls-ss-disabled", disabled); + } + function renderList() { + list.innerHTML = ""; + const term = search.value.trim().toLowerCase(); + const shown = term + ? options.filter(option => String(option.label).toLowerCase().includes(term)) + : options; + if (!shown.length) { + const empty = document.createElement("div"); + empty.className = "slls-ss-empty"; + empty.textContent = options.length ? "No matches" : emptyLabel; + list.appendChild(empty); + return; + } + shown.forEach(option => { + const row = document.createElement("button"); + row.type = "button"; + row.className = "slls-ss-opt"; + row.classList.toggle("slls-ss-selected", String(option.value) === String(value)); + row.textContent = option.label; + row.addEventListener("click", () => { + value = option.value; + renderValue(); + close(); + if (cfg.onChange) cfg.onChange(option); + }); + list.appendChild(row); + }); + } + btn.addEventListener("click", event => { + event.stopPropagation(); + if (disabled) return; + const opening = !wrap.classList.contains("slls-ss-open"); + wrap.classList.toggle("slls-ss-open", opening); + btn.setAttribute("aria-expanded", String(opening)); + if (opening) { + search.value = ""; + renderList(); + search.focus(); + } + }); + panel.addEventListener("click", event => event.stopPropagation()); + search.addEventListener("input", renderList); + search.addEventListener("keydown", event => { + if (event.key === "Escape") { close(); btn.focus(); } + }); + document.addEventListener("click", close); + renderValue(); + return { + el: wrap, + get value() { return value; }, + get label() { + const selected = options.find(option => String(option.value) === String(value)); + return selected ? selected.label : ""; + }, + focus() { btn.focus(); }, + setOptions(items, selectedValue) { + options = items || []; + value = selectedValue || ""; + renderValue(); + renderList(); + }, + setEmptyLabel(text) { emptyLabel = text || "No items"; renderList(); }, + setDisabled(flag) { disabled = !!flag; if (disabled) close(); renderValue(); }, + }; +} +""" + +_FALLBACK_DAX_PERFORMANCE_ICON = ( + '" +) +_FALLBACK_ACTIVITY_ICON = ( + '' +) +_FALLBACK_HAMMER_ICON = ( + '' + '' + '' + '' +) +_FALLBACK_LIST_TREE_ICON = ( + '' + '' + '' + '' +) +_FALLBACK_GIT_BRANCH_ICON = ( + '' + '' + '' +) +_FALLBACK_WORKFLOW_ICON = ( + '' + '' + '' +) +_FALLBACK_SHIELD_CHECK_ICON = ( + '' + '' +) +_FALLBACK_USERS_ICON = ( + '' + '' +) +_FALLBACK_USER_ICON = ( + '' + '' +) +_FALLBACK_ERASER_ICON = ( + '' + '' + '' +) + + +def _visualize_dax_test( + df: pd.DataFrame, + total_duration: int, + fe_duration: int, + se_duration: int, + cpu_time: int, + dax_string: str, + dataset_id: Optional[str], + workspace_id: Optional[str], + dataset_name: Optional[str] = None, + workspace_name: Optional[str] = None, + dark_mode: bool = False, + clear_cache: bool = True, + result_df: Optional[pd.DataFrame] = None, + effective_user_name: Optional[str] = None, + role: Optional[str] = None, +) -> None: + """Render an interactive editable DAX widget for :func:`test` results.""" + + try: + import anywidget + import traitlets + except ImportError as e: + raise ImportError( + "Visualizing 'test()' requires the 'anywidget' package. " + "Install it with: pip install anywidget" + ) from e + + from IPython.display import display + from sempy_labs._daxformatter import _format_dax + from sempy_labs import _ui_components + from sempy_labs._ui_components import ( + LIGHT_THEME_VARS as _UI_LIGHT_VARS, + DARK_THEME_VARS as _UI_DARK_VARS, + SYNTAX_HIGHLIGHT_VARS as _UI_SYNTAX_VARS, + HEADER_CSS as _UI_HEADER_CSS, + ATTRIBUTION_CSS as _UI_ATTRIBUTION_CSS, + ICONS as _UI_ICONS, + ) + _UI_SEARCH_SELECT_CSS = getattr( + _ui_components, "SEARCH_SELECT_CSS", _FALLBACK_SEARCH_SELECT_CSS + ) + _UI_SEARCH_SELECT_JS = getattr( + _ui_components, "SEARCH_SELECT_JS", _FALLBACK_SEARCH_SELECT_JS + ) + _UI_TABLE_COLUMN_RESIZE_JS = _ui_components.TABLE_COLUMN_RESIZE_JS + + # The DAX is intentionally NOT auto-formatted on load (formatting calls + # the external DAX Formatter service and would slow down ``test()``). + # The user can format on demand via the "Format" button in the widget. + # Still normalize line endings to "\n" so the classified token text + # matches the ' + + '' + + '
' + + '' + + '' + + '
'; + nlOverlay.appendChild(nlModal); + root.appendChild(nlOverlay); + + const nlInput = nlModal.querySelector(".dtx-nl-input"); + const nlError = nlModal.querySelector(".dtx-nl-error"); + const nlSubmit = nlModal.querySelector(".dtx-nl-submit"); + const nlCancel = nlModal.querySelector(".dtx-nl-cancel"); + const nlClose = nlModal.querySelector(".dtx-nl-close"); + + function renderNlBtn() { + nlBtn.disabled = model.get("dataset_chosen") !== true; + } + + function renderNlModal() { + const loading = model.get("nl_to_dax_loading") === true; + nlSubmit.disabled = loading; + nlSubmit.textContent = loading ? "Generating\u2026" : "Submit"; + nlInput.disabled = loading; + } + + function openNlModal() { + if (model.get("dataset_chosen") !== true) return; + nlError.textContent = ""; + nlError.style.display = "none"; + nlOverlay.style.display = "flex"; + renderNlModal(); + setTimeout(() => { try { nlInput.focus(); } catch (e) {} }, 0); + } + + function closeNlModal() { + nlOverlay.style.display = "none"; + } + + function submitNl() { + if (model.get("nl_to_dax_loading") === true) return; + const text = String(nlInput.value || "").trim(); + if (!text) { try { nlInput.focus(); } catch (e) {} return; } + nlError.textContent = ""; + nlError.style.display = "none"; + model.set("nl_to_dax_error", ""); + model.set("nl_to_dax_text", text); + model.set("nl_to_dax_loading", true); + model.set("nl_to_dax_trigger", + (model.get("nl_to_dax_trigger") || 0) + 1); + model.save_changes(); + renderNlModal(); + } + + nlBtn.addEventListener("click", () => { + if (nlBtn.disabled) return; + openNlModal(); + }); + nlSubmit.addEventListener("click", submitNl); + nlCancel.addEventListener("click", closeNlModal); + nlClose.addEventListener("click", closeNlModal); + nlOverlay.addEventListener("click", (e) => { + if (e.target === nlOverlay + && model.get("nl_to_dax_loading") !== true) closeNlModal(); + }); + nlInput.addEventListener("keydown", (e) => { + if ((e.ctrlKey || e.metaKey) && e.key === "Enter") { + e.preventDefault(); + submitNl(); + } else if (e.key === "Escape" + && model.get("nl_to_dax_loading") !== true) { + e.preventDefault(); + closeNlModal(); + } + }); + + const undoBtn = document.createElement("button"); + undoBtn.type = "button"; + undoBtn.className = "dtx-hist-btn"; + undoBtn.innerHTML = UNDO_SVG; + undoBtn.title = "Undo (Ctrl/Cmd+Z)"; + undoBtn.setAttribute("aria-label", "Undo"); + qTitleGroup.appendChild(undoBtn); + + const redoBtn = document.createElement("button"); + redoBtn.type = "button"; + redoBtn.className = "dtx-hist-btn"; + redoBtn.innerHTML = REDO_SVG; + redoBtn.title = "Redo (Ctrl/Cmd+Shift+Z or Ctrl/Cmd+Y)"; + redoBtn.setAttribute("aria-label", "Redo"); + qTitleGroup.appendChild(redoBtn); + + const cutBtn = document.createElement("button"); + cutBtn.type = "button"; + cutBtn.className = "dtx-hist-btn"; + cutBtn.innerHTML = CUT_SVG; + cutBtn.title = "Cut selected DAX text (Ctrl/Cmd+X)"; + cutBtn.setAttribute("aria-label", "Cut"); + qTitleGroup.appendChild(cutBtn); + + const copyBtn = document.createElement("button"); + copyBtn.type = "button"; + copyBtn.className = "dtx-hist-btn"; + copyBtn.innerHTML = COPY_SVG; + copyBtn.title = "Copy selected DAX text (Ctrl/Cmd+C)"; + copyBtn.setAttribute("aria-label", "Copy"); + qTitleGroup.appendChild(copyBtn); + + const pasteBtn = document.createElement("button"); + pasteBtn.type = "button"; + pasteBtn.className = "dtx-hist-btn"; + pasteBtn.innerHTML = PASTE_SVG; + pasteBtn.title = "Paste text at the cursor (Ctrl/Cmd+V)"; + pasteBtn.setAttribute("aria-label", "Paste"); + qTitleGroup.appendChild(pasteBtn); + + const expandBtn = document.createElement("button"); + expandBtn.type = "button"; + expandBtn.className = "dtx-fmt-btn dtx-expand-btn"; + expandBtn.innerHTML = EXPAND_SVG; + expandBtn.title = "Open the DAX editor in a large pop-out window"; + expandBtn.setAttribute("aria-label", "Expand DAX editor to full screen"); + qTitleGroup.appendChild(expandBtn); + expandBtn.addEventListener("click", () => openEditorPop()); + + // Cut/copy/paste operate on the DAX query textarea's current selection. + function writeClipboard(text) { + try { + if (navigator.clipboard && navigator.clipboard.writeText) { + return navigator.clipboard.writeText(text); + } + } catch (e) {} + // Fallback for notebook hosts where the Clipboard API is unavailable. + try { + const copyTarget = document.createElement("textarea"); + copyTarget.value = text; + copyTarget.setAttribute("readonly", ""); + copyTarget.style.position = "fixed"; + copyTarget.style.opacity = "0"; + document.body.appendChild(copyTarget); + copyTarget.select(); + const copied = document.execCommand("copy"); + copyTarget.remove(); + return copied ? Promise.resolve() : Promise.reject(new Error("Copy failed")); + } catch (e) { + return Promise.reject(e); + } + } + function doCopy() { + const s = textarea.selectionStart || 0; + const e = textarea.selectionEnd || 0; + if (e <= s) return; + writeClipboard(textarea.value.slice(s, e)); + textarea.focus(); + } + function doCut() { + const s = textarea.selectionStart || 0; + const e = textarea.selectionEnd || 0; + if (e <= s) return; + const sel = textarea.value.slice(s, e); + writeClipboard(sel); + textarea.value = textarea.value.slice(0, s) + textarea.value.slice(e); + textarea.selectionStart = textarea.selectionEnd = s; + textarea.focus(); + commitHistory(textarea.value, false); + model.set("dax_query", textarea.value); + model.save_changes(); + renderHighlight(); + renderFmtBtn(); + } + function doPaste() { + const insert = (text) => { + if (text == null || text === "") { textarea.focus(); return; } + insertAtCursor(String(text)); + renderFmtBtn(); + }; + try { + if (navigator.clipboard && navigator.clipboard.readText) { + navigator.clipboard.readText().then(insert).catch(() => { + textarea.focus(); + }); + return; + } + } catch (e) {} + // Fallback: rely on the textarea's native paste. + textarea.focus(); + document.execCommand("paste"); + } + cutBtn.addEventListener("click", () => doCut()); + copyBtn.addEventListener("click", () => doCopy()); + pasteBtn.addEventListener("click", () => doPaste()); + + undoBtn.addEventListener("click", () => doUndo()); + redoBtn.addEventListener("click", () => doRedo()); + + // Cut/Copy require a non-empty selection in the DAX query textarea. + function renderClipBtns() { + let hasSelection = false; + try { + hasSelection = (textarea.selectionEnd || 0) + > (textarea.selectionStart || 0); + } catch (e) {} + cutBtn.disabled = !hasSelection; + copyBtn.disabled = !hasSelection; + } + + function renderHistBtns() { + undoBtn.disabled = undoStack.length === 0; + redoBtn.disabled = redoStack.length === 0; + renderClipBtns(); + } + + const cacheLabel = document.createElement("label"); + cacheLabel.className = "dtx-cache-label"; + cacheLabel.title = "Clear the dataset cache before running (cold-cache run)"; + const cacheCb = document.createElement("input"); + cacheCb.type = "checkbox"; + cacheCb.checked = model.get("clear_cache") === true; + cacheCb.addEventListener("change", () => { + model.set("clear_cache", cacheCb.checked); + model.save_changes(); + }); + cacheLabel.appendChild(cacheCb); + const cacheSwitch = document.createElement("span"); + cacheSwitch.className = "dtx-cache-switch"; + cacheSwitch.setAttribute("aria-hidden", "true"); + cacheLabel.appendChild(cacheSwitch); + const cacheText = document.createElement("span"); + cacheText.textContent = "Clear cache before run (cold-cache timings)"; + cacheLabel.appendChild(cacheText); + function renderCacheBtn() { + cacheCb.checked = model.get("clear_cache") === true; + } + + // Impersonation: none / user (effective_user_name) / role (role). + const impWrap = document.createElement("div"); + impWrap.className = "dtx-imp-wrap"; + const impLabel = document.createElement("span"); + impLabel.className = "dtx-imp-label"; + impLabel.textContent = "RUN AS"; + const impSegment = document.createElement("div"); + impSegment.className = "dtx-imp-segment"; + impSegment.setAttribute("role", "group"); + impSegment.setAttribute("aria-label", "Run query as"); + const impButtons = {}; + [ + ["none", "No impersonation", SHIELD_CHECK_SVG], + ["role", "Role", USERS_SVG], + ["user", "User", USER_SVG], + ].forEach(([mode, label, icon]) => { + const button = document.createElement("button"); + button.type = "button"; + button.className = "dtx-imp-mode"; + button.innerHTML = icon + `${label}`; + button.title = mode === "none" ? "Run without impersonation" : `Run as ${label.toLowerCase()}`; + button.setAttribute("aria-pressed", "false"); + impButtons[mode] = button; + impSegment.appendChild(button); + }); + // Text input for user impersonation (effective_user_name). + const impInput = document.createElement("input"); + impInput.type = "text"; + impInput.className = "dtx-imp-input"; + // Dropdown of model roles for role impersonation. + const impRoleSel = document.createElement("select"); + impRoleSel.className = "dtx-imp-select dtx-imp-role-select"; + impWrap.appendChild(impLabel); + impWrap.appendChild(impSegment); + impWrap.appendChild(impInput); + impWrap.appendChild(impRoleSel); + queryOptions.appendChild(impWrap); + + function hasRoles() { + const roles = model.get("model_roles") || []; + return roles.length > 0; + } + + function renderRoleOption() { + // Disable the "Role impersonation" choice when the model has no roles. + impButtons.role.disabled = !hasRoles(); + impButtons.role.title = hasRoles() + ? "Run as a security role" + : "Role impersonation is unavailable because this model has no roles"; + } + + function renderRoleChoices() { + const roles = model.get("model_roles") || []; + const current = model.get("impersonation_value") || ""; + impRoleSel.innerHTML = ""; + roles.forEach(rn => { + const o = document.createElement("option"); + o.value = rn; + o.textContent = rn; + impRoleSel.appendChild(o); + }); + if (roles.length) { + impRoleSel.value = roles.includes(current) ? current : roles[0]; + } + } + + function renderImpersonation() { + renderRoleOption(); + let mode = model.get("impersonation_mode") || "none"; + // If role impersonation is selected but the model has no roles, notify + // the user and fall back to no impersonation. + if (mode === "role" && !hasRoles()) { + model.set("impersonation_mode", "none"); + model.set("impersonation_value", ""); + model.set("error_message", + "Role impersonation is not available: this model does not " + + "contain any roles."); + model.save_changes(); + mode = "none"; + } + Object.entries(impButtons).forEach(([buttonMode, button]) => { + const active = buttonMode === mode; + button.classList.toggle("dtx-active", active); + button.setAttribute("aria-pressed", active ? "true" : "false"); + }); + if (mode === "user") { + impInput.style.display = ""; + impRoleSel.style.display = "none"; + impInput.placeholder = "user@domain.com"; + impInput.title = "User to impersonate (effective_user_name)"; + const val = model.get("impersonation_value") || ""; + if (impInput.value !== val) impInput.value = val; + } else if (mode === "role") { + impInput.style.display = "none"; + impRoleSel.style.display = ""; + impRoleSel.title = "Security role to impersonate (role)"; + renderRoleChoices(); + // Persist the resolved selection (e.g. defaulted to first role). + if (impRoleSel.value !== (model.get("impersonation_value") || "")) { + model.set("impersonation_value", impRoleSel.value); + model.save_changes(); + } + } else { + impInput.style.display = "none"; + impRoleSel.style.display = "none"; + } + scheduleResponsiveQueryOptions(); + } + Object.entries(impButtons).forEach(([mode, button]) => { + button.addEventListener("click", () => { + if (mode === "role" && !hasRoles()) return; + model.set("impersonation_mode", mode); + // Reset the value so a stale user string isn't reused as a role, etc. + model.set("impersonation_value", ""); + model.save_changes(); + renderImpersonation(); + }); + }); + impInput.addEventListener("input", () => { + model.set("impersonation_value", impInput.value); + model.save_changes(); + }); + impRoleSel.addEventListener("change", () => { + model.set("impersonation_value", impRoleSel.value); + model.save_changes(); + }); + + const reportCapture = document.createElement("div"); + reportCapture.className = "dtx-report-capture"; + const reportLabel = document.createElement("span"); + reportLabel.className = "dtx-report-label"; + reportLabel.textContent = "Reports"; + const reportSelectBtn = document.createElement("button"); + reportSelectBtn.type = "button"; + reportSelectBtn.className = "dtx-report-select"; + reportSelectBtn.setAttribute("aria-haspopup", "menu"); + reportSelectBtn.setAttribute("aria-expanded", "false"); + const reportSelectIcon = document.createElement("span"); + reportSelectIcon.className = "dtx-report-select-icon"; + reportSelectIcon.innerHTML = REPORT_FILE_SVG; + const reportSelectText = document.createElement("span"); + reportSelectText.className = "dtx-report-select-text"; + const reportSelectChevron = document.createElement("span"); + reportSelectChevron.className = "dtx-report-select-chevron"; + reportSelectChevron.innerHTML = CHEVRON_DOWN_SVG; + reportSelectBtn.appendChild(reportSelectIcon); + reportSelectBtn.appendChild(reportSelectText); + reportSelectBtn.appendChild(reportSelectChevron); + const reportMenu = document.createElement("div"); + reportMenu.className = "dtx-report-menu"; + reportMenu.style.display = "none"; + const reportCaptureBtn = document.createElement("button"); + reportCaptureBtn.type = "button"; + reportCaptureBtn.className = "dtx-report-capture-btn"; + reportCaptureBtn.innerHTML = CAMERA_SVG; + reportCaptureBtn.setAttribute("aria-label", "Capture report queries"); + reportCaptureBtn.title = "Embed the selected report(s), cycle their pages and capture their DAX queries into Trace history"; + const reportProgress = document.createElement("span"); + reportProgress.className = "dtx-report-progress"; + let selectedReportIds = new Set(); + let reportMenuOpen = false; + + function renderReportCapture() { + const reports = model.get("available_reports") || []; + const capturing = model.get("report_capture_loading") === true; + selectedReportIds = new Set( + [...selectedReportIds].filter(id => reports.some(report => report.id === id)) + ); + const count = selectedReportIds.size; + reportSelectText.textContent = count === 0 + ? "Select report(s)…" + : count === 1 ? "1 report" : `${count} reports`; + reportSelectBtn.setAttribute("aria-expanded", reportMenuOpen ? "true" : "false"); + reportSelectBtn.disabled = reports.length === 0 || capturing; + reportSelectBtn.title = reports.length + ? "Choose reports that use this semantic model" + : "No reports use this semantic model"; + reportCaptureBtn.disabled = count === 0 || capturing; + reportProgress.textContent = model.get("report_capture_progress") || ""; + reportMenu.innerHTML = ""; + if (count > 0) { + const clearSelection = document.createElement("button"); + clearSelection.type = "button"; + clearSelection.className = "dtx-report-clear"; + clearSelection.textContent = "Clear selection"; + clearSelection.addEventListener("click", () => { + selectedReportIds.clear(); + renderReportCapture(); + }); + reportMenu.appendChild(clearSelection); + } + reports.forEach(report => { + const option = document.createElement("label"); + option.className = "dtx-report-option"; + option.setAttribute("role", "menuitemcheckbox"); + option.setAttribute("aria-checked", selectedReportIds.has(report.id) ? "true" : "false"); + const checkbox = document.createElement("span"); + checkbox.className = "dtx-report-check" + + (selectedReportIds.has(report.id) ? " dtx-checked" : ""); + checkbox.innerHTML = CHECK_SVG; + const label = document.createElement("span"); + label.className = "dtx-report-option-text"; + label.textContent = report.name; + label.title = report.name; + option.appendChild(checkbox); + option.appendChild(label); + option.addEventListener("click", event => { + event.preventDefault(); + if (selectedReportIds.has(report.id)) selectedReportIds.delete(report.id); + else selectedReportIds.add(report.id); + renderReportCapture(); + }); + reportMenu.appendChild(option); + }); + reportMenu.style.display = reportMenuOpen && reports.length ? "" : "none"; + renderRunBtn(); + scheduleResponsiveQueryOptions(); + } + reportSelectBtn.addEventListener("click", () => { + reportMenuOpen = !reportMenuOpen; + renderReportCapture(); + }); + function hideReportMenuOnOutsidePointer(event) { + if (!reportMenuOpen || reportCapture.contains(event.target)) return; + reportMenuOpen = false; + renderReportCapture(); + } + document.addEventListener("pointerdown", hideReportMenuOnOutsidePointer); + reportCaptureBtn.addEventListener("click", () => { + if (!selectedReportIds.size || model.get("report_capture_loading") === true) return; + reportMenuOpen = false; + model.set("capture_report_ids", [...selectedReportIds]); + model.set("report_capture_loading", true); + model.set("report_capture_progress", "Starting trace…"); + model.set("report_capture_start_trigger", + (model.get("report_capture_start_trigger") || 0) + 1); + model.save_changes(); + renderReportCapture(); + }); + reportCapture.appendChild(reportLabel); + reportCapture.appendChild(reportSelectBtn); + reportCapture.appendChild(reportMenu); + reportCapture.appendChild(reportCaptureBtn); + reportCapture.appendChild(reportProgress); + queryOptions.appendChild(reportCapture); + + let responsiveOptionsFrame = null; + function updateResponsiveQueryOptions() { + responsiveOptionsFrame = null; + queryOptions.classList.remove( + "dtx-hide-report-capture", + "dtx-hide-impersonation", + "dtx-no-optional-controls", + ); + const availableWidth = queryOptions.clientWidth; + if (!availableWidth) return; + const optionsStyle = window.getComputedStyle(queryOptions); + const gap = parseFloat(optionsStyle.gap) || 0; + const impersonationWidth = Math.ceil(impWrap.scrollWidth); + const reportStyle = window.getComputedStyle(reportCapture); + const reportGap = parseFloat(reportStyle.gap) || 0; + const reportParts = [reportLabel, reportSelectBtn, reportCaptureBtn, reportProgress] + .filter(part => part.getClientRects().length && part.getBoundingClientRect().width > 0); + const reportWidth = Math.ceil( + reportParts.reduce( + (width, part) => width + part.getBoundingClientRect().width, + 0, + ) + reportGap * Math.max(0, reportParts.length - 1), + ); + const stacked = optionsStyle.flexDirection === "column"; + const requiredWidth = stacked + ? Math.max(impersonationWidth, reportWidth) + : impersonationWidth + gap + reportWidth; + const capturing = model.get("report_capture_loading") === true; + let hideReport = false; + let hideImpersonation = false; + if (requiredWidth > availableWidth + 2) { + if (capturing) hideImpersonation = true; + else hideReport = true; + } + if (reportWidth > availableWidth + 2) hideReport = true; + if (impersonationWidth > availableWidth + 2) hideImpersonation = true; + const focusedControl = document.activeElement; + if (hideReport) { + queryOptions.classList.add("dtx-hide-report-capture"); + reportMenuOpen = false; + reportMenu.style.display = "none"; + reportSelectBtn.setAttribute("aria-expanded", "false"); + } + if (hideImpersonation) { + queryOptions.classList.add("dtx-hide-impersonation"); + } + if ((hideReport && reportCapture.contains(focusedControl)) + || (hideImpersonation && impWrap.contains(focusedControl))) { + textarea.focus({ preventScroll: true }); + } + queryOptions.classList.toggle( + "dtx-no-optional-controls", + queryOptions.classList.contains("dtx-hide-report-capture") + && queryOptions.classList.contains("dtx-hide-impersonation"), + ); + } + function scheduleResponsiveQueryOptions() { + if (responsiveOptionsFrame !== null) return; + responsiveOptionsFrame = window.requestAnimationFrame(updateResponsiveQueryOptions); + } + const queryOptionsObserver = typeof ResizeObserver === "undefined" + ? null : new ResizeObserver(scheduleResponsiveQueryOptions); + queryOptionsObserver?.observe(queryOptions); + window.addEventListener("resize", scheduleResponsiveQueryOptions); + scheduleResponsiveQueryOptions(); + + const runBtn = document.createElement("button"); + runBtn.type = "button"; + runBtn.className = "dtx-btn"; + function renderRunBtn() { + const running = model.get("is_running") === true; + const capturing = model.get("report_capture_loading") === true; + const chosen = model.get("dataset_chosen") === true; + runBtn.disabled = capturing || (!running && !chosen); + if (running) { + runBtn.classList.add("dtx-btn-stop"); + runBtn.innerHTML = STOP_SVG; + runBtn.title = "Cancel running query"; + runBtn.setAttribute("aria-label", "Cancel running query"); + } else { + runBtn.classList.remove("dtx-btn-stop"); + runBtn.innerHTML = PLAY_SVG; + runBtn.title = "Run DAX query (Ctrl/Cmd+Enter)"; + runBtn.setAttribute("aria-label", "Run DAX query"); + } + runProgress.classList.toggle("dtx-active", running); + runProgress.setAttribute("aria-hidden", running ? "false" : "true"); + } + runBtn.addEventListener("click", () => { + if (model.get("is_running") === true) { + // Cancel + model.set("cancel_trigger", (model.get("cancel_trigger") || 0) + 1); + model.save_changes(); + return; + } + // Run + model.set("dax_query", textarea.value); + model.set("error_message", ""); + model.set("is_running", true); + model.set("run_trigger", (model.get("run_trigger") || 0) + 1); + model.save_changes(); + }); + toolbar.appendChild(runBtn); + + const clearModelCacheBtn = document.createElement("button"); + clearModelCacheBtn.type = "button"; + clearModelCacheBtn.className = "dtx-fmt-btn dtx-clear-model-cache-btn"; + clearModelCacheBtn.innerHTML = ERASER_SVG; + clearModelCacheBtn.setAttribute("aria-label", "Clear model cache"); + function renderClearModelCacheBtn() { + const chosen = model.get("dataset_chosen") === true; + const loading = model.get("cache_clear_loading") === true; + const capturing = model.get("report_capture_loading") === true; + clearModelCacheBtn.disabled = !chosen || loading || capturing; + clearModelCacheBtn.title = loading + ? "Clearing model cache…" + : "Clear model cache"; + } + clearModelCacheBtn.addEventListener("click", () => { + if (model.get("dataset_chosen") !== true + || model.get("cache_clear_loading") === true) return; + model.set("error_message", ""); + model.set("cache_clear_loading", true); + model.set("cache_clear_trigger", + (model.get("cache_clear_trigger") || 0) + 1); + model.save_changes(); + }); + toolbar.appendChild(clearModelCacheBtn); + + const reportCaptureFrame = document.createElement("iframe"); + reportCaptureFrame.className = "dtx-report-host"; + reportCaptureFrame.setAttribute("aria-hidden", "true"); + reportCaptureFrame.setAttribute("tabindex", "-1"); + reportCaptureFrame.title = "Report query capture"; + root.appendChild(reportCaptureFrame); + + let powerBiClientPromise = null; + function reportCaptureContext() { + const captureWindow = reportCaptureFrame.contentWindow; + const captureDocument = reportCaptureFrame.contentDocument; + if (!captureWindow || !captureDocument) return null; + let host = captureDocument.getElementById("report-capture-host"); + if (!host) { + captureDocument.documentElement.style.width = "100%"; + captureDocument.documentElement.style.height = "100%"; + captureDocument.body.style.width = "100%"; + captureDocument.body.style.height = "100%"; + captureDocument.body.style.margin = "0"; + host = captureDocument.createElement("div"); + host.id = "report-capture-host"; + host.style.width = "100%"; + host.style.height = "100%"; + captureDocument.body.appendChild(host); + } + return { captureWindow, captureDocument, host }; + } + function resolvePowerBiClient(context) { + const client = context.captureWindow["powerbi-client"]; + const models = client?.models; + if (!models) return null; + let powerbi = context.captureWindow.powerbi; + if (client?.service?.Service && client?.factories) { + powerbi = new client.service.Service( + client.factories.hpmFactory, + client.factories.wpmpFactory, + client.factories.routerFactory, + ); + } + return powerbi?.embed && powerbi?.reset + ? { models, powerbi, host: context.host } + : null; + } + function ensurePowerBiClient() { + if (powerBiClientPromise) return powerBiClientPromise; + powerBiClientPromise = new Promise((resolve, reject) => { + const context = reportCaptureContext(); + if (!context) { + reject(new Error("The isolated report capture frame is unavailable")); + return; + } + const loadedClient = resolvePowerBiClient(context); + if (loadedClient) { + resolve(loadedClient); + return; + } + const existing = context.captureDocument.querySelector( + 'script[data-sll-powerbi-client="true"]' + ); + if (existing) existing.remove(); + const script = context.captureDocument.createElement("script"); + script.dataset.sllPowerbiClient = "true"; + script.src = "https://cdn.jsdelivr.net/npm/powerbi-client@2.23.1/dist/powerbi.min.js"; + script.async = true; + script.onload = () => { + const client = resolvePowerBiClient(context); + if (client) { + resolve(client); + } else { + reject(new Error("The Power BI client did not initialize correctly")); + } + }; + script.onerror = () => { + script.remove(); + reject(new Error("Failed to load the Power BI client")); + }; + context.captureDocument.head.appendChild(script); + }).catch(error => { + powerBiClientPromise = null; + throw error; + }); + return powerBiClientPromise; + } + + async function cycleReportPages(embedUrl, accessToken) { + const { models, powerbi, host } = await ensurePowerBiClient(); + powerbi.reset(host); + const report = powerbi.embed(host, { + type: "report", + tokenType: models.TokenType.Embed, + accessToken, + embedUrl, + permissions: models.Permissions.Read, + viewMode: models.ViewMode.View, + settings: { panes: { filters: { visible: false }, pageNavigation: { visible: false } } }, + }); + await new Promise(resolve => { + let pages = []; + let index = 0; + let started = false; + let done = false; + let pageTimer = 0; + const finish = () => { + if (done) return; + done = true; + window.clearTimeout(pageTimer); + window.clearTimeout(overallTimer); + resolve(); + }; + const overallTimer = window.setTimeout(finish, 120000); + const activateNext = () => { + if (done) return; + if (index >= pages.length) { finish(); return; } + const page = pages[index++]; + window.clearTimeout(pageTimer); + pageTimer = window.setTimeout(activateNext, 30000); + Promise.resolve(page.setActive()).catch(activateNext); + }; + report.on("loaded", () => { + report.getPages().then(value => { + pages = value; + started = true; + activateNext(); + }).catch(finish); + }); + report.on("rendered", () => { + if (!started || done) return; + window.clearTimeout(pageTimer); + pageTimer = window.setTimeout(activateNext, 1800); + }); + report.on("error", finish); + }); + powerbi.reset(host); + } + + function checkpointReportCapture(payload, report, index) { + const checkpointId = `${payload.nonce}-${index}-${Date.now()}`; + return new Promise((resolve, reject) => { + let settled = false; + const finish = (error) => { + if (settled) return; + settled = true; + window.clearTimeout(timer); + model.off("change:report_capture_checkpoint_ack", onAck); + if (error) reject(error); + else resolve(); + }; + const onAck = () => { + if (model.get("report_capture_checkpoint_ack") === checkpointId) { + finish(); + } + }; + const timer = window.setTimeout( + () => finish(new Error(`Timed out collecting queries for ${report.name}`)), + 30000, + ); + model.on("change:report_capture_checkpoint_ack", onAck); + model.set("report_capture_checkpoint", { + nonce: payload.nonce, + checkpoint_id: checkpointId, + report_id: report.id, + report_name: report.name, + workspace_name: report.workspace_name || "", + }); + model.set("report_capture_checkpoint_trigger", + (model.get("report_capture_checkpoint_trigger") || 0) + 1); + model.save_changes(); + onAck(); + }); + } + + async function runReportCapture(payload) { + if (!payload || !payload.nonce || !payload.token) return; + let clientError = ""; + try { + const reports = payload.reports || []; + for (let index = 0; index < reports.length; index++) { + const report = reports[index]; + model.set("report_capture_progress", + `Cycling pages — ${report.name} (${index + 1}/${reports.length})`); + model.save_changes(); + await cycleReportPages(report.embed_url, payload.token); + model.set("report_capture_progress", + `Collecting queries — ${report.name} (${index + 1}/${reports.length})`); + model.save_changes(); + await checkpointReportCapture(payload, report, index); + } + } catch (error) { + clientError = error && error.message ? error.message : String(error); + } finally { + model.set("report_capture_client_error", clientError); + model.set("report_capture_progress", "Collecting captured queries…"); + model.set("report_capture_finish_trigger", + (model.get("report_capture_finish_trigger") || 0) + 1); + model.save_changes(); + } + } + + // ---------- Analyze (DAX performance analysis) ---------- + const analyzeBtn = document.createElement("button"); + analyzeBtn.type = "button"; + analyzeBtn.className = "dtx-fmt-btn dtx-analyze-btn"; + analyzeBtn.innerHTML = ANALYZE_SVG; + analyzeBtn.title = + "Generate a DAX performance analysis (query, model metadata, " + + "dependencies, trace, query plan, Vertipaq Analyzer)"; + analyzeBtn.setAttribute("aria-label", "Generate DAX performance analysis"); + function renderAnalyzeBtn() { + const chosen = model.get("dataset_chosen") === true; + const loading = model.get("performance_loading") === true; + analyzeBtn.disabled = !chosen || loading; + analyzeBtn.classList.toggle("dtx-fmt-loading", loading); + } + analyzeBtn.addEventListener("click", () => { + if (analyzeBtn.disabled) return; + // Persist the current editor text so the analysis reflects it. + model.set("dax_query", textarea.value); + triggerPerformanceAnalysis(true); + }); + toolbar.appendChild(analyzeBtn); + + const textarea = document.createElement("textarea"); + textarea.className = "dtx-query"; + textarea.spellcheck = false; + textarea.value = model.get("dax_query") || ""; + + // ---------- Undo / redo history (DAX query pane only) ---------- + // A dedicated history stack for the editor text. A custom stack is used + // (instead of the textarea's native undo) so programmatic edits such as + // drag-and-drop, "Define" and "Format" are also undoable. + const undoStack = []; + const redoStack = []; + const HISTORY_LIMIT = 200; + let histPresent = { value: textarea.value, s: 0, e: 0 }; + let lastTypingTs = 0; + let lastEditWasTyping = false; + + // Record a change to the editor text in the history. ``coalesce`` merges + // rapid consecutive typing into a single undo entry. + function commitHistory(newValue, coalesce) { + if (newValue === histPresent.value) return; + const now = Date.now(); + const canCoalesce = coalesce && lastEditWasTyping + && (now - lastTypingTs < 500) && undoStack.length > 0; + if (!canCoalesce) { + undoStack.push({ + value: histPresent.value, s: histPresent.s, e: histPresent.e }); + if (undoStack.length > HISTORY_LIMIT) undoStack.shift(); + } + histPresent = { + value: newValue, + s: textarea.selectionStart || 0, + e: textarea.selectionEnd || 0, + }; + redoStack.length = 0; + lastTypingTs = now; + lastEditWasTyping = !!coalesce; + renderHistBtns(); + } + + function applyHistoryState(st) { + textarea.value = st.value; + try { + textarea.selectionStart = st.s; + textarea.selectionEnd = st.e; + } catch (e) {} + histPresent = { value: st.value, s: st.s, e: st.e }; + lastEditWasTyping = false; + model.set("dax_query", textarea.value); + model.save_changes(); + renderHighlight(); + renderFmtBtn(); + renderHistBtns(); + autoGrowEditor(); + textarea.focus(); + } + + function doUndo() { + if (!undoStack.length) return; + redoStack.push({ + value: histPresent.value, + s: textarea.selectionStart || 0, + e: textarea.selectionEnd || 0, + }); + applyHistoryState(undoStack.pop()); + } + + function doRedo() { + if (!redoStack.length) return; + undoStack.push({ + value: histPresent.value, + s: textarea.selectionStart || 0, + e: textarea.selectionEnd || 0, + }); + applyHistoryState(redoStack.pop()); + } + + const queryWrap = document.createElement("div"); + queryWrap.className = "dtx-query-wrap"; + const hl = document.createElement("pre"); + hl.className = "dtx-query-hl"; + hl.setAttribute("aria-hidden", "true"); + queryWrap.appendChild(hl); + queryWrap.appendChild(textarea); + queryBlock.appendChild(queryWrap); + const queryCacheRow = document.createElement("div"); + queryCacheRow.className = "dtx-query-cache-row"; + queryCacheRow.appendChild(cacheLabel); + queryBlock.appendChild(queryCacheRow); + + // ---------- Auto-grow ---------- + // Grow the editor vertically to fit its content, up to EDITOR_MAX_ROWS + // rows, so generated queries (Query Builder / natural language) are fully + // visible without scrolling. Grow-only, so a manual resize is preserved. + const EDITOR_MAX_ROWS = 20; + function autoGrowEditor() { + // The pop-out editor manages its own full-height layout. + if (root.classList.contains("dtx-editor-pop")) return; + const lineH = 18; // 12px font-size * 1.5 line-height + const chrome = 26; // 12+12 padding + 2 border (border-box) + const maxH = lineH * EDITOR_MAX_ROWS + chrome; + const prevH = textarea.getBoundingClientRect().height; + textarea.style.height = "auto"; + const needed = Math.min(textarea.scrollHeight + 2, maxH); + textarea.style.height = Math.max(needed, prevH) + "px"; + hl.scrollTop = textarea.scrollTop; + hl.scrollLeft = textarea.scrollLeft; + } + + // ---------- Pop-out (full-screen) editor ---------- + const editorOverlay = document.createElement("div"); + editorOverlay.className = "dtx-editor-overlay"; + const editorModal = document.createElement("div"); + editorModal.className = "dtx-editor-modal"; + const editorHead = document.createElement("div"); + editorHead.className = "dtx-editor-head"; + const editorTitle = document.createElement("div"); + editorTitle.className = "dtx-editor-title"; + editorTitle.textContent = "DAX Query"; + const editorClose = document.createElement("button"); + editorClose.type = "button"; + editorClose.className = "dtx-hist-btn"; + editorClose.innerHTML = CLOSE_SVG; + editorClose.title = "Close the full-screen editor (Esc)"; + editorClose.setAttribute("aria-label", "Close the full-screen editor"); + editorHead.appendChild(editorTitle); + editorHead.appendChild(editorClose); + const editorBody = document.createElement("div"); + editorBody.className = "dtx-editor-body"; + editorModal.appendChild(editorHead); + editorModal.appendChild(editorBody); + editorOverlay.appendChild(editorModal); + root.appendChild(editorOverlay); + + // A placeholder marking the editor's home so it can be returned exactly + // where it was when the pop-out closes. Relocating the same DOM nodes + // keeps all existing wiring (history, drag-drop, highlight) intact. + const editorHome = document.createComment("dtx-editor-home"); + let editorPopped = false; + + function openEditorPop() { + if (editorPopped) return; + editorPopped = true; + queryWrap.parentNode.insertBefore(editorHome, queryWrap); + editorBody.appendChild(queryWrap); + root.classList.add("dtx-editor-pop"); + editorOverlay.classList.add("dtx-open"); + // Hand height control to the full-height layout. + textarea.style.height = ""; + renderHighlight(); + setTimeout(() => { try { textarea.focus(); } catch (e) {} }, 0); + } + + function closeEditorPop() { + if (!editorPopped) return; + editorPopped = false; + editorOverlay.classList.remove("dtx-open"); + root.classList.remove("dtx-editor-pop"); + if (editorHome.parentNode) { + editorHome.parentNode.insertBefore(queryWrap, editorHome); + editorHome.parentNode.removeChild(editorHome); + } else { + queryBlock.appendChild(queryWrap); + } + renderHighlight(); + autoGrowEditor(); + try { textarea.focus(); } catch (e) {} + } + + editorClose.addEventListener("click", closeEditorPop); + editorOverlay.addEventListener("click", (e) => { + if (e.target === editorOverlay) closeEditorPop(); + }); + document.addEventListener("keydown", (e) => { + if (e.key === "Escape" && editorPopped) { + e.preventDefault(); + closeEditorPop(); + } + }); + + function renderDaxTokens(tokens, text) { + let total = 0; + for (const token of tokens) total += (token.text || "").length; + if (!tokens.length || total !== text.length) return escapeHtml(text); + return tokens.map(token => { + const tokenText = escapeHtml(token.text); + return token.kind + ? `${tokenText}` + : tokenText; + }).join(""); + } + + function renderHighlight() { + const tokens = model.get("dax_tokens") || []; + const text = textarea.value; + if (text.length === 0) { + // The textarea's own text is transparent, so render the + // placeholder helper text in the highlight overlay instead. It + // disappears as soon as the user types anything. + hl.innerHTML = '' + + 'EVALUATE — type a DAX query here, drag model objects in ' + + 'from the left, or use the Query Builder.' + + ''; + hl.scrollTop = textarea.scrollTop; + hl.scrollLeft = textarea.scrollLeft; + return; + } + // An out-of-sync token list (while the user is typing) falls back to + // escaped plain text until Python reclassifies the query. + hl.innerHTML = renderDaxTokens(tokens, text) + "\n"; + hl.scrollTop = textarea.scrollTop; + hl.scrollLeft = textarea.scrollLeft; + } + + textarea.addEventListener("input", () => { + commitHistory(textarea.value, true); + model.set("dax_query", textarea.value); + model.save_changes(); + renderHighlight(); + renderFmtBtn(); + }); + // Keep the Cut/Copy buttons in sync with the current text selection. + ["select", "keyup", "mouseup", "focus", "blur", "input"].forEach((evt) => { + textarea.addEventListener(evt, renderClipBtns); + }); + document.addEventListener("selectionchange", () => { + if (document.activeElement === textarea) renderClipBtns(); + }); + textarea.addEventListener("scroll", () => { + hl.scrollTop = textarea.scrollTop; + hl.scrollLeft = textarea.scrollLeft; + }); + // Ctrl/Cmd+Enter to run; Ctrl/Cmd+Z / Ctrl/Cmd+Shift+Z / Ctrl/Cmd+Y for + // undo/redo (handled by the editor's own history stack). + textarea.addEventListener("keydown", (e) => { + if ((e.ctrlKey || e.metaKey) && e.key === "Enter") { + e.preventDefault(); + runBtn.click(); + return; + } + if (e.ctrlKey || e.metaKey) { + const k = e.key.toLowerCase(); + if (k === "z" && !e.shiftKey) { + e.preventDefault(); + doUndo(); + return; + } + if ((k === "z" && e.shiftKey) || k === "y") { + e.preventDefault(); + doRedo(); + return; + } + } + }); + + // Drag-and-drop model objects into the editor. + function insertAtCursor(text) { + const start = textarea.selectionStart != null + ? textarea.selectionStart : textarea.value.length; + const end = textarea.selectionEnd != null + ? textarea.selectionEnd : textarea.value.length; + const before = textarea.value.slice(0, start); + const after = textarea.value.slice(end); + textarea.value = before + text + after; + const caret = start + text.length; + textarea.selectionStart = textarea.selectionEnd = caret; + textarea.focus(); + commitHistory(textarea.value, false); + model.set("dax_query", textarea.value); + model.save_changes(); + renderHighlight(); + autoGrowEditor(); + } + + // Prepend a `DEFINE MEASURE` block for the given measure above the + // existing query text. If a DEFINE block already exists, the new + // measure line is added under it instead of duplicating the keyword. + function defineMeasure(meta) { + const table = meta.table || ""; + const name = meta.name || ""; + const expr = String(meta.expression == null ? "" : meta.expression).trim(); + const ref = (table ? daxTableRef(table) : "") + + "[" + String(name).replace(/\]/g, "]]") + "]"; + const measureLine = " MEASURE " + ref + " = " + expr; + const existing = textarea.value; + const lead = /^(\s*)DEFINE\b[^\n]*/i.exec(existing); + let newText; + if (lead) { + const idx = lead.index + lead[0].length; + newText = existing.slice(0, idx) + "\n" + measureLine + + existing.slice(idx); + } else { + newText = "DEFINE\n" + measureLine + "\n\n" + existing; + } + textarea.value = newText; + commitHistory(textarea.value, false); + model.set("dax_query", textarea.value); + model.save_changes(); + renderHighlight(); + autoGrowEditor(); + textarea.focus(); + } + textarea.addEventListener("dragover", (e) => { + if (dragPayload != null) { + e.preventDefault(); + e.dataTransfer.dropEffect = "copy"; + textarea.classList.add("dtx-drop-target"); + } + }); + textarea.addEventListener("dragleave", () => { + textarea.classList.remove("dtx-drop-target"); + }); + textarea.addEventListener("drop", (e) => { + const text = (e.dataTransfer && e.dataTransfer.getData("text/plain")) + || dragPayload; + if (text) { + e.preventDefault(); + textarea.classList.remove("dtx-drop-target"); + // Position the caret where the drop happened, when supported. + if (document.caretRangeFromPoint) { + const range = document.caretRangeFromPoint(e.clientX, e.clientY); + if (range && range.startContainer === textarea.firstChild) { + textarea.selectionStart = textarea.selectionEnd = + range.startOffset; + } + } + insertAtCursor(text); + } + }); + + // ---------- Error message ---------- + const errorEl = document.createElement("div"); + errorEl.className = "dtx-error"; + errorEl.style.display = "none"; + main.appendChild(errorEl); + function renderError() { + const msg = model.get("error_message") || ""; + pickerError.textContent = msg; + pickerError.style.display = msg ? "" : "none"; + if (msg) { + errorEl.textContent = msg; + errorEl.style.display = ""; + } else { + errorEl.textContent = ""; + errorEl.style.display = "none"; + } + } + + // ---------- View toggle (Trace details / Query result) ---------- + const viewToolbar = document.createElement("div"); + viewToolbar.className = "dtx-view-toolbar"; + main.appendChild(viewToolbar); + + const viewTitle = document.createElement("div"); + viewTitle.className = "dtx-view-title"; + viewTitle.textContent = "Results"; + viewToolbar.appendChild(viewTitle); + + const seg = document.createElement("div"); + seg.className = "dtx-seg"; + const segTrace = document.createElement("button"); + segTrace.type = "button"; + segTrace.className = "dtx-seg-btn"; + segTrace.textContent = "Trace details"; + const segResult = document.createElement("button"); + segResult.type = "button"; + segResult.className = "dtx-seg-btn"; + segResult.textContent = "Query result"; + const segChart = document.createElement("button"); + segChart.type = "button"; + segChart.className = "dtx-seg-btn"; + segChart.textContent = "Chart"; + const segHistory = document.createElement("button"); + segHistory.type = "button"; + segHistory.className = "dtx-seg-btn"; + segHistory.textContent = "Trace history"; + const segQueryPlan = document.createElement("button"); + segQueryPlan.type = "button"; + segQueryPlan.className = "dtx-seg-btn"; + segQueryPlan.textContent = "DAX query plan"; + const segDependencies = document.createElement("button"); + segDependencies.type = "button"; + segDependencies.className = "dtx-seg-btn"; + segDependencies.textContent = "Query dependencies"; + const segVertipaq = document.createElement("button"); + segVertipaq.type = "button"; + segVertipaq.className = "dtx-seg-btn"; + segVertipaq.textContent = "Vertipaq analyzer"; + const segPerf = document.createElement("button"); + segPerf.type = "button"; + segPerf.className = "dtx-seg-btn"; + segPerf.textContent = "Performance analysis"; + const segExecMetrics = document.createElement("button"); + segExecMetrics.type = "button"; + segExecMetrics.className = "dtx-seg-btn"; + segExecMetrics.textContent = "Execution metrics"; + seg.appendChild(segTrace); + seg.appendChild(segResult); + seg.appendChild(segQueryPlan); + seg.appendChild(segChart); + seg.appendChild(segDependencies); + seg.appendChild(segVertipaq); + seg.appendChild(segPerf); + seg.appendChild(segExecMetrics); + seg.appendChild(segHistory); + viewToolbar.appendChild(seg); + + // ---------- DAX Query Plan toggle (Logical / Physical) ---------- + const planSeg = document.createElement("div"); + planSeg.className = "dtx-seg dtx-plan-seg"; + planSeg.style.display = "none"; + const planLogicalBtn = document.createElement("button"); + planLogicalBtn.type = "button"; + planLogicalBtn.className = "dtx-seg-btn"; + planLogicalBtn.textContent = "Logical Query Plan"; + const planPhysicalBtn = document.createElement("button"); + planPhysicalBtn.type = "button"; + planPhysicalBtn.className = "dtx-seg-btn"; + planPhysicalBtn.textContent = "Physical Query Plan"; + planSeg.appendChild(planLogicalBtn); + planSeg.appendChild(planPhysicalBtn); + // Show the Logical/Physical toggle to the left of the tab group. + viewToolbar.insertBefore(planSeg, seg); + planLogicalBtn.addEventListener("click", () => { + model.set("query_plan_type", "Logical"); + model.save_changes(); + }); + planPhysicalBtn.addEventListener("click", () => { + model.set("query_plan_type", "Physical"); + model.save_changes(); + }); + + // ---------- Query Dependencies toggle (Tree / Columns) ---------- + const depSeg = document.createElement("div"); + depSeg.className = "dtx-seg dtx-dep-seg"; + depSeg.style.display = "none"; + const depTreeBtn = document.createElement("button"); + depTreeBtn.type = "button"; + depTreeBtn.className = "dtx-seg-btn"; + depTreeBtn.textContent = "Tree"; + const depColumnsBtn = document.createElement("button"); + depColumnsBtn.type = "button"; + depColumnsBtn.className = "dtx-seg-btn"; + depColumnsBtn.textContent = "Columns"; + depSeg.appendChild(depTreeBtn); + depSeg.appendChild(depColumnsBtn); + // Show the Tree/Columns toggle to the left of the tab group. + viewToolbar.insertBefore(depSeg, seg); + depTreeBtn.addEventListener("click", () => { + model.set("dependency_view", "tree"); + model.save_changes(); + }); + depColumnsBtn.addEventListener("click", () => { + model.set("dependency_view", "columns"); + model.save_changes(); + }); + + // ---------- Vertipaq Analyzer section toggle ---------- + // The Vertipaq Analyzer returns several dataframes (Model Summary, + // Tables, Partitions, Columns, Relationships, Hierarchies). This + // segmented control (built dynamically from the returned sections) lets + // the user switch between them. It is shown only on the Vertipaq tab. + const vpSeg = document.createElement("div"); + vpSeg.className = "dtx-seg dtx-vp-seg"; + vpSeg.style.display = "none"; + main.appendChild(vpSeg); + + function buildVertipaqSeg() { + const sections = model.get("vertipaq_sections") || []; + let active = model.get("vertipaq_section") || ""; + if (!sections.some(s => s.name === active)) { + active = sections.length ? sections[0].name : ""; + } + vpSeg.innerHTML = ""; + sections.forEach(s => { + const b = document.createElement("button"); + b.type = "button"; + b.className = "dtx-seg-btn"; + b.textContent = s.name; + b.classList.toggle("dtx-seg-btn-on", s.name === active); + b.addEventListener("click", () => { + model.set("vertipaq_section", s.name); + model.save_changes(); + }); + vpSeg.appendChild(b); + }); + } + + const histDownloadBtn = document.createElement("button"); + histDownloadBtn.type = "button"; + histDownloadBtn.className = "dtx-hist-download"; + histDownloadBtn.innerHTML = DOWNLOAD_SVG; + histDownloadBtn.title = "Download the trace history as an Excel file"; + histDownloadBtn.setAttribute("aria-label", "Download trace history as Excel"); + histDownloadBtn.style.display = "none"; + viewToolbar.appendChild(histDownloadBtn); + histDownloadBtn.addEventListener("click", () => { + const hist = model.get("trace_history") || []; + if (!hist.length) return; + model.set("download_history_trigger", (model.get("download_history_trigger") || 0) + 1); + model.save_changes(); + }); + + const histClearBtn = document.createElement("button"); + histClearBtn.type = "button"; + histClearBtn.className = "dtx-hist-download"; + histClearBtn.innerHTML = TRASH_SVG; + histClearBtn.title = "Clear trace history"; + histClearBtn.setAttribute("aria-label", "Clear trace history"); + histClearBtn.style.display = "none"; + viewToolbar.appendChild(histClearBtn); + + const clearHistoryOverlay = document.createElement("div"); + clearHistoryOverlay.className = "dtx-confirm-overlay"; + clearHistoryOverlay.innerHTML = ` + `; + root.appendChild(clearHistoryOverlay); + const clearHistoryCancel = clearHistoryOverlay.querySelector(".dtx-confirm-cancel"); + const clearHistoryConfirm = clearHistoryOverlay.querySelector(".dtx-confirm-clear"); + + function closeClearHistoryDialog() { + clearHistoryOverlay.classList.remove("dtx-open"); + histClearBtn.focus(); + } + function openClearHistoryDialog() { + if (!(model.get("trace_history") || []).length) return; + clearHistoryOverlay.classList.add("dtx-open"); + window.setTimeout(() => clearHistoryCancel.focus(), 0); + } + histClearBtn.addEventListener("click", openClearHistoryDialog); + clearHistoryCancel.addEventListener("click", closeClearHistoryDialog); + clearHistoryConfirm.addEventListener("click", () => { + model.set("trace_history", []); + model.save_changes(); + clearHistoryOverlay.classList.remove("dtx-open"); + segHistory.focus(); + showToast("Trace history cleared"); + }); + clearHistoryOverlay.addEventListener("click", event => { + if (event.target === clearHistoryOverlay) closeClearHistoryDialog(); + }); + clearHistoryOverlay.addEventListener("keydown", event => { + if (event.key === "Escape") { + event.preventDefault(); + closeClearHistoryDialog(); + } else if (event.key === "Tab") { + if (!event.shiftKey && document.activeElement === clearHistoryConfirm) { + event.preventDefault(); + clearHistoryCancel.focus(); + } else if (event.shiftKey && document.activeElement === clearHistoryCancel) { + event.preventDefault(); + clearHistoryConfirm.focus(); + } + } + }); + + const resultDownloadBtn = document.createElement("button"); + resultDownloadBtn.type = "button"; + resultDownloadBtn.className = "dtx-hist-download"; + resultDownloadBtn.innerHTML = DOWNLOAD_SVG; + resultDownloadBtn.title = "Download the query result as an Excel file"; + resultDownloadBtn.setAttribute("aria-label", "Download query result as Excel"); + resultDownloadBtn.style.display = "none"; + viewToolbar.appendChild(resultDownloadBtn); + resultDownloadBtn.addEventListener("click", () => { + if ((model.get("result_total_rows") || 0) <= 0) return; + model.set("download_result_trigger", (model.get("download_result_trigger") || 0) + 1); + model.save_changes(); + }); + // Tracks the DAX query text that the currently displayed dependency tree + // was computed for, so dependencies are only recomputed when it changes. + let lastDepQuery = null; + segTrace.addEventListener("click", () => { + model.set("view_mode", "trace"); + model.save_changes(); + }); + segResult.addEventListener("click", () => { + model.set("view_mode", "result"); + model.save_changes(); + }); + segChart.addEventListener("click", () => { + if (segChart.disabled) return; + model.set("view_mode", "chart"); + model.save_changes(); + }); + segHistory.addEventListener("click", () => { + model.set("view_mode", "history"); + model.save_changes(); + }); + segQueryPlan.addEventListener("click", () => { + model.set("view_mode", "queryplan"); + model.save_changes(); + }); + segDependencies.addEventListener("click", () => { + model.set("view_mode", "dependencies"); + // Only recompute dependencies when the DAX query text has changed + // since the last computation; otherwise reuse the existing tree. + const curQuery = model.get("dax_query") || ""; + if (curQuery !== lastDepQuery) { + lastDepQuery = curQuery; + model.set("dependencies_trigger", (model.get("dependencies_trigger") || 0) + 1); + } + model.save_changes(); + }); + + // Tracks the dataset id the displayed Vertipaq Analyzer results were + // computed for, so they are only recomputed when the active model changes. + let lastVertipaqDataset = null; + segVertipaq.addEventListener("click", () => { + model.set("view_mode", "vertipaq"); + // Vertipaq stats are model-level: only (re)compute when the active + // model changes since the last run, otherwise reuse the results. + const curDs = model.get("active_dataset_id") || ""; + if (curDs && curDs !== lastVertipaqDataset) { + lastVertipaqDataset = curDs; + model.set("vertipaq_trigger", (model.get("vertipaq_trigger") || 0) + 1); + } + model.save_changes(); + }); + + // Trigger a fresh DAX performance analysis and switch to its tab. The + // analysis combines the DAX query, model metadata, query dependencies, + // trace details, the DAX query plan and (when meaningful) Vertipaq + // Analyzer statistics. When ``force`` is false the analysis is only run if + // no results exist yet (used when simply switching to the tab); the + // Analyze button passes ``force = true`` to always recompute. + function triggerPerformanceAnalysis(force) { + model.set("view_mode", "performance"); + const hasResults = + Object.keys(model.get("performance_summary") || {}).length > 0; + if (model.get("dataset_chosen") === true && + model.get("performance_loading") !== true && + (force === true || !hasResults)) { + model.set("performance_loading", true); + model.set("performance_trigger", + (model.get("performance_trigger") || 0) + 1); + } + model.save_changes(); + } + segPerf.addEventListener("click", () => triggerPerformanceAnalysis(false)); + segExecMetrics.addEventListener("click", () => { + model.set("view_mode", "execmetrics"); + model.save_changes(); + }); + // Maximum number of rows for which we render an interactive chart. + // Beyond this, the Chart option is disabled to keep the widget responsive. + const CHART_MAX_ROWS = 200; + + function chartEligibility() { + const cols = model.get("result_columns") || []; + const rows = model.get("result_rows") || []; + const total = model.get("result_total_rows") || 0; + const truncated = model.get("result_truncated") === true; + if (!cols.length || !rows.length) { + return { ok: false, reason: "No query result available." }; + } + if (truncated || total > CHART_MAX_ROWS) { + return { + ok: false, + reason: `Too many rows to chart (${total.toLocaleString()}; limit ${CHART_MAX_ROWS.toLocaleString()}).`, + }; + } + const numericCols = cols.map((_, i) => + rows.some(r => typeof r[i] === "number") && + rows.every(r => r[i] === null || typeof r[i] === "number") + ); + if (!numericCols.some(Boolean)) { + return { ok: false, reason: "No numeric column to chart." }; + } + return { ok: true, numericCols }; + } + + function renderSeg() { + const mode = model.get("view_mode") || "trace"; + segTrace.classList.toggle("dtx-seg-btn-on", mode === "trace"); + segResult.classList.toggle("dtx-seg-btn-on", mode === "result"); + segChart.classList.toggle("dtx-seg-btn-on", mode === "chart"); + segHistory.classList.toggle("dtx-seg-btn-on", mode === "history"); + segQueryPlan.classList.toggle("dtx-seg-btn-on", mode === "queryplan"); + segDependencies.classList.toggle("dtx-seg-btn-on", mode === "dependencies"); + segVertipaq.classList.toggle("dtx-seg-btn-on", mode === "vertipaq"); + segPerf.classList.toggle("dtx-seg-btn-on", mode === "performance"); + segExecMetrics.classList.toggle("dtx-seg-btn-on", mode === "execmetrics"); + const elig = chartEligibility(); + segChart.disabled = !elig.ok; + segChart.title = elig.ok ? "Show simple chart of the result" : elig.reason; + const hist = model.get("trace_history") || []; + histDownloadBtn.style.display = (mode === "history") ? "" : "none"; + histDownloadBtn.disabled = !hist.length; + histClearBtn.style.display = (mode === "history") ? "" : "none"; + histClearBtn.disabled = !hist.length; + resultDownloadBtn.style.display = (mode === "result") ? "" : "none"; + resultDownloadBtn.disabled = (model.get("result_total_rows") || 0) <= 0; + // Logical/Physical toggle is only relevant on the DAX Query Plan tab. + const planType = model.get("query_plan_type") || "Logical"; + planSeg.style.display = (mode === "queryplan") ? "" : "none"; + planLogicalBtn.classList.toggle("dtx-seg-btn-on", planType === "Logical"); + planPhysicalBtn.classList.toggle("dtx-seg-btn-on", planType === "Physical"); + // Tree/Columns toggle is only relevant on the Query Dependencies tab. + const depView = model.get("dependency_view") || "tree"; + depSeg.style.display = (mode === "dependencies") ? "" : "none"; + depTreeBtn.classList.toggle("dtx-seg-btn-on", depView === "tree"); + depColumnsBtn.classList.toggle("dtx-seg-btn-on", depView === "columns"); + // Section toggle is only relevant on the Vertipaq Analyzer tab. + const vpVisible = (mode === "vertipaq"); + vpSeg.style.display = vpVisible ? "" : "none"; + if (vpVisible) buildVertipaqSeg(); + } + + const resultMeta = document.createElement("div"); + resultMeta.className = "dtx-result-meta"; + resultMeta.style.display = "none"; + main.appendChild(resultMeta); + + const tableWrap = document.createElement("div"); + tableWrap.className = "dtx-table-wrap"; + main.appendChild(tableWrap); + + const outputColumnWidths = new Map(); + function outputTableKey(table) { + const tableName = table.className || "output-table"; + const headings = Array.from(table.querySelectorAll("thead tr:first-child th")) + .map(th => th.textContent.trim()).join("|"); + return `${tableName}:${headings}`; + } + function installColumnResizers(table) { + sllsInstallColumnResizers(table, { + widths: outputColumnWidths, + key: outputTableKey, + minWidth: 56, + handleClass: "dtx-column-resizer", + resizableClass: "dtx-resizable", + resizingClass: "dtx-resizing", + onWidthsChanged: updateVertipaqFrozen, + }); + } + function enhanceOutputTables() { + tableWrap.querySelectorAll("table").forEach(installColumnResizers); + } + const outputTableObserver = new MutationObserver(enhanceOutputTables); + outputTableObserver.observe(tableWrap, { childList: true, subtree: true }); + + // ---------- Workspace monitoring ---------- + let monitoringOpen = false; + let monitoringFullscreen = false; + let monitoringSort = null; + let monitoringSearch = ""; + let monitoringContentHeight = 260; + const monitoringPane = document.createElement("section"); + monitoringPane.className = "dtx-monitoring"; + const monitoringResizer = document.createElement("div"); + monitoringResizer.className = "dtx-monitoring-resizer"; + monitoringResizer.setAttribute("role", "separator"); + monitoringResizer.setAttribute("aria-orientation", "horizontal"); + monitoringResizer.setAttribute("aria-label", "Resize workspace monitoring panel"); + monitoringPane.appendChild(monitoringResizer); + const monitoringHead = document.createElement("div"); + monitoringHead.className = "dtx-monitoring-head"; + monitoringPane.appendChild(monitoringHead); + const monitoringTitleBtn = document.createElement("button"); + monitoringTitleBtn.type = "button"; + monitoringTitleBtn.className = "dtx-monitoring-title-btn"; + monitoringTitleBtn.innerHTML = `${CHEVRON_DOWN_SVG}` + + `${ACTIVITY_SVG}` + + `Workspace monitoring` + + `· slowest recent queries`; + monitoringHead.appendChild(monitoringTitleBtn); + const monitoringControls = document.createElement("div"); + monitoringControls.className = "dtx-monitoring-controls"; + monitoringHead.appendChild(monitoringControls); + + const monitoringSearchInput = document.createElement("input"); + monitoringSearchInput.type = "search"; + monitoringSearchInput.className = "dtx-monitoring-search"; + monitoringSearchInput.placeholder = "Search monitoring results"; + monitoringSearchInput.setAttribute("aria-label", "Search workspace monitoring results"); + monitoringControls.appendChild(monitoringSearchInput); + + const monitoringActions = document.createElement("div"); + monitoringActions.className = "dtx-monitoring-actions"; + monitoringHead.appendChild(monitoringActions); + + const rangeLabel = document.createElement("label"); + rangeLabel.className = "dtx-monitoring-field"; + rangeLabel.textContent = "Range"; + const rangeSelect = document.createElement("select"); + [ + ["15m", "Last 15 min"], ["1h", "Last hour"], ["4h", "Last 4 hours"], + ["12h", "Last 12 hours"], ["1d", "Last 24 hours"], + ["3d", "Last 3 days"], ["7d", "Last 7 days"], ["30d", "Last 30 days"], + ].forEach(([value, label]) => { + const option = document.createElement("option"); + option.value = value; + option.textContent = label; + if (value === "1d") option.selected = true; + rangeSelect.appendChild(option); + }); + rangeLabel.appendChild(rangeSelect); + monitoringActions.appendChild(rangeLabel); + + const topLabel = document.createElement("label"); + topLabel.className = "dtx-monitoring-field"; + topLabel.textContent = "Top"; + const topInput = document.createElement("input"); + topInput.type = "number"; + topInput.min = "1"; + topInput.max = "200"; + topInput.value = "20"; + topLabel.appendChild(topInput); + monitoringActions.appendChild(topLabel); + + const monitoringReloadBtn = document.createElement("button"); + monitoringReloadBtn.type = "button"; + monitoringReloadBtn.className = "dtx-monitoring-action"; + monitoringReloadBtn.innerHTML = REFRESH_SVG; + monitoringReloadBtn.title = "Reload workspace monitoring"; + monitoringReloadBtn.setAttribute("aria-label", "Reload workspace monitoring"); + monitoringActions.appendChild(monitoringReloadBtn); + const monitoringFullscreenBtn = document.createElement("button"); + monitoringFullscreenBtn.type = "button"; + monitoringFullscreenBtn.className = "dtx-monitoring-action"; + monitoringActions.appendChild(monitoringFullscreenBtn); + + const monitoringContent = document.createElement("div"); + monitoringContent.className = "dtx-monitoring-content"; + monitoringContent.id = `dtx-monitoring-content-${Math.random().toString(36).slice(2)}`; + monitoringTitleBtn.setAttribute("aria-controls", monitoringContent.id); + monitoringPane.appendChild(monitoringContent); + container.appendChild(monitoringPane); + + function renderMonitoringChrome() { + const chosen = model.get("dataset_chosen") === true; + monitoringShowBtn.style.display = chosen ? "" : "none"; + monitoringPane.classList.toggle("dtx-monitoring-hidden", !chosen || !monitoringVisible); + monitoringPane.classList.toggle("dtx-monitoring-fullscreen", monitoringFullscreen); + monitoringPane.classList.toggle("dtx-monitoring-collapsed", !monitoringOpen); + monitoringShowBtn.classList.toggle("dtx-active", monitoringVisible); + const showLabel = monitoringVisible + ? "Hide workspace monitoring" : "Show workspace monitoring"; + monitoringShowBtn.title = showLabel; + monitoringShowBtn.setAttribute("aria-label", showLabel); + monitoringShowBtn.setAttribute("aria-pressed", String(monitoringVisible)); + monitoringTitleBtn.setAttribute("aria-expanded", String(monitoringOpen)); + monitoringControls.style.display = monitoringOpen ? "" : "none"; + monitoringActions.style.display = monitoringOpen ? "" : "none"; + monitoringContent.style.display = monitoringOpen ? "" : "none"; + monitoringContent.style.height = monitoringFullscreen + ? "" : `${monitoringContentHeight}px`; + monitoringFullscreenBtn.innerHTML = monitoringFullscreen + ? FULLSCREEN_EXIT_SVG : FULLSCREEN_SVG; + const fullscreenLabel = monitoringFullscreen ? "Exit full screen" : "Full screen"; + monitoringFullscreenBtn.title = fullscreenLabel; + monitoringFullscreenBtn.setAttribute("aria-label", fullscreenLabel); + } + + function renderMonitoringContent() { + const loading = model.get("workspace_monitoring_loading") === true; + const loaded = model.get("workspace_monitoring_loaded") === true; + const enabled = model.get("workspace_monitoring_enabled") !== false; + const error = String(model.get("workspace_monitoring_error") || ""); + const columns = model.get("workspace_monitoring_columns") || []; + const rows = model.get("workspace_monitoring_rows") || []; + const monitoringTokens = model.get("workspace_monitoring_tokens") || []; + const hasQueryOutput = loaded && enabled && !loading && !error && rows.length > 0; + monitoringSearchInput.hidden = !hasQueryOutput; + monitoringReloadBtn.disabled = loading || model.get("dataset_chosen") !== true; + if (error) { + monitoringContent.innerHTML = `
` + + `${ACTIVITY_SVG}${escapeHtml(error)}
`; + return; + } + if (loading) { + monitoringContent.innerHTML = `
` + + `${ACTIVITY_SVG}Reading workspace monitoring…
`; + return; + } + if (!loaded) { + monitoringContent.innerHTML = `
` + + `${ACTIVITY_SVG}Press Reload to read the slowest recent queries.` + + `Queries the workspace's monitoring database for QueryEnd events for this model.
`; + return; + } + if (!enabled) { + monitoringContent.innerHTML = `
` + + `${ACTIVITY_SVG}Workspace monitoring is not enabled` + + `Enable workspace monitoring in the workspace settings to see query history.
`; + return; + } + if (!rows.length) { + monitoringContent.innerHTML = `
` + + `${ACTIVITY_SVG}No queries found for this model and range.
`; + return; + } + const isNumeric = column => /ms$/i.test(String(column)); + const isTime = column => String(column).toLowerCase() === "timestamp"; + const isQuery = column => String(column).toLowerCase() === "eventtext"; + const headerLabel = column => ({ + durationms: "Duration (MS)", cputimems: "CPU", eventtext: "Query", + visualid: "Visual ID", reportid: "Report ID", + executinguser: "Executing User", + reportname: "Report Name", reportworkspace: "Report Workspace", + }[String(column).toLowerCase()] || String(column)); + const columnWidth = column => ({ + reportid: 320, visualid: 320, executinguser: 260, + reportname: 240, reportworkspace: 240, + }[String(column).toLowerCase()] + || (isQuery(column) ? 480 : isTime(column) ? 180 : isNumeric(column) ? 120 : 200)); + const displayValue = (column, value) => { + if (value == null || value === "") return ""; + if (isTime(column)) { + const date = new Date(value); + return Number.isNaN(date.getTime()) ? String(value) : date.toLocaleString(); + } + if (isNumeric(column)) { + const number = Number(value); + return Number.isFinite(number) ? Math.round(number).toLocaleString() : String(value); + } + return String(value); + }; + const queryIndex = columns.findIndex(isQuery); + let viewRows = rows.map((row, index) => ({ row, index })); + const search = monitoringSearch.trim().toLowerCase(); + if (search) { + viewRows = viewRows.filter(({ row }) => row.some((value, index) => { + const searchable = index === queryIndex + ? cleanDaxQuery(value) : displayValue(columns[index], value); + return String(searchable).toLowerCase().includes(search); + })); + } + if (monitoringSort) { + const { index, direction } = monitoringSort; + const column = columns[index]; + viewRows.sort((left, right) => { + const a = left.row[index] ?? ""; + const b = right.row[index] ?? ""; + let result; + if (isTime(column)) result = new Date(a).getTime() - new Date(b).getTime(); + else if (isNumeric(column)) result = Number(a) - Number(b); + else result = String(a).localeCompare(String(b), undefined, { numeric: true }); + if (Number.isNaN(result)) result = String(a).localeCompare(String(b)); + return direction === "ascending" ? result : -result; + }); + } + const head = columns.map((column, index) => { + const width = columnWidth(column); + const sort = monitoringSort?.index === index ? monitoringSort.direction : "none"; + return ``; + }).join(""); + const bodyHtml = viewRows.map(({ row, index: rowIndex }) => `${columns.map((column, index) => { + const rawValue = row[index] == null ? "" : String(row[index]); + if (index === queryIndex) { + const query = cleanDaxQuery(rawValue); + const isDax = /^\s*(?:EVALUATE|DEFINE)\b/i.test(query); + const queryHtml = isDax + ? renderDaxTokens(monitoringTokens[rowIndex] || [], query) + : escapeHtml(query); + return ``; + } + return ``; + }).join("")}`).join(""); + monitoringContent.innerHTML = `
${escapeHtml(headerLabel(column))}
${queryHtml}
${escapeHtml(displayValue(column, rawValue))}
${head}${bodyHtml}
`; + const useQuery = cell => { + const row = rows[Number(cell.dataset.monitoringIndex)] || []; + const query = queryIndex >= 0 ? cleanDaxQuery(row[queryIndex]) : ""; + if (!query) return; + model.set("dax_query", query); + model.save_changes(); + showToast("Monitoring query loaded into editor"); + }; + monitoringContent.querySelectorAll(".dtx-monitoring-query").forEach(cell => { + cell.addEventListener("click", () => useQuery(cell)); + cell.addEventListener("keydown", event => { + if (event.key === "Enter" || event.key === " ") { + event.preventDefault(); + useQuery(cell); + } + }); + }); + monitoringContent.querySelectorAll("th[data-monitoring-sort]").forEach(header => { + header.addEventListener("click", event => { + if (event.target.closest(".dtx-column-resizer")) return; + const index = Number(header.dataset.monitoringSort); + monitoringSort = monitoringSort?.index === index + ? monitoringSort.direction === "ascending" + ? { index, direction: "descending" } : null + : { index, direction: "ascending" }; + renderMonitoringContent(); + }); + }); + installColumnResizers(monitoringContent.querySelector("table")); + } + + monitoringSearchInput.addEventListener("input", () => { + monitoringSearch = monitoringSearchInput.value; + renderMonitoringContent(); + }); + monitoringResizer.addEventListener("pointerdown", event => { + if (monitoringFullscreen || !monitoringOpen) return; + event.preventDefault(); + const startY = event.clientY; + const startHeight = monitoringContent.getBoundingClientRect().height; + monitoringResizer.classList.add("dtx-monitoring-resizing"); + monitoringResizer.setPointerCapture(event.pointerId); + const onMove = moveEvent => { + const maxHeight = Math.max(260, window.innerHeight - 150); + monitoringContentHeight = Math.min( + maxHeight, + Math.max(190, startHeight + startY - moveEvent.clientY), + ); + monitoringContent.style.height = `${monitoringContentHeight}px`; + }; + const onEnd = () => { + monitoringResizer.classList.remove("dtx-monitoring-resizing"); + monitoringResizer.removeEventListener("pointermove", onMove); + monitoringResizer.removeEventListener("pointerup", onEnd); + monitoringResizer.removeEventListener("pointercancel", onEnd); + }; + monitoringResizer.addEventListener("pointermove", onMove); + monitoringResizer.addEventListener("pointerup", onEnd); + monitoringResizer.addEventListener("pointercancel", onEnd); + }); + + monitoringTitleBtn.addEventListener("click", () => { + monitoringOpen = !monitoringOpen; + if (!monitoringOpen) monitoringFullscreen = false; + renderMonitoringChrome(); + }); + monitoringReloadBtn.addEventListener("click", () => { + const top = Math.min(200, Math.max(1, Math.round(Number(topInput.value) || 20))); + topInput.value = String(top); + model.set("workspace_monitoring_request", { range: rangeSelect.value, top }); + model.set("workspace_monitoring_trigger", + (model.get("workspace_monitoring_trigger") || 0) + 1); + model.save_changes(); + }); + monitoringFullscreenBtn.addEventListener("click", () => { + monitoringFullscreen = !monitoringFullscreen; + renderMonitoringChrome(); + }); + + const chartControls = document.createElement("div"); + chartControls.className = "dtx-chart-controls"; + chartControls.style.display = "none"; + main.appendChild(chartControls); + + const chartWrap = document.createElement("div"); + chartWrap.className = "dtx-chart-wrap"; + chartWrap.style.display = "none"; + main.appendChild(chartWrap); + + // Persisted per-render chart axis selections (not synced to Python). + const chartState = { xIdx: null, yIdx: null }; + + function renderTraceTable() { + const rows = model.get("trace_rows") || []; + const fmt = (n) => Number(n).toLocaleString(); + let body; + if (!rows.length) { + body = `No trace events captured.`; + } else { + body = rows.map(r => ( + ` + ${escapeHtml(r.event_class)} + ${escapeHtml(r.event_subclass)} + ${escapeHtml(fmt(r.duration))} ms + ${escapeHtml(fmt(r.cpu))} ms + ${r.rows == null ? "" : escapeHtml(fmt(r.rows))} + ${r.kb == null ? "" : escapeHtml(Number(r.kb).toLocaleString(undefined, { maximumFractionDigits: 2 }))} + 
${renderTraceText(r.text, r.event_class)}
+ ` + )).join(""); + } + tableWrap.innerHTML = ` + + + + + + + + + + + ${body} +
EventSubclassDurationCPURowsKBText
`; + } + + function renderResultTable() { + const cols = model.get("result_columns") || []; + const rows = model.get("result_rows") || []; + const total = model.get("result_total_rows") || 0; + const truncated = model.get("result_truncated") === true; + if (!cols.length) { + tableWrap.innerHTML = `
No query result available.
`; + return; + } + const isNum = cols.map((_, i) => rows.every(r => r[i] === null || typeof r[i] === "number")); + const fmtCell = (v, i) => { + if (v === null || v === undefined) return ""; + if (typeof v === "number") return escapeHtml(Number(v).toLocaleString(undefined, { maximumFractionDigits: 6 })); + return escapeHtml(String(v)); + }; + const head = cols.map((c, i) => `${escapeHtml(c)}`).join(""); + const body = rows.length + ? rows.map(r => `${r.map((v, i) => `${fmtCell(v, i)}`).join("")}`).join("") + : `Query returned no rows.`; + tableWrap.innerHTML = `${head}${body}
`; + if (truncated) { + resultMeta.textContent = `Showing first ${rows.length.toLocaleString()} of ${total.toLocaleString()} rows.`; + } else { + resultMeta.textContent = `${total.toLocaleString()} row${total === 1 ? "" : "s"}.`; + } + } + + const historySortState = { key: "", direction: "ascending" }; + function historySortValue(entry, key) { + if (["duration", "fe_duration", "se_duration", "cpu"].includes(key)) { + return Number(entry[key] || 0); + } + if (key === "execution_metrics") return JSON.stringify(entry.execution_metrics || {}); + if (key === "query") return String(entry.dax_query || ""); + if (key === "report") return String(entry.report_name || ""); + if (key === "workspace") { + return String(entry.report_workspace_name || entry.workspace_name || ""); + } + return String(entry[key] || ""); + } + function renderHistoryTable() { + const hist = model.get("trace_history") || []; + const fmt = (n) => Number(n).toLocaleString(); + const renderMetrics = (metrics) => { + if (!metrics || typeof metrics !== "object") return ""; + const lines = Object.entries(metrics).map(([key, value]) => { + const jsonValue = JSON.stringify(value); + const renderedValue = typeof value === "number" && Number.isFinite(value) + ? `${escapeHtml(jsonValue)}` + : escapeHtml(jsonValue); + return ` ${escapeHtml(JSON.stringify(key))}: ${renderedValue}`; + }); + return lines.length ? `{\n${lines.join(",\n")}\n}` : ""; + }; + const fmtRunTime = (value) => { + const run = String(value || ""); + const time = run.includes(" ") ? run.split(" ").pop() : run; + const parts = time.split(":").map(Number); + if (parts.length !== 3 || parts.some(part => !Number.isFinite(part))) return time; + const date = new Date(2000, 0, 1, parts[0], parts[1], parts[2]); + return date.toLocaleTimeString("en-US", { + hour: "numeric", minute: "2-digit", second: "2-digit", hour12: true, + }); + }; + if (!hist.length) { + tableWrap.innerHTML = `
No queries have been executed in this session yet.
`; + return; + } + const indexedHistory = hist.map((h, index) => ({ h, index })); + if (historySortState.key) { + indexedHistory.sort((left, right) => { + const a = historySortValue(left.h, historySortState.key); + const b = historySortValue(right.h, historySortState.key); + const result = typeof a === "number" && typeof b === "number" + ? a - b + : String(a).localeCompare(String(b), undefined, { + numeric: true, sensitivity: "base", + }); + return historySortState.direction === "ascending" ? result : -result; + }); + } + const body = indexedHistory.map(({ h, index }) => { + const q = cleanDaxQuery(h.dax_query); + const run = String(h.start_time || ""); + const runTime = fmtRunTime(run); + const metrics = renderMetrics(h.execution_metrics); + const method = String(h.method || "Query"); + const reportName = method === "Report" ? String(h.report_name || "") : ""; + const reportWorkspace = method === "Report" + ? String(h.report_workspace_name || h.workspace_name || "") : ""; + return ` + ${escapeHtml(runTime)} + ${escapeHtml(fmt(h.duration))} ms + ${escapeHtml(fmt(h.fe_duration))} ms + ${escapeHtml(fmt(h.se_duration))} ms + ${escapeHtml(fmt(h.cpu))} ms + ${escapeHtml(String(h.cache || ""))} + ${metrics ? `
${metrics}
` : ""} + ${escapeHtml(method)} + 
${escapeHtml(q)}
+ ${escapeHtml(reportName)} + ${escapeHtml(reportWorkspace)} + `; + }).join(""); + tableWrap.innerHTML = ` + + + + + + + + + + + + + + + ${body} +
RunTotalFESECPUCacheExecution metricsMethodQueryReportWorkspace
`; + + tableWrap.querySelectorAll("th[data-history-sort]").forEach(header => { + const key = header.dataset.historySort; + if (key === historySortState.key) { + header.setAttribute("aria-sort", historySortState.direction); + } else { + header.setAttribute("aria-sort", "none"); + } + header.addEventListener("click", event => { + if (event.target.closest(".dtx-column-resizer")) return; + historySortState.direction = historySortState.key === key + && historySortState.direction === "ascending" + ? "descending" : "ascending"; + historySortState.key = key; + renderHistoryTable(); + }); + }); + + const copyHistoryQuery = (cell) => { + const index = Number(cell.dataset.historyIndex); + const entry = (model.get("trace_history") || [])[index]; + const query = entry ? cleanDaxQuery(entry.dax_query) : ""; + if (!query) return; + writeClipboard(query) + .then(() => showToast("Query copied to clipboard")) + .catch(() => showToast("Unable to copy query")); + }; + tableWrap.querySelectorAll(".dtx-hist-query").forEach(cell => { + cell.addEventListener("click", () => copyHistoryQuery(cell)); + cell.addEventListener("keydown", event => { + if (event.key === "Enter" || event.key === " ") { + event.preventDefault(); + copyHistoryQuery(cell); + } + }); + }); + } + + function renderQueryPlanTable() { + const rows = model.get("query_plan_rows") || []; + const planType = model.get("query_plan_type") || "Logical"; + const matching = rows.filter(r => String(r.plan_type || "") === planType); + if (!rows.length) { + tableWrap.innerHTML = `
No DAX query plan captured. Run a query to capture its query plan.
`; + return; + } + if (!matching.length) { + tableWrap.innerHTML = `
No ${escapeHtml(planType)} Query Plan was captured for the last query.
`; + return; + } + // Render the whole plan as a single pane. A 
 preserves the plan's
+        // own indentation, so there is no per-line row striping.
+        const planText = matching.map(r => String(r.text || "")).join("\n");
+        tableWrap.innerHTML = `
+            
+                
+                
+            
${escapeHtml(planType)} Query Plan
${escapeHtml(planText)}
`;
+    }
+
+    function renderExecMetricsTable() {
+        const rows = model.get("execution_metrics") || [];
+        const fmt = (n) => Number(n).toLocaleString();
+        if (!rows.length) {
+            tableWrap.innerHTML = `
No execution metrics captured. Run a query to capture its execution metrics.
`;
+            return;
+        }
+        const body = rows.map(r => (
+            `
+                ${escapeHtml(String(r.label || r.key || ""))}
+                ${escapeHtml(fmt(r.value))}
+            `
+        )).join("");
+        tableWrap.innerHTML = `
+            
+                
+                    
+                    
+                
+                ${body}
+            
MetricValue
`;
+    }
+
+    // Collapsed-state set for the query-dependencies tree (keyed by node path).
+    const depCollapsed = new Set();
+
+    function depIcon(kind) {
+        switch (kind) {
+            case "model": return TABLE_SVG;
+            case "group": return FOLDER_SVG;
+            case "table": return TABLE_SVG;
+            case "column": return COLUMN_SVG;
+            case "measure": return MEASURE_SVG;
+            case "hierarchy": return HIERARCHY_SVG;
+            case "calc_group": return CALC_GROUP_SVG;
+            case "relationship": return SWAP_SVG;
+            default: return COLUMN_SVG;
+        }
+    }
+
+    function renderDepNode(node, path, depth) {
+        const hasChildren = !!(node.children && node.children.length);
+        const collapsed = depCollapsed.has(path);
+        const caret = hasChildren
+            ? `${CARET_SVG}`
+            : ``;
+        const detail = node.detail
+            ? `${escapeHtml(String(node.detail))}`
+            : "";
+        const pad = 8 + depth * 16;
+        let html = `
`
+            + caret
+            + `${depIcon(node.kind)}`
+            + `${escapeHtml(String(node.label || ""))}`
+            + detail
+            + `
`;
+        if (hasChildren && !collapsed) {
+            html += node.children
+                .map((c, i) => renderDepNode(c, path + "/" + i, depth + 1))
+                .join("");
+        }
+        return html;
+    }
+
+    function renderDependencyColumns() {
+        const cols = model.get("dependency_columns") || [];
+        if (!cols.length) {
+            tableWrap.innerHTML = `
No columns referenced. Run this on a non-empty DAX query.
`;
+            return;
+        }
+        const body = cols.map(c => {
+            return ``
+                + `${escapeHtml(String(c.table || ""))}`
+                + `${escapeHtml(String(c.column || ""))}`
+                + ``;
+        }).join("");
+        tableWrap.innerHTML = `
+            
+                
+                    
+                    
+                
+                ${body}
+            
Table NameColumn Name
`;
+    }
+
+    function renderDependenciesTable() {
+        if (model.get("dependencies_loading") === true) {
+            tableWrap.innerHTML = `
Computing query dependencies…
`;
+            return;
+        }
+        const view = model.get("dependency_view") || "tree";
+        if (view === "columns") {
+            renderDependencyColumns();
+            return;
+        }
+        const tree = model.get("dependency_tree") || [];
+        if (!tree.length) {
+            tableWrap.innerHTML = `
No dependencies found. Run this on a non-empty DAX query.
`;
+            return;
+        }
+        const html = tree.map((n, i) => renderDepNode(n, String(i), 0)).join("");
+        tableWrap.innerHTML = `
${html}
`;
+        tableWrap.querySelectorAll(".dtx-dep-row[data-haschildren='1']").forEach(row => {
+            row.addEventListener("click", () => {
+                const p = row.getAttribute("data-path");
+                if (depCollapsed.has(p)) depCollapsed.delete(p);
+                else depCollapsed.add(p);
+                renderDependenciesTable();
+            });
+        });
+    }
+
+    const vertipaqSortBySection = new Map();
+    function updateVertipaqFrozen(table) {
+        if (!table?.classList.contains("dtx-vertipaq-table")) return;
+        const headers = Array.from(table.querySelectorAll("thead th"));
+        const rows = table.querySelectorAll("tbody tr");
+        let left = 0;
+        headers.forEach((header, index) => {
+            if (!header.classList.contains("dtx-vp-frozen")) return;
+            const offset = `${left}px`;
+            header.style.left = offset;
+            rows.forEach(row => {
+                if (row.cells[index]) row.cells[index].style.left = offset;
+            });
+            left += header.getBoundingClientRect().width;
+        });
+    }
+
+    function renderVertipaqTable() {
+        if (model.get("vertipaq_loading") === true) {
+            tableWrap.innerHTML = `
Running Vertipaq Analyzer…
`;
+            return;
+        }
+        const sections = model.get("vertipaq_sections") || [];
+        if (!sections.length) {
+            tableWrap.innerHTML = `
No Vertipaq Analyzer results available.
`;
+            return;
+        }
+        let section = sections.find(s => s.name === (model.get("vertipaq_section") || ""));
+        if (!section) section = sections[0];
+        const cols = section.columns || [];
+        const rows = section.rows || [];
+        const frozenNames = {
+            Tables: ["Table Name"],
+            Partitions: ["Table Name", "Partition Name"],
+            Columns: ["Table Name", "Column Name"],
+        }[section.name] || [];
+        const frozenIndexes = cols
+            .map((column, index) => frozenNames.includes(column) ? index : -1)
+            .filter(index => index >= 0);
+        const frozenEdge = frozenIndexes.length
+            ? frozenIndexes[frozenIndexes.length - 1] : -1;
+        const frozenClasses = index => frozenIndexes.includes(index)
+            ? ` dtx-vp-frozen${index === frozenEdge ? " dtx-vp-frozen-edge" : ""}`
+            : "";
+        const parseNumeric = value => {
+            if (typeof value === "number" && !Number.isFinite(value)) return null;
+            if (typeof value !== "number" && typeof value !== "string") return null;
+            const text = String(value).trim();
+            const match = /^([+-]?)(\d*)(?:\.(\d*))?(?:[eE]([+-]?\d+))?$/.exec(text);
+            if (!match || !(match[2] || match[3])) return null;
+            const exponent = Number(match[4] || 0);
+            if (!Number.isSafeInteger(exponent) || Math.abs(exponent) > 10000) return null;
+            let digits = `${match[2]}${match[3] || ""}`.replace(/^0+/, "");
+            if (!digits) return { sign: 0, digits: "0", scale: 0 };
+            let scale = exponent - (match[3] || "").length;
+            while (digits.endsWith("0")) {
+                digits = digits.slice(0, -1);
+                scale += 1;
+            }
+            return { sign: match[1] === "-" ? -1 : 1, digits, scale };
+        };
+        const compareNumeric = (left, right) => {
+            if (left.sign !== right.sign) return left.sign - right.sign;
+            if (left.sign === 0) return 0;
+            const leftMagnitude = left.digits.length + left.scale;
+            const rightMagnitude = right.digits.length + right.scale;
+            let result = leftMagnitude - rightMagnitude;
+            if (result === 0) {
+                const width = Math.max(left.digits.length, right.digits.length);
+                result = left.digits.padEnd(width, "0").localeCompare(
+                    right.digits.padEnd(width, "0")
+                );
+            }
+            return left.sign * result;
+        };
+        const formatNumeric = parsed => {
+            if (parsed.sign === 0) return "0";
+            const point = parsed.digits.length + parsed.scale;
+            const integer = point <= 0
+                ? "0"
+                : point >= parsed.digits.length
+                    ? parsed.digits + "0".repeat(point - parsed.digits.length)
+                    : parsed.digits.slice(0, point);
+            const fraction = point <= 0
+                ? "0".repeat(-point) + parsed.digits
+                : point < parsed.digits.length ? parsed.digits.slice(point) : "";
+            const grouped = integer.replace(/\B(?=(\d{3})+(?!\d))/g, ",");
+            return `${parsed.sign < 0 ? "-" : ""}${grouped}${fraction ? `.${fraction}` : ""}`;
+        };
+        const isBlank = value => value === null || value === undefined
+            || (typeof value === "string" && value.trim() === "");
+        const numericColumns = cols.map((_, index) => {
+            const values = rows.map(row => row[index]).filter(value => !isBlank(value));
+            return values.length > 0 && values.every(value => parseNumeric(value) !== null);
+        });
+        const sortState = vertipaqSortBySection.get(section.name) || null;
+        const viewRows = rows.map((row, index) => ({ row, index }));
+        if (sortState && sortState.index < cols.length) {
+            viewRows.sort((left, right) => {
+                const a = left.row[sortState.index];
+                const b = right.row[sortState.index];
+                const aBlank = isBlank(a);
+                const bBlank = isBlank(b);
+                if (aBlank !== bBlank) return aBlank ? 1 : -1;
+                let result = numericColumns[sortState.index]
+                    ? compareNumeric(parseNumeric(a), parseNumeric(b))
+                    : String(a ?? "").localeCompare(String(b ?? ""), undefined, {
+                        numeric: true, sensitivity: "base",
+                    });
+                if (result === 0) return left.index - right.index;
+                return sortState.direction === "ascending" ? result : -result;
+            });
+        }
+        const head = cols.map((column, index) => {
+            const direction = sortState?.index === index ? sortState.direction : "none";
+            return `${escapeHtml(String(column))}`;
+        }).join("");
+        const displayValue = (value, index) => {
+            if (isBlank(value)) return "";
+            if (!numericColumns[index]) return String(value);
+            return formatNumeric(parseNumeric(value));
+        };
+        let body;
+        if (!rows.length) {
+            body = `No rows.`;
+        } else {
+            body = viewRows.map(({ row }) => ``
+                + row.map((value, index) => `${escapeHtml(displayValue(value, index))}`).join("")
+                + ``).join("");
+        }
+        tableWrap.innerHTML = `
+            
+                ${head}
+                ${body}
+            
`;
+        requestAnimationFrame(() => updateVertipaqFrozen(
+            tableWrap.querySelector(".dtx-vertipaq-table")
+        ));
+        const sortColumn = header => {
+            const index = Number(header.dataset.vertipaqSort);
+            const direction = sortState?.index === index && sortState.direction === "ascending"
+                ? "descending" : "ascending";
+            vertipaqSortBySection.set(section.name, { index, direction });
+            renderVertipaqTable();
+        };
+        tableWrap.querySelectorAll("th[data-vertipaq-sort]").forEach(header => {
+            header.addEventListener("click", () => sortColumn(header));
+            header.addEventListener("keydown", event => {
+                if (event.key === "Enter" || event.key === " ") {
+                    event.preventDefault();
+                    sortColumn(header);
+                }
+            });
+        });
+    }
+
+    function renderChart() {
+        const cols = model.get("result_columns") || [];
+        const rows = model.get("result_rows") || [];
+        const elig = chartEligibility();
+        chartControls.innerHTML = "";
+        chartWrap.innerHTML = "";
+        if (!elig.ok) {
+            const msg = document.createElement("div");
+            msg.className = "dtx-chart-empty";
+            msg.textContent = elig.reason;
+            chartWrap.appendChild(msg);
+            chartControls.style.display = "none";
+            return;
+        }
+        const numericCols = elig.numericCols;
+        const numericIdxs = numericCols.map((b, i) => b ? i : -1).filter(i => i >= 0);
+
+        // Modern, clean qualitative color palette for multi-series / stacked bars.
+        const PALETTE = [
+            "#4f8cff", "#34d399", "#fbbf24", "#f472b6", "#a78bfa",
+            "#22d3ee", "#fb7185", "#84cc16", "#f59e0b", "#38bdf8",
+        ];
+
+        // A "category" column is one whose values are strings (not numbers/bools).
+        const isCategoryCol = (i) =>
+            rows.some(r => typeof r[i] === "string") &&
+            rows.every(r => r[i] === null || typeof r[i] === "string");
+
+        // Single result row: numeric columns become the values to chart. If a
+        // string column exists, its value is used as the X-axis category label;
+        // multiple numeric columns are drawn as a stacked bar with a legend.
+        const singleRow = rows.length === 1 && numericIdxs.length >= 1;
+        const stackedMode = singleRow && numericIdxs.length >= 2;
+
+        let data;
+        let legendSegments = null;
+        if (singleRow) {
+            chartControls.style.display = "none";
+            const catIdx = cols.findIndex((_, i) => isCategoryCol(i));
+            const catLabel = catIdx >= 0 ? String(rows[0][catIdx] ?? "") : "";
+            if (stackedMode) {
+                const segments = numericIdxs.map((i, k) => ({
+                    name: cols[i],
+                    value: typeof rows[0][i] === "number" ? rows[0][i] : 0,
+                    color: PALETTE[k % PALETTE.length],
+                }));
+                legendSegments = segments;
+                data = [{ label: catLabel || "Total", segments }];
+            } else {
+                const i = numericIdxs[0];
+                data = [{
+                    label: catLabel || cols[i],
+                    value: typeof rows[0][i] === "number" ? rows[0][i] : 0,
+                }];
+            }
+        } else {
+            // Default y = first numeric col; x = first non-numeric col or row index.
+            if (chartState.yIdx == null || !numericIdxs.includes(chartState.yIdx)) {
+                chartState.yIdx = numericIdxs[0];
+            }
+            const nonNumIdxs = cols.map((_, i) => numericCols[i] ? -1 : i).filter(i => i >= 0);
+            if (chartState.xIdx == null || (chartState.xIdx !== -1 &&
+                (chartState.xIdx >= cols.length || chartState.xIdx === chartState.yIdx))) {
+                chartState.xIdx = nonNumIdxs.length ? nonNumIdxs[0] : -1;
+            }
+
+            // Axis selectors.
+            const xLabel = document.createElement("label");
+            xLabel.innerHTML = "X";
+            const xSel = document.createElement("select");
+            const idxOpt = document.createElement("option");
+            idxOpt.value = "-1";
+            idxOpt.textContent = "(row index)";
+            xSel.appendChild(idxOpt);
+            cols.forEach((c, i) => {
+                if (i === chartState.yIdx) return;
+                const o = document.createElement("option");
+                o.value = String(i);
+                o.textContent = c;
+                xSel.appendChild(o);
+            });
+            xSel.value = String(chartState.xIdx);
+            xSel.addEventListener("change", () => {
+                chartState.xIdx = parseInt(xSel.value, 10);
+                renderChart();
+            });
+            xLabel.appendChild(xSel);
+            chartControls.appendChild(xLabel);
+
+            const yLabel = document.createElement("label");
+            yLabel.innerHTML = "Y";
+            const ySel = document.createElement("select");
+            numericIdxs.forEach(i => {
+                const o = document.createElement("option");
+                o.value = String(i);
+                o.textContent = cols[i];
+                ySel.appendChild(o);
+            });
+            ySel.value = String(chartState.yIdx);
+            ySel.addEventListener("change", () => {
+                chartState.yIdx = parseInt(ySel.value, 10);
+                renderChart();
+            });
+            yLabel.appendChild(ySel);
+            chartControls.appendChild(yLabel);
+            chartControls.style.display = "";
+
+            // Build data.
+            const yIdx = chartState.yIdx;
+            const xIdx = chartState.xIdx;
+            data = rows.map((r, i) => ({
+                label: xIdx === -1 ? String(i + 1) : (r[xIdx] == null ? "" : String(r[xIdx])),
+                value: typeof r[yIdx] === "number" ? r[yIdx] : 0,
+            }));
+        }
+
+        // SVG bar chart.
+        const n = data.length;
+        const barWidth = data.length === 1 ? 64 : 28;
+        const barGap = 8;
+        const leftPad = 56;
+        const rightPad = 16;
+        const topPad = 12;
+        const bottomPad = 56;
+        const plotWidth = Math.max(n * (barWidth + barGap), 200);
+        const width = leftPad + plotWidth + rightPad;
+        const height = 280;
+        const plotHeight = height - topPad - bottomPad;
+        const totalOf = d => d.segments ? d.segments.reduce((s, sg) => s + sg.value, 0) : d.value;
+        const values = data.map(totalOf);
+        const dataMin = Math.min(0, ...values);
+        const dataMax = Math.max(0, ...values);
+
+        // Compute "nice" integer-only axis bounds and step.
+        function niceStep(raw) {
+            if (raw <= 0) return 1;
+            const mag = Math.pow(10, Math.floor(Math.log10(raw)));
+            const norm = raw / mag;
+            let nice;
+            if (norm <= 1) nice = 1;
+            else if (norm <= 2) nice = 2;
+            else if (norm <= 5) nice = 5;
+            else nice = 10;
+            return Math.max(1, Math.round(nice * mag));
+        }
+        const ticks = 5;
+        const rawSpan = (dataMax - dataMin) || 1;
+        const step = niceStep(rawSpan / ticks);
+        const axisMin = Math.floor(dataMin / step) * step;
+        const axisMax = Math.ceil(dataMax / step) * step;
+        const span = (axisMax - axisMin) || 1;
+        const yScale = v => topPad + plotHeight - ((v - axisMin) / span) * plotHeight;
+        const fmtNum = v => Number(v).toLocaleString();
+        // Compact axis labels: abbreviate large magnitudes (e.g. 1B, 10M, 100K).
+        const fmtAxis = v => {
+            const n = Number(v);
+            const abs = Math.abs(n);
+            const sign = n < 0 ? "-" : "";
+            const compact = (val, suffix) => {
+                let s = val.toFixed(1);
+                if (s.endsWith(".0")) s = s.slice(0, -2);
+                return sign + s + suffix;
+            };
+            if (abs >= 1e12) return compact(abs / 1e12, "T");
+            if (abs >= 1e9) return compact(abs / 1e9, "B");
+            if (abs >= 1e6) return compact(abs / 1e6, "M");
+            if (abs >= 1e3) return compact(abs / 1e3, "K");
+            return n.toLocaleString();
+        };
+
+        let gridLines = "";
+        let yTicks = "";
+        for (let v = axisMin; v <= axisMax + 0.5; v += step) {
+            const iv = Math.round(v);
+            const y = yScale(iv);
+            gridLines += ``;
+            yTicks += `${escapeHtml(fmtAxis(iv))}`;
+        }
+        const baselineY = yScale(Math.max(axisMin, Math.min(0, axisMax)));
+
+        let bars = "";
+        let xLabels = "";
+        data.forEach((d, i) => {
+            const x = leftPad + i * (barWidth + barGap) + barGap / 2;
+            if (d.segments) {
+                let cum = 0;
+                d.segments.forEach((sg) => {
+                    const y0 = yScale(cum);
+                    const y1 = yScale(cum + sg.value);
+                    const top = Math.min(y0, y1);
+                    const h = Math.max(1, Math.abs(y1 - y0));
+                    const tip = `${sg.name}: ${fmtNum(sg.value)}`;
+                    bars += `${escapeHtml(tip)}`;
+                    cum += sg.value;
+                });
+            } else {
+                const y = d.value >= 0 ? yScale(d.value) : baselineY;
+                const h = Math.max(1, Math.abs(yScale(d.value) - baselineY));
+                const tip = `${d.label}: ${fmtNum(d.value)}`;
+                bars += `${escapeHtml(tip)}`;
+            }
+            const cx = x + barWidth / 2;
+            const labelTxt = d.label.length > 16 ? d.label.slice(0, 15) + "\u2026" : d.label;
+            const ly = height - bottomPad + 14;
+            xLabels += ``
+                + `${escapeHtml(d.label)}${escapeHtml(labelTxt)}`;
+        });
+
+        const svg = ``
+            + `${gridLines}`
+            + ``
+            + ``
+            + ``
+            + `${yTicks}${xLabels}`
+            + ``
+            + `${bars}`
+            + ``;
+        let legendHtml = "";
+        if (legendSegments) {
+            legendHtml = `
`
+                + legendSegments.map(sg =>
+                    ``
+                    + ``
+                    + `${escapeHtml(sg.name)}`
+                ).join("")
+                + `
`;
+        }
+        chartWrap.innerHTML = svg + legendHtml;
+    }
+
+    function renderPerformance() {
+        if (model.get("performance_loading") === true) {
+            tableWrap.innerHTML = `
Generating DAX performance analysis…
`;
+            return;
+        }
+        const findings = model.get("performance_findings") || [];
+        const summary = model.get("performance_summary") || {};
+        if (!summary || !Object.keys(summary).length) {
+            tableWrap.innerHTML = `
No performance analysis yet. Click Analyze to generate one.
`;
+            return;
+        }
+        const sev = summary.severity_counts || {};
+        const fmtMs = v => (v == null ? "—" : Number(v).toLocaleString() + " ms");
+        const fmtPct = v => (v == null ? "" : Number(v).toFixed(1) + "%");
+        const chip = (label, val, cls) =>
+            `${escapeHtml(label)}: ${val}`;
+
+        // Engine-balance bar (FE vs SE). fe_pct/se_pct are fractions (0-1).
+        const fePct = Number(summary.fe_pct || 0) * 100;
+        const sePct = Number(summary.se_pct || 0) * 100;
+        const balanceBar = `
+            

+                

+                

+            

+            

+                 FE ${fmtPct(fePct)}
+                 SE ${fmtPct(sePct)}
+            
`;
+
+        const sevChips = [
+            sev.high ? chip("High", sev.high, "dtx-perf-chip-high") : "",
+            sev.medium ? chip("Medium", sev.medium, "dtx-perf-chip-medium") : "",
+            sev.low ? chip("Low", sev.low, "dtx-perf-chip-low") : "",
+            sev.info ? chip("Info", sev.info, "dtx-perf-chip-info") : "",
+        ].filter(Boolean).join("");
+
+        const header = `
+            

+                

+                    ${chip("Total", fmtMs(summary.total_duration_ms))}
+                    ${chip("FE", fmtMs(summary.fe_duration_ms))}
+                    ${chip("SE", fmtMs(summary.se_duration_ms))}
+                    ${chip("Findings", summary.total_findings != null ? summary.total_findings : 0)}
+                    ${sevChips}
+                

+                ${balanceBar}
+            
`;
+
+        let cards;
+        if (!findings.length) {
+            cards = `
No optimization findings — this query looks healthy based on the available signals.
`;
+        } else {
+            cards = findings.map(f => {
+                const sevCls = "dtx-perf-card-" + escapeHtml(String(f.severity || "info"));
+                const refs = (f.references || []).map(r =>
+                    `${escapeHtml(String(r))}`
+                ).join("");
+                const refsHtml = refs
+                    ? `
${refs}
` : "";
+                const rec = f.recommendation
+                    ? `
Recommendation ${escapeHtml(String(f.recommendation))}
`
+                    : "";
+                return `
+                    

+                        

+                            ${escapeHtml(String(f.severity || "info").toUpperCase())}
+                            ${escapeHtml(String(f.title || f.id || ""))}
+                            ${escapeHtml(String(f.category || ""))}
+                        

+                        
${escapeHtml(String(f.message || ""))}

+                        ${rec}
+                        ${refsHtml}
+                    
`;
+            }).join("");
+        }
+        tableWrap.innerHTML = `
${header}
${cards}
`;
+    }
+
+    function renderTable() {
+        const mode = model.get("view_mode") || "trace";
+        // Default visibility — chart/table swap below.
+        tableWrap.style.display = "";
+        chartWrap.style.display = "none";
+        chartControls.style.display = "none";
+        if (mode === "chart") {
+            tableWrap.style.display = "none";
+            chartWrap.style.display = "";
+            resultMeta.style.display = "none";
+            renderChart();
+        } else if (mode === "result") {
+            renderResultTable();
+            resultMeta.style.display = (model.get("result_columns") || []).length ? "" : "none";
+        } else if (mode === "history") {
+            renderHistoryTable();
+            resultMeta.style.display = "none";
+        } else if (mode === "queryplan") {
+            renderQueryPlanTable();
+            resultMeta.style.display = "none";
+        } else if (mode === "dependencies") {
+            renderDependenciesTable();
+            resultMeta.style.display = "none";
+        } else if (mode === "vertipaq") {
+            renderVertipaqTable();
+            resultMeta.style.display = "none";
+        } else if (mode === "performance") {
+            renderPerformance();
+            resultMeta.style.display = "none";
+        } else if (mode === "execmetrics") {
+            renderExecMetricsTable();
+            resultMeta.style.display = "none";
+        } else {
+            renderTraceTable();
+            resultMeta.style.display = "none";
+        }
+        renderSeg();
+    }
+
+    // ---------- Attribution ----------
+    const attribution = document.createElement("div");
+    attribution.className = "sl-attribution";
+    attribution.innerHTML = 'Powered by Semantic Link Labs';
+    container.appendChild(attribution);
+
+    // ---------- Wiring ----------
+    model.on("change:dark_mode", applyTheme);
+    model.on("change:dataset_name", renderSubtitle);
+    model.on("change:workspace_name", renderSubtitle);
+    model.on("change:total_duration", renderCards);
+    model.on("change:fe_duration", renderCards);
+    model.on("change:se_duration", renderCards);
+    model.on("change:cpu_time", renderCards);
+    model.on("change:query_executed", renderCards);
+    model.on("change:trace_rows", renderTable);
+    model.on("change:result_columns", renderTable);
+    model.on("change:result_rows", renderTable);
+    model.on("change:result_total_rows", renderTable);
+    model.on("change:result_truncated", renderTable);
+    model.on("change:view_mode", renderTable);
+    model.on("change:trace_history", renderTable);
+    model.on("change:workspace_monitoring_loading", renderMonitoringContent);
+    model.on("change:workspace_monitoring_loaded", renderMonitoringContent);
+    model.on("change:workspace_monitoring_enabled", renderMonitoringContent);
+    model.on("change:workspace_monitoring_error", renderMonitoringContent);
+    model.on("change:workspace_monitoring_columns", renderMonitoringContent);
+    model.on("change:workspace_monitoring_rows", renderMonitoringContent);
+    model.on("change:workspace_monitoring_tokens", renderMonitoringContent);
+    model.on("change:dataset_name", renderMonitoringChrome);
+    model.on("change:workspace_name", renderMonitoringChrome);
+    model.on("change:query_plan_rows", renderTable);
+    model.on("change:query_plan_type", renderTable);
+    model.on("change:execution_metrics", renderTable);
+    model.on("change:dependency_tree", renderTable);
+    model.on("change:dependencies_loading", renderTable);
+    model.on("change:dependency_columns", renderTable);
+    model.on("change:dependency_view", renderTable);
+    model.on("change:object_dependencies_loading", renderObjectDependencies);
+    model.on("change:object_dependencies_loaded", renderObjectDependencies);
+    model.on("change:object_dependency_edges", renderObjectDependencies);
+    model.on("change:object_dependency_error", renderObjectDependencies);
+    model.on("change:vertipaq_sections", () => {
+        vertipaqSortBySection.clear();
+        renderTable();
+    });
+    model.on("change:vertipaq_section", renderTable);
+    model.on("change:vertipaq_loading", renderTable);
+    model.on("change:performance_findings", renderTable);
+    model.on("change:performance_summary", renderTable);
+    model.on("change:performance_loading", () => { renderAnalyzeBtn(); renderTable(); });
+    model.on("change:dataset_chosen", renderAnalyzeBtn);
+    model.on("change:history_excel_b64", () => {
+        const b64 = model.get("history_excel_b64") || "";
+        if (!b64) return;
+        try {
+            const bin = atob(b64);
+            const len = bin.length;
+            const bytes = new Uint8Array(len);
+            for (let i = 0; i < len; i++) bytes[i] = bin.charCodeAt(i);
+            const blob = new Blob([bytes], {
+                type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+            });
+            const url = URL.createObjectURL(blob);
+            const a = document.createElement("a");
+            a.href = url;
+            a.download = model.get("history_excel_name") || "trace_history.xlsx";
+            document.body.appendChild(a);
+            a.click();
+            document.body.removeChild(a);
+            setTimeout(() => URL.revokeObjectURL(url), 1000);
+        } catch (e) {}
+        // Clear so a subsequent identical download still fires a change.
+        model.set("history_excel_b64", "");
+        model.save_changes();
+    });
+    model.on("change:result_excel_b64", () => {
+        const b64 = model.get("result_excel_b64") || "";
+        if (!b64) return;
+        try {
+            const bin = atob(b64);
+            const len = bin.length;
+            const bytes = new Uint8Array(len);
+            for (let i = 0; i < len; i++) bytes[i] = bin.charCodeAt(i);
+            const blob = new Blob([bytes], {
+                type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+            });
+            const url = URL.createObjectURL(blob);
+            const a = document.createElement("a");
+            a.href = url;
+            a.download = model.get("result_excel_name") || "query_result.xlsx";
+            document.body.appendChild(a);
+            a.click();
+            document.body.removeChild(a);
+            setTimeout(() => URL.revokeObjectURL(url), 1000);
+        } catch (e) {}
+        // Clear so a subsequent identical download still fires a change.
+        model.set("result_excel_b64", "");
+        model.save_changes();
+    });
+    model.on("change:is_running", () => {
+        root.classList.toggle("dtx-running", model.get("is_running") === true);
+        renderRunBtn();
+    });
+    model.on("change:error_message", renderError);
+    model.on("change:dax_query", () => {
+        if (textarea.value !== model.get("dax_query")) {
+            const newVal = model.get("dax_query") || "";
+            textarea.value = newVal;
+            // External update (e.g. Format from Python) — record
+            // it as a discrete, undoable history entry.
+            commitHistory(newVal, false);
+        }
+        renderHighlight();
+        renderFmtBtn();
+        // Generated/programmatic query (Query Builder, natural language,
+        // Format) — grow the editor to fit it (up to 20 rows).
+        autoGrowEditor();
+    });
+    model.on("change:dax_tokens", renderHighlight);
+    model.on("change:format_loading", renderFmtBtn);
+    model.on("change:nl_to_dax_loading", () => {
+        renderNlModal();
+        const loading = model.get("nl_to_dax_loading") === true;
+        const err = String(model.get("nl_to_dax_error") || "").trim();
+        // Generation finished without an error -> close the modal.
+        if (!loading && !err && nlOverlay.style.display !== "none") {
+            nlInput.value = "";
+            closeNlModal();
+        }
+    });
+    model.on("change:nl_to_dax_error", () => {
+        const err = String(model.get("nl_to_dax_error") || "");
+        nlError.textContent = err;
+        nlError.style.display = err ? "" : "none";
+    });
+    model.on("change:dataset_chosen", renderNlBtn);
+    model.on("change:clear_cache", renderCacheBtn);
+    model.on("change:cache_clear_loading", renderClearModelCacheBtn);
+    model.on("change:impersonation_mode", renderImpersonation);
+    model.on("change:impersonation_value", renderImpersonation);
+    model.on("change:model_roles", renderImpersonation);
+    model.on("change:available_reports", renderReportCapture);
+    model.on("change:report_capture_loading", renderReportCapture);
+    model.on("change:report_capture_loading", renderClearModelCacheBtn);
+    model.on("change:report_capture_progress", renderReportCapture);
+    model.on("change:report_capture_payload", () => {
+        void runReportCapture(model.get("report_capture_payload") || {});
+    });
+    model.on("change:sidebar_collapsed", renderSidebarChrome);
+    model.on("change:metadata_loading", () => { renderSidebarChrome(); renderTree(); });
+    model.on("change:model_tree", renderTree);
+    model.on("change:dataset_chosen", () => {
+        if (model.get("dataset_chosen") === true) {
+            connectingToModel = false;
+            pickerOpen = false;
+        }
+        renderPicker(); renderRunBtn(); renderClearModelCacheBtn(); renderSubtitle();
+        renderBuildBtn(); renderBuilderChrome(); renderModelViewChrome();
+        renderMonitoringChrome(); renderMonitoringContent();
+    });
+    model.on("change:available_workspaces", renderPicker);
+    model.on("change:available_datasets", renderPicker);
+    model.on("change:selected_workspace_id", renderPicker);
+    model.on("change:selected_dataset_id", renderPicker);
+    model.on("change:active_workspace_id", renderPicker);
+    model.on("change:active_dataset_id", () => {
+        // A new model finished activating — close the picker. This covers
+        // switching between two already-chosen models, where dataset_chosen
+        // does not change and so would not otherwise close the picker.
+        // During the first connection active_dataset_id arrives before
+        // dataset_chosen. Keep the connecting guard set until dataset_chosen
+        // changes so renderPicker cannot briefly reopen the picker.
+        if (model.get("dataset_chosen") === true) connectingToModel = false;
+        pickerOpen = false;
+        resetBuilderForModelChange();
+        monitoringSort = null;
+        monitoringSearch = "";
+        monitoringSearchInput.value = "";
+        // Force a fresh dependency computation for the newly activated model.
+        lastDepQuery = null;
+        closeObjectDependencies();
+        renderPicker();
+    });
+    model.on("change:picker_loading", () => {
+        const loading = model.get("picker_loading") === true;
+        const selected = String(model.get("selected_dataset_id") || "");
+        const active = String(model.get("active_dataset_id") || "");
+        const activationError = String(model.get("error_message") || "").trim();
+        if (connectingToModel && !loading && selected !== active && activationError) {
+            connectingToModel = false;
+            pickerOpen = true;
+        }
+        renderPicker();
+        renderSubtitle();
+    });
+
+    applyTheme();
+    renderSubtitle();
+    renderCards();
+    renderRunBtn();
+    renderClearModelCacheBtn();
+    renderCacheBtn();
+    renderImpersonation();
+    renderReportCapture();
+    renderError();
+    renderTable();
+    renderAnalyzeBtn();
+    renderSidebarChrome();
+    renderPicker();
+    renderFmtBtn();
+    renderNlBtn();
+    renderHistBtns();
+    renderTree();
+    renderHighlight();
+    autoGrowEditor();
+    renderBuilderChrome();
+    renderMonitoringChrome();
+    renderMonitoringContent();
+    renderBuilderZones();
+    renderBuildBtn();
+
+    // Request the initial workspace list only after the front-end is fully
+    // rendered and its comm listeners are active. Starting the worker directly
+    // after display(widget) can race the comm handshake and strand the browser
+    // in its loading state.
+    if (model.get("dataset_chosen") !== true
+        && (model.get("available_workspaces") || []).length === 0
+        && model.get("picker_loading") !== true) {
+        model.set("load_workspaces_trigger",
+            (model.get("load_workspaces_trigger") || 0) + 1);
+        model.save_changes();
+    }
+
+    // Notify Python to tear down the long-running trace when this view is
+    // disposed (cell re-run, widget removed, notebook closed).
+    return () => {
+        try {
+            document.removeEventListener("pointerdown", hideReportMenuOnOutsidePointer);
+            window.removeEventListener("resize", scheduleResponsiveQueryOptions);
+            queryOptionsObserver?.disconnect();
+            if (responsiveOptionsFrame !== null) {
+                window.cancelAnimationFrame(responsiveOptionsFrame);
+            }
+            outputTableObserver.disconnect();
+            reportCaptureFrame.remove();
+            model.set("close_trigger", (model.get("close_trigger") || 0) + 1);
+            model.save_changes();
+        } catch (e) {}
+    };
+}
+export default { render };
+"""
+    )
+    widget_js = (
+        widget_js.replace("__DTX_SUN__", sun_icon)
+        .replace("__DTX_MOON__", moon_icon)
+        .replace("__DTX_INFO__", info_icon)
+        .replace("__DTX_TABLE__", table_icon)
+        .replace("__DTX_CALC_GROUP__", calc_group_icon)
+        .replace("__DTX_CALC_ITEM__", calc_item_icon)
+        .replace("__DTX_COLUMN__", column_icon)
+        .replace("__DTX_MEASURE__", measure_icon)
+        .replace("__DTX_HIERARCHY__", hierarchy_icon)
+        .replace("__DTX_CARET__", caret_icon)
+        .replace("__DTX_FOLDER__", folder_icon)
+        .replace("__DTX_LEVEL__", level_icon)
+        .replace("__DTX_PLAY__", play_icon)
+        .replace("__DTX_STOP__", stop_icon)
+        .replace("__DTX_ERASER__", eraser_icon)
+        .replace("__DTX_REFRESH__", refresh_icon)
+        .replace("__DTX_SWAP__", swap_icon)
+        .replace("__DTX_SORT_ASC__", sort_asc_icon)
+        .replace("__DTX_SORT_DESC__", sort_desc_icon)
+        .replace("__DTX_PANEL_COLLAPSE__", panel_collapse_icon)
+        .replace("__DTX_PANEL_EXPAND__", panel_expand_icon)
+        .replace("__DTX_BUILDER__", builder_icon)
+        .replace("__DTX_LIST_TREE__", list_tree_icon)
+        .replace("__DTX_GIT_BRANCH__", git_branch_icon)
+        .replace("__DTX_WORKFLOW__", workflow_icon)
+        .replace("__DTX_SHIELD_CHECK__", shield_check_icon)
+        .replace("__DTX_USERS__", users_icon)
+        .replace("__DTX_USER__", user_icon)
+        .replace("__DTX_CLOSE__", close_icon)
+        .replace("__DTX_DAXFORMAT__", daxformat_icon)
+        .replace("__DTX_UNDO__", undo_icon)
+        .replace("__DTX_REDO__", redo_icon)
+        .replace("__DTX_DOWNLOAD__", download_icon)
+        .replace("__DTX_TRASH__", trash_icon)
+        .replace("__DTX_CAMERA__", camera_icon)
+        .replace("__DTX_REPORT_FILE__", report_file_icon)
+        .replace("__DTX_CHEVRON_DOWN__", chevron_down_icon)
+        .replace("__DTX_CHECK__", check_icon)
+        .replace("__DTX_CUT__", cut_icon)
+        .replace("__DTX_COPY__", copy_icon)
+        .replace("__DTX_PASTE__", paste_icon)
+        .replace("__DTX_ANALYZE__", analyze_icon)
+        .replace("__DTX_NLDAX__", nldax_icon)
+        .replace("__DTX_EXPAND__", expand_icon)
+        .replace("__DTX_FULLSCREEN__", fullscreen_icon)
+        .replace("__DTX_FULLSCREEN_EXIT__", fullscreen_exit_icon)
+        .replace("__DTX_DAX_PERFORMANCE__", dax_performance_icon)
+        .replace("__DTX_ACTIVITY__", activity_icon)
+        .replace("__DTX_CPU__", cpu_icon)
+        .replace("__DTX_DATABASE__", database_icon)
+        .replace("__DTX_VERTIPAQ__", vertipaq_icon)
+        .replace("__DTX_ZAP__", zap_icon)
+    )
+
+    class DaxTestWidget(anywidget.AnyWidget):
+        _esm = widget_js
+        _css = widget_css
+
+        dax_query = traitlets.Unicode("").tag(sync=True)
+        dax_tokens = traitlets.List([]).tag(sync=True)
+        dataset_name = traitlets.Unicode("").tag(sync=True)
+        workspace_name = traitlets.Unicode("").tag(sync=True)
+        dark_mode = traitlets.Bool(False).tag(sync=True)
+        clear_cache = traitlets.Bool(True).tag(sync=True)
+        cache_clear_trigger = traitlets.Int(0).tag(sync=True)
+        cache_clear_loading = traitlets.Bool(False).tag(sync=True)
+        total_duration = traitlets.Int(0).tag(sync=True)
+        fe_duration = traitlets.Int(0).tag(sync=True)
+        se_duration = traitlets.Int(0).tag(sync=True)
+        cpu_time = traitlets.Int(0).tag(sync=True)
+        query_executed = traitlets.Bool(False).tag(sync=True)
+        trace_rows = traitlets.List([]).tag(sync=True)
+        query_plan_rows = traitlets.List([]).tag(sync=True)
+        query_plan_type = traitlets.Unicode("Logical").tag(sync=True)
+        execution_metrics = traitlets.List([]).tag(sync=True)
+        result_columns = traitlets.List([]).tag(sync=True)
+        result_rows = traitlets.List([]).tag(sync=True)
+        result_total_rows = traitlets.Int(0).tag(sync=True)
+        result_truncated = traitlets.Bool(False).tag(sync=True)
+        view_mode = traitlets.Unicode("trace").tag(sync=True)
+        trace_history = traitlets.List([]).tag(sync=True)
+        download_history_trigger = traitlets.Int(0).tag(sync=True)
+        history_excel_b64 = traitlets.Unicode("").tag(sync=True)
+        history_excel_name = traitlets.Unicode("").tag(sync=True)
+        download_result_trigger = traitlets.Int(0).tag(sync=True)
+        result_excel_b64 = traitlets.Unicode("").tag(sync=True)
+        result_excel_name = traitlets.Unicode("").tag(sync=True)
+        is_running = traitlets.Bool(False).tag(sync=True)
+        error_message = traitlets.Unicode("").tag(sync=True)
+        run_trigger = traitlets.Int(0).tag(sync=True)
+        cancel_trigger = traitlets.Int(0).tag(sync=True)
+        dependency_tree = traitlets.List([]).tag(sync=True)
+        dependencies_loading = traitlets.Bool(False).tag(sync=True)
+        dependencies_trigger = traitlets.Int(0).tag(sync=True)
+        dependency_columns = traitlets.List([]).tag(sync=True)
+        dependency_view = traitlets.Unicode("tree").tag(sync=True)
+        object_dependency_target = traitlets.Dict({}).tag(sync=True)
+        object_dependency_trigger = traitlets.Int(0).tag(sync=True)
+        object_dependencies_loading = traitlets.Bool(False).tag(sync=True)
+        object_dependencies_loaded = traitlets.Bool(False).tag(sync=True)
+        object_dependency_edges = traitlets.List([]).tag(sync=True)
+        object_dependency_error = traitlets.Unicode("").tag(sync=True)
+        vertipaq_sections = traitlets.List([]).tag(sync=True)
+        vertipaq_section = traitlets.Unicode("").tag(sync=True)
+        vertipaq_loading = traitlets.Bool(False).tag(sync=True)
+        vertipaq_trigger = traitlets.Int(0).tag(sync=True)
+        performance_findings = traitlets.List([]).tag(sync=True)
+        performance_summary = traitlets.Dict({}).tag(sync=True)
+        performance_loading = traitlets.Bool(False).tag(sync=True)
+        performance_trigger = traitlets.Int(0).tag(sync=True)
+        model_tree = traitlets.List([]).tag(sync=True)
+        sidebar_collapsed = traitlets.Bool(False).tag(sync=True)
+        refresh_metadata_trigger = traitlets.Int(0).tag(sync=True)
+        metadata_loading = traitlets.Bool(False).tag(sync=True)
+        impersonation_mode = traitlets.Unicode("none").tag(sync=True)
+        impersonation_value = traitlets.Unicode("").tag(sync=True)
+        model_roles = traitlets.List([]).tag(sync=True)
+        available_reports = traitlets.List([]).tag(sync=True)
+        capture_report_ids = traitlets.List([]).tag(sync=True)
+        report_capture_start_trigger = traitlets.Int(0).tag(sync=True)
+        report_capture_finish_trigger = traitlets.Int(0).tag(sync=True)
+        report_capture_loading = traitlets.Bool(False).tag(sync=True)
+        report_capture_progress = traitlets.Unicode("").tag(sync=True)
+        report_capture_payload = traitlets.Dict({}).tag(sync=True)
+        report_capture_client_error = traitlets.Unicode("").tag(sync=True)
+        report_capture_checkpoint = traitlets.Dict({}).tag(sync=True)
+        report_capture_checkpoint_trigger = traitlets.Int(0).tag(sync=True)
+        report_capture_checkpoint_ack = traitlets.Unicode("").tag(sync=True)
+        dataset_chosen = traitlets.Bool(False).tag(sync=True)
+        available_workspaces = traitlets.List([]).tag(sync=True)
+        available_datasets = traitlets.List([]).tag(sync=True)
+        selected_workspace_id = traitlets.Unicode("").tag(sync=True)
+        selected_dataset_id = traitlets.Unicode("").tag(sync=True)
+        active_workspace_id = traitlets.Unicode("").tag(sync=True)
+        active_dataset_id = traitlets.Unicode("").tag(sync=True)
+        picker_loading = traitlets.Bool(False).tag(sync=True)
+        select_workspace_trigger = traitlets.Int(0).tag(sync=True)
+        select_dataset_trigger = traitlets.Int(0).tag(sync=True)
+        load_workspaces_trigger = traitlets.Int(0).tag(sync=True)
+        format_query_trigger = traitlets.Int(0).tag(sync=True)
+        format_loading = traitlets.Bool(False).tag(sync=True)
+        nl_to_dax_text = traitlets.Unicode("").tag(sync=True)
+        nl_to_dax_trigger = traitlets.Int(0).tag(sync=True)
+        nl_to_dax_loading = traitlets.Bool(False).tag(sync=True)
+        nl_to_dax_error = traitlets.Unicode("").tag(sync=True)
+        query_builder_state = traitlets.Unicode("").tag(sync=True)
+        build_query_trigger = traitlets.Int(0).tag(sync=True)
+        workspace_monitoring_request = traitlets.Dict({}).tag(sync=True)
+        workspace_monitoring_trigger = traitlets.Int(0).tag(sync=True)
+        workspace_monitoring_loading = traitlets.Bool(False).tag(sync=True)
+        workspace_monitoring_loaded = traitlets.Bool(False).tag(sync=True)
+        workspace_monitoring_enabled = traitlets.Bool(True).tag(sync=True)
+        workspace_monitoring_error = traitlets.Unicode("").tag(sync=True)
+        workspace_monitoring_columns = traitlets.List([]).tag(sync=True)
+        workspace_monitoring_rows = traitlets.List([]).tag(sync=True)
+        workspace_monitoring_tokens = traitlets.List([]).tag(sync=True)
+        close_trigger = traitlets.Int(0).tag(sync=True)
+
+    initial_result = _result_payload_from_df(result_df)
+
+    # Mutable model context so the front-end model picker can switch the
+    # active dataset/workspace at runtime (the run/metadata workers read the
+    # current ids from this dict rather than closing over fixed values).
+    model_ctx = {"dataset_id": dataset_id, "workspace_id": workspace_id}
+    dataset_chosen = dataset_id is not None
+
+    # Collect the model metadata tree synchronously before constructing the
+    # widget. Loading it in a background thread that sets traits right after
+    # display() races with the widget comm being opened: the finished-tree
+    # update can be sent before the front-end is listening, leaving the
+    # sidebar stuck on "Loading model metadata…". The tree collection is
+    # fast, so building it up-front (and shipping it as initial state) is
+    # both reliable and quick.
+    if dataset_chosen:
+        try:
+            initial_tree, initial_roles = _collect_model_metadata(
+                dataset_id, workspace_id
+            )
+        except Exception:
+            initial_tree = []
+            initial_roles = []
+        try:
+            initial_reports = _list_reports_for_capture(dataset_id, workspace_id)
+        except Exception:
+            initial_reports = []
+    else:
+        initial_tree = []
+        initial_roles = []
+        initial_reports = []
+
+    # Avoid blocking the initial picker screen on workspace enumeration. For a
+    # supplied dataset, retain the existing eager picker data so Change Model
+    # is immediately ready.
+    if dataset_chosen:
+        try:
+            initial_workspaces = _list_workspaces_for_picker()
+        except Exception:
+            initial_workspaces = []
+        try:
+            initial_datasets = _list_datasets_for_picker(workspace_id)
+        except Exception:
+            initial_datasets = []
+    else:
+        initial_workspaces = []
+        initial_datasets = []
+
+    widget = DaxTestWidget(
+        dax_query=formatted_initial or "",
+        dax_tokens=_classify_dax_spans(formatted_initial or ""),
+        dataset_name=dataset_name or "",
+        workspace_name=workspace_name or "",
+        dark_mode=bool(dark_mode),
+        clear_cache=bool(clear_cache),
+        total_duration=int(total_duration),
+        fe_duration=int(fe_duration),
+        se_duration=int(se_duration),
+        cpu_time=int(cpu_time),
+        query_executed=bool(dataset_chosen and dax_string and dax_string.strip()),
+        trace_rows=initial_rows,
+        query_plan_rows=initial_query_plan_rows,
+        query_plan_type="Logical",
+        execution_metrics=initial_execution_metrics,
+        result_columns=initial_result["columns"],
+        result_rows=initial_result["rows"],
+        result_total_rows=int(initial_result["total_rows"]),
+        result_truncated=bool(initial_result["truncated"]),
+        view_mode="trace",
+        is_running=False,
+        error_message="",
+        run_trigger=0,
+        cancel_trigger=0,
+        dependency_tree=[],
+        dependencies_loading=False,
+        dependencies_trigger=0,
+        dependency_columns=[],
+        dependency_view="tree",
+        object_dependency_target={},
+        object_dependency_trigger=0,
+        object_dependencies_loading=False,
+        object_dependencies_loaded=False,
+        object_dependency_edges=[],
+        object_dependency_error="",
+        vertipaq_sections=[],
+        vertipaq_section="",
+        vertipaq_loading=False,
+        vertipaq_trigger=0,
+        performance_findings=[],
+        performance_summary={},
+        performance_loading=False,
+        performance_trigger=0,
+        model_tree=initial_tree,
+        sidebar_collapsed=False,
+        refresh_metadata_trigger=0,
+        metadata_loading=False,
+        dataset_chosen=dataset_chosen,
+        available_workspaces=initial_workspaces,
+        available_datasets=initial_datasets,
+        selected_workspace_id=str(workspace_id) if workspace_id else "",
+        selected_dataset_id="",
+        active_workspace_id=str(workspace_id) if workspace_id else "",
+        active_dataset_id=str(dataset_id) if dataset_id else "",
+        picker_loading=False,
+        select_workspace_trigger=0,
+        select_dataset_trigger=0,
+        load_workspaces_trigger=0,
+        format_query_trigger=0,
+        format_loading=False,
+        nl_to_dax_text="",
+        nl_to_dax_trigger=0,
+        nl_to_dax_loading=False,
+        nl_to_dax_error="",
+        query_builder_state="",
+        build_query_trigger=0,
+        workspace_monitoring_request={},
+        workspace_monitoring_trigger=0,
+        workspace_monitoring_loading=False,
+        workspace_monitoring_loaded=False,
+        workspace_monitoring_enabled=True,
+        workspace_monitoring_error="",
+        workspace_monitoring_columns=[],
+        workspace_monitoring_rows=[],
+        workspace_monitoring_tokens=[],
+        impersonation_mode=(
+            "user" if effective_user_name else ("role" if role else "none")
+        ),
+        impersonation_value=(effective_user_name or role or ""),
+        model_roles=initial_roles,
+        available_reports=initial_reports,
+        capture_report_ids=[],
+        report_capture_start_trigger=0,
+        report_capture_finish_trigger=0,
+        report_capture_loading=False,
+        report_capture_progress="",
+        report_capture_payload={},
+        report_capture_client_error="",
+        report_capture_checkpoint={},
+        report_capture_checkpoint_trigger=0,
+        report_capture_checkpoint_ack="",
+    )
+
+    # Expose the most recent dataframes for programmatic access.
+    widget.last_df = df  # type: ignore[attr-defined]
+    widget.last_result_df = result_df  # type: ignore[attr-defined]
+    # Most recent Vertipaq Analyzer result (dict of dataframes), populated
+    # when the user opens the Vertipaq Analyzer tab. Stored for later use.
+    widget.last_vertipaq = {}  # type: ignore[attr-defined]
+
+    # State shared between the run/cancel observers.
+    import threading
+
+    run_state = {
+        "thread": None,
+        "current_run_id": 0,
+        "canceled_run_ids": set(),
+        # The DAX query that the currently-populated trace artifacts (trace
+        # rows, durations, query plan, execution metrics) belong to, and the
+        # query the cached dependency columns belong to. Used by the
+        # performance analysis to decide whether those details can be reused or
+        # must be (re)captured for the query currently in the query pane.
+        "traced_query": None,
+        "deps_query": None,
+    }
+    state_lock = threading.Lock()
+
+    # ---- Persistent (long-running) trace shared by all queries in the UI ----
+    # Instead of creating a fresh trace per query, the widget keeps a single
+    # trace running for the active model. It is started when the model
+    # metadata is loaded and torn down when the UI is closed. Each query reads
+    # the rows it produced live via ``trace.get_trace_logs()`` (tracked with a
+    # baseline row count) without stopping the trace.
+    trace_ctx: dict = {
+        "connection": None,
+        "trace": None,
+        "dataset_id": None,
+        "workspace_id": None,
+        "baseline": 0,
+        "started": False,
+        "warmed_up": False,
+    }
+    report_capture_state = {
+        "active": False,
+        "baseline": 0,
+        "nonce": "",
+        "entries": [],
+    }
+    trace_lock = threading.Lock()
+
+    def _teardown_trace_locked() -> None:
+        """Stop/drop the running trace and dispose its connection. Caller must
+        hold ``trace_lock``."""
+        tr = trace_ctx.get("trace")
+        conn = trace_ctx.get("connection")
+        if tr is not None:
+            try:
+                tr.drop()
+            except Exception:
+                pass
+        if conn is not None:
+            try:
+                conn.disconnect_and_dispose()
+            except Exception:
+                pass
+        trace_ctx["connection"] = None
+        trace_ctx["trace"] = None
+        trace_ctx["started"] = False
+        trace_ctx["baseline"] = 0
+        trace_ctx["dataset_id"] = None
+        trace_ctx["workspace_id"] = None
+        trace_ctx["warmed_up"] = False
+
+    def _ensure_trace(ds_id: Optional[str], ws_id: Optional[str]) -> None:
+        """Ensure a long-running trace is active for the given model. Starts a
+        new trace (rebinding from any previous model) when needed. Safe to call
+        repeatedly; a no-op when already running for the same model."""
+        if not ds_id:
+            return
+        with trace_lock:
+            if (
+                trace_ctx["started"]
+                and trace_ctx["dataset_id"] == ds_id
+                and trace_ctx["workspace_id"] == ws_id
+            ):
+                return
+            # Switching models (or first start): tear down any existing trace.
+            _teardown_trace_locked()
+            try:
+                conn = fabric.create_trace_connection(dataset=ds_id, workspace=ws_id)
+                trace = conn.create_trace(_TEST_EVENT_SCHEMA)
+                trace.start()
+                # Prime the trace: a freshly started trace does not begin
+                # capturing server-side events instantly, so the very first
+                # real query's events can be missed. Run the throwaway warm-up
+                # query now and wait until it actually shows up in the trace
+                # logs, which confirms the trace is live. The baseline is then
+                # advanced past these warm-up rows so the first real query is
+                # captured from a known-good state.
+                baseline = 0
+                warmed = False
+                try:
+                    fabric.evaluate_dax(
+                        dataset=ds_id,
+                        workspace=ws_id,
+                        dax_string="EVALUATE {1}",
+                    )
+                    _deadline = time.monotonic() + 5.0
+                    _qe_seen_at: Optional[float] = None
+                    _last_len = -1
+                    _stable_at: Optional[float] = None
+                    while time.monotonic() < _deadline:
+                        time.sleep(0.1)
+                        try:
+                            _logs = _get_trace_logs(trace)
+                        except Exception:
+                            continue
+                        if _logs is None or _logs.empty:
+                            continue
+                        _ec = (
+                            "Event Class"
+                            if "Event Class" in _logs.columns
+                            else "EventClass"
+                        )
+                        if _ec not in _logs.columns:
+                            continue
+                        if (
+                            _qe_seen_at is None
+                            and not _logs[_logs[_ec] == "QueryEnd"].empty
+                        ):
+                            _qe_seen_at = time.monotonic()
+                            warmed = True
+                        if _qe_seen_at is None:
+                            continue
+                        # The warm-up's QueryEnd has been seen. Let any of its
+                        # trailing rows (e.g. a late DAXQueryPlan) settle before
+                        # fixing the baseline, so the warm-up's plan is never
+                        # mis-attributed to the first real query.
+                        if len(_logs) != _last_len:
+                            _last_len = len(_logs)
+                            _stable_at = time.monotonic()
+                        baseline = len(_logs)
+                        if _stable_at is not None and (
+                            time.monotonic() - _stable_at >= 0.6
+                        ):
+                            break
+                        if time.monotonic() - _qe_seen_at >= 2.0:
+                            break
+                except Exception:
+                    pass
+                trace_ctx["connection"] = conn
+                trace_ctx["trace"] = trace
+                trace_ctx["dataset_id"] = ds_id
+                trace_ctx["workspace_id"] = ws_id
+                trace_ctx["baseline"] = baseline
+                trace_ctx["started"] = True
+                trace_ctx["warmed_up"] = warmed
+            except Exception:
+                # Tracing could not be started; queries fall back to a
+                # one-shot trace via ``_run_dax_trace``.
+                _teardown_trace_locked()
+
+    def _stop_persistent_trace(*_args) -> None:
+        """Tear down the long-running trace (called when the UI is closed)."""
+        with trace_lock:
+            _teardown_trace_locked()
+
+    def _start_report_capture() -> None:
+        try:
+            if widget.is_running:
+                raise RuntimeError("Wait for the current DAX query to finish first.")
+            report_ids = [str(value) for value in (widget.capture_report_ids or [])]
+            available = {
+                str(report.get("id")): report
+                for report in (widget.available_reports or [])
+            }
+            reports = [available[value] for value in report_ids if value in available]
+            reports = [report for report in reports if report.get("embed_url")]
+            if not reports:
+                raise ValueError("Select at least one embeddable report.")
+
+            ds_id = model_ctx["dataset_id"]
+            ws_id = model_ctx["workspace_id"]
+            _ensure_trace(ds_id, ws_id)
+            with trace_lock:
+                trace = trace_ctx["trace"] if trace_ctx["started"] else None
+                if trace is None:
+                    raise RuntimeError("Unable to start the semantic model trace.")
+                logs = _get_trace_logs(trace)
+                baseline = 0 if logs is None else len(logs)
+                trace_ctx["baseline"] = baseline
+
+            if widget.clear_cache:
+                from sempy_labs._clear_cache import clear_cache as _clear_cache_fn
+
+                _clear_cache_fn(dataset=ds_id, workspace=ws_id)
+
+            from sempy_labs.report._generate_embed_token import generate_embed_token
+
+            token = generate_embed_token(
+                dataset_ids=[ds_id],
+                report_ids=[report["id"] for report in reports],
+            )
+            if not token:
+                raise RuntimeError("Power BI did not return an embed token.")
+
+            nonce = str(time.time_ns())
+            report_payload = [
+                {**report, "workspace_name": str(widget.workspace_name or "")}
+                for report in reports
+            ]
+            report_capture_state.update(
+                {
+                    "active": True,
+                    "baseline": baseline,
+                    "nonce": nonce,
+                    "entries": [],
+                }
+            )
+            with state_lock:
+                run_state["current_run_id"] += 1
+            widget.report_capture_client_error = ""
+            widget.report_capture_checkpoint = {}
+            widget.report_capture_checkpoint_ack = ""
+            widget.report_capture_payload = {
+                "nonce": nonce,
+                "token": token,
+                "reports": report_payload,
+            }
+        except Exception as exc:  # noqa: BLE001
+            report_capture_state.update(
+                {"active": False, "baseline": 0, "nonce": "", "entries": []}
+            )
+            widget.report_capture_payload = {}
+            widget.report_capture_loading = False
+            widget.report_capture_progress = ""
+            widget.error_message = f"Failed to start report query capture: {exc}"
+
+    def _checkpoint_report_capture() -> None:
+        checkpoint = dict(widget.report_capture_checkpoint or {})
+        checkpoint_id = str(checkpoint.get("checkpoint_id") or "")
+        try:
+            if not report_capture_state["active"]:
+                return
+            if str(checkpoint.get("nonce") or "") != report_capture_state["nonce"]:
+                return
+            time.sleep(0.8)
+            baseline = int(report_capture_state["baseline"])
+            with trace_lock:
+                trace = trace_ctx["trace"] if trace_ctx["started"] else None
+                logs = _get_trace_logs(trace) if trace is not None else None
+                total_count = 0 if logs is None else len(logs)
+                if trace_ctx["trace"] is trace:
+                    trace_ctx["baseline"] = total_count
+            report_capture_state["baseline"] = total_count
+            if logs is None or total_count <= baseline:
+                new_logs = pd.DataFrame()
+            else:
+                new_logs = logs.iloc[baseline:].reset_index(drop=True)
+            captured = _captured_queries_from_df(new_logs)
+            stamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
+            report_name = str(checkpoint.get("report_name") or "")
+            report_workspace_name = str(checkpoint.get("workspace_name") or "")
+            for index, item in enumerate(captured):
+                report_capture_state["entries"].append(
+                    {
+                        "run_id": f"report-{report_capture_state['nonce']}-{checkpoint_id}-{index}",
+                        "method": "Report",
+                        "report_name": report_name,
+                        "report_workspace_name": report_workspace_name,
+                        "dax_query": item["dax_query"],
+                        "start_time": stamp,
+                        "end_time": stamp,
+                        "rows": 0,
+                        "duration": item["duration"],
+                        "cpu": item["cpu"],
+                        "fe_duration": item["fe_duration"],
+                        "se_duration": item["se_duration"],
+                        "cache": "Cold" if widget.clear_cache else "Warm",
+                        "execution_metrics": {},
+                        "dataset_name": str(widget.dataset_name or ""),
+                        "workspace_name": str(widget.workspace_name or ""),
+                        "impersonation_type": "None",
+                        "impersonation": "None",
+                    }
+                )
+        except Exception as exc:  # noqa: BLE001
+            widget.report_capture_client_error = str(exc)
+        finally:
+            if checkpoint_id:
+                widget.report_capture_checkpoint_ack = checkpoint_id
+
+    def _finish_report_capture() -> None:
+        if not report_capture_state["active"]:
+            widget.report_capture_loading = False
+            widget.report_capture_progress = ""
+            return
+        try:
+            entries = list(report_capture_state["entries"])
+            if entries:
+                widget.trace_history = list(reversed(entries)) + list(
+                    widget.trace_history
+                )
+                widget.view_mode = "history"
+
+            client_error = (widget.report_capture_client_error or "").strip()
+            if client_error:
+                widget.error_message = f"Report capture ended early: {client_error}"
+            elif not entries:
+                widget.error_message = (
+                    "No DAX queries were captured. The selected reports may have "
+                    "no visible data visuals or may have returned cached results."
+                )
+            else:
+                widget.error_message = ""
+        except Exception as exc:  # noqa: BLE001
+            widget.error_message = f"Failed to collect report queries: {exc}"
+        finally:
+            report_capture_state.update(
+                {"active": False, "baseline": 0, "nonce": "", "entries": []}
+            )
+            widget.report_capture_payload = {}
+            widget.report_capture_checkpoint = {}
+            widget.report_capture_loading = False
+            widget.report_capture_progress = ""
+
+    def _on_report_capture_start(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_start_report_capture, daemon=True).start()
+
+    def _on_report_capture_finish(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_finish_report_capture, daemon=True).start()
+
+    def _on_report_capture_checkpoint(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_checkpoint_report_capture, daemon=True).start()
+
+    def _update_history_execution_metrics(history_id, metric_rows: list) -> None:
+        metrics = _execution_metrics_dict(metric_rows)
+        if not metrics:
+            return
+        history = []
+        changed = False
+        for entry in widget.trace_history:
+            if entry.get("run_id") == history_id:
+                entry = dict(entry)
+                entry["execution_metrics"] = metrics
+                changed = True
+            history.append(entry)
+        if changed:
+            widget.trace_history = history
+
+    def _run_query_persistent(
+        query: str,
+        clear_cache_flag: bool,
+        effective_user: Optional[str],
+        role_name: Optional[str],
+    ) -> Tuple[pd.DataFrame, int, int, int, int, pd.DataFrame, Optional[int]]:
+        """Run a query against the long-running trace, capturing only the rows
+        it produced. Falls back to a one-shot trace if the persistent trace is
+        not available.
+
+        The final tuple element is the trace-log baseline (row count) at which
+        this query started, or ``None`` for the one-shot fallback. The caller
+        uses it to back-fill a late-arriving DAX query plan from the persistent
+        trace after the results have already been shown."""
+        from sempy_labs._clear_cache import clear_cache as _clear_cache_fn
+
+        ds_id = model_ctx["dataset_id"]
+        ws_id = model_ctx["workspace_id"]
+        _ensure_trace(ds_id, ws_id)
+        with trace_lock:
+            trace = trace_ctx["trace"] if trace_ctx["started"] else None
+            baseline = trace_ctx["baseline"]
+            run_warmup = not trace_ctx["warmed_up"]
+        if trace is None:
+            # Persistent tracing unavailable: one-shot fallback (no back-fill).
+            return _run_dax_trace(
+                dataset_id=ds_id,
+                workspace_id=ws_id,
+                dax_string=query,
+                clear_cache=clear_cache_flag,
+                effective_user_name=effective_user,
+                role=role_name,
+            ) + (None,)
+        if clear_cache_flag:
+            _clear_cache_fn(dataset=ds_id, workspace=ws_id)
+        result_df, new_logs, new_count = _execute_and_capture(
+            trace,
+            ds_id,
+            ws_id,
+            query,
+            effective_user,
+            role_name,
+            baseline,
+            run_warmup=run_warmup,
+            wait_for_optional_events=False,
+        )
+        with trace_lock:
+            # Only advance the baseline if the trace wasn't rebound meanwhile.
+            if trace_ctx["trace"] is trace:
+                trace_ctx["baseline"] = new_count
+                trace_ctx["warmed_up"] = True
+        df, total, fe, se, cpu = _compute_trace_stats(new_logs)
+        return df, total, fe, se, cpu, result_df, baseline
+
+    import atexit
+
+    atexit.register(_stop_persistent_trace)
+
+    def _backfill_query_plan(run_id: int, start_baseline: int, query: str = "") -> None:
+        """Watch the persistent trace after a query has returned and back-fill
+        the DAX query plan (and execution metrics) if they flush late.
+
+        A query's ``DAXQueryPlan`` and ``ExecutionMetrics`` events are delivered
+        to the trace buffer asynchronously and, on a busy capacity, can arrive a
+        few seconds after the query itself completed — i.e. after the inline
+        capture in ``_execute_and_capture`` has already returned the results.
+        This keeps polling the trace (slicing from this query's start baseline)
+        and sets ``query_plan_rows`` / ``execution_metrics`` the moment they
+        appear, so those tabs are populated reliably without ever delaying the
+        result grid.
+
+        The poll stops as soon as a newer run starts (or this run is canceled):
+        the trace rows between this query's start baseline and the next query's
+        events belong exclusively to this query, so stopping there prevents
+        attributing a later query's plan to this one."""
+        _deadline = time.monotonic() + 25.0
+        _need_plan = not (widget.query_plan_rows or [])
+        _need_metrics = not (widget.execution_metrics or [])
+        while time.monotonic() < _deadline:
+            time.sleep(0.3)
+            with state_lock:
+                superseded = run_state["current_run_id"] != run_id
+                canceled = run_id in run_state["canceled_run_ids"]
+            if superseded or canceled:
+                return
+            with trace_lock:
+                trace = trace_ctx["trace"] if trace_ctx["started"] else None
+            if trace is None:
+                return
+            try:
+                logs = _get_trace_logs(trace)
+            except Exception:
+                continue
+            if logs is None or logs.empty or len(logs) <= start_baseline:
+                continue
+            _new = logs.iloc[start_baseline:]
+            # Restrict to the request that ran the query pane's query so a late
+            # plan from an auxiliary query is never shown for the user's query.
+            _req_id = _resolve_query_request_id(_new, query)
+            if _req_id is not None:
+                _new = _filter_logs_to_request_id(_new, _req_id)
+            if _need_plan:
+                plan_rows = _query_plan_rows_from_df(_new)
+                if plan_rows:
+                    widget.query_plan_rows = plan_rows
+                    _need_plan = False
+            if _need_metrics:
+                metric_rows = _execution_metrics_from_df(_new)
+                if metric_rows:
+                    widget.execution_metrics = metric_rows
+                    _update_history_execution_metrics(run_id, metric_rows)
+                    _need_metrics = False
+            if not _need_plan and not _need_metrics:
+                return
+
+    def _worker(
+        query: str,
+        clear_cache_flag: bool,
+        run_id: int,
+        effective_user: Optional[str],
+        role_name: Optional[str],
+    ) -> None:
+        _start_dt = datetime.now(timezone.utc)
+        try:
+            (
+                new_df,
+                new_total,
+                new_fe,
+                new_se,
+                new_cpu,
+                new_result,
+                start_baseline,
+            ) = _run_query_persistent(
+                query=query,
+                clear_cache_flag=clear_cache_flag,
+                effective_user=effective_user,
+                role_name=role_name,
+            )
+        except Exception as exc:  # noqa: BLE001
+            with state_lock:
+                canceled = run_id in run_state["canceled_run_ids"]
+            if canceled:
+                return
+            widget.error_message = f"{type(exc).__name__}: {exc}"
+            widget.is_running = False
+            return
+
+        with state_lock:
+            canceled = run_id in run_state["canceled_run_ids"]
+        if canceled:
+            # User canceled — discard results. The underlying engine may
+            # still have completed the query in the background.
+            return
+
+        widget.last_df = new_df  # type: ignore[attr-defined]
+        widget.last_result_df = new_result  # type: ignore[attr-defined]
+        widget.total_duration = int(new_total)
+        widget.fe_duration = int(new_fe)
+        widget.se_duration = int(new_se)
+        widget.cpu_time = int(new_cpu)
+        widget.query_executed = True
+        widget.trace_rows = _trace_rows_from_df(new_df)
+        widget.query_plan_rows = _query_plan_rows_from_df(new_df)
+        metric_rows = _execution_metrics_from_df(new_df)
+        widget.execution_metrics = metric_rows
+        with state_lock:
+            run_state["traced_query"] = query
+        payload = _result_payload_from_df(new_result)
+        widget.result_columns = payload["columns"]
+        widget.result_rows = payload["rows"]
+        widget.result_total_rows = int(payload["total_rows"])
+        widget.result_truncated = bool(payload["truncated"])
+        widget.error_message = ""
+        widget.is_running = False
+
+        # Append to the session trace history (newest first).
+        _end_dt = datetime.now(timezone.utc)
+        try:
+            # Row count is the true number of rows in the query's result
+            # dataframe (not the truncated display payload).
+            try:
+                _row_count = int(len(new_result)) if new_result is not None else 0
+            except Exception:
+                _row_count = int(payload["total_rows"])
+            if role_name:
+                _imp_type = "Role"
+                _imp_value = str(role_name)
+            elif effective_user:
+                _imp_type = "User"
+                _imp_value = str(effective_user)
+            else:
+                _imp_type = "None"
+                _imp_value = "None"
+            _entry = {
+                "run_id": run_id,
+                "method": "Query",
+                "report_name": "",
+                "report_workspace_name": "",
+                "dax_query": query,
+                "start_time": _start_dt.strftime("%Y-%m-%d %H:%M:%S"),
+                "end_time": _end_dt.strftime("%Y-%m-%d %H:%M:%S"),
+                "rows": _row_count,
+                "duration": int(new_total),
+                "cpu": int(new_cpu),
+                "fe_duration": int(new_fe),
+                "se_duration": int(new_se),
+                "cache": "Cold" if clear_cache_flag else "Warm",
+                "execution_metrics": _execution_metrics_dict(metric_rows),
+                "dataset_name": str(widget.dataset_name or ""),
+                "workspace_name": str(widget.workspace_name or ""),
+                "impersonation_type": _imp_type,
+                "impersonation": _imp_value,
+            }
+            widget.trace_history = [_entry] + list(widget.trace_history)
+        except Exception:
+            pass
+
+        # If the DAX query plan or execution metrics were not captured inline
+        # (they can flush into the trace a few seconds after the query
+        # completes), keep watching the persistent trace and back-fill the
+        # Query Plan / Execution Metrics tabs once they arrive.
+        _missing_plan = not (widget.query_plan_rows or [])
+        _missing_metrics = not (widget.execution_metrics or [])
+        if start_baseline is not None and (_missing_plan or _missing_metrics):
+            try:
+                _backfill_query_plan(run_id, int(start_baseline), query)
+            except Exception:
+                pass
+
+    def _on_run(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.report_capture_loading:
+            widget.error_message = "Wait for report query capture to finish first."
+            widget.is_running = False
+            return
+        if model_ctx["dataset_id"] is None:
+            widget.error_message = (
+                "No semantic model selected. Choose a workspace and a "
+                "semantic model first."
+            )
+            widget.is_running = False
+            return
+        query = widget.dax_query or ""
+        if not query.strip():
+            widget.error_message = "DAX query is empty."
+            widget.is_running = False
+            return
+        mode = widget.impersonation_mode or "none"
+        imp_value = (widget.impersonation_value or "").strip()
+        effective_user = imp_value if mode == "user" else None
+        role_name = imp_value if mode == "role" else None
+        if mode in ("user", "role") and not imp_value:
+            label = "user" if mode == "user" else "role"
+            widget.error_message = (
+                f"Impersonation is set to '{label}' but no {label} value "
+                "was provided."
+            )
+            widget.is_running = False
+            return
+        with state_lock:
+            run_state["current_run_id"] += 1
+            run_id = run_state["current_run_id"]
+        thread = threading.Thread(
+            target=_worker,
+            args=(query, bool(widget.clear_cache), run_id, effective_user, role_name),
+            daemon=True,
+        )
+        with state_lock:
+            run_state["thread"] = thread
+        thread.start()
+
+    def _on_cancel(change):
+        if change["new"] == change["old"]:
+            return
+        with state_lock:
+            run_id = run_state["current_run_id"]
+            run_state["canceled_run_ids"].add(run_id)
+        widget.is_running = False
+        widget.error_message = (
+            "Query canceled. Note: the DAX engine may still finish the "
+            "query in the background; results have been discarded."
+        )
+
+    def _clear_model_cache() -> None:
+        try:
+            dataset_id = model_ctx["dataset_id"]
+            if dataset_id is None:
+                raise ValueError("No semantic model selected.")
+            from sempy_labs._clear_cache import clear_cache as _clear_cache_fn
+
+            _clear_cache_fn(
+                dataset=dataset_id,
+                workspace=model_ctx["workspace_id"],
+            )
+            widget.error_message = ""
+        except Exception as exc:
+            widget.error_message = f"Failed to clear the model cache: {exc}"
+        finally:
+            widget.cache_clear_loading = False
+
+    def _on_clear_model_cache(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.report_capture_loading:
+            widget.cache_clear_loading = False
+            widget.error_message = "Wait for report query capture to finish first."
+            return
+        threading.Thread(target=_clear_model_cache, daemon=True).start()
+
+    def _compute_dependencies() -> None:
+        """Compute the model objects (tables, columns, measures,
+        relationships) referenced by the current DAX query via
+        ``INFO.CALCDEPENDENCY`` and push the resulting hierarchical tree to the
+        front-end. The trace data for this helper query is not captured."""
+        try:
+            if model_ctx["dataset_id"] is None:
+                widget.dependency_tree = []
+                widget.dependency_columns = []
+                return
+            dax_query = widget.dax_query or ""
+            if not dax_query.strip():
+                widget.dependency_tree = []
+                widget.dependency_columns = []
+                return
+            # Escape double quotes for embedding as a DAX string literal.
+            escaped_dax = dax_query.replace('"', '""')
+            query = (
+                "EVALUATE\n"
+                "SELECTCOLUMNS(\n"
+                "    INFO.CALCDEPENDENCY(\n"
+                '        "Query",\n'
+                f'        "{escaped_dax}"\n'
+                "    ),\n"
+                '    "Referenced Object Type", [REFERENCED_OBJECT_TYPE],\n'
+                '    "Referenced Table", [REFERENCED_TABLE],\n'
+                '    "Referenced Object", [REFERENCED_OBJECT]\n'
+                ")"
+            )
+            dep_df = fabric.evaluate_dax(
+                dataset=model_ctx["dataset_id"],
+                dax_string=query,
+                workspace=model_ctx["workspace_id"],
+            )
+            rows = []
+            for _, r in dep_df.iterrows():
+                rows.append(
+                    {
+                        "object_type": str(r.get("[Referenced Object Type]", "") or ""),
+                        "table": str(r.get("[Referenced Table]", "") or ""),
+                        "object": str(r.get("[Referenced Object]", "") or ""),
+                    }
+                )
+            # Enrich relationships with the columns they join (read via TOM)
+            # only when the query actually depends on a relationship. Also read
+            # the model's RowNumber columns (also via TOM) so they can be
+            # excluded from the output whenever the query references columns.
+            rel_lookup: dict = {}
+            rel_columns: dict = {}
+            rownumber_cols: set = set()
+            needs_rel = any(
+                "RELATIONSHIP" in (row["object_type"] or "").upper() for row in rows
+            )
+            needs_cols = any(
+                "COLUMN" in (row["object_type"] or "").upper() for row in rows
+            )
+            if needs_rel or needs_cols:
+                try:
+                    from sempy_labs.tom import connect_semantic_model
+
+                    with connect_semantic_model(
+                        dataset=model_ctx["dataset_id"],
+                        workspace=model_ctx["workspace_id"],
+                        readonly=True,
+                    ) as tom:
+                        if needs_rel:
+                            rel_lookup = _build_relationship_lookup(tom)
+                            rel_columns = _build_relationship_columns(tom)
+                        rownumber_cols = _build_rownumber_columns(tom)
+                except Exception:
+                    rel_lookup = {}
+                    rel_columns = {}
+                    rownumber_cols = set()
+            widget.dependency_tree = _build_dependency_tree(
+                rows, rel_lookup, widget.dataset_name or "Model", rownumber_cols
+            )
+            widget.dependency_columns = _build_dependency_columns(
+                rows, rel_columns, rownumber_cols
+            )
+            with state_lock:
+                run_state["deps_query"] = dax_query
+            widget.error_message = ""
+        except Exception as exc:  # noqa: BLE001
+            widget.dependency_tree = []
+            widget.dependency_columns = []
+            widget.error_message = f"Failed to compute query dependencies: {exc}"
+        finally:
+            widget.dependencies_loading = False
+
+    def _on_dependencies(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.report_capture_loading:
+            widget.dependencies_loading = False
+            widget.error_message = "Wait for report query capture to finish first."
+            return
+        if widget.dependencies_loading:
+            return
+        widget.dependencies_loading = True
+        threading.Thread(target=_compute_dependencies, daemon=True).start()
+
+    def _compute_object_dependencies(request_id: int) -> None:
+        dataset_snapshot = model_ctx["dataset_id"]
+        workspace_snapshot = model_ctx["workspace_id"]
+
+        def _is_current_request() -> bool:
+            return (
+                str(model_ctx["dataset_id"]) == str(dataset_snapshot)
+                and str(model_ctx["workspace_id"]) == str(workspace_snapshot)
+                and widget.object_dependency_trigger == request_id
+            )
+
+        try:
+            if _is_current_request():
+                widget.object_dependencies_loading = True
+                widget.object_dependencies_loaded = False
+                widget.object_dependency_error = ""
+            if dataset_snapshot is None:
+                if _is_current_request():
+                    widget.object_dependency_edges = []
+                    widget.object_dependencies_loaded = True
+                return
+            dependency_df = fabric.evaluate_dax(
+                dataset=dataset_snapshot,
+                workspace=workspace_snapshot,
+                dax_string="""
+                SELECT
+                    [OBJECT_TYPE] AS [Object Type],
+                    [TABLE] AS [Table],
+                    [OBJECT] AS [Object],
+                    [REFERENCED_OBJECT_TYPE] AS [Referenced Object Type],
+                    [REFERENCED_TABLE] AS [Referenced Table],
+                    [REFERENCED_OBJECT] AS [Referenced Object]
+                FROM $SYSTEM.DISCOVER_CALC_DEPENDENCY
+                """,
+            )
+
+            def _value(row, name: str) -> str:
+                value = row.get(f"[{name}]", row.get(name, ""))
+                return "" if value is None or pd.isna(value) else str(value)
+
+            edges = [
+                {
+                    "object_type": _value(row, "Object Type"),
+                    "table": _value(row, "Table"),
+                    "object": _value(row, "Object"),
+                    "referenced_object_type": _value(
+                        row, "Referenced Object Type"
+                    ),
+                    "referenced_table": _value(row, "Referenced Table"),
+                    "referenced_object": _value(row, "Referenced Object"),
+                }
+                for _, row in dependency_df.iterrows()
+            ]
+            if _is_current_request():
+                widget.object_dependency_edges = edges
+                widget.object_dependencies_loaded = True
+                widget.object_dependency_error = ""
+        except Exception as exc:  # noqa: BLE001
+            if _is_current_request():
+                widget.object_dependency_edges = []
+                widget.object_dependencies_loaded = False
+                widget.object_dependency_error = (
+                    f"Failed to read object dependencies: {exc}"
+                )
+        finally:
+            if _is_current_request():
+                widget.object_dependencies_loading = False
+
+    def _on_object_dependencies(change):
+        if change["new"] == change["old"]:
+            return
+        request_id = int(change["new"])
+        threading.Thread(
+            target=_compute_object_dependencies,
+            args=(request_id,),
+            daemon=True,
+        ).start()
+
+    def _compute_vertipaq() -> None:
+        """Run the Vertipaq Analyzer against the active semantic model and push
+        its result tables (Model Summary, Tables, Partitions, Columns,
+        Relationships, Hierarchies) to the front-end. The full result dict is
+        also stored on ``widget.last_vertipaq`` for later programmatic use."""
+        try:
+            if model_ctx["dataset_id"] is None:
+                widget.vertipaq_sections = []
+                return
+            from sempy_labs.semantic_model._vertipaq_analyzer import (
+                vertipaq_analyzer,
+            )
+            from IPython.utils.capture import capture_output
+
+            # vertipaq_analyzer renders its own HTML visualization via
+            # display(); capture (and discard) it so it does not appear as a
+            # separate output below this widget. The returned dataframes are
+            # rendered inside the Vertipaq Analyzer tab instead.
+            with capture_output():
+                result = vertipaq_analyzer(
+                    dataset=model_ctx["dataset_id"],
+                    workspace=model_ctx["workspace_id"],
+                )
+            # Store the raw result for later programmatic access.
+            widget.last_vertipaq = result  # type: ignore[attr-defined]
+            sections = []
+            for name, sdf in (result or {}).items():
+                embedded_df = _prepare_embedded_vertipaq_dataframe(str(name), sdf)
+                payload = _result_payload_from_df(embedded_df)
+                sections.append(
+                    {
+                        "name": str(name),
+                        "columns": payload["columns"],
+                        "rows": payload["rows"],
+                    }
+                )
+            widget.vertipaq_sections = sections
+            if sections and not (widget.vertipaq_section or "").strip():
+                widget.vertipaq_section = sections[0]["name"]
+            widget.error_message = ""
+        except Exception as exc:  # noqa: BLE001
+            widget.vertipaq_sections = []
+            widget.error_message = f"Failed to run Vertipaq Analyzer: {exc}"
+        finally:
+            widget.vertipaq_loading = False
+
+    def _on_vertipaq(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.report_capture_loading:
+            widget.vertipaq_loading = False
+            widget.error_message = "Wait for report query capture to finish first."
+            return
+        if widget.vertipaq_loading:
+            return
+        widget.vertipaq_loading = True
+        threading.Thread(target=_compute_vertipaq, daemon=True).start()
+
+    def _ensure_trace_captured(query: str) -> None:
+        """Ensure the trace artifacts for ``query`` are populated before a
+        performance analysis runs.
+
+        If the trace rows, durations, DAX query plan and execution metrics were
+        already captured for the same query currently in the query pane, they
+        are reused as-is. Otherwise the query is executed once against the
+        persistent trace (synchronously), the trace traitlets are populated, and
+        the method waits briefly for the DAX query plan and execution metrics to
+        flush into the trace (they can arrive a few seconds after the query
+        completes)."""
+
+        if not (query or "").strip():
+            return
+        with state_lock:
+            already_traced = run_state.get("traced_query") == query
+        if already_traced and (widget.trace_rows or []):
+            return
+
+        # Derive impersonation from the current UI state (mirrors _on_run).
+        mode = widget.impersonation_mode or "none"
+        imp_value = (widget.impersonation_value or "").strip()
+        effective_user = imp_value if mode == "user" else None
+        role_name = imp_value if mode == "role" else None
+
+        _start_dt = datetime.now(timezone.utc)
+        (
+            new_df,
+            new_total,
+            new_fe,
+            new_se,
+            new_cpu,
+            new_result,
+            start_baseline,
+        ) = _run_query_persistent(
+            query=query,
+            clear_cache_flag=bool(widget.clear_cache),
+            effective_user=effective_user,
+            role_name=role_name,
+        )
+
+        widget.last_df = new_df  # type: ignore[attr-defined]
+        widget.last_result_df = new_result  # type: ignore[attr-defined]
+        widget.total_duration = int(new_total)
+        widget.fe_duration = int(new_fe)
+        widget.se_duration = int(new_se)
+        widget.cpu_time = int(new_cpu)
+        widget.trace_rows = _trace_rows_from_df(new_df)
+        widget.query_plan_rows = _query_plan_rows_from_df(new_df)
+        metric_rows = _execution_metrics_from_df(new_df)
+        widget.execution_metrics = metric_rows
+        payload = _result_payload_from_df(new_result)
+        widget.result_columns = payload["columns"]
+        widget.result_rows = payload["rows"]
+        widget.result_total_rows = int(payload["total_rows"])
+        widget.result_truncated = bool(payload["truncated"])
+        with state_lock:
+            run_state["traced_query"] = query
+
+        # Append a row to the session trace history (newest first), exactly as a
+        # normal query run does.
+        _end_dt = datetime.now(timezone.utc)
+        _history_id = f"analysis-{time.time_ns()}"
+        try:
+            try:
+                _row_count = int(len(new_result)) if new_result is not None else 0
+            except Exception:
+                _row_count = int(payload["total_rows"])
+            if role_name:
+                _imp_type = "Role"
+                _imp_value = str(role_name)
+            elif effective_user:
+                _imp_type = "User"
+                _imp_value = str(effective_user)
+            else:
+                _imp_type = "None"
+                _imp_value = "None"
+            _entry = {
+                "run_id": _history_id,
+                "method": "Query",
+                "report_name": "",
+                "report_workspace_name": "",
+                "dax_query": query,
+                "start_time": _start_dt.strftime("%Y-%m-%d %H:%M:%S"),
+                "end_time": _end_dt.strftime("%Y-%m-%d %H:%M:%S"),
+                "rows": _row_count,
+                "duration": int(new_total),
+                "cpu": int(new_cpu),
+                "fe_duration": int(new_fe),
+                "se_duration": int(new_se),
+                "cache": "Cold" if bool(widget.clear_cache) else "Warm",
+                "execution_metrics": _execution_metrics_dict(metric_rows),
+                "dataset_name": str(widget.dataset_name or ""),
+                "workspace_name": str(widget.workspace_name or ""),
+                "impersonation_type": _imp_type,
+                "impersonation": _imp_value,
+            }
+            widget.trace_history = [_entry] + list(widget.trace_history)
+        except Exception:
+            pass
+
+        # Wait (briefly, synchronously) for a late-arriving DAX query plan and
+        # execution metrics so the analysis has the complete picture.
+        _need_plan = not (widget.query_plan_rows or [])
+        _need_metrics = not (widget.execution_metrics or [])
+        if start_baseline is not None and (_need_plan or _need_metrics):
+            _deadline = time.monotonic() + 25.0
+            while time.monotonic() < _deadline and (_need_plan or _need_metrics):
+                time.sleep(0.3)
+                with trace_lock:
+                    trace = trace_ctx["trace"] if trace_ctx["started"] else None
+                if trace is None:
+                    break
+                try:
+                    logs = _get_trace_logs(trace)
+                except Exception:
+                    continue
+                if logs is None or logs.empty or len(logs) <= int(start_baseline):
+                    continue
+                _new = logs.iloc[int(start_baseline):]
+                _req_id = _resolve_query_request_id(_new, query)
+                if _req_id is not None:
+                    _new = _filter_logs_to_request_id(_new, _req_id)
+                if _need_plan:
+                    plan_rows = _query_plan_rows_from_df(_new)
+                    if plan_rows:
+                        widget.query_plan_rows = plan_rows
+                        _need_plan = False
+                if _need_metrics:
+                    metric_rows = _execution_metrics_from_df(_new)
+                    if metric_rows:
+                        widget.execution_metrics = metric_rows
+                        _update_history_execution_metrics(_history_id, metric_rows)
+                        _need_metrics = False
+
+    def _compute_performance() -> None:
+        """Generate a DAX performance analysis for the current query and push
+        the resulting findings (and a summary) to the front-end.
+
+        The analysis combines the DAX query, the semantic model metadata
+        (``model_tree``), the query dependencies, the captured trace details,
+        the DAX query plan and -- when the Data column cardinalities are not
+        all trivial -- Vertipaq Analyzer statistics. The trace (and its query
+        plan / execution metrics), query dependencies and Vertipaq stats are
+        captured on demand when they have not already been produced for the
+        query currently in the query pane, and reused otherwise."""
+        try:
+            if model_ctx["dataset_id"] is None:
+                widget.performance_findings = []
+                widget.performance_summary = {}
+                return
+            from sempy_labs.semantic_model._dax_optimization import (
+                analyze_dax_performance,
+            )
+
+            query = widget.dax_query or ""
+
+            # Ensure the trace (rows, durations, DAX query plan and execution
+            # metrics) is captured for the current query. Reuses what was
+            # already captured for this query in the query pane, otherwise runs
+            # the query against the trace now.
+            try:
+                _ensure_trace_captured(query)
+            except Exception:  # noqa: BLE001
+                pass
+
+            # Reuse query dependencies when they were already computed for this
+            # exact query in the query pane; only (re)compute them when they are
+            # missing or were captured for a different query.
+            with state_lock:
+                deps_fresh = run_state.get("deps_query") == query
+            dep_cols = list(widget.dependency_columns or [])
+            if not deps_fresh and query.strip():
+                try:
+                    _compute_dependencies()
+                    dep_cols = list(widget.dependency_columns or [])
+                except Exception:  # noqa: BLE001
+                    dep_cols = []
+
+            # Reuse Vertipaq Analyzer results if the tab has been opened,
+            # otherwise compute them now (capturing the analyzer's own HTML).
+            # Vertipaq stats describe the model, not the query, so they are
+            # always safe to reuse once computed.
+            vertipaq = getattr(widget, "last_vertipaq", None) or {}
+            if not vertipaq:
+                try:
+                    from sempy_labs.semantic_model._vertipaq_analyzer import (
+                        vertipaq_analyzer,
+                    )
+                    from IPython.utils.capture import capture_output
+
+                    with capture_output():
+                        vertipaq = vertipaq_analyzer(
+                            dataset=model_ctx["dataset_id"],
+                            workspace=model_ctx["workspace_id"],
+                        )
+                    widget.last_vertipaq = vertipaq  # type: ignore[attr-defined]
+                except Exception:  # noqa: BLE001
+                    vertipaq = {}
+
+            result = analyze_dax_performance(
+                dax_query=widget.dax_query or "",
+                trace_rows=list(widget.trace_rows or []),
+                total_duration_ms=int(widget.total_duration or 0),
+                fe_duration_ms=int(widget.fe_duration or 0),
+                se_duration_ms=int(widget.se_duration or 0),
+                cpu_time_ms=int(widget.cpu_time or 0),
+                query_plan_rows=list(widget.query_plan_rows or []),
+                dependency_columns=dep_cols,
+                model_tree=list(widget.model_tree or []),
+                vertipaq=vertipaq,
+                cold_cache=bool(widget.clear_cache),
+            )
+            widget.performance_findings = result["findings"]
+            widget.performance_summary = result["summary"]
+            widget.error_message = ""
+        except Exception as exc:  # noqa: BLE001
+            widget.performance_findings = []
+            widget.performance_summary = {}
+            widget.error_message = f"Failed to generate performance analysis: {exc}"
+        finally:
+            widget.performance_loading = False
+
+    def _on_performance(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.report_capture_loading:
+            widget.performance_loading = False
+            widget.error_message = "Wait for report query capture to finish first."
+            return
+        threading.Thread(target=_compute_performance, daemon=True).start()
+
+    def _load_workspace_monitoring() -> None:
+        allowed_ranges = {"15m", "1h", "4h", "12h", "1d", "3d", "7d", "30d"}
+        request = dict(widget.workspace_monitoring_request or {})
+        time_range = str(request.get("range") or "1d")
+        if time_range not in allowed_ranges:
+            time_range = "1d"
+        try:
+            top_n = min(200, max(1, int(request.get("top") or 20)))
+        except (TypeError, ValueError):
+            top_n = 20
+        dataset = str(widget.dataset_name or "")
+        workspace = model_ctx["workspace_id"]
+        safe_dataset = dataset.replace("\\", "\\\\").replace('"', '\\"')
+        query = (
+            "SemanticModelLogs\n"
+            '| where OperationName == "QueryEnd" and '
+            '(EventText startswith "EVALUATE" or EventText startswith "DEFINE")\n'
+            f'| where ItemName == "{safe_dataset}"\n'
+            f"| where Timestamp >= ago({time_range})\n"
+            "| extend ctx = parse_json(dynamic_to_json(ApplicationContext))\n"
+            "| extend ReportId = tostring(ctx.Sources[0].ReportId)\n"
+            "| extend VisualId = tostring(ctx.Sources[0].VisualId)\n"
+            "| project Timestamp, DurationMs, CpuTimeMs, ExecutingUser, "
+            "ReportId, VisualId, EventText\n"
+            f"| top {top_n} by DurationMs desc"
+        )
+        widget.workspace_monitoring_loading = True
+        widget.workspace_monitoring_error = ""
+        try:
+            if not dataset or not workspace:
+                raise ValueError("Choose a semantic model first.")
+            from sempy_labs import query_workspace_monitoring
+
+            monitoring_df = query_workspace_monitoring(
+                query=query,
+                workspace=workspace,
+            )
+            if (
+                dataset != str(widget.dataset_name or "")
+                or workspace != model_ctx["workspace_id"]
+            ):
+                return
+            if "ReportId" in monitoring_df.columns:
+                report_lookup = {
+                    str(report.get("id") or "").lower(): str(
+                        report.get("name") or ""
+                    )
+                    for report in (widget.available_reports or [])
+                }
+                monitoring_df = monitoring_df.copy()
+                monitoring_df["ReportName"] = monitoring_df["ReportId"].map(
+                    lambda value: report_lookup.get(str(value or "").lower(), "")
+                )
+                monitoring_df["ReportWorkspace"] = monitoring_df[
+                    "ReportName"
+                ].map(lambda value: str(widget.workspace_name or "") if value else "")
+            payload = _result_payload_from_df(monitoring_df, max_rows=top_n)
+            query_index = next(
+                (
+                    index
+                    for index, column in enumerate(payload["columns"])
+                    if column.lower() == "eventtext"
+                ),
+                -1,
+            )
+            monitoring_tokens = [
+                _monitoring_dax_spans(str(row[query_index] or ""))
+                if query_index >= 0
+                else []
+                for row in payload["rows"]
+            ]
+            widget.workspace_monitoring_columns = payload["columns"]
+            widget.workspace_monitoring_tokens = monitoring_tokens
+            widget.workspace_monitoring_rows = payload["rows"]
+            widget.workspace_monitoring_enabled = True
+            widget.workspace_monitoring_loaded = True
+        except Exception as exc:  # noqa: BLE001
+            message = str(exc)
+            if "Monitoring KQL database" in message:
+                widget.workspace_monitoring_columns = []
+                widget.workspace_monitoring_tokens = []
+                widget.workspace_monitoring_rows = []
+                widget.workspace_monitoring_enabled = False
+                widget.workspace_monitoring_loaded = True
+            else:
+                widget.workspace_monitoring_error = (
+                    f"Failed to read workspace monitoring: {message}"
+                )
+        finally:
+            widget.workspace_monitoring_loading = False
+
+    def _on_workspace_monitoring(change):
+        if change["new"] == change["old"] or widget.workspace_monitoring_loading:
+            return
+        threading.Thread(target=_load_workspace_monitoring, daemon=True).start()
+
+    widget.observe(_on_run, names="run_trigger")
+    widget.observe(_on_cancel, names="cancel_trigger")
+    widget.observe(_on_clear_model_cache, names="cache_clear_trigger")
+    widget.observe(_on_dependencies, names="dependencies_trigger")
+    widget.observe(_on_object_dependencies, names="object_dependency_trigger")
+    widget.observe(_on_vertipaq, names="vertipaq_trigger")
+    widget.observe(_on_performance, names="performance_trigger")
+    widget.observe(_on_workspace_monitoring, names="workspace_monitoring_trigger")
+    widget.observe(_on_report_capture_start, names="report_capture_start_trigger")
+    widget.observe(
+        _on_report_capture_checkpoint, names="report_capture_checkpoint_trigger"
+    )
+    widget.observe(_on_report_capture_finish, names="report_capture_finish_trigger")
+
+    def _build_history_excel() -> None:
+        import base64
+        import io
+
+        history = list(widget.trace_history)
+        columns = [
+            "Run",
+            "Total",
+            "FE",
+            "SE",
+            "CPU",
+            "Cache",
+            "Execution metrics",
+            "Method",
+            "Query",
+            "Report",
+            "Workspace",
+        ]
+        rows = [
+            {
+                "Run": entry.get("start_time", ""),
+                "Total": entry.get("duration", ""),
+                "FE": entry.get("fe_duration", ""),
+                "SE": entry.get("se_duration", ""),
+                "CPU": entry.get("cpu", ""),
+                "Cache": entry.get("cache", ""),
+                "Execution metrics": json.dumps(
+                    entry.get("execution_metrics") or {}, indent=2
+                ),
+                "Method": entry.get("method", "Query"),
+                "Query": entry.get("dax_query", ""),
+                "Report": entry.get("report_name", ""),
+                "Workspace": entry.get("report_workspace_name", ""),
+            }
+            for entry in history
+        ]
+        df_hist = pd.DataFrame(rows, columns=columns)
+        buf = io.BytesIO()
+        # Use whichever Excel engine is available (openpyxl is standard in
+        # Fabric notebooks; xlsxwriter is an accepted fallback).
+        engine = None
+        for _eng in ("openpyxl", "xlsxwriter"):
+            try:
+                __import__(_eng)
+                engine = _eng
+                break
+            except Exception:
+                continue
+        if engine is None:
+            widget.error_message = (
+                "Could not export to Excel: no Excel engine is installed. "
+                "Install 'openpyxl' (pip install openpyxl) and try again."
+            )
+            return
+        try:
+            with pd.ExcelWriter(buf, engine=engine) as writer:
+                df_hist.to_excel(writer, index=False, sheet_name="Trace History")
+        except Exception as exc:  # noqa: BLE001
+            widget.error_message = f"Failed to build the Excel file: {exc}"
+            return
+        b64 = base64.b64encode(buf.getvalue()).decode("ascii")
+        stamp = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
+        # Reset first so the front-end always observes a change event.
+        widget.history_excel_name = f"trace_history_{stamp}.xlsx"
+        widget.history_excel_b64 = ""
+        widget.history_excel_b64 = b64
+
+    def _on_download_history(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_build_history_excel, daemon=True).start()
+
+    widget.observe(_on_download_history, names="download_history_trigger")
+
+    def _build_result_excel() -> None:
+        import base64
+        import io
+
+        df_result = getattr(widget, "last_result_df", None)
+        if df_result is None or len(df_result) == 0:
+            widget.error_message = "There is no query result to download."
+            return
+        buf = io.BytesIO()
+        # Use whichever Excel engine is available (openpyxl is standard in
+        # Fabric notebooks; xlsxwriter is an accepted fallback).
+        engine = None
+        for _eng in ("openpyxl", "xlsxwriter"):
+            try:
+                __import__(_eng)
+                engine = _eng
+                break
+            except Exception:
+                continue
+        if engine is None:
+            widget.error_message = (
+                "Could not export to Excel: no Excel engine is installed. "
+                "Install 'openpyxl' (pip install openpyxl) and try again."
+            )
+            return
+        try:
+            with pd.ExcelWriter(buf, engine=engine) as writer:
+                df_result.to_excel(writer, index=False, sheet_name="Query Result")
+        except Exception as exc:  # noqa: BLE001
+            widget.error_message = f"Failed to build the Excel file: {exc}"
+            return
+        b64 = base64.b64encode(buf.getvalue()).decode("ascii")
+        stamp = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
+        # Reset first so the front-end always observes a change event.
+        widget.result_excel_name = f"query_result_{stamp}.xlsx"
+        widget.result_excel_b64 = ""
+        widget.result_excel_b64 = b64
+
+    def _on_download_result(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_build_result_excel, daemon=True).start()
+
+    widget.observe(_on_download_result, names="download_result_trigger")
+
+    def _on_query_change(change):
+        # Re-classify on every edit so the syntax-highlight overlay stays
+        # in sync. The DAX tokenizer is cheap relative to comm latency.
+        try:
+            widget.dax_tokens = _classify_dax_spans(change["new"] or "")
+        except Exception:
+            pass
+
+    widget.observe(_on_query_change, names="dax_query")
+
+    def _load_metadata() -> None:
+        if model_ctx["dataset_id"] is None:
+            widget.metadata_loading = False
+            return
+        try:
+            tree, roles = _collect_model_metadata(
+                model_ctx["dataset_id"], model_ctx["workspace_id"]
+            )
+        except Exception as exc:  # noqa: BLE001
+            widget.metadata_loading = False
+            widget.error_message = f"Failed to load model metadata: {exc}"
+            return
+        widget.model_tree = tree
+        widget.model_roles = roles
+        try:
+            widget.available_reports = _list_reports_for_capture(
+                model_ctx["dataset_id"], model_ctx["workspace_id"]
+            )
+        except Exception:
+            widget.available_reports = []
+        widget.metadata_loading = False
+        # Start (or keep) the long-running trace for this model now that its
+        # metadata has loaded.
+        _ensure_trace(model_ctx["dataset_id"], model_ctx["workspace_id"])
+
+    def _on_refresh_metadata(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.metadata_loading:
+            return
+        widget.metadata_loading = True
+        threading.Thread(target=_load_metadata, daemon=True).start()
+
+    widget.observe(_on_refresh_metadata, names="refresh_metadata_trigger")
+
+    def _load_datasets_for_selected_workspace() -> None:
+        ws_id = (widget.selected_workspace_id or "").strip()
+        if not ws_id:
+            widget.available_datasets = []
+            widget.picker_loading = False
+            return
+        try:
+            datasets = _list_datasets_for_picker(ws_id)
+        except Exception as exc:  # noqa: BLE001
+            widget.available_datasets = []
+            widget.picker_loading = False
+            widget.error_message = f"Failed to list semantic models: {exc}"
+            return
+        widget.available_datasets = datasets
+        widget.picker_loading = False
+
+    def _on_select_workspace(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.picker_loading:
+            return
+        widget.picker_loading = True
+        threading.Thread(
+            target=_load_datasets_for_selected_workspace, daemon=True
+        ).start()
+
+    widget.observe(_on_select_workspace, names="select_workspace_trigger")
+
+    def _activate_selected_dataset() -> None:
+        if widget.report_capture_loading:
+            widget.metadata_loading = False
+            widget.picker_loading = False
+            widget.error_message = "Wait for report query capture to finish first."
+            return
+        ws_id = (widget.selected_workspace_id or "").strip()
+        ds_id = (widget.selected_dataset_id or "").strip()
+        if not ws_id or not ds_id:
+            widget.metadata_loading = False
+            widget.picker_loading = False
+            return
+        try:
+            from sempy_labs._helper_functions import (
+                resolve_workspace_name_and_id as _rwni,
+            )
+
+            ws_name, ws_id_resolved = _rwni(ws_id)
+            ds_name, ds_id_resolved = resolve_item_name_and_id(
+                item=ds_id, type="SemanticModel", workspace=ws_id_resolved
+            )
+            model_ctx["workspace_id"] = ws_id_resolved
+            model_ctx["dataset_id"] = ds_id_resolved
+            tree, roles = _collect_model_metadata(ds_id_resolved, ws_id_resolved)
+        except Exception as exc:  # noqa: BLE001
+            widget.error_message = f"Failed to load semantic model: {exc}"
+            widget.metadata_loading = False
+            widget.picker_loading = False
+            return
+        widget.dataset_name = str(ds_name) if ds_name else str(ds_id)
+        widget.workspace_name = str(ws_name) if ws_name else ""
+        widget.active_workspace_id = str(ws_id_resolved)
+        widget.active_dataset_id = str(ds_id_resolved)
+        widget.model_tree = tree
+        widget.model_roles = roles
+        try:
+            widget.available_reports = _list_reports_for_capture(
+                ds_id_resolved, ws_id_resolved
+            )
+        except Exception:
+            widget.available_reports = []
+        widget.metadata_loading = False
+        # Clear Vertipaq Analyzer results from any previously selected model so
+        # stale stats aren't shown; they are recomputed on the next tab open.
+        widget.vertipaq_sections = []
+        widget.vertipaq_section = ""
+        widget.last_vertipaq = {}  # type: ignore[attr-defined]
+        # Clear any performance analysis produced for the previous model.
+        widget.performance_findings = []
+        widget.performance_summary = {}
+        widget.object_dependency_target = {}
+        widget.object_dependencies_loading = False
+        widget.object_dependencies_loaded = False
+        widget.object_dependency_edges = []
+        widget.object_dependency_error = ""
+        widget.workspace_monitoring_loading = False
+        widget.workspace_monitoring_loaded = False
+        widget.workspace_monitoring_enabled = True
+        widget.workspace_monitoring_error = ""
+        widget.workspace_monitoring_columns = []
+        widget.workspace_monitoring_rows = []
+        widget.workspace_monitoring_tokens = []
+        widget.query_executed = False
+        # Reset impersonation so a stale role/user from a prior model isn't
+        # reused against a model that may not define it.
+        widget.impersonation_mode = "none"
+        widget.impersonation_value = ""
+        widget.dataset_chosen = True
+        widget.error_message = ""
+        widget.picker_loading = False
+        # Rebind the long-running trace to the newly selected model.
+        _ensure_trace(ds_id_resolved, ws_id_resolved)
+
+    def _on_select_dataset(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.picker_loading:
+            return
+        widget.picker_loading = True
+        threading.Thread(target=_activate_selected_dataset, daemon=True).start()
+
+    widget.observe(_on_select_dataset, names="select_dataset_trigger")
+
+    def _load_workspaces() -> None:
+        try:
+            workspaces = _list_workspaces_for_picker()
+        except Exception as exc:  # noqa: BLE001
+            widget.picker_loading = False
+            widget.error_message = f"Failed to list workspaces: {exc}"
+            return
+        widget.available_workspaces = workspaces
+        # Refresh the dataset list for the currently selected workspace too.
+        ws_id = (widget.selected_workspace_id or "").strip()
+        if ws_id:
+            try:
+                widget.available_datasets = _list_datasets_for_picker(ws_id)
+            except Exception:
+                pass
+        widget.picker_loading = False
+
+    def _on_load_workspaces(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.picker_loading:
+            return
+        widget.picker_loading = True
+        threading.Thread(target=_load_workspaces, daemon=True).start()
+
+    widget.observe(_on_load_workspaces, names="load_workspaces_trigger")
+
+    def _format_query() -> None:
+        dax = widget.dax_query or ""
+        if not dax.strip():
+            widget.format_loading = False
+            return
+        try:
+            formatted = _format_dax(dax)
+            dax_out = formatted[0] if formatted else dax
+        except Exception as exc:  # noqa: BLE001
+            widget.format_loading = False
+            widget.error_message = f"Failed to format the DAX query: {exc}"
+            return
+        dax_out = dax_out.replace("\r\n", "\n").replace("\r", "\n")
+        widget.dax_query = dax_out
+        widget.dax_tokens = _classify_dax_spans(dax_out)
+        widget.error_message = ""
+        widget.format_loading = False
+
+    def _on_format_query(change):
+        if change["new"] == change["old"]:
+            return
+        if widget.format_loading:
+            return
+        widget.format_loading = True
+        threading.Thread(target=_format_query, daemon=True).start()
+
+    widget.observe(_on_format_query, names="format_query_trigger")
+
+    def _nl_to_dax_run() -> None:
+        try:
+            if model_ctx["dataset_id"] is None:
+                widget.nl_to_dax_error = (
+                    "No semantic model selected. Choose a workspace and a "
+                    "semantic model first."
+                )
+                return
+            question = (widget.nl_to_dax_text or "").strip()
+            if not question:
+                widget.nl_to_dax_error = "Enter a question first."
+                return
+            import asyncio
+            from sempy_labs.semantic_model._nl_to_dax import nl_to_dax
+
+            result = asyncio.run(
+                nl_to_dax(
+                    dataset=model_ctx["dataset_id"],
+                    question=question,
+                    workspace=model_ctx["workspace_id"],
+                )
+            )
+            if isinstance(result, str) and result.strip():
+                dax_out = result.replace("\r\n", "\n").replace("\r", "\n")
+                widget.dax_query = dax_out
+                widget.dax_tokens = _classify_dax_spans(dax_out)
+                widget.error_message = ""
+                widget.nl_to_dax_error = ""
+            elif isinstance(result, dict):
+                err = result.get("error")
+                if isinstance(err, dict):
+                    msg = err.get("message") or str(err)
+                else:
+                    msg = (
+                        err
+                        or result.get("message")
+                        or "Failed to generate a DAX query from the question."
+                    )
+                widget.nl_to_dax_error = str(msg)
+            else:
+                widget.nl_to_dax_error = (
+                    "Could not generate a DAX query from the question. "
+                    "Try rephrasing it."
+                )
+        except Exception as exc:  # noqa: BLE001
+            widget.nl_to_dax_error = f"Failed to generate the DAX query: {exc}"
+        finally:
+            widget.nl_to_dax_loading = False
+
+    def _on_nl_to_dax(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_nl_to_dax_run, daemon=True).start()
+
+    widget.observe(_on_nl_to_dax, names="nl_to_dax_trigger")
+
+    def _build_query() -> None:
+        if model_ctx["dataset_id"] is None:
+            widget.error_message = (
+                "No semantic model selected. Choose a workspace and a "
+                "semantic model first."
+            )
+            return
+        import json as _json
+
+        try:
+            state = _json.loads(widget.query_builder_state or "{}")
+        except Exception:
+            widget.error_message = "Could not read the query builder state."
+            return
+        fields = state.get("fields") or []
+        if not fields:
+            widget.error_message = (
+                "Add at least one column or measure to the query builder "
+                "before building a query."
+            )
+            return
+        try:
+            dax = _build_summarize_dax(
+                state, model_ctx["dataset_id"], model_ctx["workspace_id"]
+            )
+        except Exception as exc:  # noqa: BLE001
+            widget.error_message = f"Failed to build a DAX query: {exc}"
+            return
+        if not dax:
+            widget.error_message = (
+                "Could not build a DAX query from the current selection."
+            )
+            return
+        try:
+            formatted = _format_dax(dax)
+            dax_out = formatted[0] if formatted else dax
+        except Exception:
+            dax_out = dax
+        dax_out = dax_out.replace("\r\n", "\n").replace("\r", "\n")
+        widget.dax_query = dax_out
+        widget.dax_tokens = _classify_dax_spans(dax_out)
+        widget.error_message = ""
+
+    def _on_build_query(change):
+        if change["new"] == change["old"]:
+            return
+        threading.Thread(target=_build_query, daemon=True).start()
+
+    widget.observe(_on_build_query, names="build_query_trigger")
+
+    def _on_close_trigger(change):
+        if change["new"] == change["old"]:
+            return
+        _stop_persistent_trace()
+
+    widget.observe(_on_close_trigger, names="close_trigger")
+
+    # Start the long-running trace for the initially selected model (if any)
+    # so it is ready by the time the first query runs.
+    if model_ctx["dataset_id"] is not None:
+        threading.Thread(
+            target=lambda: _ensure_trace(
+                model_ctx["dataset_id"], model_ctx["workspace_id"]
+            ),
+            daemon=True,
+        ).start()
+
+    display(widget)
+
+    # Backstop for trace teardown if the comm is closed without the JS
+    # cleanup hook firing (e.g. kernel/cell disposal).
+    try:
+        widget.comm.on_close(lambda *_a: _stop_persistent_trace())
+    except Exception:
+        pass
diff --git a/src/sempy_labs/semantic_model/_direct_lake_manager.py b/src/sempy_labs/semantic_model/_direct_lake_manager.py
index 7d5fbb75b..36ee1307c 100644
--- a/src/sempy_labs/semantic_model/_direct_lake_manager.py
+++ b/src/sempy_labs/semantic_model/_direct_lake_manager.py
@@ -412,6 +412,17 @@
     renderThemeBtn();
     header.appendChild(themeBtn);
 
+    // Full-screen toggle button (expands the manager to fill the screen).
+    const FULLSCREEN_SVG = `__SLLS_ICON_FULLSCREEN__`;
+    const FULLSCREEN_EXIT_SVG = `__SLLS_ICON_FULLSCREEN_EXIT__`;
+    const fullscreenBtn = document.createElement("button");
+    fullscreenBtn.className = "slls-dle-btn slls-dle-btn-icon";
+    fullscreenBtn.type = "button";
+    header.appendChild(fullscreenBtn);
+    sllsSetupFullscreen(
+        root, fullscreenBtn, "slls-dle-fullscreen", FULLSCREEN_SVG, FULLSCREEN_EXIT_SVG
+    );
+
     // ----------- Status banner (shared) -----------
     const status = document.createElement("div");
     status.className = "slls-dle-status";
@@ -3217,6 +3228,28 @@
 """
 
 
+# Inject the shared full-screen helper + CSS so the manager can expand to fill
+# the screen, staying in sync with the other Semantic Link Labs widgets. The
+# ``.slls-dle`` root carries the card styling itself, so no inner container
+# selector is needed.
+from sempy_labs._ui_components import (  # noqa: E402
+    ICONS as _UI_ICONS,
+    fullscreen_css as _ui_fullscreen_css,
+    fullscreen_setup_js as _ui_fullscreen_setup_js,
+)
+
+_WIDGET_JS = _ui_fullscreen_setup_js() + _WIDGET_JS.replace(
+    "__SLLS_ICON_FULLSCREEN__", _UI_ICONS["fullscreen"]
+).replace("__SLLS_ICON_FULLSCREEN_EXIT__", _UI_ICONS["fullscreen_exit"])
+_WIDGET_CSS = (
+    _WIDGET_CSS
+    + "\n"
+    + _ui_fullscreen_css(
+        ".slls-dle", "slls-dle-fullscreen", bg_var="var(--slls-bg-solid)"
+    )
+)
+
+
 def _build_tables_payload(tom):
     """Return a list of dicts describing Direct Lake tables and their partitions."""
     import Microsoft.AnalysisServices.Tabular as TOM
@@ -3427,9 +3460,7 @@ def _list_source_tables_payload(
             if source_type == "Lakehouse":
                 from sempy_labs.lakehouse import get_lakehouse_tables
 
-                dfT = get_lakehouse_tables(
-                    lakehouse=source_id, workspace=workspace_id
-                )
+                dfT = get_lakehouse_tables(lakehouse=source_id, workspace=workspace_id)
                 items = []
                 for _, r in dfT.iterrows():
                     schema = str(r.get("Schema Name") or "")
@@ -3454,9 +3485,7 @@ def _list_source_tables_payload(
                         if not table:
                             continue
                         items.append({"schema": schema, "table": table})
-            items.sort(
-                key=lambda x: ((x["schema"] or "").lower(), x["table"].lower())
-            )
+            items.sort(key=lambda x: ((x["schema"] or "").lower(), x["table"].lower()))
             return {"items": items}
         except Exception as e:
             return {"error": str(e)}
@@ -3691,7 +3720,12 @@ def _on_run(_change):
                         continue
                     key = f"{ws_id}::{src_type}::{src_id}::{schema}::{table}"
                     new_map[key] = _list_source_columns_payload(
-                        ws_id, src_type, src_id, schema, table, use_sql,
+                        ws_id,
+                        src_type,
+                        src_id,
+                        schema,
+                        table,
+                        use_sql,
                     )
                 # Single assignment triggers one sync to the frontend, which
                 # then refreshes every visible column picker.
@@ -3807,12 +3841,14 @@ def _on_run(_change):
                                 if c.Type == TOM.ColumnType.RowNumber:
                                     continue
                                 src_col = getattr(c, "SourceColumn", "") or ""
-                                if src_col and src_col not in wanted and c.Name not in wanted:
+                                if (
+                                    src_col
+                                    and src_col not in wanted
+                                    and c.Name not in wanted
+                                ):
                                     t.Columns.Remove(c.Name)
                     if refresh_after:
-                        refresh_semantic_model(
-                            dataset=ds_id_resolved, workspace=ws_id
-                        )
+                        refresh_semantic_model(dataset=ds_id_resolved, workspace=ws_id)
                 widget.workspace_id = str(ws_id)
                 widget.workspace_name = _resolve_ws_name(ws_id)
                 widget.dataset_id = str(ds_id_resolved)
@@ -3849,6 +3885,7 @@ def _on_run(_change):
                 changes = data.get("changes") or []
                 if not changes:
                     return
+
                 # Defer table renames to the end so other staged changes
                 # that reference the original table name resolve correctly.
                 # Run add_source first so subsequent reassign_table /
@@ -3862,6 +3899,7 @@ def _change_order(ch):
                     if k == "rename_table":
                         return 3
                     return 2
+
                 changes = sorted(changes, key=_change_order)
                 summary = []
                 with connect_semantic_model(
@@ -3981,7 +4019,9 @@ def _change_order(ch):
                                 # can override the entity name.
                                 if isinstance(spec, dict):
                                     raw_spec = (spec.get("spec") or "").strip()
-                                    custom_name = (spec.get("name") or "").strip() or None
+                                    custom_name = (
+                                        spec.get("name") or ""
+                                    ).strip() or None
                                 else:
                                     raw_spec = str(spec).strip()
                                     custom_name = None
@@ -4038,9 +4078,7 @@ def _change_order(ch):
                             table_name = p.get("table_name")
                             cols = p.get("columns") or []
                             if not table_name:
-                                raise ValueError(
-                                    "Table is required to edit columns."
-                                )
+                                raise ValueError("Table is required to edit columns.")
                             if not cols:
                                 continue
                             renames = []
@@ -4080,8 +4118,7 @@ def _change_order(ch):
                                         f"'{table_name}': {exc}"
                                     )
                             summary.append(
-                                f"updated {len(cols)} column(s) in "
-                                f"'{table_name}'"
+                                f"updated {len(cols)} column(s) in " f"'{table_name}'"
                             )
                         elif kind == "rename_table":
                             old_name = change.get("key") or p.get("table_name")
@@ -4093,12 +4130,9 @@ def _change_order(ch):
                                 )
                             if old_name == new_name:
                                 continue
-                            if old_name not in [
-                                t.Name for t in tom.model.Tables
-                            ]:
+                            if old_name not in [t.Name for t in tom.model.Tables]:
                                 raise ValueError(
-                                    f"Table '{old_name}' not found "
-                                    "in the model."
+                                    f"Table '{old_name}' not found " "in the model."
                                 )
                             try:
                                 tom.model.Tables[old_name].Name = new_name
@@ -4119,15 +4153,10 @@ def _change_order(ch):
                             add_cols = p.get("add") or []
                             remove_cols = p.get("remove") or []
                             if not table_name:
+                                raise ValueError("Table is required to sync columns.")
+                            if table_name not in [t.Name for t in tom.model.Tables]:
                                 raise ValueError(
-                                    "Table is required to sync columns."
-                                )
-                            if table_name not in [
-                                t.Name for t in tom.model.Tables
-                            ]:
-                                raise ValueError(
-                                    f"Table '{table_name}' not found "
-                                    "in the model."
+                                    f"Table '{table_name}' not found " "in the model."
                                 )
                             added = 0
                             removed = 0
@@ -4142,10 +4171,7 @@ def _change_order(ch):
                                 if dtype == "Binary":
                                     continue
                                 existing = {
-                                    c.Name
-                                    for c in tom.model.Tables[
-                                        table_name
-                                    ].Columns
+                                    c.Name for c in tom.model.Tables[table_name].Columns
                                 }
                                 target_name = col_name
                                 suffix = 1
@@ -4161,9 +4187,9 @@ def _change_order(ch):
                                 added += 1
                             for col_name in remove_cols:
                                 try:
-                                    tom.model.Tables[
-                                        table_name
-                                    ].Columns.Remove(col_name)
+                                    tom.model.Tables[table_name].Columns.Remove(
+                                        col_name
+                                    )
                                     removed += 1
                                 except Exception:
                                     continue
diff --git a/src/sempy_labs/semantic_model/_generate.py b/src/sempy_labs/semantic_model/_generate.py
index bd981b912..9a72b37bc 100644
--- a/src/sempy_labs/semantic_model/_generate.py
+++ b/src/sempy_labs/semantic_model/_generate.py
@@ -101,7 +101,7 @@ def generate_direct_lake_semantic_model(
     if source is None:
         raise ValueError(f"{icons.red_dot} The 'source' parameter must be provided.")
 
-    (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace)
+    workspace_name, workspace_id = resolve_workspace_name_and_id(workspace)
     source_workspace_id = resolve_workspace_id(source_workspace)
     source_id = resolve_item_id(
         item=source, type=source_type, workspace=source_workspace_id
diff --git a/src/sempy_labs/semantic_model/_infer_relationships.py b/src/sempy_labs/semantic_model/_infer_relationships.py
index df0aa13e3..81978c8b2 100644
--- a/src/sempy_labs/semantic_model/_infer_relationships.py
+++ b/src/sempy_labs/semantic_model/_infer_relationships.py
@@ -34,11 +34,9 @@ def is_unique(source, schema, table, column, workspace):
             with ConnectMirroredAzureDatabricksCatalog(
                 mirrored_azure_databricks_catalog=source, workspace=workspace
             ) as sql:
-                df = sql.query(
-                    f"""SELECT COUNT(DISTINCT {column}) AS ct_col,
+                df = sql.query(f"""SELECT COUNT(DISTINCT {column}) AS ct_col,
                             COUNT({column}) AS ct_tbl
-                        FROM {schema}.{table}"""
-                )
+                        FROM {schema}.{table}""")
             ct_col, ct_tbl = df.iloc[0]
             cardinality_cache[key] = ct_col == ct_tbl
 
diff --git a/src/sempy_labs/semantic_model/_nl_to_dax.py b/src/sempy_labs/semantic_model/_nl_to_dax.py
new file mode 100644
index 000000000..070ef0fee
--- /dev/null
+++ b/src/sempy_labs/semantic_model/_nl_to_dax.py
@@ -0,0 +1,119 @@
+import httpx
+import json
+from typing import Optional
+from uuid import UUID
+from sempy_labs._helper_functions import (
+    resolve_workspace_id,
+    resolve_item_id,
+)
+
+MCP_SERVER_URL = "https://msitapi.fabric.microsoft.com/v1/mcp/powerbi"
+
+
+async def nl_to_dax(
+    dataset: str | UUID, question: str, workspace: Optional[str | UUID] = None
+):
+    """
+    Complete workflow to ask a question against Power BI
+    """
+    import notebookutils
+
+    token = notebookutils.credentials.getToken("pbi")
+
+    workspace_id = resolve_workspace_id(workspace)
+    item_id = resolve_item_id(
+        item=dataset, type="SemanticModel", workspace=workspace_id
+    )
+
+    headers = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
+
+    async with httpx.AsyncClient(timeout=120.0) as client:
+
+        # Step 1: Get the semantic model schema
+        schema_payload = {
+            "jsonrpc": "2.0",
+            "id": 1,
+            "method": "tools/call",
+            "params": {
+                "name": "GetSemanticModelSchema",
+                "arguments": {"artifactId": item_id},
+            },
+        }
+
+        response = await client.post(
+            MCP_SERVER_URL, headers=headers, json=schema_payload
+        )
+        schema_data = parse_sse_response(response.text)
+        # return schema_data
+
+        if "error" in schema_data:
+            print(f"Error getting schema: {schema_data['error']}")
+            return schema_data
+
+        # Extract structured schema
+        structured_schema = json.loads(
+            schema_data.get("result", {}).get("content", {})[0]["text"]
+        )["schema"]
+        # return structured_schema
+        tables = structured_schema.get("Tables", [])
+
+        # Step 2: Generate DAX query
+        # Auto-select relevant tables based on question keywords
+        schema_selection = build_schema_selection(tables)
+
+        generate_payload = {
+            "jsonrpc": "2.0",
+            "id": 2,
+            "method": "tools/call",
+            "params": {
+                "name": "GenerateQuery",
+                "arguments": {
+                    "artifactId": item_id,
+                    "userInput": question,
+                    "schemaSelection": schema_selection,
+                },
+            },
+        }
+
+        response = await client.post(
+            MCP_SERVER_URL, headers=headers, json=generate_payload
+        )
+        query_data = parse_sse_response(response.text)
+
+        if "error" in query_data:
+            print(f"Error generating query: {query_data['error']}")
+            return query_data
+
+        result = query_data.get("result", {})
+        content = result.get("content", [])
+        dax_query = (
+            json.loads(content[0].get("text", {})).get("daxQuery")
+            if len(content) > 0
+            else None
+        )
+        return dax_query
+
+
+def build_schema_selection(tables):
+    """Build schema selection based on available tables"""
+    # For simplicity, include main tables - you can make this smarter
+    schema_tables = []
+
+    for table in tables:
+        table_selection = {
+            "name": table["Name"],
+            "columns": [col["Name"] for col in table.get("Columns", [])],
+            "measures": [m["Name"] for m in table.get("Measures", [])],
+        }
+        schema_tables.append(table_selection)
+
+    return {"tables": schema_tables}
+
+
+def parse_sse_response(text):
+    """Parse Server-Sent Events response"""
+    lines = text.split("\n")
+    for line in lines:
+        if line.startswith("data: "):
+            return json.loads(line[6:])
+    return {}
diff --git a/src/sempy_labs/semantic_model/_perspective_editor.py b/src/sempy_labs/semantic_model/_perspective_editor.py
index 28507d4ed..eb80bccb4 100644
--- a/src/sempy_labs/semantic_model/_perspective_editor.py
+++ b/src/sempy_labs/semantic_model/_perspective_editor.py
@@ -2,7 +2,6 @@
 from uuid import UUID
 from sempy._utils._log import log
 
-
 _WIDGET_CSS = """
 .slls-pe {
     --slls-bg-solid: #ffffff;
@@ -488,26 +487,6 @@
             (ws ? escapeHtml(ws) : "");
     }
 
-    // Theme toggle button (light/dark)
-    const SUN_SVG = `__SLLS_ICON_SUN__`;
-    const MOON_SVG = `__SLLS_ICON_MOON__`;
-    const themeBtn = document.createElement("button");
-    themeBtn.className = "slls-pe-btn slls-pe-btn-icon";
-    themeBtn.type = "button";
-    function renderThemeBtn() {
-        const isDark = model.get("dark_mode") === true;
-        themeBtn.innerHTML = isDark ? SUN_SVG : MOON_SVG;
-        themeBtn.title = isDark ? "Switch to light mode" : "Switch to dark mode";
-        themeBtn.setAttribute("aria-label", themeBtn.title);
-    }
-    themeBtn.addEventListener("click", () => {
-        model.set("dark_mode", !(model.get("dark_mode") === true));
-        model.save_changes();
-    });
-    model.on("change:dark_mode", renderThemeBtn);
-    renderThemeBtn();
-    header.appendChild(themeBtn);
-
     const select = document.createElement("select");
     select.className = "slls-pe-select";
     header.appendChild(select);
@@ -541,6 +520,43 @@
     createRow.appendChild(cancelBtn);
     header.appendChild(createRow);
 
+    // Theme toggle + full-screen buttons live at the far right of the header
+    // (to the right of the "new perspective" button). They are appended last
+    // so that, because the header cluster is right-aligned, they stay pinned
+    // to the right edge — including while the "create perspective" row is
+    // shown (which grows toward the left).
+
+    // Theme toggle button (light/dark)
+    const SUN_SVG = `__SLLS_ICON_SUN__`;
+    const MOON_SVG = `__SLLS_ICON_MOON__`;
+    const themeBtn = document.createElement("button");
+    themeBtn.className = "slls-pe-btn slls-pe-btn-icon";
+    themeBtn.type = "button";
+    function renderThemeBtn() {
+        const isDark = model.get("dark_mode") === true;
+        themeBtn.innerHTML = isDark ? SUN_SVG : MOON_SVG;
+        themeBtn.title = isDark ? "Switch to light mode" : "Switch to dark mode";
+        themeBtn.setAttribute("aria-label", themeBtn.title);
+    }
+    themeBtn.addEventListener("click", () => {
+        model.set("dark_mode", !(model.get("dark_mode") === true));
+        model.save_changes();
+    });
+    model.on("change:dark_mode", renderThemeBtn);
+    renderThemeBtn();
+    header.appendChild(themeBtn);
+
+    // Full-screen toggle button (expands the editor to fill the screen).
+    const FULLSCREEN_SVG = `__SLLS_ICON_FULLSCREEN__`;
+    const FULLSCREEN_EXIT_SVG = `__SLLS_ICON_FULLSCREEN_EXIT__`;
+    const fullscreenBtn = document.createElement("button");
+    fullscreenBtn.className = "slls-pe-btn slls-pe-btn-icon";
+    fullscreenBtn.type = "button";
+    header.appendChild(fullscreenBtn);
+    sllsSetupFullscreen(
+        root, fullscreenBtn, "slls-pe-fullscreen", FULLSCREEN_SVG, FULLSCREEN_EXIT_SVG
+    );
+
     // ----------- Toolbar -----------
     const toolbar = document.createElement("div");
     toolbar.className = "slls-pe-toolbar";
@@ -1087,6 +1103,10 @@
 # Inject SVG icons from the shared UI components module so they stay in
 # sync with other widgets (e.g. ``vertipaq_analyzer``).
 from sempy_labs._ui_components import ICONS as _UI_ICONS  # noqa: E402
+from sempy_labs._ui_components import (  # noqa: E402
+    fullscreen_css as _ui_fullscreen_css,
+    fullscreen_setup_js as _ui_fullscreen_setup_js,
+)
 
 _WIDGET_JS = (
     _WIDGET_JS.replace("__SLLS_ICON_COLUMN__", _UI_ICONS["column"])
@@ -1097,6 +1117,20 @@
     .replace("__SLLS_ICON_SUN__", _UI_ICONS["sun"])
     .replace("__SLLS_ICON_MOON__", _UI_ICONS["moon"])
     .replace("__SLLS_ICON_PLUS__", _UI_ICONS["plus"])
+    .replace("__SLLS_ICON_FULLSCREEN__", _UI_ICONS["fullscreen"])
+    .replace("__SLLS_ICON_FULLSCREEN_EXIT__", _UI_ICONS["fullscreen_exit"])
+)
+
+# Prepend the shared full-screen helper so ``render`` can call it, and append
+# the full-screen CSS (the ``.slls-pe`` root carries the card styling itself,
+# so no inner container selector is needed).
+_WIDGET_JS = _ui_fullscreen_setup_js() + _WIDGET_JS
+_WIDGET_CSS = (
+    _WIDGET_CSS
+    + "\n"
+    + _ui_fullscreen_css(
+        ".slls-pe", "slls-pe-fullscreen", bg_var="var(--slls-bg-solid)"
+    )
 )
 
 
diff --git a/src/sempy_labs/semantic_model/_vertipaq_analyzer.py b/src/sempy_labs/semantic_model/_vertipaq_analyzer.py
index 6369abe3f..efcdda6dd 100644
--- a/src/sempy_labs/semantic_model/_vertipaq_analyzer.py
+++ b/src/sempy_labs/semantic_model/_vertipaq_analyzer.py
@@ -1,6 +1,5 @@
 import sempy.fabric as fabric
 import pandas as pd
-from IPython.display import display, HTML
 import zipfile
 import os
 import uuid
@@ -32,6 +31,9 @@
     render_header_html as _ui_render_header_html,
     render_attribution_html as _ui_render_attribution_html,
     theme_toggle_script as _ui_theme_toggle_script,
+    fullscreen_css as _ui_fullscreen_css,
+    fullscreen_toggle_script as _ui_fullscreen_toggle_script,
+    display_html_widget as _ui_display_html_widget,
 )
 
 
@@ -214,8 +216,8 @@ def vertipaq_analyzer(
 
     from sempy_labs.tom import connect_semantic_model
 
-    (workspace_name, workspace_id) = resolve_workspace_name_and_id(workspace)
-    (dataset_name, dataset_id) = resolve_dataset_name_and_id(dataset, workspace_id)
+    workspace_name, workspace_id = resolve_workspace_name_and_id(workspace)
+    dataset_name, dataset_id = resolve_dataset_name_and_id(dataset, workspace_id)
     save_prefix = "vertipaqanalyzer_"
     save_table_name = f"{save_prefix}model"
 
@@ -1056,9 +1058,7 @@ def create_dfs(column_formatting: str = "format"):
             df_datasets["Dataset Id"] == dataset_id, "Configured By"
         ].iloc[0]
 
-        (capacity_id, capacity_name) = resolve_workspace_capacity(
-            workspace=workspace_id
-        )
+        capacity_id, capacity_name = resolve_workspace_capacity(workspace=workspace_id)
 
         base_metadata = {
             "Capacity Name": capacity_name,
@@ -1191,6 +1191,8 @@ def visualize_vertipaq(
     uid = uuid.uuid4().hex[:8]
     root_selector = f".vpx-{uid}"
     theme_btn_id = f"vpx-theme-{uid}"
+    fullscreen_btn_id = f"vpx-fullscreen-{uid}"
+    fullscreen_class = "vpx-fullscreen"
     # Scope the shared header CSS under the root selector so its rules win
     # against notebook host styles (e.g. Jupyter's ``.jp-RenderedHTMLCommon
     # button`` rules that would otherwise override the theme toggle
@@ -1199,6 +1201,12 @@ def visualize_vertipaq(
     # are NOT subject to f-string escaping and don't need doubling.
     ui_header_css_scoped = _ui_scoped_header_css(root_selector)
     ui_attribution_css_scoped = _ui_scoped_attribution_css(root_selector)
+    ui_fullscreen_css = _ui_fullscreen_css(
+        root_selector,
+        fullscreen_class,
+        container_selector=".vpx-container",
+        bg_var="var(--vpx-bg)",
+    )
 
     # ── CSS ──────────────────────────────────────────────────────────────
     # Light theme is the default; the ``.vpx-dark`` modifier on the root
@@ -1208,6 +1216,7 @@ def visualize_vertipaq(
     styles = f"""