refactor: support multiple markdown flavors in marimo export md#9586
refactor: support multiple markdown flavors in marimo export md#9586peter-gy wants to merge 20 commits into
marimo export md#9586Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
4 issues found across 21 files
Architecture diagram
sequenceDiagram
participant CLI as CLI (export md)
participant API as Export API
participant Pyodide as Pyodide Session
participant Exporter as Exporter (export_as_md)
participant Converter as MarimoConverter
participant FlavorReg as Flavor Registry
participant PymdownF as PymdownFlavor
participant QmdF as QmdFlavor
participant MystF as MystFlavor
participant ToIR as Markdown → IR Converter
participant FromIR as Markdown Export Builder
participant App as InternalApp / Notebook IR
Note over CLI,Pyodide: Entry points for markdown export
CLI->>CLI: Parse --flavor (pymdown/qmd/mystmd)
CLI->>Exporter: export_as_md(path, flavor)
Exporter->>Converter: from_ir(ir).to_markdown(flavor)
API->>Exporter: export_as_md(path, flavor)
Exporter->>Converter: from_ir(ir).to_markdown(flavor)
Pyodide->>Pyodide: export_markdown(request with flavor)
Pyodide->>FromIR: convert_from_ir_to_markdown(ir, flavor)
alt PyMdown flavor
Converter->>FlavorReg: normalize_markdown_flavor("pymdown")
FlavorReg-->>Converter: PymdownMarkdownFlavor
Converter->>FromIR: _notebook_to_markdown_export_document(ir)
FromIR->>App: Iterate cells
App-->>FromIR: Code + metadata
FromIR->>FromIR: Build MarkdownExportDocument with MarkdownCellBlock, CodeCellBlock
FromIR-->>Converter: MarkdownExportDocument
Converter->>PymdownF: render_document(document)
PymdownF->>PymdownF: render_preamble (YAML frontmatter)
PymdownF->>PymdownF: transform_blocks (passthrough)
PymdownF->>PymdownF: render_code_cell (fenced code with .marimo class)
PymdownF->>PymdownF: render_directive (/// blocks unchanged)
PymdownF->>PymdownF: render_tab_set (consecutive /// tab blocks)
PymdownF-->>Converter: Complete markdown string
else Qmd flavor
Converter->>FlavorReg: normalize_markdown_flavor("qmd")
FlavorReg-->>Converter: QmdMarkdownFlavor
Converter->>FromIR: _notebook_to_markdown_export_document(ir)
FromIR->>App: Iterate cells
App-->>FromIR: Code + metadata
FromIR->>FromIR: Build MarkdownExportDocument (no filters injection)
FromIR-->>Converter: MarkdownExportDocument
Converter->>QmdF: render_document(document)
QmdF->>QmdF: render_preamble (YAML frontmatter, no marimo filter)
QmdF->>QmdF: transform_blocks (split pymdown blocks, group tabs)
QmdF->>QmdF: render_code_cell (```{marimo .python} fence)
QmdF->>QmdF: render_directive
alt Known admonition → callout
QmdF->>QmdF: :: {.callout-tip title="..."}
else Unrecognized → pandoc div
QmdF->>QmdF: :: {.details title="..." #id .class}
end
QmdF->>QmdF: render_tab_set (:: {.panel-tabset})
QmdF-->>Converter: Complete markdown string
else Myst flavor
Converter->>FlavorReg: normalize_markdown_flavor("mystmd")
FlavorReg-->>Converter: MystMarkdownFlavor
Converter->>FromIR: _notebook_to_markdown_export_document(ir)
FromIR->>App: Iterate cells
App-->>FromIR: Code + metadata
FromIR->>FromIR: Build MarkdownExportDocument with header/pyproject metadata
FromIR-->>Converter: MarkdownExportDocument
Converter->>MystF: render_document(document)
MystF->>MystF: render_preamble
MystF->>MystF: YAML frontmatter (title, etc.)
MystF->>MystF: {marimo-config} block (header, pyproject)
MystF->>MystF: transform_blocks (split pymdown blocks, group tabs)
MystF->>MystF: render_code_cell (```{marimo} python with :options:)
MystF->>MystF: render_directive (:::{tip} / :::{dropdown})
MystF->>MystF: render_tab_set (::::{tab-set})
MystF-->>Converter: Complete markdown string
end
Converter-->>Exporter: Markdown string
Exporter-->>CLI/API: ExportResult with contents
Note over ToIR,App: Import path (reverse direction)
ToIR->>ToIR: convert_from_md_to_marimo_ir(markdown)
ToIR->>App: Parse frontmatter, header
ToIR->>ToIR: MystMarimoPreprocessor normalizes {marimo} directives
ToIR->>ToIR: Extract options from :hide-code: style lines
ToIR->>App: Build InternalApp with cell configurations
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
dmadisetti
left a comment
There was a problem hiding this comment.
I had a couple reservations initially, but I think it's a good abstraction.
I think my major concern is handling admonitions differently. Ideally we don't want to change user code- so if anything I think they should be less of a parse concern and more of a render one.
There was a problem hiding this comment.
1 issue found across 9 files (changes from recent commits).
Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.
Re-trigger cubic
5866953 to
5a79966
Compare
|
@dmadisetti as per our conversation I reduced the scope of this PR to carry only the flavor-based refactoring and the introduction of |
There was a problem hiding this comment.
1 issue found across 10 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
bf4cece to
f9c54e2
Compare
There was a problem hiding this comment.
1 issue found across 11 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
7f579b0 to
fc0a8bc
Compare
|
You're iterating quickly on this pull request. To help protect your rate limits, cubic has paused automatic reviews on new pushes for now—when you're ready for another review, comment |
marimo export mdmarimo export md
📝 Summary
Refactors the internals of
marimo export mdso markdown export can target multiple markdown flavors through a shared renderer interface. Supports https://github.com/marimo-team/quarto-marimo/ and https://github.com/marimo-team/jupyter-book-marimo.Prior to this, markdown export only needed to distinguish between marimo's default markdown format (pymdown) and Quarto, so an
is_qmdbranch in the exporter was simple enough to handle markdown output differences. As we add MyST /Jupyter Book support, that does not scale well as each target has its own way to represent executable cells, page-level metadata, callouts, options, and tabsets.
This PR introduces explicit markdown flavors for:
enhancementlabel on this PR): MyST markdown, including the{marimo} / {marimo-config}surface used by marimo-team/jupyter-book-marimo#1The source authoring surface is still assumed to be marimo markdown / pymdown. The flavor layer maps from that source representation into each target format.
Details
This refactor sets up marimo core to support markdown flavor-specific rendering of callouts, tabs, etc. It allows us to first build a small intermediate export document instead of directly concatenating strings and if-else branching on detected markdown flavor.
As part of a follow-up, flavor implementations can then decide how to render:
{marimo-config}for mystmdFor Quarto, PyMdown admonitions are mapped to Quarto callouts and tab groups are mapped to panel tabsets.
The QMD exporter no longer injects
filters: marimo-team/marimo, since the Quarto marimo engine extension now auto-detects marimo code blocks.For MyST, marimo cells are emitted as
{marimo}directives and page-level execution config is emitted as{marimo-config}, because executable plugins do not currently get a clean frontmatter access path. The spelling remainsunparsablethroughout; no support forunparseablespelling to stay coherent and to ensure we do not express the same thing in multiple ways in the codebase.Flavor Differences
PyMdown
https://facelessuser.github.io/pymdown-extensions/
/// Tip | Data sources panel Click the database "barrel" icon in the left toolbar to see all dataframes and in-memory tables that your notebook has access to. ///QMD
https://quarto.org/docs/authoring/markdown-basics.html
::: {.callout-tip title="Data sources panel"} Click the database "barrel" icon in the left toolbar to see all dataframes and in-memory tables that your notebook has access to. :::MyStmd
https://mystmd.org/
:::{tip} Data sources panel Click the database "barrel" icon in the left toolbar to see all dataframes and in-memory tables that your notebook has access to. :::