diff --git a/development_docs/adding_lint_rules.md b/development_docs/adding_lint_rules.md index 6e49c29b7da..de50b605fca 100644 --- a/development_docs/adding_lint_rules.md +++ b/development_docs/adding_lint_rules.md @@ -4,11 +4,12 @@ This guide explains how to add new lint rules to marimo's linting system. ## Overview -marimo's lint system helps users write better, more reliable notebooks by detecting various issues that could prevent notebooks from running correctly. The system is organized around three severity levels: +marimo's lint system helps users write better, more reliable notebooks by detecting various issues that could prevent notebooks from running correctly. The system is organized around four severity levels: - **Breaking (MB)**: Errors that prevent notebook execution - **Runtime (MR)**: Issues that may cause runtime problems - **Formatting (MF)**: Style and formatting issues +- **WASM (MW)**: Compatibility issues for WASM/Pyodide notebooks *(off by default)* ## Rule Code Assignment @@ -19,6 +20,7 @@ Rule codes follow a specific pattern: `M[severity][number]` - **MB001-MB099**: Breaking rules - **MR001-MR099**: Runtime rules - **MF001-MF099**: Formatting rules +- **MW001-MW099**: WASM compatibility rules *(off by default)* ### Assigning New Codes @@ -45,6 +47,7 @@ Create your rule in the appropriate directory: - Breaking rules: `marimo/_lint/rules/breaking/` - Runtime rules: `marimo/_lint/rules/runtime/` - Formatting rules: `marimo/_lint/rules/formatting/` +- WASM rules: `marimo/_lint/rules/wasm/` **Template for a new rule**: @@ -315,6 +318,24 @@ the `--unsafe-fixes` flag. To do this, implement an `async def apply_unsafe_fixes(self, notebook, diagnostics) -> Notebook` method in your rule class, and inherit from `UnsafeFixRule` instead of `LintRule`. +## Default vs Opt-in Rules + +Rules in `RULE_CODES` are the full set of all rules. Rules in `DEFAULT_RULE_CODES` are those enabled when no `--select` is specified. + +To make a category **off by default** (like WASM rules), include it in `RULE_CODES` but exclude it from `DEFAULT_RULE_CODES` in `marimo/_lint/rules/__init__.py`: + +```python +# Rules enabled by default (excludes opt-in categories like WASM). +DEFAULT_RULE_CODES: dict[str, type[LintRule]] = ( + BREAKING_RULE_CODES | RUNTIME_RULE_CODES | FORMATTING_RULE_CODES +) + +# All known rules (including opt-in). Used when --select is provided. +RULE_CODES: dict[str, type[LintRule]] = DEFAULT_RULE_CODES | WASM_RULE_CODES +``` + +Users opt in via `marimo check --select MW` or `--select ALL`. The `resolve_rules()` function in `rule_selector.py` uses `DEFAULT_RULE_CODES` when no `select` is specified, and `RULE_CODES` when it is. + ## Common Patterns ### Checking All Cells diff --git a/docs/guides/lint_rules/index.md b/docs/guides/lint_rules/index.md index a914611a818..96c48f9460a 100644 --- a/docs/guides/lint_rules/index.md +++ b/docs/guides/lint_rules/index.md @@ -22,7 +22,7 @@ marimo check --fix . ## Rule Categories -marimo's lint rules are organized into three main categories based on their severity: +marimo's lint rules are organized into categories based on their severity: ### 🚨 Breaking Rules @@ -60,6 +60,16 @@ These are style and formatting issues. | [MF006](rules/misc_log_capture.md) | misc-log-capture | Miscellaneous log messages during processing | ❌ | | [MF007](rules/markdown_indentation.md) | markdown-indentation | Markdown cells in `mo.md()` should be properly indented. | 🛠️ | +### 🌐 WASM Rules + +These issues affect WASM/Pyodide compatibility (off by default). + +| Code | Name | Description | Fixable | +|------|------|-------------|----------| +| [MW001](rules/incompatible_import.md) | incompatible-import | Importing a module unavailable in WASM/Pyodide | ❌ | +| [MW002](rules/unsafe_system_call.md) | unsafe-system-call | System call that fails in WASM/Pyodide | ❌ | +| [MW003](rules/incompatible_package.md) | incompatible-package | Package with native extensions not available in Pyodide | ❌ | + ## Legend - 🛠️ = Automatically fixable with `marimo check --fix` diff --git a/docs/guides/lint_rules/rules/incompatible_import.md b/docs/guides/lint_rules/rules/incompatible_import.md new file mode 100644 index 00000000000..0e4eb28e22b --- /dev/null +++ b/docs/guides/lint_rules/rules/incompatible_import.md @@ -0,0 +1,39 @@ +# MW001: incompatible-import + +🌐 **WASM** ❌ Not Fixable + +MW001: Importing modules unavailable in WASM/Pyodide. + +## What it does + +Checks each cell's imports against a blocklist of stdlib modules that +either don't exist in Pyodide or are stubs that fail at runtime. + +## Why is this bad? + +WASM notebooks run in the browser via Pyodide, which cannot support +modules that depend on OS-level process control, terminal I/O, or +native GUI toolkits. Importing these modules will raise ImportError +or produce broken stubs. + +## Examples + +**Problematic:** +```python +import subprocess + +result = subprocess.run(["ls"]) +``` + +**Problematic:** +```python +from multiprocessing import Pool +``` + +**Solution:** +Remove or replace the import with a WASM-compatible alternative. + +## References + +- https://pyodide.org/en/stable/usage/wasm-constraints.html + diff --git a/docs/guides/lint_rules/rules/incompatible_package.md b/docs/guides/lint_rules/rules/incompatible_package.md new file mode 100644 index 00000000000..3270e0cd4d8 --- /dev/null +++ b/docs/guides/lint_rules/rules/incompatible_package.md @@ -0,0 +1,40 @@ +# MW003: incompatible-package + +🌐 **WASM** ❌ Not Fixable + +MW003: Packages in the dependency tree incompatible with WASM. + +## What it does + +Reads the notebook's PEP 723 ``dependencies``, walks their transitive +dependency tree via installed metadata, then queries PyPI's JSON API +to check whether each package has a ``py3-none-any`` or emscripten +wheel available. Packages only in pyodide-lock.json are also accepted. + +## Why is this bad? + +Pyodide can only install pure-Python wheels via micropip, or packages +that are pre-built in the Pyodide distribution. Packages with only +platform-specific native wheels will fail to install in the browser. + +## Examples + +**Problematic:** +```python +import jax # jaxlib (transitive dep) has only native wheels +``` + +**Not flagged:** +```python +import numpy # Native, but pre-built in Pyodide +``` + +**Not flagged:** +```python +import requests # Pure Python wheel on PyPI +``` + +## References + +- https://pyodide.org/en/stable/usage/packages-in-pyodide.html + diff --git a/docs/guides/lint_rules/rules/unsafe_system_call.md b/docs/guides/lint_rules/rules/unsafe_system_call.md new file mode 100644 index 00000000000..aa45467baa6 --- /dev/null +++ b/docs/guides/lint_rules/rules/unsafe_system_call.md @@ -0,0 +1,40 @@ +# MW002: unsafe-system-call + +🌐 **WASM** ❌ Not Fixable + +MW002: System calls that fail in WASM/Pyodide. + +## What it does + +Walks the AST of each cell looking for calls to functions like +``os.system()``, ``os.fork()``, ``signal.signal()``, and +``breakpoint()`` that have no meaningful implementation in WASM. + +## Why is this bad? + +These functions depend on OS features (process spawning, signal +handling, debugger attachment) that don't exist in a browser +environment. They will raise ``OSError``, ``NotImplementedError``, +or hang silently. + +## Examples + +**Problematic:** +```python +import os + +os.system("ls") +``` + +**Problematic:** +```python +breakpoint() +``` + +**Solution:** +Remove or guard these calls behind a WASM detection check. + +## References + +- https://pyodide.org/en/stable/usage/wasm-constraints.html + diff --git a/marimo/_cli/export/commands.py b/marimo/_cli/export/commands.py index 598bc6ecc65..8e3a2b01cd5 100644 --- a/marimo/_cli/export/commands.py +++ b/marimo/_cli/export/commands.py @@ -931,6 +931,16 @@ def html_wasm( if execute: cli_args = parse_args(args) + # Run WASM compatibility lint pass. When bootstrapped, this runs + # inside the uv sandbox so MW003 introspects the resolved env. + from marimo._lint import run_check + + run_check( + (name,), + lint_config={"select": ["MW"]}, + pipe=lambda msg: echo(msg, err=True), + ) + def export_callback(file_path: MarimoPath) -> ExportResult: return asyncio_run( run_app_then_export_as_wasm( diff --git a/marimo/_lint/__init__.py b/marimo/_lint/__init__.py index a9f88a4fdc2..2e885c80b15 100644 --- a/marimo/_lint/__init__.py +++ b/marimo/_lint/__init__.py @@ -20,6 +20,7 @@ Severity.BREAKING: 0, Severity.RUNTIME: 1, Severity.FORMATTING: 2, + Severity.WASM: 3, } diff --git a/marimo/_lint/context.py b/marimo/_lint/context.py index 783a3d2fef7..8c2c553fc5f 100644 --- a/marimo/_lint/context.py +++ b/marimo/_lint/context.py @@ -24,6 +24,7 @@ Severity.BREAKING: 0, Severity.RUNTIME: 1, Severity.FORMATTING: 2, + Severity.WASM: 3, } diff --git a/marimo/_lint/diagnostic.py b/marimo/_lint/diagnostic.py index 448e1e54ed5..61dc099ab57 100644 --- a/marimo/_lint/diagnostic.py +++ b/marimo/_lint/diagnostic.py @@ -14,6 +14,7 @@ class Severity(Enum): FORMATTING = "formatting" # prefix: MF0000 RUNTIME = "runtime" # prefix: MR0000 BREAKING = "breaking" # prefix: MB0000 + WASM = "wasm" # prefix: MW0000 def line_num(line: int) -> str: diff --git a/marimo/_lint/rule_engine.py b/marimo/_lint/rule_engine.py index 9660240672a..a9e2e111c36 100644 --- a/marimo/_lint/rule_engine.py +++ b/marimo/_lint/rule_engine.py @@ -6,7 +6,7 @@ from marimo._lint.context import LintContext, RuleContext from marimo._lint.diagnostic import Severity -from marimo._lint.rules import RULE_CODES +from marimo._lint.rules import DEFAULT_RULE_CODES from marimo._schemas.serialization import NotebookSerialization if TYPE_CHECKING: @@ -154,5 +154,5 @@ def create_default( rules = resolve_rules(lint_config) else: - rules = [rule() for rule in RULE_CODES.values()] + rules = [rule() for rule in DEFAULT_RULE_CODES.values()] return cls(rules, early_stopping) diff --git a/marimo/_lint/rule_selector.py b/marimo/_lint/rule_selector.py index b32ae12ce43..13368bd828a 100644 --- a/marimo/_lint/rule_selector.py +++ b/marimo/_lint/rule_selector.py @@ -45,13 +45,16 @@ def resolve_rules( all_rules = RULE_CODES - codes = set(all_rules.keys()) select = config.get("select") ignore = config.get("ignore") - # Step 1: base set + # Step 1: base set — use only default rules unless explicitly selected if select: - codes = {c for c in codes if _matches_any_prefix(c, select)} + codes = {c for c in all_rules if _matches_any_prefix(c, select)} + else: + from marimo._lint.rules import DEFAULT_RULE_CODES + + codes = set(DEFAULT_RULE_CODES.keys()) & set(all_rules.keys()) # Step 2: remove ignored if ignore: diff --git a/marimo/_lint/rules/__init__.py b/marimo/_lint/rules/__init__.py index d94e70a1188..840294f1946 100644 --- a/marimo/_lint/rules/__init__.py +++ b/marimo/_lint/rules/__init__.py @@ -5,14 +5,21 @@ from marimo._lint.rules.breaking import BREAKING_RULE_CODES from marimo._lint.rules.formatting import FORMATTING_RULE_CODES from marimo._lint.rules.runtime import RUNTIME_RULE_CODES +from marimo._lint.rules.wasm import WASM_RULE_CODES -RULE_CODES: dict[str, type[LintRule]] = ( +# Rules enabled by default (excludes opt-in categories like WASM). +DEFAULT_RULE_CODES: dict[str, type[LintRule]] = ( BREAKING_RULE_CODES | RUNTIME_RULE_CODES | FORMATTING_RULE_CODES ) +# All known rules (including opt-in). Used when --select is provided. +RULE_CODES: dict[str, type[LintRule]] = DEFAULT_RULE_CODES | WASM_RULE_CODES + __all__ = [ "BREAKING_RULE_CODES", + "DEFAULT_RULE_CODES", "FORMATTING_RULE_CODES", "RULE_CODES", "RUNTIME_RULE_CODES", + "WASM_RULE_CODES", ] diff --git a/marimo/_lint/rules/wasm/__init__.py b/marimo/_lint/rules/wasm/__init__.py new file mode 100644 index 00000000000..12667b0a354 --- /dev/null +++ b/marimo/_lint/rules/wasm/__init__.py @@ -0,0 +1,24 @@ +# Copyright 2026 Marimo. All rights reserved. +from __future__ import annotations + +from marimo._lint.rules.base import LintRule +from marimo._lint.rules.wasm.incompatible_imports import ( + IncompatibleImportsRule, +) +from marimo._lint.rules.wasm.incompatible_packages import ( + IncompatiblePackagesRule, +) +from marimo._lint.rules.wasm.unsafe_system_calls import UnsafeSystemCallsRule + +WASM_RULE_CODES: dict[str, type[LintRule]] = { + "MW001": IncompatibleImportsRule, + "MW002": UnsafeSystemCallsRule, + "MW003": IncompatiblePackagesRule, +} + +__all__ = [ + "WASM_RULE_CODES", + "IncompatibleImportsRule", + "IncompatiblePackagesRule", + "UnsafeSystemCallsRule", +] diff --git a/marimo/_lint/rules/wasm/incompatible_imports.py b/marimo/_lint/rules/wasm/incompatible_imports.py new file mode 100644 index 00000000000..be2beaa5b7a --- /dev/null +++ b/marimo/_lint/rules/wasm/incompatible_imports.py @@ -0,0 +1,102 @@ +# Copyright 2026 Marimo. All rights reserved. +from __future__ import annotations + +from typing import TYPE_CHECKING + +from marimo._lint.diagnostic import Diagnostic, Severity +from marimo._lint.rules.breaking.graph import GraphRule + +if TYPE_CHECKING: + from marimo._lint.context import RuleContext + from marimo._runtime.dataflow import DirectedGraph + +# Stdlib modules that don't exist or are non-functional stubs in Pyodide. +INCOMPATIBLE_MODULES = frozenset( + { + "subprocess", + "multiprocessing", + "pdb", + "dbm", + "resource", + "fcntl", + "termios", + "readline", + "curses", + "tkinter", + } +) + + +class IncompatibleImportsRule(GraphRule): + """MW001: Importing modules unavailable in WASM/Pyodide. + + This rule detects imports of standard library modules that are missing + or non-functional in the Pyodide runtime used by WASM notebooks. + + ## What it does + + Checks each cell's imports against a blocklist of stdlib modules that + either don't exist in Pyodide or are stubs that fail at runtime. + + ## Why is this bad? + + WASM notebooks run in the browser via Pyodide, which cannot support + modules that depend on OS-level process control, terminal I/O, or + native GUI toolkits. Importing these modules will raise ImportError + or produce broken stubs. + + ## Examples + + **Problematic:** + ```python + import subprocess + + result = subprocess.run(["ls"]) + ``` + + **Problematic:** + ```python + from multiprocessing import Pool + ``` + + **Solution:** + Remove or replace the import with a WASM-compatible alternative. + + ## References + + - https://pyodide.org/en/stable/usage/wasm-constraints.html + """ + + code = "MW001" + name = "incompatible-import" + description = "Importing a module unavailable in WASM/Pyodide" + severity = Severity.WASM + fixable = False + + async def _validate_graph( + self, graph: DirectedGraph, ctx: RuleContext + ) -> None: + for cell_id, cell_impl in graph.cells.items(): + for variable, var_data_list in cell_impl.variable_data.items(): + for var_data in var_data_list: + if var_data.import_data is None: + continue + + top_level = var_data.import_data.module.split(".")[0] + if top_level not in INCOMPATIBLE_MODULES: + continue + + line, column = self._get_variable_line_info( + cell_id, variable, ctx + ) + await ctx.add_diagnostic( + Diagnostic( + message=( + f"Module '{top_level}' is not available in " + "WASM/Pyodide and will fail to import." + ), + line=line, + column=column, + fix=f"Remove or replace '{top_level}' with a WASM-compatible alternative.", + ) + ) diff --git a/marimo/_lint/rules/wasm/incompatible_packages.py b/marimo/_lint/rules/wasm/incompatible_packages.py new file mode 100644 index 00000000000..8c1010e0d00 --- /dev/null +++ b/marimo/_lint/rules/wasm/incompatible_packages.py @@ -0,0 +1,224 @@ +# Copyright 2026 Marimo. All rights reserved. +from __future__ import annotations + +import functools +import importlib.metadata +import json +import re +import urllib.request +from typing import TYPE_CHECKING + +from marimo._lint.diagnostic import Diagnostic, Severity +from marimo._lint.rules.base import LintRule + +if TYPE_CHECKING: + from marimo._lint.context import RuleContext + + +def _normalize_name(name: str) -> str: + """Normalize package name per PEP 503.""" + return re.sub(r"[-_.]+", "-", name).lower() + + +def _get_pyodide_packages() -> set[str] | None: + """Fetch the set of normalized package names available in Pyodide.""" + try: + from marimo._pyodide.pyodide_constraints import ( + fetch_pyodide_package_versions, + ) + + versions = fetch_pyodide_package_versions() + return {_normalize_name(name) for name in versions} + except Exception: + return None + + +@functools.cache +def _has_wasm_compatible_wheel(package_name: str) -> bool: + """Check PyPI for a pure-python or emscripten wheel. + + Returns True if micropip can install this package (has a + py3-none-any wheel, a py2.py3-none-any wheel, or an + emscripten/wasm32 wheel). Returns True on network failure + (fail open). Cached so a single export with N transitive deps + hits PyPI at most once per unique package name. + """ + url = f"https://pypi.org/pypi/{package_name}/json" + try: + with urllib.request.urlopen(url, timeout=10) as resp: + data = json.loads(resp.read()) + except Exception: + return True # Can't check — assume compatible. + + urls = data.get("urls", []) + if not urls: + return True # No files — likely a namespace package. + + for file_info in urls: + filename = file_info.get("filename", "") + if filename.endswith(".whl"): + if ( + "none-any" in filename + or "emscripten" in filename + or "wasm" in filename + ): + return True + # Source distributions can be built as pure-python by micropip. + if filename.endswith((".tar.gz", ".zip")): + return True + + return False + + +def _get_notebook_deps(ctx: RuleContext) -> set[str] | None: + """Extract PEP 723 dependencies from the notebook's script metadata.""" + if not ctx.notebook.filename: + return None + + try: + from marimo._utils.inline_script_metadata import PyProjectReader + + reader = PyProjectReader.from_filename(ctx.notebook.filename) + deps = reader.dependencies + if not deps: + return None + names: set[str] = set() + for dep in deps: + match = re.match( + r"^([A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?)", dep + ) + if match: + names.add(_normalize_name(match.group(1))) + return names + except Exception: + return None + + +def _resolve_dep_tree(root_deps: set[str]) -> set[str]: + """Walk installed metadata to resolve the transitive dependency tree.""" + installed: dict[str, importlib.metadata.Distribution] = {} + for dist in importlib.metadata.distributions(): + name = dist.metadata.get("Name") + if name: + installed[_normalize_name(name)] = dist + + resolved: set[str] = set() + queue = list(root_deps) + while queue: + pkg = queue.pop() + if pkg in resolved: + continue + resolved.add(pkg) + + entry: importlib.metadata.Distribution | None = installed.get(pkg) + if entry is None: + continue + + requires = entry.metadata.get_all("Requires-Dist") or [] + for req in requires: + if "extra ==" in req: + continue + match = re.match( + r"^([A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?)", req + ) + if match: + dep_name = _normalize_name(match.group(1)) + if dep_name not in resolved: + queue.append(dep_name) + + return resolved + + +class IncompatiblePackagesRule(LintRule): + """MW003: Packages in the dependency tree incompatible with WASM. + + This rule resolves the notebook's PEP 723 dependency tree and checks + each package against PyPI for WASM-compatible wheels. Catches + transitive dependencies like ``jaxlib`` (pulled in by ``jax``) that + have no pure-python or emscripten wheel on PyPI. + + ## What it does + + Reads the notebook's PEP 723 ``dependencies``, walks their transitive + dependency tree via installed metadata, then queries PyPI's JSON API + to check whether each package has a ``py3-none-any`` or emscripten + wheel available. Packages only in pyodide-lock.json are also accepted. + + ## Why is this bad? + + Pyodide can only install pure-Python wheels via micropip, or packages + that are pre-built in the Pyodide distribution. Packages with only + platform-specific native wheels will fail to install in the browser. + + ## Examples + + **Problematic:** + ```python + import jax # jaxlib (transitive dep) has only native wheels + ``` + + **Not flagged:** + ```python + import numpy # Native, but pre-built in Pyodide + ``` + + **Not flagged:** + ```python + import requests # Pure Python wheel on PyPI + ``` + + ## References + + - https://pyodide.org/en/stable/usage/packages-in-pyodide.html + """ + + code = "MW003" + name = "incompatible-package" + description = "Package with native extensions not available in Pyodide" + severity = Severity.WASM + fixable = False + + async def check(self, ctx: RuleContext) -> None: + pyodide_packages = _get_pyodide_packages() + if pyodide_packages is None: + return + + notebook_deps = _get_notebook_deps(ctx) + if notebook_deps is None: + return + + notebook_deps.discard("marimo") + if not notebook_deps: + return + + dep_tree = _resolve_dep_tree(notebook_deps) + + incompatible: list[str] = [] + for pkg in sorted(dep_tree): + # In Pyodide's lockfile — definitely available. + if pkg in pyodide_packages: + continue + + # Check PyPI for a WASM-compatible wheel. + if not _has_wasm_compatible_wheel(pkg): + incompatible.append(pkg) + + if not incompatible: + return + + pkg_list = ", ".join(incompatible) + await ctx.add_diagnostic( + Diagnostic( + message=( + f"Package(s) without WASM-compatible wheels on PyPI: " + f"{pkg_list}. " + f"These will fail to install in Pyodide." + ), + line=1, + column=0, + fix=( + "Remove these packages or replace with pure-Python " + "alternatives available in Pyodide." + ), + ) + ) diff --git a/marimo/_lint/rules/wasm/unsafe_system_calls.py b/marimo/_lint/rules/wasm/unsafe_system_calls.py new file mode 100644 index 00000000000..449096e221a --- /dev/null +++ b/marimo/_lint/rules/wasm/unsafe_system_calls.py @@ -0,0 +1,149 @@ +# Copyright 2026 Marimo. All rights reserved. +from __future__ import annotations + +import ast +from typing import TYPE_CHECKING + +from marimo._ast.parse import ast_parse +from marimo._lint.diagnostic import Diagnostic, Severity +from marimo._lint.rules.base import LintRule + +if TYPE_CHECKING: + from marimo._lint.context import RuleContext + +# Functions that trap at runtime even though their parent module imports fine. +UNSAFE_ATTR_CALLS: dict[str, set[str]] = { + "os": { + "system", + "popen", + "fork", + "kill", + "killpg", + "getuid", + "getgid", + }, + "signal": {"signal", "alarm"}, +} + +# Prefixes for os.exec*, os.spawn* families. +UNSAFE_ATTR_PREFIXES: dict[str, tuple[str, ...]] = { + "os": ("exec", "spawn"), +} + +UNSAFE_BUILTINS = frozenset({"breakpoint"}) + + +class _UnsafeCallVisitor(ast.NodeVisitor): + """Collect unsafe calls with their line/column info.""" + + def __init__(self) -> None: + self.findings: list[tuple[int, int, str]] = [] + + def visit_Call(self, node: ast.Call) -> None: + # Check module.func() calls like os.system() + if isinstance(node.func, ast.Attribute): + value = node.func.value + if isinstance(value, ast.Name): + module = value.id + attr = node.func.attr + + # Exact matches + exact = UNSAFE_ATTR_CALLS.get(module) + if exact and attr in exact: + self.findings.append( + (node.lineno, node.col_offset, f"{module}.{attr}()") + ) + + # Prefix matches (os.execl, os.spawnv, etc.) + prefixes = UNSAFE_ATTR_PREFIXES.get(module) + if prefixes and any(attr.startswith(p) for p in prefixes): + # Avoid double-reporting if also in exact set + if not (exact and attr in exact): + self.findings.append( + ( + node.lineno, + node.col_offset, + f"{module}.{attr}()", + ) + ) + + # Check bare builtins like breakpoint() + elif isinstance(node.func, ast.Name): + if node.func.id in UNSAFE_BUILTINS: + self.findings.append( + (node.lineno, node.col_offset, f"{node.func.id}()") + ) + + self.generic_visit(node) + + +class UnsafeSystemCallsRule(LintRule): + """MW002: System calls that fail in WASM/Pyodide. + + This rule detects calls to OS-level functions that silently fail or + raise errors in the Pyodide runtime, even though their parent modules + import successfully. + + ## What it does + + Walks the AST of each cell looking for calls to functions like + ``os.system()``, ``os.fork()``, ``signal.signal()``, and + ``breakpoint()`` that have no meaningful implementation in WASM. + + ## Why is this bad? + + These functions depend on OS features (process spawning, signal + handling, debugger attachment) that don't exist in a browser + environment. They will raise ``OSError``, ``NotImplementedError``, + or hang silently. + + ## Examples + + **Problematic:** + ```python + import os + + os.system("ls") + ``` + + **Problematic:** + ```python + breakpoint() + ``` + + **Solution:** + Remove or guard these calls behind a WASM detection check. + + ## References + + - https://pyodide.org/en/stable/usage/wasm-constraints.html + """ + + code = "MW002" + name = "unsafe-system-call" + description = "System call that fails in WASM/Pyodide" + severity = Severity.WASM + fixable = False + + async def check(self, ctx: RuleContext) -> None: + for cell in ctx.notebook.cells: + try: + tree = ast_parse(cell.code) + except SyntaxError: + continue + + visitor = _UnsafeCallVisitor() + visitor.visit(tree) + + for lineno, col_offset, call_name in visitor.findings: + await ctx.add_diagnostic( + Diagnostic( + message=( + f"'{call_name}' is not supported in WASM/Pyodide " + "and will fail at runtime." + ), + line=cell.lineno + lineno - 1, + column=cell.col_offset + col_offset, + fix=f"Remove or guard '{call_name}' for WASM compatibility.", + ) + ) diff --git a/mkdocs.yml b/mkdocs.yml index 0cb04f65178..b8804e6d2ef 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -209,6 +209,9 @@ nav: - SQL parse error: guides/lint_rules/rules/sql_parse_error.md - Misc parse log: guides/lint_rules/rules/misc_log_capture.md - Reusable ordering: guides/lint_rules/rules/reusable_definition_order.md + - Incompatible import: guides/lint_rules/rules/incompatible_import.md + - Unsafe system call: guides/lint_rules/rules/unsafe_system_call.md + - Incompatible package: guides/lint_rules/rules/incompatible_package.md - Testing: - Testing notebooks: guides/testing/index.md - pytest: guides/testing/pytest.md diff --git a/scripts/generate_lint_docs.py b/scripts/generate_lint_docs.py index 2614d4eaec0..a2da0dec378 100755 --- a/scripts/generate_lint_docs.py +++ b/scripts/generate_lint_docs.py @@ -33,6 +33,7 @@ class Severity(Enum): FORMATTING = "formatting" RUNTIME = "runtime" BREAKING = "breaking" + WASM = "wasm" @dataclass @@ -109,16 +110,18 @@ def extract_rule_info_from_file(file_path: Path) -> list[RuleInfo]: def discover_all_rules() -> dict[str, RuleInfo]: """Discover all lint rules that are actually registered in the codebase.""" # First, get the registered rule codes from the init files - breaking_init = MARIMO_ROOT / "marimo" / "_lint" / "rules" / "breaking" / "__init__.py" - formatting_init = MARIMO_ROOT / "marimo" / "_lint" / "rules" / "formatting" / "__init__.py" - runtime_init = MARIMO_ROOT / "marimo" / "_lint" / "rules" / "runtime" / "__init__.py" + rules_dir = MARIMO_ROOT / "marimo" / "_lint" / "rules" + breaking_init = rules_dir / "breaking" / "__init__.py" + formatting_init = rules_dir / "formatting" / "__init__.py" + runtime_init = rules_dir / "runtime" / "__init__.py" + wasm_init = rules_dir / "wasm" / "__init__.py" registered_codes = set() def process(init_file: Path, prefix: str, registered_codes: set[str]) -> None: try: content = init_file.read_text() - # Extract codes from BREAKING_RULE_CODES dictionary + # Extract codes from *_RULE_CODES dictionary for line in content.split('\n'): if f'"{prefix}' in line and ':' in line: # Extract the code between quotes @@ -131,10 +134,11 @@ def process(init_file: Path, prefix: str, registered_codes: set[str]) -> None: except Exception as e: print(f"Warning: Could not parse rules init: {e}") - # Parse the breaking rules init file + # Parse the rules init files process(breaking_init, "MB", registered_codes) process(runtime_init, "MR", registered_codes) process(formatting_init, "MF", registered_codes) + process(wasm_init, "MW", registered_codes) # Now discover rules from source files rules_dir = MARIMO_ROOT / "marimo" / "_lint" / "rules" @@ -166,6 +170,7 @@ def get_severity_info(severity: Severity) -> tuple[str, str, str]: Severity.BREAKING: ("🚨", "Breaking", "These errors prevent notebook execution"), Severity.RUNTIME: ("⚠️", "Runtime", "These issues may cause runtime problems"), Severity.FORMATTING: ("✨", "Formatting", "These are style and formatting issues"), + Severity.WASM: ("🌐", "WASM", "These issues affect WASM/Pyodide compatibility (off by default)"), } return severity_map.get(severity, ("❓", "Unknown", "")) @@ -177,8 +182,8 @@ def validate_rule_info(rule: RuleInfo) -> list[str]: # Check required attributes are present and valid if not rule.code: issues.append("Missing rule code") - elif not re.match(r'^M[BRF]\d{3}$', rule.code): - issues.append(f"Invalid rule code format: {rule.code} (expected MB###, MR###, or MF###)") + elif not re.match(r'^M[BRFW]\d{3}$', rule.code): + issues.append(f"Invalid rule code format: {rule.code} (expected MB###, MR###, MF###, or MW###)") if not rule.name: issues.append("Missing rule name") @@ -208,7 +213,8 @@ def validate_rule_info(rule: RuleInfo) -> list[str]: expected_prefixes = { Severity.BREAKING: "MB", Severity.RUNTIME: "MR", - Severity.FORMATTING: "MF" + Severity.FORMATTING: "MF", + Severity.WASM: "MW", } expected_prefix = expected_prefixes.get(rule.severity) if expected_prefix and code_prefix != expected_prefix: @@ -296,11 +302,11 @@ def generate_main_index_page(rules_by_severity: dict[Severity, list[dict[str, An ## Rule Categories -marimo's lint rules are organized into three main categories based on their severity: +marimo's lint rules are organized into categories based on their severity: """ - for severity in [Severity.BREAKING, Severity.RUNTIME, Severity.FORMATTING]: + for severity in [Severity.BREAKING, Severity.RUNTIME, Severity.FORMATTING, Severity.WASM]: if severity not in rules_by_severity: continue diff --git a/tests/_lint/test_files/wasm_incompatible.py b/tests/_lint/test_files/wasm_incompatible.py new file mode 100644 index 00000000000..1c9a7c43413 --- /dev/null +++ b/tests/_lint/test_files/wasm_incompatible.py @@ -0,0 +1,45 @@ +import marimo + +__generated_with = "0.23.2" +app = marimo.App() + + +@app.cell +def _(): + import subprocess + + subprocess.run(["ls"]) + return + + +@app.cell +def _(): + import os + + os.system("echo hello") + return + + +@app.cell +def _(): + breakpoint() + return + + +@app.cell +def _(): + import pdb + + pdb.set_trace() + return + + +@app.cell +def _(): + import multiprocessing + + return + + +if __name__ == "__main__": + app.run() diff --git a/tests/_lint/test_lint_config_integration.py b/tests/_lint/test_lint_config_integration.py index 36f9e09a3bf..c0c0229a062 100644 --- a/tests/_lint/test_lint_config_integration.py +++ b/tests/_lint/test_lint_config_integration.py @@ -10,9 +10,9 @@ class TestRuleEngineConfig: def test_create_default_no_config(self): engine = RuleEngine.create_default() - from marimo._lint.rules import RULE_CODES + from marimo._lint.rules import DEFAULT_RULE_CODES - assert len(engine.rules) == len(RULE_CODES) + assert len(engine.rules) == len(DEFAULT_RULE_CODES) def test_create_default_with_select(self): engine = RuleEngine.create_default(lint_config={"select": ["MB"]}) @@ -26,9 +26,9 @@ def test_create_default_with_ignore(self): def test_create_default_empty_config(self): engine = RuleEngine.create_default(lint_config={}) - from marimo._lint.rules import RULE_CODES + from marimo._lint.rules import DEFAULT_RULE_CODES - assert len(engine.rules) == len(RULE_CODES) + assert len(engine.rules) == len(DEFAULT_RULE_CODES) class TestLinterConfig: @@ -53,9 +53,9 @@ def test_linter_explicit_rules_ignores_config(self): def test_linter_no_config(self): linter = Linter() - from marimo._lint.rules import RULE_CODES + from marimo._lint.rules import DEFAULT_RULE_CODES - assert len(linter.rule_engine.rules) == len(RULE_CODES) + assert len(linter.rule_engine.rules) == len(DEFAULT_RULE_CODES) class TestPep723LintConfig: diff --git a/tests/_lint/test_rule_selector.py b/tests/_lint/test_rule_selector.py index 4338ae55756..74631698d2d 100644 --- a/tests/_lint/test_rule_selector.py +++ b/tests/_lint/test_rule_selector.py @@ -4,13 +4,15 @@ from __future__ import annotations from marimo._lint.rule_selector import resolve_rules -from marimo._lint.rules import RULE_CODES +from marimo._lint.rules import DEFAULT_RULE_CODES, RULE_CODES class TestResolveRules: - def test_empty_config_returns_all_rules(self): + def test_empty_config_returns_default_rules(self): rules = resolve_rules({}) - assert len(rules) == len(RULE_CODES) + assert len(rules) == len(DEFAULT_RULE_CODES) + # WASM rules are off by default + assert not any(r.code.startswith("MW") for r in rules) def test_select_by_category_prefix(self): rules = resolve_rules({"select": ["MB"]}) @@ -36,7 +38,7 @@ def test_ignore_single_rule(self): rules = resolve_rules({"ignore": ["MF004"]}) codes = {r.code for r in rules} assert "MF004" not in codes - assert len(rules) == len(RULE_CODES) - 1 + assert len(rules) == len(DEFAULT_RULE_CODES) - 1 def test_ignore_category(self): rules = resolve_rules({"ignore": ["MF"]}) @@ -80,3 +82,13 @@ def test_select_all_ignore_some(self): rules = resolve_rules({"select": ["ALL"], "ignore": ["MF"]}) assert not any(r.code.startswith("MF") for r in rules) assert any(r.code.startswith("MB") for r in rules) + + def test_select_wasm_rules(self): + rules = resolve_rules({"select": ["MW"]}) + assert all(r.code.startswith("MW") for r in rules) + assert len(rules) == 3 + + def test_select_all_includes_wasm(self): + rules = resolve_rules({"select": ["ALL"]}) + assert len(rules) == len(RULE_CODES) + assert any(r.code.startswith("MW") for r in rules) diff --git a/tests/_lint/test_wasm_rules.py b/tests/_lint/test_wasm_rules.py new file mode 100644 index 00000000000..806ba7483a7 --- /dev/null +++ b/tests/_lint/test_wasm_rules.py @@ -0,0 +1,122 @@ +# Copyright 2026 Marimo. All rights reserved. +"""Tests for WASM compatibility lint rules (MW001, MW002, MW003).""" + +from __future__ import annotations + +from marimo._ast.parse import parse_notebook +from marimo._lint.diagnostic import Severity +from tests._lint.utils import lint_notebook + +TEST_FILE = "tests/_lint/test_files/wasm_incompatible.py" + + +def _load_notebook(): + with open(TEST_FILE) as f: + code = f.read() + return parse_notebook(code, filepath=TEST_FILE), code + + +class TestWasmRulesOffByDefault: + """MW rules should not fire without explicit --select MW.""" + + def test_no_wasm_diagnostics_by_default(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook(notebook, contents) + assert not any(d.code.startswith("MW") for d in diagnostics) + + +class TestMW001IncompatibleImports: + """MW001: Importing modules unavailable in WASM/Pyodide.""" + + def test_subprocess_flagged(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW001"]} + ) + codes = [d.code for d in diagnostics] + messages = [d.message for d in diagnostics] + assert "MW001" in codes + assert any("subprocess" in m for m in messages) + + def test_multiprocessing_flagged(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW001"]} + ) + messages = [d.message for d in diagnostics] + assert any("multiprocessing" in m for m in messages) + + def test_pdb_flagged(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW001"]} + ) + messages = [d.message for d in diagnostics] + assert any("pdb" in m for m in messages) + + def test_severity_is_wasm(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW001"]} + ) + for d in diagnostics: + assert d.severity == Severity.WASM + + def test_safe_imports_not_flagged(self): + """Standard safe imports like os should not be flagged.""" + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW001"]} + ) + messages = " ".join(d.message for d in diagnostics) + # os is importable in Pyodide; only specific os.* calls are bad (MW002) + assert "Module 'os'" not in messages + + +class TestMW002UnsafeSystemCalls: + """MW002: System calls that fail in WASM/Pyodide.""" + + def test_os_system_flagged(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW002"]} + ) + messages = [d.message for d in diagnostics] + assert any("os.system()" in m for m in messages) + + def test_breakpoint_flagged(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW002"]} + ) + messages = [d.message for d in diagnostics] + assert any("breakpoint()" in m for m in messages) + + def test_severity_is_wasm(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW002"]} + ) + for d in diagnostics: + assert d.severity == Severity.WASM + + def test_line_numbers_are_set(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW002"]} + ) + for d in diagnostics: + assert d.line > 0 + + +class TestMW001AndMW002Together: + """Both MW001 and MW002 should fire on the test file.""" + + def test_combined_diagnostics(self): + notebook, contents = _load_notebook() + diagnostics = lint_notebook( + notebook, contents, lint_config={"select": ["MW"]} + ) + codes = {d.code for d in diagnostics} + assert "MW001" in codes + assert "MW002" in codes