fix: preserve mixed-script token boundaries in BM25 search

[emecii]

What does this PR do?

Fixes BM25 retrieval of mixed Chinese/Japanese/Korean and Latin text. A passage such as Python数据库SQL currently fails searches for Python and SQL because n-gram expansion joins those words to the edge n-grams. Mixed queries such as Python数据库 also fail because Unicode \w consumes both scripts.

Add boundaries around indexed CJK n-grams and keep CJK runs separate when parsing query terms. Existing OR-between-terms and AND-between-CJK-bigrams semantics are preserved. Extend the FTS5 tests with mixed-script retrieval after database reopen and negative controls.

Previously built artifacts remain readable; affected indexes need rebuilding to regenerate missing tokens. No new dependency or public API change.

Related Issues

Follow-up correctness fix to merged #401. No matching open issue or competing open PR was found. Does not address the metadata-sharing/incremental work in open #398.

Validation

• Nine regression cases fail on the base production code.
• 47 focused FTS5 and metadata-filter tests pass with the fix.
• Changed-file Ruff and pre-commit checks pass; repository-wide formatting passes.
• Whole-repository Ruff has seven pre-existing errors in untouched CLI code; targeted ty reports four unchanged baseline diagnostics.
• Full runtime/CI matrix not run.

AI assistance: implemented and tested with Codex (GPT-6); no claim of human review is made here.

Checklist

• Full test suite (uv run pytest); focused 47-test suite passed.
• Repository-wide lint clean; seven baseline CLI errors remain. Formatting and changed-file lint passed.
• Full pre-commit run; changed-file applicable hooks passed.

CI follow-up

The lint job reports exactly the seven pre-existing F811 duplicate index_* methods in cli.py documented above. Formatting passes. None of the errors is in the tokenizer or regression tests, and the same diagnostics were verified on the base. This PR leaves those handlers unchanged.