QA Jan v0.8.3

[thinhlpg]

QA List:

• Draft release: https://github.com/janhq/jan/releases/tag/untagged-ee8d7b381c0ee877cab4

Install / First-run

• Clean install .dmg → confirm bundle id jan.ai.app, version 0.8.3
• Factory reset returns to first-run on macOS (race-free webview profile prune) (flaky, might not work)
• open_app_directory opens Jan/, not jan.ai.app/
• First launch does not auto-download model; only on explicit click

Chat / Generation

• Deleting the only user turn → clear send-time error, no Jinja "No user query"
• Backend OOM/context overflow → failed turn hidden behind error banner; effectiveStatus reset to ready (shimmer stops)
• Regenerate: no "Loading model…" flash (monotonic stream token)
• Tool approval/exec: sidebar spinner on, send disabled, prompt-progress bar hidden
• LLM auto-title does not overwrite manually set titles (titleSetManually)
• Embedding shimmer persists across thread navigation

Message versioning

• Edit user/assistant message → new version, original preserved
• < n/m > navigates versions
• Switching version re-threads downstream
• Version tree survives restart (parent pointer in metadata)

Assistant switcher

• Button shows only when >1 assistant
• Cmd+J cycle works in normal + project threads
• Switching does not drop the model from local state
• Switching to default assistant is not a no-op

Sampling defaults

• Model Settings → "Default sampling parameters" section visible for llamacpp + mlx
• Editing temperature/top_k/top_p/min_p/repeat persists to model.yml
• llama.cpp router preset emits CLI defaults; per-request JSON still overrides
• MLX requests see defaults (temperature/top_p/repetition_penalty only)
• Sliders no longer pinned to minimum (stored ?? def fix); hidden on embedding models
• Defaults aligned: temperature=0.8, repeat_penalty=1.0

llama.cpp / MTP / Video

• Gemma-4 MTP model → companion mtp-*.gguf auto-downloads, not shown as standalone variant
• Vision model + router mode: drag/pick video renders <video> in the thread
• Old backend + mtmd video → "corrupted size vs. prev_size" surfaces as backend-error banner, not silent relaunch
• Linux ROCm versioned dir (/opt/rocm-*) → HIP backend starts, GPUs detected
• Windows "Install CUDA Runtime" button → installs cudart-llama-bin DLL into existing backend
• Manual install flat-root zip (Windows) → stages then renames build/bin, no EINVAL

Markdown / Render perf

• $x_{i}$ no longer triggers KaTeX "Unrecognized Unicode 8203" (latex code/math lift)
• Inline \[…\] converts correctly
• "home**,**" renders bold correctly (zwsp flanking)
• Long-stream response does not freeze on reasoning (pre-wrapped text path)
• In-progress code block during stream renders as plain text, no Shiki crash mid-token
• Raw <svg> in fence renders via HtmlArtifact (no-script mode)

Artifacts

• Settings → Interface → "Artifact rendering" toggle exists, marked Experimental, default OFF
• ```html block: Code|Preview toggle, sandboxed iframe, network blocked by CSP, no Tauri/IPC access
• CSP meta cannot be loosened by misplaced model markup (always-wrap injection)
• SVG fence preview renders with script-src dropped, sandbox without allow-scripts
• Copy on SVG block works

Reasoning / Tool trace

• Reasoning + tool call + narration fold into one collapsible timeline
• Header verb updates with the current step
• Citation superscripts align with global index after multi-tool RAG (per-message run token)

UI / Window

• Sidebar: selected thread has a persistent highlight distinct from the streaming spinner
• "Colored user message bubble" toggle: off reverts to muted assistant-style bubble
• Model Settings sheet (Win/Linux): offset below in-app window controls; close + header pinned while body scrolls
• Delete-project dialog: no <p> inside DialogDescription (no hydration error)
• Hub Fuse search: threshold 0.3, ignoreLocation, "developer" returns correct results
• HuggingFace author/model routes via ?repo= (not single-segment)
• Download popover with sidebar collapsed: no clipping past window edge

MCP

• Browser MCP teardown kills whole process group (no port leak)
• wait_for_port_free is bounded (no hang)
• Switching to non-vision model shows the auto-disable toast
• MLX STDIO MCP handshake: falls back to system npx/uvx when bundled fails; stderr captured

Server / Proxy

• Pretty-printed JSON body proxy no longer hangs llama-server (stale Content-Length stripped)
• x-anthropic-billing-header stripped at buffered-body chokepoint for both POST arms → Anthropic cache no longer leaks CLI metadata

Provider

• Azure: base URL field editable again, resource-name substitution works, sits above the API key card

Misc / Regression

• HMR: DataProvider events.off, deeplink unsubscribe, theme subscribe via import.meta.hot.dispose — no leaked subscriptions
• Build type-check passes: HtmlArtifact language cast, ResizeDirection inline, filteredModels: CatalogModel[]
• i18n: backend + CUDA install strings localized; "No assistant" fallback

Test suite

• install-from-file by i18n key
• installCudaRuntime 6 cases
• data-provider subscription cleanup mocks
• assistant switcher cycle/wrap
• threadlist useParams stub
• EditModel IconVideo mock
• SetupScreen download-click

[thinhlpg]

Window, Mac

• Draft release: https://github.com/janhq/jan/releases/tag/untagged-ee8d7b381c0ee877cab4

Install / First-run

• Clean install .dmg → confirm bundle id jan.ai.app, version 0.8.3. Window (.exe)
• Factory reset returns to first-run on macOS (race-free webview profile prune) (flaky, might not work)
• open_app_directory opens Jan/, not jan.ai.app/
• First launch does not auto-download model; only on explicit click

Chat / Generation

• Deleting the only user turn → clear send-time error, no Jinja "No user query"
• Backend OOM/context overflow → failed turn hidden behind error banner; effectiveStatus reset to ready (shimmer stops)
• Regenerate: no "Loading model…" flash (monotonic stream token)
• Tool approval/exec: sidebar spinner on, send disabled, prompt-progress bar hidden
• LLM auto-title does not overwrite manually set titles (titleSetManually)
• Embedding shimmer persists across thread navigation

Message versioning

• Edit user/assistant message → new version, original preserved
• < n/m > navigates versions
• Switching version re-threads downstream
• Version tree survives restart (parent pointer in metadata)

Assistant switcher

• Button shows only when >1 assistant
• Cmd+J cycle works in normal + project threads
• Switching does not drop the model from local state
• Switching to default assistant is not a no-op

Sampling defaults

• Model Settings → "Default sampling parameters" section visible for llamacpp + mlx
• Editing temperature/top_k/top_p/min_p/repeat persists to model.yml
• llama.cpp router preset emits CLI defaults; per-request JSON still overrides
• MLX requests see defaults (temperature/top_p/repetition_penalty only)
• Sliders no longer pinned to minimum (stored ?? def fix); hidden on embedding models
• Defaults aligned: temperature=0.8, repeat_penalty=1.0

llama.cpp / MTP / Video

• Gemma-4 MTP model → companion mtp-*.gguf auto-downloads, not shown as standalone variant
• Vision model + router mode: drag/pick video renders <video> in the thread
• Old backend + mtmd video → "corrupted size vs. prev_size" surfaces as backend-error banner, not silent relaunch
• Linux ROCm versioned dir (/opt/rocm-*) → HIP backend starts, GPUs detected
• Windows "Install CUDA Runtime" button → installs cudart-llama-bin DLL into existing backend
• Manual install flat-root zip (Windows) → stages then renames build/bin, no EINVAL

Markdown / Render perf

• $x_{i}$ no longer triggers KaTeX "Unrecognized Unicode 8203" (latex code/math lift)
• Inline \[…\] converts correctly
• "home**,**" renders bold correctly (zwsp flanking)
• Long-stream response does not freeze on reasoning (pre-wrapped text path)
• In-progress code block during stream renders as plain text, no Shiki crash mid-token
• Raw <svg> in fence renders via HtmlArtifact (no-script mode)

Artifacts

• Settings → Interface → "Artifact rendering" toggle exists, marked Experimental, default OFF
• ```html block: Code|Preview toggle, sandboxed iframe, network blocked by CSP, no Tauri/IPC access
• CSP meta cannot be loosened by misplaced model markup (always-wrap injection)
• SVG fence preview renders with script-src dropped, sandbox without allow-scripts
• Copy on SVG block works

Reasoning / Tool trace

• Reasoning + tool call + narration fold into one collapsible timeline
• Header verb updates with the current step
• Citation superscripts align with global index after multi-tool RAG (per-message run token)

UI / Window

• Sidebar: selected thread has a persistent highlight distinct from the streaming spinner
• "Colored user message bubble" toggle: off reverts to muted assistant-style bubble
• Model Settings sheet (Win/Linux): offset below in-app window controls; close + header pinned while body scrolls
• Delete-project dialog: no <p> inside DialogDescription (no hydration error)
• Hub Fuse search: threshold 0.3, ignoreLocation, "developer" returns correct results
• HuggingFace author/model routes via ?repo= (not single-segment)
• Download popover with sidebar collapsed: no clipping past window edge

MCP

• Browser MCP teardown kills whole process group (no port leak)
• wait_for_port_free is bounded (no hang)
• Switching to non-vision model shows the auto-disable toast
• MLX STDIO MCP handshake: falls back to system npx/uvx when bundled fails; stderr captured

Server / Proxy

• Pretty-printed JSON body proxy no longer hangs llama-server (stale Content-Length stripped)
• x-anthropic-billing-header stripped at buffered-body chokepoint for both POST arms → Anthropic cache no longer leaks CLI metadata

Provider

• Azure: base URL field editable again, resource-name substitution works, sits above the API key card

Misc / Regression

• HMR: DataProvider events.off, deeplink unsubscribe, theme subscribe via import.meta.hot.dispose — no leaked subscriptions
• Build type-check passes: HtmlArtifact language cast, ResizeDirection inline, filteredModels: CatalogModel[]
• i18n: backend + CUDA install strings localized; "No assistant" fallback

Test suite

• install-from-file by i18n key
• installCudaRuntime 6 cases
• data-provider subscription cleanup mocks
• assistant switcher cycle/wrap
• threadlist useParams stub
• EditModel IconVideo mock
• SetupScreen download-click

[qnixsynapse]

QA List:

Linux

• Draft release: https://github.com/janhq/jan/releases/tag/untagged-ee8d7b381c0ee877cab4

Install / First-run

• Clean install .appimage → confirm bundle id jan.ai.app, version 0.8.3
• open_app_directory opens Jan/, not jan.ai.app/
• First launch does not auto-download model; only on explicit click

Chat / Generation

• Deleting the only user turn → clear send-time error, no Jinja "No user query"
• Backend OOM/context overflow → failed turn hidden behind error banner; effectiveStatus reset to ready (shimmer stops)
• Regenerate: no "Loading model…" flash (monotonic stream token)
• Tool approval/exec: sidebar spinner on, send disabled, prompt-progress bar hidden
• LLM auto-title does not overwrite manually set titles (titleSetManually)
• Embedding shimmer persists across thread navigation

Message versioning

• Edit user/assistant message → new version, original preserved
• < n/m > navigates versions
• Switching version re-threads downstream
• Version tree survives restart (parent pointer in metadata)

Assistant switcher

• Button shows only when >1 assistant
• Cmd+J cycle works in normal + project threads
• Switching does not drop the model from local state
• Switching to default assistant is not a no-op

Sampling defaults

• Model Settings → "Default sampling parameters" section visible for llamacpp
• Editing temperature/top_k/top_p/min_p/repeat persists to model.yml
• llama.cpp router preset emits CLI defaults; per-request JSON still overrides
• Sliders no longer pinned to minimum (stored ?? def fix); hidden on embedding models
• Defaults aligned: temperature=0.8, repeat_penalty=1.0

llama.cpp / MTP / Video

• Gemma-4 MTP model → companion mtp-*.gguf auto-downloads, not shown as standalone variant
• Vision model + router mode: drag/pick video renders <video> in the thread
• Old backend + mtmd video → "corrupted size vs. prev_size" surfaces as backend-error banner, not silent relaunch
• Linux ROCm versioned dir (/opt/rocm-*) → HIP backend starts, GPUs detected - need AMDGPU deferred

Markdown / Render perf

• $x_{i}$ no longer triggers KaTeX "Unrecognized Unicode 8203" (latex code/math lift)
• Inline \[…\] converts correctly
• "home**,**" renders bold correctly (zwsp flanking)
• Long-stream response does not freeze on reasoning (pre-wrapped text path)
• In-progress code block during stream renders as plain text, no Shiki crash mid-token
• Raw <svg> in fence renders via HtmlArtifact (no-script mode)

Artifacts

• Settings → Interface → "Artifact rendering" toggle exists, marked Experimental, default OFF
• ```html block: Code|Preview toggle, sandboxed iframe, network blocked by CSP, no Tauri/IPC access
• CSP meta cannot be loosened by misplaced model markup (always-wrap injection)
• SVG fence preview renders with script-src dropped, sandbox without allow-scripts
• Copy on SVG block works

Reasoning / Tool trace

• Reasoning + tool call + narration fold into one collapsible timeline
• Header verb updates with the current step
• Citation superscripts align with global index after multi-tool RAG (per-message run token)

UI / Window

• Sidebar: selected thread has a persistent highlight distinct from the streaming spinner
• "Colored user message bubble" toggle: off reverts to muted assistant-style bubble
• Model Settings sheet (Win/Linux): offset below in-app window controls; close + header pinned while body scrolls
• Delete-project dialog: no <p> inside DialogDescription (no hydration error)
• Hub Fuse search: threshold 0.3, ignoreLocation, "developer" returns correct results
• HuggingFace author/model routes via ?repo= (not single-segment)
• Download popover with sidebar collapsed: no clipping past window edge

MCP

• Browser MCP teardown kills whole process group (no port leak)
• wait_for_port_free is bounded (no hang)
• Switching to non-vision model shows the auto-disable toast
• MLX STDIO MCP handshake: falls back to system npx/uvx when bundled fails; stderr captured

Server / Proxy

• Pretty-printed JSON body proxy no longer hangs llama-server (stale Content-Length stripped)
• x-anthropic-billing-header stripped at buffered-body chokepoint for both POST arms → Anthropic cache no longer leaks CLI metadata

Provider

• Azure: base URL field editable again, resource-name substitution works, sits above the API key card

Misc / Regression

• HMR: DataProvider events.off, deeplink unsubscribe, theme subscribe via import.meta.hot.dispose — no leaked subscriptions - not needed in production (dev only)
• Build type-check passes: HtmlArtifact language cast, ResizeDirection inline, filteredModels: CatalogModel[]
• i18n: backend + CUDA install strings localized; "No assistant" fallback

Test suite

• install-from-file by i18n key
• installCudaRuntime 6 cases (Windows only)
• data-provider subscription cleanup mocks
• assistant switcher cycle/wrap
• threadlist useParams stub
• EditModel IconVideo mock
• SetupScreen download-click

[thinhlpg]

Video input: model ignores video (silent, no error)

Env: Windows, draft 0.8.3 (Jan_0.8.3_x64-setup.exe), backend b9743 win-cuda-13, model Qwen3_5-4B-IQ4_XS (vision + mmproj.gguf), RTX 4060 Ti.

Symptom: drag/pick a video ? <video> renders in the thread (CSP #8330 OK), but the model replies text-only and never "sees" the video. No error banner, no crash.

Root cause (from logs): the video frames never reach the vision encoder. Jan sends the video as a base64 input_video content part straight to llama-server (web-app/src/lib/video-sentinel.ts ? model-factory.ts); ffmpeg is not involved at all. The bundled llama-server build silently drops input_video for this model.

Proof ? llama.cpp itself warns a frame needs ?1024 image tokens:

W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024

But every prompt eval after sending a video is tiny ? no image tokens decoded:

slot print_timing: task 0   | prompt eval time = 446.13 ms /  925 tokens
slot print_timing: task 226 | prompt eval time = 159.78 ms /  282 tokens
slot print_timing: task 550 | prompt eval time = 230.14 ms /  369 tokens

If a video were decoded into frames, prompts would be thousands of image tokens. They are not ? input_video is ignored server-side.

Notes:

• ffmpeg installed on the host is irrelevant (Jan never calls it).
• No "corrupted size vs. prev_size" crash here ? it fails silently instead of surfacing a backend-error banner, so the related QA item ("mtmd video ? backend-error banner, not silent relaunch") is not satisfied for this path.

Relates to QA item: llama.cpp / MTP / Video ? "Vision model + router mode: drag/pick video".