feat: add record-replay system for playground HTTP calls#3336
Open
feat: add record-replay system for playground HTTP calls#3336
Conversation
…Phase 1) Add RecordReplay<K,V> generic store, RequestKey, RecordedResponse types, and ReplayHttp wrapper implementing IoNamespaceHttp + IoClassHttpResponse. Wire ReplayHttp into the LSP path wrapping PlaygroundHttp with shared fetch_id allocator. Every HTTP response is now automatically recorded in a per-project replay store.
… (Phase 2) Add ToggleReplay WebSocket message for pin/unpin commands. Implement replay-hit path in ReplayHttp::send() that returns cached responses instantly without network calls. Wire on_replay callback to emit FetchLogNew (with replayed flag) and FetchLogUpdate via broadcast.
Add toggleReplay protocol types and wire through WebSocketRuntimePort. Add pin/unpin button and purple REPLAYED badge to all three fetch log render sites in ExecutionPanel. Add no-op toggleReplay handler in WASM worker for exhaustiveness.
…e 4) Wrap WasmHttp in ReplayHttp with fetch_id starting at 1M to avoid collision with JS-side IDs. Add replay_notify JS callback for posting fetch log events on replay hits. Add toggle_replay wasm_bindgen method to BamlWasmRuntime. Wire toggleReplay in worker.
…spector Replace inline pin icons with a Record/Replay dialog accessible from the status bar. The dialog has a left/right split layout: - Left: domain → endpoint → request navigation tree - Right: response list with replay toggle, request/response body inspector Backend changes: - Add RequestDisplayInfo, snapshot structs, and snapshot() enumeration API - Store timestamps (via web-time) and request metadata per recording - Fix fetch_id race condition by pre-allocating IDs in send() - Add RequestReplayState/ReplayState WS message pair - Add replayState() wasm_bindgen method Frontend changes: - Add ReplayGroup/ReplayRecording types and wire through both transports - Create ReplayManagerPopover.tsx with 4-level tree view - Add shadcn Switch component - Suppress browser password manager on API Keys dialog inputs - Remove Pin/PinOff icons and onTogglePin from ExecutionPanel
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📝 WalkthroughWalkthroughAdds HTTP record-and-replay support across the playground: new replay store and HTTP wrapper, LSP server integration with per-project replay maps and WS handlers, WASM bridge callbacks and exports, worker/protocol and UI components for inspecting/pinning replayed responses. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Browser Client
participant Worker as LSP Worker (WASM)
participant Server as LSP Server
participant Store as ReplayStore
participant HTTP as ReplayHttp
Note over Client,HTTP: Recording Phase
Client->>Worker: perform HTTP request
Worker->>Server: sys_ops request
Server->>HTTP: send(request)
HTTP->>HTTP: compute RequestKey
HTTP->>Store: check pinned (miss)
HTTP->>HTTP: allocate fetch_id
HTTP->>HTTP: delegate to inner HTTP
HTTP-->>Server: response placeholder
Server->>HTTP: text() -> actual body
HTTP->>Store: record(fetch_id, key, response metadata)
Server-->>Worker: response logged
Worker-->>Client: emit FetchLogNew
Note over Client,HTTP: Pin & Replay Phase
Client->>Worker: toggleReplay(fetchId, pinned=true)
Worker->>Server: WsInMessage::ToggleReplay
Server->>Store: set_pinned(fetch_id, true)
Client->>Worker: send same HTTP request
Worker->>Server: sys_ops request
Server->>HTTP: send(same_request)
HTTP->>HTTP: compute RequestKey
HTTP->>Store: check pinned (hit)
Store-->>HTTP: return RecordedResponse
HTTP->>HTTP: on_replay callback (emit event)
HTTP-->>Server: synthetic response (replay)
Server-->>Worker: response logged with replayed=true
Worker-->>Client: emit FetchLogNew(replayed: true)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merging this PR will not alter performance
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
typescript2/pkg-playground/src/components/ApiKeysDialog.tsx (1)
146-152: 🧹 Nitpick | 🔵 TrivialConsider adding autofill suppression to import Textarea.
While password managers rarely interact with textareas, the .env import field can contain sensitive API keys. Adding
autoComplete="off"would provide consistency with the other inputs and an extra layer of protection.🔒 Optional enhancement
<Textarea value={importText} onChange={(e) => setImportText(e.target.value)} placeholder={"Paste .env contents here...\nKEY=value\nANOTHER_KEY=value"} rows={5} className="text-[11px] font-vsc-mono resize-none" + autoComplete="off" />
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: f6c17e53-11bf-4f66-9ffc-ad2edcd821f6
⛔ Files ignored due to path filters (1)
baml_language/Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (16)
baml_language/Cargo.tomlbaml_language/crates/baml_lsp_server/src/lib.rsbaml_language/crates/baml_lsp_server/src/playground_http.rsbaml_language/crates/baml_lsp_server/src/playground_server.rsbaml_language/crates/baml_lsp_server/src/playground_ws.rsbaml_language/crates/bridge_wasm/src/lib.rsbaml_language/crates/sys_ops/Cargo.tomlbaml_language/crates/sys_ops/src/lib.rsbaml_language/crates/sys_ops/src/replay.rstypescript2/app-promptfiddle/src/playground/baml-lsp-worker.tstypescript2/pkg-playground/src/ExecutionPanel.tsxtypescript2/pkg-playground/src/ReplayManagerPopover.tsxtypescript2/pkg-playground/src/components/ApiKeysDialog.tsxtypescript2/pkg-playground/src/components/ui/switch.tsxtypescript2/pkg-playground/src/ports/WebSocketRuntimePort.tstypescript2/pkg-playground/src/worker-protocol.ts
Comment on lines
22
to
27
| pub struct PlaygroundHttpState { | ||
| broadcast_tx: broadcast::Sender<WsOutMessage>, | ||
| next_fetch_id: AtomicU64, | ||
| next_fetch_id: Arc<AtomicU64>, | ||
| /// Maps response body pointer → (call_id, fetch_id) for response_text tracking. | ||
| response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>, | ||
| pub response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>, | ||
| } |
Contributor
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Keep response_to_fetch out of the public API.
Exposing a mutable Mutex<HashMap<...>> field makes this fetch/body correlation invariant externally mutable. If the replay wiring only needs it inside baml_lsp_server, pub(crate) or a small accessor keeps the API surface tighter.
🔒 Suggested change
pub struct PlaygroundHttpState {
broadcast_tx: broadcast::Sender<WsOutMessage>,
next_fetch_id: Arc<AtomicU64>,
/// Maps response body pointer → (call_id, fetch_id) for response_text tracking.
- pub response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>,
+ pub(crate) response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>,
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| pub struct PlaygroundHttpState { | |
| broadcast_tx: broadcast::Sender<WsOutMessage>, | |
| next_fetch_id: AtomicU64, | |
| next_fetch_id: Arc<AtomicU64>, | |
| /// Maps response body pointer → (call_id, fetch_id) for response_text tracking. | |
| response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>, | |
| pub response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>, | |
| } | |
| pub struct PlaygroundHttpState { | |
| broadcast_tx: broadcast::Sender<WsOutMessage>, | |
| next_fetch_id: Arc<AtomicU64>, | |
| /// Maps response body pointer → (call_id, fetch_id) for response_text tracking. | |
| pub(crate) response_to_fetch: std::sync::Mutex<HashMap<usize, (u64, u64)>>, | |
| } |
Comment on lines
+429
to
+439
| WsInMessage::ToggleReplay { | ||
| project, | ||
| fetch_id, | ||
| pinned, | ||
| } => { | ||
| let stores = state.replay_stores.lock().unwrap(); | ||
| if let Some(store) = stores.get(&project) { | ||
| store.write().unwrap().set_pinned(fetch_id, pinned); | ||
| } else { | ||
| tracing::warn!("ToggleReplay: no replay store for project {project}"); | ||
| } |
Contributor
There was a problem hiding this comment.
Handle unknown fetch_ids explicitly.
set_pinned() already tells you when the requested recording is missing. Ignoring that return value turns stale replay-manager state into a silent no-op.
💡 Suggested change
WsInMessage::ToggleReplay {
project,
fetch_id,
pinned,
} => {
let stores = state.replay_stores.lock().unwrap();
if let Some(store) = stores.get(&project) {
- store.write().unwrap().set_pinned(fetch_id, pinned);
+ if !store.write().unwrap().set_pinned(fetch_id, pinned) {
+ tracing::warn!("ToggleReplay: unknown fetch_id {fetch_id} for project {project}");
+ }
} else {
tracing::warn!("ToggleReplay: no replay store for project {project}");
}
}
Comment on lines
+164
to
+171
| replay_store: std::sync::Arc< | ||
| std::sync::RwLock< | ||
| sys_ops::replay::RecordReplay< | ||
| sys_ops::replay::RequestKey, | ||
| sys_ops::replay::RecordedResponse, | ||
| >, | ||
| >, | ||
| >, |
Contributor
There was a problem hiding this comment.
Scope replay state per project in the WASM runtime.
This stores one replay_store on BamlWasmRuntime and exposes toggleReplay() / replayState() without a project. If the runtime ever has multiple playground projects loaded, recordings and pin toggles will bleed across projects even though the main-thread protocol already sends project.
Also applies to: 232-237, 464-487
Comment on lines
+29
to
30
| | { type: 'replayState'; entries: unknown[] }; | ||
|
|
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Decode replay state in one shared helper.
This snake_case → camelCase mapper is duplicated here and in typescript2/app-promptfiddle/src/playground/baml-lsp-worker.ts on Lines 679-697. A shared raw wire type plus normalizer in pkg-playground would remove the unknown[]/any casts and keep the WebSocket and WASM paths from drifting.
Also applies to: 322-340
Binary size checks passed✅ 7 passed
Generated by |
…esponsive dialog - Record SysOpOutput::Ready(Ok) responses in ReplayHttp::text() (sync backends) - Send full request/response bodies in snapshot instead of 1000-char previews - Add aria-label to Replay Manager icon button for accessibility - Make Replay Manager dialog width responsive for narrow viewports - Add TODO comment for per-project WASM replay store scoping Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
baml_language/crates/bridge_wasm/src/lib.rs (1)
164-176:⚠️ Potential issue | 🟠 MajorKeep replay storage project-scoped in the WASM runtime.
BamlWasmRuntimestill owns one sharedreplay_store, andtoggleReplay()/replayState()never take a project key. If a single runtime holds more than one project, recordings and pins will bleed across projects even though the worker protocol is already project-scoped.Also applies to: 205-207, 469-491
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2b528822-61fd-4d0f-8e18-6b4f3b31576d
📒 Files selected for processing (7)
baml_language/crates/bridge_wasm/src/lib.rsbaml_language/crates/sys_ops/src/replay.rstypescript2/app-promptfiddle/src/playground/baml-lsp-worker.tstypescript2/pkg-playground/src/ExecutionPanel.tsxtypescript2/pkg-playground/src/ReplayManagerPopover.tsxtypescript2/pkg-playground/src/ports/WebSocketRuntimePort.tstypescript2/pkg-playground/src/worker-protocol.ts
Comment on lines
+176
to
+183
| let mut sorted_headers: Vec<_> = request.headers.iter().collect(); | ||
| sorted_headers.sort_by_key(|(k, _)| k.as_str()); | ||
| for (k, v) in &sorted_headers { | ||
| hasher.update(k.as_bytes()); | ||
| hasher.update(b":"); | ||
| hasher.update(v.as_bytes()); | ||
| hasher.update(b"\0"); | ||
| } |
Contributor
There was a problem hiding this comment.
Normalize header names before hashing.
HTTP header names are case-insensitive. Hashing Authorization and authorization differently means the same request can miss an existing replay pin after a harmless casing change.
💡 Suggested fix
- let mut sorted_headers: Vec<_> = request.headers.iter().collect();
- sorted_headers.sort_by_key(|(k, _)| k.as_str());
- for (k, v) in &sorted_headers {
- hasher.update(k.as_bytes());
+ let mut sorted_headers: Vec<_> = request
+ .headers
+ .iter()
+ .map(|(k, v)| (k.to_ascii_lowercase(), v))
+ .collect();
+ sorted_headers.sort_by(|(a, _), (b, _)| a.cmp(b));
+ for (k, v) in &sorted_headers {
+ hasher.update(k.as_bytes());
hasher.update(b":");
hasher.update(v.as_bytes());
hasher.update(b"\0");
}
Comment on lines
+421
to
+649
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| fn dummy_display() -> RequestDisplayInfo { | ||
| RequestDisplayInfo { | ||
| method: "GET".to_string(), | ||
| url: "https://example.com".to_string(), | ||
| body: String::new(), | ||
| } | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_record_and_lookup() { | ||
| let mut store = RecordReplay::new(); | ||
| let key = RequestKey([0u8; 32]); | ||
| store.record(key.clone(), 1, "response body".to_string(), dummy_display()); | ||
|
|
||
| assert!(store.get_pinned(&key).is_none()); | ||
|
|
||
| assert!(store.set_pinned(1, true)); | ||
| let pinned = store.get_pinned(&key).unwrap(); | ||
| assert_eq!(pinned.value, "response body"); | ||
| assert_eq!(pinned.fetch_id, 1); | ||
|
|
||
| assert!(store.set_pinned(1, false)); | ||
| assert!(store.get_pinned(&key).is_none()); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_multiple_recordings_per_key() { | ||
| let mut store = RecordReplay::new(); | ||
| let key = RequestKey([0u8; 32]); | ||
| store.record(key.clone(), 1, "first".to_string(), dummy_display()); | ||
| store.record(key.clone(), 2, "second".to_string(), dummy_display()); | ||
|
|
||
| assert!(store.set_pinned(2, true)); | ||
| let pinned = store.get_pinned(&key).unwrap(); | ||
| assert_eq!(pinned.value, "second"); | ||
|
|
||
| assert!(store.set_pinned(1, true)); | ||
| let pinned = store.get_pinned(&key).unwrap(); | ||
| assert_eq!(pinned.value, "first"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_request_key_deterministic() { | ||
| let req = owned::http::Request { | ||
| method: "POST".to_string(), | ||
| url: "https://api.openai.com/v1/chat".to_string(), | ||
| headers: indexmap::IndexMap::from([ | ||
| ("content-type".to_string(), "application/json".to_string()), | ||
| ("authorization".to_string(), "Bearer sk-xxx".to_string()), | ||
| ]), | ||
| body: r#"{"model":"gpt-4","messages":[]}"#.to_string(), | ||
| }; | ||
| let key1 = RequestKey::from_request(&req); | ||
| let key2 = RequestKey::from_request(&req); | ||
| assert_eq!(key1, key2); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_request_key_header_order_independent() { | ||
| let req1 = owned::http::Request { | ||
| method: "POST".to_string(), | ||
| url: "https://api.example.com".to_string(), | ||
| headers: indexmap::IndexMap::from([ | ||
| ("a".to_string(), "1".to_string()), | ||
| ("b".to_string(), "2".to_string()), | ||
| ]), | ||
| body: "body".to_string(), | ||
| }; | ||
| let req2 = owned::http::Request { | ||
| method: "POST".to_string(), | ||
| url: "https://api.example.com".to_string(), | ||
| headers: indexmap::IndexMap::from([ | ||
| ("b".to_string(), "2".to_string()), | ||
| ("a".to_string(), "1".to_string()), | ||
| ]), | ||
| body: "body".to_string(), | ||
| }; | ||
| assert_eq!( | ||
| RequestKey::from_request(&req1), | ||
| RequestKey::from_request(&req2) | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_request_key_different_body() { | ||
| let req1 = owned::http::Request { | ||
| method: "POST".to_string(), | ||
| url: "https://api.example.com".to_string(), | ||
| headers: indexmap::IndexMap::new(), | ||
| body: "body1".to_string(), | ||
| }; | ||
| let req2 = owned::http::Request { | ||
| method: "POST".to_string(), | ||
| url: "https://api.example.com".to_string(), | ||
| headers: indexmap::IndexMap::new(), | ||
| body: "body2".to_string(), | ||
| }; | ||
| assert_ne!( | ||
| RequestKey::from_request(&req1), | ||
| RequestKey::from_request(&req2) | ||
| ); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_set_pinned_unknown_fetch_id() { | ||
| let mut store: RecordReplay<RequestKey, String> = RecordReplay::new(); | ||
| assert!(!store.set_pinned(999, true)); | ||
| } | ||
|
|
||
| // --------------------------------------------------------------------------- | ||
| // snapshot() tests | ||
| // --------------------------------------------------------------------------- | ||
|
|
||
| fn make_key(s: &str) -> RequestKey { | ||
| let mut hasher = sha2::Sha256::new(); | ||
| hasher.update(s.as_bytes()); | ||
| RequestKey(hasher.finalize().into()) | ||
| } | ||
|
|
||
| fn make_display(method: &str, url: &str) -> RequestDisplayInfo { | ||
| RequestDisplayInfo { | ||
| method: method.to_string(), | ||
| url: url.to_string(), | ||
| body: String::new(), | ||
| } | ||
| } | ||
|
|
||
| fn make_response(status: i64, body: &str) -> RecordedResponse { | ||
| RecordedResponse { | ||
| status, | ||
| headers: indexmap::IndexMap::new(), | ||
| body: body.to_string(), | ||
| url: "https://example.com".to_string(), | ||
| } | ||
| } | ||
|
|
||
| #[test] | ||
| fn snapshot_returns_all_groups_with_display_info() { | ||
| let mut store = RecordReplay::new(); | ||
| let key_a = make_key("request-a"); | ||
| let key_b = make_key("request-b"); | ||
|
|
||
| store.record( | ||
| key_a.clone(), | ||
| 1, | ||
| make_response(200, "resp1"), | ||
| make_display("POST", "https://a.com"), | ||
| ); | ||
| store.record( | ||
| key_a, | ||
| 2, | ||
| make_response(200, "resp2"), | ||
| make_display("POST", "https://a.com"), | ||
| ); | ||
| store.record( | ||
| key_b, | ||
| 3, | ||
| make_response(404, "not found"), | ||
| make_display("GET", "https://b.com"), | ||
| ); | ||
|
|
||
| let snap = store.snapshot(); | ||
| assert_eq!(snap.len(), 2); | ||
|
|
||
| let group_a = snap | ||
| .iter() | ||
| .find(|g| g.display.url == "https://a.com") | ||
| .unwrap(); | ||
| assert_eq!(group_a.display.method, "POST"); | ||
| assert_eq!(group_a.recordings.len(), 2); | ||
| assert_eq!(group_a.pinned_fetch_id, None); | ||
|
|
||
| let group_b = snap | ||
| .iter() | ||
| .find(|g| g.display.url == "https://b.com") | ||
| .unwrap(); | ||
| assert_eq!(group_b.recordings.len(), 1); | ||
| assert_eq!(group_b.recordings[0].status, 404); | ||
| } | ||
|
|
||
| #[test] | ||
| fn snapshot_shows_pinned_fetch_id() { | ||
| let mut store = RecordReplay::new(); | ||
| let key = make_key("req"); | ||
| store.record( | ||
| key.clone(), | ||
| 10, | ||
| make_response(200, "ok"), | ||
| make_display("GET", "https://x.com"), | ||
| ); | ||
| store.record( | ||
| key, | ||
| 11, | ||
| make_response(200, "ok2"), | ||
| make_display("GET", "https://x.com"), | ||
| ); | ||
| store.set_pinned(11, true); | ||
|
|
||
| let snap = store.snapshot(); | ||
| assert_eq!(snap.len(), 1); | ||
| assert_eq!(snap[0].pinned_fetch_id, Some(11)); | ||
| } | ||
|
|
||
| #[test] | ||
| fn display_info_stored_only_on_first_record() { | ||
| let mut store = RecordReplay::new(); | ||
| let key = make_key("req"); | ||
| store.record( | ||
| key.clone(), | ||
| 1, | ||
| make_response(200, "a"), | ||
| make_display("POST", "https://first.com"), | ||
| ); | ||
| store.record( | ||
| key, | ||
| 2, | ||
| make_response(200, "b"), | ||
| make_display("POST", "https://second.com"), | ||
| ); | ||
|
|
||
| let snap = store.snapshot(); | ||
| // Display info should be from the first recording, not overwritten. | ||
| assert_eq!(snap[0].display.url, "https://first.com"); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Add unit coverage for ReplayHttp's wrapper branches.
These tests stop at RecordReplay / RequestKey. The highest-risk logic is in ReplayHttp::send and ReplayHttp::text—especially the SysOpOutput::Ready(Ok(_)) recording path, replay-hit synthetic bodies, and pending correlation—and those branches are still untested.
As per coding guidelines, "Prefer writing Rust unit tests over integration tests where possible" and "Always run cargo test --lib if you changed any Rust code."
Comment on lines
+216
to
+233
| const [selectedRecFetchId, setSelectedRecFetchId] = useState<number | null>(null); | ||
| const [activeTab, setActiveTab] = useState<"request" | "response">("request"); | ||
| const isEnabled = group.pinnedFetchId != null; | ||
|
|
||
| const effectiveSelectedId = selectedRecFetchId | ||
| ?? group.pinnedFetchId | ||
| ?? group.recordings[0]?.fetchId | ||
| ?? null; | ||
|
|
||
| const selectedRec = group.recordings.find((r) => r.fetchId === effectiveSelectedId) ?? null; | ||
|
|
||
| const handleToggle = (checked: boolean) => { | ||
| if (!checked) { | ||
| if (group.pinnedFetchId != null) onToggleReplay(group.pinnedFetchId, false); | ||
| } else if (effectiveSelectedId != null) { | ||
| onToggleReplay(effectiveSelectedId, true); | ||
| } | ||
| }; |
Contributor
There was a problem hiding this comment.
Remount RightPanel when the selected request changes.
RightPanel keeps selectedRecFetchId and activeTab in local state, but the component is reused across different groups. After selecting a recording in group A and then switching to group B, effectiveSelectedId can still point at A's fetchId, so enabling replay in B can pin A's response.
💡 Suggested fix
- {selectedGroup ? (
- <RightPanel group={selectedGroup} onToggleReplay={onToggleReplay} />
+ {selectedGroup ? (
+ <RightPanel key={selectedGroup.key} group={selectedGroup} onToggleReplay={onToggleReplay} />
) : (Also applies to: 386-387
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Record-Replay System | Artifacts | Task
What problems was I solving
When iterating on BAML functions that make multiple LLM calls (chains, retries, conditional branches), every click of "Run" re-executes all HTTP fetches from scratch. If the user is only changing the prompt for one call in a chain, they still wait for every other call to complete. There's no way to "pin" a previous response and skip the network round-trip.
This PR adds automatic HTTP response recording and user-controlled replay to the BAML playground. After running a function once, users can pin any recorded response so that subsequent runs with the same request key return instantly — eliminating redundant LLM calls during prompt iteration.
What user-facing changes did I ship
How I implemented it
Core Infrastructure (Rust —
sys_ops/src/replay.rs)New 635-line module with:
RecordReplay<K, V>— generic recording/pinning/lookup store withrecord(),get_pinned(),set_pinned(), andsnapshot()methodsRequestKey— SHA-256 hash of(method, url, sorted_headers, body)for content-addressable request matchingRecordedResponse— stores status, headers, body, and URLReplayHttp— wraps anydyn IoNamespaceHttpto interceptsend()(checks for pinned replay hit) andtext()(records response after successful fetch). UsesReplayResponseBodymarker in_bodyfor replay hits andpending_recordingsmap forsend()→text()correlation viaArc::as_ptrReplayFetchEvent+on_replaycallback — notifies the UI when a replay hit occursLSP Path Wiring (
baml_lsp_server/)lib.rs—ReplayStoreMaptype alias for per-project replay stores;build_playground_sys_ops()now takes areplay_storeand wrapsPlaygroundHttpinReplayHttpwith anon_replaycallback that broadcastsFetchLogNew+FetchLogUpdateeventsplayground_http.rs—next_fetch_idchanged from ownedAtomicU64to sharedArc<AtomicU64>soReplayHttpcan allocate IDs for replayed calls without collisionplayground_server.rs—WsStategainsreplay_storesfield; newToggleReplayandRequestReplayStateWebSocket message handlersplayground_ws.rs— NewWsInMessage::ToggleReplayandWsInMessage::RequestReplayStatevariants;FetchLogNewgains optionalreplayedfield; newWsOutMessage::ReplayStateWASM Path Wiring (
bridge_wasm/src/lib.rs)BamlWasmRuntimegainsreplay_storefieldWasmHttpinReplayHttpwith areplay_notifyJS callback that posts replay events to the main thread viajs_sys::ArraytoggleReplay()andreplayState()methods exposed to JS via#[wasm_bindgen]Frontend (TypeScript)
worker-protocol.ts— NewReplayGroup,ReplayRecordingtypes;FetchLogEntrygainsreplayedandpinnedfields; newtoggleReplayandrequestReplayStateworker messagesbaml-lsp-worker.ts—replay_notifycallback in WASM callbacks;toggleReplayandrequestReplayStatemessage handlersWebSocketRuntimePort.ts— MapstoggleReplay/requestReplayStateto/from WebSocket; transformsreplayStateandfetchLogNew.replayedin both directionsExecutionPanel.tsx— Purple "REPLAYED" badge on fetch log rows (all 3 render sites); Replay Manager button in toolbar; state management forreplayEntriesReplayManagerPopover.tsx(new, 397 lines) — Full Replay Manager dialog with domain→endpoint→request tree navigation (left panel) and response inspector with replay toggle (right panel)components/ui/switch.tsx(new) — Radix-based Switch component for replay toggleApiKeysDialog.tsx— Suppresses password manager autofill on API key inputs (quality-of-life fix)Deviations from the plan
Implemented as planned
RecordReplay<K,V>store,RequestKey(SHA-256),RecordedResponse,ReplayResponseBody,ReplayHttpwrapper insys_opsArc<AtomicU64>fetch ID allocatorset_pinned()/get_pinned(), replay-hit path insend(),ToggleReplay/RequestReplayStateWebSocket protocol#7c3aedstyling at all 3 render sitesReplayHttpwrappingWasmHttp,toggleReplay/replayStateonBamlWasmRuntime,replay_notifycallbackDeviations/surprises
RecordedEntryhas arecorded_attimestamp (usingweb-timefor WASM compat) — not in plan but needed for the "time ago" display in the Replay Manageron_replaycallback pattern for broadcasting replay-hit fetch log events — plan didn't specify how replay hits would appear in the fetch log, this approach keepsReplayHttpdecoupled from any specific transportAdditions not in plan
RequestDisplayInfoand snapshot types (ReplayGroupSnapshot,RecordingSnapshot) for the popover data modelSwitchUI componentApiKeysDialog.tsxpassword-manager suppression (unrelated fix)Items planned but not implemented
FetchLogRowcomponent extraction — plan called for refactoring the 3 identical fetch log render sites into a shared component; this was deferred (REPLAYED badge was added inline at all 3 sites)How to verify it
Setup
Manual Testing (VS Code / LSP path)
Manual Testing (promptfiddle.com / WASM path)
toggleReplayandreplayStatework through the Web WorkerAutomated Tests
Description for the changelog
Add record-replay system to the playground: HTTP responses are automatically recorded during function runs, and users can pin recorded responses via the new Replay Manager dialog for instant replay on subsequent runs — eliminating redundant LLM calls during prompt iteration.
Summary by CodeRabbit
New Features
Improvements