Skip to content

Migrate healthcare-assistant demo app from galileo to splunk-ao SDK - #735

Closed
etserend wants to merge 8 commits into
splunk:mainfrom
etserend:HYBIM-977-healthcare-assistant-splunk-ao-migration
Closed

Migrate healthcare-assistant demo app from galileo to splunk-ao SDK#735
etserend wants to merge 8 commits into
splunk:mainfrom
etserend:HYBIM-977-healthcare-assistant-splunk-ao-migration

Conversation

@etserend

@etserend etserend commented Aug 14, 2026

Copy link
Copy Markdown

Summary

  • Migrates all three `healthcare-assistant` workshop variations (`2-app-with-instrumentation`, `3-app-with-experiments`, `4-app-with-controls`) from the `galileo` SDK to `splunk-ao`
  • Updates credentials (`GALILEO_` → `SPLUNK_AO_`), callbacks (`GalileoAsyncCallback` → `SplunkAOAsyncCallback`), logger (`GalileoLogger` → `SplunkAOLogger`), context manager (`galileo_context` → `splunk_ao_context`), and k8s secret references (`galileo-secret` → `splunk-ao-secrets`)
  • Updates Agent Control SDK deps (`agent-control-sdk[galileo]` → `agent-control-sdk[splunk-ao]`) and API key header (`Galileo-API-Key` → `Splunk-AO-Key`) in 4-app
  • Updates all corresponding workshop content pages (en + ja) with new package names, credential names, image references, and console UI steps
  • Adds `OPENAI_EMBEDDING_BASE_URL` support in `rag.py` and `k8s.yaml` so Azure deployments that use separate chat/embedding endpoints work correctly
  • Patches both sync and async OpenAI clients in `app.py` to inject `api-version` query param required by Azure APIM
  • Wraps hallucination demo retriever+llm spans in a workflow span so the Splunk AO console shows a single named root trace instead of two orphan spans (2-app and 3-app)

Validation — Phase 1 (Galileo SDK baseline) ✅

Ran full test session against Galileo staging REST via `_validate_galileo.py`:

  • Lisinopril RAG query → 24 spans, full retriever tree with real context docs
  • Patient P001 SQL query → 10 spans, SQL generation LLM call captured
  • Hallucination demo → 2 flat root spans (retriever + llm), as expected for Galileo SDK

Validation — Phase 2 (splunk-ao SDK → Galileo staging REST) ✅

Ran full test session via Streamlit UI at localhost:8540:

  • All three interactions confirmed working end-to-end
  • Span structure matches Phase 1: no spans dropped, retriever output populated
  • Naming differences are systematic OTel verb prefixes (`invoke_workflow`, `execute_tool`, `retrieval`) — cosmetic only
  • Hallucination demo now shows a single `invoke_workflow Hallucination Demo` root — improvement over Phase 1

Test plan

  • `2-app-with-instrumentation` starts and traces appear in Splunk AO console (validated Phase 2)
  • RAG query returns answers from knowledge base with retriever spans populated
  • SQL query returns patient data with SQL-generation LLM span captured
  • Hallucination demo logs successfully with workflow root span
  • `3-app-with-experiments` dataset creation and experiment run complete with `SplunkAOEvaluators` metrics
  • `4-app-with-controls` Agent Control initializes on startup, blocking/steering controls work
  • Workshop content pages render correctly (no broken image refs, correct code snippets)

Closes HYBIM-977

@etserend etserend closed this Aug 14, 2026
…pp and 3-app

- app.py: patch both sync and async OpenAI clients to inject api-version query
  param (Azure APIM requires it; openai 3.x non-Azure client omits it)
- rag.py: honour OPENAI_EMBEDDING_BASE_URL so embeddings route to the correct
  Azure deployment instead of the chat deployment (which returns 400)
- hallucination_helpers.py (2-app, 3-app): wrap retriever+llm spans in a
  workflow span so the Galileo console shows "Hallucination Demo" as a single
  root trace instead of two orphan spans
- k8s.yaml: add OPENAI_EMBEDDING_BASE_URL, SPLUNK_AO_PROJECT, SPLUNK_AO_AGENT_STREAM;
  fix secret names (openai-api→openai-secrets, splunk-ao-secret→splunk-ao-secrets);
  remove splunk-ao-config ConfigMap ref

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@etserend etserend reopened this Aug 31, 2026
…1y support

- config.py: add create_chat_llm() and create_embeddings() factory functions that
  return AzureChatOpenAI/AzureOpenAIEmbeddings when AZURE_OPENAI_ENDPOINT is set,
  otherwise fall back to ChatOpenAI/OpenAIEmbeddings — replaces the monkey-patch
- agent.py: use create_chat_llm(); wrap start_session in try/except so O11y CRUD
  failures (no session write permission) are non-fatal
- app.py: remove Azure monkey-patch (now in create_chat_llm); pass only latest user
  message per trace; use splunk_ao_context directly for hallucination existing_logger
- rag.py: use create_embeddings()/create_chat_llm() from config; remove inline
  OPENAI_EMBEDDING_BASE_URL handling (subsumed by factory)
- hallucination_helpers.py: call set_session() when reusing existing logger so
  O11y deployments that lack CRUD token don't fail on start_session()
- setup_env.py: accept either O11y (REALM+TOKEN) or standalone (API_KEY+CONSOLE_URL)
  credential sets; remove OPENAI_BASE_URL from required (not needed for Azure)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@dmitchsplunk dmitchsplunk self-assigned this Aug 31, 2026
etserend and others added 3 commits August 31, 2026 15:22
…bility

Plain ChatOpenAI in generate_sql() was routing through OPENAI_BASE_URL (Azure
chat deployment) without the api-version param, returning 404. Switching to
create_chat_llm() picks up AZURE_OPENAI_ENDPOINT when set so SQL generation
works on both Azure and standard OpenAI deployments.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
start_session(external_id=) requires a CRUD POST which returns 403 on
ingest-only O11y tokens — silently failing and generating a new SDK
session ID per query instead of using the shared UUID from Streamlit
session state. Passing session_id= to the __call__ uses set_session()
locally, no network call, and correctly groups all traces under one
session across multiple user turns.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…pp-with-instrumentation only

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@etserend etserend closed this Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants