Skip to content

feat(observability): add LangSmith as alternative tracing backend - #23

Merged
ngoclam9415 merged 1 commit into
developfrom
feat/langsmith-tracing-backend
Jul 7, 2026
Merged

feat(observability): add LangSmith as alternative tracing backend#23
ngoclam9415 merged 1 commit into
developfrom
feat/langsmith-tracing-backend

Conversation

@ngoclam9415

Copy link
Copy Markdown
Contributor

What

Adds LangSmith as an alternative tracing backend for @observable, mirroring the existing Langfuse integration. ONE backend active per process; selection at decoration time: LangSmith > Langfuse > no-op. No call-site changes — all 30+ @observable sites (agents, workflows, LLM calls, codecs) traced automatically.

Why

Operators want LangSmith as an option alongside Langfuse. Both are OTLP-capable, but a parallel backend behind the existing decorator is lower-risk than an OTLP pivot and matches the established LANGFUSE_ENABLED toggle pattern.

How to enable

pip install dana[observability]
export LANGSMITH_TRACING=true LANGSMITH_API_KEY=lsv2_...
# optional: LANGSMITH_PROJECT, LANGSMITH_ENDPOINT
# alias: DANA_LANGSMITH_ENABLED=true|1|yes

Toggle back to Langfuse by unsetting LANGSMITH_TRACING and setting LANGFUSE_ENABLED=true.

Changes

  • dana/common/observable.py — dispatch (LANGSMITH>LANGFUSE>noop), _langsmith_kwargs adapter (as_typerun_type, session_id/user_idmetadata), bare-form normalization (avoids double-call), factored _langfuse_wrap preserving byte-equivalent flush behavior on the langfuse path.
  • dana/__init__/init_environment.py_install_langsmith_shim mirrors the langfuse shim (clean import without the extra).
  • pyproject.tomllangsmith>=0.8,<0.9 in [observability] extra (langfuse pins untouched).
  • tests/unit/test_observable_langsmith_dispatch.py — 11 mocked cases incl. SC7 langfuse flush regression (sync/bare/async) and async exclusivity.
  • docs/codebase-summary.md — tracing-backends section + silent no-op caveat.
  • docs/project-changelog.md — Added + Changed entries.

Verification

  • 11/11 new tests pass; full unit suite 1345 pass, 20 skip, 1 xfail — no regressions.
  • ruff + ruff-format clean; pre-commit hooks pass.
  • Smoke-tested all 3 dispatch modes + shim path against real langsmith==0.8.5.
  • Code-reviewed: langfuse path verified byte-equivalent (flush fires once for sync/bare/async); exclusivity confirmed live.

Scope / Out of scope

  • Phase 1 (this PR): tracing dispatch only.
  • Phase 2 (deferred): LangSmithPromptRepository (prompt management) — separate PR. LangSmith native versioning will also let us drop the dana_versions config-stashing hack currently in LangfusePromptRepository.
  • Deferred follow-ups: symmetric runtime-warning when an env flag is set but its package is missing (currently silent no-op, parity with langfuse); per-call-site run_type="llm" for llm_caller.py spans; process_inputs scrubbing for self (parity with langfuse).

Breaking changes

None. Default behavior (no env set) is unchanged except @observable now returns the decorated function unchanged (identity) on the no-op path instead of a passthrough wrapper — improves inspect.signature(), no introspection-sensitive call sites affected.

@observable dispatches to langsmith.traceable when LANGSMITH_TRACING=true
or DANA_LANGSMITH_ENABLED truthy; exclusive with Langfuse (LangSmith wins).
No call-site changes — all 30+ @observable sites traced automatically.

- observable.py: LANGSMITH>LANGFUSE>noop dispatch, _langsmith_kwargs adapter
  (as_type->run_type, session_id/user_id folded into metadata), bare-form
  normalization to avoid double-call, factored _langfuse_wrap preserves
  byte-equivalent flush behavior on the langfuse path
- init_environment.py: _install_langsmith_shim mirrors the langfuse shim so
  the module imports cleanly without the extra installed
- pyproject.toml: langsmith>=0.8,<0.9 added to [observability] extra
- tests: 11 cases incl. SC7 langfuse flush regression (sync/bare/async) and
  async exclusivity
- docs: tracing-backends section + silent no-op caveat + changelog entry

Enable: pip install dana[observability]; LANGSMITH_TRACING=true LANGSMITH_API_KEY=...
@ngoclam9415
ngoclam9415 merged commit 9a3f5f8 into develop Jul 7, 2026
1 check passed
@ngoclam9415
ngoclam9415 deleted the feat/langsmith-tracing-backend branch July 7, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant