LangRails

Unified LLM provider interface for Go. One API, many providers.

import "github.com/promptrails/langrails/llm"

provider := llm.MustNew(llm.OpenAI, "sk-...")
resp, _ := provider.Complete(ctx, &langrails.CompletionRequest{
    Model:    "gpt-4o",
    Messages: []langrails.Message{{Role: "user", Content: "Hello!"}},
})
fmt.Println(resp.Content)

Switch providers by changing one constant:

provider := llm.MustNew(llm.Anthropic, "sk-ant-...")  // or Gemini, Ollama, ...

Install

go get github.com/promptrails/langrails

Features

• 25 providers — OpenAI, Anthropic, Gemini, DeepSeek, Groq, Fireworks, xAI, OpenRouter, Together, Mistral, Cohere, Perplexity, Ollama, Chutes AI, Z.AI, Moonshot (Kimi), Novita AI, DeepInfra, Friendli AI, Cerebras, SambaNova, Hyperbolic, Alibaba DashScope (Qwen), Hugging Face Router, Amazon Bedrock (Converse API)
• Streaming — Channel-based, idiomatic Go
• Tool calling — Unified interface + automatic tool execution loop, with ToolChoice control
• Reasoning — Provider-agnostic ReasoningEffort (minimal/low/medium/high), reasoning text + token accounting
• Web search & citations — Provider-native search (ServerTools) with unified Citations in the response
• Prompt caching — CacheControl + cached/cache-creation token reporting
• Agents — Middleware-driven tool-calling loop (summarization, PII redaction, human-in-the-loop)
• Chain — Sequential multi-step prompt pipelines
• Graph — LangGraph-style stateful workflows: parallel fan-out, subgraphs, streaming, durable execution
• MCP — Model Context Protocol client
• A2A — Agent-to-Agent protocol client + server
• Structured output — JSON schema + JSON mode across all providers
• Vision / Multimodal — Images + text in messages (all providers)
• Prompt templates — Jinja-style {{ variable }} syntax
• Memory — Conversation history with token limits
• Retry & Fallback — Composable resilience decorators
• Zero dependencies — Only Go standard library

Documentation

Getting Started	Installation, first request, error handling
Providers	All providers, config examples
Parameters	All parameters, provider support matrix
Streaming	Real-time token streaming
Reasoning	Reasoning effort, thinking output, tokens
Web Search & Citations	Provider-native search + citations
Prompt Caching	Cache control + cached-token reporting
Vision / Multimodal	Images + text in messages
Structured Output	JSON schema + JSON mode
Prompt Templates	Jinja-style variable substitution
Memory	Conversation history management
Tool Calling	Function calling + automatic tool loop
Agents (Middleware)	Middleware agent loop + built-in middleware
Chain	Sequential prompt pipelines
Graph	Stateful workflows, fan-out, subgraphs, streaming
Durable Execution	Checkpointing, resume, time travel
MCP	Model Context Protocol integration
A2A	Agent-to-Agent protocol client + server
Retry & Fallback	Resilience patterns

Full docs with search: promptrails.github.io/langrails

License

MIT — PromptRails

Part of the PromptRails AI Toolkit

• LangRails — Unified LLM provider
• GuardRails — Content safety scanning
• MemoryRails — Agent memory
• MediaRails — AI media generation
• Go AI Toolkit — Demo app