Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
[flake8]
max-line-length = 88
exclude = .git,.github,.chglog,__pycache__,docs,venv,env,mypy_cache
max-complexity = 10
max-complexity = 10
extend-ignore = E203
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,24 @@ For more detailed examples and integration patterns, check out:

---

## Streaming

The SDK auto-detects the response wire format from `Content-Type`:

- `application/vnd.amazon.eventstream` — AWS Bedrock event-stream framing (used by `invoke-with-response-stream` and `converse-stream`). Frames are parsed via `botocore.eventstream`; Bedrock's `{"bytes": "<base64-json>"}` envelope is unwrapped automatically.
- `text/event-stream` / `application/x-ndjson` — SSE (OpenAI, Anthropic direct, Gemini).

Each route configures a `stream_response_path` (JSONPath) that extracts the per-chunk text from the model's native chunk schema. Common values:

| Model family | `stream_response_path` |
| ------------------------ | -------------------------- |
| Anthropic Claude | `delta.text` |
| Meta Llama (Bedrock) | `generation` |
| Cohere Command (Bedrock) | `text` |
| OpenAI chat | `choices[0].delta.content` |

If `stream_response_path` doesn't match a chunk, the SDK logs a single structured WARNING with the payload keys it saw — check that warning against the `ModelSpec` on the gateway.

## Type Hints & py.typed Marker

This package includes a `py.typed` marker file, which indicates to type checkers (like `mypy`, `pyright`, `pylance`) that the package supports type checking. This allows IDEs and static analysis tools to provide better autocomplete, type checking, and refactoring support.
Expand Down
Loading