Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions skills/openrouter-analytics-query/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,15 +82,16 @@ cd <openrouter-analytics-skill-path>/scripts && npx tsx query-analytics.ts --met
### Filter Object Shape

```json
{ "field": "<dimension_name>", "operator": "<op>", "value": "<value>" }
{ "field": "<dimension_name>", "operator": "<op>", "value": "<value>", "include_unset": true }
```

- Scalar operators (`eq`, `neq`, `gt`, `gte`, `lt`, `lte`): `value` is a string or number
- Array operators (`in`, `not_in`): `value` is an array of strings or numbers
- `include_unset` is optional and applies only to `in` and `not_in` filters on dimensions with an unset bucket: `api_key_id` (`-1`), `app` (`-1`), and `user` (empty string). With `in`, unset rows are included with the listed values. With `not_in`, unset rows are excluded along with the listed values. An empty `value` array with `in` matches only unset rows. Other dimensions return the validation error `Dimension "<field>" has no unset bucket`.
- Several dimensions are **label-resolved** in query results (returned as human-readable names), but filters must use the underlying ID:
- `api_key_id` — numeric ID (from generation metadata) or 64-char SHA-256 hash (from `GET /api/v1/keys`). Hashes are auto-resolved to numeric IDs before querying.
- `user` — Clerk user ID (e.g. `user_abc123`), not the display name/email shown in results.
- `workspace` — workspace UUID, not the workspace name shown in results.
- `workspace` — workspace UUID, not the workspace name shown in results. Filtering or grouping by the account default workspace also covers activity recorded before workspace resolution existed, which is folded from the legacy all-zero UUID.
- `app` — numeric app ID, not the app title shown in results.
- `model` — permaslug (e.g. `openai/gpt-4o`), not the display name.
- Other dimensions (`provider`, `origin`, `country`, `finish_reason`, `external_user`, etc.) are not enriched — filter values match what's returned in results.
Expand Down Expand Up @@ -190,7 +191,7 @@ Classifier filters narrow results to generations matching specific classificatio

> **Numeric types:** Count metrics (`request_count`, `tokens_*`, etc.) are returned as strings (`"1523"`). Cost and rate metrics (`total_usage`, `cache_hit_rate`, latency, throughput) are returned as numbers (`4.27`). Parse count values with `Number()` or `parseInt()` before arithmetic.

> **Label resolution:** Dimensions `api_key_id`, `app`, `user`, and `workspace` return human-readable labels in data rows (key names, app titles, user names, workspace names), not raw IDs.
> **Label resolution:** Dimensions `api_key_id`, `app`, `user`, and `workspace` return human-readable labels in data rows (key names, app titles, user names, workspace names), not raw IDs. The unset buckets `api_key_id = -1` and `app = -1` resolve to `Chatroom` and `Unknown`. The unresolved-workspace sentinel `ffffffff-ffff-ffff-ffff-ffffffffffff` resolves to `Unattributed` (a different value from the legacy all-zero UUID, which is folded into the default workspace).

## CLI Reference

Expand Down Expand Up @@ -339,5 +340,5 @@ Classifier dimensions and classifier filters always force the 31-day time range
If a query times out, try:
- Narrowing the time range
- Removing latency/throughput metrics
- Removing per-generation dimensions (`provider`, `origin`, `country`, `finish_reason`, etc.)
- Removing per-generation dimensions (`provider`, `origin`, `country`, `data_region`, `finish_reason`, etc.)
- Removing classifier dimensions/filters (they are more expensive to compute)
14 changes: 10 additions & 4 deletions skills/openrouter-analytics-schema/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Each metric has:

### Time Range Limits

Most volume and cost metrics support time ranges up to **365 days** with daily granularity. Latency/throughput metrics and some dimensions (`provider`, `origin`, `country`, `finish_reason`, `external_user`, `context_length_bucket`, `generation_id`) are limited to **31-day** time ranges. If a query times out, try narrowing the time range or removing latency/throughput metrics and per-generation dimensions.
Most volume and cost metrics support time ranges up to **365 days** with daily granularity. Latency/throughput metrics and some dimensions (`provider`, `origin`, `country`, `data_region`, `finish_reason`, `external_user`, `context_length_bucket`, `generation_id`, `session_id`) are limited to **31-day** time ranges. If a query times out, try narrowing the time range or removing latency/throughput metrics and per-generation dimensions.

### Metric Categories

Expand Down Expand Up @@ -95,6 +95,7 @@ Most volume and cost metrics support time ranges up to **365 days** with daily g

**Efficiency metrics** (how well):
- `cache_hit_rate` — ratio of cached tokens to prompt tokens (0–1)
- `blended_cost_per_million_tokens` — total spend, including BYOK inference, divided by prompt plus completion tokens and multiplied by 1M. Currency rate metric in USD per 1M tokens (up to 365 days). NULL when there are no tokens.
- `guardrail_invoked_rate` — ratio of requests that triggered guardrails
- `response_cached_rate` — ratio of responses served from cache

Expand Down Expand Up @@ -122,7 +123,7 @@ Some dimensions have their raw IDs automatically resolved to human-readable labe
| `user` | User name or email address |
| `workspace` | Workspace name |

All other dimensions (e.g., `model`, `provider`, `country`) are returned as-is without resolution.
The `api_key_id` value `-1` resolves to `Chatroom`, `app` value `-1` resolves to `Unknown`, and the unresolved-workspace sentinel `ffffffff-ffff-ffff-ffff-ffffffffffff` resolves to `Unattributed` (distinct from the legacy all-zero UUID, which is folded into the default workspace). All other dimensions (e.g., `model`, `provider`, `country`) are returned as-is without resolution.

> Rows with an empty `user` value represent traffic not attributed to a specific org member (e.g., API keys created at the org level).

Expand All @@ -133,17 +134,19 @@ All other dimensions (e.g., `model`, `provider`, `country`) are returned as-is w
- `variant` — model variant (e.g., standard, extended)
- `api_key_id` — which API key made the request
- `user` — the creator user ID (for org-level queries)
- `workspace` — workspace ID
- `workspace` — workspace ID. Filtering or grouping by the account default workspace also covers activity recorded before workspace resolution existed, which is folded from the legacy all-zero UUID. Grouping does not return the all-zero UUID as a separate group.
- `app` — application ID

**Limited to 31-day time ranges:**
- `generation_id` — unique ID for each generation (use to drill down to individual requests, then inspect via the `openrouter-generations` skill)
- `provider` — upstream provider name
- `origin` — request origin/source
- `country` — request country
- `data_region` — request data region (`global`, `europe`, or `us`). Rows predating the default are reported as `global`.
- `finish_reason` — why the generation ended (stop, length, etc.)
- `external_user` — custom user ID passed by the caller
- `context_length_bucket` — bucketed context length (1K, 10K, 100K, etc.)
- `session_id` — session ID. Sessionless requests are returned as the literal `none`.

## Classifier Dimensions

Expand Down Expand Up @@ -178,6 +181,8 @@ Filter operators for the `filters` array in query requests:
| `in` | array | In list |
| `not_in` | array | Not in list |

The optional `include_unset` flag is supported only with `in` and `not_in`. It is valid for `api_key_id` with sentinel `-1`, `app` with sentinel `-1`, and `user` with sentinel empty string. With `in`, it adds unset rows to the matched set. With `not_in`, unset rows are excluded along with the listed values. An empty array with `in` matches only unset rows. Other dimensions fail validation with `Dimension "<field>" has no unset bucket`. The `/meta` response does not indicate which dimensions have an unset bucket.

## Understanding Granularities

Time bucketing for time-series queries:
Expand All @@ -204,6 +209,7 @@ Use this guide to translate natural-language questions into the right metric/dim
| "How many tokens?" | `tokens_total` | — | Use `tokens_prompt` / `tokens_completion` for split |
| "Which provider is fastest?" | `avg_latency`, `p90_latency` | `provider` | 31-day limit |
| "What's my cache hit rate?" | `cache_hit_rate` | `model` | Rate metric — shows per-model caching |
| "What's my effective cost per million tokens?" | `blended_cost_per_million_tokens` | `model` | Rate metric in USD per 1M tokens. Includes BYOK inference spend and is NULL when there are no tokens. |
| "Which API key uses the most?" | `request_count`, `total_usage` | `api_key_id` | — |
| "Usage over time" | `request_count` or `total_usage` | — | Set `granularity: "day"` |
| "Latency trends" | `p90_latency` | — | Set `granularity: "hour"`, 31d limit |
Expand All @@ -229,7 +235,7 @@ Several dimensions are **label-resolved** in query results — the response show
|---|---|---|
| `api_key_id` | Numeric ID **or** 64-char SHA-256 hash | Numeric ID: generation metadata (`api_key_id` field). Hash: `GET /api/v1/keys` (`key_hash` field). Hashes are auto-resolved server-side. If a hash can't be resolved, a sentinel value returns zero rows (no error). |
| `user` | Clerk user ID (e.g. `user_abc123`) | User settings or org member list — not the display name/email shown in results. |
| `workspace` | Workspace UUID | Workspace settings page or `GET /api/v1/workspaces` — not the workspace name shown in results. |
| `workspace` | Workspace UUID | Workspace settings page or `GET /api/v1/workspaces` — not the workspace name shown in results. The account default workspace also matches legacy all-zero UUID rows, which are folded into that workspace for grouping. |
| `app` | Numeric app ID | Generation metadata (`app_id` field) or app settings — not the app title shown in results. |
| `model` | Permaslug (e.g. `openai/gpt-4o`) | Model page URL or `GET /api/v1/models` — not the display name. |

Expand Down
2 changes: 1 addition & 1 deletion skills/openrouter-analytics/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ When interpreting results for the user:
- **Rates** (`cache_hit_rate`) are 0–1 ratios
- **Throughput** (`avg_throughput`) is tokens per second
- When `granularity` is set, rows include a `date__<granularity>` field for the time bucket (e.g., `date__day`, `date__hour`, `date__month`)
- **Label resolution**: dimensions `api_key_id`, `app`, `user`, and `workspace` have their raw IDs replaced with human-readable names (key name, app title, user name, workspace name) directly in the data rows; `generation_id` and `session_id` return raw values
- **Label resolution**: dimensions `api_key_id`, `app`, `user`, and `workspace` have their raw IDs replaced with human-readable names (key name, app title, user name, workspace name) directly in the data rows; `api_key_id = -1` resolves to `Chatroom`, `app = -1` resolves to `Unknown`, and the unresolved-workspace sentinel `ffffffff-ffff-ffff-ffff-ffffffffffff` resolves to `Unattributed` (distinct from the legacy all-zero UUID, which is folded into the default workspace); `generation_id` and `session_id` return raw values
- **Truncation**: when consuming output programmatically, check `metadata.truncated`. If `true`, the result was capped at `--limit` and is a *partial* dataset — raise `--limit` or paginate before reporting totals or rankings

### Cost Optimization Guidance
Expand Down