Skip to content

Add w3-router predictions - #202

Open
Orange1640hahaha wants to merge 2 commits into
RouteWorks:mainfrom
Orange1640hahaha:submit/w3-router
Open

Add w3-router predictions#202
Orange1640hahaha wants to merge 2 commits into
RouteWorks:mainfrom
Orange1640hahaha:submit/w3-router

Conversation

@Orange1640hahaha

Copy link
Copy Markdown

No description provided.

@Orange1640hahaha

Copy link
Copy Markdown
Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: w3-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7817
Accuracy 78.55%
Total Cost $0.567731
Avg Cost per Query $0.000068
Avg Cost per 1K Queries $0.0676
Number of Queries 8400
Abnormal Entries 11
Robustness Score 0.3048

⚠️ 11 of 8400 queries (0.1%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.


Evaluation completed by RouterArena automated workflow

@Orange1640hahaha

Copy link
Copy Markdown
Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: w3-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7814
Accuracy 78.56%
Total Cost $0.588048
Avg Cost per Query $0.000070
Avg Cost per 1K Queries $0.0700
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.3048

Evaluation completed by RouterArena automated workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant