Skip to content

Actions: EleutherAI/lm-evaluation-harness

Actions

All workflows

Actions

Loading...
Loading

Showing runs from all workflows
2,500+ workflow runs
2,500+ workflow runs

Filter by Workflow

⌥ + click/return to exclude

Filter by Event

⌥ + click/return to exclude

Filter by Status

⌥ + click/return to exclude

Filter by Branch

⌥ + click/return to exclude

Filter by Actor

⌥ + click/return to exclude
fix: predict all CoQA turn answers instead of only the last turn Tasks Modified #6175: Pull request #3704 opened by rahulraj-jhawar-devrev
Action required rahulraj-jhawar-devrev:rahulraj/contributions
rahulraj-jhawar-devrev:rahulraj/contributions
Action required
fix: predict all CoQA turn answers instead of only the last turn Unit Tests #6148: Pull request #3704 opened by rahulraj-jhawar-devrev
Action required rahulraj-jhawar-devrev:rahulraj/contributions
rahulraj-jhawar-devrev:rahulraj/contributions
Action required
Add ifeval_pt to harness Unit Tests #6147: Pull request #3622 synchronize by Nkluge-correa
Action required Polygl0t:ifeval_pt
Polygl0t:ifeval_pt
Action required
Add ifeval_pt to harness Tasks Modified #6174: Pull request #3622 synchronize by Nkluge-correa
Action required Polygl0t:ifeval_pt
Polygl0t:ifeval_pt
Action required
Fix BigBench multiple-choice crash on mixed-format tasks Unit Tests #6146: Pull request #3702 opened by Chessing234
Action required Chessing234:fix/bigbench-mc-filter
Chessing234:fix/bigbench-mc-filter
Action required
Fix BigBench multiple-choice crash on mixed-format tasks Tasks Modified #6173: Pull request #3702 opened by Chessing234
Action required Chessing234:fix/bigbench-mc-filter
Chessing234:fix/bigbench-mc-filter
Action required
fix: add ray as dependency of lm-eval[vllm] Tasks Modified #6172: Pull request #3701 opened by Anai-Guo
Action required Anai-Guo:fix-vllm-ray-dependency
Anai-Guo:fix-vllm-ray-dependency
Action required
fix: add ray as dependency of lm-eval[vllm] Unit Tests #6145: Pull request #3701 opened by Anai-Guo
Action required Anai-Guo:fix-vllm-ray-dependency
Anai-Guo:fix-vllm-ray-dependency
Action required
fix: don't pass task stop sequences to vLLM for reasoning models Unit Tests #6144: Pull request #3700 opened by jwmacd
Action required jwmacd:fix/reasoning-stop-sequences
jwmacd:fix/reasoning-stop-sequences
Action required
fix: don't pass task stop sequences to vLLM for reasoning models Tasks Modified #6171: Pull request #3700 opened by jwmacd
Action required jwmacd:fix/reasoning-stop-sequences
jwmacd:fix/reasoning-stop-sequences
Action required
Adding Cruxeval Tasks Modified #6170: Pull request #3699 opened by ThomasHeap
Action required ThomasHeap:main
ThomasHeap:main
Action required
Adding Cruxeval Unit Tests #6143: Pull request #3699 opened by ThomasHeap
Action required ThomasHeap:main
ThomasHeap:main
Action required
Fix median aggregation returning arbitrary element instead of median Unit Tests #6142: Pull request #3696 opened by Chessing234
Action required Chessing234:fix/median-missing-sort
Chessing234:fix/median-missing-sort
Action required
Fix median aggregation returning arbitrary element instead of median Tasks Modified #6169: Pull request #3696 opened by Chessing234
Action required Chessing234:fix/median-missing-sort
Chessing234:fix/median-missing-sort
Action required
Fix acc_all_stderr grouping by question_id only (drops paragraph_id) Unit Tests #6141: Pull request #3695 opened by Chessing234
Action required Chessing234:fix/acc-all-stderr-paragraph-key
Chessing234:fix/acc-all-stderr-paragraph-key
Action required
Fix acc_all_stderr grouping by question_id only (drops paragraph_id) Tasks Modified #6168: Pull request #3695 opened by Chessing234
Action required Chessing234:fix/acc-all-stderr-paragraph-key
Chessing234:fix/acc-all-stderr-paragraph-key
Action required
[Feature] tool_calls and reasoning: Tracking and evaluation Unit Tests #6140: Pull request #3685 synchronize by RawthiL
2m 14s pnyxai:feature-tool-calls-evaluation
pnyxai:feature-tool-calls-evaluation
2m 14s
[Feature] tool_calls and reasoning: Tracking and evaluation Tasks Modified #6167: Pull request #3685 synchronize by RawthiL
38s pnyxai:feature-tool-calls-evaluation
pnyxai:feature-tool-calls-evaluation
38s
fix(vllm): add ray to vllm extras (#3688) Tasks Modified #6166: Pull request #3694 opened by kvr06-ai
Action required kvr06-ai:fix/3688-ray-vllm-extras
kvr06-ai:fix/3688-ray-vllm-extras
Action required
fix(vllm): add ray to vllm extras (#3688) Unit Tests #6139: Pull request #3694 opened by kvr06-ai
Action required kvr06-ai:fix/3688-ray-vllm-extras
kvr06-ai:fix/3688-ray-vllm-extras
Action required
Fix mmlu_pro fewshot answers leaking into user role under chat template Unit Tests #6138: Pull request #3693 opened by kiwaku
Action required kiwaku:fix/mmlu-pro-chat-template-fewshot
kiwaku:fix/mmlu-pro-chat-template-fewshot
Action required
Fix mmlu_pro fewshot answers leaking into user role under chat template Tasks Modified #6165: Pull request #3693 opened by kiwaku
Action required kiwaku:fix/mmlu-pro-chat-template-fewshot
kiwaku:fix/mmlu-pro-chat-template-fewshot
Action required
[Feat] Add native Tensor Parallelism support for HF backend Tasks Modified #6164: Pull request #3692 synchronize by YangKai0616
Action required YangKai0616:hf-tp
YangKai0616:hf-tp
Action required
[Feat] Add native Tensor Parallelism support for HF backend Unit Tests #6137: Pull request #3692 synchronize by YangKai0616
Action required YangKai0616:hf-tp
YangKai0616:hf-tp
Action required
[Feat] Add native Tensor Parallelism support for HF backend Tasks Modified #6163: Pull request #3692 opened by YangKai0616
Action required YangKai0616:hf-tp
YangKai0616:hf-tp
Action required