Issues · Toloka/tolokaforge · GitHub

Labels Milestones

Characterise per_trial cost/latency premium with proper sample size

#164

· CiroGamboa opened

on Jul 3, 2026

Cross-trial prompt-cache reuse within a domain (measure first, then decide)

#158

· azorej opened

on Jul 2, 2026

[design] Library-shaped composition — expose agentic loop, grader, user simulator as independently callable components

#156

· CiroGamboa opened

on Jul 2, 2026

[design] Observability seam for external collectors (OpenTelemetry, Prometheus, Datadog)

#155

· CiroGamboa opened

on Jul 2, 2026

[test] Verify int→float dump does not diverge from the current writer JSON

#153

· CiroGamboa opened

on Jul 2, 2026

·

Typed contracts & decoupling

[test] Assert aggregate.json.schema_version survives the writer path under exclude_unset

#152

· CiroGamboa opened

on Jul 2, 2026

·

Typed contracts & decoupling

[docs] Task-pack image-layering guide (SWE-bench 3-tier pattern for cache-with-isolation)

#146

· CiroGamboa opened

on Jul 2, 2026

[perf] LocalRuntimeBackend: cold-start reduction + hardening (pre-pull, template DB, container pool, orphan sweep, resource caps, benchmark harness)

#145

· CiroGamboa opened

on Jul 2, 2026

[design] EnvironmentManifest: endpoint-resolution flexibility (runner_port, db_service, db_port, rag_service, rag_port)

#144

· CiroGamboa opened

on Jul 2, 2026

[docs] Review ADR practice + write guideline for when to author one

#142

· CiroGamboa opened

on Jul 2, 2026

[refactor] Lift the rubric judge to a Judge Protocol with InMemoryJudge fixture

#131

· CiroGamboa opened

on Jul 1, 2026

[design] Manifest: trusted-output declarations for adversarial-content evaluations

#127

· CiroGamboa opened

on Jul 1, 2026