From cc7047d05b316e15245116ec54450f23db5ebb09 Mon Sep 17 00:00:00 2001 From: remi Date: Wed, 14 Jan 2026 23:58:12 +0100 Subject: [PATCH 1/3] Listing possible improvements based on paper & GH issue --- todo.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 todo.md diff --git a/todo.md b/todo.md new file mode 100644 index 0000000..c4d24a2 --- /dev/null +++ b/todo.md @@ -0,0 +1,3 @@ +- [ ] come up w/ more effective approach for scaling training turns +- [ ] reduce reliance on external ground-truth verifiers to provide reward signals during training +- [ ] check for model collapse \ No newline at end of file From ca59e530e3ecdbcbcc05f74dbe3134cbc99fbc3a Mon Sep 17 00:00:00 2001 From: remi Date: Fri, 16 Jan 2026 00:48:46 +0100 Subject: [PATCH 2/3] More potential improvements as shown in the paper --- todo.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/todo.md b/todo.md index c4d24a2..7a9d1fe 100644 --- a/todo.md +++ b/todo.md @@ -1,3 +1,6 @@ -- [ ] come up w/ more effective approach for scaling training turns +- [ ] come up w/ more effective approach for scaling # of training and inference turns - [ ] reduce reliance on external ground-truth verifiers to provide reward signals during training -- [ ] check for model collapse \ No newline at end of file +- [ ] check for model collapse +- [ ] investigate scalability of PAG to larger models (e.g., 32B/70B parameters) +- [ ] integration with advanced reasoning techniques, such as long chain-of-thought +- [ ] RL algorithms for multi-turn training besides PPO (GRPO, DAPO, VAPO; REINFORCE was unstable) \ No newline at end of file From 3482ef84083f2ca89d4bfc3004f902b2b0fd845f Mon Sep 17 00:00:00 2001 From: remi Date: Fri, 16 Jan 2026 00:49:01 +0100 Subject: [PATCH 3/3] More potential improvements as shown in the paper --- todo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/todo.md b/todo.md index 7a9d1fe..df2712a 100644 --- a/todo.md +++ b/todo.md @@ -1,4 +1,4 @@ -- [ ] come up w/ more effective approach for scaling # of training and inference turns +- [ ] come up w/ more effective approach for scaling # of training and inference turns (diminishing returns) - [ ] reduce reliance on external ground-truth verifiers to provide reward signals during training - [ ] check for model collapse - [ ] investigate scalability of PAG to larger models (e.g., 32B/70B parameters)