Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning

Let an AI judge point at the key moves, then test them with fresh replays before rewarding them.

9.91%

Relative gain over GRPO

Average across ALFWorld, WebShop, SearchQA with Qwen3.5-2B

How it works

Credit spread evenly
GRPO gives every token in a trajectory the same reward signal, so key decisions look no different from filler.
Propose, verify, credit
A judge LLM picks a likely pivotal segment; fresh rollouts before and after it measure how much it raises success.
Better agents, cheaply
Positive measured gains boost that segment's advantage, giving the best average scores on three agent benchmarks at both sizes.

From the paper