Feed
Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills
Test agent skills by running the agent with and without them, not just reading them.
0.2134
Mean composite Skill Lift
Skills measurably help live agents across 947 paired cases.
How it works
Scans miss runtime
Skill checks read the document only. Structure and LLM-judge scores barely agree, and neither predicts live usefulness.
Paired live trials
Run the same task twice in a sandbox, with and without the target skill, and grade both traces on six metrics.
Measured skill value
Skills lifted scores in 72.8% of cases, mostly in skill execution, behavior checks and efficiency.
From the paper