Your agent harness can change a rollout's training weight
Agent Lightning shows how one task can become several training rows. Preserve rollout lineage so incidental fragmentation does not silently redefine the optimisation objective.
Only 36% of rollouts remained a single training sample in the coding-agent experiment reported in Agent Lightning v1.0. The average rollout produced 2.41 samples.
That makes a training row a poor default synonym for a task. A harness can branch into subagents, summarise context or break token-prefix continuity, leaving one task represented by several pieces when the trainer receives it.
My working rule is to make the intended unit of influence explicit before optimising the batch. If the objective is defined around task outcomes, an incidental increase in rows should not quietly change how much a task counts.
The reward can stay fixed while the baseline moves
The paper offers a small example. Two rollouts receive rewards of one and zero. Their mean reward is one half. If the successful rollout becomes three samples and each inherits its reward, averaging over samples gives three quarters instead.
No task improved. The representation changed the average. The authors also examine loss normalisation and favour a rollout-level approach in their experiments. That is a design choice supported by their study, not a guarantee that every training objective should weight every task identically.
For a practitioner, the important question is whether the weighting matches the objective intentionally. A system may choose to emphasise certain tasks or tokens. It should not get that emphasis accidentally because one harness operation happened to split a trajectory.
As an illustration, I would inspect a hypothetical coding-agent training batch before launching an expensive run. Select several tasks with different numbers of fragments. For each, trace the final reward, the advantage calculation and the contribution to the normalised loss.
Then I would perform a controlled representation check on a small synthetic fixture. Keep the underlying task outcomes and token losses fixed while varying a permitted grouping of those tokens into rows. The expected invariance depends on the chosen objective, so I would write that expectation down before looking at the result.
Keep lineage through the batching boundary
A rollout identifier should survive flattening into training samples. So should the grouping used to compare alternative rollouts from the same prompt. Once that relationship is lost, a later reader cannot reliably explain whether a weighting change came from behaviour or sample construction.
I would monitor fragment counts alongside task outcomes when changing the harness. A new summarisation policy might improve execution while also changing sample shape. Treating the model and reward as unchanged is not enough to call the training configuration unchanged.
The same caution applies to reconstructed text. The paper explains that retokenising an apparently unchanged string can produce different token boundaries. Training records should preserve what the model actually consumed rather than assuming a later reconstruction is equivalent.
These checks belong before a large benchmark comparison. If two runs differ in their effective weighting, a higher score may still be real, but the team needs to describe what experiment produced it. Otherwise an implementation change can be mistaken for an improvement in the learning recipe.
I have not reproduced the paper's training results. The contribution I would borrow first is its accounting question: how does a task become data, and how does that data acquire influence over an update?
That question is inexpensive to ask on a small batch. Discovering the answer after a long training run is a much less pleasant way to learn it.
Preserve task lineage through training; a harness-generated fragment should not acquire extra influence by accident.
