An observation needs a boundary
A group of agents can appear coordinated without solving the right problem. Define the task, shared resources, and allowed actions before interpreting the result. Keep a record of which instruction each participant received and which state it could see. This article outlines an evaluation method; it does not claim a Spinrun swarm benchmark.
Make ownership explicit
Give each part of the work a concrete owner and define how results are handed off. If two participants can change the same record, decide who has authority to write. A shared goal does not by itself establish mutual exclusion or prevent duplicate external actions.
Separate messages from state
A participant reporting completion is an observation, not a committed change in every connected system. Record the external result and the message independently. Coordination is easier to assess when the experiment can tell whether a claim was made, a task was attempted, and its effect was confirmed.
Test interruption and recovery
Interrupt a participant after it has read a source and before it reports completion. Observe whether the remaining system can identify the last confirmed action. Repeat with a provider timeout after a possible write. These cases reveal whether recovery duplicates work or preserves a useful audit trail.
Evaluate the output
Judge the group's final result against the same standard as a single-agent run. Include review effort and the cost of reconciling contradictory answers. More participants are useful when they improve the result enough to justify that coordination work.
A reproducible record
Keep the initial task, source snapshot, participant configuration, tool results, approval decisions, and reviewer rubric. Publish only data you have permission to share. An experiment can then support a bounded conclusion about the task it actually tested.