Înapoi la cercetare

Echipa Spinrun · 3 octombrie 2026

Designing a fair model comparison for agent work

A rubric for comparing models on the work your team needs, without inventing benchmark results.

Acest articol este disponibil în engleză.

Start with a task set

Select tasks that represent the actual workflows your team wants to delegate. Include known successful cases, missing-data cases, and a case that must stop for approval. Freeze the source material so each model sees the same information.

Keep the harness constant

Use the same tool schemas, access rules, runtime behavior, and output requirements. Record model configuration and any retries. A change in available context or permission can dominate a model difference and make an apparently precise score misleading.

Score observable outcomes

A reviewer should check correctness, source attribution, appropriate actions, and explicit handling of uncertainty. Add a criterion for stopping at the required review boundary. Measure preparation, review, and corrections together rather than timing only the first response.

Record cost and variance

Run enough representative examples to observe inconsistency. Record input size, tool activity, latency, and billed usage under the current plan. Report the spread of results and the failure cases alongside any average.

Report only what was tested

This page provides a method, not a published Spinrun score for GPT-5.6 or Claude. A responsible comparison names the task set, configuration, dates, and limitations. Check current model availability in your workspace before applying the method.

Înainte să se încarce ceva

Am vrea să activăm analiza traficului, ca să vedem ce pagini merită păstrate. Nu s-a încărcat încă nimic și nu se va încărca nimic până nu alegi. Cookie-urile care te țin autentificat și îți rețin limba nu fac parte din asta.

Citește Politica de cookie-uri