#model routing

Same Agent Workflow, Three Model Routes: A Real Crazyrouter Benchmark

Dynamic workflows are exciting, but they introduce a practical question: which model should run each step?

If a workflow has a planner, implementer, adversarial reviewer, and verifier, you can route all steps to one strong model. Or you can route different steps to different models.

The wrong answer is to guess.

So we ran a small real benchmark through Crazyrouter.

Test setup

We used the Crazyrouter OpenAI-compatible endpoint:

https://cn.crazyrouter.com/v1

The workflow task was:

Add a CSV export for account billing history with user-level authorization, timezone-safe timestamps, CSV escaping, tests, and rollback notes.

The workflow had four steps:

1. planner

2. implementer

3. adversarial reviewer

4. verifier

We tested three routing policies:

PolicyPlannerImplementerReviewerVerifierall_opus_47claude-opus-4-7claude-opus-4-7claude-opus-4-7claude-opus-4-7all_opus_48claude-opus-4-8claude-opus-4-8claude-opus-4-8claude-opus-4-8routed_47_48claude-opus-4-7claude-opus-4-8claude-opus-4-7claude-opus-4-8

Raw benchmark artifact:

generated/dynamic_workflow_routing_20260603/benchmark_results.json

Results

RouteCallsTotal latencyTotal tokensOutput tokensScoreall Opus 4.74100.939s8,8535,97714/17all Opus 4.8482.598s8,3575,78215/17routed 4.7/4.8485.975s8,6525,87315/17

In this run, all_opus_48 won on latency, total tokens, and score.

That does not mean every workflow should use Opus 4.8 everywhere. It means routing needs evidence.

What the score measured

This was not a generic benchmark. Each workflow step had step-specific checks.

For example:

planner needed affected files, risks, acceptance criteria, tests, rollback;

implementer needed CSV handling, authorization, timestamps, tests;

reviewer needed security, privacy, tests, rollback;

verifier needed commands, tests, evidence, inspection.

The score was a simple keyword-based quality gate. It is not a perfect human evaluation, but it catches whether the output covered required workflow concerns.

Why this matters

Dynamic workflows can multiply model calls.

A simple AI coding request might become:

planner call + implementer call + reviewer call + verifier call + retry calls + patch-fix calls + final summary call

If you always use the most expensive model, cost can grow quickly. If you always use the cheapest model, failures and retries can grow quickly.

The useful metric is not token price.

The useful metric is:

cost and latency per successful workflow

What we learned

1. Opus 4.8 was faster in this workflow

The all-4.8 route finished in 82.598 seconds. The all-4.7 route took 100.939 seconds.

That is an 18.341 second difference for the same four-step workflow.

In a single call, that may not matter. In a background agent system with many workflow steps, it does.

2. Mixed routing was close, but not better here