Route GPT-6 Luna to Sol: Check Errors and Total Cost

A Luna-first, Sol-second workflow is useful only if its accepted results meet your error limits and its complete cost beats the alternative. Counting the cheaper first call alone misses the second model, retries and human handling. This guide builds the measurement plan before claiming savings.

The Luna and Sol documentation was checked on September 24, 2026. Choosing these models as first-pass and escalation candidates is an engineering hypothesis, not a measured quality ranking. Our downloadable lab contains authored routing records; it makes no live API calls.

Start with an observable acceptance rule

Use the extraction task from the Structured Outputs tutorial: classify a ticket and copy an order ID and supporting quote. The first response must be complete, parseable, conform to the schema and satisfy source-grounding checks. Passing those checks makes the answer eligible for further review; it does not prove the category is right.

Define three actions explicitly: accept, escalate to Sol, or send to a person. Do not make every failure another model request. A source missing essential information may need clarification. A request involving an action such as a refund needs a separately authorized workflow, even when its category is clear.

Keep rules versioned. If you revise a threshold halfway through an evaluation, retain the earlier version and rerun the held-out set rather than combining incompatible results.

Inspect the records that were accepted

Teams often review escalations because they look difficult. The quieter failure is an incorrect answer that passes the checks and never gets escalated. That is where a cheap first pass can conceal business risk.

Sample accepted records independently of their confidence scores. Track incorrect accepted records over all accepted records, and also over the entire workload. Report the sample size and sampling method. A tiny, handpicked sample cannot establish a production error rate. Pay attention to rare but expensive categories and to language-specific errors.

The model’s statement that it is confident is not an independently calibrated probability. If you use a score, fit and assess thresholds on labeled data with a separate held-out set. A score produced by the same system that made the mistake should not be its only reviewer.

Count the whole route

For each original ticket, collect the first call, any Sol call, failed retries, external tools and human handling. Use actual usage records and applicable vendor prices for the correct provider and date. Do not replace invoice history with a new price table.

route cost = first-pass cost + escalation cost
           + retry/tool cost + human handling cost

Compare total cost per completed acceptable task, not only mean token price. Keep unresolved cases visible. If one workflow drops difficult inputs, it can appear cheaper without delivering the same service. Human handling needs a stated time-and-cost assumption or should be shown separately in minutes; silently setting it to zero distorts the result.

Run the authored cost counterexample

Unzip the lab and run:

python3 lab.py routing

The four records use arbitrary cost units. They are not dollar prices, not model outputs, and not a measured savings estimate.

The total is 17 units, versus a hypothetical all-Sol token-only baseline of 16. The baseline has no measured quality or human-handling cost and is deliberately incomplete; it cannot select a winner. The example shows why four cheap first calls do not establish savings. Two records were accepted and one was wrong, but those authored labels are a logic demonstration, not an observed 50% error rate for Luna.

Make the all-Sol baseline comparable

Run the same held-out inputs with the same extraction policy, output schema and adjudication rules. Fix the provider, model IDs, effort settings, request budget and available tools, while documenting unavoidable differences. Randomize ordering when service conditions could affect latency, and record wall-clock time rather than adding only model-reported durations.

Count baseline retries and human review just as you count them for the routed system. Compare quality by category, cost per accepted task, unresolved work, and latency percentiles with sample sizes. Several repeated runs can show instability; they do not automatically make a small sample representative.

Only choose a threshold after its error-cost tradeoff meets a requirement you set in advance. For an initial pilot, manual approval may remain the appropriate endpoint. Add automation gradually when the evidence supports it. Use the Sol pricing explanation and Luna pricing explanation to interpret usage, not to substitute for the task-level experiment.