How to check an LLM API provider before you trust it

To check an LLM API provider, compare its documented model and routing policy with your request, response metadata and billing record. A model saying “I am GPT” is not proof of which weights handled a request. A cheap price, a familiar model name or one good answer is not sufficient evidence either.

This guide is for developers choosing a gateway for coding agents or production requests. It provides a repeatable evidence checklist; it does not certify Ofox or any competitor. Apply the same standard to every provider.

Start with a documented model ID

Save the model identifier you sent, the API endpoint and the date. Distinguish the original model developer’s name from a gateway’s namespaced ID. An alias can be legitimate, but the provider should document what it resolves to and whether that resolution can change.

Screenshot of the OpenAI model directory, captured September 16, 2026. This is the developer’s directory, not proof of what an unrelated gateway served.

For Ofox, start with the relevant page in the model catalog. Record the selected provider or route if the product exposes it. Do not assume that a field missing from a catalog means a feature is unsupported: endpoint lists, protocol support and pricing fields answer different questions.

Collect evidence at four layers

Keep API keys, authorization headers and private prompts out of screenshots or support tickets. A redacted request ID and a minimal synthetic example are usually a better starting point than uploading a production conversation.

OpenAI’s API reference documents its request and response conventions. A compatible gateway may expose different metadata. Check that gateway’s contract rather than treating a missing OpenAI-specific header as proof of misconduct.

Compare behavior without pretending it proves identity

Use a small task set that represents your application: one structured response, one tool interaction, one long-input task and one difficult task with a known acceptance criterion. Use material you may send to both services. Fix prompts, settings and reference inputs where both routes support them.

Repeat a task if the result matters. Different answers can arise from sampling, instructions, tools, context handling or implementation differences. Conversely, matching answers do not prove identical weights. The useful question is whether the route reliably meets the contract you are buying.

For a coding task, accept the result only after checking the diff and the relevant tests. For extraction, compare required fields against a reference. Record failures and manual repairs instead of selecting the best-looking response from each service.

If the same model behaves differently between clients, use the client comparison checklist before attributing the difference to the model provider.

Reconcile a request with its charge

Read the price for the exact route and unit. Separate uncached input, cache reads, cache writes, output and additional tool fees when applicable. Do not multiply a combined token counter by a single input price.

The cache-cost worksheet explains how to avoid counting cache tokens twice. Treat a request that times out as an unresolved execution outcome until you inspect activity records; receiving no answer does not by itself prove that the upstream did no work or charged nothing.

If a request stops without a complete answer, use the timeout diagnostic checklist to separate connection, response and stream failures.

Use this evidence record for a trial:

Date and time:
Provider / endpoint / documented model ID:
Requested and reported model:
Route and fallback policy, if available:
Request ID / finish state:
Input / cache read / cache write / output usage:
Rate source, currency and unit:
Observed charge and reconciliation notes:
Acceptance criterion / result / manual repair:
Unknown or undisclosed fields:

This is an empty template, not a fabricated test log. Keep unknown fields marked unknown. Ask the provider to explain material discrepancies before increasing usage. If the response does not provide enough evidence, that uncertainty belongs in the purchasing decision.