
Quick Summary
- Open-weight models let you choose the model and its host separately. Where requests are processed depends on who runs inference, not where the model was developed.
- In Zoolatech’s bounded four-feature evaluation, routing reduced model charges from $86.95 to $42.79—about 51%. About three-quarters of the router’s work went to a lower-cost model; this was not a hosting comparison.
- Token rates tell only part of the story. Handovers between models accounted for 41% of the router bill while representing less than 3% of the text.
- Independent hosting and self-hosting offer different controls and costs. Check processing location, request recipients, and retention; self-hosting also adds GPU capacity, maintenance, and operational costs.
- Compare deployments on the same workflow using total charges divided by accepted results, alongside verification quality and response times. Both sides passed the evaluation’s checks, but premium-only sessions produced about four times as many automated tests.
“Developed in China” is not the same as “runs in China.”
That one sentence changes the economics of open-weight models. Our AI Router evaluation shows how, in dollars.
In September 2026, the router sent an API-contract task to GLM-5.2 instead of Claude Opus 4.8. GLM-5.2 wrote the contract and found a live bug: an invalid API key returned an empty string, so the documented 401 never fired.
A capable model did real work for a fraction of the price. The open question is where that model should run.

What Open Weights Change
Three layers, three separate decisions:
- The assistant: the tool a developer talks to, such as a coding agent or our router.
- The model: the engine that generates responses.
- The host: the infrastructure that runs the model and processes each request.
When a developer publishes its weights, anyone with the hardware and license can run the model. That gives a business options beyond the developer’s own API.
It does not make inference free, and licenses still apply. GLM-5.2 is MIT; Kimi K2.6 uses a modified MIT license with its own conditions.
Where Your Data Actually Goes
Where a request goes depends on the service and deployment you use, not on where the weights were trained. Three routes exist:
- Developer’s API: the model author runs inference. Check its processing location and data policy.
- Independent provider: a hosting company runs its own copy of the weights. Verify it is hosted inference, not a gateway forwarding requests upstream.
- Your own deployment: you control location, network and logging.
Providers do put this in writing. Together states that models on its own infrastructure do not call out to the model author. DeepInfra states its hosted models run in US data centers.
Those are provider statements, not audits. Whichever route you pick, three checks remain:
- Where inference runs
- Who receives request content, including fallbacks and tools
- What is stored, for how long, and whether it trains anything
One Model, Four Hosts: Kimi K2.6
Kimi K2.6 is the worked example from our July hosting research. It is not a model we have tested in the router.
Published rates, checked November 9, 2026, USD per million tokens:
| Provider | Uncached input | Cached input | Output | Status |
| DeepInfra | 0.75 | 0.15 | $3.50 | Serverless; deployment labeled FP4 |
| Novita AI | 0.80 | 0.16 | $3.40 | Serverless; quantization not stated |
| Fireworks AI | Billed by GPU time | Serverless retired Sept 25, 2026; dedicated deployment only | ||
| Together AI | Not verified | July rate ($1.20 / $4.50) no longer in current listings | ||
Two token-priced options remain. DeepInfra has the lower input rate; Novita has the lower output rate. Which wins depends on the workload.
What the Bill Looks Like
Model charges = uncached input + cached input + output + any separately billed usage.

Illustrative totals at the rates above, not measured traffic:
| Token mix | DeepInfra | Novita AI |
| 10M uncached input + 1M output | 11.00 | 11.40 |
| 1M uncached input + 1M output | 4.25 | 4.20 |
| 1M uncached + 9M cached input + 1M output | 5.60 | 5.64 |
Input-heavy work favors DeepInfra. Output-heavy work favors Novita. The third row shows how much cache pricing matters when 90% of input is a reread.
Our own evaluation shows why that row deserves attention:
- Rereading conversation history was 72% of the premium-only bill
- Handovers between models were 41% of the router bill, under 3% of the text
- Caches are scoped per model and account and can expire
Switching providers mid-session is not free, even when per-token rates look identical.
Dedicated Deployment and Self-Hosting
Fireworks answers a different question: a dedicated deployment instead of a shared endpoint. It is a researched option, not part of our router.
| Fireworks documents | Limit |
| GLM-5.2 and Kimi K2.6 on dedicated GPUs | Availability and quota must match the account |
| Regional deployments; Enterprise data-residency controls | Enterprise feature, not default |
| No storage or logging of inference without opt-in | Batch and training workflows persist differently |
Residency has a price. Together’s US-only endpoint charges 1.25 times its standard rate, and it supports GLM-5.3 and Kimi K3, not the models above.
Self-hosting replaces token prices with capacity costs:
- Provisioned GPUs, busy or idle
- Storage and networking
- Upgrades and maintenance
- Monitoring and on-call
Moonshot’s deployment guide gives a single-node example of eight H200 GPUs. One configuration, not a minimum.
Same Model, Different Results
Two deployments of the same model are not guaranteed to behave the same way. DeepInfra labels its Kimi K2.6 FP4; Moonshot documents native INT4. Neither label proves degraded or identical quality.
Benchmarks help shortlist. Z.ai’s GLM-5.2 model card reports:
| Benchmark | GLM-5.2 | Claude Opus 4.8 |
| Terminal Bench 2.1, Terminus-2 | 81.0 | 85.0 |
| NL2Repo | 48.9 | 69.7 |
Developer-reported, not our tests. The gap is small on terminal work and wide on repository-level generation.
Our four-feature run adds what benchmarks cannot. Both sides passed every test and check, but the premium-only sessions left about four times as many automated tests behind. Same outcome, different depth of verification.
Test One Workflow Before You Route to It
Before any model and host enters a routing configuration, measure five things on a representative workflow:
- Accepted results against agreed requirements.
- Total model charges, including every attempt and reread.
- Verification performed and human corrections needed.
- Response times and failures under expected load.
- Approved processing destinations, including fallbacks.
The metric that ties them together: total charges divided by accepted results.
Our evaluation is that assessment at small scale. Four features, $86.95 versus $42.79. The saving came from routing and shorter conversations, not from any host or self-hosting.
Open weights let you judge the model and the service separately. Judge them on the same workflow.
Questions You May Have
Does a model developed in China send my data to China?
Not automatically. It depends on who runs the model: the developer’s API, an independent provider, or your own deployment.
Which Kimi K2.6 host is cheapest?
It depends on the workload: DeepInfra wins on input-heavy mixes and Novita on output-heavy ones, with a few percent between them.
Why are Fireworks and Together missing prices?
Fireworks moved Kimi K2.6 to GPU-time billing, and Together’s old token rate could not be verified in current listings.
Did the 51% saving come from cheaper hosting?
No, it came from routing and shorter conversations in a bounded test, not from a provider price difference or self-hosting.
What should we measure before routing to a new model?
Measure total charges divided by accepted results, alongside verification performed, response times under load, and approved processing destinations.












