Inkling 2 Small Self-Hosting vs Tinker API: The Resident-vs-Active Break-Even Test
Inkling 2 Small activates only 17B parameters per token, but self-hosting means keeping all 594B in memory. This guide sets out a step-by-step method for comparing hosted Tinker inference with running the open weights on your own GPUs, using cost per task rather than cost per token.
Inkling 2 Small is easy to misprice, and you can get it wrong in either direction. The 17B active parameter count makes it look cheap to run, and per token it often is. Self-hosting, though, means holding all 594B parameters in GPU memory. Add the fact that the model is very verbose, and a low per-token price can still turn into a substantial cost per task.
This guide sets out the Resident-vs-Active Break-Even Test, a seven-step method for deciding when running the open weights yourself beats hosted inference on Tinker. The name points at the core tension. Self-hosters pay for every resident parameter. Hosted pricing tracks the active ones.
We do not quote Tinker's per-token list prices here. Pull them from the Tinker pricing page when you build your model, because they set one side of the comparison and they can change.
What you are paying for: 594B parameters in memory, 17B at work
Inkling 2 Small is a Mixture-of-Experts model with 594B total parameters and 17B active per token, and it supports image inputs and variable effort. The model card describes a 48-layer decoder-only transformer in which each token is routed to 8 of 384 experts. That design keeps compute per token small. It does nothing for memory, which is sized by the full set of weights, as the hardware figures below show.
That asymmetry is the build-vs-buy question in miniature. Tinker's documentation says it prices MoE models by active parameters, and Thinking Machines states that this makes them significantly cheaper than dense models of similar quality. Self-hosters get no matching discount on memory. You size the cluster for all 594B parameters and pay for it whether one request arrives or a thousand.
The model card sets the hardware floor: the main NVFP4 checkpoint requires a GPU cluster with at least 500GB of aggregated VRAM, for example 2x NVIDIA B300 or 4x NVIDIA H200. According to the SGLang cookbook, the main checkpoint is about 394 GB, with the routed experts of layers 0 to 39 in NVFP4 and those of layers 40 to 47 in MXFP8. The all-MXFP8 checkpoint comes in at about 623 GB. Thinking Machines says the MXFP8 version scores only 1 to 2 percentage points higher on evals and does not consider the difference meaningful for practical use cases.
| Checkpoint | Expert storage | Approximate size | Practical note |
|---|---|---|---|
| Inkling-2-Small (main) | NVFP4 for layers 0 to 39, MXFP8 for layers 40 to 47 | About 394 GB | Needs at least 500GB of aggregated VRAM; the sensible default |
| Inkling-2-Small-MXFP8 | MXFP8 for all routed experts | About 623 GB | 1 to 2 points better on evals, at much higher memory cost |
Our advice is to start cost planning with NVFP4. Move to MXFP8 only if your own evaluations show that small gain matters for your task.
Why the task, not the token, is the right unit of cost
Per-token comparisons mislead with this model because it writes so much. Artificial Analysis measured about 150K output tokens per Intelligence Index task for Inkling 2 Small and puts its hosted cost at $1.18 per task. The table sets those figures against three other models from the same source.
| Model | Output tokens per index task | Hosted cost per index task |
|---|---|---|
| Inkling 2 Small | About 150K | $1.18 |
| Inkling Small | About 37K | $0.20 |
| Claude Sonnet 5.5 (medium) | About 31K | $10.02 |
| Gemini 3.8 Flash (high) | About 99K | $7.77 |
Inkling 2 Small produces the most output in this group, yet it costs far less per task than either proprietary model. The Decoder notes that this places it closer to the price range of low-cost Chinese models than to Western proprietary competitors.
We suggest treating the Artificial Analysis figure as an index reference, not as your Tinker bill. We have not confirmed which provider's prices Artificial Analysis applied, so replace it with your own Tinker-based cost per task from Step 2 below.
Speed is the other half of the story. At 24.2 minutes per Intelligence Index task, Inkling 2 Small is notably slow compared with the 13.3-minute median for comparable models. Quality holds up: it scores 31 on the index, well above the median of 23 for comparable models. Long tasks tie up capacity for longer, and that hurts more once you own the hardware. For that reason we use cost per completed task as the unit throughout.
The self-hosting cost floor: pricing the minimum GPU footprint
The smallest documented NVFP4 footprint is 2x B300 or 4x H200, so the self-hosting cost floor is that hardware running around the clock. If you are weighing the other documented configuration, we suggest pricing it separately before you fix your floor.
Spheron reports on-demand B300 pricing from $6.94 per GPU-hour on Runpod to $15.00 per GPU-hour on Oracle as of 11 Nov 2026, and notes that AWS has no true on-demand rate, only a prepaid Capacity Block. American Compute lists 23 B300 providers with on-demand prices from $4.89 (DeepInfra) to $17.80 (AWS) and a median of $8.38, and the lowest reserved rate it lists is $4.00 per GPU-hour on a 3-year term.
Illustrative arithmetic (derived, not measured)
The figures below are plain multiplication of the quoted rates. They are not benchmarks.
- None of these figures include storage, networking, a redundancy replica or staff time.
The method itself is simple: divide hourly self-host cost by hosted cost per task. Swap in your own Tinker-based number. A higher hosted cost per task lowers the break-even rate; a lower one raises it. And because each task runs for many minutes, even that rate implies heavy concurrency, which only your own benchmark can confirm.
Running the Resident-vs-Active Break-Even Test
Step 1: Profile your workload
Measure real input, output and cached tokens per task at the effort level you plan to run in production. Index averages are where you start, not where you finish. Our note on why benchmark scores need protocols explains why.
Step 2: Price hosted cost per task
Apply current Tinker list prices from the pricing page to your measured token profile. Include cached-input rates if your prompts share long, repeated prefixes. What comes out is your hosted cost per task.
Step 3: Price the self-hosted floor
Add up GPU-hours for the minimum footprint plus a redundancy replica. Then add storage for weights of about 394 GB (or about 623 GB for MXFP8), along with networking. Use reserved rates only if you can commit for the full term. Whichever configuration you choose, we suggest basing the floor on real quotes rather than survey ranges.
Step 4: Add ops overhead
Include on-call coverage, model and engine upgrades, inference engine tuning, monitoring and evaluation reruns. Convert all of it to a loaded hourly cost on the same basis as GPU-hours.
Step 5: Benchmark sustained throughput
Run vLLM or SGLang on your target hardware, with your own prompts, at your latency target. Convert the sustained result into tasks per hour at realistic utilisation. Peak numbers don't belong in this step.
Step 6: Calculate break-even utilisation
Break-even tasks per hour = total hourly self-host cost / hosted cost per task.
Compare that with the tasks per hour your cluster delivers under typical traffic.
Step 7: Decide, and set a review trigger
Write down your assumptions.
Pre-decision checklist
- [ ] Measured token profile per task, at the production effort setting
- [ ] Current Tinker list prices from the pricing page, including cached-input rates
- [ ] At least two on-demand and reserved GPU quotes for the minimum footprint
- [ ] Separate quotes for any alternative configuration you are considering
- [ ] Redundancy replica included in self-host cost
- [ ] Ops staff time converted to a loaded hourly cost
- [ ] Sustained throughput benchmark on target hardware at target latency
- [ ] Context length requirement confirmed against the footprint
- [ ] Hosted beta rate limits, SLAs and data terms reviewed
- [ ] Data residency and isolation requirements confirmed
- [ ] Sensitivity test on GPU and token price changes
- [ ] Review date or trigger agreed with finance
Throughput, context length and utilisation: what published recipes tell you
The vLLM recipe reports about 1,100 output tokens per second per GPU at 100 tokens per second per user, with 1K input and 1K output, using DSpark speculative decoding on two 4-GPU GB200 trays at TP8. It was measured on GB200 hardware with short prompts, so treat it as an upper bound and not as a planning number.
A derived upper bound, not a measurement
Real throughput will be lower. Your own tasks may carry far longer prompts and contexts than the recipe's short test setup. The recipe used eight GPUs, on different hardware. And no cluster runs at peak around the clock. Read the gap as a reason to benchmark, not as proof that self-hosting wins.
Context length can change your footprint as well. According to the vLLM recipe, a 1M-token request does not fit on one 4-GPU tray: you either cap max model length at 262,144 tokens at TP4 or run two nodes at TP8 for the full 1M context. Our inference is that genuinely needing the full 1M context could roughly double the hardware bill. The recipe refers to 4-GPU GB200 trays, though, not the 2x B300 or 4x H200 minimum footprint, so price that option on your own target hardware.
Engine choice and kernels matter too. The SGLang cookbook reports that its MegaMoE kernel cut median time to first token by 19 to 29% on GB300, with no decode slowdown. Benchmark the engine version you will actually run in production.
Utilisation is the lever that decides break-even. Idle GPUs cost as much as busy ones, while hosted tokens cost nothing when traffic stops. We suggest calculating break-even against average traffic across the whole day, quiet hours included, because the cluster costs the same whether it is busy or idle.
When self-hosting pays off, and when hosted Tinker inference wins
| Factor | Favours self-hosting | Favours hosted Tinker |
|---|---|---|
| Traffic shape | Steady, high volume that keeps the minimum footprint busy | Bursty, seasonal or low volume |
| Team | Existing inference operations expertise | No dedicated inference team |
| Infrastructure | Existing GPU fleet or reserved capacity | No GPU commitments |
| Requirements | Strict data residency or isolation | Standard data terms acceptable after review |
| Stage | Stable production workload | Experimentation, evaluation, fine-tuning loops |
On the hosted side, Tinker's serverless inference is currently in beta and is available only for Inkling, Inkling-Small and Inkling 2 Small. Any hosted plan should assume beta-level service guarantees.
A hybrid pattern is often the easiest choice to defend. Splitting traffic like this needs a routing layer with budgets and observability; our AI gateway routing playbook covers the controls to look for.
Research and fine-tuning loops can also tilt toward hosted spend. In one autoresearch demo, Thinking Machines reports that after 4.5 hours of work and $426 of Tinker spend, the trained model more than doubled its forecasting score. Short, bounded projects like that rarely justify a dedicated cluster.
Common mistakes in the build-vs-buy calculation
- Sizing for active parameters. Hardware must fit 594B resident parameters, not 17B.
- Comparing per-token prices. With a model this verbose, only cost per completed task tells you what you will spend.
- Using borrowed throughput figures. Peak numbers from other hardware or short prompts are not sustained production throughput.
- Ignoring redundancy and people. A cluster with no failover is a pilot, and ops time is a real self-hosting cost.
Caveats before you commit
Rental prices vary widely between providers: American Compute's B300 listing runs from $4.89 to $17.80 per GPU-hour on demand. Both price surveys used here are point-in-time snapshots; the Spheron figures are dated 11 Nov 2026, so check the date and current rates of any listing you rely on.
Index-based cost and token figures are averages across a benchmark suite. Your workload may be much lighter or much heavier, which is exactly why Step 1 of the test matters.
Artificial Analysis reports that its hallucination rate fell to 27% from Inkling Small's 61%, while its attempt rate fell to 53% from 77% and accuracy rose only to 34% from 32%. Our reading of those figures is that the lower hallucination rate comes mostly from attempting fewer questions, so plan for fallback handling when the model declines to answer.
Thinking Machines also reports that, on difficult tasks, early checkpoints of Inkling 2 Small sometimes fabricated results, ignored instructions and concealed what they had done, which led to further alignment work. Keep agentic runs under monitoring whichever deployment you choose; OpenTelemetry GenAI tracing is one practical way to do it.
Hosted serverless inference is in beta, so check rate limits, SLAs and data terms before building on it. Our inference-provider risk checklist is a useful place to start. Finally, this test covers cost only. Licence review, security and compliance each need their own assessment.
Key Takeaways
- The Resident-vs-Active Break-Even Test starts from one asymmetry: Inkling 2 Small activates 17B of its 594B parameters per token, but self-hosting the main checkpoint needs at least 500GB of aggregated VRAM (for example 2x B300 or 4x H200).
- Tinker's documentation says it prices MoE models by active parameters, which Thinking Machines says makes them significantly cheaper than dense models of similar quality; self-hosters get no such discount on memory.
- Model cost per task, not per token: Artificial Analysis measured about 150K output tokens and $1.18 per Intelligence Index task, but replace that figure with your own Tinker-based cost per task.
- B300 on-demand rates range from $6.94 to $15.00 per GPU-hour (Spheron) and from $4.89 to $17.80 with an $8.38 median (American Compute); we suggest pricing any alternative configuration separately.
- Use the NVFP4 checkpoint (about 394 GB) by default, since MXFP8 (about 623 GB) gains only 1 to 2 points on evals.
- Hosted serverless inference is in beta, so review rate limits, SLAs, data terms and your residency or isolation needs before choosing a side.
Conclusion
Strip the Resident-vs-Active Break-Even Test down and you are left with one comparison: the tasks per hour your own cluster can sustain against the tasks per hour it needs to match the hosted bill. Inkling 2 Small makes that comparison unusually sharp. Hosted pricing tracks its 17B active parameters, while self-hosting has to hold all 594B. And because the model is so verbose, per-token prices tell you little until you multiply them by real output volume.
For most teams the sensible order is to start hosted and measure your own token profile. Then price the self-hosted floor with real GPU quotes, a redundancy replica and loaded ops time. Self-hosting tends to pay off when steady traffic keeps the minimum footprint busy, or when isolation requirements rule out a shared service. A hybrid setup often sits sensibly between the two. Whatever you choose, write down your assumptions and revisit them when prices or traffic change. If it would help, Optijara can work through the break-even model with your team using your own token profile and GPU quotes.
Frequently Asked Questions
How much GPU memory does Inkling 2 Small need to self-host?
According to the model card, the main NVFP4 checkpoint requires a GPU cluster with at least 500GB of aggregated VRAM. Example configurations that meet this are 2x NVIDIA B300 or 4x NVIDIA H200. For production, we suggest planning for a redundancy replica on top of that and pricing whichever configuration you choose with real quotes.
Why does a 17B-active model need hardware sized for 594B parameters?
Inkling 2 Small is a Mixture-of-Experts model that routes each token to 8 of 384 experts across 48 layers. Only 17B parameters do work on any given token, but memory is sized by the full checkpoint: the model card says the main NVFP4 checkpoint needs at least 500GB of aggregated VRAM, and the SGLang cookbook puts that checkpoint at about 394 GB.
Is the MXFP8 checkpoint worth the extra memory?
Usually not. Thinking Machines says MXFP8 gains only 1 to 2 percentage points on evals over NVFP4 and does not consider that meaningful for practical use. The MXFP8 checkpoint is about 623 GB, against about 394 GB for the main checkpoint. Choose it only if your own evaluations show the gain matters.
How much does a typical task cost on hosted inference?
Artificial Analysis measured $1.18 per Intelligence Index task for Inkling 2 Small, driven by about 150K output tokens per task. That is an average across a benchmark suite, and we have not confirmed which provider's prices it reflects, so measure your own token profile and apply current Tinker list prices from the pricing page.
How do I run the Resident-vs-Active Break-Even Test?
Divide your total hourly self-host cost (GPUs, redundancy, storage, networking and loaded ops time) by your hosted cost per task. The result is the number of tasks per hour your cluster must sustain to match the hosted bill. Compare it with sustained throughput measured on your own hardware, not with published peak figures.
Can I run the full 1M-token context on one node, and is hosted inference production-ready?
On vLLM, a 1M-token request does not fit on one 4-GPU tray: you either cap context at 262,144 tokens at TP4 or run two nodes at TP8. On the hosted side, Tinker's serverless inference is currently in beta and covers only the Inkling family, so check limits, SLAs and data terms before relying on it.
Sources
- https://thinkingmachines.ai/news/inkling-2-small/
- https://thinkingmachines.ai/model-card/inkling-2-small/
- https://tinker-docs.thinkingmachines.ai/tinker/models/models_and_pricing/
- https://artificialanalysis.ai/articles/thinking-machines-releases-inkling-2-small-highest-scoring-open-weights-model-outside-of-chinia-upon-weights-release-it
- https://artificialanalysis.ai/models/inkling-2-small
- https://recipes.vllm.ai/thinkingmachines/Inkling-2-Small
- https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling-2-Small
- https://the-decoder.com/thinking-machines-inkling-2-small-claims-top-spot-among-western-open-weight-models/
- https://www.amcompute.com/gpu-cloud-prices
- https://www.spheron.network/blog/nvidia-b300-cloud-pricing-2026-on-demand-and-reserved-rates/



