GGUF Per-Tensor Layout Maps: Compare Recipes, Not Filenames

What GGUF per-tensor layout maps actually change

GGUF per-tensor layout maps specify which tensors receive which quantization types. A familiar filename can make two artifacts look equivalent when their tensor allocations are not.

Bartowski's September 10, 2026 release note explains his generator's allocation recipes. It does not redefine GGUF or establish a naming standard for other publishers. The generator uses model shape and a sensitivity prior to assign precision, with maps published beside supported artifacts.

Consider a hypothetical migration, not an Optijara deployment: an operator replaces an older GGUF with a newer Q4_K_M download. A smoke test passes, but the matching suffix does not establish matching tensor assignments, storage cost or workload behavior.

Keep the quant family, tensor recipe and downloaded artifact separate. A same-name comparison reveals migration differences. To compare recipes at similar storage cost, select candidates under a declared byte envelope, even when their suffixes differ. This article uses a pinned MiniCPM5 recipe as an example, not as a model-launch recap. Optijara performed no quantization or benchmarks.

Old names, new allocation budgets

Read _S, _M and _L in this generator's terms

The release describes S, M and L with 90%, 70% and 50% base-type rules. The pinned generator makes the denominator precise: TIER_SHARE is a minimum share of solved body bytes held at the base type.

These are not tensor counts or percentages of the complete GGUF file. Embedding-class tensors, pinned tensors and retained data require separate accounting. The solver has no absolute byte target; bitrate is reported after solving. Here, allocation budget means a body-share constraint, not a promised download size.

The historical variants and retirements are described in What ships. They are not claims about every GGUF publisher. Keeping IQ tensor types out of K-quant recipes is also Bartowski's compatibility choice, not a universal format restriction.

Read an actual Q4_K_M recipe

The pinned MiniCPM5 layout JSON records base_type q4_k, variant shr0.70, rung_share 0.7 and auto_crush false. Its tensor-types file assigns output.weight to q6_k and token_embd.weight to q4_k. In block zero, attn_k and attn_v use q8_0 while attn_q uses q4_k.

Q4_K_M therefore does not mean every tensor is Q4_K. The manifest separately records predicted file bytes and its quantizable, unpinned body-plus-embedding comparison. Neither is a measurement of the downloaded file. Its generator_key is a69a855b91615d22 and llama_cpp_version is b10883. These identify inspected recipe metadata, not your source weights or serving runtime.

What the release evidence supports

KLD is distribution fidelity, not task accuracy

Bartowski reports KL divergence between quantized and BF16 token probabilities on wikitext-2-raw at context 512. Head-to-head comparisons use 100 chunks, increasing to 300 when gaps are close. These are his experimental settings, not a universal evaluation budget.

Lower KLD indicates closer reference distributions under that setup. It does not establish extraction accuracy, long-context quality or inference speed. The llama.cpp perplexity documentation explains reference-logit comparisons, implementation dependence and uncertainty estimates.

The release reports that dense models below roughly three bits mostly tied the older method, while improvements above roughly Q5 approached noise. The prior initially used two small Qwen models, with later Granite-derived corrections. Read those as boundaries of the author's evidence, not architecture-wide performance guarantees.

Release iteration is not universal coverage

The article describes early MiniCPM canary failure and subsequent generator/prior corrections, then identifies MiniCPM and Gryphe Pantheon as releases using the method. An earlier fallback and a later mapped artifact reflect iteration, not proof of a contradiction.

The MiniCPM card explicitly says some files use computed layouts. The pinned Q4_K_M files above provide one concrete example. The Gryphe Pantheon card separately documents layouts and canary comparisons. Neither establishes mapped coverage for every quant.

Bartowski's expectation that sensitivity transfers between matching shapes remains a hypothesis to test for fine-tunes and unusual architectures. The article does not independently validate that expectation.

The Quant Recipe Migration Map

Pin provenance and diff the assignments

Optijara's proposed Quant Recipe Migration Map is a decision record connecting a replacement artifact to its recipe, comparison and rollback. It is not an external standard or an executed benchmark.

Diff assignments by exact tensor name, including retained higher-precision tensors. Preserve pattern order: the pinned generator documents that the first matching pattern wins. An unordered set can lose reconstruction information.

Hugging Face commit 689246ff8d3d9b7f80495a9c883ac6f19a582630 pins the inspected recipe files. It does not establish the upstream source-weight revision. Likewise, a local source path in the manifest is not a portable source identity.

Compare actual bytes before quality

Declare the storage envelope before choosing replacements. Keep source weights and calibration inputs constant. Select nearby old and mapped candidates, then record the remaining byte mismatch. Matching filenames is not a substitute for this accounting.

If exact byte matching is unavailable, plot measured KLD against measured size for nearby candidates. Label interpolation as an estimate, never as an observed benchmark. Keep raw KLD and bits per weight visible. The author's phrase KLD per bit describes a size-aware comparison curve; this article does not establish a standardized KLD-divided-by-bits score.

The quantization documentation covers importance matrices and tensor overrides. Confirm option support in the exact executable used to build the artifact. Current documentation may differ from an older build.

Keep the experiment inside one serving route. Our Qwen quantized-route comparison addresses the separate problem of moving between BF16, GGUF and SGLang. Changing routes during a recipe test adds another explanation for observed differences.

Run canaries and retain a reversible switch

Bartowski's canary policy compares mapped recipes with his own llama.cpp changes at Q4_K_M, Q3_K_M and IQ2_XS, or the smallest target. A map is rejected when a mapped candidate sits above the KLD-versus-bits comparison curve beyond noise. These are his targets and fallback rules, not universal application thresholds.

Reproducing that experiment and comparing against an existing download are different exercises. Name the baseline. Hold the BF16 reference, corpus, tokenization, context, chunk selection and evaluator constant. Preserve raw output and uncertainty information.

Then test representative tasks and runtime behavior against predeclared requirements. Keep the old verified artifact and serving configuration. A favorable fidelity curve cannot overrule a failed workload requirement.

This illustrative record contains no observed results. Fill null fields with verified identifiers or measurements.

{
  "framework": "Quant Recipe Migration Map",
  "status": "proposed_not_executed",
  "provenance": {"sourceRevision": null, "generatorCommit": null, "priorRevision": null},
  "oldArtifact": {"sha256": null, "bytes": null},
  "candidateArtifact": {"sha256": null, "bytes": null},
  "recipeDiff": null,
  "comparisonSettings": {"reference": null, "corpus": null, "context": null, "runtimeRevision": null},
  "measurements": {"kld": null, "taskQuality": null, "latency": null, "peakMemory": null},
  "rollbackArtifact": null
}

Measure usefulness separately from KLD

Keep separate columns for separate decisions

A comparison sheet should not hide trade-offs inside one score. Storage, distribution fidelity, workload acceptance, latency and memory answer different questions. Our Qdrant benchmark analysis makes the analogous distinction between reference fidelity and application usefulness, using different measurements.

For extraction, score required fields. For structured output, separate schema validity from content correctness. Include longer-context cases where needed: a context-512 reference test cannot validate them. Keep held-out tasks rather than selecting repeatedly against one evaluation set.

For local RAG, freeze retrieved passages and the prompt template while swapping the GGUF. Score answer correctness, citation support and abstention separately. That keeps retrieval changes out of the generator comparison.

File bytes are not peak memory

Downloaded size does not account for the workload's KV cache, runtime workspace or allocator behavior. Measure host and device peaks under intended settings. Our MiniCPM5 device-budget guide discusses that separate deployment question.

Keep serving settings fixed for timing tests. Record cold-start and warmed behavior separately when both matter. Neither a smaller artifact nor lower KLD proves faster inference. If a runtime upgrade is part of the test, label that additional variable instead of attributing the timing change solely to allocation.

Common mistakes and remaining limits

Matching filenames instead of artifacts

Avoid name-only swaps, whole-file interpretations of body shares and nearest-looking replacements for retired labels. Use assignments and measured bytes. Mutable branch links help discovery but do not establish reproducibility: retain immutable revisions for the card, recipe and artifact used in a decision.

Do not confuse a pinned generator with pinned source weights, or predicted tensor storage with measured download size. Missing provenance limits what a comparison can establish even when the application test passes.

Treating the prior as architecture-wide support

The release's limitations explicitly identify PLE n-gram tables and Hy4's gated MLA as unhandled gaps. Earlier dense-model corrections justify continued canaries, not claims of universal coverage. Small higher-precision differences may also be difficult to separate from noise.

Allow time for regeneration, validation and runtime compatibility testing. A narrow reference corpus can miss workload regressions. For private evaluation documents, define access, logging and retention rules before testing; local execution alone does not settle them.

Quantization packaging is not evidence of new licensing rights, benchmark superiority over Unsloth or another publisher, or promised savings. When provenance or evaluation is incomplete, retain the existing verified artifact rather than describing the migration as validated.