Can this model be modified at all?

Some models cannot be modified, and no hardware fixes it

Four of the nineteen checkpoints in our collection are structurally impossible to modify. Not slow, not expensive — impossible. They are mixture-of-experts models that store all their experts in a single fused tensor rather than as individual linear layers.

That matters because both tools in the chain need those layers to be separate. The tool that shrinks the model cannot handle one giant merged block, so the model stays full-size and will not fit on your card. And the tool that applies the modification works by attaching to individual named layers — if they are merged into one, there is nothing to attach to.

The check takes seconds

Build the model structure without loading any weights, then count what you can actually target:

from accelerate import init_empty_weights
from transformers import AutoConfig, AutoModelForCausalLM
import torch

cfg = AutoConfig.from_pretrained(path)
with init_empty_weights():
    m = AutoModelForCausalLM.from_config(cfg)

targets = sum(1 for n, mod in m.named_modules()
              if isinstance(mod, torch.nn.Linear)
              and n.rsplit(".", 1)[-1] in ("o_proj", "down_proj"))

fused = [(n, tuple(p.shape)) for n, p in m.named_parameters()
         if "expert" in n.lower() and p.dim() >= 3]

print(targets, "targetable modules;", len(fused), "fused expert tensors")

init_empty_weights is what makes this cheap — it builds the module graph with no allocated tensors, so a 35B checkpoint answers as fast as a 1B one.

Reading the answer

• Targetable modules above zero, no fused tensors. Go ahead.
• Any fused expert tensor. Stop. A shape like (256, 1024, 2048) is 256 experts packed into one parameter. There is no fix at the checkpoint level. The model still serves perfectly well under vLLM or llama.cpp — it just cannot go through this pipeline.
• Zero targetable modules and no fused tensors. A new architecture the tool does not recognise. Sometimes fixable by teaching it the new module names, which is how we handled a hybrid Mamba model.