Parameters · demo vs model scale
| | Demo · exact | 7B · illustrative |
|---|
| Base | 144 | 7.00B |
| Trainable | 48 | 0.52M |
| Train % | 33.3% | 0.007% |
| Adpt mem | 0.19 KB | 1.0 MB |
A 2×12 + B 12×2 FP32 | 32L · d4096 · FP16
Frozen
Trainable
ΔW
Target
Simplified on purpose. B·A really trains here (Adam) against a
seeded target ΔW, but real LoRA learns from data through a whole network.
Rank does not map cleanly to quality, module choice matters, training still needs
data and compute, and adapters do not add new facts or remove unsafe behavior.
The tiny 12×12 demo makes adapters look big; the 7B column is an
illustrative estimate of where LoRA actually wins.