tiny transformer

Checkpoint: CP-005 Status: QUEUED — not yet run. No results exist for this experiment.

Question

Does attention change the cost structure — do softmax/layernorm or the matmuls dominate?

Why it matters

Attention introduces data-dependent access patterns and two more non-linearities. It is also where segment/checkpoint verification becomes testable, because a transformer has natural layer boundaries.

Gate

Blocked on CP-004.

---

This is a stub. It exists so the roadmap's structure is visible, not to imply work has been done. Nothing in ../../benchmarks/results/ refers to it yet.