| 1 | Multi-head attention is the next move. Instead of running one attention function over the full 512-dimensional vectors, the authors split into h equals eight independent heads, each operating on 64-dimensional projections of Q, K, and V. The projections are learned, so each head can specialize on a different kind of relationship. The heads run in parallel, their outputs are concatenated, and a final linear projection maps back to 512. The total compute is roughly the same as a single full-dimensional head, but the model gains the ability to attend to different representation subspaces at different positions simultaneously. |