| 1 | Today we are reading what is probably the single most influential machine learning paper of the last decade: Attention Is All You Need, by Vaswani and seven coauthors at Google Brain, Google Research, and the University of Toronto, presented at NeurIPS 2017. The title is the thesis: drop recurrence and convolution, use only attention, and you get a model that is faster to train, easier to parallelize, and stronger on machine translation than anything that came before. Over the next sixteen slides we'll walk through what the Transformer actually is, how the attention mechanism works, why the authors chose this design, and what the experiments showed. |