| 1 | <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1280 720" width="1280" height="720"> |
| 2 | <defs> |
| 3 | <linearGradient id="peLeftScrim" x1="0" y1="0" x2="1" y2="0"> |
| 4 | <stop offset="0%" stop-color="#FFFFFF" stop-opacity="0.96"/> |
| 5 | <stop offset="45%" stop-color="#FFFFFF" stop-opacity="0.78"/> |
| 6 | <stop offset="80%" stop-color="#FFFFFF" stop-opacity="0"/> |
| 7 | </linearGradient> |
| 8 | </defs> |
| 9 | |
| 10 | <g id="bg"> |
| 11 | <rect width="1280" height="720" fill="#FFFFFF"/> |
| 12 | <image href="../images/positional_encoding.png" x="0" y="0" width="1280" height="720" preserveAspectRatio="xMidYMid slice"/> |
| 13 | <rect x="0" y="0" width="1280" height="720" fill="url(#peLeftScrim)"/> |
| 14 | </g> |
| 15 | |
| 16 | <g id="header"> |
| 17 | <rect x="80" y="80" width="60" height="3" fill="#3182CE"/> |
| 18 | <text x="80" y="108" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="13" font-weight="700" fill="#3182CE" letter-spacing="2.5">§3.5 · POSITIONAL ENCODING</text> |
| 19 | </g> |
| 20 | |
| 21 | <g id="title"> |
| 22 | <text x="80" y="166" font-family="Georgia, "Microsoft YaHei", serif" font-size="38" font-weight="700" fill="#1A365D">Recovering order</text> |
| 23 | <text x="80" y="208" font-family="Georgia, "Microsoft YaHei", serif" font-size="38" font-weight="700" fill="#1A365D">without recurrence</text> |
| 24 | </g> |
| 25 | |
| 26 | <g id="explanation"> |
| 27 | <line x1="80" y1="252" x2="120" y2="252" stroke="#3182CE" stroke-width="1.5"/> |
| 28 | <text x="80" y="286" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">No recurrence and no convolution means the model has</text> |
| 29 | <text x="80" y="308" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">no built-in notion of token order. Add a deterministic</text> |
| 30 | <text x="80" y="330" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">sinusoidal position vector to each input embedding —</text> |
| 31 | <text x="80" y="352" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">order is injected, not learned.</text> |
| 32 | |
| 33 | <text x="80" y="402" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="12" font-weight="700" fill="#3182CE" letter-spacing="1.5">WHY SINUSOIDS</text> |
| 34 | <text x="80" y="426" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">Wavelengths form a geometric progression</text> |
| 35 | <text x="80" y="448" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">from 2π up to 10000·2π. For any fixed offset k,</text> |
| 36 | <text x="80" y="470" font-family="Consolas, "Courier New", monospace" font-size="14" fill="#1A202C">PE(pos + k)</text> |
| 37 | <text x="80" y="492" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">is a linear function of <tspan font-family="Consolas, monospace" font-weight="700" fill="#3182CE">PE(pos)</tspan> — so the model</text> |
| 38 | <text x="80" y="514" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="15" fill="#1A202C">can easily learn to attend by relative position.</text> |
| 39 | </g> |
| 40 | |
| 41 | <g id="formula-band"> |
| 42 | <rect x="80" y="560" width="540" height="110" rx="10" fill="#FFFFFF" stroke="#3182CE" stroke-width="1.5"/> |
| 43 | <text x="100" y="586" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="12" font-weight="700" fill="#3182CE" letter-spacing="1.5">EQUATION</text> |
| 44 | <image href="../images/formula_003.png" x="120" y="596" width="460" height="72" preserveAspectRatio="xMidYMid meet"/> |
| 45 | </g> |
| 46 | |
| 47 | <g id="footer"> |
| 48 | <text x="1220" y="708" text-anchor="end" font-family="Arial, "Microsoft YaHei", sans-serif" font-size="11" fill="#4A5568">11</text> |
| 49 | </g> |
| 50 | </svg> |
| 51 |