| 1 | ## Backbones quick introduction |
| 2 | |
| 3 | |
| 4 | ### unett.py |
| 5 | - flat unet transformer |
| 6 | - structure same as in e2-tts & voicebox paper except using rotary pos emb |
| 7 | - possible abs pos emb & convnextv2 blocks for embedded text before concat |
| 8 | |
| 9 | ### dit.py |
| 10 | - adaln-zero dit |
| 11 | - embedded timestep as condition |
| 12 | - concatted noised_input + masked_cond + embedded_text, linear proj in |
| 13 | - possible abs pos emb & convnextv2 blocks for embedded text before concat |
| 14 | - possible long skip connection (first layer to last layer) |
| 15 | |
| 16 | ### mmdit.py |
| 17 | - stable diffusion 3 block structure |
| 18 | - timestep as condition |
| 19 | - left stream: text embedded and applied a abs pos emb |
| 20 | - right stream: masked_cond & noised_input concatted and with same conv pos emb as unett |
| 21 |