attention-mechanisms

https://github.com/lucidrains/x-transformers A simple but complete full-attention transformer with a set of promising experimental features from various papers

https://github.com/lucidrains/vit-pytorch mplementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

https://github.com/lucidrains/audiolm-pytorch Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

https://github.com/lucidrains/soundstorm-pytorch Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

https://github.com/lucidrains/magvit2-pytorch Implementation of MagViT2 Tokenizer in Pytorch

https://github.com/lucidrains/meshgpt-pytorch

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch https://github.com/lucidrains/make-a-video-pytorch

https://github.com/lucidrains/self-reasoning-tokens-pytorch Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto

https://github.com/lucidrains/musiclm-pytorch Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

https://github.com/lucidrains/En-transformer Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network