LayerNorm in ViT Models
An introduction to LayerNorm used in ViT and CLIP Models.
Hai's Path to AI
Ideas, systems, and experiments on the path to AI.
Technical notes on building, understanding, and applying AI systems.
Read the writingFrom the notebook
Latest writing
Positional Embedding Advance - RoPE
Advance to PE technologies: RoPE.
The anatomy of 2D positional embedding
An in-depth analysis into 2D positional embedding.