Study note

Using group theory to explore the space of positional encodings for attention

View original Back to list

Properties

Type: Blogs
Status: 已读
Domain: AI / ML
Category: 深度学习、自动微分与实践
Source: blog.janestreet.com
Vault note: library/articles/ai_ml/Using-group-theory-to-explore-the-space-of-positional-encodings-for-attention-071c8a5b5f4f173e.md

Jane Street 文章，用群论视角探索 attention positional encoding 的设计空间，将位置编码放进更抽象的对称性和群表示框架中理解。

适合把 Transformer 位置编码从工程 trick 提升到数学结构层面理解，对 ML + 数学交叉学习很有价值。

Jane Street 用群论理解 attention positional encoding。

Differentiable Programming from Scratch – Max Slater – Computer Graphics, Programming, and Math Blogs · 深度学习、自动微分与实践
Get Started With Deep Learning Performance Blogs · 深度学习、自动微分与实践
Hands-On Modern RL Blogs · 深度学习、自动微分与实践
How to Build a Diffusion Language Model Blogs · 深度学习、自动微分与实践
MiniTorch Blogs · 深度学习、自动微分与实践
TensorTonic | Learn ML through code Blogs · 深度学习、自动微分与实践

Loading comments...