Skip to content

Instantly share code, notes, and snippets.

@furixturi

furixturi/sinusoidal_position_encoding.md

Created September 12, 2025 06:19
Show Gist options
  • Save furixturi/137157839aae50fbf329c58abc3c463a to your computer and use it in GitHub Desktop.
Save furixturi/137157839aae50fbf329c58abc3c463a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
sinusoidal_position_encoding.md

$$ \text{PE}{(pos, 2i)} = \sin\left(\frac{pos}{10000^{\frac{2i}{d}}}\right) $$ $$ \text{PE}{(pos, 2i+1)} = \cos\left(\frac{pos}{10000^{\frac{2i}{d}}}\right) $$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment