Self-attention
A layer in which each token's output is a weighted sum of every token's value, weighted by how well its query matches their keys.
Self-attention
A layer in which each token's output is a weighted sum of every token's value, weighted by how well its query matches their keys.