Ctrl+Enter to process
6
Transformer Layers
A transformer is built by stacking many identical layers on top of each other — typically 12 to 96. Each layer refines the model's understanding a little more. The early layers tend to pick up simple patterns like grammar and word endings; the middle layers handle longer-range relationships; the deeper layers seem to encode more abstract, task-relevant knowledge. By the final layer, each token's representation has been shaped by the entire context.
Common misconception: The layers do not work like a pipeline where each one finishes before the next starts. Instead, each layer adds small corrections to a running "representation" of every token through a mechanism called the residual stream. The original information is never erased — it flows straight through and gets refined.
Enter a query above to begin.