Latest
Everything, newest first.
4 essays across 2 questions.
- The transformer is a learned probability model A joint probability over sequences is impossible to store. The chain rule turns it into one function you can learn.
- From Bellman to Rainbow RL isn’t solvable — it can only be estimated. This essay collects everything you need to know about reinforcement learning.
- The residual stream is the model Every explanation of transformers spends itself on attention, which is a third of the model. The other two thirds are MLPs — and the thing they all write into is where the model actually lives.
- Attention is a lookup table you learn Before any of the scale questions, the mechanism. Attention is not “the model focusing” — it is a differentiable dictionary, and once you see it that way every design choice downstream stops being arbitrary.