Featured
LLMs from First Principles
For engineers and technical managers: from basic transformers to a working nano-scale GPT.
- Explain the structure of the transformer, compared with a feed-forward network.
- Build character-level language models, from RNNs and GRUs to the autoregressive transformer.
- Pretrain a small GPT, fine-tune it to follow instructions, and apply reinforcement learning from rewards.
- Build a software-engineering (SWE) agent.