The Hundred-Page Language Models Book

Master language models through mathematics, illustrations, and code―and build your own from scratch!

The Hundred-Page Language Models Book by Andriy Burkov, the follow-up to his bestselling The Hundred-Page Machine Learning Book (now in 12 languages), offers a concise yet thorough journey from language modeling fundamentals to the cutting edge of modern Large Language Models (LLMs). Within Andriy's famous "hundred-page" format, readers will master both theoretical concepts and practical implementations, making it an invaluable resource for developers, data scientists, and machine learning engineers.

The Hundred-Page Language Models Book allows you to:

- Master the mathematical foundations of modern machine learning and neural networks

- Build and train three architectures of language models in Python

- Understand and code a Transformer language model from scratch in PyTorch

- Work with LLMs, including instruction finetuning and prompt engineering

Written in a hands-on style with working Python code examples, this book progressively builds your understanding from basic machine learning concepts to advanced language model architectures. All code examples run on Google Colab, making it accessible to anyone with a modern laptop.

Endorsements

Vint Cerf, Internet pioneer and Turing Award recipient: "This book cleared up a lot of conceptual confusion for me about how Machine Learning actually works - it is a gem of clarity."

Tomás Mikolov, the author of word2vec and FastText: "The book is a good start for anyone new to language modeling who aspires to improve on state of the art."