Transformers from First Principles
Free lecture notes as a 148-page PDF: build a transformer one block at a time, from the next-word problem to designing your own model, with every matrix drawn and every number computed and checked.
Read article →Program update · Live cohorts closed
All live cohorts are officially closed. November 2026 is Ayush Singh’s final live cohort, and enrolment is already closed. No further live cohorts will open.
Ayush is introducing the FDE + Agentic Engineering Program: an annual community membership starting at ₹24,000 + GST. It includes recordings from all three cohorts, a structured learning route, access to Ayush through the community, and one live group Q&A each month.
Every two months, we will add a detailed write-up of a production-grade project. This is a community membership with recordings and a monthly Q&A, rather than a new live teaching cohort.
We will admit up to 50 new members per month. Invitations for the first 50 seats will be emailed in January 2027. Join the waitlist with your name, LinkedIn profile, phone number, and email address to receive a broadcast notification when monthly applications open.
Join the January 2027 waitlist ↗Interactive diagrams and long-form studies on transformers, agentic systems, and AI engineering.
Free lecture notes as a 148-page PDF: build a transformer one block at a time, from the next-word problem to designing your own model, with every matrix drawn and every number computed and checked.
Read article →Follow a token through a 3D transformer. Inspect the matrices, attention, gradients and training batches in a small, working Qwen-style teaching model.
Open diagram →A ground-up technical study of Kirti's cost-sensitive CI-triage system: labels, trust gates, observers, evidence contracts, fusion, distillation, and human judgment around AI-assisted coding.
Read article →A first-principles architecture study of a production-grade AI pull-request review agent: specialist reasoners, grounded context, orchestration, memory, evaluation, observability, and human review.
Read article →Reliable AI agents need more than longer context windows. They need explicit state, graphs, task DAGs, validators, proof logs, and feedback loops around the model.
Read article →An AI coding plan is not a pile of prompts. It is a small machine that writes the to-do list, delegates work, checks results, and records what got done.
Read article →