Deep learning foundations

30 hours · enough, and deliberately no more

This pillar exists to stop one specific failure: hesitating on a theory question and watching the interviewer downgrade everything you said before it. It is not an ML education, and treating it as one is the fastest way to blow this budget on material that won't be tested.

For an AI engineering target, the bar is fluency on fundamentals, not depth on research. You need to explain backprop, training dynamics, and the standard architectural choices without hedging. You do not need to derive the Adam update from scratch or know what a Hessian is.

Most of this pillar is covered by the same course that anchors LLM internals, which is why 30 hours is realistic.

Modules

1 · Backprop, actually understood · ~12h

The one piece of theory that must be genuinely internalised rather than memorised. Everything else in deep learning is a variation on it.

• Neural Networks: Zero to Hero — Andrej Karpathy · work through, lectures 1–3. micrograd builds a working autograd engine in about a hundred lines; you finish it understanding backprop rather than reciting it. The highest-value ~8 hours in this entire roadmap.
• 3Blue1Brown · Neural Networks — skim. The clearest visual intuition that exists for gradient descent and backprop. Watch before Karpathy if the maths feels distant; skip if it doesn't.

Done when: you can derive the gradient of a two-layer network by hand on a whiteboard, and explain what a vanishing gradient is and which architectural choices exist to prevent it.

2 · Training dynamics · ~10h

The interview questions that separate "used a framework" from "understands the thing."

Cover: initialisation and why it matters · normalisation, and specifically why layer norm rather than batch norm in transformers · residual connections · optimisers (SGD, momentum, Adam, AdamW) at intuition level · learning rate schedules and warmup · overfitting, dropout, weight decay · what a loss curve is telling you when it plateaus, spikes, or diverges.

Karpathy's later lectures cover most of this in the course of building the models, which is a better teacher than a dedicated theory course would be.

Done when: you can answer "why layer norm and not batch norm", "what does warmup do", and "your loss spiked at step 10k, what happened" without notes.

3 · Classical ML breadth — the compressed version · ~8h

Folded in here rather than given its own pillar, because AI engineering loops rarely probe it deeply. What survives is what still gets asked.

• StatQuest — Josh Starmer · skim the relevant videos. Unusually clear on statistics and classical methods. Use the index as a lookup, not a curriculum.
• An Introduction to Statistical Learning — reference. Free PDF, the standard bridge from statistics to ML. Read chapters when a gap surfaces; do not read it cover to cover for this target.
• Hands-On Machine Learning — Aurélien Géron, O'Reilly · reference. Practical scikit-learn and the classical model zoo. Reference only.

Worth having automatic: bias-variance · overfitting and regularisation · precision/recall and when accuracy lies · cross-validation · why class imbalance breaks naive metrics · the common model families and when each is the right call.

Done when: you can answer standard classical-ML screening questions confidently and briefly, then steer the conversation to systems — which is where you're strong.

What I'm skipping

• A full deep learning course. fast.ai and d2l.ai are both excellent and both cost 100+ hours. Wrong budget, wrong target. Revisit if the goal shifts to ML Engineer.
• Computer vision. CNNs, detection, segmentation. Not tested for this role.
• Maths as a curriculum. Linear algebra and calculus refreshers on demand only, via 3Blue1Brown. This is the single most seductive rabbit hole in ML prep: it feels like rigorous progress and produces almost no interview value.
• Deriving optimisers. Know what Adam does and why AdamW differs. Skip the proofs.