Become an AI Expertv2.1
0 of 0 steps Continue
Stage 2 · The model from the insidePerformance evidence

Milestone B

Each milestone is scored out of twenty under the peer-review protocol: five criteria, 0 to 4 each. Score 0 for missing evidence, 1 to 2 for an incomplete artifact, 3 for a sound artifact with limited measurement, and 4 for reproducible evidence with the measurement, its noise and its limitations stated. 14 of 20 passes. A high score never overrides a failed mastery gate.

  • A from-scratch GPT that trains and samples, with the forward pass whiteboarded with shapes.
  • The nanochat run: eval table, loss curve, cost line, and one change with its measured effect.
  • A KV cache with the identical-output proof, timing and cache size.
  • SFT and GRPO runs with separate capability and general-ability columns, and what the KL coefficient did.
  • One ablation: one variable, three settings, one plot, one paragraph.
  • Target 05 (stale cache): all five flags print, and the repo holds your prediction list, an explanation of each invariant, the negative tests you added beyond the grader's, and a limits paragraph.

Challenge: your GRPO row improved the math eval and hurt the general eval. Explain the mechanism (Principle 8), what the KL term does, and what you would change in the reward.

Submission checklist

For each milestone include: commit identifier; environment and setup; exact commands; the measurement, its denominator and its noise; what you expected before running; what you observed; limitations; next steps. Anything not run (a GPU run you could not afford, users you could not recruit) must say "not done", not be implied. Grade reasoning and reproducibility, not screenshots.

Use the instructor keys for module reasoning and the capstone for the final practical assessment.