Skip to content
rraushan
Overview
Initializing search
Home
ML
Maths
CS336
ML Sys Design
Guides
rraushan
Home
ML
ML
Overview
Applied Ml
Applied Ml
Communication Primitives
Data Parallelism (DDP)
Exploding / Vanishing Gradients
Floating Point Representation
FSDP / ZeRO
Gradient Accumulation
Gradient Checkpointing
Gradient Clipping
JAX vs PyTorch vs TensorFlow
JIT Compilation
Mixed Precision Training
Numerical Precision Tricks
Pipeline Parallelism
Profiling
Tensor Parallelism
General Ml
General Ml
Activation Functions
Adam / AdamW / Adagrad
Autoencoders
Backpropagation
Bagging
BatchNorm / LayerNorm / RMSNorm
Bayes Theorem
Bias–Variance Tradeoff
Boosting
Clustering (k-means and friends)
CNNs
Confidence Intervals
Convex Functions
Cross Validation
Curse of Dimensionality
Data Whitening
Decision Trees
Dimensionality Reduction
Domain Adaptation
Early Stopping
Ensembles
Entropy
Expectation
Few-Shot / Zero-Shot Learning
Gradient Descent / SGD
Gumbel-Softmax
Jensen–Shannon Divergence
KL Divergence
K-Nearest Neighbours
Linear Regression
Loss Functions
MLE vs MAP
Newton's Method
No Free Lunch Theorem
Overfitting / Underfitting
PDF / PMF
Precision / Recall / F1 / AUC-ROC
Regularisation Methods
RNNs / LSTMs
S4 / State Space Models
Second Order Methods
SVMs
Transfer Learning
Unsupervised vs Supervised
Variance and Covariance
Weight Initialisation
Generative
Generative
Classifier-Free Guidance
Diffusion: Forward Process
Diffusion: Reverse Process (DDPM / DDIM)
Diffusion as SDEs
Flow Matching
GANs
The Score Function
VAEs and the ELBO
ML Notes
Llms
Llms
Causal Attention
Cross Attention
Decoding Techniques
Finetuning
Flash Attention
Griffin
The 1/√d Attention Scaling Factor
Transformer vs RNN vs S4
LoRA
Mixture of Experts
Perceiver
Pretraining
Relative Positional Embeddings
RLHF
RoPE — Rotary Position Embeddings
Scaling Laws
Sinusoidal Embeddings
Tokenisation
Transformer-XL
Maths
Maths
Overview
Linear Algebra
Linear Algebra
Determinant
Dot Product
Eigenvectors and Eigenvalues
Hessian
Jacobian
Linear Independence
Inverse of a Matrix
Null Space and Image Space
Orthogonality
Positive Semi-Definite Matrices
Rank and Span
Singular Matrices
CS336
CS336
Overview
CS336 · Language Modeling from Scratch — Notes
CS336 · Lecture 1 — Overview & Tokenization
CS336 · Lecture 2 — Resource Accounting
CS336 · Lecture 3 — Architectures & Hyperparameters
CS336 · Lecture 4 — Attention Alternatives & Mixture of Experts
CS336 · Lecture 5 — GPUs & Making Them Go Fast
CS336 · Lecture 6 — Triton Kernels, Benchmarking & Profiling
CS336 · Lecture 7 — Multi-GPU Parallelism
CS336 · Lecture 8 — Parallelism Deep Dive (ZeRO & 4D)
CS336 · Lecture 9 — Scaling Laws (Basics)
CS336 · Lecture 10 — Inference
CS336 · Lecture 11 — Scaling Laws II (Recipes, Optimizers, μP)
CS336 · Lecture 12 — Evaluation
CS336 · Lecture 13 — Data I (Sources, Copyright, Datasets)
CS336 · Lecture 14 — Data II (Pipeline & Post-training)
CS336 · Lecture 15 — Alignment: SFT & RLHF
CS336 · Lecture 16 — RLVR & Reasoning Models
CS336 · Lecture 17 — Multimodality
CS336 · Lecture 18 — Inference Systems (Guest)
ML Sys Design
ML Sys Design
Overview
FB News Feed Ranking — ML System Design Master Notes
Deep Neural Networks for YouTube Recommendations — Master Notes
Guides
Guides
Overview
Python Typing
UV for Python
Weight Initialization in DL
Language Modeling From Scratch
¶
Course content CS336
Back to top