Skip to main content
🔒 Preview mode. The first 15 Foundations lessons are free; this one is Pro. Start a 7-day trial to unlock the editor, AI hints and the rest of the curriculum. Card required, cancel any time in Dashboard.
Start 7-day trial →
⚡
← Courses
›
Data Science Applied
›
Module 6 · Deep Learning + MLOps
›
Optimizers: SGD vs Adam vs AdamW
quiz
87 / 104
🇺🇸
EN
▼
↗
Share
⋯
More
+100 XP
📋
Task
📖
Theory
🤖
AI Help
Task
📝 **Question:** Default optimizer for a transformer fine-tune? 📋 Pick the right answer. 💡 **Hint:** Re-read the theory above if unsure.
🎯 Quiz
Question
📝 **Question:** Default optimizer for a transformer fine-tune? 📋 Pick the right answer. 💡 **Hint:** Re-read the theory above if unsure.
A
SGD with Nesterov momentum — its lookahead step converges faster on large transformer batch sizes.
B
AdamW — decoupled weight decay outperforms standard Adam on transformers; modern default.
C
Adagrad — per-parameter learning-rate decay handles attention's sparse gradient pattern out of the box.
D
RMSProp — the original adaptive optimizer behind GPT-2 pre-training; still the BERT fine-tuning default.
Submit answer
💬 Discussion
Be the first to ask a question or share a tip.
Sign in
to join the discussion. Reading is free.
Loading discussion…