Skip to main content
🔒 Preview mode. The first 15 Foundations lessons are free; this one is Pro. Start a 7-day trial to unlock the editor, AI hints and the rest of the curriculum. Card required, cancel any time in Dashboard.Start 7-day trial →
← CoursesAI Engineering with PythonModule 6 · Frontier AI EngineeringSelf-improving prompt: eval-driven prompt loopwrite103 / 105
+140 XP
Task
📝 **Task:** Implement `compare_prompts(golden, run_fn, prompt_old, prompt_new, min_delta=0.0) -> dict`. `golden` is a list of `(input, expected)`. Return: * `old_score` — fraction (rounded to 2 decimals) where `run_fn(prompt_old, input) == expected`. * `new_score` — same for `prompt_new`. * `delta` — `round(new_score - old_score, 2)`. * `verdict` — `'ship'` if `delta > min_delta`, else `'reject'`. Then run the demo: a fake `run_fn` returns `input.upper()` regardless of prompt — so both prompts score identically and we should reject. Expected output: ``` old_score=1.0 new_score=1.0 delta=0.0 verdict=reject ``` 📋 Implement the function above. Tests run automatically. 💡 **Hint:** Re-read the theory if stuck.
✏️ Write your code here
🐍
Loading Python...
One-time download (~1 MB). Then it runs instantly.
📊 Result
Press Run to see result...

💬 Discussion

Be the first to ask a question or share a tip.
Sign in to join the discussion. Reading is free.
Loading discussion…