Validate
AI Evaluation
Testing an AI’s behavior against clear expectations, across the cases it will really face.
What is AI Evaluation?
Testing an AI’s behavior against clear expectations, across the cases it will really face.
What you learn
Test AI on the hard cases people will really bring.
When it helps
an AI feature works in demos but varies in use, or a change needs comparing.
Use it at work
- Write what a good answer looks like.
- Collect real, messy cases, including ones it should refuse.
- Score the results against your expectations.
- Rerun the tests after every change.
Your output
Add 20 real, messy cases to your AI tests: typos, edge cases and questions it should decline. Run the tests after every change.
Watch for
Testing only clean, typical cases.
Related practices
Sources
Enable JavaScript to try the interactive activity.