← All Elements

Validate

AI Evaluation

Testing an AI’s behavior against clear expectations, across the cases it will really face.

What is AI Evaluation?

Testing an AI’s behavior against clear expectations, across the cases it will really face.

What you learn

Test AI on the hard cases people will really bring.

When it helps

an AI feature works in demos but varies in use, or a change needs comparing.

Use it at work

  1. Write what a good answer looks like.
  2. Collect real, messy cases, including ones it should refuse.
  3. Score the results against your expectations.
  4. Rerun the tests after every change.

Your output

Add 20 real, messy cases to your AI tests: typos, edge cases and questions it should decline. Run the tests after every change.

Watch for

Testing only clean, typical cases.

Related practices

Sources

Enable JavaScript to try the interactive activity.

Elements.
48 elements · 8 families
Filters
DESIGN CONTEXT
What are you trying to do?

Scroll sideways within the map to explore all columns.

Created by Alan Hurt. Activities and notes are stored on your device. Privacy · Usage counts

ELEMENTS FIELD GUIDE