# Evals: Test Your AI Agent on Real Questions

> Evals pair your real questions with the SQL that answers them. Run them against a Supaboard agent, inspect what fails, and compare runs after every change.

HTML version: https://supaboard.ai/product/evals

Test your agent on the questions that matter.

Pair real questions with the SQL that answers them, then test how your agent answers. Inspect what fails, refine its context, and rerun the same questions to see what changed.

## How it works

1. **Write the eval.** A question, the SQL that answers it, and its source.
2. **Run it.** Each answer is graded 0–100; 70 or more passes.
3. **Refine and rerun.** Update a rule or an example, then run the same evals again.

## Start with questions whose answers you know

Pick the questions your team asks every month and the tricky ones: refunds, test accounts, reporting periods. Add each with the SQL that answers it, and include variations your examples don’t cover.

- A question, the expected SQL and its source
- Save & Run checks the SQL straight away

## See what needs attention

Each run grades the agent’s answer to every eval from 0 to 100 against your SQL, and 70 or above passes. A failing eval comes with a note on how its answer differed, pointing to the rule, example or table note to look at.

- Failing, Passing and Not run, at a glance
- A note on how each answer differed

## Change the agent. Compare the next run.

Refine the context, then rerun the same evals to see what moved. With five evals the agent shows its accuracy on the set you’ve written, so keep adding questions as your team asks them.

- Accuracy shows once there are five evals
- Run history keeps every run to compare

## FAQ

### What is an eval?

A question your team asks, the SQL that answers it correctly, and the source it runs on.

### How is an eval scored?

The agent answers the question, and its query is graded from 0 to 100 against yours. 70 or above is a pass.

### What does the accuracy figure mean?

Once an agent has five evals, it shows accuracy: the average score across them. It describes the questions you’ve written, so add new ones as your team asks them.

### What should I include?

Common questions, your important definitions, and cases that have caused mistakes. Add variations that weren’t used as examples when you set up the agent.

### What should I do when an eval fails?

Read its note and check the expected SQL, then update the rule, example or table note it points to and run the evals again.

Part of [Agents](https://supaboard.ai/product/agents).

---

- Site index for agents: https://supaboard.ai/llms.txt
- Sitemap: https://supaboard.ai/sitemap.xml
- Blog feed: https://supaboard.ai/blog/rss.xml
- Product documentation: https://docs.supaboard.ai

Any page on https://supaboard.ai is available as markdown: append `.md` to its path, or send `Accept: text/markdown`.