# Agent eval tools?

**URL:** <https://community.crewai.com/t/agent-eval-tools/7453>\
**Category:** General\
**Tags:** agent\
**Created:** [April 7, 2026, 7:32pm UTC](https://community.crewai.com/t/agent-eval-tools/7453 "2026-04-07T19:32:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Derfe](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/derfe/32/4994_2.png) [@Derfe](https://community.crewai.com/u/Derfe)\
**Post date:** [April 7, 2026, 7:32pm UTC](https://community.crewai.com/t/agent-eval-tools/7453/1 "2026-04-07T19:32:03Z")

</div>

Hiya all,

I’m curious how people here are evaluating their CrewAI agents once things start getting a bit more complex?

I mean stuff like:

- output quality
- if tool usage is actually improving
- regressions after changes
- comparing runs over time
- sharing results that is easy to understand to other peeps

I tried [test.qlankr.com](http://test.qlankr.com), but I’m wondering what other tools or approaches people here are using?

Are you mostly doing this manually, building your own eval setup or using some other tools?

Would love to hear what people do

---

<div class="post-metadata">

**Author:** ![ian\_chou](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/ian_chou/32/5047_2.png) [@ian\_chou](https://community.crewai.com/u/ian_chou)\
**Post date:** [May 1, 2026, 4:20pm UTC](https://community.crewai.com/t/agent-eval-tools/7453/2 "2026-05-01T16:20:18Z")

</div>

Hey! I had the same questions, so I built a minimal quickstart that maps each of your questions to a concrete [promptfoo](https://www.promptfoo.dev/) pattern:

| Your question | How this quickstart addresses it |
| --- | --- |
| Output quality | `is-json` + JavaScript assertions + `llm-rubric` for quality grading |
| Tool usage improvement | Agent reports `tools_used` in structured output; assertions verify specific tools were called |
| Regressions after changes | Saved test cases + `promptfoo eval` re-runs = automatic regression detection |
| Comparing runs over time | `promptfoo view` web viewer + JSON/YAML result exports |
| Sharing results | Exported reports or viewer screenshots |

**Repo** : [ianTPE/crewai-promptfoo-eval-quickstart](https://github.com/ianTPE/crewai-promptfoo-eval-quickstart)

It’s a single-agent CrewAI crew that triages support tickets — nothing fancy, just enough to demonstrate each eval pattern end-to-end. Clone, add your `OPENAI_API_KEY`, run `promptfoo eval`.

Complementary to `crewai test` — that one’s great for overall health; promptfoo is for specific output contracts and regression guarding.

This is a reference quickstart, not a maintained library. PRs welcome if something breaks on newer versions. Hope it helps!
