How does training & evaluation actually work in CrewAI?

CrewAI Community Support
1

Hey everyone! :wave:

I’ve been digging through the CrewAI source code and docs to understand the training and evaluation features, but I’m a bit confused about how they’re meant to be used.

From what I can see, CrewAI lets you:

  • Collect human feedback using crew.train()

  • Save training data to JSON files

  • Log some basic evaluation info with crew.test()

But I’ve got a few questions:

  1. What’s the end goal of collecting the training data? I don’t see how it’s actually used to make the agents better

  2. Are there any actual metrics for measuring how well a crew is doing?

  3. What’s the best way to use the human feedback to improve my crews?

I’ve looked through the code (TaskEvaluator, CrewEvaluator, etc.) but it seems like it’s mostly set up to collect data and log results without doing much with them afterward.

The docs are also pretty vague.

Thanks in advance! :pray:

2

When you run training on a crew, it will prompt you for feedback. It will then capture that feedback and store in a pickle file.
And then the next time it runs, it will Inject that file into your system prompts

Hope That makes sense

1 Like
3

Thanks, that part makes sense! Should I open separate topics for my other related questions?

4

That would be better jsut so it’s easier for others to find :+1:

1 Like
5

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.