Loading trained pkl file for flow

General
1

I have a flow which is running multiple crews to get the final output. Here is how the flow works : crew_A → crew_B → crew_C → output. Now, i want to train crew_A, crew_B and crew_C seperately and load the pkl file to respective crews while running the flow next time. Is there any way to do it?

2

tl;dr

Train your Crew, give its output a filename "trained_agents_data.pkl", and then it’ll automagically be picked up by each of your Agents when they need it. In a nutshell, that’s all there is to it. If you’ve got time for the longer version, here it is:

How to Train your Dragon (sorry, I mean your Crew)?

Well, training your Crew is basically a supervised learning loop. You guide the agents, an LLM reflects on your guidance to cook up some general instructions, and then those instructions get fed back to the agents on later runs to boost their performance – assuming you put the final training file where the agent expects to find it.

As the documentation explains:

from crewai import Crew, Agent, Task

# Define your Agents...

# Define your Tasks...

my_crew = Crew(
    agents=[agent_1, agent_2],
    tasks=[task_1, task_2]
)

# Start training
my_crew.train(
    n_iterations=2,                      # How many times to run the practice session
    filename="trained_agents_data.pkl",  # Where to save the final training results
    inputs={"topic": "AI Agents"}        # Any initial inputs your crew needs
)

So, you can set how many times it runs (n_iterations), pass inputs just like you would for my_crew.kickoff(), and in theory, you can specify the output file for your training results. I say in theory because, from what I can tell digging into the source code, when it’s time to actually use the training data later, you don’t get to specify the filename. It specifically looks for a file named trained_agents_data.pkl. So, yeah, you don’t have quite as much freedom with the filename as you might think.

What Really Goes Down During Training?

When you call my_crew.train(), CrewAI temporarily tweaks a copy of your crew just for the training session. All Tasks are flipped to require human input (task.human_input = True), meaning the process will pause and wait for your feedback during each task. Also, Agents are temporarily blocked from delegating tasks to each other (agent.allow_delegation = False).

The crew then runs through its tasks sequentially for the number of iterations you set:

  1. An Agent takes a crack at a Task.
  2. The Agent’s initial output is logged.
  3. You step in and provide feedback, usually the corrected or ideal output.
  4. Your feedback/corrected output is logged.
  5. For every Agent and every iteration, the agent’s initial shot, your feedback, and the improved output (which is typically just your feedback) get saved into a temporary file called training_data.pkl (don’t mix this up with the final trained_agents_data.pkl file).

And What Happens After Training?

Once all the training rounds are done, CrewAI loads up all the raw practice data collected in that temp training_data.pkl file. Then, each Agent’s own Language Model (LLM) analyzes the data it was involved with (initial_output, human_feedback, improved_output). Based on this analysis, the LLM generates a list of clear, actionable instructions (suggestions) designed to help the agent do a better job next time, make sense? This happens for every Agent that took part in the training.

Finally, these refined suggestions are saved to that file you should have been able to name freely, but can’t. So, at this point, it gets saved as trained_agents_data.pkl in the directory where the training script was run.

Finally, How is This Automatically Picked Up?

When you run my_crew.kickoff(), before executing a Task, each Agent automagically tries to load those (hopefully) awesome suggestions. It specifically looks in the same directory where you run your script for a hardcoded file named trained_agents_data.pkl (the name is defined over in crewai/utilities/constants.py). Yeah, I know, that probably shouldn’t be hardcoded :roll_eyes:, but moving on. If the Agent finds this file and, inside it, finds an entry matching its role, it grabs the suggestions from that file. These suggestions are then added to the Task prompt, usually under a header like: "You MUST follow these instructions: ...".

The End

And that’s how training wraps up. Hopefully, that clears up any questions about how to train your crew and how it automatically finds the training results later on.

1 Like
3

I do not get the point about allowing to specify a different name for the training output file but hard code the loading of the training file name? According to the code @Lorenze_Jay has added this feature 2 months ago *(February 19th, 2025 10:52 PM).
Dear Jay, what is the idea about allowing only to specify the training output file ?