Training crew : how to be sur that the crew use pkl files?

Hello, I am using the following versions:

  • crewai==0.70.1
  • crewai-tools==0.12.1

I have trained my crew and now have two .pkl files: training_data.pkl and usecase.pkl. Everything has worked well up to this point.

Now, how can I ensure that my crew is properly trained and is using these data?

To launch my crew, I use the command:
python main.py run

Thank you for your help.

4 Likes

Hi, the same doubt here.

After training my crew, I ran it, and it seems not to be using the training data.

Additionally, when I train the same crew a second time, the training data is reset, loosing everything that was trained before.

1 Like

I too am interested in figuring out how to manage multiple iterations of .pkl file from training crew.

Is there a way to manually check and see if the pkl file is being loaded? If not, is there a way to manually load crew training files?

3 Likes

Same here! Maybe someone in the CrewAI staff could answer? @matt ?

2 Likes

@plume69 , you can use “crewai test” command.

Indeed, I created a local project, with two tasks :

  1. The agent says “Hello {name}”
  2. The agent sums up two numbers.

I did not provided any input running the crew at first.

Then I ran “crewai train” saying that everyone in the world is named “Paul” and it should ALWAYS sums up these two numbers 1245 and 963.

After I used the “crewai test” command and the “crewai run” commands, and it always said “Hello Paul” and 2208. That means that it automatically loads the training files.

Anyone found a answer on how to kickoff the Crew using the training data? it seems to ignore it.

Using agentops you can trace the api requests including the embeddings used by crewai. By looking deep you can find out if trained knowledge was used as part of the prompt.

Guys, I found a way to determine that it is used by CrewAI.

    @crew
    def crew(self) -> Crew:
        """Creates the Write Book Chapter Crew"""
        crew = Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential,
            verbose=True,
            memory=True
        )
        output = self.agents[0]._use_trained_data(".")
        print(output)
        return crew

If you see something like this: .\n\nYou MUST follow these instructions: \n - [Your list of recommendation for the first agent you have in your crew] then this means that the crew can load your training data.

Also I found these constants:
TRAINING_DATA_FILE = “training_data.pkl”
TRAINED_AGENTS_DATA_FILE = “trained_agents_data.pkl”

So, make sure to keep them at the root of the directory from where you execute the crew (if you execute it with a python file).