Additional docs/clarification on what is/isn't permitted in config YAML files

[nmvega]

Hello Friends,

I recently used vsCode Copilot (which has knowledge of CrewAi) to generate the sample ./agents.yaml file below. I wanted to see all possible attribute/value pairs I could specify in the YAML file instead of inside CreaAi code directly.

I’ll stipulate that vsCode Copilot could be wrong, but let’s go with it for the purpose of the question.

My main concern is how robust can entries in the YAML file be – for any attribute?

Let us use the ./agents.yaml :: llm attribute as an example. The Agent attributes docs section specifies these possible types for the llm: attribute:

LLM (optional)	  llm	    Union[str, LLM, Any]

Those make sense when using the LLM() class. However can, say, the LLM type also be specified in the YAML file by specifying a Python identifier to one? For example:

First, define this in, say, a helper module...: my_llm = LLM(...)
Then, in the YAML file........................: llm: my_llm

The documentation doesn’t detail what is permitted for YAML file attributes and values - it only provides an example.

Anyway, the reason this question arose, apart from the documentation not offering detail, is that attempting the following raised an exception:

YAML snippet:

[ ... ] # Again, this was suggested by vsCode Copilot and could be wrong.
   llm: # This translates to a Python dict() type.
      model: "gpt-3.5-turbo"  # Default model
      temperature: 0.7  # Default temperature
[ ... ]

Type Exception (dict):

TypeError: unhashable type: 'dict' # Refers to the YAML llm dict().

The above outcome suggests that complex types are not permitted for YAML attribute values, only simple types like this:

llm: ollama/phi4:latest # This translates to a simply str() type.

Does anyone have detailed information on what it allowed in these YAML files? See llm:, tools: [...], etc. below.

Thank you.

# Generated by vsCode Copilot:
researcher:
  role: {topic} Senior Data Researcher
  goal: Uncover cutting-edge developments in {topic}
  backstory: Some backstory.

  verbose: true
  max_iter: 5
  max_rpm: 10
  allow_delegation: false
  function_calling_llm: null
  knowledge: null
  knowledge_sources: []
  embedder: null
  step_callback: null
  tools: []  # Do I specify an identifier (variable) for a tool here.
  llm:
    model: "gpt-3.5-turbo"
    temperature: 0.7

[rthidden]

I asked the same question. I hope there is an answer soon.

[nmvega]

Awesome. Thank you for letting me know and for asking it.

[maxmoura]

Hi everyone,

This solution I’m going to present aims to clarify how to use the tools and llm attributes in the YAML configuration files for agents.

Understanding the problem

When replicating the problem, you’ll basically encounter an error in a dictionary key, specifically in the crewai/project/crew_base.py file. Let’s look at the first lines of the method that raises the exception:

def map_agent_variables(
    self,
    agent_name: str,
    agent_info: Dict[str, Any],
    agents: Dict[str, Callable],
    llms: Dict[str, Callable],
    tool_functions: Dict[str, Callable],
    cache_handler_functions: Dict[str, Callable],
    callbacks: Dict[str, Callable],
) -> None:
    if llm := agent_info.get("llm"):
        try:
            self.agents_config[agent_name]["llm"] = llms[llm]()
        except KeyError:
            self.agents_config[agent_name]["llm"] = llm
    if tools := agent_info.get("tools"):
        self.agents_config[agent_name]["tools"] = [
            tool_functions[tool]() for tool in tools
        ]

Ok, let’s see what it does with the llm attribute from the YAML. It first searches for a key with the name you set to llm (a string). If it finds this key in the llms dict, then it runs the callable associated with that key. If the string you provided is not a key (which happens often since llms = {} by default), then it simply uses that string as the model parameter for a crewai.LLM object later. If you don’t believe me, go read the code yourself.

And what about the tools parameter from the YAML, which is a list of strings? Well, it tests each string as a key in the tools dict then it runs the callable associated with that key.

Finding the solution

The llms and tools dicts are populated in a scan that’s done in a previous method named map_all_agent_variables, which I’ll omit.

This method scans the decorators that are defined in crewai/project/__init__.py. You’re certainly already used to saying from crewai.project import CrewBase, agent, task, crew, the new thing is that you can (and should) say from crewai.project import tool, llm. That’s the key insight.

Once you do this, the callables decorated with @tool will be automatically listed in the tools dict and so the callables decorated with @llm will be automatically listed in the llms dict. This is elegant.

Semi-functional example

File max_agents.yaml:

max_agent:
  role: >
    Max Role
  goal: >
    Max Goal
  backstory: >
    Max Backstory
  verbose: true
  llm: max_llm
  tools:
    - max_tool

File max_tasks.yaml:

max_task:
  description: >
    Max's question is: {user_question}
  expected_output: >
    Max answer
  agent: max_agent

File crew.py:

from crewai import Agent, Crew, Task, Process, LLM
from crewai.project import CrewBase, agent, task, crew, tool, llm
from crewai.tools import BaseTool
from crewai_tools import DirectoryReadTool
import os

os.environ['LLM_API_KEY'] = 'YOUR_KEY_NOT_MINE'

@CrewBase
class MaxCrew:
    agents_config = 'max_agents.yaml' 
    tasks_config = 'max_tasks.yaml'
    
    @agent
    def max_agent(self) -> Agent:
        return Agent(
            config=self.agents_config['max_agent'],
        )
    
    @task
    def max_task(self) -> Task:
        return Task(
            config=self.tasks_config['max_task'],
        )
    
    @tool
    def max_tool(self) -> BaseTool:
        return DirectoryReadTool(directory='./')
    
    @llm
    def max_llm(self) -> LLM:
        return LLM(
            model='max/maxgpt-999b-chat',
            temperature=0.7,
            timeout=90,
            max_tokens=512,
        )
    
    @crew
    def crew(self) -> Crew:
        return Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential,
            verbose=True,
        )

Suggestions

I suggest that the CrewAI team enriches the documentation on this topic and perhaps even make the crewai create crew command add this import to the crew.py file, as this would encourage the use of this approach.

[nmvega]

@maxmoura Thank you for this delightfully well-written walkthrough. I’ll have a careful study of what you wrote (probably a few times) so I can understand the underlying mechanics and best-practices. I may have a follow-on question.

[nmvega]

@maxmoura Ah, I see. Using the @llm decorator allows one to fully define custom LLM(...) configurations arbitrarily - perhaps even storing collections of them in a dedicated module for reuse. Then, these named configurations are assigned to specific Agent stanzas (in ./agents.yaml) based on LLM (model) capabilities sought for that specific Agent.

In a similar manner, the @tool decorator works the same way.

Therefore, YAML attribute values needn’t be complex structures, since those details are supplied in the decorated and/or subclass definitions; and one simply just references their identifiers (variables).