Connect-crewai-to-llms

[TzurV]

I’m failing to use different llm in agents beside the OpenAI ones.
I’ve tried following the instructions in

Connect CrewAI to LLMs - crewAI

I’m using crewai 0.51.1.

do I need to create a Custom agent and LLM wrapper that uses different LLMs, such as Anthropic or Cohere ?

[Dabnis]

This is how I manage my LLMs:
llms.py

import os
from langchain_community.llms import OpenAI, Ollama
from langchain_openai import ChatOpenAI
from langchain_groq import ChatGroq

@staticmethod
class LLMS:
    def __init__(self):
        self.OpenAIGPT35 = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.7)
        self.OpenAIGPT4oMini = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.8)
        self.OpenAIGPT4o = ChatOpenAI(model_name="gpt-4o", temperature=0.8)
        self.OpenAIGPT4 = ChatOpenAI(model_name="gpt-4", temperature=0.8)
        # self.Phi3 = Ollama(model="phi3:mini")
        self.Llama3_1 = Ollama(model="llama3.1")
        self.Phi3 = Ollama(model="phi3:medium-128k")
        # self.Phi3 = ChatOpenAI(model_name="phi3:medium-128k", temperature=0, api_key="ollama", base_url="http://localhost:11434")
        self.groqLama3_8B_3192 = ChatGroq(temperature=0.5, groq_api_key=os.environ.get("GROQ_API_KEY"),
                                          model_name="llama3-8b-8192")

From within the crew/agent set-up:

#Import the LLMs class into your crew.py
from base.llms import LLMS

# Add it to the crew
@CrewBase
class PydanticCrew:
    """Pydantic crew"""
    agents_config = 'config/agents.yaml'
    tasks_config = 'config/tasks.yaml'

    def __init__(self, pd_mdl: Any):
        super().__init__()
        self.llms = LLMS()  # imported from llms.py
        self.pyd_model = pd_mdl
        self.OnlineSearchTool = SerperDevTool()

# Access it from your agent set-up:
@agent
    def online_researcher(self) -> Agent:
        return Agent(
            config=self.agents_config['online_researcher'],
            # tools=[self.OnlineSearchTool], # Example of custom tool, loaded on the beginning of file
            verbose=True,
            llm=self.llms.groqLama3_8B_3192
        )

Remember to set up your .env file

OPENAI_API_KEY=''
OPENAI_MODEL_NAME='gpt-4o-mini'
OPENAI_ORGANIZATION_ID = ''
GROQ_API_KEY = ''
SERPER_API_KEY=''
FIRECRAWL_API_KEY=''

# Not all are required!

I’m relatively new to Python so there may be a better way, but hope this helps you to get going.

[Dabnis]

FYI: I have just ran the agent below and it does appear to work!

 @agent
    def model_prompt_tuning(self) -> Agent:
        return Agent(
            config=self.agents_config['model_prompt_tuning'],
            # tools=[self.OnlineSearchTool],  # Example of custom tool, loaded on the beginning of file
            verbose=True,
            allow_delegation=False,
            llm='claude-2'
        )

Maybe you should use this!

A day when you learn something new is a good day

Given your situation I would suggest looking at your .env file.

[TzurV]

Hey @Dabnis,
Thanks for your reply.

A couple of questions:

• Which CrewAI version are you using?
• I’m not using agents_config or tasks_config. What are they doing, and what information do they hold?

[matt]

Be sure to upgrade to 0.60.0 you can see how that versions handles llms in our docs

[TzurV]

@Dabnis

CrewBase

Thanks. upgraging to 0.60.0 sorted the issues.

note that hithub documentation hyper link goes to crewAI when top right has v0.55.2. not sure it this is correct.

[matt]

it will be updated soon