Error while kicking off the Crew with custom LLM - AzureOpenAI

[dhavalkaku]

Hello Team,

I’m performing one operation using PDF Rag search, and in that I have added configuration of AzureOpenAI to read the PDF file for the user query.
Then I have created agent and tasks to perform the operations, but when I’m kicking off the crew, it is giving me the error as below:

llm.py-llm:161 - ERROR: LiteLLM call failed: litellm.NotFoundError: NotFoundError: OpenAIException - Error code: 404 - {‘error’: {‘code’: ‘404’, ‘message’: ‘Resource not found’}}

I double checked all the credentials which I’m trying but still no luck.

I’m attaching my code snapshot here for the reference.

[olaservo]

What does your Azure url look like?

[dhavalkaku]

It is like this : “https://{my-resource-name}.openai.azure.com/”

[dhavalkaku]

Is there any update for the same ?

[bigilad]

this is happenning to me as well. there is an issue when using crewai with azure openai endpoint due to crewai using litellm
CrewAI is using LiteLLM internally to make API calls
When we pass our model name gpt-4o_2024-08-06, LiteLLM doesn’t know which provider (Azure) to use…

[olaservo]

Are you setting the model using this format?

model = “azure/<your_deployment_name>”

[rokbenko]

@bigilad As @olaservo mentioned above, make sure to set the LLM provider before configuring the LLM. For Azure OpenAI, use azure/<your deployment name>. See the official LiteLLM docs on Azure OpenAI.

If you’re unsure how to do this for a specific LLM provider, refer to the LiteLLM Providers page for guidance.

from crewai import Agent, LLM

my_llm = LLM(
    api_key=os.getenv("AZURE_API_KEY"),
    model="azure/<your_deployment_name>",
),

my_agent = Agent(
    ...,
    llm=my_llm,
)

[bigilad]

in here it says:

AZURE_API_KEY= # “my-azure-api-key”
AZURE_API_BASE= # “https://example-endpoint.openai.azure.com”
AZURE_API_VERSION= # “2023-05-15”
AZURE_AD_TOKEN= # Optional
AZURE_API_TYPE= # Optional

and finally what actually worked is:

llm = AzureChatOpenAI(
        #azure_deployment="gpt-4",#gpt-4o_2024-08-06
        azure_deployment="gpt-4o",
        openai_api_version="2024-02-15-preview",
        azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
        api_key=os.getenv("AZURE_OPENAI_API_KEY"),
        temperature=0.7
    )

which is different when NOT using crewai:

llm = AzureChatOpenAI(
openai_api_key=os.getenv(“AZURE_OPENAI_API_KEY”),
azure_endpoint=os.getenv(“AZURE_OPENAI_ENDPOINT”),
AZURE_OPENAI_DEPLOYMENT_NAME=“gpt-4o_2024-08-06”,
openai_api_version=“2024-02-15-preview”,
temperature=0.7
)

azure is letting me define DEPLOYMENT_NAME and if , as i did define it, without crewai it will work with its full name, but when using crewai with litellm than

[dhavalkaku]

Yes, after initializing model with this format it worked.
Thank you !

[dhavalkaku]

Yes, after initializing model with this format for LLM it worked.
Thank you !