Azure OpenAI and CrewAI

tlallema · October 2, 2024, 1:45pm

Hello, I’m struggling to go through Azure Openai while crewai insists in calling openai. Is there a way to avoid this ? I didn’t find any useful info on this. Thanks for the support

nfomis · October 4, 2024, 5:51am

Run into the same problem. The solution is in setting the environment variables LiteLLM expects: see here Azure OpenAI | liteLLM. Then define the agent using LLM(model='azure/<model-name>')

My example looks as follows

# api key is an environment variable
AZURE_OPENAI_API_KEY = os.getenv('AZURE_OPENAI_API_KEY')

# configuration are stored in a config dict
API_VERSION = config['api-version']
AZURE_OPENAI_ENDPOINT = config['openai-endpoint']
AZURE_OPENAI_LLM_DEPLOYMENT_NAME = config['llm-deployment-name']
LLM_MODEL_NAME = config['llm-model-name']

# set environment as LiteLLM expects
os.environ['AZURE_API_KEY'] = AZURE_OPENAI_API_KEY
os.environ["AZURE_API_BASE"] = AZURE_OPENAI_ENDPOINT
os.environ["AZURE_API_VERSION"] = API_VERSION

# define the agent
from crewai import Agent, LLM
agent = Agent(
   role='...',
   goal='...',
   ... ,
   llm=LLM(model=f'azure/{LLM_MODEL_NAME}')
)

tlallema · October 10, 2024, 2:56pm

Thank you ! I will try your suggestion

SMAshhar · October 30, 2024, 6:04pm

So I am facing a similar problem while using GROQ. During the initiation of the project, I selected groq to be the llm, but when I run “crewai run” it give me OPENAI_API_KEY error. Here is a full output:

PS E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew> crewai run
Running the Crew
# Agent: Experienced Business 
## Task: <some task>

2024-10-30 10:44:18,851 - 24252 - llm.py-llm:161 - ERROR: LiteLLM call failed: litellm.AuthenticationError: AuthenticationError: OpenAIException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
# Agent: Experienced Business 
## Task: <some task>
2024-10-30 10:44:19,303 - 24252 - llm.py-llm:161 - ERROR: LiteLLM call failed: litellm.AuthenticationError: AuthenticationError: OpenAIException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
# Agent: Experienced Business 
## Task: <some task>

2024-10-30 10:44:19,739 - 24252 - llm.py-llm:161 - ERROR: LiteLLM call failed: litellm.AuthenticationError: AuthenticationError: OpenAIException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
Traceback (most recent call last):
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\llms\OpenAI\openai.py", line 854, in completion
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\llms\OpenAI\openai.py", line 790, in completion
    self.make_sync_openai_chat_completion_request(
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\llms\OpenAI\openai.py", line 649, in make_sync_openai_chat_completion_request
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\llms\OpenAI\openai.py", line 631, in make_sync_openai_chat_completion_request
    raw_response = openai_client.chat.completions.with_raw_response.create(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\_legacy_response.py", line 356, in wrapped
    return cast(LegacyAPIResponse[R], func(*args, **kwargs))
                                      ^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\_utils\_utils.py", line 274, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\resources\chat\completions.py", line 815, in create
    return self._post(
           ^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\_base_client.py", line 1277, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\_base_client.py", line 954, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\openai\_base_client.py", line 1058, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\main.py", line 1587, in completion
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\main.py", line 1560, in completion
    response = openai_chat_completions.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\llms\OpenAI\openai.py", line 864, in completion
    raise OpenAIError(
litellm.llms.OpenAI.openai.OpenAIError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Scripts\run_crew.exe\__main__.py", line 8, in <module>
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\src\test_multi_crew\main.py", line 21, in run
    TestMultiCrewCrew().crew().kickoff(inputs=inputs)
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\crew.py", line 494, in kickoff
    result = self._run_sequential_process()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\crew.py", line 598, in _run_sequential_process
    return self._execute_tasks(self.tasks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\crew.py", line 696, in _execute_tasks
    task_output = task.execute_sync(
                  ^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\task.py", line 191, in execute_sync
    return self._execute_core(agent, context, tools)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\task.py", line 247, in _execute_core
    result = agent.execute_task(
             ^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agent.py", line 248, in execute_task
    result = self.execute_task(task, context, tools)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agent.py", line 248, in execute_task
    result = self.execute_task(task, context, tools)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agent.py", line 247, in execute_task
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agent.py", line 236, in execute_task
    result = self.agent_executor.invoke(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agents\crew_agent_executor.py", line 93, in invoke
    formatted_answer = self._invoke_loop()
                       ^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agents\crew_agent_executor.py", line 175, in _invoke_loop
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\agents\crew_agent_executor.py", line 115, in _invoke_loop
    answer = self.llm.call(
             ^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\crewai\llm.py", line 155, in call 
    response = litellm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\utils.py", line 1013, in wrapper
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\utils.py", line 903, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\main.py", line 2999, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 2116, in exception_type
    raise e
  File "E:\Work\Practice\7. GenAI Practice\00.PRACTICE\Learn\test_multi_crew\.venv\Lib\site-packages\litellm\litellm_core_utils\exception_mapping_utils.py", line 343, in exception_type
    raise AuthenticationError(
litellm.exceptions.AuthenticationError: litellm.AuthenticationError: AuthenticationError: OpenAIException - Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk_askdm**************************************234t. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
An error occurred while running the crew: Command '['uv', 'run', 'run_crew']' returned non-zero exit status 1.```

SMAshhar · October 31, 2024, 6:43am

I tried using LLM function to pass a custom LLM,
Here is my code:

import os
from crewai import Agent, Crew, Process, Task, LLM
from crewai.project import CrewBase, agent, crew, task

GROQ_API_KEY = os.getenv('GROQ_API_KEY')


llm = LLM(
    model="groq/llama-3.1-70b-versatile",
    base_url="https://api.groq.com",
    api_key=GROQ_API_KEY
)

@CrewBase
class TestMultiCrewCrew():
	"""TestMultiCrew crew"""

	@agent
	def planner(self) -> Agent:
		return Agent(
			config=self.agents_config['planner'],
			# tools=[MyCustomTool()], # Example of custom tool, loaded on the beginning of file
			verbose=True,
			llm=llm
		)

	@agent
	def writer(self) -> Agent:
		return Agent(
			config=self.agents_config['writer'],
			verbose=True,
			llm=llm
		)

And here is the error that I faced:

raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: GroqException - Error code: 404 - {'error': {'message': 'Unknown request URL: POST /chat/completions. Please check the URL for typos, or see the docs at https://console.groq.com/docs/', 'type': 'invalid_request_error', 'code': 'unknown_url'}}
An error occurred while running the crew: Command '['uv', 'run', 'run_crew']' returned non-zero exit status 1.

I did confirm several times, that the base_url is indeed https://api.groq.com.

rokbenko · October 31, 2024, 10:22am

Isn’t the base URL https://api.groq.com/openai/v1/?

SMAshhar · October 31, 2024, 11:12am

okay… okay… my 15 hrs … gone… to waste… TO WASTE…!!!

btw… thanks a lot man. You saved me. Thank you so much.

rokbenko · October 31, 2024, 11:32am

@SMAshhar I would be happy if you like my answer above with clicking the heart button.

Nisar_Khan · December 9, 2024, 8:24am

Have these set up:
os.environ[“OPENAI_API_KEY”] = “”

Azure_model = AzureChatOpenAI(
api_version=“2024-08-06”,
azure_deployment=“gpt4o”,
azure_endpoint=“”,
)
And pass your Azure_model to your Agent:
writer = Agent(
role=‘Content Writer’,
goal=‘Craft engaging blog posts about the AI industry’,
backstory=‘A skilled writer with a passion for technology.’,
tools=[docs_tool, file_tool],
verbose=True,
llm = Azure_model
)
Andd thats it.
I solved my problem using the below link, though:

Topic		Replies	Views
Use Case: Configuring LLM with Azure API Management Crews openai , agent , crewai	1	249	September 10, 2024
Error while kicking off the Crew with custom LLM - AzureOpenAI CrewAI Community Support tools_issues , agent , crewai	10	186	November 15, 2024
Automated Project notebook Gemini CrewAI Community Support	4	76	January 5, 2025
Parameters and LLM settings in crewai v0.61.0 CrewAI Community Support	12	230	September 24, 2024
litellm.exceptions.AuthenticationError: litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable CrewAI Community Support	2	141	December 22, 2024

Azure OpenAI and CrewAI

Related topics