APIError: litellm.APIError: AzureException APIError - argument of type 'NoneType' is not iterable

Hi,
I’m unable to access Azure OpenAI from CrewAI agents.
This is my python notebook code. Can someone please check this and help me fix this?

os.environ["AZURE_OPENAI_API_KEY"] = <KEY1>
os.environ["AZURE_OPENAI_ENDPOINT"] = <Endpoint URL>

##Agents
extract_cli_marker = Agent(
    role="Senior Medical Researcher",
    goal="Identify and analyze clinical markers for heart failure in provided data",
    verbose=True,
    memory=True,
    backstory=(
        "With a strong background in cardiology and clinical research, "
        "you specialize in identifying key biomarkers and diagnostic criteria "
        "for heart failure. Your expertise allows you to analyze complex clinical data "
        "and provide actionable insights for diagnosis and treatment."
    ),
    tools=[ScrapeWebsiteTool()],
    llm="azure/gpt-4o",
    allow_delegation=True
)
identify_hf = Agent(
    role="Cardiology Specialist",
    goal=f"Determine if a patient has heart failure based on upon the given medical documents.",
    verbose=True,
    memory=True,
    backstory=(
        """As a cardiology specialist, you have extensive experience in diagnosing heart failure
        using clinical data, lab results, and imaging studies. Your expertise ensures accurate
        and timely diagnosis for patients.
        """
    ),
    llm="azure/gpt-4o",
    allow_delegation=False
)

##tasks
task_extract_cli_marker = Task(
    description=(
        "Analyze the provided clinical data to identify key markers for heart failure using the top validated websites at internet and with your own knowledge. Combine them come up with a consildate marker"
        "Focus on identifying relevant biomarkers, symptoms, and diagnostic criteria. "
        "Your final report should clearly articulate the clinical markers, "
        "their significance, and how they contribute to the diagnosis of heart failure."
    ),
    expected_output=(
        "A detailed report identifying the clinical markers for heart failure, "
        "including biomarkers, symptoms, and diagnostic criteria. "
        "The report should be 3 paragraphs long and provide a clear explanation "
        "of the significance of each marker."
    ),
    agent=extract_cli_marker,
)
task_identify_hf = Task(
    description=(
        f"Analyze the medical documents of the given patient to determine if they have heart failure. Medical document : {subject_notes}"
        "Use tools to retrieve the patient's medical records, including lab results, imaging reports, and clinical notes. "
        "Focus on identifying key markers such as elevated BNP/NT-proBNP levels, reduced ejection fraction (LVEF < 40%), "
        "and symptoms like shortness of breath, fatigue, or peripheral edema. "
        "Provide a clear yes/no answer and a line of evidence supporting the diagnosis."
    ),
    expected_output=(
        "A concise answer (yes/no) indicating whether the patient has heart failure, "
        "followed by a line of evidence supporting the diagnosis. "
        "For example: 'Yes, the patient has heart failure due to an elevated BNP level of 800 pg/mL.'"
    ),
    agent=identify_hf
)

crew=Crew(
    agents=[extract_cli_marker, identify_hf],
    tasks=[task_extract_cli_marker, task_identify_hf],
    process=Process.sequential,
)

result=crew.kickoff(inputs={'topic':'Identify if the patient has Heart Failure'})

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\litellm\llms\azure\azure.py:353, in AzureChatCompletion.completion(self, model, messages, model_response, api_key, api_base, api_version, api_type, azure_ad_token, dynamic_params, print_verbose, timeout, logging_obj, optional_params, litellm_params, logger_fn, acompletion, headers, client)
    351 ### CHECK IF CLOUDFLARE AI GATEWAY ###
    352 ### if so - set the model as part of the base url
--> 353 if "gateway.ai.cloudflare.com" in api_base:
    354     ## build base url - assume api base includes resource name
    355     if client is None:

TypeError: argument of type 'NoneType' is not iterable

During handling of the above exception, another exception occurred:

AzureOpenAIError                          Traceback (most recent call last)
File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\litellm\main.py:1185, in completion(model, messages, timeout, temperature, top_p, n, stream, stream_options, stop, max_completion_tokens, max_tokens, modalities, prediction, audio, presence_penalty, frequency_penalty, logit_bias, user, response_format, seed, tools, tool_choice, logprobs, top_logprobs, parallel_tool_calls, deployment_id, extra_headers, functions, function_call, base_url, api_version, api_key, model_list, **kwargs)
   1184     ## COMPLETION CALL
-> 1185     response = azure_chat_completions.completion(
   1186         model=model,
   1187         messages=messages,
   1188         headers=headers,
   1189         api_key=api_key,
   1190         api_base=api_base,
   1191         api_version=api_version,
   1192         api_type=api_type,
   1193         dynamic_params=dynamic_params,
...
   (...)
   2045         request=httpx.Request(method="POST", url="https://openai.com/"),
   2046     )

APIError: litellm.APIError: AzureException APIError - argument of type 'NoneType' is not iterable

I will try my best to help you with this as getting things to work with Azure is really cool but can be tricky. Here are the things that should either be in your .env file or your OS Environment variables as you have them. I will post next on how to find these in Azure.

Finding these items in Azure:

  • login to the Azure Portal
  • Click on your OpenAi resource
  • you should see your LLM listed with an icon like this
    image
    *Click on that to open your subsciption to MS Cognitive services
    *Scroll down and click on this
    image

Menu on left scroll down and click on Deployments
image

Select View Code

Be sure to click the second tab which shows code for Key Authentication.

Everything you need is in there… in fact it is best to get this code working first and then try your Crew. Please let me know if it worked or if you still have questions.

Hi @joab.io
Thanks for the response.

I have these env variables in the code.
os.environ[“AZURE_OPENAI_API_KEY”] =
os.environ[“AZURE_OPENAI_ENDPOINT”] =
os.environ[“AZURE_API_VERSION”] = “2024-11-20”

Still, getting the same error.


TypeError Traceback (most recent call last)
File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\litellm\llms\azure\azure.py:353, in AzureChatCompletion.completion(self, model, messages, model_response, api_key, api_base, api_version, api_type, azure_ad_token, dynamic_params, print_verbose, timeout, logging_obj, optional_params, litellm_params, logger_fn, acompletion, headers, client)
351 ### CHECK IF CLOUDFLARE AI GATEWAY ###
352 ### if so - set the model as part of the base url
→ 353 if “gateway.ai.cloudflare.com” in api_base:
354 ## build base url - assume api base includes resource name
355 if client is None:

TypeError: argument of type ‘NoneType’ is not iterable

During handling of the above exception, another exception occurred:

AzureOpenAIError Traceback (most recent call last)
File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\litellm\main.py:1185, in completion(model, messages, timeout, temperature, top_p, n, stream, stream_options, stop, max_completion_tokens, max_tokens, modalities, prediction, audio, presence_penalty, frequency_penalty, logit_bias, user, response_format, seed, tools, tool_choice, logprobs, top_logprobs, parallel_tool_calls, deployment_id, extra_headers, functions, function_call, base_url, api_version, api_key, model_list, **kwargs)
1184 ## COMPLETION CALL
→ 1185 response = azure_chat_completions.completion(
1186 model=model,
1187 messages=messages,
1188 headers=headers,
1189 api_key=api_key,
1190 api_base=api_base,
1191 api_version=api_version,
1192 api_type=api_type,
1193 dynamic_params=dynamic_params,

(…)
2045 request=httpx.Request(method=“POST”, url=“https://openai.com/”),
2046 )

APIError: litellm.APIError: AzureException APIError - argument of type ‘NoneType’ is not iterablestrong text

you have these variables :

The variable names that I have that are working great with Azure are:

“AZURE_API_KEY”] =
“AZURE_API_BASE”] =
“AZURE_API_VERSION”] = “2024-11-20”

Did you try using these same exact environment variable names, your variable names are different?

@joab.io
Thanks again for the response.

After changing the env variables, I’m getting this error now.
APIError: litellm.APIError: AzureException APIError - Error code: 404 - {‘error’: {‘code’: ‘404’, ‘message’: ‘Resource not found’}}

This is the model deplyment in Azure AI Foundry. I checked network and the OpenAI instance should be accessible.

Any idea what I’m missing?

This is good progress! Try one thing at the command line
az login

follow the instructions - should only have to do this once. See if that helps. Should not matter but try it just in case.

It looks to me like the Endpoint is not being found. Click on the blue link ‘gpt-4o’ in the Name column above that you showed. This brings you to the

Chat playground

In the prompt box to the right, ask the LLM to tell you a joke. If that works then move on.

Next click on View code and be sure about your API Version

this is what mine is showing today:
2024-05-01-preview

Set your api version key to whatever it says in the code. Also double check the endpoint is the same as what is shown in the sample code.

If that still give a 404 error then we need to check your Crew code for how you are specifying the LLM. Best to use YAML files. Let me know how this goes and then we can dig into the next possibility.

It worked now.
The problem was, I was not giving the full URL for the AZURE_API_BASE. There was no need to run the “az login” command.

I really appreciate your responses.
Thank you @joab.io

2 Likes

@joab.io
Code is running fine when I use ScrapeWebsiteTool. I’m getting an error when I try to use WebsiteSearchTool.

Please point me to the documentation that can help me in using the WebsiteSearchTool.


KeyError Traceback (most recent call last)
Cell In[95], line 1
----> 1 web_rag_tool = WebsiteSearchTool()

File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\crewai_tools\tools\website_search\website_search_tool.py:34, in WebsiteSearchTool.init(self, website, **kwargs)
33 def init(self, website: Optional[str] = None, **kwargs):
—> 34 super().init(**kwargs)
35 if website is not None:
36 kwargs[“data_type”] = DataType.WEB_PAGE

[... skipping hidden 1 frame]

File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\crewai_tools\tools\rag\rag_tool.py:46, in RagTool._set_default_adapter(self)
42 from embedchain import App
44 from crewai_tools.adapters.embedchain_adapter import EmbedchainAdapter
—> 46 app = App.from_config(config=self.config) if self.config else App()
47 self.adapter = EmbedchainAdapter(
48 embedchain_app=app, summarize=self.summarize
49 )
51 return self

File c:\Users\userID\AppData\Local\anaconda3\envs\crewai\lib\site-packages\embedchain\app.py:121, in App.init(self, id, name, config, db, embedding_model, llm, config_data, auto_deploy, chunker, cache_config, memory_config, log_level)
118 if name is not None:
119 self.name = name

679 # raise KeyError with the original key value
→ 680 raise KeyError(key) from None
681 return self.decodevalue(value)

KeyError: ‘OPENAI_API_KEY’

I think this is related to the difference between accessing the LLM directly with OPENAI and through Azure. Step 1:

Run this code and be sure you do not have any keys loading for OPENAI from past work.

Delete any keys relating to OpenAi and remove anything related to OPENAI_API_KEY as you want to access your LLM through Azure just like before. They might look like __CONDA_SHLVL_1_OPENAI_API_KEY:

Next, specify your LLM to be azure and your LLM model gpt4o following this guide:
Website RAG Search - CrewAI

Use the same keys as before using a .env file if possible.

Best of luck, I hope this works. Let me know.

I ran the command you provided and did not find any OpenAI related values in the output.

According to the https://docs.crewai.com/tools/websitesearchtool#customization-options, I tried different configs, but none worked.

tool = WebsiteSearchTool(
config=dict(
llm=dict(
provider=“azure_openai”,
config=dict(
api_key=<AZURE_API_KEY>,
model=“gpt-4o”,
deployment_name=“gpt-4o”, # Azure deployment name
),
),
embedder=dict(
provider=“azure_openai”,
config=dict(
api_key=<AZURE_API_KEY>
model=“text-embedding-3-large”,
deployment_name=“text-embedding-3-large”, # Azure deployment name
),
),
)
)

OpenAIError: Missing credentials. Please pass one of api_key, azure_ad_token, azure_ad_token_provider, or the AZURE_OPENAI_API_KEY or AZURE_OPENAI_AD_TOKEN environment variables.

looks like a different error and it is NOT finding your
AZURE_OPENAI_API_KEY

Try using a .env file with the following:
AZURE_API_BASE=https://<yourname>.openai.azure.com
AZURE_API_KEY=<your_key>
AZURE_OPENAI_API_KEY=<your_same_key>
AZURE_API_VERSION=<your_api_version>

also try:

provider=“azure" above in your dictionaries

Thanks @joab.io

I’m able to use WebsiteSearchTool after adding AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT to the environment variables.

Also, looks like the provider has to be azure_openai. I got this error when I changed it to azure.

SchemaError: Or(‘openai’, ‘azure_openai’, ‘anthropic’, ‘huggingface’, ‘cohere’, ‘together’, ‘gpt4all’, ‘ollama’, ‘jina’, ‘llama2’, ‘vertexai’, ‘google’, ‘aws_bedrock’, ‘mistralai’, ‘clarifai’, ‘vllm’, ‘groq’, ‘nvidia’) did not validate ‘azure’
‘openai’ does not match ‘azure’
‘azure_openai’ does not match ‘azure’
‘anthropic’ does not match ‘azure’
‘huggingface’ does not match ‘azure’
‘cohere’ does not match ‘azure’
‘together’ does not match ‘azure’
‘gpt4all’ does not match ‘azure’
‘ollama’ does not match ‘azure’
‘jina’ does not match ‘azure’
‘llama2’ does not match ‘azure’

‘mistralai’ does not match ‘azure’
‘clarifai’ does not match ‘azure’
‘vllm’ does not match ‘azure’
‘groq’ does not match ‘azure’
‘nvidia’ does not match ‘azure’

1 Like

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.