Why am I getting the "Invalid response from LLM call - None or empty" error with my custom tool if using Anthropic LLM but not with OpenAI LLM?

[rokbenko]

Looks like Anthropic LLMs, even the more capable ones like Claude 3.5 Sonnet, sometimes have difficulties and produce the Received None or empty response from LLM call error as @frieda-huang reported in another topic.

Custom tool throws error `ValueError: Invalid response from LLM call - None or empty`

I switched from claude-3-5-sonnet-20241022 to gpt-4o and it now works! Is there a way we can have a fallback option where if one model fails to accomplish the task, another equally or more capable model can pick it up? I belive LiteLLM has this feature (Reliability - Retries, Fallbacks | liteLLM)

In case you’re using one of the Anthropic LLMs, set the use_system_prompt parameter to False to the agent as @rtk suggested above, but that might cause some other issues down the road since system prompts in CrewAI exist to help agents to create the best final answer possible, not to cause issues. Or, you could simply switch the LLM provider to OpenAI, for example, if you’re LLM provider agnostic.

So, keep this in mind besides the two general rules I’ve suggested above:

@rokbenko

There are two general rules:

• The newer the LLM family, the better. For example, Llama 3.2 family is a better choice than Llama 3.1 family.
• The higher the LLM parameter number, the better. For example, Llama 3.2 11B is a better choice than Llama 3.2 3B.

There’s no formula to say a specific LLM will work. Simply try and see.