Model does not exist

General
,
1

Can someone tell me what i should do? I am using hugging face API key and declaring the LLM in my agent as per the official documentation of crewai still I am facing this persistent issue.
ERROR:root:LiteLLM call failed: litellm.APIError: HuggingfaceException - {“error”:“Model mistralai/Mistral-7B-Instruct does not exist”}
Error during LLM call: litellm.APIError: HuggingfaceException - {“error”:“Model mistralai/Mistral-7B-Instruct does not exist”}
An unknown error occurred. Please check the details below.
Error details: litellm.APIError: HuggingfaceException - {“error”:“Model mistralai/Mistral-7B-Instruct does not exist”}

2

Hope this helps:

from crewai import LLM
import os

os.environ['HUGGINGFACE_API_KEY'] = '<your-hf-key>'

llm = LLM(
    model='huggingface/mistralai/Mistral-7B-Instruct-v0.3',
    api_base=('https://api-inference.huggingface.co/models/'
              'mistralai/Mistral-7B-Instruct-v0.3'),
    temperature=0.7
)

response = llm.call("Say 'hi Aadishree' in 300 chars. "
                    "She's a member of the CrewAI community!")

print(f'\n[ 🤖 ] {response.strip()}\n')
3

It didnt solve the problem and I had to buy credits for OpenAI API as any other API refused to integrate with the agent

2 Likes
4

The code provided here is still giving an error.

ERROR:root:LiteLLM call failed: Invalid message type: <class 'str'>. Expected dict or Pydantic model.

5

Which model are you using?

6

Okay, that’s pretty confusing. Are you running the code above exactly as it’s written there? Like, you’re just copying the entire block, setting your HUGGINGFACE_API_KEY, and running it? And you’re still getting an error?

7

Thats correct. I am not changing anything except adding the key. Other models such as those from Ollama works without any issue.

image
image727×666 45.9 KB

8

I’m starting to suspect it might be something related to the version of the LiteLLM library, which CrewAI uses to interface with the LLMs.

I ran the following code, based on the LiteLLM documentation, and got a valid output without any errors:

import litellm
from litellm import completion
import os

print(f"LiteLLM version: {litellm._version.version}\n")

os.environ["HUGGINGFACE_API_KEY"] = "hf_???"

response = completion(
  model="huggingface/mistralai/Mistral-7B-Instruct-v0.3",
  api_base="https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3",
  messages=[{ "content": "Hello, how are you?","role": "user"}],
)

print(response)

Ouput:

LiteLLM version: 1.60.2

ModelResponse(id='chatcmpl-1240b043-266c-431c-ab6f-90016a1df45f', created=1743806174[...]
1 Like
9

was experiencing same problem, but got able to keep up free tier of mistral
but I still have problems integrating non-openai embedding models for memory in crew

10

@Gurpreet_Grover, may I suggest that you start a new thread detailing your issues with embeddings? Ideally, include a small example that reproduces the errors you’ve been running into.

That way, topics stay better organized, making it easier for others facing the same challenges to find help.

11

I was on liteLLM version 1.57.1. The example you provided is for completion command, which worked for me even with 1.57.1. Then I upgraded to LiteLLM version: 1.60.2. The completion command work with this version as well. The problem still persists with the CrewAI LLM() function call. Can you please provide the updated example with CrewAI LLM() call? Thanks

1 Like
12

Alright, if the example provided worked with LiteLLM, then the one below should also work with CrewAI, since it’s basically a direct translation of that same example. Under the hood, CrewAI is just calling litellm.completion for you:

from crewai import LLM
import os

os.environ["HUGGINGFACE_API_KEY"] = "hf_???"

llm = LLM(
    model="huggingface/mistralai/Mistral-7B-Instruct-v0.3",
    api_base="https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3"
)

response = llm.call("Hello, how are you?")

print(f'\n[ 🤖 Response ]\n\n{response.strip()}\n')

If everything goes smoothly, now you know how to set up your Hugging Face LLM: just make sure to specify your model correctly and point to the right api_base URL. Good luck!

13

Thanks.Worked for me. Looks like the message was badly formatted.