Unable to connect CrewAI with the huggingface models

CrewAI Community Support
1 
pip install crewai==0.28.8 crewai_tools==0.1.6

...
 huggingface = HuggingFaceHub(
     repo_id=repo_id,
    huggingfacehub_api_token=api_key_read,
    task="text-generation",)

The above older version is working fine, but on using the below syntax based on my understanding, I was getting error at the time of execution. please let me know how to fix it

crewai==0.80.0
crewai-tools==0.14.0

...
huggingface = LLM(
    model="huggingface/meta-llama/Llama-3.2-3B-Instruct",  # or another available model
    base_url="https://api-inference.huggingface.co/models/",
    api_key=api_key_write
)
2

What error did you get?

3 
(new_env_name_t3) (base) vinaymittal@dyn-118-139-36-168 my_content_app % python app1.py     
# Agent: Content Planner
## Task: 1. Prioritize the latest trends, key players, and noteworthy news on AI.
2. Identify the target audience, considering their interests and pain points.
3. Develop a detailed content outline including an introduction, key points, and a call to action.
4. Include SEO keywords and relevant data or sources.
# Agent: Content Planner
## Task: 1. Prioritize the latest trends, key players, and noteworthy news on AI.
2. Identify the target audience, considering their interests and pain points.
3. Develop a detailed content outline including an introduction, key points, and a call to action.
4. Include SEO keywords and relevant data or sources.
# Agent: Content Planner
## Task: 1. Prioritize the latest trends, key players, and noteworthy news on AI.
2. Identify the target audience, considering their interests and pain points.
3. Develop a detailed content outline including an introduction, key points, and a call to action.
4. Include SEO keywords and relevant data or sources.
Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 2035, in completion
    model_response = huggingface.completion(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 725, in completion
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 714, in completion
    return self.convert_to_model_response_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 358, in convert_to_model_response_object
    raise HuggingfaceError(
litellm.llms.huggingface_restapi.HuggingfaceError: response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 282, in execute_task
    result = self.agent_executor.invoke(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 93, in invoke
    formatted_answer = self._invoke_loop()
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 185, in _invoke_loop
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 115, in _invoke_loop
    answer = self.llm.call(
             ^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/llm.py", line 155, in call
    response = litellm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 960, in wrapper
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 849, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 3060, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2136, in exception_type
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1438, in exception_type
    raise APIError(
litellm.exceptions.APIError: litellm.APIError: HuggingfaceException - response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 2035, in completion
    model_response = huggingface.completion(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 725, in completion
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 714, in completion
    return self.convert_to_model_response_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 358, in convert_to_model_response_object
    raise HuggingfaceError(
litellm.llms.huggingface_restapi.HuggingfaceError: response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 282, in execute_task
    result = self.agent_executor.invoke(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 93, in invoke
    formatted_answer = self._invoke_loop()
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 185, in _invoke_loop
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 115, in _invoke_loop
    answer = self.llm.call(
             ^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/llm.py", line 155, in call
    response = litellm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 960, in wrapper
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 849, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 3060, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2136, in exception_type
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1438, in exception_type
    raise APIError(
litellm.exceptions.APIError: litellm.APIError: HuggingfaceException - response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 2035, in completion
    model_response = huggingface.completion(
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 725, in completion
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 714, in completion
    return self.convert_to_model_response_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/llms/huggingface_restapi.py", line 358, in convert_to_model_response_object
    raise HuggingfaceError(
litellm.llms.huggingface_restapi.HuggingfaceError: response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vinaymittal/Desktop/Trial/my_content_app/app1.py", line 53, in <module>
    crew_output = crew.kickoff(inputs={"topic": "AI"})
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/crew.py", line 495, in kickoff
    result = self._run_sequential_process()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/crew.py", line 599, in _run_sequential_process
    return self._execute_tasks(self.tasks)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/crew.py", line 697, in _execute_tasks
    task_output = task.execute_sync(
                  ^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/task.py", line 192, in execute_sync
    return self._execute_core(agent, context, tools)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/task.py", line 248, in _execute_core
    result = agent.execute_task(
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 294, in execute_task
    result = self.execute_task(task, context, tools)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 294, in execute_task
    result = self.execute_task(task, context, tools)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 293, in execute_task
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agent.py", line 282, in execute_task
    result = self.agent_executor.invoke(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 93, in invoke
    formatted_answer = self._invoke_loop()
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 185, in _invoke_loop
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/agents/crew_agent_executor.py", line 115, in _invoke_loop
    answer = self.llm.call(
             ^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/crewai/llm.py", line 155, in call
    response = litellm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 960, in wrapper
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/utils.py", line 849, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/main.py", line 3060, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2136, in exception_type
    raise e
  File "/Users/vinaymittal/Desktop/Trial/new_env_name_t3/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1438, in exception_type
    raise APIError(
litellm.exceptions.APIError: litellm.APIError: HuggingfaceException - response is not in expected format - [{'error': 'HfApiJson(Deserialize(Error("invalid type: map, expected a string", line: 1, column: 1)))'}]

This is what i am getting