# How to use the qwen2.5-vl-3b-instruct model with the CrewAi?

**URL:** <https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456>\
**Category:** LLMs\
**Tags:** llama-31-8b\
**Created:** [March 7, 2025, 5:38am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456 "2025-03-07T05:38:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dhruvin\_5179](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/dhruvin_5179/32/2262_2.png) [@Dhruvin\_5179](https://community.crewai.com/u/Dhruvin_5179)\
**Post date:** [March 7, 2025, 5:38am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456/1 "2025-03-07T05:38:30Z")

</div>

"I am trying to use the Qwen series model with the Hugging Face provider locally, but it does not support CrewAI. I am using vLLM to host the model on an API endpoint, but when I call the model through the `LLM` class, I encounter an error.

I have tried multiple approaches, including using FastAPI to serve the model with `api_base` or `base_url`, but it still doesn’t work. I am using the Qwen model for image-based question answering. Below is the code I used:

python

CopyEdit

```auto
llm = LLM(
    model="huggingface/Qwen/Qwen2-VL-2B-Instruct",
    api_base="http://0.0.0.0:8000/v1/chat/completions",
)

```

Can you help me resolve this issue? Also, does `litellm` support the Qwen-Vision model? If yes, how can I use the Qwen model with CrewAI?

Thanks in advance!"

---

<div class="post-metadata">

**Author:** ![zinyando](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/zinyando/32/4435_2.png) [@zinyando](https://community.crewai.com/u/zinyando)\
**Post date:** [March 7, 2025, 10:00am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456/2 "2025-03-07T10:00:48Z")

</div>

LiteLLM supports all models on VLLM. [VLLM | liteLLM](https://docs.litellm.ai/docs/providers/vllm)

Try changing to:

```auto
llm = LLM(
    model="hosted_vllm/huggingface/Qwen/Qwen2-VL-2B-Instruct",
    api_base="http://0.0.0.0:8000/v1/chat/completions",
)

```

I think your issue is on the model path or the api\_base.

---

<div class="post-metadata">

**Author:** ![Dhruvin\_5179](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/dhruvin_5179/32/2262_2.png) [@Dhruvin\_5179](https://community.crewai.com/u/Dhruvin_5179)\
**Post date:** [March 11, 2025, 9:41am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456/3 "2025-03-11T09:41:39Z")

</div>

Hi @zinyando, I tried using the LLM class, but it didn’t work. Below is my full code. Please suggest how to use the Qwen model. If there is another idea or technique to use the Qwen model, please let me know.

```
from crewai import Agent, Task, Crew, LLM
llm = LLM(
    model="hosted_vllm/huggingface/Qwen/Qwen2-VL-2B-Instruct",
    api_base="http://0.0.0.0:8000/v1/chat/completions",
)

image_analyst = Agent(
    role="Image Analyst",
    goal="Analyze the given image and provide detailed insights based on the provided question",
    backstory="Expert in visual analysis with deep knowledge of design, composition, objects, patterns, and features. Can accurately describe and interpret images across various contexts.",
    multimodal=True,
    verbose=True,
    llm=llm
)

task = Task(
    description=(
        "Analyze the image located at https://as1.ftcdn.net/v2/jpg/10/64/66/34/1000_F_1064663493_PG2uY9VYvZPVxXvmFwIOvGDSNlfMDeIL.jpg and describe the word written on the image."
    ),
    expected_output="A detailed analysis and response based on the given image and question.",
    agent=image_analyst
)

crew = Crew(
    agents=[image_analyst],
    tasks=[task]
)

result = crew.kickoff()

```

Error:  
`litellm.exceptions.NotFoundError: litellm.NotFoundError: NotFoundError: Hosted_vllmException - Error code: 404 - {'detail': 'Not Found'}`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex025/uploads/crewai/original/1X/ecda97e72f77c7100e0a761444b1c991925cc789.png) [@system](https://community.crewai.com/u/system)\
**Post date:** [April 6, 2025, 5:38am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456/4 "2025-04-06T05:38:41Z")

</div>

This topic was automatically closed after 30 days. New replies are no longer allowed.