# \#llama-31-8b **URL:** https://community.crewai.com/tag/llama-31-8b/5.md [Latest](https://community.crewai.com/latest.md) · [Categories](https://community.crewai.com/categories.md) · [Tags](https://community.crewai.com/tags.md) --- ## [How are people handling execution auditability and governance in production CrewAI deployments?](https://community.crewai.com/t/how-are-people-handling-execution-auditability-and-governance-in-production-crewai-deployments/7589)
**Author:** [@Tanishq1030](https://community.crewai.com/u/Tanishq1030)\ **Replies:** 0\ **Last updated:** [June 19, 2026, 10:35am UTC](https://community.crewai.com/t/how-are-people-handling-execution-auditability-and-governance-in-production-crewai-deployments/7589 "2026-06-19T10:35:07Z")
As agent systems become more autonomous and gain access to tools, external APIs, file systems, databases, and execution environments, I’ve become increasingly curious about how teams are approaching governance and audita… --- ## [Anyone managed to use mcp server with ollama model provider?](https://community.crewai.com/t/anyone-managed-to-use-mcp-server-with-ollama-model-provider/6427)
**Author:** [@smrati\_katiyar](https://community.crewai.com/u/smrati_katiyar)\ **Replies:** 2\ **Last updated:** [July 25, 2025, 10:36pm UTC](https://community.crewai.com/t/anyone-managed-to-use-mcp-server-with-ollama-model-provider/6427 "2025-07-25T22:36:29Z")
Hey everyone , I am trying to use a local llm while using mcp servers in crew ai i can see in verbose log that llm is able to identify correct tool and able to run it and receive output from tool but after that step it… --- ## [Help ::: How to use a custom (local) LLM with vLLM](https://community.crewai.com/t/help-how-to-use-a-custom-local-llm-with-vllm/5746)
**Author:** [@likhithananda](https://community.crewai.com/u/likhithananda)\ **Replies:** 1\ **Last updated:** [May 13, 2025, 6:15am UTC](https://community.crewai.com/t/help-how-to-use-a-custom-local-llm-with-vllm/5746 "2025-05-13T06:15:33Z")
Help me understand what I am missing while trying to use a custom LLM with vLLM (by loading a model from local path - offline inferencing). Here is the full code. # Install required packages (run in terminal if needed):… --- ## [Gemini 2.5 Flash Preview and Gemma3:1b/27b, Big Difference in Output for same task definition](https://community.crewai.com/t/gemini-2-5-flash-preview-and-gemma3-1b-27b-big-difference-in-output-for-same-task-definition/5577)
**Author:** [@hasanfarag199](https://community.crewai.com/u/hasanfarag199)\ **Replies:** 1\ **Last updated:** [April 29, 2025, 6:07pm UTC](https://community.crewai.com/t/gemini-2-5-flash-preview-and-gemma3-1b-27b-big-difference-in-output-for-same-task-definition/5577 "2025-04-29T18:07:59Z")
Hi All, I am experimenting with crew ai for Industry Intelligence reporting with the below agents and tasks definitions, the results sounds satifsfactory using Gemini 2.5 Flash \[ev\_report.pdf\] but quite short for Gemma3… --- ## [Handling LLM Errors in Hierarchical CrewAI Process with Callbacks](https://community.crewai.com/t/handling-llm-errors-in-hierarchical-crewai-process-with-callbacks/4694)
**Author:** [@dhanushkv67](https://community.crewai.com/u/dhanushkv67)\ **Replies:** 8\ **Last updated:** [April 10, 2025, 3:12pm UTC](https://community.crewai.com/t/handling-llm-errors-in-hierarchical-crewai-process-with-callbacks/4694 "2025-04-10T15:12:48Z")
I am running a CrewAI process in a hierarchical setup, but sometimes I encounter LLM errors during execution. I would like to know: Is there a way to add a callback functionality that triggers automatically whenever a… --- ## [LLM Response Error: ValueError: Invalid response from LLM call - None or empty](https://community.crewai.com/t/llm-response-error-valueerror-invalid-response-from-llm-call-none-or-empty/4425)
**Author:** [@sranjan710](https://community.crewai.com/u/sranjan710)\ **Replies:** 4\ **Last updated:** [March 13, 2025, 5:22am UTC](https://community.crewai.com/t/llm-response-error-valueerror-invalid-response-from-llm-call-none-or-empty/4425 "2025-03-13T05:22:05Z")
Hey CrewAi team, Can someone please prioritize this error? I’m noticing that multiple users of crewai are also running into the same issue when using open-source LLMs. Here’s the problem: :white\_check\_mark: Everything… --- ## [How to use the qwen2.5-vl-3b-instruct model with the CrewAi?](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456)
**Author:** [@Dhruvin\_5179](https://community.crewai.com/u/Dhruvin_5179)\ **Replies:** 2\ **Last updated:** [March 11, 2025, 9:41am UTC](https://community.crewai.com/t/how-to-use-the-qwen2-5-vl-3b-instruct-model-with-the-crewai/4456 "2025-03-11T09:41:39Z")
"I am trying to use the Qwen series model with the Hugging Face provider locally, but it does not support CrewAI. I am using vLLM to host the model on an API endpoint, but when I call the model through the LLM class, I e… --- ## [Problem with using locally deployed custom llm](https://community.crewai.com/t/problem-with-using-locally-deployed-custom-llm/4056)
**Author:** [@Mateusz\_Wosinski](https://community.crewai.com/u/Mateusz_Wosinski)\ **Replies:** 1\ **Last updated:** [February 21, 2025, 8:30pm UTC](https://community.crewai.com/t/problem-with-using-locally-deployed-custom-llm/4056 "2025-02-21T20:30:42Z")
Hi there! I have Llama-3.2-3B-Instruct model deployed on internal infrastracture. I want to use it as the llm for one of my agents. This what I prepared: llm = LLM( model="alpindale/Llama-3.2-3B-Instruct", … --- ## [Error: llm.py-llm:426 - ERROR: Failed to get supported params: argument of type 'NoneType' is not iterable](https://community.crewai.com/t/error-llm-py-llm-426-error-failed-to-get-supported-params-argument-of-type-nonetype-is-not-iterable/3915)
**Author:** [@Raj3](https://community.crewai.com/u/Raj3)\ **Replies:** 1\ **Last updated:** [February 18, 2025, 7:26pm UTC](https://community.crewai.com/t/error-llm-py-llm-426-error-failed-to-get-supported-params-argument-of-type-nonetype-is-not-iterable/3915 "2025-02-18T19:26:55Z")
Getting the above error when trying to use integrate Groq\_llm into agents from crewai import Agent, Task, Crew, Process, LLM from crewai\_tools import SerperDevTool groq\_llm=LLM(model="llama3-8b-8192", tempe… --- ## [Nineteenai doesnt work with the usual llm setup code](https://community.crewai.com/t/nineteenai-doesnt-work-with-the-usual-llm-setup-code/3892)
**Author:** [@Ren\_Kaizel](https://community.crewai.com/u/Ren_Kaizel)\ **Replies:** 0\ **Last updated:** [February 16, 2025, 6:20pm UTC](https://community.crewai.com/t/nineteenai-doesnt-work-with-the-usual-llm-setup-code/3892 "2025-02-16T18:20:23Z")
NineteenAI gives free access to Lllama 70B which is what im trying to use. The api and stuff can be found here: NineteenAI | Decentralised Inference. (I have given the specific error below, but its basically that i can… --- ## [Using Custom LLM](https://community.crewai.com/t/using-custom-llm/3210)
**Author:** [@gaurav](https://community.crewai.com/u/gaurav)\ **Replies:** 0\ **Last updated:** [January 25, 2025, 2:58pm UTC](https://community.crewai.com/t/using-custom-llm/3210 "2025-01-25T14:58:57Z")
I was working on a data analyst agent, and I want to use a custom LLM. I have a model hosted on a server with a URL, and I need to use the LLM via that URL. I tried creating a custom class, but it did not work. --- ## [Using ollama(llama3.2-vision) to extract text from image](https://community.crewai.com/t/using-ollama-llama3-2-vision-to-extract-text-from-image/2724)
**Author:** [@yaki6](https://community.crewai.com/u/yaki6)\ **Replies:** 1\ **Last updated:** [January 17, 2025, 6:04am UTC](https://community.crewai.com/t/using-ollama-llama3-2-vision-to-extract-text-from-image/2724 "2025-01-17T06:04:54Z")
Hi team, I am trying to use llama3.2-vision on ollama to spin up a crew to extract text. I use the following to do the job: import ollama response = ollama.chat( model='llama3.2-vision', messages=\[{ 'r… --- ## [When native integrations with other LLMs?](https://community.crewai.com/t/when-native-integrations-with-other-llms/1944)
**Author:** [@achris7](https://community.crewai.com/u/achris7)\ **Replies:** 1\ **Last updated:** [December 11, 2024, 8:50am UTC](https://community.crewai.com/t/when-native-integrations-with-other-llms/1944 "2024-12-11T08:50:30Z")
Having quite a difficult time for ~2 weeks using for example Gemini via Litellm…prob need to check out another option like ag2 if this continues. welcome thoughts or plans --- ## [Tried to both perform Action and give a Final Answer at the same time, I must do one or the other](https://community.crewai.com/t/tried-to-both-perform-action-and-give-a-final-answer-at-the-same-time-i-must-do-one-or-the-other/936)
**Author:** [@Yash](https://community.crewai.com/u/Yash)\ **Replies:** 7\ **Last updated:** [November 18, 2024, 4:09pm UTC](https://community.crewai.com/t/tried-to-both-perform-action-and-give-a-final-answer-at-the-same-time-i-must-do-one-or-the-other/936 "2024-11-18T16:09:35Z")
Error parsing LLM output, agent will retry: I did it wrong. Tried to both perform Action and give a Final Answer at the same time, I must do one or the other getting these in my logs , if any one can help --- ## [BadRequestError: litellm.BadRequestError: LLM Provider NOT provided](https://community.crewai.com/t/badrequesterror-litellm-badrequesterror-llm-provider-not-provided/918)
**Author:** [@Yash](https://community.crewai.com/u/Yash)\ **Replies:** 4\ **Last updated:** [October 21, 2024, 12:37pm UTC](https://community.crewai.com/t/badrequesterror-litellm-badrequesterror-llm-provider-not-provided/918 "2024-10-21T12:37:33Z")
I am getting the below error when trying to use litellm code snippet : from crewai import LLM llm = LLM( model="my-model, openai\_api\_base=“http://localhost:4000”, api\_key=“api-key” ) get\_research = CustomResearch… --- ## [ImportError: cannot import name 'LLM' from 'crewai'](https://community.crewai.com/t/importerror-cannot-import-name-llm-from-crewai/916)
**Author:** [@Yash](https://community.crewai.com/u/Yash)\ **Replies:** 6\ **Last updated:** [October 21, 2024, 11:07am UTC](https://community.crewai.com/t/importerror-cannot-import-name-llm-from-crewai/916 "2024-10-21T11:07:20Z")
I am not able to import LLM from crewai, gives me the import error, i want to use the litellm as my LLM provider.