# A question regarding using open source LLMs through Ollama with CrewAI

**URL:** <https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397>\
**Category:** CrewAI Community Support\
**Created:** [September 15, 2024, 10:22am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397 "2024-09-15T10:22:41Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![aslansd](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/aslansd/32/301_2.png) [@aslansd](https://community.crewai.com/u/aslansd)\
**Post date:** [September 15, 2024, 10:22am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/1 "2024-09-15T10:22:41Z")

</div>

Hi all,

I was wondering if any one has been used CrewAI with one of the open source LLMs of Ollama. Over the weekend I worked on a small project using CrewAI with Ollama. When I run it, it does not give any error, at the same time, it does not produce any results. I can share the code if any wants. Thanks in advance for any hint!

Best regards,

Aslan

---

<div class="post-metadata">

**Author:** ![Dabnis](https://avatars.discourse-cdn.com/v4/letter/d/858c86/32.png) [@Dabnis](https://community.crewai.com/u/Dabnis)\
**Post date:** [September 15, 2024, 10:41am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/2 "2024-09-15T10:41:48Z")

</div>

Hi @aslansd,  
First question: What operting system re you running: Linux, Win, Mac?

I tend to be online most days, please feel free to PM me.

I am relatively new to CrewAI, but may be able to help you with this one.

---

<div class="post-metadata">

**Author:** ![aslansd](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/aslansd/32/301_2.png) [@aslansd](https://community.crewai.com/u/aslansd)\
**Post date:** [September 15, 2024, 11:35am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/3 "2024-09-15T11:35:05Z")

</div>

Hi @Dabnis, thanks for your response. I am using Mac!

---

<div class="post-metadata">

**Author:** ![alexcovo](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@alexcovo](https://community.crewai.com/u/alexcovo)\
**Post date:** [September 15, 2024, 3:33pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/4 "2024-09-15T15:33:03Z")

</div>

Hi @aslansd,

I use local models often through ollama. Here’s a simple crewai repo that uses several local LLMs. Feel free to change to any models you’ve downloaded with Ollama.  
Also, you can comment out gemini-1.5 as the LLM Manager and change the process back from hierarchical to sequential.

The crew will ask you for a topic when you start-up. I find it more flexible.  
Check out the code if it helps.

> **[GitHub - dax8it/Local-CrewAI: CrewAI Local Agents and LLM Manager](https://github.com/dax8it/Local-CrewAI)**
>
> CrewAI Local Agents and LLM Manager

Cheers,  
Alex

---

<div class="post-metadata">

**Author:** ![jklre](https://avatars.discourse-cdn.com/v4/letter/j/b4bc9f/32.png) [@jklre](https://community.crewai.com/u/jklre)\
**Post date:** [September 15, 2024, 5:26pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/5 "2024-09-15T17:26:47Z")

</div>

I primarily use open source LLM’s with both ollama and LMstudio.

You basically install ollama. Do ollama pull (model name)

```auto
from langchain_community.chat_models import ChatOllama
from langchain_community.llms import Ollama

llm_ollama = ChatOllama(model="llama3-gradient:latest")
llm_ollama2 = ChatOllama(model="mistral:latest")
llm_ollama3 = ChatOllama(model="phi3:latest")
llm_ollama4 = ChatOllama(model="dolphin-llama3:8b-256k")
llm_ollama5 = ChatOllama(model="gemma2:latest")

```

etc. Then set your LLM in your crew. llm=llm\_olamma, etc You may need to run the ollama serve command first but I dont need to.

```auto
    @agent
    def reporting_analyst(self) -> Agent:
        return Agent(
            config=self.agents_config['reporting_analyst'],
            verbose=True,
            allow_delegation=True,
            llm=llm_ollama,
            max_iter=800,
            memory=True,
            max_rpm=15
        )

```

---

<div class="post-metadata">

**Author:** ![aslansd](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/aslansd/32/301_2.png) [@aslansd](https://community.crewai.com/u/aslansd)\
**Post date:** [September 15, 2024, 6:21pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/6 "2024-09-15T18:21:55Z")

</div>

Thanks for your comments @alexcovo @jklre!

---

<div class="post-metadata">

**Author:** ![alexcovo](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@alexcovo](https://community.crewai.com/u/alexcovo)\
**Post date:** [September 16, 2024, 11:37pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/7 "2024-09-16T23:37:03Z")

</div>

You’re welcome. Just FYI I have an issue with langchain-core 0.3.0 now after I updated crewai to latest version today. I have to fix. Just a heads up!

Works with crewai 0.51.1

---

<div class="post-metadata">

**Author:** ![aslansd](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/aslansd/32/301_2.png) [@aslansd](https://community.crewai.com/u/aslansd)\
**Post date:** [September 17, 2024, 5:43am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/8 "2024-09-17T05:43:33Z")

</div>

Thanks for info @alexcovo!

---

<div class="post-metadata">

**Author:** ![alexcovo](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@alexcovo](https://community.crewai.com/u/alexcovo)\
**Post date:** [September 17, 2024, 3:29pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/9 "2024-09-17T15:29:27Z")

</div>

if you’re going to use the repo use with crewai 0.51.1.

Having issues with update 0.6 and langchain-core and dependencies locally.

---

<div class="post-metadata">

**Author:** ![matt](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/matt/32/3_2.png) [@matt](https://community.crewai.com/u/matt)\
**Post date:** [September 17, 2024, 4:29pm UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/10 "2024-09-17T16:29:44Z")

</div>

Langchain is no longer a dependency for crewAI,

What issue are you having?

---

<div class="post-metadata">

**Author:** ![alexcovo](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@alexcovo](https://community.crewai.com/u/alexcovo)\
**Post date:** [September 18, 2024, 4:44am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/12 "2024-09-18T04:44:49Z")

</div>

@aslansd Here is the updated code for 0.60.00. Unfortunately what I posted previously doesn’t work anymore. I’ve updated the githhub repository as well.

Here’s the updated crew.py code:

import os  
from crewai import Agent, Crew, Process, Task  
from crewai.project import CrewBase, agent, crew, task  
from langchain\_community.llms import OpenAI, Ollama  
from dotenv import load\_dotenv  
import litellm # Added for using google API  
from crewai\_tools import SerperDevTool

search\_tool = SerperDevTool()

load\_dotenv()

# Set the Google API key for LiteLLM to use Gemini LLM Models

# litellm.api\_key = os.getenv(‘GOOGLE\_API\_KEY’)

# Uncomment the following line to use an example of a custom tool

# from conundrum\_crew.tools.custom\_tool import MyCustomTool

@CrewBase  
class ConundrumCrew():  
“”“Conundrum crew”“”  
agents\_config = ‘config/agents.yaml’  
tasks\_config = ‘config/tasks.yaml’

```
@agent
def researcher(self) -> Agent:
	return Agent(
		config=self.agents_config['researcher'],
		# tools=[MyCustomTool()], # Example of custom tool, loaded on the beginning of file
		verbose=True,
		llm='ollama/hermes3:latest',
    	max_iterations=5,
    	max_time=120,
		tools=[search_tool]
	)

@agent
def reporting_analyst(self) -> Agent:
	return Agent(
		config=self.agents_config['reporting_analyst'],
		verbose=True,
		llm='ollama/gemma2:27b',
    	max_iterations=10,
    	max_time=120,	
	)

@task
def research_task(self) -> Task:
	return Task(
		config=self.tasks_config['research_task'],
		output_file='research.md'
	)

@task
def reporting_task(self) -> Task:
	return Task(
		config=self.tasks_config['reporting_task'],
		output_file='report.md'
	)

@crew
def crew(self) -> Crew:
	"""Creates the Conundrum crew"""
	return Crew(
		agents=self.agents, # Automatically created by the @agent decorator
		tasks=self.tasks, # Automatically created by the @task decorator
		manager_llm='ollama/gemma2:27b', #you can also use Gemini, for example: 'gemini/gemini-1.5-flash-exp-0827',
		process=Process.hierarchical, # In case you wanna use that instead https://docs.crewai.com/how-to/Hierarchical/
		# process=Process.sequential,
		verbose=True,
	)

```

---

<div class="post-metadata">

**Author:** ![alexcovo](https://avatars.discourse-cdn.com/v4/letter/a/c37758/32.png) [@alexcovo](https://community.crewai.com/u/alexcovo)\
**Post date:** [September 18, 2024, 4:49am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/13 "2024-09-18T04:49:05Z")

</div>

I had issues with my conda environment but sorted it out. I still wasn’t sure how to implement Gemini but currently solved with following. If there’s an easier way let me know!

> [@alexcovo](#):
>
> import litellm # Added for using google API
> 
> litellm.api\_key = os.getenv(‘GOOGLE\_API\_KEY’)

---

<div class="post-metadata">

**Author:** ![aslansd](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/aslansd/32/301_2.png) [@aslansd](https://community.crewai.com/u/aslansd)\
**Post date:** [September 18, 2024, 5:21am UTC](https://community.crewai.com/t/a-question-regarding-using-open-source-llms-through-ollama-with-crewai/397/14 "2024-09-18T05:21:31Z")

</div>

Thanks @alexcovo for your comments!