How to embed knowledge source with ollama2

CrewAI Community Support
1

Hi,

I am trying to use a local knowledge source like the TextFileKnowledgeSource together with an Agent using the llama3.2 model.

llm = LLM(model="ollama/llama3.2", base_url="http://localhost:11434")
ks = TextFileKnowledgeSource(
            file_path="user_preference.txt", metadata={"foo": "bar"}
       )
return Agent(
            config=self.agents_config["dummy_agent"],
            verbose=True,
            knowledge_sources=[ks],
            llm=llm,
            embedder={"provider": "ollama", "config": {"model": "llama2"}},
        )

I already got this running with the default openAI model. However it looks like that the embedding does not work, since the model can not access the information located in the textfile.

I See others are using a custom embedder configuration when they use a different model. I can’t find any information on how to embed the knowledge source together with ollama.

Can anyone help?

2

I think I found a solution. I need to do two things:

  1. Configure the LLM with the pretty low temperature. Not sure why this is the reason, but I guess with higher temperature the model tends to be more creative on its own model.
  2. Actually tell the agent in his configuration (goal) to make use of his local knowledge sources. :thinking:

When setting the temperature to <=0.3 I get the expected response pretty stable.

Is this behaviour expected?

Further I would like to learn more about custom embedder. I found out that a did not need the embedder at all in my configuration. So when do I need a custom embedder and where do I find infos on how to configure it. I found the config I used my accident here in the form.

3

Hey Torsten! As you’ve experienced, lower temperature should be better on embedding/RAG related projects where we need the information correctly. Here’s the explanation of temperature from OpenAI’s documents:

-I’m also trying to better understand the embedding stuff so i won’t be able to say anything there, but good luck! :slight_smile:

1 Like
4

Hey Ali, Thank you for your answer.

Since crewai uses chroma as application db to make knowledge available to the LLM I found Chroma Docs useful for a start. At least I was able to find the configuration there.

1 Like
5

Hi,
Do you have a sample code on how you got the embedding working using ollama model?
I have a custom knowledge source that I have created in my application. On creating the crew, like this
crew = Crew(
agents=[json_analyst, summarizer_agent],
tasks=[ analysis_task, summary_task],
verbose=True,
knowledge_sources=[knowledge_source],
embedder={
“provider”: “ollama”,
“config”: {
“model”: “nomic-embed-text”
}
},
process=Process.sequential
)
the script fails while creating the embedding with the following error -
Failed to upsert documents: Expected Embedings to be non-empty list or numpy array, got in upsert.

Any pointers around this?