How to embed knowledge source with ollama2

[toirl]

Hi,

I am trying to use a local knowledge source like the TextFileKnowledgeSource together with an Agent using the llama3.2 model.

llm = LLM(model="ollama/llama3.2", base_url="http://localhost:11434")
ks = TextFileKnowledgeSource(
            file_path="user_preference.txt", metadata={"foo": "bar"}
       )
return Agent(
            config=self.agents_config["dummy_agent"],
            verbose=True,
            knowledge_sources=[ks],
            llm=llm,
            embedder={"provider": "ollama", "config": {"model": "llama2"}},
        )

I already got this running with the default openAI model. However it looks like that the embedding does not work, since the model can not access the information located in the textfile.

I See others are using a custom embedder configuration when they use a different model. I can’t find any information on how to embed the knowledge source together with ollama.

Can anyone help?

[toirl]

I think I found a solution. I need to do two things:

1. Configure the LLM with the pretty low temperature. Not sure why this is the reason, but I guess with higher temperature the model tends to be more creative on its own model.
2. Actually tell the agent in his configuration (goal) to make use of his local knowledge sources.

When setting the temperature to <=0.3 I get the expected response pretty stable.

Is this behaviour expected?

Further I would like to learn more about custom embedder. I found out that a did not need the embedder at all in my configuration. So when do I need a custom embedder and where do I find infos on how to configure it. I found the config I used my accident here in the form.

[progressiveOverload]

Hey Torsten! As you’ve experienced, lower temperature should be better on embedding/RAG related projects where we need the information correctly. Here’s the explanation of temperature from OpenAI’s documents:

• “What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.”
  Also here’s the link for it: https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature (Just scroll a bit to temperature if link doesn’t land you strictly on that)

-I’m also trying to better understand the embedding stuff so i won’t be able to say anything there, but good luck!