Unable to use MDXSearchTool with local Ollama LLM — OpenAI API key still required

General
1

Hi CrewAI community,

I’m trying to use MDXSearchTool with a fully local setup using Ollama (llama3.1:8b) as the LLM and embeddinggemma:latest as the embedder. I want to avoid using OpenAI entirely.

Here’s what I tried:

from crewai_tools import MDXSearchTool

tool = MDXSearchTool(
    mdx="docs/example.mdx",
    config=dict(
        llm=dict(
            provider="ollama",
            config=dict(
                model="llama3.1:8b",
                base_url="http://localhost:11434",
                temperature=0.3,
            ),
        ),
        embedder=dict(
            provider="ollama",
            config=dict(
                model="embeddinggemma:latest",
                base_url="http://localhost:11434",
                task_type="retrieval_document",
            ),
        ),
    ),
)

However, I still get the following error:

ValidationError: Value error, Please provide an OpenAI API key.

It seems like the tool is still validating OpenAI by default, even though I’ve provided both llm and embedder in the config.

My questions:

  1. Is it possible to fully bypass OpenAI and use local models with MDXSearchTool?

  2. If yes, what is the correct config schema to make it work?

  3. Are there any known workarounds (patching, subclassing, or using a local provider flag) to avoid the OpenAI key validation?

Thanks in advance for your guidance!

2

How are you declaring your real LLM, the one your Agents use? If not, then LiteLLM/CrewAI will default to OpenAI

3

Yep, I’m declaring my own LLM so it doesn’t fall back to OpenAI. Since I’m running locally with Ollama, I just spin up the LLM instance and then pass it directly into the agent:

from crewai import LLM, Agent

model = "ollama/llama3.1:8b"
llm = LLM(
    model=model,
    base_url="http://localhost:11434"
)

researcher = Agent(
    role="Tech Job Researcher",
    goal="Make sure to do amazing analysis on job posting to help job applicants",
    tools=[scrape_tool, search_tool],
    verbose=True,
    backstory=(
        "As a Job Researcher, your prowess in navigating and extracting "
        "critical information from job postings is unmatched. Your skills "
        "help pinpoint the necessary qualifications and skills sought by "
        "employers, forming the foundation for effective application tailoring."
    ),
    llm=llm  # this makes sure it uses my local model
)

That llm=llm line is what keeps it from defaulting to OpenAI. Since I’ve set the base_url to my local Ollama server, the agent always runs through that instead.

—
The issue is only for the MDXSearchTool which does not allow me to override to use my local llm + embedder.

4

You’re right. I was able to replicate your error.

You know, until recently, CrewAI’s RAGTools (MDXSearchTool, PDFSearchTool, etc.) were outsourced to the Embedchain library.

Then lately, from what I’ve seen in the commits, the CrewAI team decided to implement their own machinery to handle embeddings and vector databases, so the RAGTools are now using this new in-house version.

Therefore, the error you’re encountering is related to a bug in this new implementation. I’d advise you to open a GitHub issue to report it.

1 Like
5

Thanks . Created : [BUG]Unable to use MDXSearchTool with local Ollama LLM — OpenAI API key still required · Issue #3622 · crewAIInc/crewAI · GitHub

1 Like