CSVSearchTool always returns only 3 results - How to increase the limit?

General
2

Hey Valclemir,

Try using a custom Adapter like the one I’ve laid out below. I took the chance to swap out the search functionality between Adapters: the original one relies on Embedchain’s .query() method, but the new one uses the .search() method instead, which provides the flexibility to adjust the number of results you get back.

I’ve also beefed up the text that’s returned to the LLM. You really want to think about the tool output as something that enriches the LLM’s understanding. Whenever you’re building custom tools, consider this output as part of the LLM’s actual prompt.

Finally, you’ll notice I got rid of the llm attribute you were passing in your configuration and just stuck with the embedder. Even the original Adapter doesn’t require the llm attribute. It’s only used (in the default Adapter) if you set summarize=True. In that case, Embedchain itself hands your Agent a result that’s been summarized by the LLM specified in that parameter. Since you’re using the raw data from your search, that llm parameter is pretty much never necessary.

from typing import Any
from crewai_tools.tools.rag.rag_tool import Adapter
from embedchain import App

class CustomEmbedchainAdapter(Adapter):
    embedchain_app: App

    def query(self, question: str) -> str:
        response = "---\n"
        response += f"**Additional Context for Query '{question}':**\n"
    
        search_results = self.embedchain_app.search(
            query=question,
            num_documents=5, # Up to 5 relevant chunks
        )
        
        if search_results:
            for context, metadata in (item.values() for item in search_results):
                response += f"**Context:** '{context}' "
                response += f"(**Metadata:** '{metadata}')\n"
        else:
            response += "No relevant context found for this query.\n"
        
        response += "---\n"
        return response.strip()

    def add(self, *args: Any, **kwargs: Any) -> None:
        self.embedchain_app.add(*args, **kwargs)

#
# Test it out
#

from crewai_tools import CSVSearchTool
import os

os.environ["OPENAI_API_KEY"] = "<YOUR_OPENAI_API_KEY>"

embedchain_config = {
    "embedder": {
        "provider": "openai",
        "config": {
            "model": "text-embedding-3-small"
        }
    }
}

csv_tool = CSVSearchTool(
    csv="/path/to/your/file.csv",
    config=embedchain_config,
    adapter=CustomEmbedchainAdapter(
        embedchain_app=App.from_config(config=embedchain_config)
    )
)

print(
    csv_tool.run("<YOUR_QUERY>")
)