Local hosted searxng search

CrewAI Community Support
1

Hello.
Trying to use searxng tool from http://localhost:8080 . Tried many approaches but getting error. Any idea as to how to create tool for searxng based search.

Langchain’s searxng tool also did not work.

2

What error do you get?

3

Error for different methods I tried:

Method 1:

from langchain_community.utilities import SearxSearchWrapper
from langchain.agents import Tool

search = SearxSearchWrapper(searx_host=“http://localhost:8080”)

search_tool = Tool(
name=“search ng”,
description=“Search the internet(search_query: ‘string’)”,
func=search.run,
)

print(search_tool)
Tool(name=‘search ng’, description=“Search the internet(search_query: ‘string’)”, func=<bound method SearxSearchWrapper.run of SearxSearchWrapper(searx_host=‘http#####8080’, unsecure=True, params={‘language’: ‘en’, ‘format’: ‘json’}, headers=None, engines=, categories=, query_suffix=‘’, k=10, aiosession=None)>)

Error when adding this tool to agent:

ValidationError: 1 validation error for Agent
tools.0
Input should be a valid dictionary or instance of BaseTool [type=model_type, input_value=Tool(name=‘search ng’, de…k=10, aiosession=None)>), input_type=Tool]

######################

Method 2:

wrapping search function above in tool

from langchain.tools import tool

@tool(“Search Tool”)
def search_tool(query: str) → list:
“”“Search Internet for relevant information based on a query.”“”
results = search.results(keywords=query, num_results = 10, engines=[“google”])
return results

Gives same error.

###############

Method 3:

from langchain.agents import load_tools
search_tool = load_tools([“searx-search”],
searx_host=“http#####8080”,
engines=[“google”])
print(search_tool)
[SearxSearchRun(wrapper=SearxSearchWrapper(searx_host=‘http#####8080’, unsecure=True, params={‘language’: ‘en’, ‘format’: ‘json’, ‘engines’: ‘google’}, headers=None, engines=[‘google’], categories=, query_suffix=‘’, k=10, aiosession=None), kwargs={})]

Again same error.

#################

Mtehod 4:

from crewai.tools import BaseTool, tool
from typing import List, Union
class MySearchTool(BaseTool):
name: str = “Search Internet”
description: str = “A tool that can be used to search the internet with a search_query. (search_query: ‘string’)”

def _run(self, search_query: str) -> Union[List [str], str]:
    # Implementation goes here
    return search.results(search_query, num_results = 10, engines=["google"])

search_tool = MySearchTool()

This works untill kick_off. Then throws this error message for the agent.

Agent: Venue Coordinator

Task: Find a venue in San Francisco that meets criteria for Tech Innovation Conference.

I encountered an error while trying to use the tool. This was the error: name ‘search’ is not defined.
Tool Search Internet accepts these inputs: Tool Name: Search Internet
Tool Arguments: {‘search_query’: {‘description’: None, ‘type’: ‘str’}}
Tool Description: A tool that can be used to search the internet with a search_query. (search_query: ‘string’)

#######

I also tried repurposing serper_tool.py from crewai_tool repo. and edit to make it for seaxing.

Again same error.

#######

It will be great to have an example case on the repo with ollama and searxng.

4

Ok. Worked . Got help from your chatgpt assistant.

It suggested

To use SearxNG as a tool in CrewAI:

  1. Create a Custom Tool: Define a custom tool using CrewAI’s framework to wrap the SearxNG API.

python

Copy code

from crewai_tools import tool
import requests

@tool
def searxng_search(query: str) -> str:
    """
    Search the web using a locally hosted SearxNG instance.
    Args:
        query (str): Search term
    Returns:
        str: Top results
    """
    SEARXNG_URL = "http://localhost:8080/search"
    params = {'q': query, 'format': 'json'}
    response = requests.get(SEARXNG_URL, params=params)

    if response.status_code == 200:
        results = response.json()
        return "\n".join(
            f"{res['title']} - {res['url']}" for res in results['results']
        )
    else:
        return f"Error: {response.status_code}"
  1. Add the Tool to an Agent: Include the searxng_search tool in an agent for tasks that require internet search capabilities.

And it works. Please add it in documentation somewhere or may be just add a Searxng_search_tool in your tool repo. Thanks again.

searxng_crewai
searxng_crewai675×820 57.7 KB

1 Like
5

Thank you! A small improvement on you code for my specific use-case:

from crewai.tools import tool
import requests
from datetime import datetime

@tool
def searxng_search(query: str, tema: str = "geral", data_inicio: str = "", limite: int = 5) -> str:
    """
Executa uma busca na web utilizando o mecanismo SearxNG, 
adaptada ao contexto temático e temporal da consulta.
Parâmetros esperados:
    - query (str): É a pergunta ou termo principal que será buscado. Exemplo: "impacto da IA na educação".
    - tema (str): Define o tipo de fontes que o agente deseja consultar. As opções são:
        - "geral": busca em mecanismos amplos como Google e Bing.
        - "noticias": foca em portais de notícias como Google News, BBC e Bing News.
        - "cientifico": retorna resultados de artigos e papers em bases como arXiv e Semantic Scholar.
        - "tecnico": busca conteúdos técnicos em fontes como Stack Overflow, GitHub e sites de documentação.
    - data_inicio (str): Um filtro opcional para limitar os resultados a publicações feitas **a partir dessa data**. Deve ser informada no formato "YYYY-MM-DD".
        Exemplo: "2023-01-01" retorna apenas conteúdos publicados após 1º de janeiro de 2023.
    - limite (int): Define quantos resultados devem ser retornados. Por padrão, traz os 5 principais resultados ordenados por relevância.
Retorno:
Uma string contendo uma lista formatada com os principais resultados, incluindo:
    - Título
    - Resumo (quando disponível)
    - URL
    - Fonte original (ex: Google, arXiv, GitHub)
O objetivo é permitir que o agente LLM tenha acesso rápido e filtrado a informações da web, 
com controle sobre o tema e período de interesse.
    """
    SEARXNG_URL = "http://localhost:8887/search"

    # Filtros por tema
    tema = tema.lower()
    categorias = {
        "geral": {"engines": "google,bing", "categoria": "general"},
        "noticias": {"engines": "google news,bbc,bing news", "categoria": "news"},
        "cientifico": {"engines": "arxiv,semantic scholar", "categoria": "science"},
        "tecnico": {"engines": "stack overflow,github,docs", "categoria": "it"},
    }

    config = categorias.get(tema, categorias["geral"])

    # Aplica filtro de data se informado
    if data_inicio:
        try:
            datetime.strptime(data_inicio, "%Y-%m-%d")
            query += f" after:{data_inicio}"
        except ValueError:
            return "[Erro] Formato de data inválido. Use AAAA-MM-DD."

    params = {
        'q': query,
        'format': 'json',
        'language': 'pt',
        'safesearch': 1,
        'categories': config["categoria"],
        'engines': config["engines"]
    }

    try:
        response = requests.get(SEARXNG_URL, params=params, timeout=10)
        response.raise_for_status()
        results = response.json().get('results', [])

        if not results:
            return "Nenhum resultado encontrado."

        return "\n\n".join(
            f"{res['title']}\n{res.get('content', '').strip()}\n{res['url']} (via {res['engine']})"
            for res in results[:limite]
        )

    except requests.RequestException as e:
        return f"[Erro de conexão] {str(e)}"
    except Exception as e:
        return f"[Erro inesperado] {str(e)}"