How to use a local model for custom embedding

General
1

Hey guys, hope everyone is going well…

Im trying use the tool “JSONSearchTool”, but I want use a local model for llm and embedder, its possible ?
If yes, how can I do this ?

I try something like that

json_tool = JSONSearchTool(
config={
“llm”: {
“provider”: “openai”,
“config”: {
“model”: “gpt-4”,
“base_url”: “https://server_name/v1/”,
“api_key”: “dummy”
},
},
“embedder”: {
“provider”: “huggingface”,
“config”: {
“model”: “nomic-ai/nomic-embed-text-v1.5”,
“base_url”: “https://server_name/v1/embeddings”,
“api_key”: “dummy”
},
},
},
file_path=temp_file.name
)

trying configurate the embedder just like the model, but he got an unexpected keyword argument ‘base_url’

If someone know some tutorial, documentation, or some way to do this, just let me know plsss, I will be grateful

2

I managed to use custom embedding this way, I hope it helps someone

json_tool = JSONSearchTool(
config={
“llm”: {
“provider”: “openai”, # Other options include google, openai, anthropic, llama2, etc.
“config”: {
“model”: “gpt-4”,
“base_url”: “https://api_url/v1//”,
“api_key”: “dummy” # Valor dummy para satisfazer o parâmetro api_key
# Additional optional configurations can be specified here.
# temperature=0.5,
# top_p=1,
# stream=true,
},
},
“embedder”: {
“provider”: “openai”, # or openai, ollama, …
“config”: {
“model”: “text-embedding-nomic-embed-text-v1.5”,
“api_base”: “https://api_url/v1/”,
“api_key”: “dummy”

            # Further customization options can be added here.
        },
    },
},
json_path=temp_file.name

)

Edit: Just for clarify what was wrong in my first implementation, the json_path was being used with a wrong name, and was having difficulty to pass the embedder config, after some search I achieved the correct way

1 Like