# Issue with Litellm Perplexity API

**URL:** <https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551>\
**Category:** CrewAI Community Support\
**Created:** [March 10, 2025, 10:07am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551 "2025-03-10T10:07:02Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 10, 2025, 10:07am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/1 "2025-03-10T10:07:02Z")

</div>

Hi,

I am trying to use Perplexity as a search agent. It used to work before with the old Llama models but not anymore with the latest Sonar models. I have the following code:

```auto
from openai import OpenAI
import os
from dotenv import load_dotenv
from crewai import Agent, Task, Crew, Process, LLM
from langchain_openai import ChatOpenAI

load_dotenv(".env", override=True) 
perplexity_llm = LLM(
    api_key=os.getenv("PERPLEXITYAI_API_KEY"),
    model="perplexity/sonar",
    base_url="https://api.perplexity.ai/",
)

```

Is there a new setup for Perplexity to get it to work in CrewAI? Btw, it works fine in a non CrewAI context. With the above set up it now returns ErrorCode 400.

Thx

---

<div class="post-metadata">

**Author:** ![zinyando](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/zinyando/32/4435_2.png) [@zinyando](https://community.crewai.com/u/zinyando)\
**Post date:** [March 10, 2025, 11:38am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/2 "2025-03-10T11:38:53Z")

</div>

Seems the sonar models are not supported by LiteLLM yet [Perplexity AI (pplx-api) | liteLLM](https://docs.litellm.ai/docs/providers/perplexity#supported-models)

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 10, 2025, 1:29pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/3 "2025-03-10T13:29:01Z")

</div>

yeah, but from that site I also tried:

perplexity/pplx-70b-online

and this also does not work. Any idea which one would work?

Thx

---

<div class="post-metadata">

**Author:** ![zinyando](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/zinyando/32/4435_2.png) [@zinyando](https://community.crewai.com/u/zinyando)\
**Post date:** [March 10, 2025, 2:26pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/4 "2025-03-10T14:26:06Z")

</div>

After digging around a bit sonar models should work, can you try this one more time:

```auto
perplexity_llm = LLM(
    model="sonar",
    base_url="https://api.perplexity.ai/"
)

```

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 10, 2025, 6:52pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/5 "2025-03-10T18:52:26Z")

</div>

Thx, I tried but it still does not work. What version of crewai and litellm are you using?

---

<div class="post-metadata">

**Author:** ![zinyando](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/zinyando/32/4435_2.png) [@zinyando](https://community.crewai.com/u/zinyando)\
**Post date:** [March 10, 2025, 6:55pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/6 "2025-03-10T18:55:28Z")

</div>

CrewAI version 0.102.0, which version are you on?

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 10, 2025, 7:05pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/7 "2025-03-10T19:05:15Z")

</div>

same version, but for some reason does not work for me. this is most exasperating. I really think that CrewAI should have someone monitoring the community support to help out on queries like these. I wonder if the other multi-agent platforms are as problematic as this one.

---

<div class="post-metadata">

**Author:** ![maxmoura](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/maxmoura/32/4206_2.png) [@maxmoura](https://community.crewai.com/u/maxmoura)\
**Post date:** [March 11, 2025, 2:28am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/8 "2025-03-11T02:28:18Z")

</div>

Well, I don’t think you need to set `base_url`. Have you tried just this:

```python
perplexity_llm = LLM(
    model="perplexity/sonar",
)

```

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 11, 2025, 8:19am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/9 "2025-03-11T08:19:11Z")

</div>

Hi Max, yes I did try your suggestion. Here is the result that I got:

BadRequestError: litellm.BadRequestError: PerplexityException - Error code: 400 - {‘error’: {‘message’: ‘custom stop words are not implemented for completions.’, ‘type’: ‘unsupported\_parameter’, ‘code’: 400}}

It somehow implies that CrewAI is adding some kind of stop words to the perplexity prompt. This is baffling because my CrewAI setup used to work perfectly well in the past with the older Llama models in PerplexityAI.

---

<div class="post-metadata">

**Author:** ![maxmoura](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/maxmoura/32/4206_2.png) [@maxmoura](https://community.crewai.com/u/maxmoura)\
**Post date:** [March 11, 2025, 12:28pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/10 "2025-03-11T12:28:42Z")

</div>

Hey Moe, digging around a bit more, I saw that other people are encountering `ErrorCode 400` in the LiteLLM library when using models from the Sonar series, for example [see here](https://github.com/BerriAI/litellm/issues/8011). CrewAI relies on LiteLLM for its interfaces with LLMs. So, let’s confirm if the problem isn’t in LiteLLM itself, alright? I propose the following code:

```python
from litellm import completion
import os

os.environ['PERPLEXITYAI_API_KEY'] = 'YOUR_KEY_NOT_MINE'
MODEL = 'perplexity/sonar'

messages = [
    {'role': 'system', 'content': 'You are a helpful AI philosopher.'},
    {'role': 'user', 'content': 'What is the meaning of life in one paragraph?'},
]

litellm_response = completion(
    model=MODEL,
    messages=messages,
)

print(f'\nLiteLLM Response:\n\n{litellm_response}')

```

If the same problem shows up, then it’s something that needs to be fixed in the LiteLLM library, which CrewAI depends on.

I’m new to this area, still learning how to handle agent frameworks, but I suspect you could make the connection with LangChain, for example, using something like `langchain_community.chat_models.perplexity.ChatPerplexity`.

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 11, 2025, 8:47pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/11 "2025-03-11T20:47:21Z")

</div>

Hi Max, I tried your code above and it works perfectly fine for me. I do get a response from Perplexity. So where does that leave this issue at? Is it a problem with CrewAI that Perplexity cannot run inside CrewAI?

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 11, 2025, 9:11pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/12 "2025-03-11T21:11:07Z")

</div>

Max, does this make sense?:

> **[\[Bug\]: custom stop words are not implemented for completions ·...](https://github.com/danny-avila/LibreChat/discussions/1913)**
>
> What happened? I've tried using the openrouter.ai model "perplexity/sonar-medium-online," which is defined inside the librechat.yaml. However, when I attempt to use it now, I receive the following ...

---

<div class="post-metadata">

**Author:** ![maxmoura](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/maxmoura/32/4206_2.png) [@maxmoura](https://community.crewai.com/u/maxmoura)\
**Post date:** [March 12, 2025, 6:39pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/13 "2025-03-12T18:39:27Z")

</div>

Moe, if the code using the LiteLLM library worked, then I really can’t understand how CrewAI wouldn’t, since it uses the LiteLLM library under the hood.

For one last check, let’s take a look at a more complete code example, trying a direct connection and using a crew.

My knowledge is limited to this, so I hope it works out.

```python
from crewai import LLM, Agent, Crew, Task, Process
import os

os.environ['PERPLEXITYAI_API_KEY'] = 'YOUR_API_KEY_HERE'
MODEL = 'perplexity/sonar'

perplexity_llm = LLM(
    model=MODEL,
)

print('-------------------------')
print(' LLM.call()')
print('-------------------------')

call_output = perplexity_llm.call(
    "Tell me how many stars there are in the visible sky. "
    "Output a single paragraph."
)

print(f'LLM.call Output:\n\n{call_output}')

print('-------------------------')
print(' LLM.Crew.kickoff()')
print('-------------------------\n')

agent = Agent(
    role="Senior Astrophysicist",
    goal="Spread deep thoughts about the Universe",
    backstory="With over 10 years of experience in Astrophysic, "
              "you excel at contemplating the Universe.",
    llm=perplexity_llm,
    verbose=True,
)

task = Task(
    description="Tell me how many stars there are in the visible sky.",
    expected_output="A single paragraph.",
    agent=agent
)

crew = Crew(
    agents=[agent],
    tasks=[task],
    process=Process.sequential,
    verbose=True,
)

crew_output = crew.kickoff()

print(f"LLM.Crew.kickoff Output:\n\n{crew_output.raw}")

```

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 13, 2025, 8:15am UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/14 "2025-03-13T08:15:40Z")

</div>

Hi Max, the new code still gives the same 400 error.

Just to be clear, what is working for me currently is the following and is not litellm related:

from openai import OpenAI  
import os

client = OpenAI(  
api\_key=os.getenv(“PERPLEXITYAI\_API\_KEY”), base\_url=“[https://api.perplexity.ai](https://api.perplexity.ai)”  
)

messages = [  
{  
“role”: “system”,  
“content”: (  
"You are an artificial intelligence assistant and you need to "  
“engage in a helpful, detailed, polite conversation with a user.”  
),  
},  
{  
“role”: “user”,  
“content”: (“How many stars are in the universe?”),  
},  
]

# chat completion without streaming

response = client.chat.completions.create(  
model=“sonar”,  
messages=messages,  
)

* * *

In the past I was able to get perplexity to work through Litellm with the old Llama-128k models, but those have since been deprecated, and now with the new Sonar series this error 400 problem is coming about.

---

<div class="post-metadata">

**Author:** ![VinnieD](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/vinnied/32/2094_2.png) [@VinnieD](https://community.crewai.com/u/VinnieD)\
**Post date:** [March 14, 2025, 8:18pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/15 "2025-03-14T20:18:08Z")

</div>

Hi,

When I use a specific agent I use this moneypatch:

import litellm

# Patch unsupported parameters

original\_completion = litellm.completion

def patched\_completion(\*args, \*\*kwargs):  
kwargs.pop(‘stop’, None) # Remove problematic parameter  
return original\_completion(\*args, \*\*kwargs)

litellm.completion = patched\_completion

# Now use LiteLLM normally

perplexity\_agent = Agent(  
role=‘Researcher’,  
llm=‘perplexity/sonar-pro’  
)

At the moment I dont use a perplexity agent anymore but have defined it as a tool:

import os  
import logging  
from typing import Optional, Dict, Any  
import requests  
from tenacity import retry, wait\_exponential, stop\_after\_attempt, retry\_if\_exception\_type

from crewai.tools import BaseTool

logger = logging.getLogger( **name** )

class PerplexitySearchTool(BaseTool):  
“”"Tool that performs research using Perplexity API with citation tracking.

```
This tool allows agents to search the web for information on a given topic
and returns results with proper citations.
"""

name: str = "perplexity_search"
description: str = "Performs in-depth research on topics with citation tracking"
api_key: Optional[str] = None
max_results: int = 5
result_count: int = 3

class Config:
    extra = "allow"

def __init__ (
    self,
    api_key: Optional[str] = None,
    max_results: int = 5,
    result_count: int = 3
) -> None:
    """Initialize the Perplexity search tool.
    
    Args:
        api_key: Perplexity API key. Defaults to PERPLEXITY_API_KEY env variable.
        max_results: Maximum number of results to return.
        result_count: Number of results to include in Perplexity API query.
    """
    super(). __init__ ()
    self.api_key = api_key or os.getenv("PERPLEXITY_API_KEY")
    self.max_results = max_results
    self.result_count = result_count
    
    if not self.api_key:
        logger.warning("PERPLEXITY_API_KEY not found in environment variables")

@retry(
    wait=wait_exponential(multiplier=1, min=4, max=60),
    stop=stop_after_attempt(3),
    retry=retry_if_exception_type(requests.exceptions.RequestException)
)
def _run(self, query: str) -> str:
    """Run research query through Perplexity API and format results with citations.
    
    Args:
        query: The search query.
        
    Returns:
        str: Research results with citations.
    """
    if not self.api_key:
        return "Error: Perplexity API key not found in environment variables."
    
    try:
        headers = {
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        }
        
        data = {
            "query": query,
            "max_results": self.result_count
        }
        
        logger.info(f"Sending query to Perplexity API: {query}")
        response = requests.post(
            "https://api.perplexity.ai/search",
            headers=headers,
            json=data,
            timeout=30
        )
        response.raise_for_status()
        
        results = response.json()
        logger.debug(f"Received response from Perplexity API for query: {query}")
        
        # Format the response with citations
        formatted_response = f"## Research Results for: {query}\n\n"
        
        # Add results with citations
        for i, result in enumerate(results.get("results", [])):
            formatted_response += f"### Source {i+1}: {result.get('title', 'Untitled')}\n"
            formatted_response += f"URL: {result.get('url', 'No URL')}\n\n"
            formatted_response += f"{result.get('snippet', 'No content available')}\n\n"
        
        # Add a summary section
        formatted_response += "## Summary\n\n"
        formatted_response += results.get("summary", "No summary available")
        
        return formatted_response
        
    except requests.exceptions.RequestException as e:
        error_msg = f"Error performing Perplexity search: {str(e)}"
        logger.error(error_msg)
        
        # Fallback to a more generic response when the API fails
        return f"""Research results for: {query}

```

---

<div class="post-metadata">

**Author:** ![VinnieD](https://sea1.discourse-cdn.com/flex025/user_avatar/community.crewai.com/vinnied/32/2094_2.png) [@VinnieD](https://community.crewai.com/u/VinnieD)\
**Post date:** [March 14, 2025, 8:29pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/16 "2025-03-14T20:29:12Z")

</div>

I still have a crew with a research agent which uses a llm\_config.yaml:

esearch\_analyst:  
cache: true  
max\_retries: 5  
max\_tokens: 24000  
model: perplexity/sonar-pro  
temperature: 0.3  
timeout: 60

---

<div class="post-metadata">

**Author:** ![Moe](https://avatars.discourse-cdn.com/v4/letter/m/ad7895/32.png) [@Moe](https://community.crewai.com/u/Moe)\
**Post date:** [March 15, 2025, 6:11pm UTC](https://community.crewai.com/t/issue-with-litellm-perplexity-api/4551/17 "2025-03-15T18:11:06Z")

</div>

Hi Vinnie,

Your moneypatch code worked!! Fantastic. Thx!

With regards to it being used as a tool, I noticed that you are using BaseTool in your code. In the latest library update of crewai\_tools, BaseTool is deprecated. What should be used now instead?
