Intermittent 401 for my custom LiteLLM

General
3

Ran into the same issue. Apparently, the way to pass api key is to use extra_body, it was intermittent becuase litellm was caching, when litellm serves cached response it does not validate api key.


custom_llm = LLM(
    model=api_model,
    base_url=api_base,
    api_key=gif_api_key,
    extra_body={"api_key": gif_api_key}
)