CustomLLM Implementation Error

[George_Belsian]

I am working on creating a Custom LLM by following this documentation, Custom LLM Implementation - CrewAI . While executing that , I get into an error “TypeError: BaseLLM.init() missing 1 required positional argument: ‘model’” . Its expecting “model” as one of the parameter. But I don’t see that in the examples provided. Is there a disconnect between the python SDK and documentation ?

I am using crewai SDK Version: 0.120.1

Example snippet from the documentation
class CustomLLM(BaseLLM):
def init(self, api_key: str, endpoint: str):
super().init() # Initialize the base class to set default attributes
if not api_key or not isinstance(api_key, str):
raise ValueError(“Invalid API key: must be a non-empty string”)
if not endpoint or not isinstance(endpoint, str):
raise ValueError(“Invalid endpoint URL: must be a non-empty string”)
self.api_key = api_key
self.endpoint = endpoint
self.stop = # You can customize stop words if needed

def call(
    self,
    messages: Union[str, List[Dict[str, str]]],

[Max_Moura]

Hey George. First off, welcome to the community!

You’re right. There’s an issue with the official documentation. Take a look at how the BaseLLM abstract class is defined in the source code:

class BaseLLM(ABC):
    model: str # <= This 'model' attribute is currently required by the constructor.
    temperature: Optional[float] = None
    stop: Optional[List[str]] = None

So, you have to pass a model string. Generally, this makes sense because, after all, you’ll usually need a model name when making an API call, right? But this attribute isn’t actually used explicitly by the BaseLLM class itself after initialization, so you can just pass any string, like this:

class CustomLLM(BaseLLM):
    """
    Example of a custom LLM implementation.
    
    This class demonstrates how to initialize the mandatory 'model' 
    attribute required by the BaseLLM superclass.
    """
    def __init__(self, api_key: str, endpoint: str):
        super().__init__(model="BelsianLLM") 
        
        # ... and any other setup your LLM needs (e.g., API client).

Ideally, either the official documentation should be updated to mention that model must be passed to super().__init__(), or the abstract class code itself should be changed to model: Optional[str] = None to make it truly optional if it’s not used.

Either way, feel free to open an Issue and link back to this thread.