It seems that the issue was that litellmfetches a model cost map from an external server during import, and this network request was slow due to my ISP or routing issues.
After multiple attempts to fix the issue, one of them was enabling a VPN on my machine and routing the traffic through it, which did fix the problem.
I’ve also found (while doing some profiling) that the bottleneck resided in the get_model_cost_map function, which led me to the LITELLM_LOCAL_MODEL_COST_MAP environment variable, which in its turn, through some Google search, led me to this page from their docs where it states:
–
Don’t pull hosted model_cost_map
If you have firewalls, and want to just use the local copy of the model cost map, you can do so like this:
export LITELLM_LOCAL_MODEL_COST_MAP="True"
Note: this means you will need to upgrade to get updated pricing, and newer models.
–
I hope this can help someone going through the same issue.