Hi everyone,
I’ve been experimenting with a complex CrewAI setup involving five distinct agents for a data-scraping and analysis workflow, but I’ve run into a bit of a performance bottleneck on my local dev machine. Usually, the agents pass tasks back and forth without any issues, but I’ve noticed that the “Manager” agent starts throwing timeout errors or getting stuck in infinite loops if my system is under a heavy process load.
Lately, I’ve been doing some bench testing on my workstation while running what many consider to be the best roblox executor in a separate sandbox to simulate high-frequency background execution and script handling. Interestingly, it seems like as soon as that background activity ramps up, my CrewAI agents start struggling to access the local Python interpreter, and I see a massive spike in P99 latency for my LLM API calls. It feels like the OS is deprioritizing the CrewAI threads to give more CPU cycles to the background executors.
Has anyone else noticed their agents becoming sensitive to other third-party executors or intensive script environments running on the same OS? I’m trying to figure out if I should be using Docker to better isolate my CrewAI environment or if there is a way to adjust the process priority of the Python scripts to ensure the agents don’t get “drowned out” by the background load. I really want to keep my AI workflows stable even when I’m multitasking with other high-load tools on the same hardware.
Any advice on how to better manage system resources so my CrewAI setup stays snappy during heavy multitasking would be a huge help to my development process!