Playwright custom tool - should I use sync or async version?

CrewAI Community Support
1

I’ve created and tested separately my custom playwright_tool - it works.
But when I tried to use it in my crewai flow for 1 of agents - I got 2 types of warnings in logs:

  • if I use sync version of playwright - it logs out:
    I encountered an error while trying to use the tool. This was the error: It looks like you are using Playwright Sync API inside the asyncio loop.
    Please use the Async API instead…
  • if I use async version - it logs out coroutine, as the output of my tool:

Agent: URL My Scraper

Using tool: Playwright Scrape Website Tool

Tool Input:

“{"url": "some_url"}”

Tool Output:

<coroutine object PlaywrightTool._run at 0x000001C43A805F20>

Sure, my _run method is async:
async def _run(self, url: str) → dict:
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page()
await page.goto(url, wait_until=“networkidle”)
etc
…
…

But _run is not awaited when crew is running.
So how to handle this?

2

Hi Alex,

did you make progress with Playwright tool. I was checking if there is already a tool available. It would be great if you share your experience or progress here.

Regards
Ali

3

tl;dr: try using the async implementation, use nest_asyncio library, and attach the crewAI async loop

I’m having a very similar issue since I tried to implement a custom tool using Crawl4AI (which happens to have an async implementation). I think the problem seems to that CrewAI starts kickoff flows using async. See this code section:

<in crewai/flow/flow.py>
 def kickoff(self, inputs: Optional[Dict[str, Any]] = None) -> Any:
        """
        Start the flow execution in a synchronous context.

        This method wraps kickoff_async so that all state initialization and event
        emission is handled in the asynchronous method.
        """

        async def run_flow():
            return await self.kickoff_async(inputs)

        return asyncio.run(run_flow())

CrewAI seems to only work with non-async tools (that’s why you get the coroutine output).
I found a way to make async tools work with CrewAI, but it requires a 3rd party library called nest_asyncio (see: nest-asyncio · PyPI)

Here’s a code snippet that worked for me inside the tool:

# Keep _run synchronous as required
    def _run(self, **kwargs: Any) -> str:
        website_url = kwargs.get("website_url", self.website_url)

        try:
            loop = asyncio.get_running_loop()
            nest_asyncio.apply(loop) # this patches the loop created by CrewAI to allow nested asyncio.run
        except RuntimeError:  # No running loop
            print("No running loop")
            return asyncio.run(self._crawl_content(website_url))
        else:  # If loop is already running. This is required since crewAI kickoff executes a asyncio.run(run_flow())
            print("Attaching to existing async loop")
            return loop.run_until_complete(self._crawl_content(website_url))

    # Async helper method
    async def _crawl_content(self, website_url: str) -> str:
        # your code goes here
1 Like