Try to read all files in a folder

CrewAI Community Support
,
1

Hi, I’m trying to create a Crew that reads all source code files found in a folder (playing with some legacy source code migration scenarios) and what i’m seeing is that only a few (sometimes one, sometimes two or three but no more) files are read and processed. I have an agent with only one task that so far the only thing is trying to do is to read those source code files and generate some YAML metadata (purpose, references, callers, etc.) extracted from the source code. What i’m doing wrong?

2

@AndresVettori Just to make sure, you’re using the DirectoryReadTool, right?

3

Exactly, I have configured that tool for the task, and that works perfectly, but somehow the agent decides not to process all files but only a few and I don’t know how to control that. Those files are legacy source code I’m trying to analyze to extract metadata and want to write all resutls from all analyzed files into one output file.

4

Two suggestions come to mind:

  1. Have you tried adjusting the agent and task configurations to explicitly specify that all files in the directory should be processed? I’ve used the DirectoryReadTool tool before, and it worked flawlessly for me, recognizing all the files in the directory without any issues, but I make sure that I always set clear configurations.
  2. Have you tried using a more capable LLM? Sometimes, upgrading to a more advanced LLM can resolve such issues by providing better handling and understanding of the task.
5

What about counting the number of files in the directory first and then either ensuring that it processed that number of files --or-- use the count of files as an index and ask for file[1], file[2], and so on?

6

Thanks for your response, I will try to clarify a little more my scenario regarding the two comments you kindly posted:

  1. I think the DirectoryReadTool is not the problem, as I see it reading (and fetching the task) the whole contents of the directory (reading all file names correctly). So, because how that list of files (that is quite extensive, there are hundreds of files to process) is processed I think is the agent responsibility I have tried to be very explicit in the prompt (agent goal and task description) clearly stating that all files must be processed and no optimization should be done there, but no change in results.
  2. I am usign gpt4-o and have also tried with other advanced models (gemini, claude, etc.) with the same results so I don’t think this is something related to the models.

What I’m going to try now is to create some custom code that receives the directory name to process and explicitely read all files and process them. It seems a little counter intuitive to do that because I don’t need CrewAI to do it that way, but maybe there is something I can try and later could be adapted into a more native CrewAI way of working.

Any ideas are welcome,

Thanks again.

7

Thanks a lot for the suggestion, any pointers or sample on how to implement that in CrewAI?

Agv

8

Got it! You can force tool output as a result by setting the result_as_answer parameter to True. As stated in the docs:

This parameter ensures that the tool output is captured and returned as the task result, without any modifications by the agent.

Try the following:

from crewai.agent import Agent
from crewai_tools import DirectoryReadTool

my_agent = Agent(
    ...,
    tools=[DirectoryReadTool(result_as_answer=True)],
)
9

Hi, just tried the “force tool output as result” but is not what I need. It makes the result of the tool be returned by the agent without any processing, and I need the agent to process that file list and do something (analyze those file contents).

What I’m seeing is this (output from the Agent):

# Agent: AI LLMs Expert Software Developer

Thought: Thought: The result of the file listing has shown a vast number of files in the directory, spread across various components such as models, views, controllers, services, and more. Given the vastness, I need to start with one category and one specific file to analyze and proceed towards extracting the necessary information. I’ll begin by looking at the app/models directory and analyze a file from there to capture the structure and functionality.

To begin, I’ll read the content of one of the model files that might have complex business logic. Let’s start with app/models/account.rb since accounts are fundamental to ERP systems, and there’s a corresponding accounts table in the SQL DDL that has various fields and constraints.

Using tool: Read a file’s content

Tool Input:

“{"file_path": "/Users/andres.vettori/Projects/BonsaiERP/app/models/account.rb"}”

Tool Output:

The output here is perfect, it’s correct.

BUT…

It’s only FOR ONE FILE and the agent doesn’t continue processing more files.
And the agent has looked into the file list and it’s “vast” so he decides to choose a few (or just one as in this case).

What I also see now is that after processing a few files I get an error from the model:

Error parsing LLM output, agent will retry: I did it wrong. Invalid Format: I missed the ‘Action:’ after ‘Thought:’. I will do right next, and don’t use a tool I have already used.
If you don’t need to use any more tools, you must give your best complete final answer, make sure it satisfy the expect criteria, use the EXACT format below:
Thought: I now can give a great answer
Final Answer: my best complete final answer to the task.

And so the agent returns the results it has compiled so far (for three files).

This might be a case of context exhaustion (of some kind) in the model?

10

Why not create an agent or task just to list all files in a directory and set the result_as_answer parameter to True? Then, use another agent or task to process them.

This won’t fully solve your goal of processing all the files, but it makes the workflow easier to handle and debug by breaking it into smaller steps.

11

Ok, I have implemented this idea and now I’m testing it. Let’s see how it goes…

Agv

12

Let me know the results!

13

@AndresVettori I am curious how your adventure went here? Did this yield the results you were looking for?

14

Hi and thanks for asking. The use of “result_as_answer” worked well for things deterministic, like reading a folder’s content. But, for other kind of things where an agent is told (via prompt) to do something over a large list of files (maybe hundreds) then in all cases the agent try to “deduce” what is the best subset of files to accomplish the goals, and so never process all files and most of the time it only process one file, sometimes three, you get it.

So, in my case I need to move forward and then stick to just code calling an LLM with more reduced scope (like analyzing one file at the time) and that’s it. So, i have (for now) decided not to use CrewAI for my endeavor.

Thanks a lot,
Agv

15

I don’t know if this can help, but I used CrewAI to analyze the code of my application. I know there are probably more relevant and more complete tools, but I wanted to have a summarized and functional view.

The key point is that we don’t control the context size, so a specific LLM configuration must be used. My project being quite simple, this single parameter allows me to have my documentation, but if you exceed the LLM’s context window, it will be a bit trickier :slight_smile:

I hope this can help you, it’s quite simple but it works for me :

Here is my code :

from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from lead_score_flow.models import CandidateScore
from crewai import LLM
from crewai_tools import DirectoryReadTool,FileReadTool
from pathlib import Path

llm = LLM(
model=“gpt-4o-mini”,
temperature=0.4,
max_tokens=4000,
top_p=0.9,
frequency_penalty=0.1,
presence_penalty=0.1,
)

@CrewBase
class CodeAnalysis:
“”“Code Analyzer”“”

agents_config = "config/agents.yaml"
tasks_config = "config/tasks.yaml"

@agent
def expert_python_agent(self) -> Agent:
     # Remonter au répertoire 'lead_score_flow'
    directory_path = Path(__file__).parents[3]
    print(f"Répertoire cible : {directory_path}")
    directory_tool = DirectoryReadTool(directory=str(directory_path))
    file_read_tool = FileReadTool()

    return Agent(
        config=self.agents_config["expert_python_agent"],
        verbose=True,
        llm=llm,
        tools=[directory_tool,file_read_tool],
        allow_code_execution=False,  # Disable code execution
    )

@task
def code_analysis(self) -> Task:
    return Task(
        config=self.tasks_config["code_analysis"],
        output_pydantic=CandidateScore,
    )

@crew
def crew(self) -> Crew:
    """Creates the Code Analysis Crew"""
    return Crew(
        agents=self.agents,
        tasks=self.tasks,
        process=Process.sequential,
        verbose=True,
    )

code_analysis:
description: >
ONLY READ Python and YAML files, analyze the code to deliver a comprehensive explanation of the global application functionality.
You MUST read all Python and YAML files

Use your expertise to thoroughly examine the code, considering key aspects such as:
  - Functionality and logic


ADDITIONAL INSTRUCTIONS
-----------------------
Your final answer MUST include:
- A resume explanation  of what the application does

**Important:** Do not ask any questions to the user and provide your final answer following the instructions above.
**Important:** Do not explain each file.
**Important:** Do not read file in code_analysis_crew.
**Important:** Do not write technical terms and test or configuration information.

expected_output: >
formatted md file. A thorough analysis of the provided application, including explanations of its functionality.
agent: expert_python_agent

expert_python_agent:
role: >
Expert Python Programmer
goal: >
Analyze the Python code to provide a precise explanation of how it works.
backstory: >
As a Python programming expert, you have extensive experience in software development and the architecture of generative AI solutions using agents.
You excel at providing simple explanations of how an application works, with content that is understandable by CEOs, IT teams, or business teams
who do not have a strong knowledge of development or the functioning of AI agents.