Incomplete Data Saved in CSV While Using CrewAI

CrewAI Community Support
1

I am using CrewAI with multiple agents and tools to fetch historical stock data and save it as a CSV file. However, despite the correct data retrieval, the saved CSV only contains a subset of rows, instead of the full year’s worth of data.

Tools Used:

  1. Get Historical Data Tool:
    @tool(“Get Historical data”)
    def historical_data(ticker: str, period: str = ‘1y’) → dict:
    “”"
    Retrieves historical stock data based on the ticker symbol.
    “”"
    ticker = FinanceTools.format_ticker(ticker)
    ticker_data = yf.Ticker(ticker)
    recent = ticker_data.history(period=period)
    if recent.empty:
    return f"No historical data found for {ticker}."
    return recent

  2. Save CSV Tool:
    @tool(“Save data as CSV”)
    def save_csv(file_name: str, data_json: str) → str:
    “”“Saves JSON data as a CSV file in the ./CSVs directory.”“”
    try:
    file_path = os.path.join(CSVTools.csv_directory, file_name)
    df = pd.read_json(StringIO(data_json))
    df.to_csv(file_path, index=False)
    return f"Data saved as CSV at {file_path}"
    except Exception as e:
    return f"Error saving CSV file: {e}"

The retrieved data is correctly displayed, showing 249 rows covering the full year. However, when the data is saved into a CSV file, the saved CSV only contains a few rows instead of the full dataset.

Steps Taken

  1. Queried historical stock data for Reliance with a 1-year period.
  2. The retrieved data from Get Historical Data correctly shows 249 rows (from 2024-02-05 to 2025-02-05).
  3. Attempted to save the retrieved data using Save data as CSV.
  4. The saved CSV file only contains a handful of rows, and the values seem different from the originally retrieved data.

Expected Outcome

  • The CSV file should contain all 249 rows as retrieved by Get Historical Data.

Actual Outcome

  • The saved CSV file contains only a few rows instead of the full dataset.

Questions & Assistance Needed

  • How can I ensure that the full dataset (249 rows) is saved into the CSV file?
  • Could there be an issue in the way the Save data as CSV tool handles the data?
  • Is there a recommended way to debug where the truncation or modification occurs?

Any insights or fixes would be greatly appreciated! :rocket: