Agents Sometimes Follow Deleted or Wrong Templates Despite Full Memory Reset and Cache Cleanup

CrewAI Community Support
,
1

:memo: Issue Summary

I’m building a multi-agent system using CrewAI where each agent/task is guided and informed by structured templates. These templates are .txt files used as knowledge sources via TextFileKnowledgeSource (in crew.py), and they contain:

  • Detailed instructions
  • Numbering and structure
  • Specific formatting rules

Each template is explicitly designed for one task, and vice versa — a 1:1 mapping. however not all tasks have a dedicated template. These are critical, as the agent is expected to follow the structure and instructions in them exactly and generate outputs accordingly.

:red_exclamation_mark: The Problem

In some executions, the system behaves correctly:
:white_check_mark: The agent uses the current, correct template (knowledge file) and follows its structure precisely.

But in other executions — with absolutely no change to the system whatsoever — the agent either:

:cross_mark: Seems to follow a deleted template, producing output based on outdated instructions or numbering.

:cross_mark: or follows the wrong template. this means that instead of the knowledge file (template) that it has been explicitly referenced in task description, it follows other existing template files.

:cross_mark: or in rare cases, the agent follows instruction of other tasks (and their templates) thoroughly! this is while two different agents have been assigned to these two tasks and this mixture should not happen based on any logic!

This behavior is inconsistent and unpredictable, which makes the system unreliable when it comes to acquiring standard results across runs (and this is why i have created templates).

:white_check_mark: What I’ve Already Done

Despite multiple defensive measures, the issue persists. Here’s what I have in place:
** agent memory, cache, crew memory is turned off (i.e. are set to false)
Memory Reset at the start of every run using:
Manual deletion of the following folders before every run:

  • knowledge

  • short_term

  • entities
    (note: only when crew memory is set to true, the short_term and entities folders are created and then manually deleted; otherwise only the knowledge is automatically created and manually deleted).
    Removed additional memory-related files (in case they are created):

  • latest_kickoff_task_outputs.db

  • long_term_memory_storage.db
    Set these environment variables at the very top of the code (before any CrewAI import):
    import os
    crewai_storage_path = r"the path i have identified on my computer"
    os.environ[“CREWAI_STORAGE_DIR”] = crewai_storage_path
    os.environ[“EMBEDCHAIN_PERSIST_DIR”] = os.path.join(os.environ[“CREWAI_STORAGE_DIR”], “knowledge”)

  • All knowledge files are .txt files, and no old copies of templates exist in the file system.
    their names are up to date and correctly reflected in TextFileKnowledgeSourcein my crew.py .

  • Tasks.yaml is current and the tasks correspond to the correct knowledge file for each task.

:warning: What I Observe

  • The issue is not constant.
  • Sometimes the system works perfectly.
  • But randomly, the agent reverts to behavior consistent with wrong or deleted template, (i.e. either the file does not exist anymore or it does exist but is not the one the agent in the specific task is expected to follow.)

:folded_hands: i appreciate any solutions along with any insight into:

  1. Are embeddings or templates cached elsewhere on the system outside CREWAI_STORAGE_DIR?
  2. as there should be some cached data that are interfering across runs, i would like to ask about any other form of cache, memory data, etc that crewai saves for the purpose of using across different runs? Similarly are there any internal fallback behaviors or cache mechanisms in CrewAI that could be using stale embeddings?
  3. Do files like latest_kickoff_task_outputs.db or long_term_memory_storage.db influence which knowledge source is used, even when memory is reset and knowledge is deleted.
  4. to monitor crew execution, task outputs are set to be automatically saved to automatically created folders in a specific path (not the one that CREWAI_STORAGE_DIR or the db files are saved). do i need to delete all task output files before every new run?
  5. Can long-term memory or embedding-based retrieval interfere even if the .txt file is fully replaced?
  6. is there a way to bind the knowledge files with specific tasks in a technical way in crew.py or main.py?

I’ve been working on this CrewAI system for months, and I’d really like to make it reliable. Thank you so much for any help or suggestions!