How to use multiple independant "memory" and RAG database for different crews?

CrewAI Community Support
1

Hello,
Is there a way to have multiple crews that use fully independent “memory” and rag tools, i.e. that have separate ./db folders I guess.
I tried updating the CREWAI_STORAGE_DIR and CHROMA_DB_FOLDER variables before creating each crew but it does not seem to work and I have only one ./db in my root project folder… What I want is multiple fully different crew1/db crew2/db etc… so there is absolutely no mixup of data between the crews…
I guess that should be possible… how?

2

well it seems to work if I just chdir() to a different path before creating new crews…

3

Where in the code are you doing the chdir?

4 
    pdf_search_tool = PDFSearchTool(
        config=dict(
            embedder={
                "provider": "openai",
                "config": {
                    "model": 'text-embedding-3-small'
                }
            },
            vectordb=dict(
                provider="chroma",
                config=dict(
                    dir=f'crew-db-{uuid.uuid4()}'
                    ...
                )
        )
    )

Something like this, note the UUID creating the dir so its distinct

1 Like
5

Just “os.chdir('a speific path for each crew”)"…

I want to be able to “reset the memory” of a specifc crew and not some others. Using crewai reset_memories does not give any flexibilty to pick which part of the memory to reset.

If each crew that needs to be indpendant has a completly different “home” folder with a fully different ./db etc… then I am sure I can manage each crew memory and everything else autonomously… and also be certain that any rag query will not mix up data.

It’s not a very a very sophisticated design but it seems to work…

6

are you pipelining the crews?

7

No, simple crew with one agent and one task, default openai key… plain and standard…

But why pipeline would make any difference related to memory or rag ??

8

I was just curious and wondering how memory and context got dragged along or not when pipelining crews.

9

Great tip when processing multiple items having a different knowledge base using the same crew (eg: what is the color of my cat? with pdf1 = cat is blue, pdf2 = cat is red, …). When I store all PDFs in the same vector store, the retriever tends to mix things up between various data sources.

Is there any alternative to using a different db? For instance, with metadata?