Document proxy for RAG search

Hello,

I am still new to crew ai. I have read about the PDF/DOCX RAG Search tools, however I don’t understand how to deploy responsibilities separately. Let me try to explain.

Imagine I have a site A, and a site B. Each have documents. I want to run a global crew ai tool that uses RAG searching tools on both sites, yet it is not hosted in A nor B.

Possible solutions that I can think of:

  • VPN. Downside, what if the connection breaks?
  • Cron process overnight to convert the files to text and upload them for processing. Downside, it is not realtime.

What would be the best approach? Any thoughts?

Thanks.