The concept that motivated me to learn CrewAI

General
1

A picture paints a thousand words: look at this diagram

image
image1536×1503 132 KB

If you’ve read my profile you will know that I’m recently retired after 50+ years as a software engineer. The majority of my later working life spent designing & maintaining micro-service back-end systems. Before I left my last employer (GoStudent Gmbh) I gave a talk on what I believe the future held for BE systems. The diagram sums up my thoughts.

I believe that replacing dumb sequential messaging systems with AI driven network/service based on ‘business/decisions’ is the way to go, or at least an alternative where ‘services’ could be treated in a similar way to external tools.

The CrewAI in the diagram would act as a ‘cognitive gateway’ to the micro-services.

There’s another part to my conceptual idea, custom fine tuned LLMs. I have fine tuned a number of open source LLMs to give/improve expertise in a custom area. Assume that LLMs used in such a BE AI controlled micro-services system have been fine tuned to reflect the needs/business logic of a particular customer, etc. LLMs that are ‘business aware’

In ‘theory’ the Agent role-play scenario could eventually be ‘You are a CEO of a business, etc’ :rofl:

As I said, conceptual, but any/all input welcome.

A bit techy stuff!
In essence the inputs to the controlling AI will be in the form of events:

  1. inputs from website front-ends, etc
  2. events generated by the services themselves
  3. chronological (cron jobs) that will trigger Tasks

I imagine that output from the AI control will also be of event type .

The irony of the above is that existing network messaging such as NATS, Pulsar would probably be used for the interface between the AI control & the services themselves.

Assume that existing DB services will become AI RAG services, etc.

N.B. No reason why services could be other AI processes.

Why after 50 years I’ve only (3 months ago) just decided to learn Python! Well that’s a whole different story :slight_smile:

1 Like
2

This is a pretty cool idea. I was talking with some of the cofounders of stability.ai the other day about the future of LLM’s in managing inferastructure. The basic output of that conversation was that DevOps and NOC’s will basically be gone in 5 years and LLM’s will be able to manage and monitor your inferastructure. So basically myself being an observability, monitoring, performace engineer have my days numbered. Hence why I’m here. lol

3

As an intermediate development you could create a Crew that takes ‘events’ from your monitored systems as input and work on the Agents/Tasks to see how much of your job you can get the Crew to do. :slightly_smiling_face:

4

That’s my current project. Take a live feed of monitoring data / logs and feed it to the LLM’s. The llms act as the NOC and look up relevant info from SOP’s and act on them. If they don’t resolve the issue they will either create a ticket and assign it to the support group or page and even call the relevant support team to escalate properly. But you can take this further into your whole CI/CD pipeline. LLMs manage all your deployments and monitoring setups. Infrastructure as code but orchestrated all by agents. My current issue is getting linux agents to remotely talk to windows. Such a pain.

5

If one does not exist you could fine-tune an LLM for K8’s. Creating/managing K8’s clusters is just generating yaml files.

6

Yeah its super easy. I can train my own models and just have LORA’s for specific tasks. Its just building on top of llama.cpp and making a router api to split tasks. So i need to build my own version of ollama with the secret sause added in and then use specialised agents to refrence the fine tuned models and LORA’s per task. It’s a big project especially if you want it to be universial for all datacenters, cloud’s etc.Oh and yeah… legacy stuff… “We have a windows NT server and if we turn it off our prod breaks and we dont know why” lol

7

Have any of you had a look at this?

1 Like
8

I had heard of it, but that’s more dev-ops & not related to my concept. I’m sure that there will be many more AI driven BE tools on the horizon.

9

This is super cool. Seems they have a limited support range of tools instead of a universal one but it’s a great start. I looked them up and it seems one of the founders follows me on linkedin. Maybe they are hiring.