I’m reading about Flow and I am wondering how is it different from Pipeline. In what cases would I choose a Flow over a Pipeline? And another thing is that it seems like it’s a little bit on stealth mode. Is it reliable/usable yet?
Hi @mayonesa,
I’ve just read through the docs, and while my answer may not be definative, this is the ‘major’ difference:
Pipelines
" A pipeline in crewAI represents a structured workflow that allows for the sequential or parallel execution of multiple crews. It provides a way to organize complex processes involving multiple stages, where the output of one stage can serve as input for subsequent stages."
Flows
“Flows allow you to create structured, event-driven workflows.”
Note the ‘event driven’ part.
Within the docs take alook at the example, take note of @listen(generate_city) where generate_city is a method on the class.
The above suggests that due to Flows being ‘event driven’ the @listen decorator stipulates that the generate_fun_fact method will not be called until it is ‘triggered’ by an event that is probably fired when the generate_city is completed.
Hope that helps.
1 Like
Thank you for that! I’m just wondering if I should update to use Flow (which would require me to spend more time than i already have debugging using LiteLLM for my SageMaker endpoint since LangChain model interface is no longer possible). So far, I still don’t see what effectively I’m getting with Flow over Pipeline — the difference in mechanism by which they do the dynamic navigation alone is not compelling enough to put more time into the LiteLLM-SageMaker issue. I am hoping to get some insight beyond what is in the docs. For example, the motivation behind Flow and why would i pick it over Pipeline
Hi @mayonesa ,
Flow is for ‘event driven processes’. I’m not 100% about your prticular issue, but I’ll have a go 
The Flow process is event driven, so have you got dynamic events that could drive your Flow?
Alternately, do you have a clear process to follow that you/your code can initiate/start. Is it a continuose process, or do you have breaks in process where you await such as user input, response/input from another process/system.
These are the types of considerations that I would be evaluating if I was deciding to use Flow, or Pipeline.
A hyperthetical example:
You may have a process that you/your code can start and you have an expectation that the process will continue uninterupted until complete. (Pipleline)
If you have a process that excites another external service or requires some functionality to complete before a another method can be started (Flow) is the way to go where you would trigger the target functionality ONLY when triggered by a required method/process.
I know, sound a bit convoluted, but I do’nt klnow your code, process requirements.
To clarify this topic there’s a new You Tube Video that explains Flows 
1 Like
Deal-breaker (for me): Flow relies on mutable state which can present heisenbugs.
Not quite a deal-breaker: totally new DSL.
I’ve been experimenting with LangGraph to set up a system that would just keep running. Would Flow replace LangGraph?
My ‘gut feeling’ response would be Flow, an ‘event driven’ process. If you control the events, then you have more control over what happens when and for what length of time.
Having said that, a pipeline process could be told to start again once it is completed!
I suppose the choice would be relevant to the context within which your application is operating/working.
I have a ‘content driven’ website (we curate and create food recipes). My agents should go out and find new recipes, adapt them to our site and publish them. Then they should publish the recipe to Instagram. It should also write articles about certain topics. And finally send out some newsletter emails.
In a futture version I’d also like to employ the agents to search for offers on groceries and send out recipes based on what the current offers are etc.
It’s a pretty big setup, but it’s also quite reliable in terms of the task complexity.
I’ve looked at Crew with LangGraph before, but it seems like maybe Pipelines and Flows could do the same job with less hassle…
What do you think?
As I say to everyone, the only limits of what you can do with CrewAI are the limits of your own imagination!. True for ALL software development.
So standing back and looking at the big picture, one question jumps out: How often do you want your Agents/Crews to do these searches, checks?
From an engineering perspective ‘continuously’ just appears bad!
- How often is the searched data likely to change?
- What costs are there in terms of resources when these activities are running?
Plus anything else that you can think of.
I don’t know how much of your code is server-side based, but if you can look at setting up ‘cron’ jobs/tasks, scheduling, etc. By default all part of a Linux OS.
I have found a possible solution to scheduling your taks just using Python for you.
Best of luck 
and keep us updated
P.S. You should also take note of any copyrighted content that you may be scraping 
I was thinking of having the agents go out like a few times per day. Brandon Hancock has a nice example using CrewAI and LangGraph to check his Gmail inbox and write draft responses. He basically just sets a node that is a timer and lets it run every few minutes.
hehe… the insanely sweet thing about recipes is that a recipe isn’t protected by IP laws. At least not in the sense that you can’t reproduce them. So an agent could find a recipe. Rephrase it in the description etc. But keep all the ingredients and steps 
Will keep you posted 
FYI: The recipe may not be copyrighted, but the publish text that described it probably will be!!
If you scrape any website within the EU, USA, UK the laws states that there is an ‘implied’ copyright, regardless of whether it states it or not. Please check and be careful.
@matt Maybe a topic in general to promote the legal dangers of scraping sites. If nothing else it would keep potential bad press away from the CrewAI brand.
That’s a fair point. I’ll make sure to do some due dilligence on this. I don’t want to get anyone in trouble.
Thanks for pointing that out 
Ask a Task/Agent pair (un-copyright team) to take in copyrighted content & re-write in a way that does not reflect the structure or uniqueness of the original text, but maintains subject and context.
Is that a partial prompt
1 Like
That was the initial idea. But I didn’t consider scraping restrictions:-/
But… I’m thinking, like 4-5 recipes per day. From a wide range of sources. So it’s not like we’re going to hit rate limits with the domains 
But mostly I’m very eager to try putting together a complex flow like this. I’ve got some other, less legally challenging cases that would benefit immensely.
I just really want to become very very proficient with CrewAI:nerd_face:
Hi @Michael_Birkeland ,
Hers my ‘philosophy’:
If i read a book and learn from it, then write and publish a paper about the same subject as the book, not including any direct content from the book.
A) I own the implied copyright of my paper.
B) I have not breached the copyright of the book.
Replace book with scraped recipe, and me with LLM/Agent, job done 
N.B. I don’t believe that you can copyright the ingredients. An omlete has eggs period!
1 Like
Thanks I think this sounds reasonable. Also, it’s a very nice challenge for my crew to take something and create something “new” from it.
I’ll keep you posted as I progress.
Thanks again for valuable feedback
FYI: Iv’e been in this industry for 52 years, wrote my first code: December 12th 1972! Employed at every level, both as a softwar ‘engineer’ & tech mangament.
If I can’t take my experience with me when I pass away, then this is a good place for me to leave it.
Best of luck and anything else I can asisst with, just ask.
That’s crazy. 
You were litteraly using punch-cards at that time. I can pretty much guarantee you that I will take you up on that offer 