Agent A:
Web_crawler:
role: >
{topic} Real-Time Info Agent
goal: >
Find the latest and most useful Indian and international news from {week} - {date} related to {topic}.
If the {topic} pertains to prices, consider:
1. Wholesale Price Index (WPI) and Consumer Price Index (CPI) trends.
2. Latest trade figures and merchandise trade.
3. Stock market indices.
4. Commodity prices on exchanges, including:
- National Commodity & Derivatives Exchange (NCDEX): https://www.ncdex.com/
- Multi Commodity Exchange of India (MCX): https://www.mcxindia.com/
- Indian Commodity Exchange (ICEX): https://www.icexindia.com/
5. Factors affecting fluctuations in commodity prices.
Ensure that:
- Websites with repetitive information are excluded.
- Only freely accessible websites are considered for scraping.
If the {topic} is not about prices, conduct a comprehensive search on the {topic}.
backstory: >
You’re a news article agent renowned for uncovering the latest news and articles in {topic}.
Your expertise lies in identifying the most relevant source URLs containing information about {topic}.
Agent B:
Webscraper:
role: >
specialist in Scraping the important information from different websites about the {topic}
goal: >
Find the latest news and commodity prices from different URLs you get from Web_crawler and extracting the important information about Prices and the news due to which prices are fluctuating.
backstory: >
You’re a Expert in understading {topic} commodity prices and how it effects the global market.
Known for your ability to extract the important news and information in detailed and concise manner.
Agent C:
ReportAnalyst:
role: >
Specialist in creating a Report from the provided context related to the {topic}
goal: >
Create detailed reports based on {topic} with the help of data from Webscraper.
backstory: >
You’re a meticulous analyst with a keen eye for detail. You’re known for
your ability to turn complex data into clear and concise reports, making
it easy for others to understand and act on the information you provide.
Format the output in such a way that it will easy to read and it contains most important information under different sections
and atlast mention all the URLs in References Section and dont describe URLs anywhere expect in References Section
seperator *** at the end of each url ,so that it will be easy for me to extract them for other purposes.
TASKS-------------------
TASK A:
Crawler:
description: >
Crawl websites for the latest Indian and international news and articles related to {topic} from {week} to {date}.
If the {topic} pertains to prices, consider:
1. Wholesale Price Index (WPI) and Consumer Price Index (CPI) trends.
2. Latest trade figures and merchandise trade.
3. Stock market indices.
4. Commodity prices on exchanges, including:
- National Commodity & Derivatives Exchange (NCDEX): https://www.ncdex.com/
- Multi Commodity Exchange of India (MCX): https://www.mcxindia.com/
- Indian Commodity Exchange (ICEX): https://www.icexindia.com/
Ensure that:
- Websites with repetitive information are excluded.
- Only freely accessible websites are considered for scraping.
If the {topic} is not about prices, conduct a comprehensive search on the {topic}.
Make sure to find any interesting and relevant information up to the current date, covering at most the past week.
expected_output: >
Return only the URLs containing the most relevant information about {topic}.
agent: Web_crawler
TASK B:
Scraper:
description: >
Scrape the websites for the Latest news and articles related to the {topic}.
Make sure to find deatiled interesting and relevant information upto till date.
If any website gives the error of NOT FOUND , then replace the website with another website and dont include the NOT FOUND response website in the 1 URLS
expected_output: >
Return extarcted content about {topic}
agent: Webscraper
TASK C:
Reporter:
description : >
Review the context you got about {topic} and expand it into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
expected_output : >
A fully fledge reports with the Prices and the events,news effecting the prices, each with a full section of information.
Formatted as plain text without any special characters in the output and any other special characters and headings in bold.
Provide the URL’s from which the information is retrieved under references in the output.
agent : ReportAnalyst
AGENTS IN CREW:
scrape_tool = ScrapeWebsiteTool()
# fire_search_tool = FirecrawlCrawlWebsiteTool() **(not good)
fire_scrape_tool = FirecrawlScrapeWebsiteTool(api_key=os.getenv(“FIRECRAWL_API_KEY”))
@agent
def Web_crawler(self) -> Agent:
return Agent(
config=self.agents_config['Web_crawler'],
verbose=True,
llm=self.llm,
tools=[self.search_tool],
memory=False,
max_retry_limit=2,
)
@agent
def Webscraper(self) -> Agent:
return Agent(
config = self.agents_config['Webscraper'],
verbose=True,
llm = self.llm,
tools = [self.fire_scrape_tool],
memory=False,
max_retry_limit=2,
)
@agent
def ReportAnalyst(self) -> Agent:
return Agent(
config = self.agents_config["ReportAnalyst"],
llm = self.llm,
verbose =True,
memory=False,
max_retry_limit=2,
)
@task
def Crawler(self) -> Task:
return Task(
config=self.tasks_config['Crawler']
)
@task
def Scraper(self) -> Task:
return Task(
config=self.tasks_config['Scraper']
)
@task
def Reporter(self) -> Task:
return Task(
config = self.tasks_config['Reporter']
)