Link to demo - https://www.linkedin.com/posts/elliott-a-b82ba749_ai-machinelearning-crewai-activity-7308083034346422272-P5xO?utm_source=share&utm_medium=member_desktop&rcm=ACoAAApSM9IBsnZ4oVY9Cz2eraivuxtiukJwNUE
# Ebook Generation Process Architecture
## System Overview
The ebook generation process involves three major components:
1. Markdown Parser
2. Asset Fetcher
3. Pandoc PDF Converter
These components work together to generate a final PDF ebook from given markdown content and associated assets.
## Component Details
### Markdown Parser
**Responsibilities:**
- Reads and validates input markdown content.
- Generates a document outlining the structure of the markdown.
**Inputs:**
- Raw markdown content.
**Outputs:**
- Document structure (JSON format) detailing the arrangement of text and asset placeholders.
- A list of assets to include in the final PDF document.
**Configuration Parameters:**
- None
### Asset Fetcher
**Responsibilities:**
- Fetches necessary assets.
- Validates the existence and availability of these assets.
**Inputs:**
- List of assets from the Markdown Parser.
**Outputs:**
- Asset files to be included in the final PDF document.
**Configuration Parameters:**
- Asset source directory: where the Asset Fetcher should look for required assets (e.g., `images/`)
### Pandoc PDF Converter
**Responsibilities:**
- Converts the validated markdown content and assets into a final PDF document.
**Inputs:**
- Document structure from Markdown Parser and assets from Asset Fetcher.
**Outputs:**
- Final PDF document
**Configuration Parameters:**
- Filepath of the input Markdown document.
## Data Flows
1. The raw markdown content flows into the Markdown Parser.
2. The Markdown Parser generates a document structure, which flows into the Pandoc PDF Converter.
3. A list of required assets flows from the Markdown Parser to Asset Fetcher.
4. The Asset Fetcher retrieves the required assets, which also flow into the Pandoc PDF Converter.
5. The Pandoc PDF Converter takes in the document structure and required assets and generates a final PDF document.
## Task Dependencies
1. The Markdown Parser must finish validating the markdown content and generating the document structure and list of required assets.
2. The Asset Fetcher must finish retrieving all required assets.
3. Only then can the Pandoc PDF Converter take the document structure and assets and generate the final PDF document.
## Scaling Considerations
As the complexity and size of the markdown documents increase, the system can face scalability issues. In particular, the Pandoc PDF Converter, which requires heavy computing resources, may become a bottleneck. Parallelism, optimized asset fetching, and caching are a few strategies to handle these potential challenges