AI UGC Video Creator

An AI creative director that turns product URLs into viral short-form UGC videos through an interactive, multi-turn chat experience. Built with Next.js, FastAPI, FFmpeg, Jina Reader, and Gemini 2.0 Flash.

Architecture

graph TD
    User((User))
    Frontend[Next.js Frontend\nChatWindow & SSE]
    Backend[FastAPI Backend\nIntent Router & Pipeline]
    Jina[Jina Reader API\nWeb Scraping]
    Gemini[Gemini 2.0 Flash\nLLM Engine]
    FFmpeg[FFmpeg\nVideo Composition]
    Cloudinary[Cloudinary CDN\nVideo Hosting]

    User -->|Provides URL or Feedback| Frontend
    Frontend -->|SSE Stream Request| Backend

    Backend -->|1. Scrape Website| Jina
    Jina -->|Markdown Text| Backend
    
    Backend -->|2. Generate Profile & Concepts| Gemini
    Gemini -->|Structured JSON| Backend
    
    Backend -->|3. Overlay Caption on GIF| FFmpeg
    FFmpeg -->|Rendered MP4 Video| Backend

    Backend -->|4. Upload Final Video| Cloudinary
    Cloudinary -->|Secure CDN URL| Backend

    Backend -.->|Yield Progress Events| Frontend
    Backend -.->|Yield Final Result URL| Frontend
    Frontend -.->|Live UI Updates| User

Loading

• Frontend: Next.js App Router, Tailwind CSS, shadcn/ui. Handles real-time SSE streaming for live progress states and rich chat UI.
• Backend: FastAPI & Pydantic. Implements an Intent Router to handle conversational state, Jina Reader for website scraping, Gemini for structuring brand profiles and generating concepts, and ffmpeg-python for dynamic video composition.

Local Setup

1. Environment Variables

In the backend folder, create a .env file:

JINA_API_KEY=your_key
GEMINI_API_KEY=your_key
GROQ_API_KEY=your_key
CLOUDINARY_URL=cloudinary://API_KEY:API_SECRET@CLOUD_NAME

2. Backend

Ensure you have ffmpeg installed on your system path.

cd backend
python -m venv venv
source venv/bin/activate  # or .\venv\Scripts\activate on Windows
pip install -r requirements.txt
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

3. Frontend

cd frontend
npm install
npm run dev

Open http://localhost:3000.

Adding Assets

To add new video meme templates, simply drop an .mp4 file (vertical orientation) into backend/assets/gifs/ and update backend/assets/gifs/metadata.json with a description and emotional tags. The AI automatically parses these tags to match generated concepts to the right template video.

{
  "new_template.mp4": {
    "tags": ["happy", "celebration", "wow"],
    "description": "Guy celebrating"
  }
}

Deployment

Backend (Northflank / Koyeb / Render)

The backend requires ffmpeg. Use the included Dockerfile to deploy the backend to any container-hosting platform. A render.yaml template is provided for a quick 1-click deploy.

Frontend (Vercel)

Deploy the frontend folder directly to Vercel. Ensure you set the environment variable:

NEXT_PUBLIC_API_URL=https://your-backend-url.com

---