Language · 2025
VNASelf, a multi-agent assistant
A supervisor agent routes requests to specialist agents for calendar automation, OCR-based task capture and document questions, behind a real-time chat UI.
Supervisor + 3 specialist agents on LangGraph
- Problem
- Personal admin is scattered across a calendar, photos of receipts and notes, and documents you cannot search. One chat window should be able to handle all three without a single monolithic prompt.
- Approach
- A LangGraph supervisor agent classifies each message and routes it to a specialist: Google Calendar automation, OCR-based task management (PaddleOCR / VietOCR for Vietnamese text), or a cloud-backed RAG agent over the user's documents on NeonDB + Supabase. FastAPI backend with WebSocket chat and per-thread history; React + Vite frontend.
- Result
- Working end-to-end assistant with real-time chat, chat-history threads and three specialist agents, containerised with Docker. [Add one measured number: e.g. routing accuracy on N test prompts, or OCR field accuracy on Vietnamese receipts.]
Problem
The usual single-prompt chatbot breaks as soon as a request needs a tool: creating an event, reading a receipt photo, or answering from a document it has never seen. The goal was a small system where each capability is its own agent with its own tools and prompt, and a supervisor decides who handles what, so adding a capability does not mean rewriting one giant prompt.
Architecture
React + Vite chat. Real-time chat over WebSocket, thread history, light and dark theme. Talks to the FastAPI backend on /api/chat and /api/chat/history/{thread_id}.
FastAPI backend. Chat endpoint, history store, CORS for the frontend, Swagger at /docs. Containerised with Docker.
Supervisor agent (LangGraph). Classifies the request and routes it to one specialist; conversation state lives in the LangGraph thread so follow-ups keep context.
Calendar agent. Google Calendar automation: create, move and list events from natural-language requests.
OCR task agent. PaddleOCR / VietOCR read photos of receipts and notes (Vietnamese and English) and turn them into tasks.
RAG agent. Documents are chunked and embedded into NeonDB (Postgres) with files on Supabase; answers cite the retrieved passages.
What I would show a reviewer
- [Screenshot or short clip of the chat routing one message to the calendar agent and the next to the OCR agent.]
- [An example receipt photo, the OCR output, and the task that was created from it.]
- [A document question with the retrieved passage shown next to the answer.]
Challenges and lessons
- Routing is the product. Most failures were the supervisor picking the wrong specialist, not the specialists failing. [Describe how you evaluated routing and what you changed in the supervisor prompt or state schema.]
- Vietnamese OCR. PaddleOCR handles layout, VietOCR handles diacritics better on handwriting-like fonts; [note which one won on your receipts and why you kept both].
- State across agents. Keeping one LangGraph thread per conversation meant the RAG agent could see what the calendar agent had just done. [Mention any limitation you hit.]
Links
Code: github.com/x23d8/X23D8-muilti-agents (MIT). The README there is short; this page is the fuller write-up.