/001/
Overview
Finding the right passage inside large document sets is slow, and keyword search misses meaning.
My role
Backend and AI pipeline development with a Next.js interface.
/002/
Stack
- NestJS
- Next.js
- PostgreSQL + pgvector
- Embeddings
- RAG
- BullMQ
/003/
Architecture
NestJS services, PostgreSQL with pgvector for similarity search, an embedding and RAG pipeline, and BullMQ workers for document ingestion.
/004/
Key features
- /001/Document ingestion
- /002/Chunked embeddings stored in pgvector
- /003/AI-assisted retrieval
- /004/RAG pipeline over private documents
/005/
The hard problem
The hard problem
Ingesting large documents would block the API and time out uploads.
Solution
Ingestion moved to BullMQ background workers, so uploads stay responsive while chunks are embedded and indexed.