Northstar RAG POC
Zero-backend, fully client-side browser RAG with local embeddings and WebGPU LLMs
Episodes & demos
- S01E01 Playhttps://jpeckenpaugh.github.io/rag-poc/ ► Watch (opens in a new tab)
About
Northstar RAG POC is a static single-page application demonstrating retrieval-augmented generation entirely inside the browser with zero backend and zero external API calls. It extracts per-page text from PDFs using PDF.js, splits documents with a sliding-window chunker, generates embeddings via Transformers.js ONNX models, and stores 367 chunk vectors as contiguous Float32Array buffers in JavaScript heap memory for linear dot-product similarity search. Answers are generated by WebLLM with WebGPU acceleration, hot-swapping between Llama-3.2-1B, SmolLM2-1.7B, Qwen2.5-0.5B, and Qwen2.5-1.5B, or by four ONNX embedding models. When WebGPU is unavailable it falls back to a semantic retrieval view showing ranked matches and cosine scores. The demo is preloaded with 21 fictional Northstar Urgent Care Cooperative operational policy PDFs (64 pages) plus a 15-query evaluation benchmark including guardrail abstention cases. Built-in developer tools include a canvas PDF viewer, chunk visualizer, vector math inspector, raw prompt preview, and hardware diagnostics. It is hosted on GitHub Pages as a fully static bundle.
Summary
A 100% client-side Retrieval-Augmented Generation proof-of-concept that parses PDFs, chunks text, generates ONNX embeddings, builds an in-memory vector index, and runs local WebGPU language models entirely in the browser with no backend or external APIs.
Tech
- TypeScript
- React
- Vite
- Tailwind CSS
- Transformers.js
- WebLLM
- PDF.js
- WebGPU
Tags
- browser
- embeddings
- llm
- on-device
- rag
- vector-search