RAG Data Synthesizer
Build realistic fictional document corpora for RAG development and testing
About
RAG Data Synthesizer is a workflow and toolset for building realistic fictional document collections for retrieval-augmented generation development and evaluation. It supports collection planning, staff-authored source documents, fact and relationship tracking, visual assets, PDF rendering, and release review. The repository does not automatically generate document prose; a coordinator uses bounded author and reviewer agents to draft and reconcile content, while scripts generate scenario visuals and render approved Markdown to searchable PDFs. Its first scenario is the Northstar Urgent Care Cooperative, a fictional three-site clinic network, with 21 approved Markdown documents, rendered PDFs totaling 64 pages, a staff registry, corpus plan, shared-fact ledger, visual assets, a generated PDF index, and rendering provenance. The Python tools are scenario-neutral enough to render another Markdown collection and generate configured visuals; a second scenario, validated evaluation set, full ingestion manifest, and automated corpus-wide validator remain future work. All organizations, people, policies, events, and records are fictional and intended as educational test material.
Summary
An agentic workflow and Python toolset to draft, reconcile, visualize, and render realistic fictional document collections for developing and evaluating RAG pipelines.
Tech
- Python
- WeasyPrint
- mistune
- PyYAML
- pypdf
- Pillow
Tags
- data-generation
- documents
- evaluation
- pdf-rendering
- rag
- synthetic-data