3

AI-Powered Regulatory Whitepaper Assistant — European NGO & Crypto Alliance

RAG for regulatory whitepapers: upload PDFs → parse, chunk, embed → OpenSearch (user isolation), S3; per-field generation with guidelines; Auth0; multi-type whitepapers; FastAPI, React, AWS.

Secure, production-ready application for a client in the regulated financial sector (European scientific foundation and crypto industry alliance). Teams prepare structured regulatory disclosures (whitepapers) using RAG, with strict user isolation and integration with official registries.

Business Purpose

  • Automate first drafts from uploaded source documents (e.g. PDFs) so experts focus on review and refinement.
  • Enforce consistency with field-level guidelines and reporting standards.
  • Integrate official data via lookups (e.g. LEI and token/asset classification registries) so users get validated references without leaving the tool.
  • Single, auditable workflow: upload → generate → review → edit; support for multiple whitepaper types and follow-up refinement.

Technical Pipeline

  • Ingest: Users upload PDFs; documents are parsed (Docling), chunked with a token-aware strategy (TikToken), and embedded (OpenAI).
  • Store: Text chunks and embeddings in a vector-capable search engine (AWS OpenSearch) with strict user isolation—every query and document scoped to the authenticated user. Files in object storage (AWS S3) under user-specific paths.
  • Generate: For each form field, the system retrieves relevant chunks, combines them with field-specific guidelines and prompts, and calls an LLM to produce suggested content. Generation runs asynchronously with status polling and cancellation.
  • Multi-type support: Backend routes to different schemas and guidelines depending on whitepaper type (e.g. asset-referenced, e-money, or other token classifications), using SQLite and JSON configs for structure and prompts.
  • Security: Auth0 (JWT, RS256); API validates tokens on every request; React SPA with protected routes and secure API calls.

Challenges

  • Document complexity: Parsing and chunking long PDFs so the right context reaches each field.
  • Regulatory alignment: Structured prompts, guideline injection, evaluation metrics (relevancy, faithfulness, hallucination).
  • User isolation and migration: Tenant isolation at S3, OpenSearch, and app DB; safe migration to user-scoped model.
  • Latency and UX: Async jobs, progress reporting, cancellation.
  • External registries: Clear API boundaries, error handling, optional caching.

Outcome

Teams draft regulatory whitepapers faster with more consistent adherence to guidelines. Manual lookups reduced; reviewers spend less time on initial drafting and more on validation and refinement. Workflow scalable (async, stateless backend, user-scoped data) and auditable (logs, user isolation, clear progression). Production-ready security and ops: Auth0 SSO, JWT validation, HTTPS, rate limiting, security headers, structured logging, health endpoints.

Stack: React, React Router, Auth0 React SDK, Tailwind, React-PDF; FastAPI, PyJWT, SQLite, Pydantic; OpenAI, AWS OpenSearch; Docling, TikToken; Docker, Nginx, AWS (S3, OpenSearch).