Impact-Site-Verification: 578d421e-1081-463d-918b-ec5e29c5b9db
<Back

Major Life Insurance Group's IT Subsidiary: Technical Coaching and Development for In-House RAG Apps

Global Consulting Firm

Context and Problem

The client was a major life insurance group's IT subsidiary. The goal was for its own engineers to build and modify RAG apps themselves, bringing RAG development in-house. About 10 client engineers took part.

Rather than stopping at lectures, the work was done by building alongside the client's engineers: first a RAG app written from scratch in Python and JavaScript, then in-house development of RAG apps built on Azure's best practices.

Lectures and Teaching Material

Taught Python, Azure, and web technologies systematically and introduced agile development. For the first stage, wrote a complete RAG app and a four-chapter textbook. The textbook code is in fill-in-the-blank form so participants write it themselves (350 blanks, counted in the textbook Markdown).

First-Stage Teaching App (Built Myself)

  1. Provisioning: ten numbered Azure CLI scripts create the resource group, Azure OpenAI and model deployments, storage, Azure AI Search, App Service, and Document Intelligence in order, plus a script that pushes .env values into App Service settings.
  2. Ingestion: pypdf extracts text from PDFs in Blob Storage; the text is split into 1,000-character chunks with a 200-character overlap, embedded, and uploaded with IDs derived from the file name and chunk number. A chunk size not larger than the overlap raises an error, so a bad setting cannot split forever.
  3. Index: title, content, and a vector field searched with HNSW, plus a semantic configuration that prioritizes title and content.
  4. Retrieval and answer: keyword and vector search combined for the top 3 chunks, switching to semantic ranking only when the index has a semantic configuration. The system prompt restricts answers to the supplied context.
  5. App: a Flask JSON API (/api/chat) and a page built from a Jinja2 template and plain JavaScript, served by gunicorn on App Service.
  6. Extra experiment: converting handwritten PDFs to Markdown with Document Intelligence prebuilt-read, with an explainer document.

The System in the Client Environment

Built a new system by extending Azure's RAG sample (Azure-Samples/azure-search-openai-demo) and deployed it into the client's environment in a private network that met the security requirements. The sample it starts from provides a React frontend with a Python (Quart) backend, retrieval that combines keyword, vector, and semantic ranking, and document ingestion with Document Intelligence.

At the client's request to improve operations, added a logging platform with OpenTelemetry and Log Analytics.

Outcome

RAG app development was brought in-house, and the system was used by all of the client's engineers.

Technologies

PythonJavaScriptFlaskgunicornpypdfAzure CLIAzure App ServiceAzure AI SearchAzure OpenAI ServiceAzure AI Document IntelligenceAzure Blob StorageAzure Cosmos DBLog AnalyticsOpenTelemetryRAGAgile