Impact-Site-Verification: 578d421e-1081-463d-918b-ec5e29c5b9db
<Back

AI Scheduling Agent

Commissioned Work | Forward Deployed Engineer

Context and Problem

Scheduling depends on unwritten personal criteria such as "keep lunch free" or "this kind of meeting in the morning". Worked on a PoC that tests whether an AI agent can learn these implicit criteria through dialogue and improve its proposals: wrote the test plan and built the implementation alone.

Breaking Down the Question and Designing the Test

  1. Decomposition: split the question into acquiring implicit knowledge (can dialogue draw it out), storing it (can it be structured and kept), and using it (does stored knowledge improve proposals), with one decision criterion per part.
  2. Order of testing: test in reverse, starting from "use", with human-prepared ideal knowledge so one variable changes at a time, fixing the best method at each stage before moving on.
  3. Choice of method: the plan compared two retrieval methods and three storage methods, and implementation started with key lookup and structured JSON storage.
  4. Evaluation: a short PoC cannot reach statistical significance, so results are checked by user review and by comparing the same scenario before and after dialogue.

How It Works

A 12-node LangGraph graph that separates six LLM steps from rule-based steps, memory reads and writes, and user confirmation. To the four roles in the initial design, added request parsing and a step that judges approval or rejection of a proposal.

  1. Parse the request and estimate priority (LLM) → compute free time for all participants (rules: holidays excluded, business hours, 30-minute slots) → load memory → preference rules → propose candidates (LLM).
  2. Confirmation: interrupt() shows the candidates to the user. Approval ends the run; on rejection or no candidates, the agent picks one to three questions (LLM), structures the answer into memory, recomputes free time, and proposes again.
  3. The rejection reason goes to the next proposal and to question selection. Dates that the LLM extracts from the rejection reason are added to the excluded dates in code, and those slots are removed by rule.
  4. Memory: LangGraph's Store holds each user's preferences (one document updated by deep merge) and an append-only Q&A history. Memory is read for all participants but written only for the requester, and several people's conditions are combined by rules such as "the strictest daily meeting cap wins" and "lunch is protected if anyone protects it".

Technical Decisions

  1. Rules in code: left to the LLM, rules such as lunch protection and meeting caps were followed inconsistently, so eight rules are applied deterministically before the LLM call, and the LLM focuses on drawing conditions out of the user.
  2. Questions and relaxation: preferences are elicited step by step from 24 question templates in 6 categories; when no candidate exists, conditions relax in order: use lunch, move events, drop participants, shorten, extend the deadline.
  3. Input hardening: every value placed in a prompt is XML-escaped, input is capped at 10,000 characters, and user IDs are checked by pattern and resolved path to prevent path traversal.
  4. LLM creation goes through one factory so a model change cannot be missed in a single step, and every LLM output is received through a Pydantic schema.

Verification and Problems Found

  1. Tests (counted): 289 test functions in 33 files and 132 scenario definitions in 13 groups (a static count of the repository, not a run result). Scenario tests run the real graph with only the calendars replaced, and a checker inspects API output for preference violations (lunch, full-day events, time of day).
  2. Observed: checked against the prepared verification scenarios, the first version passed only the basic flow. The biggest cause was not recomputing free time after learning, so the fix added a recompute step, moving events, per-participant slots, and deadline and duration relaxation.
  3. Observed: during trial use, "exclude this date" was ignored because the structured request schema had no field for it and the condition was lost at the entrance. The fix added the field, skips those slots by rule, and adds dates the LLM extracts from a rejection reason to the excluded dates in code.
  4. To make trial use easy, built a FastAPI API and a Next.js screen, packaged together with Docker for hand-off.

Technologies

PythonLangGraphLangChainPydanticFastAPINext.jsReactTypeScriptFullCalendarZustandTailwind CSSpytestDocker