Tool-using agent that books field-service visits, with guardrails and an eval set

ClearFlow books about 3,000 repair and service visits a month, split between its UK and US businesses. Customers phone or message, and a team of 9 coordinators checks the engineer calendar, finds a slot, confirms the address and books. Evenings and weekends are not covered, and about 18% of messages sent after 6pm never turn into a booking.

The operations director wants an assistant that can do the booking itself by calling the company's scheduling API, not just chat about it. She also listed what must never happen: - booking a slot that does not exist or is already taken; - cancelling or moving someone else's appointment; - promising a price, because prices depend on the engineer's diagnosis; - handling a gas-leak or carbon-monoxide report as a normal booking instead of giving the emergency number at once.

You will build a mock scheduling API (availability, book, reschedule, cancel, look up customer), seeded with 40 engineers, their skills and postcodes or ZIP codes, and two weeks of slots. You will then build an agent with an LLM that supports tool calling, plus guardrails in code (not only in the prompt) that check every action before it runs. Finally you will write an evaluation set of 120 scripted conversations, run each one with a simulated customer, and score task success, policy violations, number of turns and cost.

The go-live bar: at least 85% task success, zero critical policy violations across the 120 conversations, and every emergency handled correctly. Done looks like the API, the agent, the evaluation suite and its results, and a short safety case for the operations director.