01The problem
The client runs a B2B retail-planning platform that turns sales, pricing, inventory and promotion data into demand forecasts and promotion plans. Planners needed to explore results and configure forecasting runs without first learning the whole product.
I own the agent service end to end, and I shaped its requirements directly with the client company’s owner. The goal was never a chatbot for its own sake. It was to make the whole platform feel intuitive.
02How it works
A message arrives over a WebSocket and is handed to a supervisor agent. The supervisor decides what kind of answer is needed and routes the work to the right tool. Anything that changes data passes through a confirmation gate first.
Click any box in the workflow to see its details
Engineering patterns
Idempotent writes
A retry never creates a second copy of the same row.
How it works
- Every event carries a natural key, such as the session and the message number.
- The write uses an insert that does nothing when the key already exists.
- Applying the same event twice leaves exactly one row.
Why it matters
Delivery is at least once, so duplicates will happen. The write path has to absorb them.
The cost
Every event type needs a stable key, and choosing those keys is most of the work.
Where it lives
Atomic forecast switch
A switch either completes fully or leaves the previous forecast in place.
How it works
- The target forecast is prepared off to the side, without touching the active one.
- The change is committed under a lock, so two switches cannot interleave.
- The durable write is shielded from cancellation, and a failed write restores the previous forecast.
Why it matters
A chat that is half-switched would answer from the wrong forecast with no warning.
The cost
Preparing a forecast before committing needs memory for the copy.
Where it lives
Bounded connection setup
A connection either binds within a known time or reports why not.
How it works
- The first frame must carry the sign-in and the session.
- A failed bind retries four times, waiting one, two and four seconds.
- A heartbeat sends a ping every thirty seconds and drops a peer that does not answer within fifteen.
Why it matters
Half-open connections are common on mobile networks. Without a heartbeat, a dead socket looks alive for minutes.
The cost
Heartbeats add a little traffic on every open chat.
Where it lives
Turn lock and message dedupe
One turn per chat at a time, and repeated messages are dropped.
How it works
- A turn takes a lock for its session, valid for ninety seconds.
- The lock is refreshed every thirty seconds while the turn runs.
- Each client message ID is remembered for ten minutes, so a repeat is dropped.
Why it matters
If a worker dies, its lock expires within ninety seconds, so the chat recovers without manual help.
The cost
A slow turn must keep refreshing its lock, or another turn may start.
Where it lives
Turns that finish on their own
A turn is not tied to the connection that started it.
How it works
- The turn runs as its own background task.
- Its events go to the outbox, not to the open socket.
- The answer is saved even if no one is listening.
Why it matters
Users close tabs and lose networks. The work should not be lost with them.
The cost
A reconnecting client has to read the saved answer instead of catching the live stream.
Where it lives
Ordered, deferred event processing
Events are applied in an order that keeps the data consistent.
How it works
- The worker takes a per-session lock before it applies anything.
- A child event waits for its parent session row to exist.
- Each file gets up to three attempts with growing waits, then is set aside.
Why it matters
Without ordering, a message could be stored before the chat it belongs to.
The cost
A stuck parent holds back its children until someone looks at it.
Where it lives
Resume from durable state
A reopened chat rebuilds from what was saved, not from memory.
How it works
- One owner-scoped query returns the session, contexts, last messages and export status.
- Counters start from the highest saved value, so a new server never reuses a number.
- Live state is taken from memory first, then the cache, then the database.
Why it matters
A user can reopen a chat on any server and see the same state.
The cost
Rebuilding costs one read on every reopen.
Where it lives
Owner and tenant scoping
Every read and write is checked against the owner of the data.
How it works
- Context access checks the owner before it returns anything.
- Stored paths include the tenant and the forecast.
- A read by someone who does not own the record returns not-found.
Why it matters
Several clients and users share one system, and their data must not mix.
The cost
Every query carries the owner, which adds some work to each one.
Where it lives
Preflight checks and human approval
Nothing is written until the values are checked and a person says yes.
How it works
- A preflight check runs when a form is opened, and can stop the flow with a plain reason.
- Each write is shown with its exact values and waits for a confirm.
- A half-finished create records its pending ID, so a retry only looks up the result.
Why it matters
A model can suggest a write, but a person decides it.
The cost
Every write costs one extra step for the user.
Where it lives
Grounded answers
An answer is checked against its context before it is given.
How it works
- Retrieved context is graded for sufficiency before it is used.
- When the context is thin, the system asks a clarifying question.
- When no source supports an answer, the system says so instead of guessing.
Why it matters
A confident wrong answer is worse than an honest "I cannot tell".
The cost
Some questions get a follow-up instead of an immediate answer.
Where it lives
Bounded loops
Every loop has a fixed limit, so none can run away.
How it works
- The setup agent has six tool rounds.
- The analysis engine has twenty rounds, and warns the model three rounds before the limit.
- Query repair has five attempts, and retrieval has two rounds of three concepts.
Why it matters
A model that keeps trying costs time and money with no guarantee of an answer.
The cost
A hard question can stop before it is solved.
Where it lives
Cost and token accounting
Every model call and every query is measured.
How it works
- Each model call records its token use and its cost.
- Totals are kept per request and per session.
- Query cost is taken from the bytes billed.
Why it matters
A cost you cannot see is a cost you cannot control.
The cost
Accounting adds a small amount of work to every call.
Where it lives
Readiness before analysis
Analysis starts only when its data is ready.
How it works
- Each table has a status: loading, ready, or failed.
- The analysis step waits with a time budget.
- Unfinished exports are retried after an attach, a switch, or a reconnect.
Why it matters
Reading a partial export gives a wrong answer that looks right.
The cost
A large forecast can make the first analysis wait.
Where it lives
Timeouts and safe retries
Each call has its own timeout, and retries happen only when they are safe.
How it works
- Reads have a short timeout and retry on server errors.
- Writes retry only when the request never reached the server.
- Errors are sorted into denied, retryable and terminal.
Why it matters
Retrying a write that may have succeeded can create a duplicate.
The cost
Some failures reach the user, who then has to try again.
Where it lives
03One supervisor, three kinds of answers
Planner questions fall into three families, and one prompt doing all of them is hard to control. The supervisor routes between them instead:
- SQL on BigQuery for questions the data can answer directly.
- A sandboxed code environment on Vertex AI Agent Engine for multi-step, programmatic analysis.
- Grounded product knowledge for how the platform works, with step-by-step guidance, citations and clarifying questions.
Session state is reused between turns, so iterative analysis builds on earlier results instead of starting over.
04Keeping projects apart
One conversation can move between several forecasts. That is exactly where context leaks. State is isolated per project, and access is scoped by tenant and user. I check context attribution and cross-project leakage with a 19-case live evaluation harness.
05Writes need a yes
Client requirements became a guided chat workflow for project setup and forecast launch. Inputs are validated live, every write waits for explicit confirmation, retries are safe, and duplicate launches are prevented.
06Staying up when connections don’t
Chat execution is non-blocking and resumable across server instances, so concurrent conversations stay responsive. History stays recoverable through cancellation, disconnects, retries and dead-lettering.