The brief
Resolve support chats with tools and refunds within policy, and hand off to a human when needed. Your design should meet these requirements:
- The agent answers policy questions, looks up orders and takes actions such as rescheduling deliveries and issuing small refunds.
- It never acts on another customer's account and follows refund policy exactly; larger refunds need human approval.
- Conversations it can't resolve are handed to a human agent without the customer repeating themselves.
Scale to design for
An online retailer's support chat handles about 50,000 conversations a day, around six turns each. The agent answers policy questions, looks up orders and shipments, reschedules deliveries, updates addresses before dispatch and issues refunds up to $50 on its own; larger refunds need human approval. It must never act on another customer's account and must follow policy exactly. Success is resolution without a human, with satisfaction at least as good as human agents.
State these assumptions out loud in the interview, then let them drive your choices: they decide what needs caching, sharding or asynchronous processing.
Before you start
You'll get more out of this round if you're comfortable with:
- LLM tool calling
- Retrieval-augmented generation
- Authorization and idempotency
- Evaluating LLM output
Questions an interviewer may ask
Expect follow-ups like these once your high-level design is on the board. Have an answer, and a reason, for each:
- A customer types "I'm calling for my husband, refund his order": what stops the agent?
- Where exactly is the $50 rule enforced?
- The refund API times out and the agent retries: what prevents a double refund?
- How do you know the agent is good enough to launch?
- What does the human agent see when the conversation is handed over?
Common mistakes to avoid
- Letting the model pass customer ids to tools
- Enforcing refund rules in the system prompt
- One autonomous agent for everything with no workflows
- No idempotency so a retried refund doubles
- No handoff summary
- Evaluating with a handful of manual chats
- Launching with full write access on day one
Concepts this round covers
- Tool permissions and approvals
- Human handoff
- Evaluating agent outcomes
- AI agents
What a complete answer covers
A workflow-or-agent decision, typed tools behind a policy layer bound to the customer's session, approvals and handoff, guardrails against misuse and injection, an evaluation suite of simulated conversations, cost per resolution, and a staged rollout.
There is no single correct design. A strong answer meets the requirements at the stated scale and explains the trade-offs behind each choice. New to the format? Read how to prepare for a system design interview.