Sign inStart practicing
Start practicing Sign in
System design interview question · AI engineering

Design an LLM evaluation platform

Let teams test prompt, model and agent changes against golden datasets before they ship.

Intermediate 45 minutesModel quality Plus
More AI engineering questions

Concepts this round covers

  • Golden datasets and graders
  • LLM-as-a-judge calibration
  • Release gates
  • Model quality

This is a Plus round

The brief, the scale to design for, the questions interviewers ask and the common mistakes for this round are available to Plus members. Plus is coming soon.

Try “ChatGPT” free Already on Plus? Sign in

More AI engineering system design interview questions

See all
Free
AI engineering·Assistants

Design ChatGPT

Serve a chat assistant to millions of people with streamed answers, saved history and fair usage limits.

Advanced 60 min
Free
AI engineering·AI agents

Design a customer support AI agent

Resolve support chats with tools and refunds within policy, and hand off to a human when needed.

Advanced 60 min
Plus
AI engineering·Assistants

Design an AI code assistant

Suggest code as developers type, using context from their repository, in a few hundred milliseconds.

Advanced 60 min
Plus
AI engineering·AI agents

Design a real-time voice AI agent

Answer phone calls with an AI that listens, books appointments and replies without awkward pauses.

Advanced 60 min