A voice agent can produce an impressive answer and still be a poor product. The caller experiences much more than the model’s words: how quickly the agent responds, whether it notices an interruption, what it remembers, which business tools it can use, and what happens when any of those pieces fail.
That makes reliability an architectural property. Real-time speech creates a continuous stream of partial information. The system must decide when to listen, when to act, and when new evidence invalidates an action already in motion. Latency is not a single number; it is a budget distributed across recognition, reasoning, tool calls, synthesis, and the network between them.
Tool use changes the standard from plausible to correct. If an agent books an appointment, updates a customer record, or commits a business to a price, the system needs explicit state, validation, idempotency, and recovery. A fluent sentence cannot substitute for a completed transaction.
Evaluation must therefore follow real workflows. Component metrics remain useful, but the decisive questions are end to end: Did the caller’s intent survive a messy conversation? Did the agent use the right information? Did the downstream system reach the correct state? Could a person understand and repair the failure?
The strongest voice systems treat the model as a capable participant inside a deliberately engineered environment. The model matters. The boundaries around it determine whether that capability becomes dependable work.