That gap matters once the agent can use tools, change external state, retain information, or continue without a person approving every step. The system is no longer generating an answer. It is interpreting intent, choosing actions, observing incomplete results, and deciding what to do next.

Reliable behavior requires an enforceable contract between what we intend and what the system may actually do.

1. Convert values into decisions

“Protect the user.” “Ask before consequential action.” “Finish the task.”

These are useful values. They do not yet determine behavior.

For each instruction, ask what decision it must change:

  • Which actions count as consequential?
  • What evidence constitutes approval?
  • Does approval cover one action or a sequence?
  • What invalidates it?
  • What happens when an action’s result is unknown?
  • What evidence distinguishes completion from an attempt?
  • When must the system stop?

If an instruction changes no testable decision, it is guidance-not a control.

Required output: a list of situations in which the system must act differently because of the instruction.

2. Write the behavioral contract

For every consequential action, define five things:

  1. Obligation: What must the system do?
  2. Prohibition: What must it not do?
  3. Authority: Who or what may permit the action?
  4. Evidence: What observation confirms the intended effect?
  5. Termination: When must the system finish, wait, hand off, or stop?

Consider an appointment agent. “Be helpful and confirm with the user” is intent. A behavioral contract is operational:

  • The agent may search availability without approval.
  • It may propose a time but may not book without authorization tied to that time and participant set.
  • A submitted request is not a confirmed appointment.
  • Completion requires a provider-issued confirmation.
  • If the result is unknown, the agent must reconcile the first attempt before making another.

The contract need not encode every conversational choice. It must govern the transitions where an error would change the user’s situation or make recovery difficult.

Required output: one reviewable contract for each class of consequential action.

3. Separate judgment from enforcement

Models are useful because they can interpret ambiguous requests and propose actions that were not enumerated in advance. Preserve that flexibility. Move critical enforcement into components with narrower responsibilities.

The model may propose sending an email. The system should verify that the recipient, content, and authorization match.

The model may infer that the user approved a purchase. The system should require an authorization record covering the item, amount, and scope.

The model may report that work is complete. The workflow should check the completion criteria against external evidence.

This is not an argument that deterministic code is infallible. It is an allocation of responsibility: the model interprets; the system governs.

Release bar: no consequential boundary depends only on the model correctly interpreting and enforcing the instruction that defines that boundary.

4. Enforce the contract across the trajectory

Agent behavior unfolds across observations, decisions, tool calls, external effects, retries, and handoffs. A final answer cannot show whether every transition was permitted.

A harmless closing message may follow an impermissible action. A correct action may occur without authorization. A timeout may trigger an unsafe retry. A handoff may be announced without responsibility transferring.

Before each consequential transition, the runtime should be able to answer:

  • Are the required facts present and current?
  • Is this action permitted in the present state?
  • Does the authorization match its parameters?
  • Has the action already been attempted?
  • Is an earlier outcome unresolved?
  • Has a time, cost, or action limit been reached?
  • Must control now pass to a person or another system?

The unit of reliability is the governed trajectory, not the isolated response.

Required output: a state-transition check that can allow, reject, or defer the proposed action for an explicit reason.

5. Make evidence part of correct behavior

A dependable system does not merely take the right action. It leaves enough evidence to establish what happened.

The record must answer:

  1. What happened?
  2. What remains uncertain?
  3. What is safe to do next?

A timeout is not evidence of failure. A generated success message is not evidence of success. If the system cannot distinguish a proposal, an attempt, and a confirmed effect, it cannot retry or recover safely.

Release bar: every claimed completion has an independently checkable completion condition.

6. Turn failures into contract changes

No initial contract will anticipate every relevant condition. Production behavior will expose missing distinctions.

When a failure occurs, do not stop at “improve the prompt.” Ask what the architecture allowed:

  • Was an obligation ambiguous?
  • Was a forbidden transition possible?
  • Was authorization too broad?
  • Did completion lack a verifiable condition?
  • Could the workflow proceed while an outcome remained unresolved?
  • Was there no safe stop or recovery path?

Then change the system: add a scenario, narrow the authority, introduce a state, require evidence, or create a handoff condition. Assign an owner and verify that the original failure now blocks release.

A lesson is complete when it changes future behavior.

Make behavior governable

Prompts express goals, context, and judgment in a form a model can use. They remain necessary.

The surrounding architecture must carry the rest: explicit state, scoped authority, runtime enforcement, verifiable effects, and recovery. That architecture will not eliminate mistakes. It determines whether a mistake can silently become an irreversible outcome.

The goal is not more eloquent intent. It is behavior that can be inspected, constrained, and improved.