TL;DR

AI agent verification should improve the next decision, not simply produce more tool calls. Select diagnostics that distinguish plausible explanations, enforce access and resource limits outside the model, and stop when further investigation is not justified. Keep evidence, approval, execution, and service recovery as separate states. A supported diagnosis can justify a change proposal, but it cannot grant permission to execute it. That permission must remain bound to the exact action and valid when execution occurs.

Introduction

An AI assistant investigating a checkout outage has identified several plausible explanations. Instead of choosing between them, it queries another dashboard, repeats a health check, searches previous incidents, and asks a second model to review its answer.

The investigation is now longer. The decision is not necessarily better.

Part 1, AI Confidence Is Not Evidence: Building an Evidence Contract, established how to qualify observations before using them. External checkout requests were timing out, node-health observations were normal, and a gateway configuration had recently changed. Those facts narrowed the investigation without establishing a root cause.

This article continues that illustrative scenario. The assistant has scoped diagnostic access, production changes require human approval, and ordinary incident processing does not update model weights. The workflow and policy below are proposed engineering patterns, not a validated implementation.

The next question is practical: What should the agent verify, when should it stop, and what must remain outside its authority even after the evidence improves?

Ask a Question That Could Change the Response

A useful diagnostic tests a distinction between explanations. It does not merely collect another presentation of information already available.

For the checkout incident, an approved internal application check could help distinguish a backend problem from a problem affecting the external request path. Comparing the deployed gateway route with its approved configuration could test a different hypothesis. Repeating the unchanged node-health summary may add little to either question.

Before a call is admitted, record the question, expected result classes, and how each result would affect the next response. Include an inconclusive result rather than forcing every tool response into success or failure.

Keep comparisons meaningful. The internal and external checks should exercise comparable application behavior, with differences documented. A successful internal liveness endpoint cannot establish that the externally exposed checkout operation works.

This turns the diagnostic plan into something an operator can review before the results arrive, rather than a justification reconstructed afterward.

Distinguish Information Gain from Decision Value

Poole and Mackworth’s Artificial Intelligence: Foundations of Computational Agents describes information value through its effect on decisions. Learning more is not automatically worth the cost of obtaining that information.

For a simplified, non-disruptive diagnostic, express the net value as:$$V(q)=R(E)-mathbb{E}_{zmid E,q}[R(E,z)]-C(q)$$

Here, E is the current evidence, q is a candidate diagnostic, and z is a possible result. R is the minimum expected loss across the permitted responses. C(q) is the diagnostic’s cost, including delay, expressed on a compatible scale.

This is a one-step decision model, not a production scoring algorithm. It assumes a defined outcome model and response set, here proposals and escalations rather than independently executed production changes. It does not capture every consequence of a diagnostic that materially changes the system.

For this workflow, apply access and safety constraints first. An unauthorized query is not eligible simply because it might be informative. When defensible probabilities or loss estimates are unavailable, use reviewed diagnostic playbooks rather than asking the model to invent precise scores.

The operational test is simpler: would a plausible result change the recommendation, escalation destination, or urgency? When it would not, another query may consume the time needed for recovery.

This also preserves the foundation article’s terminology boundary. Selecting informative tools is not, by itself, an implementation of formal active inference.

Build AI Agent Verification Outside the Model

The model can propose a diagnostic and explain its purpose. The surrounding runtime should decide whether the operation is permitted, execute it through a constrained adapter, and qualify the returned observation.

The important feature in this diagram is the return through admission. An earlier authorization does not create unrestricted permission for every subsequent query.

OWASP’s AI Agent Security Cheat Sheet recommends separating decision-making from execution and independently validating consequential actions. Here, that separation also provides a place to enforce diagnostic boundaries without depending on the model’s willingness to follow instructions.

Preserve a workflow identifier across the planner, adapters, evidence records, and escalation. Record actual tool parameters and collection results. The agent’s narrative can explain the assessment, but it should not be the sole record of what happened.

If a tool response contains instructions to change policy or call a different tool, treat those instructions as untrusted content. A diagnostic result can inform the assessment; it cannot modify the authorization rules governing the investigation.

Give the Workflow a Runtime-Enforced Budget

The following YAML describes an illustrative contract for the diagnostic phase. It is not configuration accepted by a named framework. Tool names, resource registries, budget profiles, and escalation destinations must be implemented in your environment.

The contract deliberately excludes production write tools. Its successful output is either a supported proposal or an accurate escalation, not an automatically executed repair.

policy_version: "1.0"
workflow: checkout_path_verification
mode: diagnose_and_propose

scope:
  tenant_id: tenant-a
  environment: production
  target_set: approved-checkout-diagnostics
  arbitrary_destinations: deny

allowed_tools:
  - monitoring.read_probe_result
  - application.run_approved_health_check
  - gateway.read_deployed_config
  - change.read_approved_revision

budget:
  accounting_scope: root_workflow
  max_tool_attempts: 4
  max_parallel_calls: 1
  workflow_deadline_seconds: 30
  per_call_timeout_seconds: 8
  automatic_retries: 0
  model_budget_profile: checkout-triage-small

results:
  require_evidence_contract: true
  incomplete: preserve_unknown

stop_behavior:
  budget_exhausted: escalate
  policy_unavailable: deny_and_escalate
  audit_unavailable: deny_and_escalate

execution:
  production_write_tools: unavailable
  proposal_destination: controlled_change_review

The numerical limits are placeholders, not recommended defaults for every incident. The named model-budget profile should define enforceable token and spending limits; it is not permission to leave model consumption unbounded.

Configure target_set through an access-controlled registry. The adapter should resolve registered resources and approved request profiles, rather than accepting an arbitrary destination or shell command supplied by the model.

Count the Work That Actually Happens

Charge attempts to the root workflow before dispatch, including retries and delegated work. Preserve counters and the original deadline across worker restarts. A new child agent should not receive a fresh budget for the same investigation.

The effective timeout for a call must not exceed the remaining workflow deadline. Four calls with eight-second limits are not guaranteed to fit into a thirty-second workflow, particularly when model processing and queueing also consume time.

AWS’s Control and limit retry calls guidance recommends bounded retries and selecting an appropriate retry layer. For this example, disable or account for automatic retries inside adapters and SDKs. Otherwise, four visible tool calls could generate more than four downstream attempts.

Track outstanding downstream requests: a client-side timeout does not prove they have stopped. When the budget expires, preserve completed evidence and stop dispatching new diagnostics. An unresolved cause is a valid result. Silently increasing the budget until the model reaches a conclusion is not.

Treat Diagnostic Access as Production Access

A diagnostic is still an action. The title’s distinction is between gathering qualified evidence and making consequential changes, not between dangerous writes and supposedly harmless reads.

NIST’s Zero Trust Architecture rejects implicit trust based solely on network location or ownership. Apply that principle to the assistant’s resource access: running inside a management network should not grant access to every service, tenant, or diagnostic endpoint.

For the checkout pilot, use approved synthetic profiles that do not create real orders, charge payment instruments, or modify customer records. A test that performs those operations belongs in a separately reviewed action class. Calling it a health check does not change its effects.

Also bound query scope and returned data. A configuration inspection should retrieve the relevant route, not export every tenant’s configuration. An application check should use a reviewed request profile rather than allowing the model to invent increasingly expensive tests.

Add service-wide protection alongside per-workflow limits. Ten individually bounded investigations can still overload the same dependency if they run together. In this proposed design, the platform owner sets aggregate concurrency and request limits for each diagnostic service.

If investigation cannot continue within those controls, escalate to the established incident process. Urgency should invoke a predefined emergency procedure, not allow the agent to grant itself broader access.

Bind Approval to the Change That Will Actually Execute

Suppose the diagnostics reveal that the deployed checkout route differs from the approved revision, while the internal application check succeeds. The agent now has support for a targeted rollback proposal. It still has not proved that every alternative explanation is wrong.

The reviewer needs the actual change, not a prompt that says “Approve the recommended fix?” For this scenario, the review packet should identify the tenant, environment, gateway route, current revision, proposed revision, expected impact, recovery procedure, and evidence supporting the proposal. Preserve unresolved risks in the same packet.

OWASP’s agent guidance calls for approval to be bound to the actor, tool, target, normalized parameters, and validity period. Its Transaction Authorization Cheat Sheet also recommends server-side enforcement and a final authorization check tied to execution.

Build the approval display and executable request from the same validated action object. The executor should obtain approval state from the trusted approval service, not accept an approved: true field produced by the model. A digest can identify an action, but it is not authorization on its own.

Revalidate Policy and Resource State

For this design, retain the policy version under which approval was granted and reevaluate current policy before dispatch. An approval created before a permission was revoked must not silently override that revocation. Changed action parameters require renewed approval.

Resource state needs a separate check. If the route changes while the proposal waits for review, the approved rollback may no longer apply to the state that will actually be modified.

RFC 9110 defines If-Match for conditional HTTP requests. Where the API supports it, an exact strong entity tag can prevent applying a write to a changed representation. An equivalent platform-specific revision precondition can serve the same purpose. An existence-only wildcard does not verify the reviewed revision.

Do not respond to a revision conflict by silently fetching the latest state and reusing the old approval. Reassess the proposed change against that state. A single-resource precondition also does not validate every dependency in a multi-resource operation.

Evidence can strengthen the proposal. It cannot expand the approved action.

A Timeout Leaves an Outcome to Reconcile

An executor submits the approved rollback, then loses the response. The agent should not translate that event into “rollback failed” and submit another change with a new identifier.

AWS’s Making retries safe with idempotent APIs describes this distributed-systems ambiguity: the operation may have completed even though the caller received no response. Client request identifiers and server-side idempotency can make retries safer when supported by the API contract.

For this workflow, maintain a durable action record and preserve the logical action identifier. Reconcile the downstream operation status and actual target state before deciding what comes next. A retry must not become an additional logical change simply because the first response was lost.

The distinction matters in the execution record:

Do not claim exactly-once effects merely because the orchestrator stores a unique identifier. The downstream system must enforce the relevant semantics. Where it cannot, an ambiguous write may require operator reconciliation rather than automatic resubmission.

Approval replay protection and safe retries solve different problems. The approval authorizes one logical operation; reconciliation or an allowed retry must remain associated with that operation, not manufacture authorization for another.

Validate Service Recovery, Not Just API Success

In the checkout example, a successful configuration response establishes that the API accepted or completed the operation according to its contract. It does not independently establish that customers can complete checkout.

Verify the service from the affected vantage point using an approved request profile. Compare relevant error rates and latency against the service owner’s recovery criteria, allowing for the configuration’s expected propagation behavior. Check for harm to neighboring routes or dependencies within the approved observation scope.

Keep the execution receipt and outcome evidence as separate records. The first answers what the executor did. The second answers what happened to the service afterward.

An unsuccessful recovery should not trigger an open-ended sequence of new repairs. Follow the approved fallback or return to incident command with the execution result and remaining uncertainty. Even a rollback of the attempted repair needs its own defined conditions and authority.

Test the Boundaries Before Increasing Autonomy

Begin with historical replay and a shadow workflow beside the existing operator process. Replay must expose only evidence available at the original decision point, not the final root-cause report. Shadow mode should create no production change authority.

The most useful tests exercise transitions that a successful demonstration can hide.

Measure whether diagnostics improve reviewed recommendations, how long qualified escalation takes, and how often the workflow respects its limits. Lower tool-call counts are not automatically better, just as higher call counts do not prove diligence. Compare decision quality and investigation cost together.

Assign ownership before the pilot. The service owner defines diagnostic meaning and recovery criteria. Platform operations owns adapters, deadlines, and execution records. Security owns authorization boundaries. The AI application owner evaluates question selection and evidence use. Incident command owns escalation and emergency handling.

Record these decisions through actual runtime events, not only a generated explanation. Retest after material changes to models, prompts, tools, policies, or retry behavior, and preserve a manual response path when the agent or its controls are unavailable.

Conclusion

AI agent verification should be a bounded decision process. Start with a question that can change the response, collect evidence through controlled tools, and stop when further investigation is not justified or permitted.

Then preserve the production boundary. A supported recommendation is not an authorization. An authorization is not an execution result. An execution result is not proof of service recovery.

Apply the pattern to one existing operational workflow. Define its diagnostic questions, aggregate budget, exact approval object, execution-time checks, and handling of unknown outcomes. Test those boundaries before allowing a more capable model to operate within them.

The objective is not an agent that always finds something else to check. It is a workflow that knows what another check could change, what it is allowed to do, and when responsibility must pass to an operator.

Part 3, AI Feedback Is Not Learning: Governing Memory and Model Updates, follows the outcome into the feedback pipeline and asks which lessons deserve to change future behavior.

External References

• NIST: Zero Trust Architecture
  Canonical URL: https://csrc.nist.gov/pubs/sp/800/207/final
• OWASP: AI Agent Security Cheat Sheet
  Canonical URL: https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html
• OWASP: Transaction Authorization Cheat Sheet
  Canonical URL: https://cheatsheetseries.owasp.org/cheatsheets/Transaction_Authorization_Cheat_Sheet.html
• RFC Editor: RFC 9110: HTTP Semantics
  Canonical URL: https://www.rfc-editor.org/rfc/rfc9110.html
• AWS: REL05-BP03 Control and limit retry calls
  Canonical URL: https://docs.aws.amazon.com/wellarchitected/latest/framework/rel_mitigate_interaction_failure_limit_retries.html
• AWS: Making retries safe with idempotent APIs
  Canonical URL: https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/
• David L. Poole and Alan K. Mackworth: 12.4 The Value of Information and Control
  Canonical URL: https://artint.info/3e/html/ArtInt3e.Ch12.S4.html

The post AI Agents Should Verify Before They Act appeared first on Digital Thought Disruption.