Agent Risk Assessment

Definition

An agent risk assessment is a documented examination of the harms a specific agent workflow could cause, the conditions that make them possible, and the controls and evidence supporting a deployment decision.

The unit of assessment is the deployed workflow. The same model can support a read-only assistant or a tool that changes business records; those uses need different evidence. Begin with an explicit description of the proposed work and the authority required to perform it.

What it covers

Workflow and effects

Identify the actions, people, data, and downstream services affected.

Exposure

Consider tool authority, frequency, reversibility, and dependency failures.

Evidence

Link controls to relevant tests and known limitations.

Decision

Record the owner, restrictions, remaining risk, and review triggers.

Separate inherent exposure from the risk remaining after controls. A planned approval step should not be treated as effective until its placement and behavior have been checked. Record uncertainty where a consequence or likelihood cannot be estimated reliably.

In practice

Practical design patterns by Min.

• Map the full action path, including tools and child agents.
• Consider both successful misuse and ordinary operational mistakes.
• Assign each material risk a control owner and an evidence requirement.
• Reassess when authority, tools, users, or data change.

Use a small risk register linking scenario, affected resource, preventive control, detection signal, evidence, and decision owner. This is an illustrative operating record. Keep it tied to the inventory entry so a tool or permission change can trigger review.

Working example

Why the controls matter

A correct invoice extraction does not establish that the recipient account is authorized. The assessment connects data interpretation to the payment service’s controls and to the reviewer who can approve a consequential action.

Common pitfalls

• Using model accuracy as the only workflow-risk measure.
• Assigning a precise score without explaining its assumptions.
• Treating accepted risk as permanently acceptable after the workflow changes.

An approval document can become stale while the deployed configuration changes. Compare the approved workflow with observed tools and permissions rather than assume the original assessment still describes the system.

Questions for operational review

Use these questions to decide what evidence your team needs. They are editorial prompts, not a compliance checklist.

• Which actions create the greatest consequence?
• What evidence shows the controls work?
• Who owns the decision to accept remaining risk?