An AI risk assessment should end with one of three decisions: go, change, or stop. If the result is a heat map that never changes a release decision, you have decorated a meeting.
Picture a support agent that can read account records and issue credits. “Customer-service chatbot” is too vague to assess. A usable boundary says which customers it serves, which records it reads, the maximum credit it may propose, whether a person approves the write, and what happens when the billing API times out. Now there is something to test.
We’re gmware, a custom software development firm in Austin, Texas, with engineering centers in Bangalore and Mohali, India. We build AI and automation around existing business systems. Our opinion is simple: assess the action path, not the model in isolation.
The NIST AI Risk Management Framework is voluntary and use-case agnostic. Its core groups work under Govern, Map, Measure, and Manage. NIST also warns that those functions are not an ordered checklist. That distinction matters. A copied checklist can show activity while missing the particular way your system can fail.
Start with a boundary that can be disproved
Write the intended use in operational terms. Include the user, the input, the permitted output or action, the connected systems, and the person accountable for the result. Then write the excluded uses. This second list is often more useful.
For the credit agent, excluded uses might include changing a payment method, deciding fraud, handling a disputed identity, or issuing a credit above the approved limit. Those exclusions become routing rules and test cases. They also stop a later product demo from quietly expanding the deployment.
This is where an AI readiness assessment and an AI risk assessment split. Readiness asks whether the organization can deliver and operate AI. Risk assessment asks what can go wrong in a named deployment, what evidence changes that judgment, and who may accept what remains.
The assessment record
Keep the working record short enough that the release team will read it. One risk per row. Attach test output and decision evidence rather than stuffing screenshots into the table.
| Field | Entry for the credit-agent example |
|---|---|
| System boundary | Support agent reads an authenticated account and proposes a credit |
| Intended action | Create a credit proposal within the written service policy |
| Excluded action | Change payment details, decide fraud, or approve its own proposal |
| Credible failure | Prompt content causes a proposal outside policy |
| Affected party | Customer whose balance or service could change |
| Evidence | Adversarial test set, tool-call trace, approval record, API response |
| Preventive control | Credit tool enforces amount and reason constraints |
| Detective control | Alert on rejected calls and unusual proposal patterns |
| Recovery | Cancel the pending proposal and route the case to billing operations |
| Residual-risk owner | Named business owner, not “the AI team” |
| Decision | Go, change, or stop, with date and approver |
| Reassessment trigger | Model, policy, tool, data, limit, or user population changes |
The row has a useful property: another reviewer can challenge it. They can ask why the failure is credible, reproduce the test, inspect whether the tool enforces the limit, and reject the residual-risk decision. “Hallucination: medium” cannot survive that conversation because it says almost nothing.
Map failure through the whole system
Model output is one link. The source data may be stale. Retrieval may select the wrong account record. An instruction inside a document may conflict with system policy. The tool may accept a field the policy meant to prohibit. A human approval screen may hide the detail that matters. The write may succeed while the interface reports failure, inviting a duplicate.
Follow one bad outcome backward through those links. Then follow recovery forward. The result is a fault path grounded in the actual architecture, not a catalog of scary nouns.
The NIST Generative AI Profile is a cross-sector companion to the AI RMF. It is useful for finding risk areas you might have missed. It still cannot decide your system boundary or risk tolerance for you.
Evidence beats labels
Choose evidence before choosing red, amber, or green. A test needs the input, expected boundary, observed result, environment, model and policy versions, and tool response. Keep abstentions and routed cases too. A test suite containing only correct answers tells you nothing about the stop path.
For an agent that can act, trace the whole trip: instruction, tool request, authorization, and target-system result. Dig deeper into the operating telemetry in our guide to AI agent observability. The assessment has a narrower job. It decides which events must exist and which failure thresholds stop the rollout.
No single number settles residual risk. Severity and likelihood scales can help sort work, but the scale needs definitions and evidence. A five-by-five grid with no decision rule is a colorful delay.
Put a person on the decision
The model owner can explain behavior. Security can challenge access and abuse paths. Privacy, legal, compliance, safety, and domain owners join when the deployment touches their authority. The business owner still needs to decide whether the remaining risk is acceptable for this use.
Record dissent. Record assumptions. If a control will exist next quarter rather than today, mark the launch decision against today’s control, not the roadmap.
An assessment also expires. A new model, added connector, broader audience, higher transaction limit, or changed approval rule can invalidate the evidence without changing the product name. Put reassessment triggers beside the release decision so change management has something concrete to check.
A practical finish line
The assessment is done when a reviewer can answer five things from the record: what system and use were examined, which bad outcomes were tested, what evidence was observed, which controls bound the release, and who accepted the remainder. It is not done because every cell has a color.
Sometimes the right decision is smaller scope. Remove the write permission, lower the limit, narrow the user group, or run in proposal-only mode while collecting evidence. That is not a failed AI project. It is risk work doing its job.
If you need an engineering team to turn a risk decision into tool boundaries, tests, and operating evidence, see our AI agents and LLM integration service. Reach out with one proposed AI action and the system it would touch. We’ll tell you what we would test before letting it write.