When AI capabilities change, revisit the threat model

Updated 17 October 2026. A more capable model can change the risk of a workflow without changing its interface. The product still looks familiar. The assumptions underneath it may not be.

Reassess reachable consequences

Begin with what the system can access and change. A capability increase matters differently in a read-only research tool and in an agent with production credentials. The relevant question is what new behaviour becomes possible in the actual environment.

An illustrative model update might improve planning across several tools. That can make the product more useful while increasing the importance of task boundaries and approval conditions. Better capability is not automatically better behaviour under every instruction.

Review the tests that supported the previous release. Identify which assumptions depended on the older capability level and which controls are enforced independently. A permission check should not rely on the model being too weak to attempt an unwanted action.

Use a safe environment to examine new failure cases. Keep observed behaviour separate from hypothetical concerns. Both can guide work, but a possible scenario should not be reported as an incident that actually occurred.

The practical next step

Treat the update as a system change, not merely a better score. Revisit the assumptions that matter before deciding whether the existing evidence still supports deployment.

Keep the resulting decision, its owner and unresolved questions in a short decision record. For claims about agent behaviour, connect the decision to evaluation evidence rather than relying on a successful demonstration alone.

Further primary reading: Anthropic, Demystifying evals for AI agents. The review map and product recommendations above are my analysis, not a certification or a legal assessment of a particular deployment.