FiscaloomTool use turns a model output into an operational consequence. The test plan must cover not only whether the agent chooses the correct action, but also whether the action is permitted, validated, observable, and recoverable.
Test instructions and data as untrusted inputs
Requests, documents, web content, and retrieved knowledge can contain misleading instructions. Separate data from control instructions, constrain tool availability, and validate high-impact actions outside the model.
Test action safety and duplicate prevention
Validate tool arguments, permission scope, target record, amount or threshold, and expected state. Test timeouts and ambiguous responses so the workflow does not repeat an action whose outcome is unknown.
- Prompt or document injection
- Unauthorized tool selection
- Invalid or manipulated arguments
- Duplicate or replayed action
- Partial completion
- Unavailable dependency
- Misleading success response
Define containment and escalation
The workflow should fail closed for actions outside its approved boundary, preserve evidence, and route the case to an accountable owner. Recovery procedures should be tested before a live incident.
Build an adversarial tool-use test suite
Include documents that contain instructions, requests that disguise unauthorized actions, manipulated identifiers, boundary amounts, stale records, duplicate events, unavailable tools, partial responses, and misleading success messages. Test with the permissions and network conditions that production will use. A sandbox with broad privileges can conceal failures that appear only under real controls.
For each case, define the expected tool selection, arguments, validation, approval, failure state, evidence, and recovery. A safe refusal or escalation can be the correct result. Retain regression cases for every material defect because model, prompt, policy, connector, and application changes can reintroduce previously resolved behavior.
Prepare a containment and recovery playbook
Define how operators pause the workflow, revoke or narrow tool access, stop queued actions, preserve evidence, notify owners, and continue critical work manually. The playbook should distinguish a single failed case from a systemic issue that requires a wider stop. Practice the procedure before the workflow handles material transactions.
Recovery must account for uncertain outcomes. Before retrying, check the owning system and external side effects. If an action may have completed, route it for reconciliation instead of repeating it automatically. Document the decision to resume and the tests performed after remediation.
Further reading
These primary references informed the operating principles in this guide.