Reviewed by TRUST-IT ·
Choose one measurable unit of work
Define the task precisely: a resolved internal question, a reviewed document or a correctly prepared service request. Record its inputs, accepted output and responsible reviewer. Avoid a pilot scope that combines several unrelated jobs, because the results become hard to interpret. Agree what the system may do, what requires approval and which failures would make the use case unacceptable.
Measure the current process
Collect a representative baseline for volume, time, error rates and exception handling. Include work performed by reviewers and the delays caused by missing information. Separate active handling time from waiting, and note where estimates replace measurements. A fair comparison needs the same task population and an explicit account of differences, such as a pilot receiving cleaner documents than the production team normally sees.
Build the evaluation before tuning the system
Use realistic cases that include routine requests, difficult inputs and examples the system should escalate. Keep a portion separate from development so repeated tuning does not simply teach the system the test. Define scoring with subject-matter experts. For a knowledge assistant, inspect whether sources support the answer; for an agent, inspect whether the right action occurred with the correct permission.
Calculate cost per accepted outcome
Count model usage, retrieval or storage, infrastructure, integration maintenance, monitoring and human review. An inexpensive generated answer can become costly if it needs extensive correction. Compare cost per accepted task and total process time with the baseline. State assumptions about volume, provider prices and support rather than presenting one pilot result as a guaranteed saving at enterprise scale.
Test boundaries as well as usefulness
Check access restrictions, sensitive information handling, unexpected tool results and behaviour when source content contains conflicting instructions. OWASP’s GenAI risk material is a useful reference for application-level risks. NIST’s AI Risk Management Framework provides a broader voluntary risk-management reference. These references inform a proportionate test plan; they are not a certificate or proof that a particular implementation is compliant.
Make the rollout decision explicit
Record the achieved results, unresolved issues and requirements for operation. A rollout should name the service owner, support route, usage controls and changes that trigger re-evaluation. The next step may be a wider controlled trial, a narrower use case or a redesign. Continuing only where the evidence supports it keeps investment connected to operational value rather than enthusiasm for the demonstration.
What must an AI supplier deliver before you accept the project?
Turn the pilot decision into an acceptance schedule before commissioning implementation. Give each requirement an identifier, a test method, evidence to retain and a named reviewer. Set thresholds for your use case and risk level; a generic accuracy percentage is not an acceptance specification. The following is a proposed assessment structure, to be tailored to the contracted system.
Swipe across the table to see all columns.
| Requirement | Test and evidence to receive | Agreed acceptance condition | Reviewer and operational owner |
|---|---|---|---|
| Traceable release and evaluation | A release manifest identifying model and provider, prompt templates, retrieval index, tool permissions and configuration. Test-case IDs, evaluation-set version, run dates and recorded failures connect each result to that release. | The receiving team can repeat the agreed evaluation and explain variation. Quality thresholds, case categories and repeat-run rules were defined before acceptance; an average score does not conceal critical failures. | Business subject-matter reviewer validates scoring; the technical owner maintains the release and evaluation records. |
| Supported answers and controlled retrieval | Held-out normal, ambiguous, unsupported and multilingual questions where relevant. Evidence connects claims to permitted source passages and records abstentions, stale-source handling and incorrect citations. | The agreed quality and escalation thresholds are met for each relevant case category. Unanswerable cases follow the agreed human-review path; access-control failures are evaluated separately from answer quality. | Knowledge or process owner accepts content quality; the application owner manages source freshness and re-evaluation. |
| Identity and action boundaries | Negative tests using other users’ documents, revoked access and stale caches. For action-taking agents, test changed approval parameters, expired approvals and repeated requests. Retain sanitised traces showing the authorisation decision and effect. | Specified prohibited access and actions are blocked in the agreed test set. Server-side controls enforce authority independently of model output; any unresolved security blocker keeps affected capabilities disabled. | Security reviewer approves boundaries; identity and application owners maintain permissions, revocation and action controls. |
| Failure handling and recovery | Demonstrate provider timeout, tool failure, interrupted work and rollback. Record the last completed action, retry decision, alert and recovery result; show the operator how to stop further actions. | The operator can stop the workflow and recover within the agreed limits. Retries do not silently duplicate business actions; uncertain or partial outcomes enter a defined reconciliation process. | Service owner accepts recovery; the operations team owns the runbook, alerts and escalation. |
Accept the operating model, cost and exit route too
A successful test does not transfer operational responsibility automatically. Review these dependencies alongside technical results, while the supplier is still responsible for delivery. Record agreed exceptions, their owners and expiry or review dates.
Swipe across the table to see all columns.
| Requirement | Test and evidence to receive | Agreed acceptance condition | Reviewer and operational owner |
|---|---|---|---|
| Operating cost and capacity | Reconcile model usage, retrieval/storage, infrastructure and human review against completed, accepted tasks. Include failed attempts, retries, monitoring and support; record volume and concurrency assumptions. | Budget limits, alerts and behaviour when a limit is reached are demonstrated. Cost per accepted outcome uses an agreed denominator; a test with no accepted outcomes cannot establish a viable unit cost. | Business owner accepts the economics; the platform owner maintains metering, quotas and capacity limits. |
| Support and change ownership | A named support route, covered hours, severity definitions, response expectations and division of responsibility among the client, implementer and model provider. List changes to models, prompts, permissions and sources that require re-testing. | The receiving team can open and escalate an issue, roll back an approved change and identify who authorises deployment. Response targets and charges for support or changes are agreed rather than assumed. | Service owner approves coverage; the operational contact owns triage, releases and re-evaluation scheduling. |
| Transfer and supplier exit | An inventory of accounts, licences, source/configuration rights, dependencies and retention settings. Demonstrate the agreed export of owned data/configuration, credential rotation and removal of supplier access. | The client can perform the agreed transfer or exit steps. Document non-portable components, export limits, deletion/retention obligations, replacement work and associated costs before acceptance. | Procurement and the business owner approve contractual dependencies; technical and data owners verify the handover. |
The handover pack should be usable without the demonstrator
- Versioned architecture, configuration and data-flow inventory, including providers and external services; secrets are transferred through an approved credential process, not embedded in documents.
- Evaluation report with test IDs, observed failures, unresolved issues and evidence references. Restrict access to sensitive traces; use redacted examples in ordinary reporting.
- Operator runbook covering access changes, alerts, safe shutdown, recovery, cost limits and escalation, plus a witnessed handover exercise with the receiving team.
- Acceptance record naming the approved release and enabled capabilities, the authorised decision-maker, any conditions and the changes that require a new decision.
A limited rollout can be conditional on specific remedies and a review date. High average quality does not offset an unresolved prohibited action or access-control failure. An exception needs an explicit business and security decision; it should not disappear inside the supplier’s final presentation.
The NIST AI RMF Playbook offers voluntary risk-management suggestions, not a mandatory acceptance checklist. Use it as a reference when tailoring the review: NIST AI RMF Playbook.
Explore the detailed RAG access-control tests. Discuss the scope of an AI pilot and supplier review.
Further reading
Put the guidance to work
AI opportunity assessment, use-case prioritisation, pilot economics, supplier evaluation, operating models and practical team training.
AI strategy & adoption