Insight · AI in procurement
AI in procurement: measure the whole decision
A faster first draft is useful only if the procurement decision improves. NatQuest’s practical approach to testing AI accounts for review time, accuracy, evidence and ownership.
- Published
- Updated
- Reading time
- 2 min
- Written by
- NatQuest
A procurement team does not need another demonstration that AI can produce text. It needs to know whether that text helps people make a sound commercial decision. Our recommendation is to test a complete piece of work, from the initial brief to an approved output, before making a savings claim.
Start with one decision
Choose a bounded workflow: preparing a specification, reviewing a supplier response or drafting a bid from approved material. Record the inputs, the person accountable and what constitutes an acceptable result. Keep the task comparable across the pilot.
For tender evaluation, establish the criteria before reviewing responses. An AI suggestion should be checked against the actual submission and the agreed criteria; it should not silently change the basis of an assessment.
Count all the work
Measure the baseline with the same boundaries you will use for the pilot. Record preparation, drafting, review, correction and approval separately. A reduction in drafting time can be outweighed by additional checking or rework.
Distinguish staff effort from elapsed cycle time. Two hours removed from a task do not necessarily remove two hours from a process waiting for an approval. Report both where they matter, and record the number and type of tasks observed.
Make quality visible
Agree acceptance criteria before testing. Check whether material claims have supporting sources, whether important requirements were missed, and whether a reviewer can reconstruct the reasoning. Track substantive corrections alongside time, so an apparently quick output does not conceal a quality problem.
Use authorised material. Before uploading tender or supplier documents, confirm access permissions, model providers, hosting, retention and any use for training. Assign someone to approve the output and record their changes.
Decide what the pilot justifies
At the end, compare the full workflow with its baseline. Explain the sample, period and limitations. A result from one task or category should remain a result from that pilot until it has been tested more widely.
Our Deploy → Optimise → Orchestrate → Sustain approach starts with this discipline: define the work, inspect the evidence, then decide what deserves to scale.