Two procurement functions complete the same AI readiness assessment. Both receive a score of four out of five for governance.
In the first organisation, leaders report that governance is established because an AI policy has been approved and a steering committee meets quarterly.
In the second, the policy is supported by a current inventory of AI systems, recorded use-case approvals, role-based access, defined human-review rules, operational logs, an incident process and evidence that exceptions are discussed and acted on.
The score is the same. The decision value is not.
A CPO reviewing the first result might reasonably conclude that governance is directionally in place but still needs to be tested. The second result could support a more consequential decision about controlled deployment. Treating the two findings as equivalent creates false precision: a numerical score that appears more reliable than the evidence beneath it.
This is a recurring weakness in AI readiness and maturity assessments. Organisations are asked what they have, what they do and how advanced they believe themselves to be. Their answers are converted into a stage, percentage or benchmark. The result may look objective even when it rests primarily on perception, aspiration or policy existence.
KOR’s Evidence Confidence Model addresses that problem by separating two findings:
- Capability: what level of AI readiness or maturity appears to exist?
- Confidence: how strongly is that conclusion supported by relevant evidence?
The model has three levels—Provisional, Substantiated and Verified—and applies them to individual findings as well as the overall assessment.
Its purpose is not to make every assessment slower or more bureaucratic. It is to ensure that the evidence standard matches the decision being made.
KOR’s central argument: An AI readiness score is only as decision-ready as the evidence supporting it. Self-report, documented capability and observed operating results should not be treated as equivalent.
What is an evidence-based AI readiness assessment?
An evidence-based AI readiness assessment combines stakeholder judgement with documentary, technical and operational evidence, then reports the strength of that evidence separately from the capability finding itself. Interviews and surveys remain useful inputs, but higher-consequence conclusions should be supported by artefacts, observed practice, telemetry, control records or validated outcomes appropriate to the decision being made.
Why an evidence-based AI readiness assessment needs more than self-reporting
Short self-assessments are useful. They can gather views quickly, identify differences between teams and create a starting point for discussion. Without them, important barriers may remain hidden because system records rarely explain how people experience a process.
The problem begins when perception data is presented as an established fact.
Consider the statement: “Our procurement team has adopted AI.”
Different respondents may interpret it in different ways:
- employees have access to an AI tool;
- most employees completed training;
- a small group uses it regularly;
- the intended population uses an approved tool inside a defined workflow;
- usage has been sustained and is producing measurable outcomes.
Those are materially different conditions. A simple questionnaire may record each as a positive answer.
The same problem appears across governance, data, supplier assurance and value:
- A policy is treated as evidence that controls operate.
- A data strategy is treated as evidence that the required data is usable.
- Standard contract clauses are treated as evidence that supplier AI risk is controlled.
- A business case is treated as evidence that benefits have been realised.
- A successful demonstration is treated as evidence of production performance.
KOR’s research into procurement AI maturity and readiness frameworks found that documentary, system, telemetry and financial evidence are not consistently used to qualify self-reported maturity. It also found that public tools often disclose little about scoring logic, calibration or the evidence required to claim a particular level. The result is a market in which two similar-looking scores can represent very different organisational realities.
This is not an argument against scoring. It is an argument against allowing the score to conceal the quality of the underlying evidence.
Evidence confidence is different from capability
A capability level and an Evidence Confidence level should be reported separately.
For example:
- Adoption capability: advanced
- Evidence Confidence: Provisional
This means the available information suggests advanced adoption, but the conclusion is based mainly on interviews, surveys or management assertion.
Another finding might be:
- Governance capability: developing
- Evidence Confidence: Verified
This means the organisation can demonstrate through operational records that governance remains incomplete. The capability finding is lower, but the conclusion is more reliable.
That distinction prevents two common errors.
The first is assuming that low confidence means low capability. It does not. An organisation may be operating effectively but recording the evidence poorly.
The second is assuming that a high capability score supported by weak evidence is safe to use for investment, benchmarking or scale decisions. It is not.
Confidence therefore qualifies the conclusion; it does not replace it.
How the Evidence Confidence Model was developed
KOR reviewed procurement-specific maturity and readiness approaches alongside broader AI governance, management-system and assurance guidance.
Several external frameworks reinforce the importance of evidence without defining KOR’s three levels.
The NIST AI Risk Management Framework supports the operationalisation of AI risk management through testing, evaluation, verification and validation, alongside documentation practices. Its structure of Govern, Map, Measure and Manage reflects the need to move from policy and context into measurement and continuing control. NIST currently notes that AI RMF 1.0 is being revised, so KOR treats it as supporting risk-management and evidence guidance rather than as a fixed readiness-scoring standard.
The UK Government’s AI Management Essentials asks organisations to maintain a current AI system record, document risk and impact assessments, record data and model information, establish issue-reporting processes and request relevant documentation from third-party providers.
The government’s Introduction to AI assurance describes assurance as the process of measuring, evaluating and communicating reliable evidence about AI capabilities, limitations and risks. It also emphasises using a proportionate combination of assurance techniques rather than relying on one mechanism.
ISO/IEC 42001 provides a management-system approach to establishing, implementing, maintaining and continually improving organisational AI governance. That lifecycle logic matters because evidence can become stale as systems, suppliers, data and processes change.
KOR’s Evidence Confidence Model translates these principles into a practical discipline for procurement AI readiness and maturity assessments.
Framework boundary: Provisional, Substantiated and Verified are KOR assessment levels. They are not an external certification, legal assurance opinion or substitute for a formal audit where one is required.
The three Evidence Confidence levels
Use the approved KOR Evidence Confidence Model visual already created for this article cluster. Preserve the oyster manuscript background, graphite technical linework, black typography, restrained electric-blue accents, handwritten annotations and scientific evidence-ladder treatment.
1. Provisional
A Provisional finding is supported primarily by self-report, management assertion, survey responses or a limited number of stakeholder interviews.
The information may be credible. It may also be incomplete, aspirational, outdated or based on inconsistent interpretations.
Typical evidence
- survey responses;
- leadership statements;
- employee perceptions;
- workshop outputs;
- uncorroborated descriptions of current practice;
- pilot anecdotes;
- estimated benefits;
- statements that a policy, process or tool exists without the item being examined.
What Provisional findings are useful for
They can support:
- early orientation;
- identifying perceived strengths and concerns;
- designing the evidence request;
- highlighting differences between leadership and operational teams;
- selecting areas for deeper assessment;
- establishing an initial hypothesis.
What they should not support on their own
They should not be the sole basis for:
- a major technology investment;
- an enterprise-scale deployment decision;
- external benchmarking claims;
- a declaration that governance is effective;
- finance-credible benefit reporting;
- a claim that supplier AI risk is controlled;
- a high-confidence maturity stage.
Procurement example
A survey shows that 82% of category managers believe AI is being used regularly. That may indicate broad awareness or enthusiasm. It does not establish which tools are used, whether usage is approved, whether the activity occurs inside the intended workflow or whether users review outputs appropriately.
The adoption finding is therefore Provisional until supported by additional evidence.
2. Substantiated
A Substantiated finding is supported by relevant documents, artefacts or technical evidence showing that a capability has been designed, approved or implemented.
This is stronger than self-report because the assessor can examine what the organisation has actually established.
Typical evidence
- policies and standards;
- named ownership and decision rights;
- process maps and operating procedures;
- approval records;
- risk and impact assessments;
- architecture diagrams;
- data-flow documentation;
- screenshots and system configuration;
- training materials and role guidance;
- implementation plans;
- supplier documentation;
- contractual controls;
- governance minutes and action records.
What Substantiated findings are useful for
They can support:
- roadmap design;
- prioritisation of gaps;
- determining whether the operating model has been designed coherently;
- deciding whether a use case can progress into controlled testing;
- comparing intended practice with actual practice;
- identifying where operational verification is still required.
The limitation of documentary evidence
Existence is not the same as operation.
An AI policy may be comprehensive but unknown to users. A human-review procedure may exist but be ignored under time pressure. A supplier contract may include audit rights that have never been exercised. An architecture diagram may describe a target state rather than the live environment.
Substantiated evidence therefore answers: has the capability been credibly designed or established?
It does not always answer: does it work consistently in practice?
Procurement example
A procurement function provides:
- an approved AI policy;
- documented use-case approval criteria;
- named owners;
- a contract-review procedure;
- training materials defining mandatory human checks;
- governance minutes showing that risks were considered.
The governance capability is now supported beyond assertion. It can be rated Substantiated.
The assessor should still examine logs, review records, exceptions and incidents before concluding that the control operates consistently.
3. Verified
A Verified finding is supported by operational evidence showing that the capability is functioning in practice and, where relevant, producing measured outcomes.
Verification may use system records, observed controls, telemetry, audit evidence, performance data or finance-reviewed results. It does not require the same method for every claim. The evidence should be proportionate to the decision and risk.
Typical evidence
- approved-tool usage data;
- workflow records;
- audit logs;
- access and permission records;
- exception, correction and escalation data;
- observed operation of human-review controls;
- incident records and remedial actions;
- model or supplier-change reviews;
- performance against a defined baseline;
- sustained adoption over time;
- finance-reviewed savings or productivity outcomes;
- evidence that underperforming use cases were redesigned or stopped.
What Verified findings are useful for
They can support:
- controlled scaling decisions;
- production approval;
- material investment decisions;
- portfolio prioritisation;
- credible internal benchmarking;
- verified benefit reporting;
- decisions to redesign, pause or retire a use case;
- higher-confidence maturity conclusions.
Verification is still contextual
Verified does not mean perfect, permanent or universally transferable.
A contract-intelligence workflow may be verified for English-language commercial agreements in one jurisdiction. That does not verify its performance across every contract type, language or legal environment.
The finding must therefore state:
- the claim being verified;
- the scope;
- the evidence period;
- known limitations;
- the decision the evidence can reasonably support.
The Evidence Confidence decision table
| Confidence level | Primary evidence | Appropriate use | Main limitation |
|---|---|---|---|
| Provisional | Survey, interview, assertion and perception data | Orientation, hypothesis formation and evidence planning | May reflect aspiration, inconsistent terminology or incomplete knowledge |
| Substantiated | Policies, process artefacts, approvals, architecture, contracts and documented controls | Roadmap design, prioritisation and readiness for controlled testing | Shows design or existence but may not show sustained operation |
| Verified | Telemetry, logs, observed controls, exceptions, outcomes and finance-reviewed evidence | Scaling, investment, portfolio and high-confidence maturity decisions | Applies only to the evidenced scope, period and conditions |
An evidence hierarchy for procurement AI assessments
Evidence can be organised into eight broad layers. The hierarchy reflects increasing proximity to actual operation and outcome, but it should not be applied mechanically.
1. Survey answers and management assertions
These provide perception, intent and context. They are often the fastest way to identify issues but are vulnerable to different interpretations and incomplete organisational visibility.
2. Stakeholder interviews and corroborating accounts
Interviews test the survey findings and expose differences between senior leaders, process owners, users and control functions.
Corroboration matters. A governance process described consistently by procurement, legal, technology and operational users is more credible than a claim made by one programme owner.
3. Policies, standards and documented ownership
These show that expectations and accountability have been formally defined.
The assessor should examine whether documents are current, approved, relevant to the use case and understood by the responsible roles.
4. Process maps, workflow artefacts and technical configuration
These show how the organisation intends the capability to operate.
Examples include approval flows, review checklists, system configuration, access rules, data lineage and integration documentation.
5. Supplier documentation, contractual controls and assurance records
Procurement functions depend heavily on third-party AI. Relevant evidence may include:
- model and system documentation;
- training-data or data-use information;
- subprocessor records;
- security and privacy assurance;
- performance testing;
- change-notification obligations;
- audit and information rights;
- incident obligations;
- service continuity and exit provisions.
Standard clauses alone are weak evidence if the organisation has not obtained the information needed to operate them.
6. Usage data, logs, exceptions and correction records
These show what happens in practice.
They can reveal whether intended users adopt the tool, whether the workflow is followed, where outputs are corrected and whether review or escalation occurs.
7. Operational outcomes against a baseline
This evidence tests whether the process has improved.
Relevant outcomes may include cycle time, rework, compliance, risk identification, throughput, quality, stakeholder experience or capacity released.
The baseline should be defined before the benefit is claimed.
8. Finance-validated savings, productivity or risk outcomes
This is the strongest form of evidence for an economic claim.
Finance validation is not required for every experimental use case. It becomes important when projected benefits are used to justify material investment, enterprise deployment or public claims.
Higher in the hierarchy does not always mean more relevant
A large volume of telemetry cannot prove that a policy is appropriate. A signed contract cannot prove that a use case is delivering value. A finance-approved business case cannot prove that employees are reviewing outputs correctly.
Evidence must match the claim.
For example:
- To assess role clarity, examine role guidance and interview affected users.
- To assess approved adoption, examine system and workflow usage by the intended population.
- To assess human oversight, examine review rules, logs, corrections and escalations.
- To assess supplier assurance, examine due diligence, system documentation, contractual rights and ongoing monitoring.
- To assess realised value, examine adoption, baseline performance, outcomes and full costs.
Confidence comes from a coherent body of relevant evidence, not from the largest file repository.
Nine tests for evidence quality
Every important item should be tested before it increases confidence in a finding.
1. Relevance
Does the evidence support the specific claim being made?
Training completion is relevant to enablement. It is not sufficient evidence of sustained adoption.
2. Completeness
Does the evidence cover the full assessed scope, or only a selected team, process or period?
A successful regional pilot should not be presented as function-wide capability.
3. Recency
Is the evidence current enough for the decision?
AI suppliers, model versions, data sources and organisational processes change quickly. Evidence that was reliable at implementation may no longer describe the live system.
4. Consistency
Do different sources tell the same story?
A policy may require approved-tool use while interviews reveal widespread use of unauthorised assistants. The contradiction is itself an important finding.
5. Operational use
Does the artefact influence real work?
A process map or control document should not increase confidence significantly if users do not follow it.
6. Ownership and accountability
Is someone responsible for maintaining the evidence and acting on what it shows?
A dashboard without an accountable decision-maker is monitoring without management.
7. Scope and environment
Does the evidence concern a demonstration, controlled pilot, production workflow or scaled process?
Performance under curated pilot conditions should not be assumed to persist at enterprise scale.
8. Independence of validation
Who generated or approved the evidence?
Supplier performance claims, programme estimates and user-reported benefits may all be useful. Higher-stakes decisions may require independent testing, finance review or control-function challenge.
9. Contradictory evidence
Has evidence that weakens the conclusion been recorded?
A credible assessment does not discard low adoption, exceptions or failed use cases because they complicate the maturity narrative.
Five procurement claims that should be challenged
| Claim | Weak evidence | Stronger evidence |
|---|---|---|
| “Our people have adopted AI.” | Training attendance, access licences and positive survey responses | Sustained approved-tool usage inside the target workflow, with role coverage, review behaviour and adoption trends |
| “Our AI governance is established.” | An AI policy and steering committee terms of reference | Use-case approvals, named accountability, access controls, logs, incidents, exceptions and evidence that governance decisions change practice |
| “The use case saves time.” | User estimates and demonstration timings | A pre-implementation baseline, measured end-to-end cycle time, quality and rework data, adoption levels and full operating cost |
| “Supplier AI risk is covered contractually.” | Standard data-protection and AI wording | System and model documentation, data-flow review, subprocessor visibility, change notification, audit information, incident obligations and ongoing monitoring |
| “The pilot proves the solution works.” | A successful demonstration using selected data | Observed performance in a live bounded workflow, including exceptions, corrections, user behaviour and outcomes across a defined period |
Worked example: a high readiness score with low confidence
A multinational procurement function completes an internal readiness survey before seeking funding for an AI sourcing assistant.
The assessment produces strong results:
- strategy: 4.2 out of 5;
- governance: 4.0;
- adoption: 3.9;
- value: 4.1.
The programme is presented as ready for controlled scaling.
The evidence review finds:
- the strategy score is supported by an approved transformation plan and named sponsorship;
- the governance score relies on a policy and steering committee, but there are no recorded use-case approvals or operational logs;
- the adoption score relies on training completion and positive sentiment, while system data shows irregular use by a narrow group;
- the value score relies on estimated time savings from the business case, with no end-to-end baseline or measurement of rework;
- the supplier contract includes general AI wording, but model-provider, data-use and material-change information is incomplete.
The Evidence Confidence result
| Finding | Capability assessment | Confidence | Decision implication |
|---|---|---|---|
| Strategic intent | Established | Substantiated | Use the strategy to prioritise controlled tests, but confirm operating ownership |
| Governance | Developing to established | Substantiated | Test approval, logging and incident controls before production expansion |
| Adoption | Potentially developing | Provisional | Do not use training completion as proof of function-wide adoption |
| Value | Unproven | Provisional | Establish a baseline and measure the full workflow before relying on the business case |
| Supplier control | Partially established | Substantiated | Close documentation and contractual gaps before increasing dependency |
The assessment does not conclude that the programme has failed. It concludes that the headline readiness score overstates the confidence available for a scale decision.
The appropriate next step is a bounded production test with explicit evidence requirements—not an enterprise rollout based on the average score.
Confidence should be attached to specific claims
An organisation should not be labelled simply “Verified”.
Evidence confidence varies by:
- capability;
- use case;
- region;
- process;
- supplier;
- system version;
- evidence period;
- type of decision.
A function may have Verified governance for one contract-intelligence workflow, Substantiated data readiness for a sourcing platform and only Provisional adoption evidence across the wider organisation.
That variation is useful. It tells leaders exactly where further investigation is necessary.
A single organisation-wide confidence label would recreate the averaging problem the model is intended to solve.
Absence of evidence is not automatically evidence of low maturity
This distinction is important.
A procurement team may operate a strong review process but fail to retain consistent records. The correct conclusion is not necessarily that the control is absent. It is that the finding cannot yet be relied upon with high confidence.
The assessment should record:
- the claimed capability;
- the evidence requested;
- what was provided;
- the evidence gap;
- the resulting confidence level;
- what would increase confidence.
Poor record-keeping may itself be a maturity weakness, especially where traceability or accountability is required. But it should not be confused with proof that the underlying activity never occurs.
Evidence has a half-life
AI evidence becomes stale.
A supplier may change its underlying model. A data source may deteriorate. A new region may introduce different process conditions. A control may weaken after the implementation team leaves. Users may develop workarounds. Benefits may decline as the task mix changes.
Verified findings should therefore include:
- the date or period assessed;
- the relevant system or model version where material;
- the operational scope;
- the review trigger;
- any known conditions that could invalidate the conclusion.
Triggers may include:
- material supplier or model changes;
- expansion into a new process, country or user population;
- significant incidents;
- deterioration in outcomes;
- changes to the data or integration architecture;
- changes in legal, regulatory or contractual requirements.
Evidence Confidence is not a permanent badge. It is a current statement about the support available for a defined conclusion.
How to apply the Evidence Confidence Model
1. Define the claim precisely
Avoid broad statements such as “data is ready” or “governance is mature”.
State what is being assessed, for which use case, process, geography and population.
2. Identify the decision the finding must support
An early roadmap requires a different evidence standard from an enterprise investment or production approval.
3. Gather perception data
Use surveys and interviews to understand intent, experience and disagreement. Treat the results as hypotheses, not proof.
4. Request relevant artefacts
Collect the policies, records, process documents, technical evidence, supplier information and baseline plans that should exist if the claimed capability is established.
5. Test operation
Review system data, logs, workflow records, exceptions, incidents, corrections and observed control activity.
6. Examine outcomes
Where value or effectiveness is claimed, compare performance with a defined baseline and include adoption, quality, risk and full cost.
7. Look for contradictions
Compare leadership statements, documents and operational evidence. Do not average away material inconsistencies.
8. Assign capability and confidence separately
State the assessed capability and whether confidence is Provisional, Substantiated or Verified.
9. Record what would change the conclusion
Specify the missing evidence, review date, scope limitation and next decision gate.
How the Evidence Confidence Model connects to KOR’s other models
The model is designed to qualify the findings produced elsewhere in the KOR assessment system.
- AI Readiness in Procurement: Why Pilots Do Not Mean You Are Ready to Scale introduces the distinction between activity, readiness, maturity and value.
- The Procurement AI Readiness–Realisation Matrix plots organisational preparedness against adopted, measurable outcomes.
- The Procurement AI Maturity Model defines five stages and the hard gates required for progression.
- The Procurement AI Adoption Model measures culture, trust and behaviour through observable operating evidence.
- Procuring AI Safely develops supplier due diligence, contracting and ongoing monitoring.
- From AI Pilot to Measurable Value develops the evidence needed for baselines, benefits and scale economics.
The Evidence Confidence Model does not compete with those frameworks. It states how much reliance leaders should place on their conclusions.
Better evidence should change the decision
Evidence collection is not valuable because it creates a larger audit file. It is valuable when it changes what the organisation does.
A Provisional finding may justify further investigation. A Substantiated finding may justify controlled testing. A Verified finding may support scale—provided the evidence matches the scope and risk of the decision.
The practical discipline is to stop reporting readiness and maturity as unsupported single numbers.
A CPO should be able to see:
- the capability claimed;
- the evidence supporting it;
- the confidence that can reasonably be placed in the finding;
- the limits of that confidence;
- the decision the evidence is strong enough to support.
That produces a less flattering assessment in some cases. It also produces a more useful one.
The question is not simply:
What did the assessment score?
It is:
What conclusion can the organisation defend, using what evidence, strongly enough to justify the next decision?
That is the difference between an AI maturity score and a decision-ready assessment.
Frequently asked questions about evidence-based AI readiness assessments
What is an evidence-based AI readiness assessment?
An evidence-based AI readiness assessment combines stakeholder views with documentary, technical, operational and financial evidence. It reports both the capability identified and the confidence that can reasonably be placed in that conclusion.
Why is AI self-assessment not enough?
Self-assessment is useful for orientation, but respondents may interpret terms such as adoption, governance and value differently. Survey answers should therefore be treated as Provisional until relevant artefacts or operating evidence substantiate them.
What evidence demonstrates AI readiness?
Relevant evidence may include ownership records, policies, process maps, architecture, data-flow documentation, supplier assurance, system configuration, approved-tool usage, logs, exceptions, observed controls, performance against a baseline and finance-reviewed outcomes. The evidence must match the specific claim being assessed.
What is the difference between Substantiated and Verified evidence?
Substantiated evidence shows that a capability has been credibly designed, approved or implemented through documents and artefacts. Verified evidence shows that it operates in practice through telemetry, logs, observed controls, measured outcomes or other operational records.
Does missing evidence prove that an organisation is not ready?
No. Missing evidence reduces confidence in the finding but does not automatically prove low capability. The assessment should record what was claimed, what evidence was requested, what was available and what would be required to increase confidence.