You Have Evidence. But Does It Prove the Control Works?

Cyber security teams rarely suffer from a complete lack of evidence.

There are policies. Procedures. Screenshots. Configuration exports. Vulnerability reports. SIEM dashboards. Penetration tests. Risk registers. Training records. Service tickets. Architecture diagrams. Supplier assessments. Meeting minutes.

Put enough of them into a SharePoint folder and it can look reassuringly like assurance.

But there is a fundamental problem.

Evidence that something exists is not necessarily evidence that it works.

And as organisations become more dependent on digital services, cloud platforms, suppliers and increasingly autonomous technologies, that distinction matters.

The question is no longer simply:

“Can you produce evidence for the control?”

It is:

What does that evidence actually prove?

The Evidence Trap

Consider something simple: privileged access management.

An organisation may provide:

• an access control policy;
• a screenshot showing MFA enabled;
• a list of privileged accounts;
• a quarterly access review;
• an identity platform configuration export.

There is certainly evidence.

But what conclusion can legitimately be drawn from it?

The policy establishes intent.

The configuration suggests a control has been implemented.

The account list describes the current environment.

The review demonstrates that somebody performed an assurance activity.

None of these, independently, necessarily proves that privileged access is being effectively controlled.

An assessor may still need to know:

• Are all privileged accounts actually covered?
• Are emergency or service accounts excluded?
• Can MFA be bypassed?
• Are privileges being removed when people change roles?
• Are dormant administrative accounts detected?
• Are privileged activities monitored?
• Were exceptions identified during the review?
• Were those exceptions remediated?
• Is the evidence current?

That is the difference between collecting evidence and performing assurance.

Intent, Implementation and Effectiveness

A useful way to think about cyber evidence is to divide it into three broad layers.

Evidence of intent

This describes what the organisation says should happen.

Policies, standards, procedures and governance documents are important. They establish expectations, responsibilities and management intent.

But a beautifully written policy proves remarkably little about the environment it governs.

A password policy stating that MFA is mandatory does not prove MFA is enabled.

A vulnerability management policy requiring critical vulnerabilities to be remediated within 14 days does not prove that they are.

A backup policy requiring annual recovery testing does not prove that a system can actually be restored.

Intent matters.

But intent is not implementation.

Evidence of implementation

The next layer demonstrates that something has actually been deployed or performed.

Configuration exports, system settings, ticket records, vulnerability scans, access reviews and monitoring dashboards can all provide stronger evidence.

They begin to answer the question:

Did the organisation actually do what its policy says?

But even implementation evidence has limitations.

A screenshot may show that a setting was enabled on one system at one moment in time.

A vulnerability report may demonstrate that a scan took place without demonstrating that its findings were resolved.

A backup console may show successful jobs without proving that the backups are recoverable.

Implementation is stronger than intention.

But implementation is not necessarily effectiveness.

Evidence of effectiveness

This is where assurance becomes considerably more valuable.

Evidence of effectiveness asks whether the security measure achieves the outcome for which it exists.

That might involve:

• successful restoration exercises;
• penetration testing;
• simulated phishing;
• detection testing;
• incident exercises;
• technical configuration validation;
• vulnerability remediation analysis;
• access control testing;
• threat hunting;
• independent assessment.

The question changes from:

“Is the control there?”

to:

“Does the control actually reduce the risk it was designed to reduce?”

That is a much harder question.

It is also a far more important one.

CAF 4.0 Makes the Distinction Explicit

This is not simply a philosophical argument about good security practice.

The NCSC Cyber Assessment Framework takes an explicitly outcome-focused approach rather than treating assurance as a checklist. CAF 4.0’s assurance guidance expects organisations to validate that security measures are effective, remain effective for as long as they are required, and be able to justify that confidence in a way that can be independently verified.

That word — confidence — is important.

Cyber assurance ultimately exists to establish justified confidence.

Not confidence because a policy exists.

Not confidence because somebody ticked a box.

Not confidence because the organisation passed an audit eighteen months ago.

Confidence because there is sufficient, relevant and credible evidence to support the conclusion being made.

And that creates another challenge.

Not all evidence deserves equal weight.

A Policy and a Penetration Test Are Not the Same Thing

Traditional compliance processes can unintentionally flatten evidence.

A requirement asks whether vulnerability management exists.

Evidence is attached.

The evidence box is now populated.

Move to the next requirement.

But consider two organisations.

Organisation A provides:

• a vulnerability management policy.

Organisation B provides:

• the policy;
• authenticated vulnerability scan results;
• remediation tickets;
• patch deployment records;
• evidence showing critical vulnerabilities were resolved within target timescales;
• independent penetration test results.

Both organisations technically possess “evidence”.

But the assurance value of those evidence sets is radically different.

One describes a process.

The other begins to demonstrate that the process operates and produces the intended outcome.

An effective assurance process therefore needs to consider not just the presence of evidence, but its:

• Relevance — does this evidence actually relate to the requirement being assessed?
• Strength — how directly does it demonstrate the outcome?
• Coverage — does it apply across the relevant systems, people and processes?
• Consistency — does other evidence support or contradict it?
• Currency — is it still representative of the environment today?
• Provenance — where did it come from, and can it be trusted?

This is considerably more sophisticated than document collection.

Evidence Can Contradict Evidence

There is another uncomfortable reality.

Organisations frequently possess evidence that contradicts itself.

The policy says dormant accounts are disabled after 90 days.

The identity platform contains accounts unused for 240 days.

The vulnerability policy requires critical findings to be resolved within 14 days.

The vulnerability report contains critical findings that have remained open for three months.

The incident response procedure requires an annual exercise.

The last exercise report is dated three years ago.

The asset management process requires a complete inventory.

The vulnerability scanner contains hundreds of systems that do not appear in it.

Each individual piece of evidence may be legitimate.

It is only when the evidence is examined together that the assurance gap becomes visible.

And this is one of the reasons cyber assurance becomes difficult at scale.

The answer may be distributed across dozens of documents, multiple systems and thousands of pages of material.

The problem is no longer simply finding evidence.

It is understanding the relationships between it.

Time Is Part of the Evidence

There is also a temporal problem.

Cyber environments change continuously.

People join and leave.

Applications are deployed.

Cloud resources appear and disappear.

Firewall rules change.

New suppliers connect.

Software vulnerabilities are discovered.

Administrative privileges are granted.

Controls that were effective six months ago may not be effective today.

This means assurance has a half-life.

A penetration test may provide strong evidence about the environment that existed when the test was performed.

It does not automatically provide the same level of confidence twelve months later.

Similarly, a screenshot captured during an audit may have accurately represented a configuration at 10:37 on a Tuesday morning.

It says nothing about whether that configuration changed on Wednesday.

Evidence therefore needs another dimension:

How long can we reasonably rely upon it?

That question becomes increasingly important as organisations move from periodic assessments towards more continuous models of assurance.

From Evidence Collection to Evidence Reasoning

This is where technology — and particularly AI — has the potential to change the economics of cyber assurance.

Not by allowing an AI system to simply declare an organisation compliant.

That would replace one overly simplistic process with another.

The more interesting opportunity is using technology to help humans reason across evidence at a scale that has historically been impractical.

For example:

• breaking framework requirements into specific obligations;
• identifying evidence relevant to each obligation;
• distinguishing between policies, technical evidence and testing;
• identifying conflicting evidence;
• highlighting missing evidence;
• examining evidence from multiple sources;
• considering the age and provenance of evidence;
• linking findings back to the original source;
• showing why an assurance conclusion was reached.

The aim is not to remove the assessor.

It is to give the assessor a much better evidence base from which to make a judgement.

Because assurance remains a judgement.

The important thing is making that judgement defensible.

This Is the Problem CyConex Is Designed to Address

CyConex was built around a simple premise:

Cyber assurance should be based on evidence, not assertions.

Organisations can provide policies, reports, technical outputs and other evidence to a CyConex assessment.

The platform analyses that material against individual obligations within frameworks such as the NCSC Cyber Assessment Framework, ISO 27001, NIST and Cyber Essentials.

But finding a matching paragraph is only the beginning.

CyConex is designed to help assessors understand what the evidence supports, where evidence is missing, where confidence may be weak and which original sources underpin a finding.

The assessor remains responsible for the judgement.

The technology reduces the extraordinary amount of manual work required to reach it.

That distinction matters.

AI should not turn cyber assurance into automated box ticking.

Used properly, it should help us move in the opposite direction.

Towards deeper scrutiny of the evidence.

The Question Boards Should Be Asking

When executives receive a cyber assurance report, they are ultimately being asked to accept a statement about risk.

A control is effective.

A requirement has been achieved.

A critical system is adequately protected.

A supplier is sufficiently secure.

Those statements carry consequences.

So perhaps the question boards should ask is not:

“Have we completed the assessment?”

Nor even:

“Do we have evidence?”

It should be:

“Why should we believe the evidence proves the control works?”

A mature assurance process should be able to answer.

It should identify the evidence.

Explain its relevance.

Recognise its limitations.

Expose contradictions.

And show how the conclusion was reached.

Because possessing evidence is easy.

Being able to prove what it means is assurance.