Are Your Security Metrics Just for Show A Veteran CISO Weighs In

We often rely on metrics that show how much effort we're putting in – the number of patched vulnerabilities, our response times, and so on. But these vulnerability management metrics often get mixed up with operational metrics. Why? Because traditional approaches to vulnerability management don't always reduce actual risk. So, we end up reporting on things like how many patches we applied using the traditional 30/60/90-day patching method.

I call these vanity metrics. They look good in reports but don't really change anything. They might make us feel better, but they don't give us real insights. Meanwhile, threats are getting more complex, and attackers are finding the gaps we aren't even measuring. I've seen this disconnect lead to serious problems for companies.

In this article, I'll explain why these vanity metrics aren't enough in today's world, and why we need to measure effectiveness instead of just activity.

Drill Down: What Are Vanity Metrics?

Vanity metrics are numbers that look impressive but don't offer much strategic value. They're easy to track and present, and they often show activity. But they rarely show real risk reduction. Here are three common types:

• Volume metrics – These are counts, like patches applied, vulnerabilities found, or scans completed. They give a sense of progress but don't tell you anything about business impact or risk.
• Time-based metrics without risk context – Things like Mean Time to Detect (MTTD) or Mean Time to Remediate (MTTR) can sound impressive. But unless you're prioritizing based on what's most critical, speed isn't the whole story.
• Coverage metrics – Percentages like "95% of assets scanned" or "90% of vulnerabilities patched" can create a false sense of security. They ignore the important question: what about the 5% we missed? And do they matter most?

Vanity metrics aren't necessarily bad, but they're incomplete. They track motion, not meaning. If they aren't connected to real threats or critical business assets, they can hurt your security strategy.

Vanity Metrics: More Harm than Good

If vanity metrics dominate your reports, they can cause more harm than good. I've seen companies waste time and money chasing numbers that look good to executives, while real vulnerabilities were left untouched.

What happens when you rely on these metrics?

• Misallocated effort – Teams focus on what's easy to fix or what improves a metric, instead of what truly reduces risk. This creates a gap between what's done and what needs to be done.
• False confidence – Rising charts can make leaders think the company is secure. But without context – like exploitability or attack paths – that belief is fragile.
• Broken prioritization – Huge lists of vulnerabilities without context can be overwhelming. High-risk issues can get lost, and remediation gets delayed where it matters most.
• Strategic stagnation – If your reports reward activity over impact, innovation slows down. The program becomes reactive – always busy, but not always safer.

I've seen breaches happen in environments with great-looking KPIs. Why? Because those KPIs weren't based on reality. A metric that doesn't reflect real business risk is dangerous.

Moving to Meaningful Metrics

Vanity metrics tell us what's been done, but meaningful metrics tell us what matters. They shift the focus from activity to impact, so security teams and business leaders share a common understanding of risk.

A meaningful metric starts with a simple formula: risk = likelihood × impact. It doesn't just ask "What vulnerabilities exist?" – it asks "Which ones can be used to reach our most important assets, and what would happen if they did?" To shift to meaningful metrics, focus on these five:

1. Risk score (tied to business impact) - A good risk score considers exploitability, asset criticality, and potential impact. It should change as threats evolve. This score helps leaders understand security in business terms – not just how many vulnerabilities exist, but how close we are to a real breach.
2. Critical asset exposure (tracked over time) - Not all assets are equal. You need to know which critical systems are exposed and how that exposure is changing. Are you reducing risk to your most important infrastructure, or just fixing low-impact issues? Tracking this over time shows if your security program is working.
3. Attack path mapping - Vulnerabilities don't exist alone. Attackers chain together vulnerabilities – misconfigurations, overprivileged accounts, unpatched CVEs – to reach valuable targets. Mapping these paths shows how an attacker could move through your environment. It helps prioritize not just individual issues, but how they create a threat.
4. Exposure class breakdown - You need to know what types of vulnerabilities are most common and dangerous. Whether it's credential misuse, missing patches, open ports, or cloud misconfigurations, this breakdown guides your response and planning. If 60% of your risk is from identity-based vulnerabilities, that should influence your decisions.
5. Mean Time to Remediate (MTTR) for critical exposures - Average MTTR is misleading. It's skewed by easy fixes and ignores the hard problems. What matters is how fast you're fixing the vulnerabilities that put you at risk. MTTR for critical exposures – those connected to exploitable attack paths or key assets – defines your operational effectiveness.

Meaningful metrics, updated regularly, give you a living view of your threat exposure. They turn security reporting into strategic insight. And they give security teams and business leaders a common language for making informed decisions.

The Bottom Line

Vanity metrics are comforting. They fill dashboards and suggest progress. But in the real world, where attackers don't care how many patches you applied, they offer little protection.

Real security means focusing on what matters, not just what's easy to measure. It means using metrics based on business risk. And frameworks like CTEM can help.