The problem with measuring a SOC

Every SOC has a dashboard. Most of them measure the wrong things.

Ticket counts, closure rates, and mean time to respond look objective. They are also trivially gameable, and analysts under pressure will game them without even meaning to. A metric that can be improved by doing worse security work is not a metric. It is a trap.

The goal of SOC measurement is simple to state: detect real intrusions faster, respond to them better, and prove it. Every number on your dashboard should trace back to one of those three things. If it does not, delete it.


Metrics that incentivize bad behavior

Start by understanding what to remove. These are the usual suspects.


Raw ticket counts

"We handled 14,000 alerts this month" tells leadership nothing except that your detection pipeline is noisy. Worse, it rewards noise: the easiest way to grow the number is to deploy bad rules. A SOC that tunes its detections properly will see this number go down, and a naive reader will interpret that as the team doing less work.


Closure rate and time-to-close

If analysts are measured on how fast they close tickets, they will close tickets fast. That means shallow triage, copy-pasted closure notes, and true positives marked benign because digging deeper would hurt the stats. Speed of closure says nothing about quality of closure.


MTTR as a single number

Mean time to respond is not useless, but averaged across all severities and all alert types it hides everything interesting. One commodity phishing email resolved in four minutes and one domain-wide compromise resolved in three weeks average out to a number that describes neither. And once MTTR becomes a target, the gaming starts: reclassify hard incidents, split them into sub-tickets, close-then-reopen. The metric survives. The meaning does not.

Metric What it claims to measure What it actually incentivizes
Ticket volume Workload Noisy detections
Closure rate Efficiency Shallow triage
Average MTTR Response speed Reclassification and ticket-splitting
Alerts closed per analyst Productivity Racing through the queue

Metrics that improve a SOC

Now the ones worth the effort. Each of these is harder to collect than a ticket count, and each one is worth ten of them.


Detection coverage

Map your active detections against MITRE ATT&CK techniques, weighted by the actors and techniques that actually target your sector. The honest version of this metric distinguishes "we have one brittle rule for T1059" from "we cover the common procedures." Track coverage of your priority techniques over time. This is the metric that tells you where you are blind, which is the single most valuable thing a SOC can know about itself.


Alert-to-incident ratio

Of everything that fired, how much was worth firing? A ratio of 5,000 alerts to 3 confirmed incidents means your pipeline is mostly noise, and your analysts know it even if your dashboard does not. Track this per detection rule, not just globally: it tells you exactly which rules to fix or delete. A rule that has fired 400 times without a single true positive is not a detection. It is a tax on your analysts.


Triage quality, not triage speed

Sample closed alerts every week and re-review them. What percentage were correctly classified? Were escalation decisions right? This is the SOC equivalent of code review, and it is the only reliable way to catch the failure mode that matters most: true positives closed as benign. A 2% misclassification rate on 10,000 alerts is 200 missed opportunities to catch an intrusion early.


Feedback loop latency

When an analyst flags a rule as noisy, how long until detection engineering fixes or retires it? If the answer is "weeks" or "never," your analysts have already learned that reporting problems is pointless, and they have stopped. Measure the time from analyst feedback to rule change. Short loops are what separate SOCs that improve from SOCs that merely operate.


Hunt findings and detection conversions

Threat hunts produce two valuable outputs: incidents you would not have caught otherwise, and new detections derived from hunt logic. Count both. A hunting program that never converts findings into automated coverage is an expensive hobby.


Time to detect, by scenario

Instead of averaging response times across everything, measure detection and response against specific scenarios you care about: credential phishing, an internet-exposed service compromise, ransomware precursors. Purple-team exercises give you real numbers here. "We detect simulated credential dumping in under 10 minutes, up from 4 hours last quarter" is a sentence leadership can actually use.


Building a small honest dashboard

Resist the urge to track twenty things. A useful SOC dashboard fits on one page:

  1. Coverage: priority ATT&CK techniques covered, and the top three gaps.
  2. Signal quality: alert-to-incident ratio, plus the five worst-performing rules this month.
  3. Quality control: triage review sample size and misclassification rate.
  4. Improvement: rules added, tuned, and deleted; feedback loop latency.
  5. Outcomes: confirmed incidents, scenario-based detection times, hunt findings.

Note what is missing: ticket volume, closure rate, per-analyst throughput. If leadership asks for volume numbers, provide them as context, never as targets. The moment a number becomes a target, re-read the section on gaming above.

Enrichment automation helps more than it seems here. A large share of "triage time" is analysts manually looking up the same IPs, domains, and hashes over and over. Wiring enrichment into the pipeline through something like the mlab.sh API removes that dead time from every metric at once, and makes your triage-quality reviews cleaner because the context an analyst had is recorded with the alert.


Reporting to leadership

Executives do not want your metrics. They want answers to three questions: are we getting better, where are we exposed, and what do you need. Structure the report accordingly.

  • Lead with trend, not snapshot. Coverage up, noise down, detection times shrinking.
  • Show one gap and its cost. "We have no detection for X, which actor Y used against our sector twice this year."
  • Attach asks to metrics. "Feedback loop latency is 3 weeks because detection engineering is one person" is a budget argument that writes itself.

Never present a metric you are not willing to see gamed, because it will be. And when a number moves in the "wrong" direction for a good reason (alert volume dropping because you deleted garbage rules), say so explicitly. Teaching leadership to read SOC metrics for what they mean, not how they look, is part of the job.


Measure outcomes, sample quality, and keep the dashboard small. A SOC that tracks five honest numbers will outperform one drowning in forty flattering ones.