The hour that decides everything

Every incident responder eventually learns the same lesson: investigations are rarely lost in week two. They are lost in hour one, when a well-meaning admin rebooted the compromised server, or reset the attacker's password while the attacker was watching, or spent forty minutes debating severity in a chat channel that the attacker could read.

Under stress, people do not rise to the occasion. They fall to the level of their preparation. That is the whole argument for a checklist: not because the steps are hard, but because at 2 a.m. with an executive on the phone, you will forget one of them. Pilots with ten thousand hours still run the pre-landing checklist. So should you.

What follows is a first-hour sequence you can adapt, print, and actually follow.


Minute 0-10: Validate before you escalate

Most alerts are not incidents. The first job is deciding whether this one is real, without contaminating anything in the process.

  • Read the raw alert, not the summary. What exactly fired? Which host, which account, which detection logic, what time (note the timezone, seriously)?
  • Corroborate with a second source. One EDR alert is a claim. The same behavior visible in proxy logs, authentication logs, or a second sensor is a fact. Enrich the external indicators as you go: a quick lookup of the IP, domain, or hash on mlab.sh tells you in seconds whether that "suspicious" address is a known C2 or just a CDN node.
  • Check for benign explanations. Change tickets, patch windows, pentests, that one admin who runs PowerShell like a caveman. Thirty seconds in the change calendar has saved countless incident bridges.
  • Do it read-only. Query logs and consoles. Do not touch the suspect host yet, do not scan it, do not log into it "just to look around."

If it is a false positive, document why and close it. If you cannot prove it is benign in ten minutes, treat it as real and move.

The mistake to avoid: validating by doing things on the compromised host. Every command you run overwrites timestamps, pushes evidence out of short log buffers, and can trip attacker tooling that watches for investigation.


Minute 10-20: Declare, and open one channel

The moment you decide "this is probably real," three things happen in quick succession:

  1. Declare an incident. Say the words. An ambiguous "we're looking into something weird" produces ambiguous effort. A declared incident with a severity level produces a response.
  2. Name an incident lead. One person owns decisions and delegation for now. Not necessarily the most senior person; the most available competent one. Everyone else reports findings to the lead instead of freelancing.
  3. Open a dedicated comms channel, and consider it burned if it lives on the affected infrastructure. If there is any chance the attacker has mail or chat access (assume yes for account compromise and BEC), coordinate out-of-band: a separate tenant, a phone bridge, an external channel. Attackers reading the defenders' chat is not a hypothetical; it is a recurring feature of post-incident reports.

Keep the initial circle small: responders, the lead, and whoever must approve containment. Wide announcements come later, deliberately.


Minute 20-40: Scope and preserve, in parallel

Now split the effort. Half the brainpower scopes the blast radius; the other half locks down evidence before it evaporates.


Scoping questions, in order

  • Which accounts? The alerting account, plus anything it authenticated to since first suspicious activity.
  • Which hosts? Pivot on the indicators you have: the C2 address across firewall and proxy logs, the file hash across the EDR fleet, the account across authentication logs.
  • Since when? Find the earliest suspicious event, then look one week further back. Your first "patient zero" is usually just the first place you happened to look.
  • What could it reach? What data and privileges do the affected accounts and hosts touch? This drives both containment urgency and, later, notification duties.

You will not finish scoping in twenty minutes. You are drawing the first circle, knowing it will grow.


Preservation, before containment destroys it

Evidence has a half-life, and the most volatile evidence is the most valuable:

  • Do not power off machines. Memory holds process lists, network connections, injected code, and encryption keys that exist nowhere on disk. Pulling the plug destroys them forever. Isolate instead.
  • Capture memory now if you have the tooling, especially on the initial host.
  • Snapshot VMs (with memory) before touching them.
  • Export and pin logs: EDR telemetry, firewall, VPN, identity provider, cloud audit. Short retention windows and rotating buffers are quietly deleting your case while you deliberate.
  • Photograph or screenshot anything ephemeral: ransom notes on screens, odd console sessions, attacker chat windows.

The mistake to avoid: the instinct to "clean up." Deleting the webshell, killing the process, wiping the host. It feels like progress; it is evidence destruction plus a signal flare telling the attacker you found them, while they still hold three other footholds you have not discovered yet.


Minute 40-55: Decide containment deliberately

Containment is a decision, not a reflex, and it has a real trade-off. Contain too early and you tip off the attacker before you know their full footprint; they go quiet, or go loud. Contain too late and the damage compounds.

A workable rule of thumb:

Situation Bias
Active destruction or encryption underway Contain immediately, everywhere you can
Data theft likely in progress Contain fast, prioritize egress and the accounts involved
Foothold present, attacker quiet Preserve first, contain in one coordinated move

Prefer containment that is quiet and reversible: network isolation via EDR keeps the host alive for forensics while cutting attacker access. Disable accounts rather than resetting passwords one by one over an hour; a partial credential response teaches the attacker to dig in with what remains. When you do move against credentials, hit everything in the known scope at once, sessions and tokens included.

Whatever you choose, the incident lead says it out loud, someone writes it down, and everyone executes together.


Minute 55-60: Start the timeline log, then keep it alive

If you do only one administrative thing in the first hour, do this: open a timeline log and appoint a scribe. Every entry gets a timestamp (pick UTC, state it once), an author, and a fact or decision:

14:32 UTC [scribe: MK] EDR alert: cobalt strike beacon behavior, host FIN-SRV-02
14:41 UTC [scribe: MK] Corroborated: proxy shows FIN-SRV-02 -> 45.x.x.x since 13:07
14:47 UTC [lead: JD] Incident declared, SEV-2. Bridge opened (out-of-band).
14:58 UTC [scribe: MK] Memory capture of FIN-SRV-02 started. Host NOT rebooted.
15:09 UTC [lead: JD] Decision: EDR network isolation, no credential resets yet.

This log is what turns chaos into an investigation. It prevents the same host being "checked" four times, it is the raw material for regulatory notifications with legal deadlines (GDPR's 72-hour clock starts at awareness, and you will need to show when awareness happened), and it is the backbone of the post-incident review. Memory is not evidence. The log is.

This is also where tooling stops being optional. A shared doc works at 2 a.m. exactly until two people edit it at once and a third forgets which version is real. A dedicated IR platform like mlab IR keeps the timeline, task assignments, and evidence references in one workflow, self-hosted so the incident record does not live on the infrastructure you are investigating.


The printable version

  • [ ] Validate with a second source, read-only; check change calendar
  • [ ] Declare the incident, assign a lead, set severity
  • [ ] Open out-of-band comms; assume attacker reads internal channels
  • [ ] Scope: accounts, hosts, earliest activity, reachable data
  • [ ] Preserve: no power-off, capture memory, snapshot VMs, pin logs
  • [ ] Decide containment deliberately; prefer quiet, reversible, coordinated
  • [ ] No cleanup, no one-off password resets, no "just reimage it"
  • [ ] Start the UTC timeline log, appoint a scribe, record every decision

The first hour is not about being brilliant. It is about not making the four or five mistakes that can never be walked back. Print the checklist before you need it, because the incident will not wait while you write one.