Webhooks as Lightweight SOAR: Patterns, Signatures and Idempotency

SOAR without the platform

A full SOAR deployment is a project: connectors, playbook designers, a case model, licences, a team to maintain it. For many SOCs the real automation need is smaller. When X happens, enrich it, decide something simple, and tell the right channel. That is an event, a function and an output.

Webhooks are the cheapest way to wire that up. The sending system makes an HTTP POST to your URL when something happens; your receiver does the work. No polling, no shared database, no vendor-specific playbook language. If you are still deciding whether you need the full platform, /compare/siem-vs-soar lays out what each layer is for.

The catch is that a webhook receiver is an internet-facing endpoint that triggers actions. Built carelessly, it is a way for anyone to inject fake alerts into your SOC, or for one network blip to create the same ticket forty times. The patterns below avoid both.

Patterns that work

Enrich and route

The most common and most valuable pattern. An alert or a finished scan arrives, the receiver extracts indicators, enriches them, and routes by result: high-confidence malicious to the on-call channel, unknown to the queue with context attached, known-benign to a log.

Fan-out

One event, several consumers: post to chat, open a ticket, append to a case, write to the SIEM. Keep each consumer independent so that a broken ticketing API does not stop the chat notification.

Gate and approve

The receiver prepares an action (block this domain, disable this account) and posts it to a channel with approve and reject buttons. The action only runs on a human click. This is where lightweight automation earns trust: the boring work is done, the risky decision stays human.

Scheduled digests

Not every event deserves a page. Accumulate low-severity events and emit one digest per shift. Your on-call engineer will thank you, and our piece on alert fatigue explains why this matters more than it looks.

Rule one: verify the signature

Your webhook URL will leak. It ends up in logs, screenshots, browser history and someone's config repo. Security cannot rest on the URL being secret. It rests on the sender signing each request so you can prove where it came from.

The standard approach is an HMAC: the sender and receiver share a secret; the sender computes HMAC-SHA256(secret, payload) and puts it in a header; the receiver recomputes it over the exact bytes it received and compares.

Details that break real implementations:

• Sign the raw body. Verify against the bytes off the wire, before any JSON parsing. Re-serializing parsed JSON changes whitespace and key order, and the signature will not match (or worse, you will "fix" it by signing your own re-serialization).
• Compare in constant time. Use hmac.compare_digest, not ==. Ordinary string comparison returns early and leaks timing information.
• Include a timestamp. If the signature covers only the body, an attacker who captures one valid request can replay it forever. Sign timestamp + "." + body and reject requests outside a short window (five minutes is common).
• Rotate secrets. Support two valid secrets during rotation so you can roll without downtime.

Header names and the exact signing string vary between providers, so check your sender's documentation rather than assuming the scheme below matches it.

Rule two: assume every event arrives more than once

Webhook delivery is almost always at-least-once. The sender POSTs, your receiver processes it, the response is lost on the way back, the sender times out and retries. From your side, the same event arrived twice. Network timeouts, sender restarts and your own deploys all cause this.

So the receiver must be idempotent: processing the same event twice has the same effect as processing it once. The standard technique is a delivery or event ID from the sender, stored with a unique constraint. Second arrival: acknowledge, do nothing.

The other half is the response contract:

The key move is acknowledge fast, process later. Store the event, return 202, then do the slow enrichment in a worker. If you enrich inline and the enrichment API is slow, the sender times out, retries, and you now have duplicate work and a backlog.

A small receiver in Python

Standard library only, so you can read every line. It verifies a timestamped HMAC, rejects replays and oversized bodies, stores each event once in SQLite, and processes it in a background worker. The header names are placeholders; map them to your sender's.

import hashlib, hmac, json, os, sqlite3, threading, time
from http.server import BaseHTTPRequestHandler, HTTPServer

SECRETS = [s.encode() for s in os.environ["WEBHOOK_SECRETS"].split(",")]  # current,previous
TOLERANCE = 300          # seconds
MAX_BODY = 1_000_000     # bytes
DB = "webhooks.db"

def init_db():
    with sqlite3.connect(DB) as c:
        c.execute("""CREATE TABLE IF NOT EXISTS events (
            id TEXT PRIMARY KEY, body TEXT, status TEXT, received INTEGER)""")

def verify(body: bytes, ts: str, sig: str) -> bool:
    try:
        if abs(time.time() - int(ts)) > TOLERANCE:
            return False
    except ValueError:
        return False
    sig = sig.removeprefix("sha256=")
    msg = ts.encode() + b"." + body
    return any(
        hmac.compare_digest(hmac.new(k, msg, hashlib.sha256).hexdigest(), sig)
        for k in SECRETS
    )

class Handler(BaseHTTPRequestHandler):
    def reply(self, code):
        self.send_response(code)
        self.end_headers()

    def do_POST(self):
        length = int(self.headers.get("Content-Length") or 0)
        if length <= 0 or length > MAX_BODY:
            return self.reply(413)
        body = self.rfile.read(length)
        if not verify(body, self.headers.get("X-Webhook-Timestamp", ""),
                      self.headers.get("X-Webhook-Signature", "")):
            return self.reply(401)
        try:
            event_id = str(json.loads(body)["id"])
        except (ValueError, KeyError):
            return self.reply(400)
        try:
            with sqlite3.connect(DB) as c:
                c.execute("INSERT INTO events VALUES (?, ?, 'pending', ?)",
                          (event_id, body.decode(), int(time.time())))
        except sqlite3.IntegrityError:
            return self.reply(200)   # duplicate delivery: already have it
        self.reply(202)

def handle(event: dict):
    # Enrich, route, notify. Keep side effects idempotent too.
    print("processing", event.get("type"), event.get("id"))

def worker():
    c = sqlite3.connect(DB)
    while True:
        row = c.execute("SELECT id, body FROM events WHERE status='pending' "
                        "ORDER BY received LIMIT 1").fetchone()
        if not row:
            time.sleep(1)
            continue
        try:
            handle(json.loads(row[1]))
            status = "done"
        except Exception as e:
            print("failed", row[0], e)
            status = "failed"
        with c:
            c.execute("UPDATE events SET status=? WHERE id=?", (status, row[0]))

if __name__ == "__main__":
    init_db()
    threading.Thread(target=worker, daemon=True).start()
    HTTPServer(("127.0.0.1", 8080), Handler).serve_forever()

Run it behind a reverse proxy that terminates TLS; never expose plain HTTP. To test, sign a payload yourself:

TS=$(date +%s)
BODY='{"id":"evt_001","type":"scan.completed","domain":"example[.]com"}'
SIG=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -hex | awk '{print $NF}')
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:8080/ \
  -H "X-Webhook-Timestamp: $TS" -H "X-Webhook-Signature: sha256=$SIG" \
  --data "$BODY"

Send it twice: the first returns 202, the second 200 and nothing is processed again. Change one character of the body and you get 401.

What this deliberately leaves out: a single worker and SQLite cap throughput at what one process handles, which is plenty for most SOC event volumes. When it is not, swap the table for a real queue. Failed events stay in the table with status='failed', so add a retry job or a daily review before you depend on it.

Operating it

• Monitor the boring numbers. Events received, rejected (401/400), duplicates, pending backlog, failures. A spike in 401s is either a rotated secret or someone probing your endpoint.
• Restrict source IPs if the sender publishes them, as defence in depth on top of signatures, not instead of them.
• Version your payload handling. Senders add fields; code that rejects unknown fields breaks on a Tuesday for no visible reason.
• Keep a replay tool. Being able to re-run a stored event through handle() after fixing a bug is worth its few lines.

For the outbound side, sending enrichment requests from your receiver, see automating IOC enrichment with the API; /integrations lists the ways mlab.sh connects to other tools, including webhooks.

Checklist

• [ ] Every webhook verified with HMAC over the raw body, constant-time compare.
• [ ] Timestamp inside the signature, with a short tolerance window.
• [ ] Two-secret support for rotation; secrets in env or a secret store.
• [ ] Event ID with a unique constraint; duplicates acknowledged and ignored.
• [ ] Acknowledge after durable storage, process in a worker.
• [ ] Destructive actions gated behind a human approval.
• [ ] Metrics on rejects, duplicates and backlog.

That is a SOAR-shaped system in under a hundred lines you fully understand. Grow it only when it hurts.