Security Automation & SOAR

Definition

Security Orchestration, Automation and Response (SOAR) is the discipline of wiring the SOC’s repeatable work into machine-executed playbooks: alerts become cases, cases trigger playbooks, playbooks orchestrate integrations (SIEM, EDR, IAM, firewall, mail, ticketing, threat intel) to enrich, triage, contain, and document — with humans making the consequential calls. It has three layers vendors blur and practitioners keep separate: orchestration (the reliable wiring between tools), automation (the machine-executed steps — enrichment, correlation, soft containment, notification), and response (the measured actions on the environment — isolation, blocking, disabling — where a mistake is an outage). The doctrine: automate the repeatable, gate the consequential, measure everything. The craft’s center: a playbook is a procedure that executes identically at 3 a.m. and 3 p.m. — automation is never better than the procedure underneath it, and automating a broken procedure produces failures faster and more consistently. SOAR sits downstream of BT1’s triage and BT3’s pipelines, consumes BT9’s detections as triggers, and hands cases to BT4’s process — connective tissue for a detection program, not a substitute for any of it.

Why It Matters

The math forces the issue. A monitored environment produces alert volume no staffing model absorbs: BT1’s triage queue shows the shape — thousands of daily detections, the overwhelming majority enrichment-and-close (reputation, asset context, dedupe). That enrichment work consumes the largest share of analyst time, is perfectly deterministic, and is exactly what machines do well — every hour spent copy-pasting hashes into a reputation portal is an hour subtracted from hunting and casework. Automation moves MTTA from minutes to seconds and makes response consistent — the same steps in the same order at 3 a.m. as at 3 p.m., which no tired human does.

The failure modes are why “just automate it” is not a plan. Auto-containment accidents: isolating a VIP’s laptop mid-board-meeting, blocking the corporate NAT and locking out a floor — containment actions are production changes, and ungated automation causes outages at machine speed. Silent brittleness: a vendor changes an API field or a token expires and playbooks fail silently — nothing pages when a non-critical automation stops running, and the drift surfaces during the incident that needed it. Flood exhaustion: the adversary sprays ten thousand low-quality alerts, the automation enriches each through rate-limited APIs, the quotas burn, and the real attack arrives into an exhausted pipeline. Sprawl and orphaning: four hundred playbooks, a third of which fire never and a third nobody has read since the author left — an unowned codebase decays. Procedure drift: the human process changed, the playbook didn’t, and the machine faithfully executes last year’s wrong answer at scale.

The governance answer is to treat playbooks exactly like production code: versioned, reviewed, tiered by blast radius, tested before deployment, monitored in operation, every consequential action gated and reversible. That governance — not the integrations, not the platform — is the discipline this module teaches.

Detailed Explanation

   SOAR OPERATING STACK
   +------------------------------------------------------+
   | TRIGGERS    SIEM rules (BT3), EDR detections, manual |
   |             request, webhook, schedule — each with   |
   |             a severity floor, dedupe window, and     |
   |             per-playbook quota (flood control first) |
   +------------------------------------------------------+
   | CASE MANAGEMENT   alerts deduped/correlated into     |
   |   cases; every automated action lands on the case    |
   |   with actor=playbook — the case IS the audit surface|
   +------------------------------------------------------+
   | PLAYBOOK ENGINE   trigger -> conditions -> typed     |
   |   steps -> gates; idempotent steps, timeouts,        |
   |   retries, rollback definitions, full run traces     |
   +------------------------------------------------------+
   | INTEGRATION LAYER   enrichment (reputation, CMDB,   |
   |   intel), containment (EDR isolate, FW block, AD     |
   |   disable, mail purge), comms (ticket, chat, page)   |
   |   — every integration carries a risk class           |
   +------------------------------------------------------+
   | HUMAN GATES   approval for consequential actions,    |
   |   on-call reachability, break-glass manual mode      |
   +------------------------------------------------------+

Playbook anatomy. A production playbook has seven parts, and omitting any of them is how the accidents happen. The trigger (SIEM rule, EDR detection, webhook, manual button) carries a severity floor and a dedupe window so one detection storm doesn’t multiply into a thousand runs. The preconditions (field checks before any action). The steps, typed by intent: enrich, decide, act, notify. The gates — steps that pause for human approval, with a timeout that fails closed (no approval, no action, case escalates). The rollback — for every containment step, the documented inverse action. The timeouts and retries on every network call (an integration that hangs must not hang the playbook). And the run trace — every step, input, output, and decision recorded on the case, because unauditable automation is untrustable. Idempotency is the quiet requirement underneath: a playbook re-run against the same alert must not double-block, double-purge, or double-notify.

Automation tiers. The blast radius of a step’s failure determines its tier, and the tier determines the gate. Tier 0 — read-only enrichment: reputation lookups, CMDB joins, intel matches; fully automatic (the failure mode is a wasted lookup). Tier 1 — soft containment: tagging a host “suspect”, adding a watchlist entry, quarantining a message to one mailbox; automatic, with post-hoc review. Tier 2 — hard containment: host isolation, IP blocks, account suspension; gated — the playbook assembles the case and pauses for approval, with a narrowly defined auto-approve criteria for the well-understood subset. Tier 3 — irreversible or broadly destructive: org-wide mail purge, firewall changes on shared infrastructure, anything touching executives or production data paths; human-executed, the playbook doing the legwork and the human pressing the button. The tier assignment is written per playbook and reviewed — the gate IS the tier, and a playbook whose gate structure doesn’t match its tier is the lint finding soarlint exists to catch.

Case management is the join. Alerts are deduplicated and correlated into cases (BT1’s triage produces the correlation logic); every automated action lands on the case with actor=playbook; analyst notes and gate approvals append to the same object. The case is what makes automation auditable — the post-incident review asks “what did the automation do and why” and the run trace answers with data, not memory.

Quota guards and circuit breakers. Every playbook carries a per-hour run quota and a per-target rate limit; the integration layer tracks third-party API budgets; a breaker trips when runs exceed threshold — better an unenriched alert than an exhausted API quota when the real attack lands. Quota exhaustion is itself an alert, because an adversary who can burn your enrichment budget controls your visibility.

Deep dive: the auto-approve criteria — where automation earns trust

Tier-2 gates beg the obvious question: if a human approves every isolation, why automate? Because the approval queue becomes the new bottleneck — the answer is narrow auto-approve criteria, written and reviewed like detection logic. A defensible example: auto-approve host isolation only when ALL hold — high-fidelity detection source (BT9’s curated rules, not a noisy analytic); the target a workstation, never a server; the user not on the VIP/executive exclusion list; and the action time-boxed (auto-restore after N hours unless extended — the worst-case false positive becomes a bounded outage). The criteria are versioned; every auto-approval lands on the case tagged auto for weekly review. Trust is earned per-detection-class by false-positive history — thirty clean auto-approvals keep the criteria; one wrong laptop loses the privilege. The contract between BT1’s triage quality and automation speed: the cleaner the detection, the wider the auto-approve.

Common Techniques

Technique	What it does	Where it fails	Notes
Trigger severity floors	Playbook wakes only above threshold	Wrong floor = noise or blindness	Tune per class
Dedupe windows	One storm = one run	Missing = multiplied runs	Standard on triggers
Tier gates + auto-approve	Human gate on consequential actions	Criteria too broad	Versioned, FP-reviewed
Idempotent steps	Re-run safe	Check-then-act races	Required for act steps
Rollback definitions	Every containment has an inverse	Undo never tested	Drill the undo path
Run traces on cases	Full automation audit	Trace gaps after API change	BT3 join
Quota + circuit breakers	Flood/exhaustion control	Static thresholds only	BT1 flood pattern
Stage dry-runs	Tested on synthetic alerts	Prod-only testing	PT4 validation
Playbook-as-code	Versioned, reviewed, diffable	Console-edited sprawl	Git is the registry

Full Working Code

soarlint — a static validator for SOAR playbook definitions, the “playbooks are code” discipline made executable: reads a playbook registry (JSON) and enforces the structural contract — approval gates on containment, registered undo on destructive actions, timeouts on network steps, quota on every playbook, no phantom integrations. It is the lint pass that runs in CI before a playbook change reaches stage.

#!/usr/bin/env python3
"""soarlint — static validation for SOAR playbook definitions.

Input playbooks.json: {"integrations": {...}, "playbooks": [...]}
  integrations: name -> {"risk": read|comms|contain|destructive,
                         "network": bool, "undo": <name or null>}
  playbooks: name, trigger, severity_min, quota_per_hour, steps[]
Validation classes: STRUCT, INTEGRATION, GATE, ROLLBACK, TIMEOUT, QUOTA
Exit 0 clean, 1 findings, 2 input error. Stdlib only."""
import argparse
import json
import sys
from collections import Counter
from pathlib import Path

TRIGGERS = {"siem", "edr", "manual", "webhook", "schedule"}
RISKS = {"read", "comms", "contain", "destructive"}


def load(path):
    if not path.exists():
        sys.exit(f"fail-loud: playbook file not found: {path}")
    try:
        return json.loads(path.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        sys.exit(f"fail-loud: unparseable JSON: {exc}")


def main():
    ap = argparse.ArgumentParser(description="SOAR playbook linter")
    ap.add_argument("playbooks", type=Path,
                    help="playbook registry JSON")
    ap.add_argument("--out", type=Path, default=Path("soarlint_out"))
    args = ap.parse_args()
    data = load(args.playbooks)
    integrations = data.get("integrations")
    playbooks = data.get("playbooks")
    if not isinstance(integrations, dict) or not integrations:
        sys.exit("fail-loud: integrations registry missing or empty")
    if not isinstance(playbooks, list) or not playbooks:
        sys.exit("fail-loud: playbooks list missing or empty")

    findings = []
    for pb in playbooks:
        name = pb.get("name") or "<unnamed>"
        if name == "<unnamed>":
            findings.append((name, "STRUCT", "playbook missing 'name'"))
        if pb.get("trigger") not in TRIGGERS:
            findings.append((name, "STRUCT",
                             f"bad trigger {pb.get('trigger')!r}"))
        steps = pb.get("steps")
        if not steps or not isinstance(steps, list):
            findings.append((name, "STRUCT", "no steps — not automatable"))
            continue
        sev = pb.get("severity_min")
        if not isinstance(sev, int) or not 1 <= sev <= 10:
            findings.append((name, "STRUCT",
                             f"severity_min {sev!r} not an int 1-10"))
        if not pb.get("quota_per_hour"):
            findings.append((name, "QUOTA",
                             "no quota_per_hour — flood exposure"))
        for i, st in enumerate(steps, 1):
            where = f"{name}[{i}]"
            integ = st.get("integration")
            if integ not in integrations:
                findings.append((where, "INTEGRATION",
                                 f"{integ!r} not in registry"))
                continue
            spec = integrations[integ]
            risk = st.get("risk") or spec.get("risk")
            if risk not in RISKS:
                findings.append((where, "STRUCT",
                                 f"bad risk {risk!r}"))
                continue
            if risk in ("contain", "destructive") and not st.get("approval"):
                findings.append((where, "GATE",
                                 f"{risk} action without approval gate"))
            if risk == "destructive":
                undo = st.get("undo") or spec.get("undo")
                if not undo:
                    findings.append((where, "ROLLBACK",
                                     "destructive action with no undo"))
                elif undo not in integrations:
                    findings.append((where, "ROLLBACK",
                                     f"undo {undo!r} not registered"))
            if spec.get("network") and not st.get("timeout_s"):
                findings.append((where, "TIMEOUT",
                                 "network-bound action without timeout_s"))

    args.out.mkdir(parents=True, exist_ok=True)
    with (args.out / "findings.csv").open("w", encoding="utf-8") as fh:
        fh.write("playbook,kind,detail\n")
        for pb, kind, detail in findings:
            fh.write(f'{pb},{kind},"{detail}"\n')

    kinds = Counter(k for _, k, _ in findings)
    verdict = "CLEAN" if not findings else f"{len(findings)} findings"
    print(f"playbooks={len(playbooks)}  integrations="
          f"{len(integrations)}  verdict={verdict}  "
          f"{dict(kinds) if kinds else ''}")
    for pb, kind, detail in findings[:12]:
        print(f"  [{kind:<12}] {pb:<26} {detail[:52]}")
    print(f"register: {args.out / 'findings.csv'}")
    print("doctrine: a playbook is a production change — version it, "
          "gate it, test it in stage, give every containment an undo")
    sys.exit(1 if findings else 0)


if __name__ == "__main__":
    main()

Code Walkthrough

The registry contract (lines 118-124) is fail-loud on the two inputs that make linting meaningless: an empty integration registry (lint against nothing) and an empty playbooks list (the tool can’t tell “no playbooks” from “wrong file”). The playbook-level sweep (lines 128-141) checks name, valid trigger, non-empty steps, integer severity floor; the QUOTA check (lines 142-144) enforces the flood doctrine at lint time — no per-hour quota means flood-exposed by construction, failing the build instead of discovering it during an alert storm.

The per-step sweep (lines 145-171) is where the tier doctrine becomes lint rules. Unknown integrations fail first (lines 147-151) — the silent-brittleness class: a renamed integration leaves steps pointing at nothing. The GATE rule (lines 158-160) encodes tier discipline: contain and destructive steps without approval: true fail — the rule that prevents the ungated auto-isolation. The ROLLBACK rule (lines 161-168) checks the full undo chain — the step or the registry must name an undo that itself exists in the registry; an undo pointing at a retired integration is rollback theater. The TIMEOUT rule (lines 169-171) keeps a hanging API call from hanging the playbook. Exit 1 on findings (line 189) makes the tool CI-native — the change that drops a gate breaks the pipeline like a failing unit test.

How It Reaches The Target

The weekly rhythm: playbook changes arrive as pull requests (the diff is the review artifact), soarlint runs in CI, and a clean lint promotes the change to stage, where PT4’s synthetic alerts exercise it before production enable. In production every run lands its trace on the case; the weekly review samples traces per playbook (failure rates, gate timing, quota headroom), and FP history per detection class tunes auto-approve. soarlint’s INTEGRATION class catches static breaks and the health probes catch dynamic ones (expired tokens, changed endpoints) — both feed the same on-call path, because silent automation failure is the state that must never persist.

The incident that exercises everything: a credential-abuse case at 02:14. The SIEM rule (BT3) fires above the severity floor; the triage playbook enriches automatically (reputation, CMDB, intel matches — tier 0, seconds); the containment playbook proposes host isolation, and because the detection class has a clean FP history and the target is a non-VIP workstation, the criteria auto-approve — the host is isolated time-boxed by 02:16, the page goes to the on-call analyst who hands to BT4’s process. The morning review walks the trace — every step, decision, and approval — and the one SKIP (the server-check condition on a decoy host) documents why the second playbook never fired. The measure loop closes in BT20: automation rate, MTTA/MTTR deltas, gate override counts, auto-approve FP rate — the numbers that decide next quarter’s automation investments.

Exploitation Chain

Inventory — enumerate integrations and credentials; classify risk (read/comms/contain/destructive)
Codify — BT1’s top recurring procedures → playbook specs with typed steps
Gate — assign tiers; narrow auto-approve criteria; undo + timeout on every containment
Lint — soarlint in CI; all six classes clean
Test — stage dry-runs on synthetic alerts (PT4); traces reviewed before enable
Deploy — quotas, dedupe windows, breakers; traces land on cases
Operate — weekly trace review; FP history tunes auto-approve; integration health probes
Measure — BT20: automation rate, MTTA/MTTR delta, gate overrides, auto-approve FP rate

Building Your Own Tool

Linting catches structural defects; it cannot tell you what the playbook would do to this alert. playrun is the dry-run simulator: walks a playbook’s steps against a sample alert without executing anything, evaluating the trigger and each step’s when condition, and recording a trace — EXECUTE (would run now), GATE (would pause for approval), SKIP (condition false), HALT (cannot evaluate — missing field, malformed condition). It is the stage harness: before any playbook enables, it runs against a synthetic-alert corpus, and the traces are the review artifact.

Design decisions

Trace-only by construction: no action layer — it cannot isolate a host because it contains no code path that could, so it is safe to run anywhere, including on production copies of real alerts. Conditions use a deliberately tiny grammar (field op value, eight operators): a dry-run interpreter that can mis-evaluate is worse than no dry-run, and the HALT-on-anything-unclear rule means ambiguity stops the trace loudly instead of guessing. Type-aware comparison: if the alert field is an integer the literal coerces, so "severity >= 8" compares numerically — the bug class that quietly inverts conditions. The trigger evaluation mirrors soarlint’s severity-floor contract so the tools agree on what a playable alert looks like, and the trace records why each step took its verdict.

Full source

#!/usr/bin/env python3
"""playrun — dry-run simulator for SOAR playbooks against a sample alert.

Walks a playbook's steps against alert.json WITHOUT executing anything:
  EXECUTE  step would run now (condition passed, no gate)
  GATE     step would pause for human approval
  SKIP     step's `when` condition evaluated false
  HALT     trace stopped (missing alert field / bad condition)
Playbook JSON: {name, trigger, severity_min, steps: [{action,
                integration, approval, when}]}
Alert JSON: {severity, ...any fields conditions reference}
Condition grammar: "field op value"; ops == != > < >= <= contains startswith
Stdlib only. Trace only — the tool cannot perform actions."""
import argparse
import json
import sys
from pathlib import Path

OPS = {
    "==": lambda a, b: a == b,
    "!=": lambda a, b: a != b,
    ">": lambda a, b: a > b,
    "<": lambda a, b: a < b,
    ">=": lambda a, b: a >= b,
    "<=": lambda a, b: a <= b,
    "contains": lambda a, b: str(b) in str(a),
    "startswith": lambda a, b: str(a).startswith(str(b)),
}


def load(path, label):
    if not path.exists():
        sys.exit(f"fail-loud: {label} not found: {path}")
    try:
        return json.loads(path.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        sys.exit(f"fail-loud: {label} unparseable JSON: {exc}")


def resolve(alert, expr):
    """Evaluate 'field op value'. Returns (True/False/None, note)."""
    parts = expr.split()
    if len(parts) < 3:
        return None, f"malformed condition {expr!r}"
    field, op = parts[0], parts[1]
    value = " ".join(parts[2:]).strip("'\"")
    if field not in alert:
        return None, f"alert lacks field {field!r}"
    if op not in OPS:
        return None, f"unknown op {op!r}"
    try:
        left = alert[field]
        if isinstance(left, (int, float)):
            right = type(left)(value)
        else:
            right = value
        return OPS[op](left, right), "ok"
    except (TypeError, ValueError) as exc:
        return None, f"compare failed: {exc}"


def main():
    ap = argparse.ArgumentParser(description="SOAR playbook dry-run")
    ap.add_argument("playbook", type=Path)
    ap.add_argument("--alert", type=Path, required=True)
    ap.add_argument("--out", type=Path, default=Path("playrun_out"))
    args = ap.parse_args()

    pb = load(args.playbook, "playbook")
    alert = load(args.alert, "alert")

    trace = []
    sev = alert.get("severity")
    sev_min = pb.get("severity_min", 0)
    if sev is None:
        trace.append(("HALT", "trigger",
                      "alert has no severity — cannot match trigger"))
        matched = False
    else:
        matched = sev >= sev_min
        verdict = "MATCH" if matched else "SKIP"
        trace.append((verdict, "trigger",
                      f"alert.severity={sev} vs severity_min={sev_min}"))

    if matched:
        for st in pb.get("steps", []):
            action = st.get("action", "<step>")
            cond = st.get("when")
            if cond:
                ok, note = resolve(alert, cond)
                if ok is None:
                    trace.append(("HALT", action, note))
                    break
                if not ok:
                    trace.append(("SKIP", action,
                                  f"when {cond!r} false"))
                    continue
            verdict = "GATE" if st.get("approval") else "EXECUTE"
            note = st.get("integration", "-")
            if cond:
                note += f" (cond passed: {cond})"
            trace.append((verdict, action, note))

    counts = {}
    for v, _, _ in trace:
        counts[v] = counts.get(v, 0) + 1
    args.out.mkdir(parents=True, exist_ok=True)
    (args.out / "trace.json").write_text(
        json.dumps({"playbook": pb.get("name"), "alert": alert,
                    "trace": trace}, indent=2),
        encoding="utf-8")

    summary = "  ".join(f"{k}={v}" for k, v in sorted(counts.items()))
    print(f"playbook={pb.get('name')}  {summary}")
    for v, step, note in trace:
        print(f"  {v:<8} {step:<24} {note[:56]}")
    print(f"trace: {args.out / 'trace.json'}")
    print("dry-run only — nothing executed; enable in production only "
          "after traces look right on the synthetic corpus")


if __name__ == "__main__":
    main()

Walkthrough

The operator table (lines 250-259) is the whole condition language — eight lambdas, deliberately closed, so a condition either evaluates or HALTs; contains/startswith stringify both sides because integration field types are unpredictable. The resolver (lines 271-290) implements type-aware comparison — the alert field’s type wins (lines 283-287), the literal coerces — which prevents "severity >= 10" being true for severity: 9 under string comparison. Fail paths return (None, note); the walk loop turns any None into a HALT and stops the trace (lines 320-324) — an ambiguous condition poisons everything after it, so the trace refuses to guess.

The trigger evaluation (lines 303-314) mirrors soarlint’s severity-floor contract and produces the first trace line; a skipped trigger ends the walk there — the honest output for an unplayable alert. The step walk (lines 316-333) implements the three-way verdict: condition false → SKIP; approval flag → GATE; otherwise EXECUTE. The trace lands as JSON on disk (lines 335-342) so a corpus of dry-runs becomes a reviewable artifact set — clean traces, then stage, then production.

Compilation and test procedure

lint the playbook registry first

$ python3 soarlint.py playbooks.json playbooks=3 integrations=9 verdict=3 findings {‘GATE’: 1, ‘ROLLBACK’: 1, ‘TIMEOUT’: 1} [GATE ] legacy-account-kill[1] destructive action without approval gate [ROLLBACK ] phish-mail-triage[3] destructive action with no undo [TIMEOUT ] legacy-account-kill[1] network-bound action without timeout_s register: soarlint_out/findings.csv

fixed — lint clean, then dry-run against a synthetic alert

$ python3 playrun.py host-isolate-suspect.json —alert alert-lsass.json playbook=host-isolate-suspect EXECUTE=2 GATE=1 MATCH=1 MATCH trigger alert.severity=9 vs severity_min=8 EXECUTE enrich context enrich_vt (cond passed: country == CN) GATE isolate endpoint edr_isolate (cond passed: severity >= 8) EXECUTE page on-call chat_notify trace: playrun_out/trace.json

alert missing the `country` field — trace HALTs loudly

$ python3 playrun.py host-isolate-suspect.json —alert alert-minimal.json playbook=host-isolate-suspect EXECUTE=0 GATE=0 HALT=1 MATCH=1 MATCH trigger alert.severity=9 vs severity_min=8 HALT enrich context alert lacks field ‘country’ trace: playrun_out/trace.json

Detection considerations

The two tools join at the promotion pipeline: soarlint the static gate (CI, per pull request), playrun the behavioral gate (stage, per synthetic corpus), neither a substitute for the production run-trace review — the weekly sampling that catches what static and dry-run checks cannot (latency drift, approval queues backing up, quota headroom shrinking). The metrics land in BT20’s dashboard: automation rate, MTTA/MTTR deltas, gate override counts (an approver always overriding is a criteria bug, not a discipline problem), auto-approve FP rate per class, and playbook reliability (runs completed / started — the silent-brittleness detector). The flood signal joins BT1’s queue: breaker trips are themselves alerts, because an adversary burning your enrichment budget is an attack on visibility.

Improvements to add

A corpus mode: playrun across a directory of synthetic alerts, emitting a per-playbook coverage matrix (steps exercised vs never touched).
Undo-awareness in the trace: mark EXECUTE steps that carry a registered undo and print the inverse sequence.
Condition unit tests embedded in the playbook spec so soarlint verifies behavior statically.
OpenTelemetry-style spans per production step for latency-baseline detection.
A VIP/exclusion pseudo-step that HALTs the trace when the target matches — modeling the tier-gate the engine enforces.

Detection — Defender View

Signal	Source	ID	Notes
Run-trace failure rate	Weekly review	per playbook	Silent-brittleness detector
Integration 401/403/timeout spikes	Health probes	realtime	Token/endpoint drift
Quota exhaustion events	Circuit breakers	per event	Flood exposure — page
Gate override counts	Case audit	weekly	Criteria bug indicator
Auto-approve FP rate	Case review	per class	Trust budget per class
Playbook runs w/o trace	Trace audit	per cycle	Audit gap = stop-the-line
Trigger rule health	BT3 pipeline	realtime	Disabled rule = dead trigger
Undo path drill recency	Ops calendar	quarterly	Rollback that’s never run

The trigger layer consumes the standard detection substrate — Windows Security 4688/4624/4625/7045, Sysmon 1/3/10/22/23 from BT3’s pipelines, Suricata custom/local sid:1000001+ — but the SOAR-specific surface is the automation’s own health: a SOC that can see its detections but not its automation is blind to the layer that acts on everything else.

Evasion Techniques

Adversaries attack the automation itself — the playbook layer has its own evasion surface. Quota exhaustion: spraying low-quality alerts (the BT1 flood pattern, automated) to burn enrichment API budgets and trip circuit breakers — the counter is per-source alert scoring before quota consumption (cheap checks first) and breaker thresholds that distinguish storm-shape from attack-shape. Trigger spoofing: forging the webhook or event that wakes a containment playbook, turning the automation into the adversary’s DoS tool; HMAC-signed webhooks and source allowlists are the counter, and unauthenticated trigger paths are a lint-class defect. Service-account inheritance: the SOAR’s integration credentials are the most powerful in the SOC (isolate, disable, purge); steal them and the adversary inherits the whole action surface — token scoping, vault storage, no interactive logon, and action audit are the controls, and the account’s baseline (it only acts via playbooks) makes off-playbook action the detection. Legit-target shaping: engineering activity that trips auto-approve criteria against a chosen victim — the exec’s laptop isolated mid-crisis is a credibility attack; exclusion lists, server/VIP hard-fails, and the time-boxed bound cap the damage. Silent-gap awareness: the adversary who knows playbook N broke in last month’s API change attacks through the gap — PT4’s continuous validation and the reliability metric shrink that window to days. Case poisoning: planting artifacts so enrichment logic routes containment at an innocent target — multi-source corroboration before tier-2 actions is the counter: no single source should cause an isolation alone.

⚠️ What caught me: our brute-force playbook — solid logic, clean FP history — auto-blocked an IP after a 4625 burst, and twenty minutes later the helpdesk calls: an entire floor of the office is offline. The “attacker” IP was the building’s NAT gateway; a misconfigured print server had been failing auth all morning and the playbook counted it as a burst. We had built automation that responded to a bad printer by DDoSing our own office — and the review found the deeper sin: the playbook had no exclusion class for internal RFC1918 space on that action, and no human had ever run the undo path, so the unblock procedure was me reading documentation at 11 a.m. while users stared at spinning wheels. The gate criteria now hard-fail on internal-source blocks, and the undo drill runs quarterly like a fire drill — because the rollback you’ve never executed is a document, not a capability.

Lab Practice

Build the promotion pipeline on synthetic data. Lint: plant one GATE, one ROLLBACK, and one TIMEOUT defect across a three-playbook registry; run soarlint, confirm all three fire with the right locations; fix and confirm CLEAN (exit 0). Dry-run: build six synthetic alerts (three that should full-execute, two that should SKIP mid-path, one missing a condition field) and run playrun on each — verify the traces match your expectations exactly, especially the HALT. Flood drill: point the breaker at a threshold your corpus exceeds and write its page rule. Gate drill: walk the auto-approve criteria on the isolation playbook and confirm the VIP exclusion and server hard-fail fire on the right targets. Debrief: which defect class would cause the worst real-world outcome if it reached production — and does your pipeline catch it before stage?

Interview Questions

Technical: Walk the seven parts of a production playbook and explain why idempotency is a requirement for act steps — what breaks on a re-run without it?
Scenario: Your playbook just isolated a shared conference-room PC during a board call. Walk the post-incident: what was missing in the criteria, the undo path, and the review process?
Conceptual: Why is “silent automation failure” the state that must never persist, and which three mechanisms (static, behavioral, operational) catch it?
Deep-dive: Design the promotion pipeline for a SOAR program: registry → lint → stage dry-run → production → run-trace review, with the failure modes each stage exists to catch.
Practical: You inherit a SOAR with 400 playbooks, no owners, no traces, and a service account with domain admin. What do you do in the first week — and what do you refuse to automate until it’s fixed?

BT1 (SOC Operations & Alert Triage) · BT3 (SIEM Engineering & Log Pipelines) · BT4 (Incident Response Process) · BT9 (Detection Engineering & Sigma) · PT4 (Automation & Continuous Validation)