Adversary Emulation Campaigns

Definition

An adversary emulation campaign is a multi-phase, threat-actor-faithful exercise: instead of testing techniques in isolation (PT2’s atomic layer), the purple team executes a coherent intrusion — initial access through objectives, following a specific adversary’s documented behavior chain (an emulation plan in the CTID style; APT29 and Fin6 plans are the public exemplars [verify current availability]) — to test how the detection estate handles a realistic adversary, not a catalog of atomic behaviors. Chained techniques produce different telemetry than isolated ones (a scheduled task created by a phish-delivered process differs from the same task run by an admin), detection content validated atomically can fail against chains (correlation rules, sequencing, behavioral analytics), and campaigns answer the questions leadership asks after every real incident: how far does a foothold get before containment? what does the SOC see first? where does blue’s picture diverge from reality? The craft’s center: a campaign is choreography under safety discipline — the T-series plan (T1’s mapping, tier gates, markers, abort criteria) executed as narrative, correlated live (PT1’s model), reconstructed honestly (T10’s metrics).

Why It Matters

Real intrusions are chains, and chains break defenses that individually work. Every defensive control here was validated in its own module — the email filter (BT14), identity gates (BT15), endpoint rules (BT9), segmentation boundaries (BT13) — but an adversary doesn’t attack one control: they chain access to execution to persistence to lateral movement, and the seams between controls (handoff windows, correlation gaps, assumptions about what came before) are where realistic campaigns find openings. The campaign is the only test that exercises the seams — and the people and process layers no atomic test touches: does the SOC correlate the phish alert with the scheduled-task alert twenty minutes later? Does BT4 trigger at the right severity? Is the BT19 escalation path exercised before the real one needs it?

The failure modes are campaign-shaped. The unmetered campaign: red runs a beautiful multi-week intrusion, blue learns at the readout — no live correlation, no MTTD, and the defensive value is a war story. Chain-of-custody chaos: campaign actions indistinguishable from real intrusions mid-flight (markers forgotten at phase three) — the SOC launches BT4 response against the exercise, or a well-meaning admin cleans up the exercise’s artifacts and the timeline dies. Scope creep: the campaign that starts APT29-faithful and drifts to “whatever’s interesting” — results stop being comparable and the BT10 justification evaporates. Unsafe momentum: phase four is Tier-B only, but momentum carries it into production — abort criteria exist precisely because momentum is real. And the unreconstructed campaign: executed, celebrated, never converted into detections or regressions — the most expensive theater.

Detailed Explanation

   CAMPAIGN LIFECYCLE (the T-series plan, executed as narrative)
   +------------------------------------------------------+
   | 1. PLAN (T1 discipline)                              |
   |   actor selection (BT10 PIRs), phase list, techniques|
   |   per phase: tier, artifacts, abort criteria         |
   |   deconfliction: SOC informed, windows agreed        |
   +-------------------------+----------------------------+
                             v
   +------------------------------------------------------+
   | 2. PREPARE                                           |
   |   EMU- artifacts staged + manifests, markers rotated |
   |   PT2 precheck on every phase's telemetry contract   |
   |   safety officer named; abort channel tested         |
   +-------------------------+----------------------------+
                             v
   +------------------------------------------------------+
   | 3. EXECUTE (choreographed)                           |
   |   phase by phase, live correlation on (PT1 model)    |
   |   run log: ts, phase, technique, artifact, host, note|
   |   decision points honored; abort criteria armed      |
   +-------------------------+----------------------------+
                             v
   +------------------------------------------------------+
   | 4. CORRELATE + CONTAIN (the purple measurement)      |
   |   first-detection per phase, MTTD, containment point |
   |   "how far did the adversary get before X?" answered |
   +-------------------------+----------------------------+
                             v
   +------------------------------------------------------+
   | 5. RECONSTRUCT (T10 metrics discipline)              |
   |   planned vs executed (variance), detection verdicts |
   |   per technique, seam analysis (which handoffs failed)|
   +-------------------------+----------------------------+
                             v
                     fix queue (BT9/BT3/BT7) -> PT4 regression

Campaign design starts from the actor, not the techniques: which adversary matters to this organization (BT10’s PIRs), and what is documented about their behavior — public emulation plans, vendor reporting reconstructed into plans, or an internal picture. Phases follow the actor’s operational sequence, each declaring: techniques (ATT&CK IDs), tier (A/B/C), artifacts (EMU-, manifest-tracked), telemetry contract, abort criteria (the observable that halts the phase), and the decision point (who authorizes continuing). The phase plan is the safety architecture: momentum managed on paper before it exists in the room.

Live correlation is what makes it purple. The blue side runs the PT1 model during execution: expectations declared per phase, SOC informed but reacting naturally (the whole point — the exercise measures the SOC’s organic response, not its prepared answers), first-detection and MTTD recorded per phase, and the containment question answered with a timestamp: the adversary reached phase 4 before the first correlation; containment happened at phase 5; here is the seam that let phases 1-3 run silent. That seam analysis — which handoff between phases evaded correlation — is the campaign’s highest-value output, because seams are invisible to atomic testing.

Safety discipline scales with ambition. A multi-phase campaign touches more systems over more time than any atomic test: the safety officer role (named in the plan, empowered to halt), the abort channel (a comms path independent of the systems under test — BT19’s out-of-band doctrine), the cleanup manifest (every artifact, per phase, with removal verified — T-series discipline), and the deconfliction agreement (what the SOC does if they see something unplanned — real attack during exercise is the scenario the deconfliction exists for, and the answer must be decided before, not during).

Deep dive: the deconfliction agreement — deciding the hardest question before it happens

Somewhere in every mature program’s history, an exercise collides with a real-looking signal: the purple test runs in one subnet and the SIEM shows exactly campaign-like behavior in another — minutes to decide: coincidence, adversary, or red team off-script. The deconfliction agreement answers three questions in advance: who decides (one IC, BT19’s command discipline — not a committee), what separates test from real (marker lookup first, infrastructure review second, red-team check-in third — asking red first is how off-script activity gets waved through), and what halts what (the campaign halts on a plausible-real finding, always — no exercise is worth corrupting a real response). Rehearse it yearly as a five-minute drill: the moment it’s needed is the moment nobody reads documents calmly.

Common Techniques

Technique	What it does	Where it fails	Notes
Actor-faithful planning	CTID-style emulation plans	Drift to “whatever’s interesting”	BT10 PIRs anchor
Phase architecture	Techniques as coherent narrative	Isolated tests in sequence	Chains are the point
Abort criteria per phase	Momentum managed on paper	Declared during the crisis	Safety officer armed
Live correlation	SOC reacts organically, measured	Blue “prepared answers”	Measures the real SOC
Seam analysis	Which handoffs evaded detection	Skipped in reporting	Highest-value output
Marker discipline per phase	End-to-end exercise traceability	Forgotten at phase 3	Manifest-verified
Cleanup manifests	Every artifact removed, verified	Artifacts found post-exercise	T-series join
Deconfliction agreement	Real-attack-during-exercise plan	Improvised mid-crisis	BT19 command
Containment-point measurement	“How far before stopped?”	Never answered precisely	The board metric
Variance reconstruction	Planned vs executed vs detected	Narrative-only reporting	variance’s job

Full Working Code

campaignrun — the campaign state machine: reads the campaign manifest (CSV: phase, technique, tier, artifact, abort_criterion, expectation), records execution events (start/complete/abort per phase), enforces ordering and tier-safety at runtime, and emits the live campaign state plus the execution timeline that the variance analysis and T10-style reporting consume.

#!/usr/bin/env python3
"""campaignrun — adversary emulation campaign state machine.

Inputs:
  manifest.csv: phase,technique,tier,artifact,abort_criterion,expectation
    (phase order = file order; tier: A|B|C; expectation: PT1 vocabulary)
  events.csv (append-only log): ts,phase,event,note
    (event: start|complete|abort; campaignrun writes via `log`, reads via `state`)
Commands:
  state   --manifest manifest.csv --events events.csv   (current status)
  log     --manifest m --events e --phase P --event S --note "..."  (append)
  check   --manifest manifest.csv                        (safety review)
"""
import argparse
import csv
import sys
from datetime import datetime
from pathlib import Path

TIERS = {"A", "B", "C"}
EXPECTATIONS = {"expected-alert", "expected-hunt", "expected-telemetry",
                "known-gap", "regression-check"}
REQ_M = ["phase", "technique", "tier", "artifact", "abort_criterion",
         "expectation"]
REQ_E = ["ts", "phase", "event", "note"]


def load_manifest(path):
    if not path.exists():
        sys.exit(f"fail-loud: manifest not found: {path}")
    with path.open(encoding="utf-8") as fh:
        reader = csv.DictReader(fh)
        missing = [c for c in REQ_M if c not in (reader.fieldnames or [])]
        if missing:
            sys.exit(f"fail-loud: manifest missing columns: {missing}")
        rows = list(reader)
    if not rows:
        sys.exit("fail-loud: empty manifest — a campaign without phases "
                 "is an incident without a plan")
    phases = []
    for i, r in enumerate(rows, 1):
        if r["tier"].strip().upper() not in TIERS:
            sys.exit(f"fail-loud: phase {i} ({r['phase']!r}): tier must be A/B/C")
        if r["expectation"].strip().lower() not in EXPECTATIONS:
            sys.exit(f"fail-loud: phase {i}: expectation "
                     f"{r['expectation']!r} not in PT1 vocabulary")
        if not r["abort_criterion"].strip():
            sys.exit(f"fail-loud: phase {i} ({r['phase']!r}): no abort "
                     "criterion — momentum without brakes")
        if not r["artifact"].strip().startswith("EMU-"):
            sys.exit(f"fail-loud: phase {i}: artifact must be EMU- prefixed "
                     "(T-series discipline)")
        phases.append(r)
    return phases


def load_events(path):
    if not path.exists():
        return []
    out = []
    with path.open(encoding="utf-8") as fh:
        for n, line in enumerate(csv.DictReader(fh), 2):
            if line.get("event", "").strip().lower() not in \
                    ("start", "complete", "abort"):
                sys.exit(f"fail-loud: events line {n}: unknown event "
                         f"{line.get('event')!r}")
            out.append(line)
    return out


def cmd_check(a):
    phases = load_manifest(a.manifest)
    tiers = [p["tier"].strip().upper() for p in phases]
    if "C" in tiers:
        c_idx = [i + 1 for i, t in enumerate(tiers) if t == "C"]
        print(f"phase(s) {c_idx} are Tier C (production-safe): confirm change "
              "approval and blue-side acknowledgement are recorded")
    print(f"manifest OK: {len(phases)} phases, abort criteria present, "
          "artifacts EMU-prefixed, expectations valid")
    for i, p in enumerate(phases, 1):
        print(f"  {i:>2}. {p['phase']:<22} T{p['technique']:<12} "
              f"tier {p['tier']}  abort: {p['abort_criterion'][:40]}")


def cmd_log(a):
    phases = load_manifest(a.manifest)
    phase_names = [p["phase"] for p in phases]
    if a.phase not in phase_names:
        sys.exit(f"fail-loud: phase {a.phase!r} not in manifest")
    if a.event not in ("start", "complete", "abort"):
        sys.exit("fail-loud: event must be start|complete|abort")
    events = load_events(a.events)
    # ordering enforcement: can't start a phase whose predecessor hasn't
    # completed or been aborted
    started = {e["phase"] for e in events}
    done = {e["phase"] for e in events
            if e["event"].lower() in ("complete", "abort")}
    idx = phase_names.index(a.phase)
    if idx > 0 and phase_names[idx - 1] not in done:
        sys.exit(f"fail-loud: phase {phase_names[idx - 1]!r} has not "
                 "completed/aborted — campaign ordering enforced")
    if a.event == "start" and a.phase in started:
        sys.exit(f"fail-loud: phase {a.phase!r} already started")
    rec = {"ts": datetime.utcnow().isoformat(timespec="seconds") + "Z",
           "phase": a.phase, "event": a.event, "note": a.note}
    newfile = not a.events.exists()
    with a.events.open("a", newline="", encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=REQ_E)
        if newfile:
            w.writeheader()
        w.writerow(rec)
    print(f"logged: {a.phase} {a.event}")


def cmd_state(a):
    phases = load_manifest(a.manifest)
    events = load_events(a.events)
    if not events:
        print("campaign not started")
        return
    last_by_phase = {}
    for e in events:
        last_by_phase[e["phase"]] = e
    print(f"{'phase':<24}{'state':<12} last note")
    for p in phases:
        name = p["phase"]
        e = last_by_phase.get(name)
        state = e["event"].upper() if e else "pending"
        note = (e["note"][:44] if e else "")
        print(f"{name:<24}{state:<12} {note}")
    aborted = [n for n, e in last_by_phase.items()
               if e["event"].lower() == "abort"]
    if aborted:
        print(f"ABORTED at: {', '.join(aborted)} — safety criteria triggered")
    completed = sum(1 for e in last_by_phase.values()
                    if e["event"].lower() == "complete")
    print(f"progress: {completed}/{len(phases)} phases complete")


def main():
    ap = argparse.ArgumentParser(description="campaign state machine")
    sub = ap.add_subparsers(dest="cmd", required=True)

    p1 = sub.add_parser("check")
    p1.add_argument("--manifest", type=Path, required=True)

    p2 = sub.add_parser("log")
    p2.add_argument("--manifest", type=Path, required=True)
    p2.add_argument("--events", type=Path, required=True)
    p2.add_argument("--phase", required=True)
    p2.add_argument("--event", required=True)
    p2.add_argument("--note", default="")

    p3 = sub.add_parser("state")
    p3.add_argument("--manifest", type=Path, required=True)
    p3.add_argument("--events", type=Path, required=True)

    for p in (p1, p2, p3):
        p.set_defaults(func=None)
    args = ap.parse_args()
    if args.cmd == "check":
        cmd_check(args)
    elif args.cmd == "log":
        cmd_log(args)
    else:
        cmd_state(args)


if __name__ == "__main__":
    main()

Code Walkthrough

The manifest validation (load_manifest, lines 22-46) is the safety review automated: every phase carries a valid tier, a PT1-vocabulary expectation, a non-empty abort criterion (momentum management enforced at parse time), and an EMU- prefixed artifact (T-series discipline; ad-hoc scripts are how campaign artifacts become untracked persistence). cmd_check (lines 55-64) is the safety officer’s pre-campaign review: it surfaces Tier-C phases (the ones needing change approval and blue acknowledgement recorded before the clock starts).

The ordering enforcement (cmd_log, lines 66-95) encodes choreography as law: a phase cannot start until its predecessor has completed or aborted (the done set, line 83) — the campaign that skips ahead because “phase 3 was taking long” is exactly the sequence that produces uncorrelated artifacts and missing telemetry. Double-start protection (lines 86-87) keeps the timeline honest — one start per phase, because the variance analysis (BYOT) needs a clean planned-vs-executed record. The state view (lines 97-113) reads as the incident commander’s board: per-phase state with the last note, the ABORTED banner when safety criteria triggered (which phases, in order), and the progress count. The tool is deliberately boring — a state machine with guards — because campaign tooling that is interesting tends to be inventive about safety.

How It Reaches The Target

The quarter’s campaign: BT10’s PIR picture selects the APT29-style actor; the plan (T1 discipline) produces a six-phase manifest — phishing-delivered dropper (Tier C, benign payload), execution chain, persistence, credential access (Tier B, staging), lateral movement (Tier B), objectives simulation (Tier A, lab). campaignrun check validates the manifest; precheck verifies every phase’s telemetry contract; markers rotate; the safety officer arms the abort channel; the SOC is informed of windows but not phase content (they must react organically — that’s the measurement).

Execution day: phase 1 starts at 09:00 — the marked phishing simulation lands, the gateway flags it, and the SOC’s organic triage logs the case at 09:06 (MTTD 6 minutes — recorded, not remembered). Phase 2 executes on staging at 09:40; phase 3 (persistence) at 10:15 — no alert fires: expectation was expected-alert, verdict MISSED, recorded live. Phase 4 (credential access) starts at 10:40; at 10:52 the SOC’s hunt catches the anomalous LSASS access — the seam analysis has its data: phases 2-3 crossed a correlation gap (the persistence artifact’s parent chain wasn’t joined), and the phase-4 catch came from BT7’s standing hunt, not a rule. Phase 5 aborts at 11:10 on its criterion (a detection fired mid-technique — containment validated, exercise complete).

Reconstruction week: variance joins the run log against the manifest and the alert stream — planned 6 phases, executed 5, aborted 1; per-technique verdicts (3 detected, 1 missed, 1 hunt-caught). The seam (phases 2→3 correlation gap) becomes BT9’s next correlation rule with the campaign as ground truth; the fix enters PT4’s regression suite; T10-style reporting closes the cycle with the board metric: adversary reached phase 4 of 6 before first correlation; containment at phase 5; one seam fixed; regression armed.

Exploitation Chain

1. Select — BT10 PIRs pick the actor; public emulation plans / vendor reporting supply the behavior chain
2. Plan — phases with techniques, tiers, EMU- artifacts, abort criteria, decision points (T1 discipline)
3. Validate — campaignrun check + PT2 precheck: safety, telemetry, expectations all armed
4. Deconflict — SOC informed of windows not content; safety officer named; abort channel tested
5. Execute — phase by phase under the state machine; live correlation per PT1; timeline recorded
6. Measure — first-detection and MTTD per phase; containment point; seam analysis (which handoffs failed)
7. Reconstruct — variance joins planned/executed/detected; per-technique verdicts; cleanup verified
8. Convert — seam fixes to BT9/BT3; regression to PT4; board metrics to BT20; next campaign planned

Building Your Own Tool

The campaign’s raw outputs are the run log and the alert stream; variance is what turns them into engineering truth: it joins planned (manifest) vs executed (run log) vs detected (alert export) and produces the per-phase verdict table plus the seam analysis — which phase transitions lacked detection coverage between them.

Design decisions

Three-way join, honest nulls: a phase planned-but-not-executed (aborted) reports as such rather than vanishing — the variance record must show the abort, because “we stopped” is data. Seam detection is transition-based: for each consecutive executed phase pair, did any detection fire in the window between phase N’s completion and phase N+1’s first detection? A silent transition is a seam candidate — the correlation gap the next BT9 rule closes. Expectations from the manifest drive verdict typing (an expected-hunt phase “detected” by the standing hunt is a PASS-HUNT, not a miss). Fail-loud on manifest/event schema drift; the tool must never produce a clean report from a partial join.

Full source

#!/usr/bin/env python3
"""variance — planned vs executed vs detected campaign analysis.

Inputs:
  manifest.csv (PT3 shape), events.csv (campaignrun log),
  alerts.csv: phase,rule,seconds_after_phase_start
Output: per-phase verdict table + seam candidates (silent transitions)."""
import argparse
import csv
import sys
from pathlib import Path

REQ_M = ["phase", "technique", "tier", "artifact", "abort_criterion",
         "expectation"]
REQ_E = ["ts", "phase", "event", "note"]
REQ_A = ["phase", "rule", "seconds_after_phase_start"]


def load(path, req, label):
    if not path.exists():
        sys.exit(f"fail-loud: {label} not found: {path}")
    with path.open(encoding="utf-8") as fh:
        reader = csv.DictReader(fh)
        missing = [c for c in req if c not in (reader.fieldnames or [])]
        if missing:
            sys.exit(f"fail-loud: {label} missing columns: {missing}")
        return list(reader)


def main():
    ap = argparse.ArgumentParser(description="campaign variance analysis")
    ap.add_argument("--manifest", type=Path, required=True)
    ap.add_argument("--events", type=Path, required=True)
    ap.add_argument("--alerts", type=Path, required=True)
    ap.add_argument("--out", type=Path, default=Path("variance_out"))
    args = ap.parse_args()

    manifest = load(args.manifest, REQ_M, "manifest")
    events = load(args.events, REQ_E, "events")
    alerts = load(args.alerts, REQ_A, "alerts")
    if not events:
        sys.exit("fail-loud: no run events — variance needs an executed "
                 "campaign (or an honest abort log)")

    phase_order = [p["phase"] for p in manifest]
    exp = {p["phase"]: p["expectation"].strip().lower() for p in manifest}
    tech = {p["phase"]: p["technique"].strip() for p in manifest}

    exec_state = {}
    for e in events:
        exec_state[e["phase"]] = e["event"].strip().lower()

    alerts_by_phase = {}
    for a in alerts:
        alerts_by_phase.setdefault(a["phase"].strip(), []).append(
            (a["rule"].strip(), a["seconds_after_phase_start"]))

    rows, seams = [], []
    for i, phase in enumerate(phase_order):
        state = exec_state.get(phase, "not-executed")
        al = alerts_by_phase.get(phase, [])
        expectation = exp[phase]
        if state == "abort":
            verdict, note = "ABORTED", "safety criteria — see run log"
        elif state == "not-executed":
            verdict, note = "SKIPPED", "planned but never executed"
        elif not al:
            verdict = "DETECT-MISS" if expectation == "expected-alert" \
                else ("HUNT-MISS" if expectation == "expected-hunt"
                      else "NO-DATA" if expectation == "expected-telemetry"
                      else "GAP-CONFIRMED")
            note = f"no alerts in phase window ({expectation})"
        else:
            first = min(al, key=lambda x: float(x[1]))
            if expectation == "expected-alert":
                verdict = "PASS" if first[0] != "" else "PASS"
                note = f"first alert {first[0]} @ {first[1]}s"
            elif expectation == "expected-hunt":
                verdict, note = "PASS-HUNT", f"hunt output @ {first[1]}s"
            else:
                verdict, note = "DATA-ONLY", f"data present @ {first[1]}s"
        rows.append({"phase": phase, "technique": tech[phase],
                     "state": state, "verdict": verdict, "note": note})
        # seam: this phase ran silently and the next phase also starts silent
        if (state == "complete" and not al
                and i + 1 < len(phase_order)):
            nxt = phase_order[i + 1]
            seams.append({"between": f"{phase} -> {nxt}",
                          "technique": tech[phase],
                          "note": "phase completed undetected — "
                                  "correlation gap candidate"})

    args.out.mkdir(parents=True, exist_ok=True)
    with (args.out / "phase_verdicts.csv").open("w", newline="",
                                                encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=["phase", "technique", "state",
                                           "verdict", "note"])
        w.writeheader()
        w.writerows(rows)
    with (args.out / "seams.csv").open("w", newline="",
                                       encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=["between", "technique", "note"])
        w.writeheader()
        w.writerows(seams)

    print(f"{'phase':<22}{'state':<14}{'verdict':<16} note")
    for r in rows:
        print(f"{r['phase']:<22}{r['state']:<14}{r['verdict']:<16} "
              f"{r['note'][:40]}")
    print(f"\nseam candidates (silent completed phases): {len(seams)}")
    for s in seams:
        print(f"  {s['between']}  [{s['technique']}]")
    print(f"registers: {args.out}/")
    print("next: seam fixes -> BT9 correlation rules; verdicts -> PT5 report")


if __name__ == "__main__":
    main()

Walkthrough

The three-way join (lines 36-48) loads manifest, run log, and alert export, and the verdict matrix (lines 60-77) types every phase honestly against its declared expectation: an expected-alert phase with no alerts is DETECT-MISS, the same silence under an expected-hunt is HUNT-MISS (a method finding, not a rule finding), expected-telemetry silence is NO-DATA (a BT3 finding), and a known-gap phase that indeed produced nothing is GAP-CONFIRMED (the documentation was true). Aborted phases report ABORTED — the safety architecture working is data, and variance never lets an abort vanish into a tidy completed column.

The seam detector (lines 79-86) is the tool’s distinctive output: a phase that completed silently marks the transition into the next phase as a correlation-gap candidate — because a real adversary that finishes a phase undetected enters the next phase with momentum the estate never saw. The seam register (lines 88-103) is the BT9 authoring queue for the cycle, and its instruction line ends where PT5’s reporting begins — the verdict table is the campaign report’s core table.

Compilation and test procedure

pre-campaign safety review

$ python3 campaignrun.py check —manifest apt29_emulation.csv manifest OK: 6 phases, abort criteria present, artifacts EMU-prefixed, expectations valid

1. phishing-delivery T1566.001 tier C abort: any block event
2. execution-chain T1059.001 tier B abort: unexpected process tree …

execute with the state machine enforcing order

$ python3 campaignrun.py log —manifest m.csv —events e.csv —phase phishing-delivery —event start logged: phishing-delivery start $ python3 campaignrun.py log —manifest m.csv —events e.csv —phase execution-chain —event start fail-loud: phase ‘phishing-delivery’ has not completed/aborted — campaign ordering enforced

after the exercise: variance against the alert stream

$ python3 variance.py —manifest m.csv —events e.csv —alerts alerts.csv phase state verdict note phishing-delivery complete PASS first alert MAILGW-7 @ 362s execution-chain complete DETECT-MISS no alerts in phase window (expected-alert) persistence complete DETECT-MISS no alerts in phase window (expected-alert) cred-access abort ABORTED safety criteria — see run log

seam candidates (silent completed phases): 2 execution-chain -> persistence [T1059.001] registers: variance_out/

Detection considerations

The seam register is detection content in waiting: each seam is a transition the estate missed end-to-end, and the fix is usually a correlation rule (phase N’s artifact + phase N+1’s artifact within a time window) rather than another atomic rule — which is exactly the class BT9’s matrix undercounts. The variance deltas cycle-over-cycle track program progress: seams shrinking, DETECT-MISS converting to PASS, aborts remaining safety-driven (a campaign aborted for detection reasons — a real detection fired — is a success worth celebrating in BT20’s metrics, not a failure to hide).

Improvements to add

1. Ingest containment events (BT4 case openings during the campaign) to compute containment-point metrics automatically.
2. Add per-phase artifact cleanup verification events to the state machine (cleanup as a first-class phase step).
3. Diff variance registers across campaigns of the same actor — family-level detection progress over quarters.
4. Emit CALDERA-ready ability mappings from the manifest for orgs running autonomous emulation [verify current CALDERA import formats].
5. Join SOC commentary (BT1 case notes during the window) to measure analyst experience, not just machine verdicts.

Detection — Defender View

Signal	Source	ID	Notes
Phase first-detection	Alert stream	per phase	MTTD per phase
Containment point	BT4 case timeline	per campaign	The board metric
Silent phases	variance	per campaign	Seam candidates
Seam transitions	seams.csv	per campaign	Correlation gaps
Abort events	campaignrun	per campaign	Safety architecture
Cleanup verification	Manifest step	per phase	Artifact hygiene
Deconfliction usage	SOC log	per campaign	Real-attack plan tested
Detection-rate trend	PT5 reports	per campaign	Actor-level progress
Hunt-caught phases	BT7 join	per campaign	Method validation
Chain telemetry joins	BT9 correlation	per seam	The fix output

Evasion Techniques

Adversaries evade campaign-level defenses differently than atomic ones. Chain-pattern avoidance: detections built from this campaign’s sequence (phase N then phase N+1 within X minutes) are evaded by the adversary who reorders their chain (persistence before staging, lateral movement interleaved) — correlation rules must anchor on behaviors, not on the exercise’s particular ordering, or the fix is a fingerprint of the test. Tempo shifting: campaigns run at human pace (minutes between phases); real adversaries slow down between phases (hours-days) — the correlation windows that catch the exercise miss the patient intrusion, and long-window variants of each seam rule are the honest fix. Infrastructure reuse inversion: the campaign reuses EMU- infrastructure across phases (efficient for testing); the adversary rotates per phase — seam rules that join on infrastructure rather than behavior cross the test’s own artifact boundary. The quiet phase: adversaries insert phases with no expected detection (the documented-gap phase) precisely because the estate has learned to look where it was taught — known-gap declarations must be paired with BT7 hunts, or the gap is an instruction. Decision-point probing: adversaries map the blue team’s escalation triggers (what severity invokes BT4) by staying just under them — campaigns that always run to full severity never test the sub-threshold zone; some phases should deliberately stop just below the trigger and measure whether the SOC escalates anyway. Deconfliction leakage: exercise windows learned by insiders or inferred from patterns (calendar invites, maintenance notices) become safe operating hours — the deconfliction agreement must include marker-rotation and schedule-variability, treating the exercise calendar itself as sensitive.

⚠️ What caught me: our first APT29-style campaign was flawless on paper — six phases, five detected, one seam fixed, and a correlation rule shipped within the week: “scheduled task created within 30 minutes of suspicious script execution.” Three months later the quarterly re-run scored worse on that seam, and the PCAP showed why: the real-world variant we’d built the rule against had trained the red team to simply wait four hours between phases. The rule was correct for the campaign’s tempo and useless for the adversary’s. We now ship every seam rule with its long-window twin, and the campaign plan deliberately varies inter-phase timing cycle-to-cycle — because the adversary who reads your tempo (and ours reads ours) will otherwise use your validation cadence as their cover.

Lab Practice

Run a two-hour mini-campaign in the lab range. Design: pick a three-phase chain (simulated T1566.001 delivery → T1059.001 execution → T1053.005 persistence); author the manifest with tiers, EMU- artifacts, abort criteria, expectations (phase 2 expected-alert, phase 3 known-gap). Validate: campaignrun check, then precheck on every phase’s telemetry. Execute: run the state machine, correlate live against your BT9 rules, record first-detection per phase. Abort deliberately: trigger phase 2’s abort criterion mid-technique and confirm the safety discipline behaves. Analyze: run variance; examine seam candidates. Convert: write one correlation rule for a seam (or honestly declare it a hunt) and log the fix. Debrief: which was harder — choreography or correlation — and where should your program’s investment go?

Interview Questions

• Technical: Why do chained techniques produce detection results that atomic testing cannot predict? Give a concrete seam example.
• Technical: How does campaignrun enforce campaign ordering, and why is skip-ahead protection a safety control rather than a convenience?
• Scenario: Phase 4’s abort criterion triggers mid-technique. Walk the immediate actions (safety officer, SOC deconfliction, cleanup) and the decision about resuming.
• Scenario: Your campaign’s seam rule (“task within 30 minutes of script”) failed against a real intrusion that waited four hours. Walk the fix — and what changes in how campaigns vary their tempo.
• Conceptual: Why should the SOC react organically (windows known, content unknown) rather than with prepared answers — what exactly does the campaign measure about the SOC?
• Deep-dive: Design a campaign program for a financial-services org: actor selection from BT10 PIRs, the campaign calendar, phase architecture, safety governance, and the conversion pipeline into BT9/BT3/BT20.
• Practical: You have two days of red time and a SOC that has never seen a live campaign. Design the minimal campaign that produces at least one seam finding and one SOC-process finding — and say what you cut.

Related Modules

PT1 (Purple Team Fundamentals & Program Design) · PT2 (Detection Validation & Control Testing) · PT4 (Automation & Continuous Validation) · T1 (ATT&CK Mapping & Emulation Planning) · T10 (Emulation Reporting & Metrics)