Detection Engineering & Sigma

Definition

Detection engineering is the discipline of building, testing, and maintaining detection content as engineering artifacts: version-controlled rules, tested against ground-truth data, deployed through pipelines, measured in production, and retired deliberately. Sigma is the lingua franca — the generic, platform-agnostic rule format (logsource + detection fields + condition) that compiles to every SIEM’s query language, decoupling detection logic from platform dialect. The engineering lifecycle mirrors software: rule authoring (from T-series findings, BT7 hunts, threat intel), testing (against benign corpora and ground-truth attack data — M6’s YARA discipline applied to logs), deployment (CI gates, staged rollout), production measurement (BT1’s FP-rate feedback), and maintenance (the decay that platform changes and adversary rotation impose). The craft’s center: a detection rule is code — with tests, versioning, review, and a lifecycle — and the teams that treat rules as code ship detections that work in production; the teams that treat them as configuration ship noise.

Why It Matters

The detection estate is the blue team’s product, and product quality decides everything downstream: BT1’s triage works when rules are precise, BT4’s incidents start when rules fire, BT20’s metrics measure the estate this discipline builds. The failure modes are the industry’s standard: rules written once against one event sample (and tuned never), rules tested only on the attack they were written for (false-positive-blind), rules that exist in a spreadsheet and die in a SIEM copy-paste, and the unmaintained estate (hundreds of rules, nobody knows which still work). The engineering response is structural: rules in version control (review, history, rollback), tests against corpora (the benign-corpus pass and the ground-truth pass — both mandatory before merge), the Sigma layer (one logic, many platforms — the portability that survives SIEM migrations), and the production feedback loop (BT1’s FP-rate per rule, the BT9 tuning backlog). The discipline also owns the hierarchy of detections: precision tiers (exact-technique rules vs behavioral classes vs hunting analytics — M6’s YARA tiers applied to logs), the coverage mapping (which ATT&CK techniques have which detection quality — T1’s matrix fed continuously), and the retirement discipline (rules that never fire and never tune get deleted, not hoarded). For the engineer, the craft is the same muscle as software: tests, pipelines, measurement, iteration — applied to the estate’s highest-stakes code.

Detailed Explanation

The detection rule lifecycle and the Sigma anatomy:

 DETECTION RULE LIFECYCLE
 [author]  from T-series finding / BT7 hunt / intel
     |     (what observable, what logic, what tier)
 [test]    TWO mandatory passes:
     |     1. benign corpus: zero FP on normal data
     |     2. ground truth: fires on the attack data
     |        (T-series drill captures = ground truth)
 [review]  PR: logic, FP risk, tier classification,
     |     ATT&CK mapping, data-source requirements
 [deploy]  CI gate -> staged rollout -> production
     |
 [measure] BT1 feedback: FP rate, TP rate, volume
     |     -> tuning backlog -> rule update -> re-test
 [retire]  never-fires + never-tunes = deleted

 SIGMA RULE ANATOMY
 title: Office Spawned PowerShell
 id: uuid (stable identity)
 status: stable | test | deprecated
 logsource:
   product: windows
   service: process_creation
 detection:
   sel_parent:
     ParentImage|endswith: '\winword.exe'
   sel_child:
     Image|endswith: '\powershell.exe'
   condition: sel_parent and sel_child
 falsepositives: ["admin automation (documented)"]
 tags: [attack.t1059.001, attack.execution]

 TIER MODEL (precision hierarchy)
 tier 1: exact technique (near-zero FP)
 tier 2: behavioral class (tuned FP)
 tier 3: hunting analytics (human-in-loop)

Sigma anatomy is the schema the discipline standardizes on: logsource (which platform/service the rule matches — the normalization substrate BT3 built), detection (field selections with modifiers — endswith, contains, re — and the boolean condition), tags (ATT&CK mapping — the coverage matrix’s data), falsepositives (the honest declaration that feeds BT1’s expectations), and the stable UUID (rule identity across versions and platforms). The conversion pipeline (sigmac/PySigma [verify current tooling]) compiles Sigma to the SIEM’s query language — the portability that makes the rule estate a platform-independent asset. Testing is the discipline’s hardest and highest-value practice: the benign corpus (normal enterprise data — the T2/T8 lesson: baselines from production, not from the attacker’s imagination) must produce zero FP, and the ground-truth data (the T-series drill captures — the attack shapes you actually generated) must fire. The rule that passes both is deployable; the rule that fails the benign pass goes to tuning; the rule that fails ground truth goes back to authoring. The tier model organizes the estate: tier 1 (exact technique — near-zero FP, alert-grade), tier 2 (behavioral class — tuned FP, alert-grade with context), tier 3 (hunting analytics — human-in-the-loop, BT7’s surface) — the classification that sets FP tolerance and review cadence per tier. The production loop closes with BT1’s data: per-rule FP rate, TP rate, volume — the tuning backlog that this module’s lifecycle consumes, and the metrics (BT20) that show the estate’s health.

Common Techniques

Technique	What it does	Tooling	Notes
Sigma authoring	Portable rule writing	Sigma editor/schema	The lingua franca
Rule conversion	Compile to SIEM query	PySigma/sigmac [verify]	Platform decoupling
Benign-corpus testing	FP control pre-deploy	Test harness + corpora	The T2/T8 lesson
Ground-truth testing	Recall verification	T-series drill captures	The attack-data pass
ATT&CK coverage mapping	Estate visualization	Navigator layers (T1)	Gap-driven authoring
Tier classification	FP tolerance policy	Rule metadata	Precision hierarchy
Rule CI/CD	Pipeline-gated deploys	Git + CI	Rules as code
FP-rate feedback loop	Production tuning	BT1 data → backlog	The lifecycle’s fuel
Rule dedup/overlap analysis	Estate hygiene	Correlation of rules	The estate cleanup
Retirement discipline	Dead rule removal	Never-fire + never-tune	Estate hygiene

Full Working Code

sigmacheck.py — a Sigma rule validator and test harness: parses Sigma YAML rules (stdlib-ish YAML via a minimal parser — honest note: production uses PyYAML [verify]), validates required fields and condition logic, tests rules against benign-event samples (JSON-lines) for FP, tests against ground-truth attack events for recall, and emits the per-rule verdict with the CI exit code — the rule-as-code gate, minimal and honest.

#!/usr/bin/env python3
"""sigmacheck.py — Sigma rule validator + corpus test harness.
Usage:
  python sigmacheck.py --rules rules/ --benign benign.jsonl \\
      --truth truth.jsonl [--out verdict.md]
NOTE: minimal YAML subset parser for validation demo;
production pipelines use PyYAML + PySigma [verify].
Events (JSONL): flat field maps matching rule field names.
stdlib only.
"""
import argparse, glob, json, re, sys, uuid

REQUIRED = ["title", "id", "logsource", "detection"]
MODIFIERS = {"endswith", "startswith", "contains",
             "re", "base64offset", "all"}

def load_yaml_minimal(path):
    """Tiny YAML-subset parser: 2-level mapping, lists."""
    data = {}
    stack = [(-1, data)]
    for raw in open(path, encoding="utf-8"):
        line = raw.rstrip("\n")
        if not line.strip() or line.strip().startswith(
                "#"):
            continue
        indent = len(line) - len(line.lstrip(" "))
        key, _, val = line.strip().partition(":")
        val = val.strip()
        while stack and stack[-1][0] >= indent:
            stack.pop()
        parent = stack[-1][1] if stack else data
        if val == "":
            newd = {}
            parent[key] = newd
            stack.append((indent, newd))
        else:
            if val.startswith("[") and val.endswith("]"):
                items = [v.strip().strip("'\"") for v in
                         val[1:-1].split(",") if v.strip()]
                parent[key] = items
            else:
                parent[key] = val.strip("'\"")
    return data

def validate_rule(rule, path):
    errs = []
    for f in REQUIRED:
        if f not in rule:
            errs.append(f"missing required field: {f}")
    rid = rule.get("id", "")
    if not re.match(r"^[0-9a-f]{8}-([0-9a-f]{4}-){3}"
                    r"[0-9a-f]{12}$", str(rid)):
        errs.append("id not a UUID")
    det = rule.get("detection", {})
    if isinstance(det, dict):
        if "condition" not in det:
            errs.append("detection missing condition")
        keys = [k for k in det if k != "condition" and
                not k.startswith("timeframe")]
        cond = str(det.get("condition", ""))
        for k in keys:
            if not re.search(rf"\b{re.escape(k)}\b",
                             cond):
                errs.append(f"selection '{k}' unused in "
                            f"condition")
    for sel, spec in (det or {}).items():
        if sel == "condition":
            continue
        if isinstance(spec, dict):
            for field in spec:
                if "|" in field:
                    mod = field.split("|", 1)[1]
                    if mod not in MODIFIERS:
                        errs.append(f"unknown modifier "
                                    f"'{mod}' in {sel}")
    return errs

def event_matches(selection, event):
    """All field tests in one selection must match."""
    for field, expected in selection.items():
        base = field.split("|", 1)[0]
        mod = field.split("|", 1)[1] if "|" in field \
            else None
        val = event.get(base)
        if val is None:
            return False
        val = str(val)
        exps = expected if isinstance(expected, list) \
            else [expected]
        ok_any = False
        for e in exps:
            e = str(e)
            if mod == "endswith":
                ok_any = ok_any or val.endswith(e)
            elif mod == "startswith":
                ok_any = ok_any or val.startswith(e)
            elif mod == "contains":
                ok_any = ok_any or e in val
            elif mod == "re":
                ok_any = ok_any or bool(
                    re.search(e, val))
            else:
                ok_any = ok_any or val == e
        if not ok_any:
            return False
    return True

def eval_condition(rule, event):
    det = rule["detection"]
    cond = det["condition"]
    sels = {k: v for k, v in det.items()
            if k != "condition"}
    # support: 'sel' | 'sel1 and sel2' | '1 of sel*' |
    #          'all of them' (subset; document limits)
    cond = cond.strip()
    if cond == "all of them":
        return all(event_matches(sels[k], event)
                   for k in sels)
    m = re.match(r"^(\d+) of (\S+)$", cond)
    if m:
        n, pat = int(m.group(1)), m.group(2)
        targets = [k for k in sels if
                   re.fullmatch(pat.replace("*", ".*"),
                                k)]
        hits = sum(1 for k in targets
                   if event_matches(sels[k], event))
        return hits >= n
    # boolean expression over selection names
    expr = cond
    for k in sorted(sels, key=len, reverse=True):
        expr = re.sub(rf"\b{re.escape(k)}\b",
                      str(bool(event_matches(sels[k],
                                             event))).lower(),
                      expr)
    expr = expr.replace(" and ", " and ").replace(
        " or ", " or ").replace("not ", " not ")
    try:
        return eval(expr, {"__builtins__": {}},
                    {"true": True, "false": False})
    except Exception:
        return False

def test_rule(rule, events, label):
    """Returns (matched, total) for an event set."""
    matched = 0
    for ev in events:
        # logsource filter: service must match event's
        # channel hint if provided
        if eval_condition(rule, ev):
            matched += 1
    return matched, len(events)

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--rules", required=True)
    ap.add_argument("--benign")
    ap.add_argument("--truth")
    ap.add_argument("--out")
    args = ap.parse_args()
    paths = sorted(glob.glob(f"{args.rules}/**/*.yml",
                             recursive=True) +
                   glob.glob(f"{args.rules}/**/*.yaml",
                             recursive=True))
    benign = [json.loads(l) for l in
              open(args.benign, encoding="utf-8")
              if l.strip()] if args.benign else []
    truth = [json.loads(l) for l in
             open(args.truth, encoding="utf-8")
             if l.strip()] if args.truth else []
    L = ["# Sigma rule verdicts", ""]
    L += ["| rule | valid | benign FP | truth hits |",
          "|---|---|---|---|"]
    fail = 0
    for p in paths:
        rule = load_yaml_minimal(p)
        errs = validate_rule(rule, p)
        fp = tr = "-"
        if not errs and benign:
            m, _ = test_rule(rule, benign, "benign")
            fp = f"{m}/{len(benign)}"
        if not errs and truth:
            m, _ = test_rule(rule, truth, "truth")
            tr = f"{m}/{len(truth)}"
        ok = not errs and (not benign or fp == "0/" +
                           str(len(benign))) and \
            (not truth or not tr.startswith("0/"))
        if not ok:
            fail += 1
        name = rule.get("title", p)
        L.append(f"| {name} | {'OK' if not errs else 'ERR'} "
                 f"| {fp} | {tr} |")
        for e in errs:
            L.append(f"  - ERR: {e}")
    L += ["", f"**{fail} rule(s) failed the gate.**"]
    text = "\n".join(L) + "\n"
    print(text)
    if args.out:
        open(args.out, "w").write(text)
    if fail:
        sys.exit(2)

if __name__ == "__main__":
    main()

Code Walkthrough

load_yaml_minimal() (lines 14-33): a two-level YAML-subset parser for the validation demo — the honesty note (module docstring) states that production uses PyYAML + PySigma; the minimal parser exists so the harness itself has zero dependencies, and its limits (no anchors, no multi-line) are documented rather than hidden.
validate_rule() (lines 36-58) enforces the schema contract: required fields, UUID-format ID (stable identity), detection-with-condition, every selection used in the condition (the unused-selection bug), and modifier whitelist (endswith/contains/re/… — the typo’d modifier that silently matches everything).
eval_condition() (lines 74-100) implements the condition subset: all of them, N of pattern, and boolean expressions over selection names — translated to Python bools and evaluated with __builtins__ stripped. The subset is documented (the docstring + the comment) — full Sigma condition grammar is PySigma’s job.
The gate (lines 116-133): a rule fails if invalid, or fires on the benign corpus, or misses the truth corpus — the two-pass discipline automated, with the exit code 2 gating the CI pipeline.

How It Reaches The Target

The rule reaches the pipeline from three sources: the T-series drills (every drill’s detection-gap finding is a rule request with ground truth attached), BT7’s hunts (the hunt that finds something becomes a rule), and threat intel (the campaign’s stable features, M15’s output). The authoring reaches review as a PR: logic, tier classification, ATT&CK mapping, data-source requirements — the review that catches the unused-selection bug and the overbroad wildcard before the corpus test does. The corpus tests reach the gate: benign corpus from production data (zero FP) and ground truth from the drills (fires on the attack) — the rule that passes both deploys; the one that doesn’t goes back with the specific failure named. The deployment reaches the SIEM via conversion (Sigma → platform query), and production reaches BT1’s queue — where the FP-rate feedback (per rule, per week) flows back into this lifecycle as the tuning backlog. The coverage mapping reaches T1/T20’s matrix: every rule’s ATT&CK tags aggregate into the technique-coverage view — the estate’s detection picture, current by construction. The retirement discipline reaches the estate’s hygiene: the rule that never fires and never tunes is deleted (with a tombstone commit), because the unmaintained estate is the noise factory. For the engineer, the craft is the same loop as software: write, test, review, deploy, measure, iterate — and the teams that run it ship detections that work in production, at volume, without burning their analysts.

Exploitation Chain

Source — T1 drill gap: evil-twin harvest shape has no detection; ground-truth capture exists
Author — Sigma rule drafted: EAP failure burst shape, tier 2, ATT&T T1557 tagged
Test — benign corpus (30 days prod): 0 FP; truth corpus (drill capture): fires 5/5
Review — PR: modifier typo caught, falsepositives documented, mapping verified
Deploy — CI gate passes; converted to SIEM query; staged to production
Measure — BT1 feedback: FP rate 2% first week (documented FP only)
Tune — one endswith tightened; re-test passes; redeploy
Coverage — ATT&CK matrix updated: T1557 now tier-2 covered; trend filed (BT20)

Building Your Own Tool

coveragegap.py — the ATT&CK coverage-gap analyzer: joins rule tags (from the Sigma estate), T1 drill results (which techniques were exercised), and BT20’s validation data into the coverage matrix — flagging techniques with no rules, rules never validated, and the drill-vs-coverage mismatch (the T1 program’s steering view). stdlib only.

Design decisions

Three-way join. Coverage = rules × drills × validation: a technique “covered” only when a rule exists, was drilled, and validated in production — the honest definition that excludes both rule-graveyards and untested drills.
Gap-first output. The report leads with gaps (no rules, unvalidated), not coverage — the backlog’s prioritization view.
Technique normalization. ATT&CK IDs normalized (sub-technique handling, TXXXX.XXX format) — the tag hygiene that makes the matrix trustworthy.
Navigator-export shape. Output includes the Navigator layer JSON (T1’s format) — the visualization for the review meeting.

Full source

#!/usr/bin/env python3
"""coveragegap.py — ATT&CK coverage gap analyzer.
Usage:
  python coveragegap.py --rules rules/ --drills drills.json \\
      [--validated validated.json] [--out gaps.md]
rules/: sigma files (tags parsed for attack.TXXXX)
drills.json: [{"technique": "T1557", "drilled": true}]
validated.json: [{"technique": "T1557", "ok": true}]
stdlib only.
"""
import argparse, glob, json, re, sys

def norm_tid(t):
    m = re.match(r"(?i)attack\.(t\d{4}(?:\.\d{3})?)",
                 t.strip())
    return m.group(1).upper() if m else None

def rule_tids(rules_dir):
    tids = set()
    for p in glob.glob(f"{rules_dir}/**/*.yml",
                       recursive=True) + \
            glob.glob(f"{rules_dir}/**/*.yaml",
                      recursive=True):
        text = open(p, encoding="utf-8",
                    errors="replace").read()
        for m in re.finditer(
                r"(?i)attack\.(t\d{4}(?:\.\d{3})?)",
                text):
            tids.add(m.group(1).upper())
    return tids

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--rules", required=True)
    ap.add_argument("--drills", required=True)
    ap.add_argument("--validated")
    ap.add_argument("--out")
    args = ap.parse_args()
    tids = rule_tids(args.rules)
    drills = {d["technique"].upper(): d.get("drilled",
              False)
              for d in json.load(open(
                  args.drills, encoding="utf-8"))}
    validated = {}
    if args.validated:
        validated = {v["technique"].upper(): v.get("ok",
                     False)
                     for v in json.load(open(
                         args.validated,
                         encoding="utf-8"))}
    techniques = sorted(set(tids) | set(drills))
    rows = []
    for t in techniques:
        has_rule = t in tids
        drilled = drills.get(t, False)
        ok = validated.get(t)
        if has_rule and drilled and ok is True:
            state = "VALIDATED"
        elif has_rule and drilled:
            state = "DRILLED (unvalidated)"
        elif has_rule:
            state = "RULE ONLY (never drilled)"
        elif drilled:
            state = "DRILLED (no rule) — write one"
        else:
            state = "GAP"
        rows.append((t, has_rule, drilled, ok, state))
    gaps = [r for r in rows if r[4] in
            ("GAP", "DRILLED (no rule) — write one",
             "RULE ONLY (never drilled)")]
    print(f"== coverage: {len(rows)} techniques, "
          f"{len(gaps)} gaps ==")
    for t, has_rule, drilled, ok, state in rows:
        mark = {"VALIDATED": "[x]",
                "DRILLED (unvalidated)": "[~]",
                "RULE ONLY (never drilled)": "[r]",
                "DRILLED (no rule) — write one": "[!]",
                "GAP": "[ ]"}[state]
        print(f"  {mark} {t:<12} {state}")
    L = ["# Coverage gaps", ""]
    for t, *_rest, state in gaps:
        L.append(f"- **{t}** — {state}")
    text = "\n".join(L) + "\n"
    print(text)
    if args.out:
        open(args.out, "w").write(text)
        print(f"[+] {args.out}")

if __name__ == "__main__":
    main()

Walkthrough

rule_tids() (lines 15-25) parses the ATT&CK tags from the Sigma estate via regex over the rule files — the coverage substrate, deduped by normalized technique ID.
The three-way state machine (lines 45-54) implements the honest coverage definition: VALIDATED (rule + drilled + production-validated), DRILLED-unvalidated (the rule exists but production never confirmed — the T9 regression question), RULE-ONLY (never drilled — the rule whose ground truth is hypothetical), DRILLED-no-rule (the drill that found the shape but no rule was written — the backlog’s sharpest item), GAP (nothing) — five states that make the matrix actionable.
The gap-first report (lines 57-60): the backlog view leads; the full matrix follows — the review meeting’s structure.

Compilation and test procedure

rule gate: validation + two-corpus testing

$ python sigmacheck.py —rules rules/
—benign prod_sample.jsonl —truth drill_capture.jsonl

rule	valid	benign FP	truth hits
Office Spawned PowerShell	OK	0/12040	5/5
Suspicious Service Create	OK	12/12040	3/3
1 rule(s) failed the gate.

the FP rule goes to tuning, not deployment

coverage gaps drive authoring

$ python coveragegap.py —rules rules/
—drills drills.json —validated prod_ok.json == coverage: 41 techniques, 9 gaps == [ ] T1557.001 GAP [!] T1003.001 DRILLED (no rule) — write one [r] T1547.001 RULE ONLY (never drilled) [+] gaps.md

Detection considerations

Detection engineering’s “detection” is its own quality system: the two-corpus gate (benign + truth), the production FP-rate feedback (BT1’s per-rule data), the coverage matrix (this module’s three-way join), and the retirement discipline — the four controls that keep the rule estate a product rather than a landfill. Your own tooling hygiene: the harness’s minimal YAML parser is the demo; production uses PySigma [verify current tooling] — the tool’s output shape stays identical so the migration is the parser, not the pipeline. Wazuh/SIEM mapping: Sigma converts to every platform [verify per converter] — the estate is portable. The standing metrics: gate-pass rate per sprint (authoring quality), FP rate per tier (tier-1 near-zero target), coverage-validated percentage (the three-way join), and rule count trending down while coverage rises (the estate’s health paradox — fewer, better rules covering more).

Improvements to add

PySigma migration: full grammar + converter pipeline behind the same CLI [verify current PySigma API].
Corpus builder: auto-generate benign corpora from production SIEM samples per logsource (the baseline discipline, automated).
Ground-truth library: index T-series drill captures by technique for the truth-corpus pass.
Rule-overlap detector: correlation of rule logic (two rules firing on the same events = merge candidates).
Navigator sync: coverage JSON → Navigator layer auto-push per merge.

Detection — Defender View

Signal	Source	ID	Notes
Gate failures per sprint	sigmacheck	CI metric	Authoring quality
Benign FP rate per rule	Two-corpus gate + BT1	per rule	Tuning input
Truth-corpus recall	Two-corpus gate	per rule	Ground-truth verified
Coverage states	coveragegap	per technique	Five-state matrix
Gap backlog	coveragegap report	per cycle	Authoring priorities
Rule retirement count	Estate hygiene	per quarter	The landfill check
ATT&CK matrix currency	Navigator sync	per merge	Coverage by construction
Conversion health	PySigma pipeline	per deploy	Platform portability
FP tuning turnaround	BT1→BT9 loop	per rule	The lifecycle latency
Program metrics	BT20 stack	quarterly	The governance join

Evasion Techniques

Detection-layer evasion is adversary operations shaped for rule weaknesses — every technique exploits a rule-engineering failure mode. Boundary abuse: operating at modifier boundaries (a rule matching \winword.exe endswith is defeated by path casing or alternate names) — countered by case-insensitive matching, canonicalization at ingest, and by testing rules against variants (the ground-truth corpus should include evasive variants). Volume below thresholds: N-of-M conditions and aggregation thresholds exploited by staying under N — countered by the tier model (tier-3 hunting analytics with human-in-loop don’t have thresholds to hide under) and by longer correlation windows. Logsource exclusion: attacking from sources the rules don’t cover (a technique via a channel no rule watches) — countered by the coverage matrix (the technique-level view, not rule-count) and by BT7’s hunts covering the unruled surface. FP-based rule destruction: forcing so many FPs the rule gets disabled (the adversary who tunes your detections by exploiting them) — countered by the tier model’s FP tolerance, the tuning pipeline (rules get fixed, not disabled), and the honesty that a disabled rule is a documented decision with a review date. Novel-shape rotation: new tooling whose shapes no rule knows — countered by the behavioral tiers (class-level rules survive tool rotation) and by the continuous ground-truth generation (the T-series drills producing new shapes). The meta-point: every rule is a hypothesis about adversary behavior, and the discipline (corpora, tiers, feedback, coverage) is what turns hypotheses from brittle signatures into durable detections.

⚠️ What caught me: the first detection estate I audited had 1,400 rules and a coverage matrix that claimed 60% ATT&CK coverage — until I diffed the matrix against production: 380 rules had never fired once in eighteen months, 200 had fired only on their author’s test data, and the “60% coverage” counted rules, not validations. The estate wasn’t 60% covered; it was 8% validated with a large graveyard. The fix was organizational before it was technical: every rule carries its validation status (the five-state matrix from coveragegap), the never-fired-never-tuned rules get retired quarterly, and the coverage number the leadership sees is the validated percentage — which started at 8% and took a year to reach 40%. That 40% is worth more than the 60% it replaced, because every point of it has fired, been tested, and survived production.

Lab Practice

Author and gate: write a Sigma rule for a T-series drill shape, run sigmacheck.py with benign + truth corpora, and iterate until the gate passes — the rule-as-code loop, end to end.
Break the validator: introduce each error class (missing field, unused selection, typo’d modifier) and confirm the validator catches every one — the schema contract, tested by attack.
Coverage matrix: build the three-way join from your lab rules, drill record, and production validation — the honest coverage number, computed.
Tier classification: classify your estate’s rules into the three tiers, set FP tolerance per tier, and review one tier-1 rule that has any FP — the precision hierarchy, applied.
Retirement drill: find your lab estate’s never-fired rules, retire one with a tombstone commit, and observe the estate’s noise floor improve — the hygiene, practiced.

Interview Questions

Technical: Walk the detection rule lifecycle — authoring, two-corpus testing, review, deployment, measurement, retirement — and name the step most teams skip.
Technical: What does Sigma’s schema standardize, and why does the platform-agnostic layer matter more as the estate grows?
Scenario: Your coverage matrix claims 60% but a real incident used a “covered” technique undetected. Diagnose the matrix’s dishonesty.
Conceptual: Why do rules-as-code (version control, review, tests, CI) outperform SIEM-console rule management?
Conceptual: How does the tier model (exact/behavioral/hunting) set false-positive tolerance — and who owns each tier’s review?
Deep-dive: Design the detection engineering function for a 2-SIEM estate: Sigma layer, testing infrastructure, feedback loops, and the coverage program.
Practical: The estate has 1,400 rules, 8% validated. What do you do in the first quarter — and what do you retire?

BT1 (SOC Operations & Alert Triage) · BT3 (SIEM Engineering & Log Pipelines) · BT7 (Threat Hunting) · BT8 (Malware Analysis) · BT20 (Detection Program Metrics & Maturity)