IDS Rules

Definition

IDS rules are the network detection layer’s unit of work: declarative signatures — action, header (protocol, addresses, ports, direction), and options (content matches, sticky buffers, flow state, thresholds, metadata) — evaluated against reassembled traffic by engines like Suricata (multithreaded, app-layer aware) and Snort (the lineage’s original). A rule is simultaneously code (exact syntax, escaping, PCRE) and insurance (it fires when something you predicted happens), and the discipline of writing them sits exactly between BT9’s detection engineering (version control, testing, lifecycle — rules are code here too) and the raw craft of protocol analysis (E17’s network detection view). The core competencies: syntax mastery (content with |hex| notation, pcre with modifiers, sticky buffers like http.uri and tls.sni, flow state established to_server), placement discipline (content anchors before pcre — cheap matches first, because every rule pays its evaluation cost on every packet), threshold hygiene (detection filters that suppress noise without hiding attacks), and SID governance (unique, registry-managed identifiers in the local range — 1000001+ by convention — with rev incremented on every change). The craft’s center: an IDS rule that fires is a claim about traffic, and the mature teams treat every claim the way BT9 treats detection logic — tested against benign corpora, measured in production, retired deliberately.

Why It Matters

Network signatures remain the fastest path from “known bad” to “blocked or alerted” — the ET Open ruleset and vendor feeds carry decades of protocol expertise, and a well-written local rule turns a BT10 intel sighting into standing coverage in minutes. When a new C2 pattern emerges, the network is where you can see it without endpoint agents, across every OS and every workload, including the devices no one can instrument (printers, cameras, the vendor box with the support contract). Rules also anchor the perimeter conversation: IPS-mode drop rules are prevention, not detection, and the difference between an IDS deployment and an IPS deployment is exactly the confidence you have in your ruleset — which is why testing discipline and IPS mode are joined at the hip.

The failure modes are specific and heavy. The pcre tax: rules whose first meaningful match is an expensive regex pay CPU on every flow — under load, the engine’s ceiling becomes the network’s blind spot, and attackers know probe windows when they see them. Threshold suicide: threshold: type both, track by_src, count 100 on a slow-ratio attack means the first 99 beacons were free. HOME_NET lies: a variable that doesn’t match reality (a new VLAN nobody added) silently blinds entire segments — the network version of BT11’s coverage decay. SID chaos: copy-pasted rules with colliding SIDs — one overwrites the other at load, and which one survived depends on file order, which is not a detection strategy. And alert desensitization: 40,000 alerts a day from six noisy signatures teaches the SOC to ignore the NIDS entirely — BT1’s FP-rate feedback loop exists to catch those signatures before they burn the channel.

Detailed Explanation

   packets / reassembly (flow engine)
          |
          v
   +----------------+     header match: proto, ips, ports, direction
   | RULE ENGINE    |     fast-pattern selection: the engine picks the
   | (Suricata: MT, |     rarest content as prefilter
   |  app-layer)    |
   +-------+--------+
           | per-rule evaluation order:
           |   1. header (cheap)
           |   2. content / sticky buffers (prefilter)
           |   3. pcre (expensive, only after anchors hit)
           |   4. flow/threshold/metadata
           v
   +----------------+     EVE JSON alert -> BT3 pipeline
   | OUTPUT: EVE    |     -> BT1 triage verdicts
   | alert records  |     -> FP rate per sid -> rule tuning (this module)
   +----------------+

Rule anatomy (Suricata syntax, which subsumes most of Snort’s): action protocol src_ip src_port -> dst_ip dst_port (options;). The action selects posture — alert (IDS), drop/reject (IPS, prevention), pass (suppression, which is its own risk: a bad pass rule disables everything after it). The options chain carries the semantics: flow:established,to_server binds to reassembled session state; content:"/api/login"; http.uri; anchors a match inside the decoded URI; pcre:"/admin[0-9]{3,}/i"; runs only where the engine’s fast-pattern selection allows. Content modifiers moved into sticky buffers in modern Suricata (http.uri; content:"...";) — same logic, cleaner nesting, and it matters because buffer discipline is where correctness lives (matching inside the wrong buffer is the classic silent-miss).

Performance is part of correctness. The engine extracts a fast pattern per rule (the rarest content) as its prefilter; rules without any content match force full evaluation paths — pcre-only rules are the worst offenders. The craft: cheap, specific content first; pcre anchored after an explicit content hit; startswith/bsize where regex isn’t needed; and the datasets/lua features for the heavy cases. Rule review includes a cost eyeball the same way code review includes a complexity eyeball.

SID governance is the boring half that prevents the worst outages: local rules live in 1000001+ (Snort’s documented local convention; verify your org’s registry against vendor ruleset overlap), every rule carries sid + rev + msg + classtype + metadata (reference to the intel or incident that justified it), the registry (Building Your Own Tool) detects collisions before deployment, and rev increments on every edit so downstream (BT3’s pipeline, retro-hunts) can distinguish v3 from v4 of the same idea. Thresholds — detection_filter: track by_src, count 5, seconds 60 — suppress the sixth-through-Nth repeat rather than the first four alerts you needed; the direction of the inequality is the difference between tuning and blinding.

Deep dive: the pass rule — the most dangerous line in your ruleset

pass rules exist for one honest purpose: suppressing known-benign-but-loud traffic so real alerts surface (a backup window’s port scan, a partner’s chatty scanner). But a pass rule is a negation engine: in Suricata’s unified evaluation, traffic matching a pass rule skips subsequent alert/drop evaluation for that flow — so one overbroad pass (wrong port, wrong direction, a /32 that grew into a /24) doesn’t just miss a detection, it disables every rule downstream for that traffic, silently. The audit posture: pass rules live in their own file, carry metadata: justification and an expiry review date, get counted in the registry (the pass-rule audit count in the Defender View), and — the discipline that catches mistakes — every pass rule’s header is reviewed against the question “what could hide inside this traffic?” A pass rule that answers “nothing, it’s just backups” has never met a real backup server.

Common Techniques

Technique	What it does	Where it fails	Notes
Content + sticky buffers	Protocol-aware exact matches	Wrong buffer = silent miss	http.uri, tls.sni, dns.query
Fast-pattern anchoring	Cheap prefilter before pcre	pcre-only rules	Cost on every flow
Flow state binding	established/to_server precision	Missing flow = dup alerts	Reassembled view
detection_filter	Suppress repeats past N	Filters the wrong direction	Keep the first alerts
Dataset matching	Large value sets (known-bad JA3)	Feed rot	Suricata datasets
Cross-packet content	Match patterns spanning segments	Reassembly depth limits	Normal traffic first
Metadata provenance	Intel/incident link per rule	Rules without justification	bt10 join
SID registry	Unique IDs, rev discipline	Copy-paste collisions	sidregistry’s job
pass-rule suppression	Known-good loud traffic	A bad pass blinds all	Audit pass rules
IPS-mode graduation	alert → drop with soak time	Dropping on untested rules	Staged rollout

Full Working Code

rulelint — a Suricata rule validator: parses rule files, checks syntax structure (header, options), enforces hygiene (sid in local range, rev present, msg present, classtype present, content-before-pcre anchoring, flow on TCP rules), detects duplicate SIDs across files, and flags the expensive patterns (pcre without preceding content) that become the pcre tax in production.

#!/usr/bin/env python3
"""rulelint — Suricata rule hygiene validator.

Usage: python3 rulelint.py rules/*.rules [--strict]
Checks: structure, sid range/registry, rev/msg/classtype presence,
        content-before-pcre anchoring, flow on TCP, duplicate sids.
Exit 1 on any FAIL (CI gate for the rules repo).
"""
import argparse
import re
import sys
from pathlib import Path

LOCAL_SID_MIN = 1000001
LOCAL_SID_MAX = 1999999
HEADER = re.compile(
    r"^(?P<action>alert|drop|reject|pass)\s+"
    r"(?P<proto>tcp|udp|icmp|ip|http|tls|dns|ftp|smb)\s+"
    r"(?P<src>\S+)\s+(?P<sport>\S+)\s+"
    r"(?P<dir><>|->|<->)\s+"
    r"(?P<dst>\S+)\s+(?P<dport>\S+)\s*\(")
OPTION_SPLIT = re.compile(r';(?=(?:[^"]*"[^"]*")*[^"]*$)')  # split on ; not in quotes


def parse_rule(line):
    """Return (header_dict, options_list) or raise ValueError."""
    m = HEADER.match(line)
    if not m:
        raise ValueError("header does not parse")
    header = m.groupdict()
    body = line[m.end():].rstrip()
    if not body.endswith(")"):
        raise ValueError("options not closed with )")
    body = body[:-1]
    opts = [o.strip() for o in OPTION_SPLIT.split(body) if o.strip()]
    return header, opts


def opt_key(opt):
    return opt.split(":", 1)[0].split()[0].lower()


def lint_rule(line, fname, lineno, seen_sids, problems):
    def fail(msg):
        problems.append(f"{fname}:{lineno}: FAIL: {msg}")

    def warn(msg):
        problems.append(f"{fname}:{lineno}: WARN: {msg}")

    line = line.strip()
    if not line or line.startswith("#"):
        return
    try:
        header, opts = parse_rule(line)
    except ValueError as e:
        fail(f"{e}")
        return

    keys = [opt_key(o) for o in opts]

    # hygiene: required fields
    if "msg" not in keys:
        fail("missing msg")
    if "sid" not in keys:
        fail("missing sid")
    else:
        sid_opt = next(o for o in opts if opt_key(o) == "sid")
        try:
            sid = int(sid_opt.split(":", 1)[1])
        except (ValueError, IndexError):
            fail(f"sid not an integer: {sid_opt!r}")
        else:
            if sid in seen_sids:
                fail(f"duplicate sid {sid} (first at {seen_sids[sid]})")
            seen_sids[sid] = f"{fname}:{lineno}"
            if LOCAL_SID_MIN <= sid <= LOCAL_SID_MAX:
                if "rev" not in keys:
                    fail("local rule missing rev")
            elif sid >= LOCAL_SID_MAX:
                warn(f"sid {sid} above local range — vendor overlap risk")
    if "classtype" not in keys and "metadata" not in keys:
        warn("no classtype/metadata — provenance missing (bt10 join)")

    # performance: pcre must follow a content anchor
    first_content = next((i for i, k in enumerate(keys) if k == "content"), None)
    pcre_positions = [i for i, k in enumerate(keys) if k == "pcre"]
    if pcre_positions and first_content is None:
        fail("pcre without any content anchor — pcre tax on every flow")
    elif pcre_positions and pcre_positions[0] < first_content:
        warn("pcre appears before first content — reorder cheap-first")

    # correctness: TCP rules should bind flow state
    if header["proto"] == "tcp" and "flow" not in keys:
        warn("tcp rule without flow: state — will match per-packet noise")

    # threshold direction sanity
    for o in opts:
        if opt_key(o) in ("detection_filter", "threshold"):
            val = o.split(":", 1)[-1]
            if "count" in val and "seconds" not in val and "port" not in val:
                warn(f"count without time window: {val[:40]!r} — "
                     "one-shot suppression may hide the attack")


def main():
    ap = argparse.ArgumentParser(description="Suricata rule hygiene linter")
    ap.add_argument("rules", nargs="+", help="rule files or globs")
    ap.add_argument("--strict", action="store_true",
                    help="treat WARN as FAIL (CI gate mode)")
    args = ap.parse_args()

    files = []
    for pattern in args.rules:
        matched = sorted(Path(".").glob(pattern))
        if not matched:
            sys.exit(f"fail-loud: pattern matched nothing: {pattern}")
        files.extend(matched)

    seen_sids, problems, linted = {}, [], 0
    for f in files:
        for lineno, line in enumerate(f.read_text(encoding="utf-8",
                                                   errors="replace").splitlines(), 1):
            lint_rule(line, f.name, lineno, seen_sids, problems)
            if line.strip() and not line.strip().startswith("#"):
                linted += 1

    fails = [p for p in problems if ": FAIL:" in p]
    warns = [p for p in problems if ": WARN:" in p]
    print(f"linted={linted} rules across {len(files)} files  "
          f"sids={len(seen_sids)}  FAIL={len(fails)}  WARN={len(warns)}")
    for p in problems[:20]:
        print("  ", p)
    if len(problems) > 20:
        print(f"   ... and {len(problems) - 20} more")
    if fails or (args.strict and warns):
        sys.exit(1)
    print("rules lint clean — registry, hygiene, anchoring all pass")


if __name__ == "__main__":
    main()

Code Walkthrough

The header regex (lines 21-27) enforces the engine’s actual grammar — action from the four postures, protocol including Suricata’s app-layer protocols (http|tls|dns|ftp|smb rules carry their own decoder semantics), and the three direction operators (<>, ->, <->). The option splitter (line 28) is the subtle piece: splitting on ; only outside quotes — msg:"code; here" must survive as one option — using a lookahead that counts unbalanced quote pairs. parse_rule() (lines 30-41) fails with which structural defect, not just “bad rule”, because the author is fixing this in CI.

The lint logic reads in the order the review conversation does: required fields first (msg/sid/rev — lines 58-63; a local rule without rev fails because retro-hunts and pipeline joins can’t distinguish versions without it), SID registry next (duplicate detection across all files in the run, lines 71-76 — the copy-paste collision that silently overwrites one rule with another at load time), provenance (classtype or metadata — line 84, the BT10 join), then the performance review: pcre-without-content fails (lines 87-90) because that rule pays regex cost on every flow the prefilter can’t pre-exclude; pcre-before-content warns (lines 91-92). The flow check (lines 95-96) catches the per-packet noise generator — TCP rules without flow:state match reassembly orphans and duplicate-alert by design. The threshold check (lines 99-104) flags count without a time window — the one-shot suppression that hides everything after the first hit forever. Exit gates (lines 133-136): FAIL always blocks; --strict promotes WARN, the CI posture after the first stabilization quarter.

How It Reaches The Target

The lifecycle runs through the repo. A BT10 sighting lands (“new C2 panel at X, TLS self-signed, URI /gate.php”): the intel becomes three candidate rules — a tls.sni match, a http.uri match, and a JA3 dataset check — each authored in the rules repo with metadata pointing at the sighting. rulelint gates the merge (hygiene, anchoring, sidregistry assigns the next free local SID); CI deploys to the IDS sensors via controlled reload (suricatasc rule-reload, no restart); the rules soak in alert mode with FP measured per SID against BT1’s triage verdicts for a week. Only then does the strongest rule graduate toward drop (IPS mode) with a staged rollout — and the graduation decision cites the FP numbers, not optimism.

The attack that reaches the network meets this standing layer: beacon traffic matching the URI rule fires with sid:1000231 rev:2 in the EVE stream (BT3’s pipeline), BT1 triages with the intel context already in the alert metadata, and the sighting loops back to BT10 — corroboration for the sharing group. Two quarters later the infrastructure burns and the rules retire (the registry marks them retired, not deleted — retro-hunt value), leaving the pattern rule (panel path + self-signed TLS + rare JA3 combination, if BT9’s log-layer adds behavioral coverage) as the durable artifact. The measure loop closes in BT20: alert-to-case conversion per SID, FP rate per source, rule retirement counts, and the pcre-tax budget reviewed quarterly — the NIDS staying fast because someone owns its cost model.

Exploitation Chain

1. Source — BT10 sighting, BT7 hunt finding, incident RCA (BT4), or vendor/ET feed update
2. Author — rule in the repo with metadata provenance; sidregistry assigns local SID
3. Gate — rulelint (hygiene + anchoring) in CI; registry collision check before merge
4. Deploy — controlled reload to sensors; alert-mode soak with FP telemetry per SID
5. Tune — BT1 FP feedback tightens or splits the rule; threshold hygiene verified
6. Graduate — IPS-mode drop for high-confidence rules, staged with rollback
7. Detect — EVE alerts into BT3 pipeline; triage with intel context; sightings loop to BT10
8. Retire — infrastructure burns, registry marks retired; behavioral coverage succeeds in BT9

Building Your Own Tool

Rule sets rot through accumulation; sidregistry is the anti-rot tool: scans every .rules file, builds the SID inventory (id → rule, file, rev, msg, status), detects collisions and range violations, assigns the next free local SID on request, and emits the registry CSV that deployment tooling and retro-hunts consume.

Design decisions

Single source of truth: the registry is derived from the repo every run — no parallel bookkeeping to drift. Assignment is deterministic: next free integer above the highest allocated (gaps reused only explicitly, because reusing a retired SID within a pipeline’s memory window reattaches old alerts to new rules). Status lives in the rule’s metadata (metadata: status retired) rather than a side file — the rule file stays the single artifact. Duplicate detection reports both locations. Fail-loud on unparseable rules so a syntax break can’t silently shrink the inventory.

Full source

#!/usr/bin/env python3
"""sidregistry — derive the SID inventory from the rules repo.

Usage: python3 sidregistry.py rules/*.rules [--assign] [--out sid_registry.csv]
  --assign   print the next free local SID (and reserve nothing —
             the caller writes the rule; next run reflects it)
"""
import argparse
import csv
import re
import sys
from pathlib import Path

LOCAL_MIN, LOCAL_MAX = 1000001, 1999999
SID_RE = re.compile(r"\bsid:(\d+)\s*;")
REV_RE = re.compile(r"\brev:(\d+)\s*;")
MSG_RE = re.compile(r'\bmsg:"([^"]*)"\s*;')
STATUS_RE = re.compile(r"\bstatus:\s*([a-z]+)")


def main():
    ap = argparse.ArgumentParser(description="SID registry from rules repo")
    ap.add_argument("rules", nargs="+")
    ap.add_argument("--assign", action="store_true",
                    help="print next free local SID and exit")
    ap.add_argument("--out", default="sid_registry.csv")
    args = ap.parse_args()

    files = []
    for pattern in args.rules:
        matched = sorted(Path(".").glob(pattern))
        if not matched:
            sys.exit(f"fail-loud: pattern matched nothing: {pattern}")
        files.extend(matched)

    entries, collisions, unparseable = [], [], 0
    seen = {}
    for f in files:
        for lineno, line in enumerate(f.read_text(
                encoding="utf-8", errors="replace").splitlines(), 1):
            line = line.strip()
            if not line or line.startswith("#"):
                continue
            sid_m = SID_RE.search(line)
            if not sid_m:
                unparseable += 1
                print(f"FAIL: {f.name}:{lineno}: no sid — unparseable rule",
                      file=sys.stderr)
                continue
            sid = int(sid_m.group(1))
            rev = int(REV_RE.search(line).group(1)) if REV_RE.search(line) else 0
            msg = MSG_RE.search(line).group(1) if MSG_RE.search(line) else ""
            status = (STATUS_RE.search(line).group(1)
                      if STATUS_RE.search(line) else "active")
            loc = f"{f.name}:{lineno}"
            if sid in seen:
                collisions.append((sid, seen[sid], loc))
            seen[sid] = loc
            entries.append({"sid": sid, "rev": rev, "status": status,
                            "msg": msg, "location": loc,
                            "range": "local" if LOCAL_MIN <= sid <= LOCAL_MAX
                            else "vendor/other"})

    if unparseable and not entries:
        sys.exit("fail-loud: zero parseable rules — repo state is broken")
    if collisions:
        print(f"COLLISIONS ({len(collisions)}):")
        for sid, a, b in collisions:
            print(f"  sid {sid}: {a}  <->  {b}")
        sys.exit(1)

    if args.assign:
        allocated = {e["sid"] for e in entries
                     if LOCAL_MIN <= e["sid"] <= LOCAL_MAX}
        nxt = LOCAL_MIN
        while nxt in allocated:
            nxt += 1
        print(nxt)
        return

    entries.sort(key=lambda e: e["sid"])
    with open(args.out, "w", newline="", encoding="utf-8") as fh:
        w = csv.DictWriter(fh, fieldnames=["sid", "rev", "status", "msg",
                                           "location", "range"])
        w.writeheader()
        w.writerows(entries)
    active = sum(1 for e in entries if e["status"] == "active")
    local = sum(1 for e in entries if e["range"] == "local")
    print(f"registry: {len(entries)} sids  active={active}  local={local}  "
          f"-> {args.out}")


if __name__ == "__main__":
    main()

Walkthrough

Extraction is regex-against-line (lines 18-22) — deliberately simple, because the registry’s contract is “SIDs are findable without full rule parsing”; rulelint owns deep syntax, the registry owns identity. Collision detection (lines 47-49) accumulates locations and reports both — the two files fighting over one SID is an outage story, and the CI gate exits 1 so the merge stops. --assign (lines 61-68) computes the next free local SID deterministically (linear scan above the floor — with 10^6 usable SIDs and realistic estates in the thousands, the scan is instant and the determinism is worth more than cleverness). The status field (lines 52-53) reads from the rule’s own metadata — status retired lives in the artifact, so retirement survives file moves and the registry never disagrees with the repo. Unparseable lines print to stderr and count (lines 43-45) — a repo that half-parses must never produce a quietly shrunken registry.

Compilation and test procedure

derive the registry from the repo (CI step)

$ python3 sidregistry.py “rules/*.rules” —out sid_registry.csv registry: 231 sids active=228 local=189 -> sid_registry.csv

a copy-paste collision must stop the merge

$ python3 sidregistry.py “rules/*.rules”; echo “exit=$?” COLLISIONS (1): sid 1000231: c2_panels.rules:12 <-> phishing_kits.rules:88 exit=1

authoring a new rule: get the next free SID first

$ python3 sidregistry.py “rules/*.rules” —assign 1000232

then write the rule with that sid and rev:1, re-run both tools

$ python3 rulelint.py “rules/*.rules” —strict linted=232 rules across 4 files sids=232 FAIL=0 WARN=0 rules lint clean — registry, hygiene, anchoring all pass

Detection considerations

The registry’s deltas are telemetry: new SIDs per week (authoring velocity), retired-per-week (hygiene), and — the subtle one — rev bumps on rules whose msg didn’t change (tuning happening without provenance updates). Join the registry with BT1’s FP-rate-per-SID output to produce the rule health table (the quarterly pcre-tax review reads from it), and with BT9’s coverage matrix so the NIDS layer’s technique coverage is counted in the same program math as endpoint detections — one coverage number, two sensor layers, no double-counting.

Improvements to add

1. Parse flowbits/xbits dependencies and flag rules whose triggering bit has no setter (orphan flowbits — classic silent-miss).
2. Validate metadata: created against rev — old rules never revved get a review flag.
3. Diff registries week-over-week and emit authoring/retirement/tuning velocity as BT20 feeds.
4. Check msg conventions (team prefix, no PII in msgs — EVE alerts ship widely).
5. Add a --sweep mode that verifies every active local SID’s rule still loads in a staged suricata -T test run.

Detection — Defender View

Signal	Source	ID	Notes
Rule hit volume per SID	EVE JSON	per sid	Base rate
FP rate per SID	BT1 verdicts join	per sid	Tuning trigger
Alert→case conversion	BT1 pipeline	per sid	Value measure
pcre-cost budget	Engine stats	per ruleset	The pcre tax
SID registry collisions	sidregistry gate	per merge	Zero expected
Local SID hygiene	Registry deltas	weekly	Authoring velocity
Reload failures	suricatasc	per deploy	Fail-loud deploy
pass-rule audit count	rulelint/review	quarterly	Blinding risk
Threshold direction check	rulelint	per merge	Suppression sanity
Coverage join (NIDS+EDR)	BT9 matrix	quarterly	One number, two layers

Evasion Techniques

Adversaries engineer against signature evaluation specifically. Reassembly games: session splicing (splitting content across packets below reassembly depth), overlapping segments, and TTL tricks aimed at the IPS’s inspection position (the classic pdels — packets the sensor sees but the endpoint drops) assume your reassembly depth and placement; normalizing engines (Suricata’s stream engine) defeat most of it, but reassembly depth stays a real knob to know. Threshold sliding: low-and-slow beaconing under count/seconds windows — the detection_filter direction lesson; the counter-rule is behavioral (BT9/BT7) not more thresholds. Content morphing: splitting markers across requests (path + header + body), case/encoding variations (double-URL-encoding, unicode overlong), and polymorphic shellcode defeat naive content matches — protocol-aware buffers with decoded views (http.uri vs raw, url_decode) and multiple anchor points blunt it. Encryption everywhere: TLS pushes the match surface to metadata — SNI, cert JA3/JA3S, cert lifetimes (self-signed, short-lived, Let’s Encrypt velocity from new infra) — the dataset approach matters more as payloads disappear. Protocol laundering: C2 over DNS, ICMP, or DoH tunneled through allowed protocols — app-layer rules per protocol plus rare-protocol alerts cover the channel list. pass-rule weaponization: adversaries who learn your pass rules (from an open-source-sourced ruleset or leaked config) shape traffic to ride them — pass rules get audited as adversarially as drop rules.

⚠️ What caught me: I wrote what I thought was my best rule ever — a beautiful pcre catching a C2’s campaign marker across HTTP body, headers, and URI variants — and deployed it alert-mode for the soak. Zero hits, zero FPs, perfect score. Three weeks later the same campaign walked past us on a customer, and the post-incident PCAP showed the marker arriving in sixty-four fragments, each 40 bytes, spread across a reassembly window my sensor had configured at 32 segments. The pcre never ran — the prefilter never assembled the pattern. The fix wasn’t a better regex; it was a reassembly-depth review, a cross-segment test in the corpus, and the humility lesson that a rule’s zero-hit soak proves nothing about a pattern it structurally cannot see.

Lab Practice

Build a full rule lifecycle in the lab. Engine: stand up Suricata on a tap/span port or an AF_PACKET test rig [verify current setup per distro]; confirm EVE JSON flows to a file. Author: write three rules for a synthetic C2 — one dns.query match, one http.uri content match, one tls.sni match — with sidregistry-assigned SIDs and metadata provenance. Gate: run rulelint --strict and sidregistry; deliberately break one rule (remove rev, put pcre first, duplicate a SID) and watch CI reject each. Test: replay a benign PCAP corpus (public datasets work [verify license]) plus a hand-crafted malicious PCAP; confirm zero FP and one TP per rule — the two-corpus discipline, network edition. Tune: inject a noisy scan and exercise a detection_filter — verify the first alerts survive and the repeat noise suppresses. Debrief: which check caught which authoring mistake, and which mistake would have reached production without the gate?

Interview Questions

• Technical: Walk a Suricata rule’s evaluation path — header, fast-pattern prefilter, sticky buffers, pcre — and explain where the pcre tax comes from.
• Technical: Why must local rules live in a registry with rev discipline, and what exactly breaks when two rules share a SID?
• Scenario: A high-value rule FPs on a critical business app hourly. Walk the tuning decision: threshold, buffer, content narrowing, or pass rule — and the risk each carries.
• Scenario: You must graduate a rule from alert to drop on a production IPS. What evidence does the soak period need to show, and what’s the rollback plan?
• Conceptual: Why is a pcre-only rule an architectural failure rather than a style nit?
• Deep-dive: Design the NIDS rules program for a multi-site estate: repo and CI, SID registry, FP feedback from triage, IPS graduation, and the metrics (BT20) you’d report.
• Practical: Your engine’s CPU sits at 85% and packet drops are climbing. What do you measure first — and which rule classes do you audit for the pcre tax?

Related Modules

BT3 (SIEM Engineering & Log Pipelines) · BT9 (Detection Engineering & Sigma) · BT12 (Wireless IDS) · BT13 (Network Segmentation) · E17 (Network Detection)