AV Engine Internals

Antivirus engines are the oldest and most misunderstood layer of the Windows security stack. Most operators treat AV as a black box that “detects bad files,” which is exactly wrong enough to burn them: engines are a pipeline of independent detection layers — byte signatures, heuristics, an emulator, behavior monitoring, and machine-learning classifiers — and each layer catches a different class of payload at a different stage of execution. This module dissects that pipeline using Microsoft Defender as the reference implementation, then makes you build a working miniature AV engine so the abstractions become concrete. Everything here is legal, defensive-adjacent knowledge: understanding how the scanner thinks is the same knowledge whether you are writing detections or testing payloads against them.

Definition

An AV engine is a usermode process that decides whether a file, memory region, script, or network stream is malicious, fed by a kernel minifilter that intercepts file I/O. In Defender’s case the pieces are:

Component	Artifact	Role
Minifilter driver	WdFilter.sys (altitude 328010)	Intercepts file create/read/write/close, passes candidates to the service
Scan service	MsMpEng.exe (Antimalware Service Executable)	Runs the engine: signatures, emulator, heuristics, ML
Network inspection	NisSrv.exe	Signature + protocol analysis on traffic (exploit kits, C2)
Definitions	*.vdm under C:\ProgramData\Microsoft\Windows Defender\Definition Updates	Signature + engine update packages, delivered ~hourly
Cloud verdicts	MAPS (Microsoft Advanced Protection Service)	Sub-second reputation/verdict for first-seen files

A detection name encodes which layer fired: Trojan:Win32/Wacatac.B!ml means family Wacatac variant B detected by a machine-learning model (!ml). Suffixes you will actually see in the wild: !ml (static/behavioral ML), !dll/!vba [verify] (script/host-specific layers), .gen (generic signature), .dam (damaged/modified family variant), @mm (mass-mailer capability). Generic and ML detections are the two layers red-team payloads hit most; exact byte signatures mostly catch commodity malware.

Why It Matters

Three reasons this module pays for itself. First, static AV is the first checkpoint your payload meets, and it is the cheapest to defeat — but only if you know which layer caught you. Dropping an unobfuscated Meterpreter binary gets a signature hit in milliseconds; the fix is not “more packing” (entropy triggers heuristics) but understanding that signatures match specific byte patterns and ML models match statistical texture. Second, defenders need to know what real-time protection actually inspects: file writes through the minifilter, script content through AMSI, process behavior through ETW — and what it does not inspect well (in-memory .NET assembly load [verify], exotic file systems, unmonitored mounts). Third, interviews at detection-engineering roles start here; “walk me through what happens when a file lands on disk” is a filter question, and the answer below is the passing one.

There is also a historical lesson. Pure signature engines lost to polymorphism in the 2000s, which forced vendors to add emulators (unpack once in emulation, match on the unpacked body) and then ML (classify the texture of the unpacked body). Every evasion technique you will learn in E4–E6 is a direct answer to one of those layers. Knowing the question tells you why the answer looks the way it does.

Detailed Explanation

The on-access scan pipeline

When any process touches a file, the I/O manager routes the operation through registered filter drivers in altitude order. WdFilter.sys sits at altitude 328010 — above most third-party filters — and on IRP_MJ_CREATE/IRP_MJ_WRITE posts the file to MsMpEng.exe:

        USER MODE                                  KERNEL MODE
  ┌─────────────────────────┐                 ┌──────────────────────┐
  │  MsMpEng.exe            │   verdict req   │  WdFilter.sys        │
  │  ┌───────────────────┐  │ ◄────────────── │  altitude 328010     │
  │  │ 1. cloud/MAPS     │  │   candidate     │  intercepts:         │
  │  │ 2. signatures     │  │   (path, hash,  │   IRP_MJ_CREATE      │
  │  │ 3. unpack/emulate │  │    context)     │   IRP_MJ_WRITE       │
  │  │ 4. heuristics     │  │ ──────────────► │   IRP_MJ_CLEANUP     │
  │  │ 5. static ML      │  │  scan request   │                      │
  │  │ 6. behavior (BM)  │  │                 │  verdict cached by   │
  │  └───────────────────┘  │                 │  FileId + USN        │
  │        │                │                 └──────────────────────┘
  │        ▼                │                          ▲
  │  clean → release I/O    │                          │ every open/write of
  │  threat → remediate     │                          │ any file, any process
  └─────────────────────────┘                          │
       (quarantine/notify)                     any process: Office,
                                               powershell.exe, explorer...

The scan pipeline inside MsMpEng.exe runs the stages roughly in that order, and the order matters for evasion: a file that satisfies an early layer (a hash on the MAPS deny-list, or an exact byte signature) never reaches the emulator; a file that the emulator cannot run (unusual entrypoint, delay execution, unsupported APIs) still gets handed to static ML, which never “crashes” and always produces a verdict.

What each layer actually matches

• Exact signatures — byte patterns (often with ?? wildcards over variable fields) matched against the file. Cheap, near-zero false positives, trivially defeated by changing one byte of the matched region.
• Generic signatures — families of related patterns (*.gen) that match a structural skeleton rather than one build, e.g. the loader stub shared by many crypter builds.
• Emulation — the engine loads the PE into an internal x86 emulator, runs the entrypoint for a bounded number of instructions, and matches signatures against the unpacked memory the payload produces. This is how packed malware gets caught without the real CPU ever executing it.
• Heuristics — static red flags: entropy per section, section name anomalies, TLS callbacks before entrypoint, known-suspicious imports, double extensions, self-modifying characteristics.
• Static ML (!ml) — Gradient-Boosted-Tree style classifiers [verify: exact model family is not public] over features extracted from the file: section entropies, import fingerprints, string statistics, resource layout. Trained on billions of verdicts, fast, tuned for low FP rate — and it is the layer that most “I only renamed variables” payloads actually trip.
• Behavior monitoring — post-execution: process + AMSI + ETW signals combined into detections like Behavior:Win32/.... This bridges into EDR territory (E2), which is exactly why modern Defender is no longer “just AV.”

How the emulator catches packed payloads — worked example

A UPX-packed shellcode loader has entrypoint code that: allocates memory, copies the packed body, decodes it, and jumps to the unpacked stage. The engine’s emulator runs those instructions in a sandboxed CPU model — no real process is created. When the decode loop finishes, the emulator holds the unpacked second stage in its virtual memory and re-runs signature matching over that memory, finding the raw shellcode pattern that was invisible in the packed file. Defeating this requires never decoding-and-jumping in a linear emulatable way: unbounded loops, sleeps, host-environment checks (E14, E15), or indirect system calls (E5) that the emulator does not model. That is precisely why modern loaders do all of those things.

Feature economics — what makes a file look “bad” to the model

The static ML layer does not read your source code; it reads statistics. The features that dominate verdicts for small utility binaries: mean/max section entropy, ratio of imported functions from VirtualAlloc/WriteProcessMemory/CreateRemoteThread, presence of URLs or base64 blobs, section name deviations (.text, .rdata vs custom names), timestamp anomalies, and overlay data after the last section. Two practical consequences: high entropy anywhere is a penalty (so naive packing increases detection), and a boring import table with one dangerous call is worse than a wide, plausible import table. Evasion is therefore a feature-shaping exercise, not an arms race of packers.

Common Techniques

Engine layer	Signal it matches	Typical detection	First-line counter
Exact signature	Byte pattern incl. wildcards	Trojan:Win32/<family>.A	Mutate the matched region; API hashing; string encryption
Generic signature	Structural skeleton	...<family>.gen	Change loader structure, not just bytes
Emulator	Unpacked body produced in emulation	Trojan:Win32/<family>!<n> [verify]	Emulation-aware: sleeps, env checks, indirect calls
Heuristics	Entropy, imports, section layout	Trojan:Win32/Sabsik_VE.A!ml	Manage entropy; legit-looking imports/sections
Static ML	Statistical texture of whole file	Trojan:Win32/Wacatac.B!ml	Reduce distinctive features; template diversity
Cloud/MAPS	First-seen hash + file rep	Immediate cloud block	Assume first execution is watched (it is)
Behavior/AMSI	Script content, process chains	Behavior:Win32/...	See E7 (ETW) and E8 (AMSI)

Full Working Code

This is the engine core of a miniature AV (scan_core.py): feature extraction (Shannon entropy, minimal PE parsing), wildcard signature matching, heuristic rules, and a weighted verdict — the same pipeline shape as a real engine, small enough to read in one sitting. Pure standard library, Python 3.8+.

{1-6}
"""
scan_core.py — miniature AV engine core.
Pipeline: features -> signatures -> heuristics -> weighted verdict.
Pure stdlib. Python 3.8+. Usage: from scan_core import scan_bytes
"""
import math
import re
import struct
from pathlib import Path

# ---------------------------------------------------------------- features
def shannon_entropy(data: bytes) -> float:
    """Entropy in bits/byte: 0.0 (uniform) .. 8.0 (perfectly random)."""
    if not data:
        return 0.0
    freq = [0] * 256
    for b in data:
        freq[b] += 1
    n = len(data)
    ent = 0.0
    for f in freq:
        if f:
            p = f / n
            ent -= p * math.log2(p)
    return ent


def parse_pe(data: bytes):
    """Minimal PE parse: machine, entrypoint, sections (+entropy per section)."""
    if len(data) < 0x40 or data[:2] != b"MZ":
        return None
    e_lfanew = struct.unpack_from("<I", data, 0x3C)[0]
    if e_lfanew + 24 > len(data) or data[e_lfanew:e_lfanew + 4] != b"PE\x00\x00":
        return None
    (machine, nsec, _ts, _symtab, _nsym,
     opt_size, _chars) = struct.unpack_from("<HHIIIHH", data, e_lfanew + 4)
    off = e_lfanew + 24                      # optional header start
    magic = struct.unpack_from("<H", data, off)[0]
    ep = struct.unpack_from("<I", data, off + 16)[0]
    dd_off = off + (96 if magic == 0x10B else 112)   # data directories
    import_rva = struct.unpack_from("<I", data, dd_off + 8)[0]
    sections, sec_off = [], off + opt_size
    for _ in range(nsec):
        name = data[sec_off:sec_off + 8].rstrip(b"\x00").decode(errors="replace")
        vsize, vaddr, rsize, raddr = struct.unpack_from("<IIII", data, sec_off + 8)
        raw = data[raddr:raddr + rsize]
        sections.append({
            "name": name, "rawsize": rsize,
            "entropy": round(shannon_entropy(raw), 3),
        })
        sec_off += 40
    return {"machine": hex(machine), "ep": ep,
            "sections": sections, "import_rva": import_rva}


# ---------------------------------------------------------------- signatures
def compile_sig(pattern: str):
    """'4D 5A ?? 00' -> compiled regex; ?? matches any single byte."""
    rx = b""
    for tok in pattern.split():
        rx += b"." if tok == "??" else re.escape(bytes([int(tok, 16)]))
    return re.compile(rx, re.DOTALL)


# suspicious API / string hints (real engines walk the IAT; we use raw hints)
API_HINTS = [b"VirtualAlloc", b"WriteProcessMemory", b"CreateRemoteThread",
             b"WinExec", b"URLDownloadToFileA", b"IsDebuggerPresent"]


# ---------------------------------------------------------------- verdict
DEFAULT_THRESHOLDS = {"malicious": 70, "suspicious": 40}


def scan_bytes(data: bytes, sigs: list, thresholds=DEFAULT_THRESHOLDS) -> dict:
    """Run the full pipeline over one file image. Returns a verdict dict."""
    report, score = {"findings": []}, 0

    # stage 1 — byte signatures (with wildcards)
    for sig in sigs:
        if sig["rx"].search(data):
            report["findings"].append(
                {"layer": "signature", "name": sig["name"], "score": sig["score"]})
            score += sig["score"]

    # stage 2 — PE heuristics
    pe = parse_pe(data)
    if pe:
        report["pe"] = {"machine": pe["machine"], "sections": pe["sections"]}
        for s in pe["sections"]:
            if s["rawsize"] > 4096 and s["entropy"] > 7.2:
                report["findings"].append(
                    {"layer": "heuristic", "name": f"high-entropy:{s['name']}",
                     "score": 30})
                score += 30
        hint_hits = [h.decode() for h in API_HINTS if h in data]
        if hint_hits:
            report["findings"].append(
                {"layer": "heuristic", "name": "suspicious-apis",
                 "detail": hint_hits, "score": 10 * len(hint_hits)})
            score += 10 * len(hint_hits)

    # stage 3 — weighted verdict (mirrors engine threshold logic)
    verdict = ("malicious" if score >= thresholds["malicious"]
               else "suspicious" if score >= thresholds["suspicious"]
               else "clean")
    report["score"], report["verdict"] = score, verdict
    return report


def scan_file(path: str, sigs: list) -> dict:
    data = Path(path).read_bytes()
    return scan_bytes(data, sigs)

Code Walkthrough

• shannon_entropy (lines 11–21) — builds a 256-bin frequency table and sums -p·log2(p). Encrypted or packed sections score near 8.0; .text code lands around 5.5–6.5 [verify for your compiler]; a 7.2+ large section is the classic packer tell, which is exactly why the heuristic stage uses that constant.
• parse_pe (lines 24–52) — validates the MZ magic, follows e_lfanew to the PE\0\0 signature, then unpacks the COFF header fields (H=u16, I=u32). magic == 0x10B distinguishes PE32 from PE32+ (0x20B) so the data-directory offset (96 vs 112 bytes into the optional header) is correct; DataDirectory[1] is the import table RVA. Sections are walked in 40-byte entries, and entropy is computed on raw bytes — the same statistic the !ml feature extractor uses.
• compile_sig (lines 57–63) — turns "4D 5A ?? 00" into a regex where ?? becomes .. re.DOTALL is required so . matches \n bytes inside binary data. Real engines use Aho–Corasick multi-pattern automata instead of N regexes; the semantics are identical.
• scan_bytes (lines 79–115) — the pipeline: signatures first (cheap, exact), then PE heuristics only when the file is a PE (scripts and blobs still get the signature pass — note report["pe"] is only set for PEs). Each finding carries its own score contribution; the verdict is a two-threshold cut of the total. Real engines add per-rule severity, cloud reputation as a prior, and ML as a separate scorer — the shape, though, is exactly this.
• Design decision to notice — findings are additive, not boolean. A file can be “suspicious” from suspicious-apis alone but “malicious” only when high entropy and injection APIs combine. That is why evasion advice is never “remove one string”: you must keep the total under the threshold.

How It Reaches The Target

End-to-end: an attacker’s dropper lands via phishing → WriteFile completes → WdFilter.sys posts the file to MsMpEng.exe → cloud/MAPS hash verdict on first sight (sub-second) → on-disk scan (signatures → emulator → heuristics → ML) → if suspicious, the file is quarantined (encrypted copy moved to C:\ProgramData\Microsoft\Windows Defender\Quarantine, original deleted) and Event 1116/1117 lands in the operational log → if clean, the minifilter caches the FileId verdict and subsequent opens are free. If the file executes before a verdict, behavior monitoring + AMSI take over: script content is scanned in-memory, child-process chains are evaluated, and the same MsMpEng.exe engine can kill the process mid-run. The defender-facing consequence: a first-stage that survives disk scan still has to survive its own execution.

Exploitation Chain

1. Delivery — phishing attachment or staged download lands the dropper on disk
2. On-access scan — WdFilter hands the file to MsMpEng within milliseconds of the write
3. Cloud verdict — MAPS answers on first-seen hash; unknown files get a bounded cloud-block window
4. Static pipeline — signatures miss, emulator runs the unpacker, heuristics + !ml score the texture
5. Verdict — clean: release I/O and cache; malicious: quarantine + Event 1116/1117
6. Execution (if it survived) — AMSI/behavior monitoring observe the running payload
7. Defender consequence — any stage firing produces telemetry the SOC sees (E11 wires this to SIEM)

Building Your Own Tool

Goal: a complete CLI scanner (avsim.py) with external JSON rules, directory recursion, JSON reports, and quarantine simulation — the tool you will actually use in labs to test whether your payloads trip a mini-engine, and to internalize feature shaping. Design decisions: rules live in JSON (not code) so you can grow a ruleset like a real vendor; the verdict is score-based like commercial engines; quarantine renames to .quar in a side directory (real quarantine encrypts — we simulate the flow, not the crypto).

Design decisions, explicitly:

1. Rules in JSON — signatures, string hints, entropy ceilings, thresholds; mirrors how .vdm definition files separate data from engine.
2. Score, not boolean — so you can watch a payload’s score move as you strip features.
3. No third-party deps — runs anywhere Python 3.8+ runs, including air-gapped lab VMs.

Full source

avsim/ ├── avsim.py ├── rules.json └── samples/ ├── benign_hello.bin ├── packed_loader.bin └── eicar.com

{1-3}
"""
avsim.py — miniature AV scanner CLI.
Usage:
  python avsim.py scan samples/ --rules rules.json --report report.json
  python avsim.py scan samples/eicar.com --rules rules.json --quarantine qdir
"""
import argparse
import json
import shutil
import sys
import uuid
from pathlib import Path

from scan_core import compile_sig, scan_bytes, DEFAULT_THRESHOLDS


def load_rules(path: str) -> list:
    """Load a rules.json and compile signature patterns."""
    cfg = json.loads(Path(path).read_text())
    sigs = []
    for s in cfg.get("signatures", []):
        sigs.append({
            "name": s["name"],
            "score": int(s.get("score", 50)),
            "rx": compile_sig(s["pattern"]),
        })
    return sigs, cfg


def string_hints(cfg) -> list:
    """String-based rules: {"name","terms","score"} — any hit counts once."""
    return cfg.get("strings", [])


def entropy_rules(cfg) -> list:
    return cfg.get("entropy", [])


def scan_one(data: bytes, sigs: list, cfg: dict) -> dict:
    """Extend the core engine with JSON string/entropy rules, then verdict."""
    rep = scan_bytes(data, sigs, cfg.get("thresholds", DEFAULT_THRESHOLDS))

    for rule in string_hints(cfg):
        hits = [t for t in rule.get("terms", []) if t.encode() in data]
        if hits:
            rep["findings"].append({
                "layer": "string", "name": rule["name"],
                "detail": hits, "score": rule.get("score", 20)})
            rep["score"] += rule.get("score", 20)

    for rule in entropy_rules(cfg):
        if rule.get("whole_file") and data:
            from scan_core import shannon_entropy
            e = shannon_entropy(data)
            if e > rule.get("above", 7.5):
                rep["findings"].append({
                    "layer": "heuristic", "name": "whole-file-entropy",
                    "detail": round(e, 3), "score": rule.get("score", 25)})
                rep["score"] += rule.get("score", 25)

    th = cfg.get("thresholds", DEFAULT_THRESHOLDS)
    rep["verdict"] = ("malicious" if rep["score"] >= th["malicious"]
                      else "suspicious" if rep["score"] >= th["suspicious"]
                      else "clean")
    return rep


def quarantine(path: Path, qdir: Path) -> Path:
    """Simulate quarantine: move to qdir with a .quar suffix + new name."""
    qdir.mkdir(parents=True, exist_ok=True)
    dest = qdir / f"{uuid.uuid4().hex[:12]}.quar"
    shutil.move(str(path), str(dest))
    return dest


def main() -> int:
    ap = argparse.ArgumentParser(description="avsim — mini AV engine")
    ap.add_argument("target", help="file or directory to scan")
    ap.add_argument("--rules", default="rules.json")
    ap.add_argument("--report", help="write JSON report here")
    ap.add_argument("--quarantine", help="move malicious files here")
    args = ap.parse_args()

    sigs, cfg = load_rules(args.rules)
    target = Path(args.target)
    files = sorted(target.rglob("*")) if target.is_dir() else [target]

    summary = []
    for f in files:
        if not f.is_file():
            continue
        data = f.read_bytes()
        rep = scan_one(data, sigs, cfg)
        rep["file"] = str(f)
        summary.append(rep)
        flag = {"clean": "[ ]", "suspicious": "[?]", "malicious": "[!]"}[rep["verdict"]]
        print(f"{flag} {rep['score']:>4}  {f}")
        for find in rep["findings"]:
            print(f"        - {find['layer']:<10} {find['name']} (+{find['score']})")
        if rep["verdict"] == "malicious" and args.quarantine:
            moved = quarantine(f, Path(args.quarantine))
            print(f"        quarantined -> {moved}")

    if args.report:
        Path(args.report).write_text(json.dumps(summary, indent=2))
        print(f"report -> {args.report}")
    return 0


if __name__ == "__main__":
    sys.exit(main())

{
  "signatures": [
    {"name": "EICAR-Test-File", "score": 100,
     "pattern": "58 35 4F 21 50 25 40 41 50 5B 34 5C 50 5A 58 35"}
  ],
  "strings": [
    {"name": "Inject-API-Set", "score": 20, "terms": [
      "VirtualAlloc", "WriteProcessMemory", "CreateRemoteThread"]},
    {"name": "Downloader-API", "score": 20, "terms": ["URLDownloadToFileA"]}
  ],
  "entropy": [
    {"whole_file": true, "above": 7.5, "score": 25}
  ],
  "thresholds": {"malicious": 70, "suspicious": 40}
}

Walkthrough

• load_rules (lines 15–26) — reads rules.json, compiles each signature pattern through the same compile_sig used by the core. Note the separation: data (JSON) vs engine (code), which is exactly how definition updates stay independent of engine updates.
• scan_one (lines 40–70) — chains the core engine with the two JSON-driven rule families. String rules fire once per rule (not per hit) to keep scores predictable; the whole-file entropy rule catches blobs that have no PE structure at all (shellcode, archives, and — importantly — over-packed payloads).
• quarantine (lines 73–79) — moves the file under a random name with a .quar suffix, mirroring the operational flow (real Defender additionally encrypts the quarantined copy with a per-file key and holds metadata to allow restore).
• main (lines 82–110) — recursion over directories, verdict glyphs ([!]/[?]/[ ]), per-finding detail lines, and optional JSON report. Exit code 0 regardless of verdicts — this is a lab tool, not a gate.

Compilation and test procedure

# no compilation — stdlib only; run from the avsim/ directory
$ python --version
Python 3.11.4
$ python -c "print('X5O!P%@AP[4\\PZX54(P^)7CC)7}\$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!\$H+H*')" > samples/eicar.com
$ python avsim.py scan samples/ --rules rules.json --report report.json
[!]  120  samples/eicar.com
        - signature EICAR-Test-File (+100)
        - heuristic  whole-file-entropy (+25)
[ ]    0  samples/benign_hello.bin
$ python avsim.py scan samples/ --quarantine qdir
[!]  120  samples/eicar.com
        quarantined -> qdir/9f3c1a2b7e8d.quar

Then the real exercise: take a benign WriteProcessMemory-style test harness from any earlier module, run it through avsim.py, and watch the score. Strip API imports one at a time (dynamic resolution via GetProcAddress shows up identically in string rules — but not in a real IAT walk) and watch “suspicious” drop to “clean.” You have just performed feature shaping against your own model, which is the same mental operation as evading !ml.

Detection considerations

If a defender wanted to detect this tool running in their environment (a legitimate use — it flags crude triage tooling), the tells are: python.exe spawning with avsim.py in the command line (Sysmon EID 1, CommandLine like %avsim.py%), burst reads of sample directories (Sysmon EID 11 sequence), and quarantine file-writes into a new directory (EID 11 with .quar naming). For the defender-side equivalents on real engines, see E11.

Improvements to add

1. Walk the real import table via import_rva instead of raw-string API hints (hint: parse RVA→file offset through section headers).
2. Add an emulator for a tiny x86 subset that runs one known unpacker stub, then re-scan the emulated memory — this demonstrates why emulation beats signatures.
3. Implement Aho–Corasick so 10k signatures scan in one pass; benchmark against N regexes.
4. Add a “cloud prior” — a JSON reputation list where unknown files get +15 score, mimicking first-seen suspicion.
5. Persist per-file verdicts keyed by content hash and skip rescans — the minifilter’s FileId cache, in usermode form.

Detection — Defender View

What the real engine emits when it processes your files — these are the events a SOC actually sees:

Source	ID	What fires	Notes
Defender Operational log (Microsoft-Windows-Windows Defender/Operational)	1116	Malware detected (detection name, path, threat ID)	The headline event; SIEM-priority
same	1117	Action taken (quarantine/remediation result)	Pair with 1116 for outcome
same	1121 / 1122	Attack Surface Reduction rule blocked / audited	ASR is Defender’s rule engine
same	5001	Real-time protection disabled	Tamper signal — page the on-call
same	5004	Preferences updated	Correlate with exclusions being added
same	5007	Configuration change (incl. exclusions, CloudBlockLevel)	Audit trail for Set-MpPreference abuse
Sysmon	11	FileCreate of the dropped payload just before 1116	Join on path to reconstruct delivery
Sysmon	1	Process create of the payload if it executed pre-verdict	Behavior-layer evidence
Wazuh	maps Defender EID 1116/1117 via its windows-security/defender decoders [verify rule IDs against your ruleset version]	Alert on 1116, watch 5001/5007 as high severity	MITRE ATT&CK mapping available in Wazuh ruleset

The 1116 event includes the detection name — which tells the defender which layer caught the payload (!ml vs signature), which is exactly the introspection you get from building your own engine above.

Evasion Techniques

• Feature shaping first, packing second: keep section entropies under ~7.0, keep a plausible import table, and avoid WriteProcessMemory + CreateRemoteThread co-occurring with VirtualAlloc in a 30 KB binary — that trio is the heuristic jackpot.
• Signature hygiene: hash your own payload before shipping; if a build trips, diff the bytes against the last clean build to find the matched region (this is the manual version of what signature engines match).
• Emulation-aware staging: never decode-and-jump linearly in one emulatable stretch — insert a bounded-but-slow spin (the emulator has an instruction budget), or gate stage two on a host property the emulator does not fake (see E14, E15).
• Living-off-the-land for the delivery stage: mshta, rundll32, and msbuild inline tasks (C# compiled in-memory) avoid dropping a suspicious-shaped first stage entirely.
• Cloud-block awareness: the first execution of a first-seen file rides a bounded cloud-block window; do not let your dropper be first-seen and API-rich in the same minute.
• Legitimate commands defenders watch for abuse of: Add-MpPreference -ExclusionPath C:\Users\Public (requires admin; Tamper Protection blocks the pure disable — Set-MpPreference -DisableRealtimeMonitoring $true — on current builds), and Set-MpPreference -CloudBlockLevel Default to weaken cloud verdicts where policy allows.
• AMSI/script-layer evasion is a separate discipline — see E8; ETW-based behavior evasion is E7; kernel-side telemetry bypass is E9’s topic. Do not conflate “AV clean” with “EDR clean”: E2 is the next wall.

⚠️ What caught me: the first time I shipped a “clean” loader, the verdict came back Trojan:Win32/Wacatac.B!ml before the payload ever executed — the !ml suffix told me exactly which layer ate it, and it wasn’t my (unmatched) shellcode signature; it was the statistical texture: one tiny section, huge overlay, entropy 7.9, and a two-function import table. I spent a week learning that evasion is a feature-budget problem, not a “pack it harder” problem — every packer I tried raised entropy and made the score worse. My rule now: if my own engine (avsim above) wouldn’t score it, no amount of obfuscation will save it from theirs.

Lab Practice

1. In a snapshot-reverted Windows VM, confirm real-time protection state with Get-MpComputerStatus and locate WdFilter.sys in the loaded filter list (fltmc filters — altitude 328010).
2. Drop the EICAR string into eicar.com and watch Events 1116/1117 arrive in eventvwr → Applications and Services → Microsoft → Windows → Windows Defender → Operational. Note the detection name and which log fields map to my avsim.py report fields.
3. Build and run avsim.py against your own earlier payloads (F-modules’ tools); record which features drive each score.
4. Read C:\ProgramData\Microsoft\Windows Defender\Support\MPLog-*.log — find EstimatedImpact entries and a scan trace of a file you just touched; this is the closest public look at engine internals [verify fields on your build].
5. YARA round-trip: write a YARA rule matching a unique string in a test payload, verify with yara64.exe rule.yar payload.bin, then port it to a rules.json signature and confirm avsim.py catches it. You now write detections and evasions against the same mental model.

Interview Questions

Technical: Walk through what happens between a file write completing and a Defender quarantine action — name every component in order. Technical: Why does naive packing often increase AV detection rates? Reference at least two engine layers in your answer. Deep-dive: The emulator is a process running your payload’s code in a sandbox. Design three independent ways to make it fail, and explain the engine’s fallback verdict path for each. Conceptual: Why do detection names ending in !ml dominate modern first-seen detections while classic exact signatures mostly catch commodity malware? Scenario: A payload scores 45/100 “suspicious” on your own engine. List five concrete feature changes that lower the score without changing functionality, and the layer each targets. Scenario: A client says “we have AV, so in-memory .NET post-exploitation is covered.” Assess that claim. Practical: Write the YARA rule you would ship to catch the EICAR test file, then explain why real engines do not use that rule for EICAR specifically.

Related Modules

E2 (EDR Internal Architecture) · E8 (AMSI Analysis) · E11 (Defender Detection Logic) · M14 (Executable Packing Analysis)