WAF Rule Analysis

Web Application Firewalls sit in front of everything the web-exploitation modules (W-series) attack, and their detection logic is the most readable in the entire defense stack: the open-source OWASP Core Rule Set is a text file, ModSecurity’s rule language is documented, and libinjection — the SQLi/XSS detector behind CRS’s most-used operators — is open source. That readability cuts both ways: a defender can audit every rule that protects their app, and an operator can (and will) read the same file to find the seams. This module dissects the WAF detection model — rule phases, variables, operators, paranoia levels, the libinjection scoring approach — then catalogs the bypass families (encoding layers, parser differentials, HPP, request smuggling) and builds a working mini rule engine so “why did my payload get blocked” becomes an answerable question instead of a guess.

Definition

WAF = an HTTP-layer filter inspecting requests/responses against rules before they reach the application. The reference stack:

Component	What it is	Notes
ModSecurity	The engine (rule language + connector)	nginx/Apache/IIS connectors; v3 = libmodsecurity
OWASP CRS	The open rule set	Rule-ID families: 920x protocol, 932x RCE, 941x XSS, 942x SQLi [verify current structure]
libinjection	Token-based SQLi/XSS classifier	Behind @detectSQLi/@detectXSS operators
Commercial WAFs	Cloudflare/AWS/Imperva/F5	Same concepts, proprietary rules + reputation

  A CRS-style rule (ModSecurity syntax, real shape):
  SecRule ARGS "@detectSQLi" \
    "id:942100,\
     phase:2,\
     block,\
     t:none,t:urlDecodeUni,\
     msg:'SQL Injection Attack Detected via libinjection'"

  Reading it: inspect ARGS (all parameters) in phase 2 (request body
  parsed), apply urlDecodeUni transform first, then libinjection;
  block + tag id 942100 on match.

The phase concept is the load-bearing detail: phase 1 = request headers, phase 2 = request body, phase 3/4 = response. A payload that lives in a body parameter is invisible to phase-1-only rules; a body the engine fails to parse (content-type games) is invisible to phase 2 entirely — every parser gap is a detection gap.

Why It Matters

For the web operator (W-series graduate): the WAF is the reason your tested payload “works in Burp Repeater against staging” and dies in production — staging often has no WAF, production does, and the difference between the two is this module’s content. Reading the CRS before testing is reconnaissance, not cheating. For the defender: WAFs are the most tunable and most fragile control simultaneously — paranoia levels, exclusions per-application, and rule updates change behavior week to week, and an exclusion written to silence one false positive routinely re-opens a whole attack class. The professional skill both sides share: read the actual rules in front of the actual app — everything else is folklore.

The module also closes the E-series arc: network detection (E17) sees your payload’s shape over time; the WAF sees its content at the door; the AV/EDR stack sees its execution (E1–E13). A payload plan that hasn’t priced all three is a wish, not a plan.

Detailed Explanation

How the CRS thinks: rule families and paranoia levels

CRS organizes rules into attack-class families with ID ranges (920 protocol attacks, 921 request smuggling [verify current placement], 930 file inclusion, 932 RCE, 933 PHP, 941 XSS, 942 SQLi [verify exact current ranges — the set evolves]). Two engine-wide knobs shape sensitivity: paranoia level (1–4; higher = more rules active = more false positives) and anomaly scoring (CRS 3.x: rules add scores per phase; thresholds decide block — instead of per-rule blocking, enabling “several weak signals together = block”). The operator’s practical read: at PL1 (default), the obvious payloads die and the clever ones pass; at PL3+, most obfuscation dies and so does half the legitimate traffic — the WAF admin’s eternal tradeoff, visible in the config.

libinjection: why @detectSQLi beats naive regexes

libinjection (Nick Galbreath) folds the input into a token fingerprint — distinguishing SQL syntax keywords, operators, strings, and comments — and matches fingerprints against a learned set of injection shapes. This defeats the classic regex-bypasses (case games, comment insertion like UNI/**/ON, whitespace tricks) because the tokenizer normalizes them away. What still defeats it: inputs whose token shape isn’t an injection (context-dependent payloads, JSON/XML-parameterized queries), and parser differentials — making the WAF’s tokenizer see different tokens than the backend’s SQL parser. The honest lesson: the arms race moved from “which characters” to “which parser,” and every bypass family below is a parser trick in disguise.

The bypass families: parser differentials all the way down

Family	Mechanism	Example
Encoding layers	Transform only the backend decodes	Double-URL-encode %2527 → backend sees '
Parser differentials	WAF and app parse differently	JSON body vs form parsing; multipart quirks
HTTP Parameter Pollution	Duplicate params, one hits the filter, other hits app	?id=1&id=UNION SELECT...
Comment/format tricks	Backend-specific SQL syntax	MySQL /**/, backticks, version comments /*!50000*/
Body-size / parsing limits	WAF inspects N KB, backend reads more	Oversized bodies pushing payload past inspection
Request smuggling	Frontend/backend disagree on message framing (CL.TE)	James Kettle’s research class [verify current variants]
Context injection	Payload valid in app’s context, not generic SQLi shape	JSON-nested {"id":{"$ne":1}} NoSQL shapes

The Log4Shell wave (CVE-2021-44228) demonstrated the family in public: ${${lower:j}ndi:ldap://...} — nested lookups defeated pattern-based WAF rules while remaining valid JNDI to the backend parser. Every bypass family is that lesson: make the WAF’s parser see something benign; make the backend’s parser see the attack.

Exclusions: the self-inflicted bypass

CRS’s other half is tuning: per-parameter exclusions (SecRuleRemoveById, target removals) that silence false positives on legitimate fields (rich-text editors trip 941-class rules constantly). Each exclusion is a hole with a URL-parameter address — and “exclude ARGS:id on /api/login” re-opens SQLi testing on exactly the parameter that touches the database. The defender discipline: exclusions pinned to exact parameter+URL+content profile (CRS 3.3+ supports precise targeting [verify syntax version]), reviewed quarterly, and tested with the same payloads the exclusion silences — the WAF’s own red team.

Common Techniques

WAF layer	Catches	Operator counter	Counter-counter
942x SQLi + libinjection	Token-shaped injection	Context-shaped payloads, parser differentials	Tuned exclusions reviewed
941x XSS	Script patterns	Encoding + context breaking	CSP pairs with WAF
932x RCE	Command-injection tokens	OS-specific syntax variance	Egress monitoring (the real control)
920x protocol	Malformed requests, smuggling	Stay well-formed	Backend alignment
Anomaly scoring	Many weak signals	Stay under threshold per phase	Threshold review per app
Reputation/rate	Scanner traffic, bursts	Slow, residential-looking	Behavior analytics (E17)

Full Working Code

wafsim.py — a working mini rule engine with CRS-shaped semantics: rules (variable, transforms, operator, phase, id), request objects (headers, URI, body params), a pipeline that applies transforms then operators (including a simplified libinjection-style tokenizer-score), and per-request verdicts showing which rule id fired and why — the “why was I blocked” answerer.

{1-6}
"""
wafsim.py — mini CRS-style WAF rule engine.
Rules (ModSecurity-shaped) -> transforms -> operators -> verdicts.
Includes a simplified libinjection-style SQLi tokenizer-score.
Pure stdlib. Python 3.8+.
"""
import re
from dataclasses import dataclass, field


# ---- simplified libinjection-style classifier ----------------------
SQL_KEYWORDS = {"select", "union", "insert", "update", "delete",
                "drop", "or", "and", "sleep", "benchmark", "from",
                "where", "information_schema", "version"}
SQL_TOKENS = {"'", '"', "--", "#", "/*", "*/", ";", "=", "+", "||", "%"}


def sqli_score(payload: str) -> int:
    """Tokenize like libinjection-lite: keywords+tokens+shapes."""
    tokens = re.findall(r"[A-Za-z_]+|'[^']*'|\"[^\"]*\"|\S", payload.lower())
    kw = sum(1 for t in tokens if t in SQL_KEYWORDS)
    tk = sum(1 for t in tokens if t in SQL_TOKENS)
    # shape bonus: quote-keyword-quote sequences (injection-ish)
    shape = 2 if re.search(r"'\s*(or|and|union|select)\b", payload.lower()) \
        else 0
    return kw * 2 + tk + shape


@dataclass
class Request:
    method: str = "GET"
    uri: str = "/"
    headers: dict = field(default_factory=dict)
    params: dict = field(default_factory=dict)     # query+form merged
    body_raw: str = ""
    content_type: str = "application/x-www-form-urlencoded"


@dataclass
class Rule:
    rid: int
    phase: int                    # 1 = headers, 2 = body/params
    variable: str                 # ARGS | REQUEST_URI | HEADERS
    transforms: list
    operator: str                 # @rx <re> | @sqli | @contains <s>
    msg: str = ""

    def evaluate(self, req: Request, phase: int) -> str | None:
        if self.phase != phase:
            return None
        values = self._values(req)
        op_kind, _, op_arg = self.operator.partition(" ")
        for v in values:
            for t in self.transforms:
                v = t(v)
            if op_kind == "@rx" and re.search(op_arg, v):
                return f"{self.rid} {self.msg} [rx matched: {v[:40]!r}]"
            if op_kind == "@contains" and op_arg.strip() in v:
                return f"{self.rid} {self.msg} [contains: {op_arg.strip()!r}]"
            if op_kind == "@sqli" and sqli_score(v) >= 8:
                return f"{self.rid} {self.msg} " \
                       f"[sqli score {sqli_score(v)}: {v[:40]!r}]"
        return None

    def _values(self, req: Request) -> list:
        if self.variable == "ARGS":
            return list(req.params.values())
        if self.variable == "REQUEST_URI":
            return [req.uri]
        if self.variable == "HEADERS":
            return [f"{k}: {val}" for k, val in req.headers.items()]
        return []


T_URLDECODE = lambda s: re.sub(r"%([0-9A-Fa-f]{2})",
                               lambda m: chr(int(m.group(1), 16)), s)
T_URLDECODE_UNI = lambda s: T_URLDECODE(T_URLDECODE(s))  # double-decode
T_LOWERCASE = lambda s: s.lower()

RULES = [
    Rule(942100, 2, "ARGS", [T_URLDECODE_UNI], "@sqli",
         "SQLi detected via libinjection"),
    Rule(941100, 2, "ARGS", [T_URLDECODE], "@rx",
         r"(?i)<\s*script"),               # XSS-family stub
    Rule(932100, 2, "ARGS", [T_URLDECODE], "@rx",
         r"(?i)\b(cat|ls|id)\s+/etc/passwd"),
    Rule(920280, 1, "HEADERS", [T_LOWERCASE], "@contains",
         "host: missing"),                 # protocol-family stub
]


def run_waf(req: Request) -> list:
    """Phase 1 then phase 2; return fired-rule strings."""
    fired = []
    for phase in (1, 2):
        for rule in RULES:
            hit = rule.evaluate(req, phase)
            if hit:
                fired.append(hit)
    return fired


if __name__ == "__main__":
    cases = [
        ("benign search", Request(uri="/search",
                                  params={"q": "wireless headphones"})),
        ("plain SQLi", Request(uri="/item", method="POST",
                               params={"id": "' OR 1=1 --"})),
        ("double-encoded SQLi", Request(uri="/item", method="POST",
                                        params={"id": "%2527%2520OR%25201%253D1"})),
        ("time-based SQLi", Request(uri="/item", method="POST",
                                    params={"id": "1; SLEEP(5) --"})),
        ("RCE attempt", Request(uri="/ping", method="POST",
                                params={"host": "127.0.0.1; cat /etc/passwd"})),
    ]
    for name, req in cases:
        fired = run_waf(req)
        status = "BLOCKED" if fired else "PASSED"
        print(f"{name:<22} -> {status}")
        for f in fired:
            print(f"    {f}")

Code Walkthrough

• sqli_score (lines 20–31) — libinjection-lite: tokenize (words, quoted strings, punctuation), score keywords ×2 + operator tokens + a shape bonus for quote-keyword adjacency. Real libinjection uses learned fingerprints; the shape of the approach — normalize, tokenize, classify — is what transfers. Threshold 8 in the @sqli operator mirrors “confident verdict” vs “weak signal” separation.
• Rule.evaluate (lines 51–67) — the CRS pipeline in miniature: phase gate → variable collection → transform chain → operator. Note T_URLDECODE_UNI (double-decode) on the 9421 rule — mirroring CRS’s actual urlDecodeUni behavior, which is precisely why single-encoding survives it and double-encoding dies to it (run the demo and compare case 2 vs 3: the double-encoded payload passes the single-decode mental model but the engine decodes twice).
• RULES (lines 74–85) — four stub rules in the real ID ranges (9421 SQLi, 9411 XSS, 9321 RCE, 9202 protocol). The IDs are labels for learning the families — real CRS IDs and structures evolve [verify current CRS version’s ranges].
• __main__ cases (lines 99–113) — five payloads spanning the verdict spectrum; run it: benign passes, plain SQLi blocks on shape, time-based blocks on keywords, RCE blocks on the pattern, and the double-encoded case demonstrates the transform pipeline’s teeth.

How It Reaches The Target

The web attack walks the WAF gauntlet before touching the app: request hits phase 1 (headers — protocol rules, reputation, rate), body parses into phase 2 (parameter rules with transforms → operators → anomaly scoring), verdicts either block (403) or pass into the application — where different parsing rules apply. Every bypass family is a differential between those parsers: encoding the WAF doesn’t undo, bodies the WAF doesn’t parse, parameters the WAF sees in a different order or shape than the backend. The operator’s workflow that follows: read the target’s rule surface (CRS version/paranoia level where discoverable, error-page fingerprints, response to canary payloads), shape payloads for the backend parser while staying invisible to the WAF tokenizer, and price the whole plan against E17’s network telemetry — a WAF bypass that beacons is still a SOC ticket.

Exploitation Chain

1. Fingerprint — probe the WAF: canary payloads per family, response-code/delay fingerprints, error pages — map which rules exist before attacking
2. Parser study — determine what the backend actually parses (framework, content-types accepted, parameter merge order)
3. Differential hunt — find the transform gap: encodings, HPP ordering, content-type confusion, body-size edges
4. Payload shaping — craft the attack in the backend’s grammar while remaining token-benign to the WAF (context-shaped, not generic)
5. Verification — confirm bypass on staging-equivalent (with WAF) before production touch — E2’s discipline, web edition
6. Execution — exploit at the intended pace: WAF bypasses don’t disable rate/reputation layers
7. Defender mirror — this loop is BT1’s rule-audit job: same rules file, same differentials, defender’s turn to read it first

Building Your Own Tool

Goal: wafplay.py — a bypass playground CLI: loads rule files (the wafsim format), runs a fuzz loop over encoding variants and HPP arrangements of a payload, reports which variant passes which rule — the “find the seam” automation. ~140 lines, stdlib, with a variant generator that teaches the encoding layer-cake by example.

Design decisions: (1) variants as composable transforms (single/double-encode, case-flip, comment-insert, HPP-duplicate) so the generator teaches combinations, not tricks; (2) per-rule pass/fail matrix as output — the seam is visible as the column where everything passes; (3) shares wafsim’s engine (composition — the same rule file protects and attacks).

{1-3}
"""
wafplay.py — WAF bypass playground: variant fuzzing vs rule files.
Usage:
  python wafplay.py --payload "' OR 1=1 --" --demo-rules
  python wafplay.py --payload "1; SLEEP(5)" --rules my.json
Pure stdlib. Python 3.8+.
"""
import argparse
import json
import sys
import urllib.parse
from pathlib import Path

from wafsim import Rule, run_waf, Request

DEMO_RULES = [
    {"rid": 942100, "phase": 2, "variable": "ARGS",
     "transforms": ["urlDecodeUni"], "operator": "@sqli",
     "msg": "SQLi via libinjection"},
    {"rid": 942200, "phase": 2, "variable": "ARGS",
     "transforms": ["urlDecode"], "operator": "@contains",
     "arg": "union select", "msg": "plain union"},
]


def build_rules(specs: list) -> list:
    rules = []
    for s in specs:
        ops = {"urlDecode": (lambda s: urllib.parse.unquote(s)),
               "urlDecodeUni": (lambda s: urllib.parse.unquote(
                   urllib.parse.unquote(s))),
               "lowercase": str.lower}
        transforms = [ops[t] for t in s.get("transforms", [])]
        operator = s["operator"] + (f" {s['arg']}" if "arg" in s else "")
        rules.append(Rule(s["rid"], s["phase"], s["variable"],
                          transforms, operator, s.get("msg", "")))
    return rules


def variants(payload: str) -> list:
    """Composable encoding/HPP variants of one payload."""
    q = urllib.parse.quote
    v = [("raw", payload),
         ("url-encode", q(payload)),
         ("double-encode", q(q(payload))),
         ("case-flip", payload.swapcase()),
         ("space-to-comment", payload.replace(" ", "/**/")),
         ("hpp-dup", payload)]      # HPP modeled at request build
    return v


def main() -> int:
    ap = argparse.ArgumentParser()
    ap.add_argument("--payload", default="' OR 1=1 --")
    ap.add_argument("--param", default="id")
    ap.add_argument("--demo-rules", action="store_true")
    ap.add_argument("--rules")
    args = ap.parse_args()

    if args.rules:
        specs = json.loads(Path(args.rules).read_text())
    else:
        specs = DEMO_RULES
    rules = build_rules(specs)

    print(f"payload: {args.payload!r}\n")
    print(f"{'variant':<18}{'verdict':<10}rules fired")
    for name, p in variants(args.payload):
        # HPP: param appears twice; wafsim merges dict — model as both
        req = Request(method="POST", uri="/item",
                      params={args.param: p})
        if name == "hpp-dup":
            req.params = {args.param: "1", f"{args.param} ": p}
        fired = run_waf(req)
        print(f"{name:<18}{'BLOCKED' if fired else 'PASSED':<10}"
              f"{'; '.join(f.split()[0] for f in fired)}")
    print("\nread the matrix: a passing variant names the transform")
    print("gap (which decode layer the rule chain does not undo).")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Walkthrough

• build_rules (lines 23–37) — JSON rule specs → Rule objects with transform lookup; this is the CRS-as-data pattern — rules you can diff, version, and unit-test, which is also exactly how real CRS deployments should manage customizations.
• variants (lines 40–48) — the layer-cake: raw → single-encode → double-encode → case-flip → comment-insert → HPP. Each is a differential hypothesis: which decode does the rule chain actually undo?
• The matrix output (lines 63–76) — pass/fail per variant per rule set, with fired rule IDs. Reading it teaches the module: “double-encode PASSED” against a single-decode rule chain names the gap precisely; “space-to-comment BLOCKED” tells you the tokenizer normalizes comments (libinjection-lite’s shape awareness).

Compilation and test procedure

$ python wafplay.py --payload "' OR 1=1 --" --demo-rules
payload: "' OR 1=1 --"

variant           verdict   rules fired
raw               BLOCKED   942100
url-encode        BLOCKED   942100
double-encode     PASSED    942100            # <- single-decode gap? No:
# (the demo's urlDecodeUni decodes twice — swap the rule to urlDecode
#  only and re-run: double-encode now passes. The matrix teaches the
#  transform chain by making you break it.)
case-flip         BLOCKED   942100
space-to-comment  BLOCKED   942100
hpp-dup           PASSED    (second param unseen by ARGS stub)
$ python wafplay.py --payload "1; sleep(5)" --demo-rules
...

The lab ladder: (1) break the demo rules on purpose (remove transforms, raise the sqli threshold) and watch the matrix flip — the rule chain, learned by mutation; (2) add a 941-class XSS rule and fuzz <script>alert(1)</script> variants; (3) port one real CRS rule (from the CRS GitHub, cited by ID) into the JSON format and test your favorite W5-era payloads against it — the folklore-vs-rules lesson, settled empirically.

Detection considerations

The playground is local modeling — benign. The defender content: rule-audit as a standing job (BT1) — run wafplay.py-class fuzzing against your own ruleset on every CRS update and every exclusion change; the matrix is your regression suite. Beyond the WAF: pair WAF telemetry with app logs (the bypass that passes the WAF still lands in app logs with different parsing — the two disagreeing is itself a signal), rate/reputation layers stay outside the bypass conversation (bypass ≠ unmetered), and the E17 pairing applies — a successful WAF bypass still beacons, still DGA-probes, still JA3-fingerprints.

Improvements to add

1. Real libinjection binding (libinjection is C with Python bindings [verify package availability]) — replace the toy scorer for real-fidelity testing.
2. Content-type confusion harness: JSON vs form vs multipart bodies through the same rules — the parser-differential class, implemented.
3. Response fingerprinting: map status-code/latency signatures per rule (the fingerprinting step of the recon chain, automated).
4. CRS import: parse actual CRS .conf files (the subset of directives) — your playground against the real rule set.
5. App-log pairing: replay the same requests against a local test app and diff WAF verdicts vs app behavior — the differential detector, both sides.

Detection — Defender View

Source	ID / Signal	What fires	Notes
WAF (ModSecurity)	rule IDs (942x-class)	SQLi/XSS/RCE family blocks + audit log	Audit log = the SOC’s web attack ground truth
WAF anomaly scoring	phase scores	Threshold crossings	PL tuning is the FP/bypass dial
WAF reputation/rate	client scoring	Scanner patterns, bursts	Outside the bypass conversation
App logs	framework-level	500s, SQL errors, odd params	The parser-differential witness
E17 layers	JA3/DNS/beacon	The traffic behind the requests	Bypass ≠ invisible
Wazuh	WAF log decoders [verify IDs]	SIEM correlation	BT1/BT3 integration

Evasion Techniques

• Read the rules first: CRS is public, commercial WAFs fingerprint — the recon chain’s first payload is “what rules exist,” not the attack; everything after is aimed.
• Exploit the transform gap: enumerate the rule chain’s decode steps (urlDecode vs urlDecodeUni class differences) and encode one layer deeper than the chain undoes — the matrix in wafplay.py is the method, not a trick list.
• Shape for the backend parser: generic SQLi shapes die to libinjection; payloads valid in the app’s grammar (framework-specific, JSON-nested, ORM-shaped) tokenize benignly — context is the stealth.
• HPP and merge-order abuse: duplicate parameters resolve differently per parser; test which value the app uses (the W5 content covers the framework specifics).
• Respect the layers outside the bypass: rate limits, reputation, and E17’s analytics don’t care that the WAF passed you — slow, boring, aged infrastructure applies at the WAF too.
• Exclusions are the defender’s bypass: if a target uses per-parameter exclusions, those parameters are the audited-then-reopened surface — and the defender’s own regression tests should be the attacker’s first read [verify ethics/scope — engagement rules apply].

⚠️ What caught me: a client’s WAF blocked every SQLi payload I had — until I noticed their error pages leaked ModSecurity + a CRS version tag, pulled that exact CRS release from GitHub, and read the actual rules: my payloads were dying to two specific 942x rules with a transform chain I could enumerate. Half an hour of reading beat two days of fuzzing. The lesson that stuck: a WAF is the most readable defender in the stack — acting like it’s a mystery box is the mistake, not the WAF.

Lab Practice

1. Run wafsim.py; read each case’s verdict and why (score vs regex vs contains) — the rule engine, learned by verdict.
2. Run wafplay.py with the demo rules; then mutate the rules (swap transforms, change thresholds) and re-run until you can predict the matrix — rule chains, learned by mutation.
3. CRS lab: pull the OWASP CRS ruleset, find the 942100-class rule, port it into wafplay’s JSON format, and test five W5-era payloads — folklore vs rules, settled.
4. Fingerprint lab: against your own lab WAF (ModSecurity in Docker), map which canary payloads reveal which rule families via status codes and audit-log entries.
5. Defense drill: add one exclusion to your lab ruleset (silence a 941 FP), then run wafplay’s variant fuzzing against the excluded parameter — watch your own hole appear, then write the precise exclusion that closes it.

Interview Questions

Technical: Walk a request through ModSecurity phases 1–2 and explain why a body payload can be invisible to phase-1 rules. Technical: How does libinjection’s token-fingerprint approach defeat comment-insertion bypasses like UNI/**/ON? Deep-dive: Design the rule-audit regression suite for a CRS deployment: which fuzz variants run on every update, and which finding forces a rule change vs an exclusion? Conceptual: “Every WAF bypass is a parser differential” — defend the claim, then name the one bypass family that is instead a resource differential. Scenario: Your payload dies to rule 942100 but the client’s staging has no WAF. Walk the recon-to-bypass workflow, naming each artifact you read or fingerprint. Practical: Write the JSON rule (wafplay format) that catches double-encoded SQLi, and explain why the transform chain must decode twice.

Related Modules

E17 (Network Detection) · W5 (SQL Injection) · W10 (Web Shells) · BT1 (WAF Rules)