Skip to content
Cybersecurity Handbook Pro

Cryptography Fundamentals

Every red-team pivot and every blue-team forensic claim rests on whether you can tell AES-GCM from AES-ECB, whether you can read a TLS 1.3 handshake, and whether you know why SHA-256(key || msg) is broken. Heartbleed, POODLE, ROCA, CurveBall, the Java ECDSA psychic-signature bug — none of these are magic. They are all consequences of small, specific failures in primitive selection or protocol framing. This module walks through the primitives, the modes, the padding, and the public-key constructions that everyone in security is expected to know cold.

The Core Concept

Section titled “The Core Concept”

Modern crypto rests on three pillars:

  1. Symmetric encryption — one key, fast, AES/ChaCha20. Used for the bulk of data.
  2. Asymmetric encryption — keypairs, slow, RSA/ECDSA/ECDH. Used for key exchange and signatures.
  3. Cryptographic hashing — one-way, fixed-length, SHA-2/SHA-3. Used for integrity, commitments, and MACs.
┌──────────────────────────────────────────────────────────────┐
│  TLS 1.3 record (after handshake)                           │
│   Ciphertext = AES-GCM(                                        │
│      key = HKDF-Expand(secret, "key", 32),                    │
│      nonce = HKDF-Expand(secret, "iv", 12) XOR seq,           │
│      aad = TLSCiphertext header,                               │
│      plaintext = HTTP request )                              │
│   + GCM auth tag (16 bytes)                                  │
└──────────────────────────────────────────────────────────────┘

Every TLS session starts with ECDHE key agreement on a curve (X25519, P-256), feeds the shared secret through HKDF to derive traffic keys, then uses AES-GCM or ChaCha20-Poly1305 to encrypt records. There is no RSA key exchange in TLS 1.3 — that died with TLS 1.2 because it lacked forward secrecy.

How It Works

Section titled “How It Works”
  1. AES — Rijndael with a fixed 128-bit block size. Key size 128/192/256 → 10/12/14 rounds. Each round: AddRoundKey → SubBytes (S-box, nonlinear) → ShiftRows → MixColumns (over GF(2⁸)). Hardware AES-NI: AESENC xmm, xmm is one instruction per round on x86; ARM has AESE/AESD. A modern CPU does 1–2 cycles/byte on AES-256-GCM.
  2. Modes of operation:
    • ECB — encrypt each block independently. Identical plaintext blocks → identical ciphertext. The ECB penguin (https://cryptolosophy.org/ecb-penguin). Never use.
    • CBC — C[i] = E(K, P[i] XOR C[i-1]), with C[0] = IV. The IV must be unpredictable (not just unique), or you get a chosen-plaintext distinguisher. CBC has padding oracles: POODLE (CVE-2014-3566) and the Lucky13 timing attack.
    • CTR — C[i] = P[i] XOR E(K, nonce || counter[i]). Turns a block cipher into a stream cipher. Nonce reuse = plaintext XOR plaintext, full break.
    • GCM — CTR plus a GHASH MAC over AAD and ciphertext. Auth tag is 128 bits. Nonce reuse is catastrophic — leaks the GHASH key H = E(K, 0).
    • ChaCha20-Poly1305 — stream cipher (20-round ARX, 256-bit key, 96-bit nonce) with Poly1305 MAC. Constant-time, software-friendly; default for mobile.
  3. RSA — n = p*q, e = 65537 (0x10001), d = e⁻¹ mod φ(n). Encrypt c = m^e mod n, decrypt m = c^d mod n. The keys live or die by:
    • Padding — PKCS#1 v1.5 (Bleichenbacher oracle since 1998, still a problem) or OAEP (RFC 8017).
    • Key generation — ROCA (CVE-2017-15361) generated keys with a fingerprintable structure of p mod M so any e = 65537 key was factorable.
    • Signing — same as encryption but inverted; PSS is the modern signature scheme.
  4. Hash functions:
    • SHA-1: 160-bit output, broken at $2⁶³·⁴ by the SHAttered collision in 2017 (chosen-prefix collision ~$2⁶³·⁴). Still common in older certs.
    • SHA-256: 64-byte blocks, 64 rounds. The default.
    • SHA-3 (Keccak): sponge construction, different from Merkle-Damgård — no length-extension attacks.
    • Length extension: H(secret || msg) is vulnerable because H(msg || extra) is computable from H(secret || msg) without knowing secret. HMAC fixes this.
  5. HMAC — HMAC(K, m) = H((K ⊕ opad) || H((K ⊕ ipad) || m)) where ipad = 0x36 and opad = 0x5c, repeated to the block size. Used in JWTs (HMAC-SHA256), AWS SigV4, IPsec.
  6. X.509 / PKI — an ASN.1 DER-encoded structure. PEM is base64(DER) wrapped in -----BEGIN CERTIFICATE-----. A chain is leaf → intermediate → root. The signature on each cert is verified against the issuer’s public key. Wildcard SANs (*.example.com) are allowed but only on the leftmost label.

Common Techniques

Section titled “Common Techniques”
TechniqueDescriptionDetection Difficulty
Heartbleed (CVE-2014-0160)OpenSSL TLS heartbeat reads n bytes past payload; n from attacker → memory leakEasy — IDS on heartbeat_request with payload_length > record_length
POODLE (CVE-2014-3566)SSLv3 CBC padding oracle: last byte = padding length, not authenticatedEasy — disable SSLv3
ROCA (CVE-2017-15361)Infineon RSA keys with p mod M pattern → factorable in $2⁵⁰Medium — fingerprint the modulus
CurveBall (CVE-2020-0601)Windows CryptoAPI ECC cert validation accepts self-signed cert if curve parameters are spoofed for a named curveMedium — alert on certs with custom EC parameters
Java ECDSA forgery (CVE-2022-21449)r=0, s=0 accepted as a valid signature for any messageTrivial — signature validation check
Lucky 13 (CVE-2013-0169)TLS CBC timing oracle on MAC vs padding checkHard — constant-time impl required
SWEET32 (CVE-2016-2183)64-bit block cipher (3DES) birthday collision after $2³² blocksEasy — disable 3DES

Hands-On Commands

Section titled “Hands-On Commands”
Terminal window
# Inspect a real TLS 1.3 handshake with cipher selection
openssl s_client -connect cloudflare.com:443 -tls1_3 -ciphersuites TLS_AES_256_GCM_SHA384
#   "-ciphersuites" is for TLS 1.3; "-cipher" is for TLS 1.2 and below.


# Show the negotiated cipher, certificate chain, and server key share group
echo | openssl s_client -connect github.com:443 -showcerts 2>&1 | grep -E 'Cipher|Server Temp Key|subject=|issuer='


# Decode an X.509 certificate from base64 to human-readable ASN.1
openssl x509 -in cert.pem -noout -text | head -40
openssl x509 -in cert.pem -noout -serial -dates -fingerprint -sha256


# Verify a chain with a custom CA bundle
openssl verify -CAfile ca-bundle.pem leaf.pem


# Generate a strong Ed25519 keypair (modern, default in OpenSSH 8.0+)
ssh-keygen -t ed25519 -a 100 -f ~/.ssh/lab_ed25519


# Generate an ECDSA P-256 certificate signing request
openssl ecparam -name prime256v1 -genkey -noout -out ec.key
openssl req -new -key ec.key -out ec.csr -subj "/CN=lab.example.com"


# Generate a 4096-bit RSA key (slow; use ECDSA or Ed25519 for new deployments)
openssl genrsa -aes256 -out rsa.key 4096


# SHA-256 a file (NOT for passwords)
sha256sum /etc/passwd


# bcrypt a password (use -c to set cost; 12 is a reasonable default in 2024)
htpasswd -nbB -c 12 user password123 | cut -d: -f2
#   Output: $2y$12$....  Cost 12 means 2^12 = 4096 rounds.


# Argon2id — modern winner of the Password Hashing Competition
argon2 somesalt -id -t 3 -m 16 -p 1 -l 32 < password.txt

Detection — Defender View

Section titled “Detection — Defender View”

Suricata SID 2209436 — “TLS invalid record length (Heartbleed probe attempt)” [verify — Emerging Threats has rules under sid 2200000+ for Heartbleed probes].

Suricata SID 2019419 — “ET POLICY SSLv3 outbound connection (POODLE vulnerable)” [verify].

Suricata SID 2028325 — “ET INFO TLS Handshake Failure” (some automated scanners hit this when probing for downgrade support).

MITRE ATT&CK T1040 — Network Sniffing (capture TLS handshakes; defensive value: detect Heartbleed probes against your public surfaces).

MITRE ATT&CK T1552.004 — Unsecured Credentials: Credentials in files (private keys on disk — find / -name id_rsa -o -name '*.pem' 2>/dev/null).

MITRE ATT&CK T1552.001 — Unsecured Credentials: Credentials In Files (the same technique, used defensively here — secret scanning on commit history).

Wazuh rule 920 / 502 — weak cipher / SSLv3 detected [verify — Wazuh has built-in Suricata integration alerting on weak-cipher events].

For application-level detection: alert on any client hello with TLS 1.0, TLS 1.1, or SSLv3; alert on any TLS 1.2 handshake that negotiates a CBC cipher (LUCKY13 surface); alert on any certificate in the chain with SHA-1 signatures after 2017.

Evasion Techniques

Section titled “Evasion Techniques”

Nonce reuse in CTR / GCM. Not really an evasion — a bug — but the consequence is catastrophic: same nonce + same key → C₁ XOR C₂ = P₁ XOR P₂. Recover P₁ from P₂ by crib-dragging. AES-GCM nonce reuse additionally leaks H = E(K, 0) and allows universal forgeries.

POODLE fallback. Force the client to downgrade from TLS 1.2 to SSLv3 by injecting a TLS handshake failure. Defense: TLS_FALLBACK_SCSV (RFC 7507), disable SSLv3 entirely (SSLProtocol All -SSLv3 -SSLv2 in Apache).

RC4 nostalgia. RC4 was removed from TLS 1.1+, but ICS and embedded devices still ship it. RC4 has distinguishers after $2²⁶ keystream bytes — never accept RC4 ciphers.

Pinning through Windows CryptoAPI bypass (CVE-2020-0601). An attacker who can mint an ECC cert with custom curve parameters where G is replaced by an attacker-chosen point can sign it with their own private key, and Windows will trust it as if it were signed by Microsoft. Defense: check that all certs use named curves (secp256r1, secp384r1, curve25519) with no embedded parameters.

Java ECDSA psychic signature (CVE-2022-21449). A signature (r=0, s=0) was accepted by Java’s ECDSA verifier for any message. Defense: confirm the verifier rejects r==0 || s==0 and r==n || s==n (test vectors in RFC 6979).

Bleichenbacher’s e=3 signature forgery. Some verifiers checked m^e mod n == expected without verifying the padding structure. With e=3 and a chosen suffix where expected is small enough, you can forge by computing a cube root. Defense: verify the full PKCS#1 v1.5 structure including leading 00 01 FF...FF 00 ASN.1 HASH.

AttackPrimitive brokenFix
HeartbleedOpenSSL TLS heartbeat parsingBounds check on record length
POODLESSLv3 CBC paddingDisable SSLv3
Lucky 13TLS CBC MAC-then-pad timingConstant-time comparison
SWEET3264-bit block birthdayDisable 3DES / Blowfish / legacy
ROCAInfineon RSA keygenRe-generate keys; check with roca-detect
CurveBallECC cert validationReject custom EC parameters
Java ECDSASignature r=s=0Reject zero r/s

⚠️ What caught me: A JWT validation library that compared HMAC(server_secret, body) to the user-supplied signature using == in Python. == on bytes is constant-time in CPython, but == on str is not constant-time — it returns False on the first non-matching character. The library did .decode() before comparison. With enough requests and a timing measurement at the millisecond level on a quiet network, you could forge a JWT signature byte-by-byte. Lesson: it’s not the algorithm that bites you; it is the comparison function next to it. Use hmac.compare_digest(a, b) and never a == b.

Lab Practice

Section titled “Lab Practice”

VM: Ubuntu 22.04 with openssl, libssl-dev, python3-cryptography, hashcat, john. Plus a second VM with a Go-based TLS server (crypto/tls with a configurable cipher list).

Objective:

  1. Generate a 2048-bit RSA keypair. Sign a 32-byte random message with PKCS#1 v1.5 SHA-256 padding. Then sign the same message with r=s=0 using the CVE-2022-21449 vector. Verify both with OpenSSL and confirm OpenSSL rejects the second.
  2. Capture a TLS 1.2 CBC-mode handshake to a deliberately vulnerable server (openssl s_server -www -tls1_2 -cipher 'AES128-SHA'); show the CBC negotiation, then identify the Lucky13 surface.
  3. Use testssl.sh --severity HIGH against https://localhost:8443 and identify every weakness flagged.
  4. Build a Python script that performs a length-extension attack on SHA256(key || msg) given only H(key || msg) and len(key). The script should append &admin=1 without knowing the key.

Expected flag format: flag{rsa_pss_ok_ecdsa_psychic_rejected_lucky13_surface_confirmed}.

Detection that should fire: testssl.sh reports LOW: Cipher AES128-SHA (1/0x2f) CBC, MEDIUM: SHA1 in certificate chain, HIGH: TLS1.0 offered (deprecated). Your signature test should produce verification Failure: 0 from OpenSSL on the forged signature.

Interview Questions

Section titled “Interview Questions”

Conceptual: explain why H(secret || message) is insecure as a MAC but HMAC(secret, message) is not, in terms of the Merkle-Damgård construction.

Technical: walk through what happens during a TLS 1.3 handshake from ClientHello to the first encrypted record. Name the four values derived through HKDF and what each protects.

Scenario: a customer insists on RSA key exchange in their TLS configuration “for compatibility.” List three concrete failures of RSA key exchange (no forward secrecy, decryption oracle via Bleichenbacher, key theft = bulk decryption) and propose one migration plan that maintains compatibility for legacy clients while protecting modern clients.

What is the difference between ECB, CBC, CTR, and GCM in terms of (a) parallelism, (b) nonce requirements, (c) authentication, and (d) failure mode on nonce/IV reuse?

Technical: describe the Bleichenbacher e=3 signature forgery from 2006 (CVE-2006-4339). Why does it require e=3 specifically, and why does it work even though m^3 < n?

Section titled “Related Modules”

→ F1 (Networking Fundamentals) · F10 (Security Models & Principles)