Skip to content
Cybersecurity Handbook Pro

Linux Internals

Every Linux exploit is a syscall story. When you read CVE-2022-0847 Dirty Pipe or CVE-2021-4034 PwnKit, what you are really reading is “an attacker found a way to make the kernel do something it shouldn’t, by feeding it carefully crafted data through a syscall.” If you cannot list the syscall table for x86_64, do not know what lives in /proc/<pid>/maps, and have never stared at a systemd unit file, you cannot reason about Linux security beyond copy-pasting payloads.

The Core Concept

Section titled “The Core Concept”

Linux is a monolithic kernel running in ring 0; user code runs in ring 3 and reaches the kernel only through the syscall gate. The transition is syscall (opcode 0x0F05) on x86_64, which switches CPU mode, swaps the stack to the kernel stack, and jumps to a dispatch table indexed by rax.

┌──────────────────────────────────────────────────────────┐
│  User space (ring 3)                                     │
│   ┌───────────┐    libc.so.6   ┌──────────┐             │
│   │  app.c    │ →  glibc       │ syscall  │             │
│   └───────────┘   wrappers     │  0x0F05  │ ──┐         │
│                                                  │      │
└──────────────────────────────────────────────────┼──────┘
                                                   │
                              SYSCALL instruction  │
                                                   ▼
┌──────────────────────────────────────────────────────────┐
│  Kernel space (ring 0)                                    │
│   sys_call_table[ rax ]  →  do_sys_openat2()              │
│   cap_capable()  →  LSM hooks  →  SELinux/AppArmor        │
│   VFS  →  ext4  →  block layer  →  NVMe driver            │
└──────────────────────────────────────────────────────────┘

The same picture applies to memory: a process sees a flat virtual address space; the kernel maps it through a 4-level (or 5-level) page table into physical frames. /proc/<pid>/maps is the userland view of that mapping — and the first thing a shellcode writer reads.

How It Works

Section titled “How It Works”

  1. Syscall numbers (x86_64, arch/x86/entry/syscalls/syscall_64.tbl):

    0   read        rdi=fd, rsi=buf, rdx=count
    1   write       rdi=fd, rsi=buf, rdx=count
    2   open        rdi=path, rsi=flags, rdx=mode   (legacy)
    3   close       rdi=fd
    9   mmap        rdi=addr, rsi=len, rdx=prot, r10=flags, r8=fd, r9=off
    10  mprotect    rdi=addr, rsi=len, rdx=prot
    41  socket      rdi=domain, rsi=type, rdx=protocol
    42  connect     rdi=fd, rsi= sockaddr*, rdx=addrlen
    56  clone       rdi=flags, rsi=stack, rdx=parent_tidptr, r10=child_tidptr, r8=tls
    57  fork        (no args)
    59  execve      rdi=filename, rsi=argv, rdx=envp
    60  exit        rdi=code
    257 openat      rdi=dirfd, rsi=path, rdx=flags, r10=mode  (modern open)

    Convention: rdi, rsi, rdx, r10, r8, r9 for args, rax for syscall number and return value. mmap and clone take six args, so r10 is used instead of rcx because syscall clobbers rcx (saved RIP) and r11 (saved RFLAGS).

  2. /proc — the kernel’s user-visible API:

    • /proc/<pid>/maps — address perms offset dev inode pathname. r-xp = read+exec+private. Watch for [heap], [stack], [vdso], [vsyscall].
    • /proc/<pid>/cmdline — argv, NULL-separated (\0). Use tr '\0' ' ' to read.
    • /proc/<pid>/environ — envp, also NULL-separated.
    • /proc/<pid>/exe — symlink to the on-disk binary. readlink /proc/1234/exe.
    • /proc/<pid>/fd/<n> — symlink to the file that fd points at. Detect deleted-but-open files here.
    • /proc/<pid>/status — human-readable VmRSS, CapEff, TracerPid (non-zero = being ptraced).
    • /proc/<pid>/net/tcp — kernel-socket view, same as ss -tan.
    • /proc/sys/kernel/randomize_va_space — 0 off, 1 conservative, 2 full ASLR (default on modern distros).
    • /proc/sys/vm/drop_caches — echo 3 > ... purges pagecache + dentries + inodes.

  3. Credentials & capabilities:

    • Pre-2.2 Linux was root-vs-non-root. Modern Linux splits root into ~40 capabilities (CAP_SYS_ADMIN, CAP_NET_RAW, CAP_DAC_OVERRIDE, CAP_SETUID…). capsh --print shows the effective set.
    • A container that drops everything except CAP_NET_BIND_SERVICE cannot mount, cannot iptables, cannot ptrace.
    • /proc/<pid>/status line CapEff: 000001ffffffffff is the bitmask; capsh --decode=000001ffffffffff resolves it.

  4. systemd unit lifecycle:

    • Unit files live in /etc/systemd/system/ (admin), /run/systemd/system/ (runtime), /usr/lib/systemd/system/ (package). Highest priority wins.
    • Type= controls how systemd considers the unit “started”: simple (PID immediately), forking (waits for parent exit), oneshot (RemainAfterExit), notify (sd_notify), dbus (DbusName acquires).
    • systemctl status shows the cgroup tree. systemd-cgls walks it. systemd-analyze blame lists slow units at boot.
    • journalctl -u unit -f --output=cat tails raw lines without metadata.

Common Techniques

Section titled “Common Techniques”
TechniqueDescriptionDetection Difficulty
LD_PRELOAD injectionPreload a malicious .so to hook libc functions in target processMedium — /proc/<pid>/maps shows non-standard libs
/proc//mem writeOpen /proc/pid/mem with PTRACE_MODE_ATTACH, patch text segment in another processHard — requires CAP_SYS_PTRACE or same uid
Dirty Pipe (CVE-2022-0847)Overwrite page-cache via pipe buffer PIPE_BUF_FLAG_CAN_MERGEEasy now — kernel >= 5.16.11, 5.15.25, 5.10.102 patched
PwnKit (CVE-2021-4034)pkexec argv out-of-bounds env injection → setuid rootEasy — patch removes vulnerable code path
nft UAF (CVE-2023-32233)Use-after-free in nft_set_elem netlink handlerHard — requires CAP_NET_ADMIN or unprivileged user ns
Hidden process via /proc unmountRemount /proc over a stub to hide specific PIDsMedium — compare ps aux vs /sys/fs/cgroup/pids/

Hands-On Commands

Section titled “Hands-On Commands”
Terminal window
# Resolve a syscall number → name → handler
ausyscall --exact=openat          # prints "257"
cat /proc/self/syscall            # what this shell is currently blocked in


# Read the memory map of a process; look for r-xp regions (code)
cat /proc/$$/maps | grep -E 'r-xp|vdso|stack'


# Effective capabilities of the current shell
capsh --print | grep -E '^Cap (Eff|Bnd|Inh)'


# ptrace scope: 0=classic, 1=restricted (default), 2=admin-only, 3=no-ptrace
cat /proc/sys/kernel/yama/ptrace_scope


# Inspect every systemd unit's exec line and cgroup
systemctl list-units --type=service --state=running --no-pager
systemd-cgls --no-pager | head -60


# Find setuid binaries — these are local privesc surface
find / -xdev -type f -perm -4000 2>/dev/null


# Audit rules on the host — this is what logs execve system-wide
auditctl -l                       # prints loaded rules
ausearch -sc execve -ts recent    # execve events in audit log

Detection — Defender View

Section titled “Detection — Defender View”

auditd rule for syscall-wide execve logging:

-a always,exit -F arch=b64 -S execve -S execveat -k exec

ausearch -k exec then yields a record of every process start, including the parent PID, args, and the calling UID.

Sysmon for Linux (now shipped by Microsoft for kernel ≥ 5.0) — Event ID 1 ProcessCreate, Event ID 3 NetworkConnect, Event ID 11 FileCreate, Event ID 23 FileDelete. Same IDs as Windows Sysmon; the schema is identical, which is the whole point.

MITRE ATT&CK T1057 — Process Discovery (ps, top, ls /proc).

MITRE ATT&CK T1087.001 — Account Discovery: Local (/etc/passwd, getent passwd).

MITRE ATT&CK T1611 — Escape to Host (container breakouts; Dirty Pipe against /proc/1/root from a container was a working exploit until patch).

Wazuh rule 31103 — “Nmap scan detected” [verify — see F1].

Wazuh rule 920 / 921 — Suspicious file in /tmp/.X11-unix style hidden-dot folders [verify].

Evasion Techniques

Section titled “Evasion Techniques”

Disabling auditd at runtime. Old trick: auditctl -e 0 disables the audit subsystem. Newer kernels (≥ 4.13) refuse this once audit_immutable is set. Defense: set audit_backlog_limit=8192 audit=1 on the kernel command line and auditctl -e 2 (immutable; only cleared on reboot).

/proc hiding via bind-mount. Malware does:

Terminal window
mkdir -p /tmp/fakeproc
mount -t proc none /tmp/fakeproc
mount --bind /tmp/fakeproc /proc
# or: bind-mount an empty dir over /proc/<evil_pid>

Detection: lsns -t pid and ps -e -o pid,mntns will show PIDs in a different mount namespace than the host’s /proc. Or compare ls -1 /proc | wc -l against ps -e --no-headers | wc -l.

YAMA ptrace_scope bypass via CAP_SYS_PTRACE. A setuid binary granting CAP_SYS_PTRACE (e.g. some package managers’ helpers) can attach to any process even with ptrace_scope=1. Capabilities are not the same as UID; audit CAP_SYS_PTRACE and CAP_SYS_ADMIN setuid binaries.

EvasionWhat it hidesHow to detect
Bind-mount over /proc/One processps -e vs ls /proc cross-check
LD_PRELOAD with nosymfollow libcHooked functions/proc/<pid>/maps unexpected .so paths
prctl(PR_SET_DUMPABLE, 0)Crash dumps (and any ptrace-based AV)Catches tools that inspect via ptrace only
Container + unshare -rEffective uid 0 in user namespacecat /proc/self/uid_map

⚠️ What caught me: I once found a backdoored /usr/lib/systemd/libsystemd-shared-<version>.so that had been patched to log all pam_sm_authenticate calls to a remote server. The file was on disk, valid ELF, loaded by every modern systemd binary. The only thing that flagged it was rpm -Va systemd-libs showing 5..T... (mode/size/time mismatch) on a single .so. Lesson: package manager verification catches what /proc and EDR cannot, because the kernel happily executes whatever is on disk regardless of provenance.

Lab Practice

Section titled “Lab Practice”

VM: Ubuntu 22.04 LTS, kernel 5.15.0-25, linux-tools-5.15.0-25-generic, auditd, systemd-coredump, strace, ltrace, bpftrace, capsh.

Objective: without writing any code, demonstrate four facts about the running kernel:

  1. Print the syscall table entry for execve (number + handler symbol in /proc/kallsyms).
  2. Identify a setuid binary, run it, and read the CapEff line of its /proc/<pid>/status to confirm it has dropped to non-root after seteuid.
  3. Write an audit rule that logs every setuid(0) call and verify ausearch -sc setuid -ts recent shows your test.
  4. Demonstrate systemd-analyze security sshd.service and explain the top three worst scores.

Expected flag format: flag{syscall_59_capEff_<hex>_audit_rule_setuid}.

Detection that should fire: ausearch -k exec -ts recent | grep execve should show your test exec; journalctl -u auditd -f should print the setuid event in real time.

Interview Questions

Section titled “Interview Questions”

Conceptual: explain the difference between fork() and clone() in terms of what kernel resources are shared, and why every modern fork() on Linux is actually clone(SIGCHLD, 0).

Technical: walk through what happens when a process calls execve("/bin/sh", ["sh", NULL], NULL) — name every kernel subsystem that runs from the syscall entry in rax=59 to the new program’s first instruction.

Scenario: a defender reports a host where ps aux | grep evil returns nothing but /sys/fs/cgroup/pids/system.slice/evil.service/cgroup.procs lists a PID. How is the process hidden, and what three commands would you run to confirm the technique?

What is the meaning of the TracerPid field in /proc/<pid>/status, and how does that field map to ATT&CK T1055 Process Injection detection?

Technical: why does CVE-2022-0847 (Dirty Pipe) work against /usr/bin/su to give you a root shell but not against /usr/bin/passwd, even though both are setuid root?

Section titled “Related Modules”

→ F1 (Networking Fundamentals) · F3 (Windows Internals) · F5 (Python for Security)