ctf-agent 1.2.0

package/references/security-events-and-intelligence.md

@@ -0,0 +1,566 @@
+# CTF Intelligence, Events & Challenge Artifact Analysis Guide (v2.0)
+## Operational Architecture for Event Monitoring, CTF Intelligence, Challenge Triage, Platform Automation, and Artifact Verification
+
+> **Scope**: This reference handbook is strictly designed for authorized CTF competitions, educational security labs, controlled challenge infrastructure, and permitted offensive research.
+
+---
+
+## 1. Scope & Authorization Boundaries
+
+In competitive CTFs and controlled educational labs, maintaining clear operational boundaries is mandatory:
+
+```
+┌────────────────────────────────────────────────────────────────────────────────────────┐
+│  OPERATIONAL BOUNDARY SEPARATION                                                       │
+├───────────────────────────────────────────┬────────────────────────────────────────────┤
+│  CHALLENGE ARTIFACTS (IN-SCOPE)           │  UNAUTHORIZED HARVESTING (OUT-OF-SCOPE)    │
+├───────────────────────────────────────────┼────────────────────────────────────────────┤
+│  - Deliberately seeded flags & tokens     │  - Attacking competition CTFd server hosts │
+│  - Challenge service source code & files  │  - Compromising organizer cloud accounts   │
+│  - Target container filesystem & state    │  - Sniffing unrelated platform traffic     │
+│  - Intentionally exposed lab metadata     │  - Destructive Denial of Service (DoS)     │
+└───────────────────────────────────────────┴────────────────────────────────────────────┘
+```
+
+- **Challenge Artifact Analysis**: The systematic inspection of challenge-provided files, binaries, network protocols, environment variables, and target filesystem locations intentionally structured by the challenge author.
+- **Rules of Engagement (RoE)**: All actions must remain strictly bounded by the rules of the specific competition or lab platform.
+
+---
+
+## 2. Security Event Monitoring (Industry Arenas & Research Conferences)
+
+Major CTF challenges frequently adapt attack techniques and root causes presented at elite security events and academic conferences:
+
+```
+┌────────────────────────────────────────────────────────────────────────────────────────┐
+│  SECURITY EVENT MONITORING HIERARCHY                                                   │
+├────────────────────────────┬────────────────────────────┬──────────────────────────────┤
+│ Offensive / Exploitation   │ Academic Security Research │ CTF / Premier Competitions   │
+│ (0-Days, Mitigations, PoC) │ (Theory, Crypto, Systems)  │ (Hands-on Challenge Puzzles) │
+├────────────────────────────┼────────────────────────────┼──────────────────────────────┤
+│ - Pwn2Own / ZDI            │ - USENIX Security          │ - CTFtime                    │
+│ - OffensiveCon (Berlin)    │ - ACM CCS                  │ - DEF CON CTF                │
+│ - Black Hat Briefings      │ - IEEE S&P (Oakland)       │ - Real World CTF             │
+│ - DEF CON Presentations    │ - NDSS                     │ - Google CTF                 │
+│ - REcon / Zero Nights      │ - WOOT / RAID              │ - PlaidCTF / picoCTF         │
+└────────────────────────────┴────────────────────────────┴──────────────────────────────┘
+```
+
+### 2.1. Pwn2Own Arena (Zero Day Initiative)
+- **Portal:** `https://www.zerodayinitiative.com/blog`
+- **Distinguishing Fact from Community Observation**:
+  - **Fact**: Pwn2Own is one of the world's most prominent public vulnerability research and exploitation competitions (organized annually across Vancouver, Toronto, and Automotive editions). Targets span major enterprise operating systems (Ubuntu, Windows 11, macOS), hypervisors (VMware Workstation, Oracle VirtualBox), modern browsers (Chrome, Edge), and connected automotive platforms.
+  - **Community Observation**: Techniques disclosed through Pwn2Own and ZDI research can be useful references for advanced CTF challenge design and vulnerability research. Authors of elite competitions (such as Real World CTF and DEF CON Quals) often construct challenges around similar primitives following public disclosure.
+
+### 2.2. Offensive & Exploitation Conferences
+- **OffensiveCon (Berlin) (`https://www.offensivecon.org/`)**:
+  - Deep technical focus on binary exploitation, modern hardware/OS mitigations (CET, MTE, PAC), hypervisors, and browser internals.
+  - *Current Meta (2026)*: Focus areas include GPU/kernel exploitation, modern Android exploit chains, browser site isolation internals, QSEE/Wi-Fi firmware exploitation, and complex relational engine (PostgreSQL) internals.
+- **Black Hat Briefings (`https://www.blackhat.com/`)**:
+  - Authoritative presentation of novel applied attack techniques (e.g., HTTP request desynchronization, kernel credential corruption, hypervisor escapes). Slides and whitepapers are publicly archived post-event.
+- **DEF CON Presentations & Media Server (`https://media.defcon.org/`)**:
+  - Canonical public archive of presentations, recordings, open-source offensive tools, and historical DEF CON CTF challenge repositories from DEF CON 1 to present.
+- **REcon (`https://recon.cx/`) & Zero Nights**:
+  - Premier conferences dedicated to reverse engineering, hardware security, firmware extraction, and low-level software analysis.
+
+### 2.3. Academic Security Research Conferences
+- **USENIX Security (`https://www.usenix.org/conferences`)**:
+  - Peer-reviewed research on applied systems security, side-channel attacks, microarchitectural vulnerabilities, and memory safety.
+  - *Tracking Note*: USENIX Security proceedings are fully open-access (e.g., USENIX Security '26 proceedings published in August 2026; Call for Papers for '27 actively maintained).
+- **ACM CCS, IEEE Symposium on Security and Privacy (S&P / Oakland), NDSS**:
+  - The "Big Four" academic conferences disclosing formal cryptanalysis (Lattice/LWE attacks, zero-knowledge proofs), network protocol weaknesses, and automated vulnerability discovery.
+- **WOOT (Workshop on Offensive Technologies) & RAID**:
+  - Specializes in unconventional offensive primitives, hardware glitches, and exploit generation methodologies.
+
+### 2.4. Premier CTF Competitions
+- **DEF CON CTF (Quals & Finals)**: The benchmark for elite binary exploitation, zero-day research, and custom architectures.
+- **Real World CTF**: Focuses exclusively on exploiting real-world open-source software, virtualization engines, and network appliances.
+- **Google CTF, PlaidCTF, SekaiCTF, DiceCTF**: Known for cutting-edge web desync, novel cryptography, and sandbox breakouts.
+
+---
+
+## 3. CTF Event Intelligence (CTFtime & Competition Architectures)
+
+CTFtime (`https://ctftime.org/`) serves as the global coordination and telemetry platform for security competitions.
+
+```
+┌────────────────────────────────────────────────────────────────────────────────────────┐
+│  CTFTIME EVENT INTELLIGENCE TAXONOMY                                                   │
+├──────────────────────────┬──────────────────────────┬──────────────────────────────────┤
+│ Lifecycle States         │ Competition Formats      │ Participation Constraints        │
+├──────────────────────────┼──────────────────────────┼──────────────────────────────────┤
+│ - Upcoming Events        │ - Jeopardy               │ - Open                           │
+│ - Currently Running      │ - Attack-Defense (A/D)   │ - Prequalified                   │
+│ - Archived Events        │ - Hack-quest / KotH      │ - Academic / High-school         │
+└──────────────────────────┴──────────────────────────┴──────────────────────────────────┘
+```
+
+### 3.1. Event Lifecycle & Format Classification
+1. **Event Lifecycle**:
+   - **Upcoming**: Track start times, registration URLs, and format rules.
+   - **Running**: Active competitions requiring real-time challenge ingestion, scoreboard tracking, and dynamic point monitoring.
+   - **Archived**: Historical repositories linking to writeups, challenge source repositories, and participant scoreboards.
+2. **Competition Formats**:
+   - **Jeopardy**: Challenges are divided into discrete categories (Web, Pwn, Crypto, Reverse, Forensics, OSINT, Misc). Teams solve puzzles independently for static or dynamic decaying points.
+   - **Attack-Defense (A/D)**:
+     - Each team receives an identical virtual machine image hosting multiple vulnerable network services.
+     - Games proceed in timed rounds or "ticks" (typically 2–5 minutes).
+     - **Offense**: Exploit opponent services and extract periodic tick flags.
+     - **Defense**: Patch local service vulnerabilities without breaking Service Level Agreements (SLA check scripts verify service functionality).
+     - **Telemetry**: Teams analyze live network traffic (PCAP) to steal and adapt opponents' attack payloads.
+   - **Hack-quest / King of the Hill (KotH)**: Sequential penetration testing labs or competitive persistent control over shared target infrastructure.
+
+### 3.2. Dynamic Top Teams & Community Tracking
+To maintain an up-to-date knowledge base, do not rely on static hardcoded lists. Monitor teams and researchers dynamically through:
+- **CTFtime Global Rankings**: Track top-performing teams across seasons.
+- **Event Scoreboards**: Identify specialized teams consistently dominating specific categories (e.g., dedicated pwn or crypto researchers).
+- **Public Writeup Repositories & Team GitHub Orgs**: Follow active research publications from leading teams.
+
+---
+
+## 4. Threat & Vulnerability Intelligence (Disaggregated)
+
+Do not conflate general threat reporting with vulnerability intelligence or CTF intelligence:
+
+```
+┌────────────────────────────────────────────────────────────────────────────────────────┐
+│  INTELLIGENCE TAXONOMY COMPARISON                                                      │
+├─────────────────────────┬─────────────────────────────┬────────────────────────────────┤
+│ Threat Intelligence     │ CVE / Exploit Intelligence  │ CTF Intelligence               │
+├─────────────────────────┼─────────────────────────────┼────────────────────────────────┤
+│ Focus: Threat actors,   │ Focus: Vulnerability IDs,   │ Focus: Challenge design,       │
+│ campaigns, botnets,     │ CVSS, patch diffs, KEV,     │ author intent, primitives,     │
+│ malware hashes, wild C2 │ EPSS, public PoC exploits   │ hints, writeups, flags         │
+└─────────────────────────┴─────────────────────────────┴────────────────────────────────┘
+```
+
+### 4.1. Priority Intelligence & Technical Disclosures
+- **Priority Intelligence — CISA KEV (`cisa.gov/known-exploited-vulnerabilities-catalog`)**:
+  - Mandatory catalog of confirmed, actively exploited vulnerabilities in the wild. If a challenge or lab replicates a CVE listed in KEV, reliable weaponized exploit techniques exist.
+- **Technical Disclosures — Openwall oss-security (`openwall.com/lists/oss-security/`)**:
+  - High-signal technical mailing list where Linux kernel, glibc, systemd, and OpenSSH maintainers discuss coordinated vulnerability disclosures and root-cause analyses.
+- **Network Threat Telemetry — SANS Internet Storm Center (ISC) (`isc.sans.edu/`)**:
+  - Global honeypot network telemetry, daily handler diaries, and real-world packet capture breakdowns.
+
+### 4.2. Security News & Secondary Reporting
+- **The Hacker News (THN) (`thehackernews.com/`)**:
+  - Rapid secondary reporting on emerging 0-days, supply-chain incidents, and emergency patches. Useful for discovery, but always follow source links to primary advisories.
+- **BleepingComputer (`bleepingcomputer.com/`)**:
+  - Detailed secondary reporting on ransomware attack vectors, active exploitation campaigns, and vendor advisories.
+
+### 4.3. Threat Research & Malware Intelligence
+- **vx-underground (`vx-underground.org/`)**:
+  - Specialized archive of malware samples, Proof-of-Concept exploits, and technical threat research papers.
+  - > [!CAUTION]
+    > **Compliance Notice**: Access and handle material from malware repositories only where legally permitted, within properly isolated analysis sandboxes, and in full compliance with organizational security policies.
+
+---
+
+## 5. Challenge Triage & Knowledge Graph
+
+### 5.1. The Challenge Triage Pipeline
+
+When ingesting a new CTF challenge, follow this systematic progression:
+
+```
+┌────────────────────────────────────────────────────────────────────────┐
+│  1. CHALLENGE INGESTION & HINT-FIRST SEMANTIC SCOPING                  │
+│     Parse Title, Prompt, and Official Hints. Prune rabbit holes.       │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  2. ARTIFACT EXTRACTION                                                │
+│     Collect challenge files: Dockerfile, binaries, PCAPs, manifests.   │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  3. TECHNOLOGY & VERSION FINGERPRINTING                                │
+│     Identify programming language, framework, dependencies, glibc.     │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  4. ARCHITECTURE BRANCHING                                             │
+│     ├─ Source Code Available? ──> Static Audit & Sink Tracing          │
+│     ├─ Known N-Day Software?  ──> Query Exploit-Databases Handbook     │
+│     └─ Black-box / Custom?   ──> Input Fuzzing & Triage Ladder         │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  5. PRIOR WRITEUP & VARIANT ANALYSIS                                   │
+│     Search CTFtime writeups and GitHub for identical/forked problems.  │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  6. HYPOTHESIS FORMULATION, REPRODUCTION & FLAG EXTRACTION             │
+│     Reproduce locally in Docker -> Trigger remote -> Validate Flag.    │
+└────────────────────────────────────────────────────────────────────────┘
+```
+
+### 5.2. The Challenge Intelligence Graph
+Structure competition knowledge as an interconnected graph rather than disconnected text files:
+
+```
+           ┌──────────────┐
+           │    Event     │
+           └───┬──────┬───┘
+               │      │
+       ┌───────┘      └────────┐
+       ▼                       ▼
+┌──────────────┐       ┌──────────────┐
+│  Organizer   │       │  Challenge   │
+└──────────────┘       └───┬──────┬───┘
+                           │      │
+           ┌───────────────┘      └──────────────┐
+           ▼                                     ▼
+┌──────────────────────┐             ┌──────────────────────┐
+│ Technology / Stack   │             │ Intended Primitive   │
+│ (e.g., glibc 2.39)   │             │ (e.g., FSOP Apple 2) │
+└──────────┬───────────┘             └──────────┬───────────┘
+           │                                    │
+           ▼                                    ▼
+┌──────────────────────┐             ┌──────────────────────┐
+│ Associated CVE/GHSA  │             │ Public Writeups /    │
+│ (Exploit Handbook)   │             │ Researcher GitHub    │
+└──────────────────────┘             └──────────────────────┘
+```
+
+**Common Knowledge Graph Queries for CTF Agents**:
+- `CVE -> CTF Challenge`: Which past challenges featured this specific vulnerability?
+- `Technology -> Primitives`: What are the standard bypasses for this framework version?
+- `Researcher -> Writeups`: How did top researchers solve similar challenges?
+
+---
+
+## 6. CTFd Platform API Automation & Integration
+
+When participating in competitions hosted on CTFd, automated interaction eliminates manual web clicking and accelerates telemetry collection.
+
+### 6.1. Authentication Standard
+- CTFd API tokens are generated via `Settings` $\rightarrow$ `Access Tokens`.
+- Send tokens in the HTTP Authorization header:
+  ```http
+  Authorization: Token <access_token>
+  ```
+  *(Do not assume a mandatory `ctfd_` prefix; store the full token verbatim).*
+
+### 6.2. Core CTFd REST API Endpoints
+The CTFd API disaggregates challenge metadata, downloadable attachments, and hints:
+
+| Method | Endpoint | Description |
+| :--- | :--- | :--- |
+| `GET` | `/api/v1/challenges` | List all challenges (id, name, category, value, state). |
+| `GET` | `/api/v1/challenges/{id}` | Retrieve detailed challenge description and connection host/port. |
+| `GET` | `/api/v1/challenges/{id}/files` | **Downloadable challenge file URLs and hashes.** |
+| `GET` | `/api/v1/challenges/{id}/hints` | List available or unlocked hints. |
+| `GET` | `/api/v1/challenges/{id}/tags` | Metadata tags (e.g., `web`, `pwn`, `sandbox`). |
+| `GET` | `/api/v1/challenges/{id}/topics` | Academic or technical topics. |
+| `POST` | `/api/v1/challenges/attempt` | Submit a flag candidate: `{"challenge_id": 1, "submission": "flag{...}"}`. |
+| `GET` | `/api/v1/scoreboard` | Real-time team standings and point progression. |
+
+### 6.3. API Discovery, Pagination & Robustness
+- **Pagination**: Large competitions paginate `/api/v1/challenges?page=2`. Scripts must iterate through all pages until `data` is empty or `meta.pagination.pages` is reached.
+- **Rate Limiting**: Handle HTTP `429 Too Many Requests` by honoring the `Retry-After` header or applying exponential backoff.
+- **Error Handling**: Gracefully handle locked challenges (`403 Forbidden`) and platform pauses.
+
+---
+
+## 7. Flag Candidate Extraction & Validation Pipeline
+
+### 7.1. Limitations of Naive Regex Matching
+Simple regular expressions like `re.findall(r'flag\{[^\}]+\}', text, re.I)` produce severe failure modes:
+1. **Format Variations**: Flags frequently omit curly braces (e.g., `FLAG-4a7b-8c9d`, `CTF:secret_token_123`, `token=abc123xyz`).
+2. **Unicode & Character Sets**: Modern challenges include extended ASCII, UTF-8 byte sequences, or base64 strings.
+3. **Case Sensitivity & False Positives**: `re.IGNORECASE` matches error messages, HTML comments, or debugging strings like `flag{example_flag}`.
+4. **Context Blindness**: A regex match does not verify whether the string originated from the challenge output or unrelated system logs.
+
+### 7.2. Multi-Stage Flag Validation Pipeline
+
+```
+┌────────────────────────────────────────────────────────────────────────┐
+│  STAGE 1: CANDIDATE EXTRACTION                                         │
+│  Extract candidate strings from stdout, HTTP response, files, memory.  │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  STAGE 2: NORMALIZATION & DEDUPLICATION                                │
+│  Strip trailing whitespace, unescape quotes, decode URL/hex encodings. │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  STAGE 3: FORMAT & REGEX VALIDATION                                    │
+│  Verify against competition flag pattern (or fall back to heuristics). │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  STAGE 4: CONTEXT & CONFIDENCE SCORING                                 │
+│  Score candidate: source reliability, keyword proximity, entropy.      │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  STAGE 5: SUBMISSION & CONFIRMATION                                    │
+│  Submit via CTFd API. Record 1.00 confidence only upon server accept.  │
+└────────────────────────────────────────────────────────────────────────┘
+```
+
+### 7.3. FlagCandidate Data Model & Confidence Scoring
+
+```python
+from dataclasses import dataclass
+from typing import Optional
+
+@dataclass
+class FlagCandidate:
+    value: str
+    source: str              # e.g., 'terminal_output', 'http_body', 'memory_carve', 'file'
+    location: Optional[str]  # e.g., '/flag.txt', 'Header:X-Flag', 'PID:1337'
+    confidence: float        # Numeric score between 0.0 and 1.0
+```
+
+**Confidence Scoring Rubric**:
+- **0.30 (Weak Candidate)**: Matched generic token or high-entropy hex string without definitive challenge flag prefix.
+- **0.60 (Probable Candidate)**: Matches known competition format prefix (`flag{...}`), but extracted from unverified or broad application logs.
+- **0.90 (Strong Candidate)**: Matches flag format and was directly exfiltrated from the primary exploit trigger (e.g., output of `/bin/sh -c cat /flag*` or decrypted cipher output).
+- **1.00 (Confirmed Flag)**: Accepted by the CTFd competition platform API with a successful solve response (`status: correct`).
+
+---
+
+## 8. Authorized Challenge Instance Artifact Analysis
+
+When operating inside an authorized target container or remote instance following RCE or Arbitrary File Read, inspect artifacts systematically across five distinct architectural levels:
+
+```
+┌────────────────────────────────────────────────────────────────────────┐
+│  LEVEL 1: FILESYSTEM ARTIFACTS                                         │
+│  - Root & application directories: /flag, /flag.txt, /flag.sh          │
+│  - Web server roots: /app/flag.txt, /var/www/html/flag*                │
+│  - User & temporary storage: /home/*/flag*, /root/flag*, /tmp/flag*    │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  LEVEL 2: ENVIRONMENT ARTIFACTS                                        │
+│  - Current process environment: /proc/self/environ, `env`, `export`    │
+│  - Container init environment: /proc/1/environ (Docker -e FLAG=...)    │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  LEVEL 3: APPLICATION STATE ARTIFACTS                                  │
+│  - SQLite databases, session files, cached tokens, redis instances     │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  LEVEL 4: PROCESS INSPECTION                                           │
+│  - Command line arguments: /proc/<pid>/cmdline                         │
+│  - Open file descriptors: /proc/<pid>/fd/                              │
+│  - Process working directories: /proc/<pid>/cwd                        │
+└──────────────────────────────────┬─────────────────────────────────────┘
+                                   │
+                                   ▼
+┌────────────────────────────────────────────────────────────────────────┐
+│  LEVEL 5: ADVANCED PROCESS MEMORY ANALYSIS (RESEARCH PRIMITIVE)        │
+│  - Inspecting virtual memory: /proc/<pid>/mem, /proc/<pid>/maps, GDB   │
+└────────────────────────────────────────────────────────────────────────┘
+```
+
+### Level 5 Caveats (Process Memory Inspection):
+Reading `/proc/<pid>/mem` or attaching debuggers is governed by strict OS security controls:
+- **User ID Matching**: You cannot read memory of processes owned by another user without root privileges.
+- **Linux Yama LSM (`kernel.yama.ptrace_scope`)**: A `ptrace_scope` of 1 or 2 blocks non-child process memory inspection even for the same UID.
+- **Container Privileges**: Standard Docker containers drop `CAP_SYS_PTRACE`. Memory inspection fails unless the container was spawned with `--cap-add=SYS_PTRACE` or `--privileged`.
+
+---
+
+## 9. Cloud & Container Challenge Artifacts
+
+In specialized cloud security CTFs (e.g., Cloud Village, HTB Cloud Labs):
+
+> [!IMPORTANT]
+> **Strict Authorization Scope**: Only inspect metadata endpoints, service-account material, or mounted container secrets when:
+> 1. The challenge explicitly includes a cloud/container environment;
+> 2. The target infrastructure is owned or authorized for testing;
+> 3. Access is strictly within the challenge scope.
+
+### Standard Cloud & Container Artifact Locations:
+1. **Kubernetes Service Account Tokens**:
+   - Path: `/var/run/secrets/kubernetes.io/serviceaccount/token`
+   - Namespace: `/var/run/secrets/kubernetes.io/serviceaccount/namespace`
+2. **Container Secret Mounts**:
+   - Path: `/run/secrets/`
+3. **Instance Metadata Services (IMDS)**:
+   - AWS IMDSv1: `http://169.254.169.254/latest/meta-data/`
+   - AWS IMDSv2: Requires `X-aws-ec2-metadata-token` retrieved via `PUT /latest/api/token`.
+   - GCP Metadata: `http://metadata.google.internal/computeMetadata/v1/` (requires `Metadata-Flavor: Google` header).
+
+---
+
+## 10. Writeup Intelligence & Knowledge Retrieval
+
+Upon solving a challenge, extract and structure operational intelligence to benefit future engagements:
+- **Challenge Taxonomy Tagging**: Categorize by framework (e.g., Next.js, glibc 2.39, Flask), vulnerability class (e.g., Prototype Pollution, FSOP), and difficulty.
+- **Searchable Writeup Archive**: Maintain writeups following the `skills/ctf-writeup/SKILL.md` standard.
+- **Correlating External Writeups**: When blocked during training, search CTFtime and GitHub using exact error strings and function names to analyze approaches taken by top teams.
+
+---
+
+## 11. Production-Ready Python Automation Utilities
+
+### 11.1. Robust Flag Candidate Extraction Engine
+Save as `scripts/extract_flags.py`:
+
+```python
+#!/usr/bin/env python3
+"""
+Flag Candidate Extraction Engine (v2.0)
+Extracts, normalizes, deduplicates, and scores flag candidates from raw text.
+"""
+
+import re
+from dataclasses import dataclass
+from typing import List, Optional
+
+@dataclass
+class FlagCandidate:
+    value: str
+    source: str
+    location: Optional[str]
+    confidence: float
+
+# Common competition flag patterns
+FLAG_PATTERNS = [
+    (r'flag\{[a-zA-Z0-9_\-\.\!\?@#\$%\^&\*\+=]+\}', 0.90),
+    (r'CTF\{[a-zA-Z0-9_\-\.\!\?@#\$%\^&\*\+=]+\}', 0.90),
+    (r'picoCTF\{[^\}]+\}', 0.90),
+    (r'Dice\{[^\}]+\}', 0.90),
+    (r'SEKAI\{[^\}]+\}', 0.90),
+    (r'HTB\{[^\}]+\}', 0.90),
+    (r'flagyard\{[^\}]+\}', 0.90),
+    (r'FLAG-[A-Za-z0-9]{16,64}', 0.80),
+    (r'[0-9a-fA-F]{32}', 0.30),  # Raw MD5-like candidate (weak)
+]
+
+def extract_flag_candidates(raw_text: str, source: str = "terminal", location: Optional[str] = None) -> List[FlagCandidate]:
+    candidates = []
+    seen = set()
+
+    for pattern, base_conf in FLAG_PATTERNS:
+        matches = re.findall(pattern, raw_text)
+        for m in matches:
+            cleaned = m.strip()
+            if cleaned not in seen:
+                seen.add(cleaned)
+                # Boost confidence if extracted from primary target locations
+                conf = base_conf
+                if location and any(k in location for k in ("flag.txt", "/flag", "environ")):
+                    conf = min(1.0, conf + 0.10)
+                candidates.append(FlagCandidate(value=cleaned, source=source, location=location, confidence=conf))
+
+    return sorted(candidates, key=lambda c: c.confidence, reverse=True)
+```
+
+### 11.2. Complete CTFd Platform API Client
+Save as `scripts/ctfd_client.py`:
+
+```python
+#!/usr/bin/env python3
+"""
+CTFd Platform Automation Client (v2.0)
+Handles authentication, pagination, challenge files, hints, and flag submission.
+"""
+
+import os
+import json
+import time
+import urllib.request
+import urllib.error
+from typing import Optional, Dict, Any, List
+
+class CTFdClient:
+    def __init__(self, base_url: str, api_token: str):
+        self.base_url = base_url.rstrip("/")
+        self.headers = {
+            "Authorization": f"Token {api_token}",
+            "Content-Type": "application/json",
+            "User-Agent": "CTF-Agent-PlatformClient/2.0"
+        }
+
+    def _request(self, method: str, endpoint: str, data: Optional[Dict[str, Any]] = None) -> Optional[Dict[str, Any]]:
+        url = f"{self.base_url}/api/v1/{endpoint.lstrip('/')}"
+        payload = json.dumps(data).encode("utf-8") if data else None
+        req = urllib.request.Request(url, data=payload, headers=self.headers, method=method)
+
+        for attempt in range(3):
+            try:
+                with urllib.request.urlopen(req, timeout=10) as resp:
+                    return json.loads(resp.read().decode("utf-8"))
+            except urllib.error.HTTPError as e:
+                if e.code == 429:
+                    time.sleep(2 ** attempt)
+                    continue
+                return None
+            except Exception:
+                return None
+        return None
+
+    def get_challenges(self) -> List[Dict[str, Any]]:
+        """Retrieve all challenges across all paginated pages."""
+        all_challenges = []
+        page = 1
+        while True:
+            res = self._request("GET", f"challenges?page={page}")
+            if not res or not res.get("success") or not res.get("data"):
+                break
+            all_challenges.extend(res["data"])
+            pagination = res.get("meta", {}).get("pagination", {})
+            if page >= pagination.get("pages", 1):
+                break
+            page += 1
+        return all_challenges
+
+    def get_challenge_details(self, challenge_id: int) -> Optional[Dict[str, Any]]:
+        res = self._request("GET", f"challenges/{challenge_id}")
+        return res.get("data") if res else None
+
+    def get_challenge_files(self, challenge_id: int) -> List[str]:
+        """Fetch distinct attachment URLs for a challenge."""
+        res = self._request("GET", f"challenges/{challenge_id}/files")
+        if res and res.get("success"):
+            return [f.get("location") for f in res.get("data", [])]
+        return []
+
+    def get_challenge_hints(self, challenge_id: int) -> List[Dict[str, Any]]:
+        res = self._request("GET", f"challenges/{challenge_id}/hints")
+        return res.get("data", []) if res else []
+
+    def submit_flag(self, challenge_id: int, flag: str) -> Dict[str, Any]:
+        data = {"challenge_id": challenge_id, "submission": flag}
+        res = self._request("POST", "challenges/attempt", data=data)
+        if res and res.get("success"):
+            status = res.get("data", {}).get("status", "incorrect")
+            return {"status": status, "message": res.get("data", {}).get("message", "")}
+        return {"status": "error", "message": "Failed to communicate with CTFd"}
+```
+
+---
+
+## 12. Verification Checklist & Operational Pre-Flight Standards
+
+Before concluding work on any CTF challenge or security research task:
+
+- [ ] **1. Scoping Check**: Verify all actions were performed strictly within challenge scope and target infrastructure.
+- [ ] **2. Hint & Metadata Documentation**: Document all hints, official descriptions, and title clues analyzed.
+- [ ] **3. Artifact Triage Completed**: Download and categorize all challenge files (source, Dockerfile, PCAPs) into `resources/`.
+- [ ] **4. Root Cause Identified**: State the precise underlying vulnerability or logic flaw.
+- [ ] **5. Flag Candidate Verification**: Confirm candidate flag achieved a 1.00 score via platform API or authoritative challenge verification.
+- [ ] **6. Reproducible Solve Script**: Deliver a standalone `solve.py` script capable of running end-to-end.
+- [ ] **7. Writeup Standard**: Complete `writeup.md` compliant with repository standards.

package/references/version-matrix.md

@@ -0,0 +1,59 @@
+# Version-Aware Exploitation & Compatibility Matrix
+## Era-Aware Reconnaissance for CTF Competitions & Security Labs
+
+Applying modern techniques to legacy environments—or legacy techniques to modern mitigated environments—results in guaranteed failure. Always fingerprint target versions before choosing an attack vector.
+
+---
+
+## 1. Binary Exploitation: GNU C Library (glibc) Matrix
+
+Fingerprint the glibc version:
+```bash
+strings libc.so.6 | grep -i "GNU C Library" | head -1
+```
+
+### Era Breakdown & Usable Heap Primitives:
+
+| Glibc Version | Era | Security Features & Changes | Usable Attack Primitives |
+|:---|:---|:---|:---|
+| **< 2.26** | 2012–2017 | No tcache. Fastbins, unsorted bins, largebins. | Fastbin dup, unsafe unlink, classic House of Orange, House of Force, `__free_hook` / `__malloc_hook`. |
+| **2.26 – 2.30** | 2018–2020 | **Tcache introduced** (max 7 chunks per bin). Low integrity checks. | Direct tcache poisoning (single write corrupts fd), tcache double-free (2.26–2.28), tcache stashing unlink. |
+| **2.31 – 2.33** | 2020–2021 | Double-free mitigation in tcache (key field). **Safe-linking introduced in 2.32** (`(addr >> 12) ^ ptr`). | Must leak heap base address to de-mangle pointers. `__free_hook` still exists and is writable. |
+| **>= 2.34** | 2021–2026+ | **`__free_hook` AND `__malloc_hook` COMPLETELY REMOVED!** Writing to hooks causes crash. | 1. **House of Apple 2** (`_IO_wfile_overflow`).<br>2. TLS destructor overwrite (`__call_tls_dtors` / `__run_exit_handlers`).<br>3. Stack leak via `environ` followed by ROP. |
+
+---
+
+## 2. Web Backend Version Matrix
+
+### PHP Version Nuances
+- **PHP 5.x / 7.x**:
+  - **Loose string-to-number comparisons**: `"0e12345" == "0e67890"` evaluates to `true` (magic hash flaw).
+  - Numeric type juggling: `0 == "admin"` evaluates to `true`.
+- **PHP 8.x**:
+  - **Breaking change**: String-to-number comparisons are now strict. `"0e123" == "0e456"` is `false`. `0 == "admin"` is `false`.
+  - **Rule**: Do NOT attempt numeric type juggling or magic hash attacks against PHP 8 targets!
+  - Focus on filter chains (`php://filter`), unserialize gadgets, or fastcgi/fpm injection.
+
+### Python Version & Bytecode Decompilation
+- **Python 2.7**:
+  - `input()` performs direct `eval()` on user input.
+  - `uncompyle6` decompiles `.pyc` files cleanly.
+- **Python 3.8 – 3.10**:
+  - Standard PyJail breakout via `__subclasses__()`, `builtins`, `sys.modules`.
+  - Bytecode formats are standard; decompilation via `uncompyle6` or `decompyle++` (`pycdc`).
+- **Python 3.11 – 3.12+**:
+  - **PEP 709**: Inlined comprehensions, modified frame object structures, optimized bytecode opcodes.
+  - `uncompyle6` fails on Python >= 3.11 bytecode!
+  - **Rule**: Always use `pycdc` (C++ decompiler) or disassembler `python3 -m dis <file.pyc>` for Python >= 3.11.
+
+---
+
+## 3. Linux Kernel Mitigation Matrix
+
+| Feature | Introduced | Bypass Technique |
+|:---|:---|:---|
+| **SMEP** (Supervisor Mode Execution Prevention) | Linux 3.0 | ROP inside kernel space (cannot jump directly to userland shellcode). |
+| **SMAP** (Supervisor Mode Access Prevention) | Linux 3.7 | Pivot stack to kernel space; cannot read/write userland stack directly. |
+| **KASLR** | Linux 3.14 | Memory leak of kernel text address (`_text`, `commit_creds`). |
+| **FG-KASLR** | Linux 5.x | Fine-grained randomization; ROP must use gadgets within leaked function sections. |
+| **SLUB Freelist Hardening** | Linux 4.14+ | Freelist pointers XORed with random cookie. Requires heap base leak. |