ctf-agent 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (152) hide show
  1. package/AGENTS.md +131 -0
  2. package/Dockerfile +192 -0
  3. package/README.md +522 -0
  4. package/agents/ctf-analyzer.md +58 -0
  5. package/agents/ctf-controller.md +151 -0
  6. package/agents/ctf-speedrun.md +74 -0
  7. package/bin/ctf-agent.js +119 -0
  8. package/docker-compose.yml +13 -0
  9. package/mcp_config.json +19 -0
  10. package/package.json +44 -0
  11. package/references/ctf-triage-ladder.md +68 -0
  12. package/references/exploit-databases.md +575 -0
  13. package/references/llm-safety-and-policy-compliance.md +173 -0
  14. package/references/multi-agent-orchestration-and-policy-routing.md +278 -0
  15. package/references/security-events-and-intelligence.md +566 -0
  16. package/references/version-matrix.md +59 -0
  17. package/rules/ctf-execution-rules.md +44 -0
  18. package/rules/ctf-safety-framing-rules.md +140 -0
  19. package/rules/flag-validation-rules.md +25 -0
  20. package/scripts/__init__.py +1 -0
  21. package/scripts/ctf-tools.lock +205 -0
  22. package/scripts/ctf_init.py +1026 -0
  23. package/scripts/ctf_update.py +583 -0
  24. package/scripts/ctfd_client.py +81 -0
  25. package/scripts/cve_lookup.py +453 -0
  26. package/scripts/extract_flags.py +155 -0
  27. package/scripts/install_as_agent.py +278 -0
  28. package/scripts/install_ctf_tools.sh +603 -0
  29. package/scripts/parallel_triage.py +395 -0
  30. package/scripts/prompt_policy_sanitizer.py +283 -0
  31. package/scripts/scope_guard.py +444 -0
  32. package/scripts/skill_validator.py +504 -0
  33. package/scripts/workspace_cleaner.py +141 -0
  34. package/skills/ctf-ai-ml/SKILL.md +120 -0
  35. package/skills/ctf-ai-ml/adversarial-ml.md +623 -0
  36. package/skills/ctf-ai-ml/llm-attacks.md +487 -0
  37. package/skills/ctf-ai-ml/model-attacks.md +422 -0
  38. package/skills/ctf-crypto/SKILL.md +320 -0
  39. package/skills/ctf-crypto/advanced-math.md +798 -0
  40. package/skills/ctf-crypto/classic-ciphers.md +651 -0
  41. package/skills/ctf-crypto/ecc-attacks.md +347 -0
  42. package/skills/ctf-crypto/exotic-crypto-2.md +380 -0
  43. package/skills/ctf-crypto/exotic-crypto.md +528 -0
  44. package/skills/ctf-crypto/historical.md +113 -0
  45. package/skills/ctf-crypto/lattice-and-lwe.md +524 -0
  46. package/skills/ctf-crypto/modern-ciphers-2.md +563 -0
  47. package/skills/ctf-crypto/modern-ciphers-3.md +453 -0
  48. package/skills/ctf-crypto/modern-ciphers.md +649 -0
  49. package/skills/ctf-crypto/prng-attacks.md +257 -0
  50. package/skills/ctf-crypto/prng.md +664 -0
  51. package/skills/ctf-crypto/rsa-attacks-2.md +792 -0
  52. package/skills/ctf-crypto/rsa-attacks.md +487 -0
  53. package/skills/ctf-crypto/stream-ciphers.md +390 -0
  54. package/skills/ctf-crypto/zkp-and-advanced.md +456 -0
  55. package/skills/ctf-forensics/3d-printing.md +121 -0
  56. package/skills/ctf-forensics/SKILL.md +379 -0
  57. package/skills/ctf-forensics/disk-advanced.md +497 -0
  58. package/skills/ctf-forensics/disk-and-memory.md +491 -0
  59. package/skills/ctf-forensics/disk-recovery.md +699 -0
  60. package/skills/ctf-forensics/linux-forensics.md +511 -0
  61. package/skills/ctf-forensics/network-advanced.md +583 -0
  62. package/skills/ctf-forensics/network.md +645 -0
  63. package/skills/ctf-forensics/peripheral-capture.md +287 -0
  64. package/skills/ctf-forensics/signals-and-hardware.md +713 -0
  65. package/skills/ctf-forensics/steganography.md +694 -0
  66. package/skills/ctf-forensics/stego-advanced-2.md +475 -0
  67. package/skills/ctf-forensics/stego-advanced.md +481 -0
  68. package/skills/ctf-forensics/stego-image.md +691 -0
  69. package/skills/ctf-forensics/windows.md +625 -0
  70. package/skills/ctf-malware/SKILL.md +181 -0
  71. package/skills/ctf-malware/c2-and-protocols.md +274 -0
  72. package/skills/ctf-malware/pe-and-dotnet.md +108 -0
  73. package/skills/ctf-malware/scripts-and-obfuscation.md +449 -0
  74. package/skills/ctf-misc/SKILL.md +498 -0
  75. package/skills/ctf-misc/bashjails.md +323 -0
  76. package/skills/ctf-misc/ctfd-navigation.md +465 -0
  77. package/skills/ctf-misc/dns.md +255 -0
  78. package/skills/ctf-misc/encodings-advanced.md +504 -0
  79. package/skills/ctf-misc/encodings.md +431 -0
  80. package/skills/ctf-misc/games-and-vms-2.md +254 -0
  81. package/skills/ctf-misc/games-and-vms-3.md +690 -0
  82. package/skills/ctf-misc/games-and-vms-4.md +229 -0
  83. package/skills/ctf-misc/games-and-vms.md +529 -0
  84. package/skills/ctf-misc/linux-privesc.md +333 -0
  85. package/skills/ctf-misc/pyjails.md +671 -0
  86. package/skills/ctf-misc/rf-sdr.md +91 -0
  87. package/skills/ctf-osint/SKILL.md +198 -0
  88. package/skills/ctf-osint/geolocation-and-media.md +464 -0
  89. package/skills/ctf-osint/social-media.md +312 -0
  90. package/skills/ctf-osint/web-and-dns.md +341 -0
  91. package/skills/ctf-pwn/SKILL.md +214 -0
  92. package/skills/ctf-pwn/advanced-exploits-2.md +579 -0
  93. package/skills/ctf-pwn/advanced-exploits-3.md +598 -0
  94. package/skills/ctf-pwn/advanced-exploits-4.md +590 -0
  95. package/skills/ctf-pwn/advanced-exploits-5.md +119 -0
  96. package/skills/ctf-pwn/advanced-exploits.md +773 -0
  97. package/skills/ctf-pwn/advanced.md +326 -0
  98. package/skills/ctf-pwn/field-notes.md +245 -0
  99. package/skills/ctf-pwn/format-string.md +694 -0
  100. package/skills/ctf-pwn/heap-fsop.md +285 -0
  101. package/skills/ctf-pwn/heap-techniques-2.md +333 -0
  102. package/skills/ctf-pwn/heap-techniques.md +513 -0
  103. package/skills/ctf-pwn/kernel-bypass.md +421 -0
  104. package/skills/ctf-pwn/kernel-techniques.md +366 -0
  105. package/skills/ctf-pwn/kernel.md +636 -0
  106. package/skills/ctf-pwn/overflow-basics.md +611 -0
  107. package/skills/ctf-pwn/rop-advanced.md +725 -0
  108. package/skills/ctf-pwn/rop-and-shellcode.md +659 -0
  109. package/skills/ctf-pwn/sandbox-escape.md +313 -0
  110. package/skills/ctf-reverse/SKILL.md +163 -0
  111. package/skills/ctf-reverse/anti-analysis-ctf.md +204 -0
  112. package/skills/ctf-reverse/anti-analysis.md +693 -0
  113. package/skills/ctf-reverse/field-notes.md +376 -0
  114. package/skills/ctf-reverse/languages-compiled.md +666 -0
  115. package/skills/ctf-reverse/languages-platforms.md +592 -0
  116. package/skills/ctf-reverse/languages.md +553 -0
  117. package/skills/ctf-reverse/patterns-ctf-2.md +397 -0
  118. package/skills/ctf-reverse/patterns-ctf-3.md +797 -0
  119. package/skills/ctf-reverse/patterns-ctf.md +670 -0
  120. package/skills/ctf-reverse/patterns-runtime.md +274 -0
  121. package/skills/ctf-reverse/patterns.md +572 -0
  122. package/skills/ctf-reverse/platforms-hardware.md +387 -0
  123. package/skills/ctf-reverse/platforms.md +664 -0
  124. package/skills/ctf-reverse/tools-advanced-2.md +421 -0
  125. package/skills/ctf-reverse/tools-advanced.md +407 -0
  126. package/skills/ctf-reverse/tools-dynamic.md +679 -0
  127. package/skills/ctf-reverse/tools-emulation.md +319 -0
  128. package/skills/ctf-reverse/tools.md +573 -0
  129. package/skills/ctf-web/SKILL.md +153 -0
  130. package/skills/ctf-web/auth-and-access-2.md +82 -0
  131. package/skills/ctf-web/auth-and-access.md +783 -0
  132. package/skills/ctf-web/auth-infra.md +321 -0
  133. package/skills/ctf-web/auth-jwt.md +186 -0
  134. package/skills/ctf-web/client-side-advanced.md +739 -0
  135. package/skills/ctf-web/client-side.md +529 -0
  136. package/skills/ctf-web/cves.md +373 -0
  137. package/skills/ctf-web/field-notes.md +482 -0
  138. package/skills/ctf-web/node-and-prototype.md +200 -0
  139. package/skills/ctf-web/server-side-2.md +337 -0
  140. package/skills/ctf-web/server-side-advanced-2.md +559 -0
  141. package/skills/ctf-web/server-side-advanced-3.md +125 -0
  142. package/skills/ctf-web/server-side-advanced-4.md +480 -0
  143. package/skills/ctf-web/server-side-advanced.md +378 -0
  144. package/skills/ctf-web/server-side-deser.md +443 -0
  145. package/skills/ctf-web/server-side-exec-2.md +799 -0
  146. package/skills/ctf-web/server-side-exec.md +457 -0
  147. package/skills/ctf-web/server-side.md +629 -0
  148. package/skills/ctf-web/sql-injection.md +790 -0
  149. package/skills/ctf-web/web3.md +374 -0
  150. package/skills/ctf-writeup/SKILL.md +90 -0
  151. package/skills/solve-challenge/SKILL.md +269 -0
  152. package/skills.json +16 -0
@@ -0,0 +1,312 @@
1
+ # Social Media OSINT
2
+
3
+ ## Table of Contents
4
+ - [Twitter/X Account Tracking](#twitterx-account-tracking)
5
+ - [Tumblr Investigation](#tumblr-investigation)
6
+ - [BlueSky Advanced Search](#bluesky-advanced-search)
7
+ - [Username OSINT](#username-osint)
8
+ - [Platform False Positives](#platform-false-positives)
9
+ - [Social Media General Tips](#social-media-general-tips)
10
+ - [Multi-Platform OSINT Chain](#multi-platform-osint-chain)
11
+ - [Gaming Platform OSINT / MMO Character Lookup (CSAW CTF 2016)](#gaming-platform-osint--mmo-character-lookup-csaw-ctf-2016)
12
+ - [MetaCTF OSINT Challenge Patterns](#metactf-osint-challenge-patterns)
13
+ - [Unicode Homoglyph Steganography on BlueSky (MetaCTF 2026)](#unicode-homoglyph-steganography-on-bluesky-metactf-2026)
14
+ - [Strava Fitness Route OSINT (MidnightCTF 2026)](#strava-fitness-route-osint-midnightctf-2026)
15
+ - [Discord API Enumeration](#discord-api-enumeration)
16
+
17
+ ---
18
+
19
+ ## Twitter/X Account Tracking
20
+
21
+ **Persistent numeric User ID (key technique):**
22
+ - Every Twitter/X account has a permanent numeric ID that never changes
23
+ - Access any account by ID: `https://x.com/i/user/<numeric_id>` -- works even after username changes
24
+ - Find user ID from archived pages (JSON-LD `"author":{"identifier":"..."}`)
25
+ - Useful when username is deleted/changed but you have the ID from forensic artifacts
26
+
27
+ **Username rename detection:**
28
+ - Twitter User IDs persist across username changes; t.co shortlinks point to OLD usernames
29
+ - Wayback CDX API to find archived profiles: `http://web.archive.org/cdx/search/cdx?url=twitter.com/USERNAME*&output=json`
30
+ - Archived pages contain JSON-LD with user ID, creation date, follower/following counts
31
+ - t.co links in archived tweets reveal previous usernames (the redirect URL contains the username at time of posting)
32
+ - Same tweet ID accessible under different usernames = confirmed rename
33
+
34
+ **Alternative Twitter data sources:**
35
+ - Nitter instances (e.g., `nitter.poast.org/USERNAME`) show tweets without login
36
+ - Syndication API: `https://syndication.twitter.com/srv/timeline-profile/screen-name/USERNAME`
37
+ - Twitter Snowflake IDs encode timestamps: `(id >> 22) + 1288834974657` = Unix ms
38
+ - memory.lol and twitter.lolarchiver.com track username history
39
+
40
+ **Wayback Machine for Twitter:**
41
+ ```bash
42
+ # Find all archived URLs for a username
43
+ curl "http://web.archive.org/cdx/search/cdx?url=twitter.com/USERNAME*&output=json&fl=timestamp,original,statuscode"
44
+
45
+ # Also check profile images
46
+ curl "http://web.archive.org/cdx/search/cdx?url=pbs.twimg.com/profile_images/*&output=json"
47
+
48
+ # Check t.co shortlinks
49
+ curl "http://web.archive.org/cdx/search/cdx?url=t.co/SHORTCODE&output=json"
50
+ ```
51
+
52
+ ## Tumblr Investigation
53
+
54
+ **Blog existence check:**
55
+ - `curl -sI "https://USERNAME.tumblr.com"` -> look for `x-tumblr-user` header (confirms blog exists even if API returns 401)
56
+ - Tumblr API may return 401 (Unauthorized) but the blog is still publicly viewable via browser
57
+
58
+ **Extracting post content from Tumblr HTML:**
59
+ - Tumblr embeds post data as JSON in the page HTML
60
+ - Search for `"content":[` to find post body data
61
+ - Posts contain `type: "text"` with `text` field, and `type: "image"` with media URLs
62
+ - Avatar URL pattern: `https://64.media.tumblr.com/HASH/HASH-XX/s512x512u_c1/FILENAME.jpg`
63
+
64
+ **Avatar as flag container:**
65
+ - Direct avatar endpoint: `https://api.tumblr.com/v2/blog/USERNAME.tumblr.com/avatar/512`
66
+ - Or simply: `https://USERNAME.tumblr.com/avatar/512` (redirects to CDN URL)
67
+ - Available sizes: 16, 24, 30, 40, 48, 64, 96, 128, 512
68
+ - Flags may be hidden as small text in avatar images (visual stego, not binary stego)
69
+ - Always download highest resolution (512) and zoom in on all areas
70
+
71
+ ## BlueSky Advanced Search
72
+
73
+ **Pattern (Ms Blue Sky):** Find target's posts on BlueSky social media.
74
+
75
+ **Search filters:**
76
+ ```text
77
+ from:username # Posts from specific user
78
+ since:2025-01-01 # Date range
79
+ has:images # Posts with images
80
+ ```
81
+
82
+ **Reference:** https://bsky.social/about/blog/05-31-2024-search
83
+
84
+ ## Username OSINT
85
+
86
+ - [namechk.com](https://namechk.com) - Check username across platforms
87
+ - [whatsmyname.app](https://whatsmyname.app) - Username enumeration (741+ sites)
88
+ - [Osint Industries](https://osint.industries) - Cross-platform people search (paid, covers fitness/niche platforms)
89
+ - Search `"username"` in quotes on major platforms
90
+
91
+ **Username metadata mining:**
92
+ Usernames often embed geographic or temporal signals in their structure. Extract and research numeric suffixes, prefixes, or embedded patterns:
93
+
94
+ | Pattern | Example | Signal |
95
+ |---------|---------|--------|
96
+ | Trailing digits = postal/ZIP code | `LinXiayu35170` | 35170 = Bruz, France |
97
+ | Birth year suffix | `jsmith1998` | Born 1998 |
98
+ | Area code | `user212nyc` | 212 = Manhattan |
99
+ | Country code | `player44uk` | +44 = United Kingdom |
100
+
101
+ Cross-reference extracted codes with postal code databases, phone number registries, or geographic gazetteers to narrow the subject's location. (MidnightCTF 2026)
102
+
103
+ **Username chain tracing (account renames):**
104
+ 1. Start with known username -> find Wayback archives
105
+ 2. Look for t.co links or cross-references to other usernames in archived pages
106
+ 3. Discovered new username -> enumerate across ALL platforms again
107
+ 4. Repeat until you find the platform with the flag
108
+
109
+ **Priority platforms for CTF username enumeration:**
110
+ - Twitter/X, Tumblr, GitHub, Reddit, Bluesky, Mastodon
111
+ - Spotify, SoundCloud, Steam, Keybase
112
+ - Strava, Garmin Connect, MapMyRun (fitness/GPS — leak physical locations)
113
+ - Pastebin, LinkedIn, YouTube, TikTok
114
+ - bio-link services (linktr.ee, bio.link, about.me)
115
+
116
+ ## Platform False Positives
117
+
118
+ Platforms that return 200 but no real profile:
119
+ - Telegram (`t.me/USER`): Always returns 200 with "Contact @USER" page; check for "View" vs "Contact" in title
120
+ - TikTok: Returns 200 with "Couldn't find this account" in body
121
+ - Smule: Returns 200 with "Not Found" in page content
122
+ - linkin.bio: Redirects to Later.com product page for unclaimed names
123
+ - Instagram: Returns 200 but shows login wall (may or may not exist)
124
+
125
+ ## Social Media General Tips
126
+
127
+ - Check Wayback Machine for deleted posts on Bluesky, Twitter, etc.
128
+ - Unlisted YouTube videos may be linked in deleted posts
129
+ - Bio links lead to itch.io, personal sites with more info
130
+ - Search `"username"` with quotes on platform-specific searches
131
+ - Challenge titles are often hints (e.g., "Linked Traces" -> LinkedIn / linked accounts)
132
+ - **Twitter strips EXIF** on upload - don't waste time on stego for Twitter-served images
133
+ - **Tumblr preserves more metadata** in avatars than in post images
134
+
135
+ ## Multi-Platform OSINT Chain
136
+
137
+ **Pattern (Massive-Equipment393):** Reddit username -> Spotify social link -> Base58-encoded string -> Spotify playlist descriptions (base64) -> first-letter acrostic from song titles.
138
+
139
+ **Key techniques:**
140
+ - Base58 decoding for non-standard encodings
141
+ - Spotify playlists encode data in descriptions and song title initials
142
+ - Platform chaining: each platform links to the next
143
+
144
+ ## Gaming Platform OSINT / MMO Character Lookup (CSAW CTF 2016)
145
+
146
+ CTF OSINT challenges may require looking up game characters, guilds, or profiles across MMO platforms.
147
+
148
+ ```text
149
+ # World of Warcraft character/guild lookup:
150
+ # - Blizzard API: https://develop.battle.net/documentation/world-of-warcraft
151
+ # - WoW Progress: https://www.wowprogress.com
152
+ # - Raider.IO: https://raider.io
153
+ # Search: guild name + realm name (e.g., "Blackfathom Deep Dish" on US-Turalyon)
154
+
155
+ # Steam profile search:
156
+ # - steamcommunity.com/id/[username]
157
+ # - steamid.io for SteamID lookups
158
+
159
+ # Minecraft player lookup:
160
+ # - NameMC: https://namemc.com
161
+ # - Shows skin, name history, servers
162
+
163
+ # Discord user lookup:
164
+ # - discord.id for user/server lookups
165
+ # - Bot: UserInfo for detailed profiles
166
+
167
+ # Gaming OSINT chain pattern:
168
+ # 1. Blog/Twitter mentions guild or game name
169
+ # 2. Look up guild on game-specific tracker site
170
+ # 3. Find character name from guild roster
171
+ # 4. Character name may be used on other platforms
172
+ # 5. Cross-reference with other OSINT findings
173
+ ```
174
+
175
+ **Key insight:** Gaming profiles are often overlooked in OSINT but contain rich metadata (play times, real names, linked accounts, server regions). Guild/clan trackers index public game APIs and cache historical data. Character names are frequently reused across platforms.
176
+
177
+ ---
178
+
179
+ ## MetaCTF OSINT Challenge Patterns
180
+
181
+ **Common flow:**
182
+ 1. Start image with hidden EXIF/metadata -> extract username
183
+ 2. Username enumeration (Sherlock/WhatsMyName) across platforms
184
+ 3. Find profile on platform X with clues pointing to platform Y
185
+ 4. Flag hidden on the final platform (Spotify bio, BlueSky post, Tumblr avatar, etc.)
186
+
187
+ **Platform-specific flag locations:**
188
+ - Spotify: playlist names, artist bio
189
+ - BlueSky: post content
190
+ - Tumblr: avatar image, post text
191
+ - Reddit: post/comment content
192
+ - Smule: song recordings or bio
193
+ - SoundCloud: track description
194
+
195
+ **Key techniques:**
196
+ - Account rename tracking via Wayback + t.co links
197
+ - Cross-platform username correlation
198
+ - Visual inspection of all profile images at max resolution
199
+ - Song lyric identification -> artist/song as flag component
200
+
201
+ ## Unicode Homoglyph Steganography on BlueSky (MetaCTF 2026)
202
+
203
+ **Pattern (Skybound Secrets):** Flag hidden in a Bluesky post using Unicode homoglyph steganography — visually identical characters from different Unicode blocks encode binary data.
204
+
205
+ **Detection:**
206
+ - Post text looks normal but character-by-character analysis reveals non-ASCII codepoints
207
+ - Characters from Cyrillic (`а` U+0430 vs `a` U+0061), Greek, Armenian, Mathematical Monospace, etc.
208
+ - Each character encodes 1 bit: ASCII = 0, homoglyph = 1
209
+
210
+ **Bluesky API search workflow:**
211
+ ```bash
212
+ # Search for posts about the CTF
213
+ curl -s "https://public.api.bsky.app/xrpc/app.bsky.feed.searchPosts?q=metactf+flash+ctf&sort=latest" | jq '.posts[].record.text'
214
+
215
+ # Search for specific accounts
216
+ curl -s "https://public.api.bsky.app/xrpc/app.bsky.actor.searchActors?q=metactf" | jq '.actors[].handle'
217
+
218
+ # Get profile
219
+ curl -s "https://public.api.bsky.app/xrpc/app.bsky.actor.getProfile?actor=metactf.bsky.social" | jq
220
+
221
+ # Get author feed (all posts)
222
+ curl -s "https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=metactf.bsky.social&limit=50" | jq '.feed[].post.record.text'
223
+
224
+ # Get post thread (including replies)
225
+ curl -s "https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread?uri=at://did:plc:.../app.bsky.feed.post/..." | jq
226
+ ```
227
+
228
+ **Decoding homoglyph steganography:**
229
+ ```python
230
+ def decode_homoglyph_stego(text):
231
+ bits = []
232
+ for ch in text:
233
+ if ch in ('\u2019',): # Platform auto-inserted right single quote
234
+ continue # Skip, not intentional homoglyph
235
+ if ord(ch) < 128:
236
+ bits.append(0) # Standard ASCII
237
+ else:
238
+ bits.append(1) # Unicode homoglyph = 1 bit
239
+
240
+ # Group into bytes (MSB first)
241
+ flag = ''
242
+ for i in range(0, len(bits) - 7, 8):
243
+ byte_val = 0
244
+ for j in range(8):
245
+ byte_val = (byte_val << 1) | bits[i + j]
246
+ flag += chr(byte_val)
247
+ return flag
248
+ ```
249
+
250
+ **Common homoglyph pairs:**
251
+ | ASCII | Homoglyph | Unicode Block |
252
+ |-------|-----------|---------------|
253
+ | `a` (U+0061) | `а` (U+0430) | Cyrillic |
254
+ | `o` (U+006F) | `о` (U+043E) | Cyrillic |
255
+ | `e` (U+0065) | `е` (U+0435) | Cyrillic |
256
+ | `s` (U+0073) | `ѕ` (U+0455) | Cyrillic DZE |
257
+ | `t` (U+0074) | `𝚝` (U+1D69D) | Math Monospace |
258
+ | `p` (U+0070) | `р` (U+0440) | Cyrillic |
259
+
260
+ **Key lessons:**
261
+ - Check ALL replies to official CTF posts, not just the main post
262
+ - Platform auto-formatting (smart quotes `'` → `'`) must be excluded from bit encoding
263
+ - Hints like "hype comes with its own secrets" suggest steganography in the social media posts themselves
264
+ - Bluesky public API requires no authentication — use `public.api.bsky.app`
265
+
266
+ ---
267
+
268
+ ## Strava Fitness Route OSINT (MidnightCTF 2026)
269
+
270
+ **Pattern (Where was Chine):** Target's physical location identified through fitness tracking data. Username discovered on Twitter → alias found in GitHub code → alias searched on Strava → running route endpoint reveals location.
271
+
272
+ **Strava public data exposure:**
273
+ - Public athlete profiles: `https://www.strava.com/athletes/<id>`
274
+ - Activity maps show GPS routes with start/end points
275
+ - Even "privacy zones" can be circumvented by analyzing route shapes outside the zone
276
+ - Segment leaderboards reveal athlete locations without following them
277
+
278
+ **Location extraction workflow:**
279
+ 1. Find target's Strava profile via username enumeration (Whatsmyname, Osint Industries)
280
+ 2. Check public activities for GPS route maps
281
+ 3. Identify route start/end points or frequent locations
282
+ 4. Search the endpoint location on Google Maps
283
+ 5. Verify with Google Maps user-submitted photos (see [geolocation-and-media.md](geolocation-and-media.md#google-maps-crowd-sourced-photo-verification-midnightctf-2026))
284
+
285
+ **Key insight:** Fitness apps are high-value OSINT targets because users rarely restrict activity visibility. A single public run reveals home/work neighborhoods. Cross-reference GPS endpoints with Google Maps to identify specific parks, buildings, or landmarks.
286
+
287
+ **Detection:** Challenge mentions exercise, running, cycling, fitness, GPS, or health tracking. Target persona has an active/athletic profile.
288
+
289
+ ---
290
+
291
+ ## Discord API Enumeration
292
+
293
+ **Pattern (Insanity 1 & 2, 0xFun 2026):** Flags hidden in Discord server metadata not visible in normal UI.
294
+
295
+ **Hiding spots:**
296
+ - Role names
297
+ - Animated GIF emoji (flag in 2nd frame with tiny duration)
298
+ - Message embeds
299
+ - Server description, stickers, events
300
+
301
+ ```bash
302
+ # Enumerate with user token
303
+ TOKEN="your_token"
304
+ # List roles
305
+ curl -H "Authorization: $TOKEN" "https://discord.com/api/v10/guilds/GUILD_ID/roles"
306
+ # List emojis
307
+ curl -H "Authorization: $TOKEN" "https://discord.com/api/v10/guilds/GUILD_ID/emojis"
308
+ # Search messages
309
+ curl -H "Authorization: $TOKEN" "https://discord.com/api/v10/guilds/GUILD_ID/messages/search?content=flag"
310
+ ```
311
+
312
+ **Animated emoji:** Download GIF, extract frames -- hidden data in brief frames invisible at normal speed.
@@ -0,0 +1,341 @@
1
+ # Web and DNS OSINT
2
+
3
+ ## Table of Contents
4
+ - [Google Dorking](#google-dorking)
5
+ - [Google Docs/Sheets in OSINT](#google-docssheets-in-osint)
6
+ - [DNS Reconnaissance](#dns-reconnaissance)
7
+ - [DNS TXT Record OSINT](#dns-txt-record-osint)
8
+ - [Tor Relay Lookups](#tor-relay-lookups)
9
+ - [GitHub Repository Comments](#github-repository-comments)
10
+ - [Telegram Bot Investigation](#telegram-bot-investigation)
11
+ - [FEC Political Donation Research](#fec-political-donation-research)
12
+ - [Wayback Machine](#wayback-machine)
13
+ - [WHOIS Investigation](#whois-investigation)
14
+ - [Shodan SSH Fingerprint Lookup (EKOPARTY CTF 2016)](#shodan-ssh-fingerprint-lookup-ekoparty-ctf-2016)
15
+ - [Fake Service Banner Detection via Fingerprinting (MetaCTF Flash 2026)](#fake-service-banner-detection-via-fingerprinting-metactf-flash-2026)
16
+ - [Git Commit Author Mining for Credentials (Hackover 2018)](#git-commit-author-mining-for-credentials-hackover-2018)
17
+ - [.DS_Store Directory Enumeration with Python-dsstore (35C3 2018)](#ds_store-directory-enumeration-with-python-dsstore-35c3-2018)
18
+ - [TTF Glyph Contour Diffing for Obfuscated CAPTCHA (Square CTF 2018)](#ttf-glyph-contour-diffing-for-obfuscated-captcha-square-ctf-2018)
19
+ - [Cross-Challenge Container IP Reuse (RITSEC 2018)](#cross-challenge-container-ip-reuse-ritsec-2018)
20
+ - [Resources](#resources)
21
+
22
+ ---
23
+
24
+ ## Google Dorking
25
+
26
+ ```text
27
+ site:example.com filetype:pdf
28
+ intitle:"index of" password
29
+ inurl:admin
30
+ "confidential" filetype:doc
31
+ ```
32
+
33
+ **Google Image TBS (To Be Searched) parameters:**
34
+
35
+ Append `&tbs=` filters to Google Image search URLs for precision filtering:
36
+
37
+ | Filter | Parameter | Example |
38
+ |--------|-----------|---------|
39
+ | Faces only | `itp:face` | Find profile photos |
40
+ | Clipart | `itp:clipart` | Logos, icons |
41
+ | Animated GIF | `itp:animated` | Animated images |
42
+ | Specific color | `ic:specific,isc:green` | Dominant color filter |
43
+ | Transparent BG | `ic:trans` | PNGs with transparency |
44
+ | Large images | `isz:l` | High resolution only |
45
+ | Min resolution | `isz:lt,islt:2mp` | Greater than 2 megapixels |
46
+
47
+ **Combined example:** Search LinkedIn for face photos of interns at a company:
48
+ ```text
49
+ https://www.google.com/search?q="orange"+"alternant"+site:linkedin.com&tbm=isch&tbs=itp:face
50
+ ```
51
+
52
+ **Key insight:** The `itp:face` filter is especially useful for OSINT — it strips out logos, banners, and UI screenshots from results, leaving only profile photos. Combine with `site:` and date range (`after:YYYY-MM-DD`) for targeted reconnaissance.
53
+
54
+ ## Google Docs/Sheets in OSINT
55
+
56
+ - Suspects may link to Google Sheets/Docs in tweets or posts
57
+ - Try public access URLs:
58
+ - `/export?format=csv` - Export as CSV
59
+ - `/pub` - Published version
60
+ - `/gviz/tq?tqx=out:csv` - Visualization API CSV export
61
+ - `/htmlview` - HTML view
62
+ - Private sheets require authentication; flag may be in the sheet itself
63
+ - Sheet IDs are stable identifiers even if sharing settings change
64
+
65
+ ## DNS Reconnaissance
66
+
67
+ Flags often in TXT records of subdomains, not root domain:
68
+ ```bash
69
+ dig -t txt subdomain.ctf.domain.com
70
+ dig -t any domain.com
71
+ dig axfr @ns.domain.com domain.com # Zone transfer
72
+ ```
73
+
74
+ ## DNS TXT Record OSINT
75
+
76
+ ```bash
77
+ dig TXT ctf.domain.org
78
+ dig TXT _dmarc.domain.org
79
+ dig ANY domain.org
80
+ ```
81
+
82
+ **Lesson:** DNS TXT records are publicly queryable. Always check TXT, CNAME, MX for CTF domains and subdomains.
83
+
84
+ ## Tor Relay Lookups
85
+
86
+ ```text
87
+ https://metrics.torproject.org/rs.html#simple/<FINGERPRINT>
88
+ ```
89
+ Check family members and sort by "first seen" date for ordered flags.
90
+
91
+ ## GitHub Repository Comments
92
+
93
+ **Pattern (Rogue, VuwCTF 2025):** Hidden information in GitHub repo comments (issue comments, PR reviews, commit messages, wiki edits).
94
+
95
+ **Check:** `gh api repos/OWNER/REPO/issues/comments`, `gh api repos/OWNER/REPO/commits`, wiki edit history.
96
+
97
+ ## Telegram Bot Investigation
98
+
99
+ **Pattern:** Forensic artifacts (browser history, chat logs) may reference Telegram bots that require active interaction.
100
+
101
+ **Finding bot references in forensics:**
102
+ ```python
103
+ # Search browser history for Telegram URLs
104
+ import sqlite3
105
+ conn = sqlite3.connect("History") # Edge/Chrome history DB
106
+ cur = conn.cursor()
107
+ cur.execute("SELECT url FROM urls WHERE url LIKE '%t.me/%'")
108
+ # Example: https://t.me/comrade404_bot
109
+ ```
110
+
111
+ **Bot interaction workflow:**
112
+ 1. Visit `https://t.me/<botname>` -> Opens in Telegram
113
+ 2. Start conversation with `/start` or bot's custom command
114
+ 3. Bot may require verification (CTF-style challenges)
115
+ 4. Answers often require knowledge from forensic analysis
116
+
117
+ **Verification question patterns:**
118
+ - "Which user account did you use for X?" -> Check browser history, login records
119
+ - "Which account was modified?" -> Check Security.evtx Event 4781 (rename)
120
+ - "What file did you access?" -> Check MRU, Recent files, Shellbags
121
+
122
+ **Example bot flow:**
123
+ ```text
124
+ Bot: "TIER 1: Which account used for online search?"
125
+ -> Answer from Edge history showing Bing/Google searches
126
+
127
+ Bot: "TIER 2: Which account name did you change?"
128
+ -> Answer from Security event log (account rename events)
129
+
130
+ Bot: [Grants access] "Website: http://x.x.x.x:5000, Username: mehacker, Password: flaghere"
131
+ ```
132
+
133
+ **Key insight:** Bot responses may reveal:
134
+ - Attacker's real identity/handle
135
+ - Credentials to secondary systems
136
+ - Direct flag components
137
+ - Links to hidden web services
138
+
139
+ ## FEC Political Donation Research
140
+
141
+ **Pattern (Shell Game):** Track organizational donors through FEC filings.
142
+
143
+ **Key resources:**
144
+ - [FEC.gov](https://www.fec.gov/data/) - Committee receipts and expenditures
145
+ - 501(c)(4) organizations can donate to Super PACs without disclosing original funders
146
+ - Look for largest organizational donors, then research org leadership (CEO/President)
147
+
148
+ ## Wayback Machine
149
+
150
+ ```bash
151
+ # Find all archived URLs for a site
152
+ curl "http://web.archive.org/cdx/search/cdx?url=example.com*&output=json&fl=timestamp,original,statuscode"
153
+ ```
154
+
155
+ - Check for deleted posts, old profiles, cached pages
156
+ - CDX API for programmatic access to archive index
157
+
158
+ ## WHOIS Investigation
159
+
160
+ ```bash
161
+ # Basic WHOIS lookup
162
+ whois example.com
163
+
164
+ # Key fields to extract:
165
+ # - Registrant name/email/org (often redacted by privacy services)
166
+ # - Creation/expiration dates (timeline correlation)
167
+ # - Name servers (shared hosting identification)
168
+ # - Registrar (can indicate sophistication level)
169
+
170
+ # Historical WHOIS (before privacy was enabled)
171
+ # Use SecurityTrails, WhoisXML API, or DomainTools
172
+ curl "https://api.securitytrails.com/v1/domain/example.com/whois" \
173
+ -H "APIKEY: YOUR_KEY"
174
+
175
+ # Reverse WHOIS — find all domains registered by same entity
176
+ # Search by registrant email, org name, or phone number
177
+ curl "https://reverse-whois-api.whoisxmlapi.com/api/v2" \
178
+ -d '{"searchType":"current","mode":"purchase","basicSearchTerms":{"include":["target@email.com"]}}'
179
+
180
+ # IP WHOIS (find network owner)
181
+ whois 1.2.3.4
182
+ # Look for: NetName, OrgName, CIDR range, abuse contact
183
+
184
+ # ASN lookup
185
+ whois -h whois.radb.net AS12345
186
+ # Or use bgp.tools: https://bgp.tools/as/12345
187
+ ```
188
+
189
+ **Key insight:** WHOIS data is most useful for timeline correlation (when was the domain registered relative to CTF events?), reverse lookups (what other domains share the same registrant?), and identifying shared infrastructure. Historical WHOIS via SecurityTrails or Wayback Machine can reveal pre-privacy registrant details.
190
+
191
+ ---
192
+
193
+ ## Shodan SSH Fingerprint Lookup (EKOPARTY CTF 2016)
194
+
195
+ Discover the real IP behind a Tor hidden service or CDN by searching Shodan for the service's SSH fingerprint.
196
+
197
+ ```bash
198
+ # Step 1: Get SSH fingerprint from target
199
+ ssh-keyscan -t rsa target.onion 2>/dev/null | ssh-keygen -lf - -E md5
200
+ # Or use a dedicated scanner:
201
+ # pip install ssh-audit
202
+ ssh-audit target.onion
203
+
204
+ # Step 2: Extract the fingerprint hash
205
+ # e.g., MD5:ab:cd:ef:12:34:56:78:90:ab:cd:ef:12:34:56:78:90
206
+
207
+ # Step 3: Search Shodan for matching fingerprint
208
+ # Via API:
209
+ import shodan
210
+ api = shodan.Shodan('YOUR_API_KEY')
211
+ results = api.search('ssh.fingerprint:"ab:cd:ef:12:34:56:78:90:ab:cd:ef:12:34:56:78:90"')
212
+ for result in results['matches']:
213
+ print(f"IP: {result['ip_str']}")
214
+ print(f"Port: {result['port']}")
215
+ print(f"Banner: {result['data'][:200]}")
216
+
217
+ # Via Shodan CLI:
218
+ shodan search 'ssh.fingerprint:"ab:cd:ef:12:34:56:78:90"'
219
+
220
+ # Via web: https://www.shodan.io/search?query=ssh.fingerprint:%22...%22
221
+
222
+ # Also works with TLS certificate fingerprints:
223
+ # shodan search 'ssl.cert.fingerprint:"SHA256_HASH"'
224
+ ```
225
+
226
+ **Key insight:** SSH host keys are unique per server. If a hidden service runs SSH, its fingerprint can be searched on Shodan/Censys to find the real IP. This technique also works to de-anonymize services behind CloudFlare or other CDNs. Search both SSH fingerprints and TLS certificate fingerprints.
227
+
228
+ ---
229
+
230
+ ## Fake Service Banner Detection via Fingerprinting (MetaCTF Flash 2026)
231
+
232
+ **Pattern (O-Syn-T):** A port appears open on a standard service port (e.g., 22/SSH), but the service behind it is not what it claims. A basic SYN scan reports the port as open, but service version detection reveals a fake or custom banner containing the flag.
233
+
234
+ ```bash
235
+ # Step 1: Basic port scan finds port 22 open
236
+ nmap -sS target.ctf
237
+ # PORT STATE SERVICE
238
+ # 22/tcp open ssh
239
+
240
+ # Step 2: Service version fingerprinting reveals the deception
241
+ nmap -sV -sC target.ctf -p 22
242
+ # PORT STATE SERVICE VERSION
243
+ # 22/tcp open ssh?
244
+ # |_banner: MetaCTF{fake_banner_flag_here}
245
+
246
+ # Step 3: Or simply connect with netcat to read the banner
247
+ nc target.ctf 22
248
+ # MetaCTF{fake_banner_flag_here}
249
+
250
+ # Alternative: use curl or openssl for TLS-wrapped banners
251
+ echo "" | timeout 3 nc -w 3 target.ctf 22
252
+ ```
253
+
254
+ **Key insight:** Never trust port numbers alone. A SYN scan only confirms the port is open, not what service is running. Always run `nmap -sV` (version detection) or connect with `nc` to read the actual banner. CTF challenges exploit the assumption that port 22 = SSH, port 80 = HTTP, etc. Custom banner services on standard ports are a common OSINT/network recon trick.
255
+
256
+ **When to recognize:** Challenge name hints at network scanning or reconnaissance ("SYN", "scan", "port"). The expected approach is to enumerate open ports, but the flag is in the service banner itself rather than requiring exploitation.
257
+
258
+ ---
259
+
260
+ ## Git Commit Author Mining for Credentials (Hackover 2018)
261
+
262
+ **Pattern:** A challenge mentions a username with no credentials and expects the attacker to pivot to a public repository (GitHub/GitLab/Bitbucket) owned by that user. `git shortlog -sne` or `git log --format="%an <%ae>"` extracts every author email from the commit history — that address is often the valid login username the target service expects, before you attempt any password-reset or SQL-injection flow.
263
+
264
+ ```bash
265
+ # Clone the target's public repo and list every contributor email
266
+ git clone https://github.com/<target-user>/<repo>.git
267
+ cd repo
268
+ git shortlog -sne
269
+ # 23 John Doe <[email protected]>
270
+ # 5 John Doe <[email protected]> ← often the real login
271
+
272
+ # Pull every historic author at once:
273
+ git log --format="%an <%ae>%n%cn <%ce>" | sort -u
274
+ ```
275
+
276
+ ```bash
277
+ # GitHub-wide enumeration — list every event from a user
278
+ gh api "users/<target-user>/events/public" --paginate \
279
+ | jq -r '.[] | .payload.commits[]?.author.email' | sort -u
280
+ ```
281
+
282
+ **Key insight:** A git repo is a signed audit log of every author, committer, and co-author who has ever touched it. Even after someone rotates an email, the history keeps the old addresses. Mine both `author.email` and `committer.email`, and also look at `.mailmap`, `CONTRIBUTORS`, and GPG-signed commits (`git log --show-signature`). Treat each recovered email as a candidate login for the target service — many CTF web boxes, HR portals, and password-reset flows accept author emails straight from a public repo.
283
+
284
+ **References:** Hackover CTF 2018 — who knows john dows?, writeups 11537, 11646
285
+
286
+ ---
287
+
288
+ ## .DS_Store Directory Enumeration with Python-dsstore (35C3 2018)
289
+
290
+ **Pattern:** macOS `.DS_Store` files leak directory listings even when the web server hides them behind `robots.txt` or obscured paths. Download `.DS_Store` wherever possible (root, `/uploads/`, `/static/`) and parse it to enumerate filenames that are otherwise un-guessable.
291
+
292
+ ```bash
293
+ curl -sO https://target/.DS_Store
294
+ python3 -m dsstore .DS_Store
295
+ # prints every file the Finder ever saw in that directory
296
+ ```
297
+
298
+ **Key insight:** `.DS_Store` is generated automatically by macOS and often pushed to production by accident. It exposes filenames, not contents, but that is enough to find hidden admin panels, backup files, and uploaded flags.
299
+
300
+ **References:** 35C3 CTF 2018 — McDonald, writeup 12763
301
+
302
+ ---
303
+
304
+ ## TTF Glyph Contour Diffing for Obfuscated CAPTCHA (Square CTF 2018)
305
+
306
+ **Pattern:** A CAPTCHA serves obfuscated characters by remapping glyph IDs to random `cmap` entries, so the browser still displays "5" but the underlying Unicode codepoint is `U+E042`. Extract the TTF, dump each glyph's contours with `ttx`, build a reference library of known digit/letter contours, and `diff` incoming glyphs against the library to recover the true character.
307
+
308
+ ```bash
309
+ ttx -t glyf -g -d glyph_out font.ttf
310
+ # glyph_out/font.glyf/<glyph_name>.ttx holds contour XML
311
+ diff glyph_out/font.glyf/zero.ttx reference/zero.ttx
312
+ ```
313
+
314
+ **Key insight:** Visual CAPTCHAs that rely on custom fonts are trivially defeatable because the glyph *shapes* are invariant under cmap remapping. Build the reference once from any standard font and reuse it across every challenge variant.
315
+
316
+ **References:** Square CTF 2018 — C8, writeup 12161
317
+
318
+ ---
319
+
320
+ ## Cross-Challenge Container IP Reuse (RITSEC 2018)
321
+
322
+ **Pattern:** In Docker-hosted CTF infrastructures, all challenges in the same subnet often share an internal IP range. Leak the container's `REMOTE_ADDR` or routing table from one challenge (typically via a command-injection or SSRF), then apply that leaked IP to any other challenge that gates on `REMOTE_ADDR` hashes, `X-Forwarded-For` checks, or MD5(IP)-based upload paths.
323
+
324
+ ```text
325
+ # Challenge A leaks REMOTE_ADDR = 10.0.10.254
326
+ # Challenge B expects upload at /uploads/md5(10.0.10.254)/md5(time()).ext
327
+ ```
328
+
329
+ **Key insight:** Multi-challenge CTFs often leak infrastructure details cross-challenge. Always map the shared subnet first, then pivot info from the weakest challenge to the most constrained one.
330
+
331
+ **References:** RITSEC CTF 2018 — Lazy Dev → Archivr, writeups 12234-12235
332
+
333
+ ---
334
+
335
+ ## Resources
336
+
337
+ - **Shodan** - Internet-connected devices
338
+ - **Censys** - Certificate and host search
339
+ - **VirusTotal** - File/URL reputation
340
+ - **WHOIS** - Domain registration
341
+ - **Wayback Machine** - Historical snapshots