bmad-plus 0.12.1 → 0.12.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (156) hide show
  1. package/CHANGELOG.md +612 -580
  2. package/README.md +90 -115
  3. package/osint-agent-package/agents/osint-investigator.md +12 -0
  4. package/osint-agent-package/skills/bmad-osint-investigate/osint/SKILL.md +491 -482
  5. package/osint-agent-package/skills/bmad-osint-investigate/osint/assets/dossier-template.md +126 -126
  6. package/osint-agent-package/skills/bmad-osint-investigate/osint/assets/lawful-basis-record.md +48 -48
  7. package/osint-agent-package/skills/bmad-osint-investigate/osint/references/content-extraction.md +100 -100
  8. package/osint-agent-package/skills/bmad-osint-investigate/osint/references/gdpr-osint.md +48 -48
  9. package/osint-agent-package/skills/bmad-osint-investigate/osint/references/platforms.md +130 -130
  10. package/osint-agent-package/skills/bmad-osint-investigate/osint/references/psychoprofile.md +69 -69
  11. package/osint-agent-package/skills/bmad-osint-investigate/osint/references/tools.md +281 -281
  12. package/osint-agent-package/skills/bmad-osint-investigate/osint/scripts/mcp-client.py +136 -136
  13. package/package.json +104 -91
  14. package/readme-international/README.de.md +596 -594
  15. package/readme-international/README.es.md +613 -611
  16. package/readme-international/README.fr.md +611 -609
  17. package/src/bmad-plus/agents/agent-shadow/SKILL.md +18 -0
  18. package/src/bmad-plus/data/role-triggers.yaml +52 -0
  19. package/src/bmad-plus/module.yaml +283 -283
  20. package/src/bmad-plus/packs/pack-animated/animated-website-agent.md +325 -325
  21. package/src/bmad-plus/packs/pack-animated/templates/animated-website-workflow.md +55 -55
  22. package/src/bmad-plus/packs/pack-backup/backup-agent.md +71 -71
  23. package/src/bmad-plus/packs/pack-backup/templates/backup-workflow.md +51 -51
  24. package/src/bmad-plus/packs/pack-dev-studio/README.md +162 -162
  25. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/analyst-agent.md +73 -73
  26. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/document-project.md +61 -61
  27. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/domain-research.md +95 -95
  28. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/market-research.md +95 -95
  29. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/prfaq.md +134 -134
  30. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/product-brief.md +80 -80
  31. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/tech-writer-agent.md +73 -73
  32. package/src/bmad-plus/packs/pack-dev-studio/categories/analysis/technical-research.md +95 -95
  33. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/architect-agent.md +73 -73
  34. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/create-architecture.md +73 -73
  35. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/create-epics-stories.md +92 -92
  36. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/generate-project-context.md +80 -80
  37. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/implementation-readiness.md +90 -90
  38. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-01-init.md +153 -153
  39. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-01b-continue.md +173 -173
  40. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-02-context.md +224 -224
  41. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-03-starter.md +329 -329
  42. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-04-decisions.md +318 -318
  43. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-05-patterns.md +359 -359
  44. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-06-structure.md +379 -379
  45. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-07-validation.md +361 -361
  46. package/src/bmad-plus/packs/pack-dev-studio/categories/architecture/steps/step-08-complete.md +81 -81
  47. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/checkpoint-preview.md +67 -67
  48. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/code-review-steps/step-01-gather-context.md +85 -85
  49. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/code-review-steps/step-02-review.md +35 -35
  50. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/code-review-steps/step-03-triage.md +49 -49
  51. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/code-review-steps/step-04-present.md +131 -131
  52. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/code-review.md +89 -89
  53. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/correct-course.md +300 -300
  54. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/create-story.md +428 -428
  55. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/dev-agent.md +73 -73
  56. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/dev-story-checklist.md +80 -80
  57. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/dev-story.md +484 -484
  58. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/investigate.md +193 -193
  59. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/qa-e2e-tests.md +175 -175
  60. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/quick-dev.md +110 -110
  61. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/retrospective.md +1511 -1511
  62. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/sprint-planning.md +298 -298
  63. package/src/bmad-plus/packs/pack-dev-studio/categories/implementation/sprint-status.md +296 -296
  64. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/create-prd.md +29 -29
  65. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/create-ux-design.md +74 -74
  66. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/edit-prd.md +29 -29
  67. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/pm-agent.md +73 -73
  68. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/prd.md +89 -89
  69. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/ux-designer-agent.md +73 -73
  70. package/src/bmad-plus/packs/pack-dev-studio/categories/planning/validate-prd.md +29 -29
  71. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/advanced-elicitation.md +141 -141
  72. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/adversarial-review.md +37 -37
  73. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/bmad-help.md +75 -75
  74. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/brainstorming.md +6 -6
  75. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/customize.md +110 -110
  76. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/distillator.md +176 -176
  77. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/edge-case-hunter.md +67 -67
  78. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/editorial-review-prose.md +86 -86
  79. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/editorial-review-structure.md +179 -179
  80. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/index-docs.md +66 -66
  81. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/party-mode.md +127 -127
  82. package/src/bmad-plus/packs/pack-dev-studio/categories/utilities/shard-doc.md +105 -105
  83. package/src/bmad-plus/packs/pack-dev-studio/dev-studio-orchestrator.md +120 -120
  84. package/src/bmad-plus/packs/pack-dev-studio/shared/architecture-decision-template.md +12 -12
  85. package/src/bmad-plus/packs/pack-dev-studio/shared/bwml-spec.md +328 -328
  86. package/src/bmad-plus/packs/pack-dev-studio/shared/module-help.csv +32 -32
  87. package/src/bmad-plus/packs/pack-dev-studio/upstream-sync.yaml +81 -81
  88. package/src/bmad-plus/packs/pack-seo/SKILL.md +171 -171
  89. package/src/bmad-plus/packs/pack-seo/checklist.md +140 -140
  90. package/src/bmad-plus/packs/pack-seo/pagespeed-playbook.md +320 -320
  91. package/src/bmad-plus/packs/pack-seo/ref/audit-schema.json +187 -187
  92. package/src/bmad-plus/packs/pack-seo/ref/cwv-thresholds.md +87 -87
  93. package/src/bmad-plus/packs/pack-seo/ref/eeat-criteria.md +123 -123
  94. package/src/bmad-plus/packs/pack-seo/ref/geo-signals.md +167 -167
  95. package/src/bmad-plus/packs/pack-seo/ref/hreflang-rules.md +153 -153
  96. package/src/bmad-plus/packs/pack-seo/ref/quality-gates.md +133 -133
  97. package/src/bmad-plus/packs/pack-seo/ref/schema-catalog.md +91 -91
  98. package/src/bmad-plus/packs/pack-seo/ref/schema-templates.json +356 -356
  99. package/src/bmad-plus/packs/pack-seo/requirements.txt +17 -0
  100. package/src/bmad-plus/packs/pack-seo/scripts/seo_apis.py +456 -0
  101. package/src/bmad-plus/packs/pack-seo/scripts/seo_crawl.py +359 -0
  102. package/src/bmad-plus/packs/pack-seo/scripts/seo_fetch.py +304 -0
  103. package/src/bmad-plus/packs/pack-seo/scripts/seo_parse.py +255 -0
  104. package/src/bmad-plus/packs/pack-seo/scripts/seo_report.py +410 -0
  105. package/src/bmad-plus/packs/pack-seo/scripts/seo_screenshot.py +202 -0
  106. package/src/bmad-plus/packs/pack-seo/seo-chief.md +294 -294
  107. package/src/bmad-plus/packs/pack-seo/seo-judge.md +241 -241
  108. package/src/bmad-plus/packs/pack-seo/seo-scout.md +171 -171
  109. package/src/bmad-plus/packs/pack-seo/templates/seo-audit-workflow.md +241 -241
  110. package/src/bmad-plus/packs/pack-shield/README.md +6 -6
  111. package/src/bmad-plus/packs/pack-shield/SKILL.md +2 -2
  112. package/src/bmad-plus/packs/pack-shield/categories/accessibility-esg/csrd-agent.md +11 -11
  113. package/src/bmad-plus/packs/pack-shield/categories/accessibility-esg/section508-agent.md +11 -11
  114. package/src/bmad-plus/packs/pack-shield/categories/accessibility-esg/wcag-agent.md +11 -11
  115. package/src/bmad-plus/packs/pack-shield/categories/ai-governance/eu-ai-act-agent.md +11 -11
  116. package/src/bmad-plus/packs/pack-shield/categories/ai-governance/iso42001-agent.md +11 -11
  117. package/src/bmad-plus/packs/pack-shield/categories/ai-governance/nist-ai-rmf-agent.md +11 -11
  118. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/cis-controls-agent.md +11 -11
  119. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/ism-agent.md +11 -11
  120. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/iso27001-agent.md +11 -11
  121. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/nis2-agent.md +11 -11
  122. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/nist-800-53-agent.md +11 -11
  123. package/src/bmad-plus/packs/pack-shield/categories/cybersecurity/nist-csf-agent.md +11 -11
  124. package/src/bmad-plus/packs/pack-shield/categories/defense-export/cmmc-agent.md +11 -11
  125. package/src/bmad-plus/packs/pack-shield/categories/defense-export/ear-agent.md +11 -11
  126. package/src/bmad-plus/packs/pack-shield/categories/defense-export/itar-agent.md +11 -11
  127. package/src/bmad-plus/packs/pack-shield/categories/defense-export/tsa-agent.md +11 -11
  128. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/dora-agent.md +11 -11
  129. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/fedramp-agent.md +11 -11
  130. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/hipaa-agent.md +11 -11
  131. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/pci-dss-agent.md +11 -11
  132. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/soc2-agent.md +11 -11
  133. package/src/bmad-plus/packs/pack-shield/categories/industry-compliance/swift-csp-agent.md +11 -11
  134. package/src/bmad-plus/packs/pack-shield/shield-orchestrator.md +1 -1
  135. package/tools/build/check-counts.js +628 -0
  136. package/tools/build/generated-adapters/.codex/AGENTS.md +1 -1
  137. package/tools/build/generated-adapters/.cursor/rules/bmad-plus.mdc +1 -1
  138. package/tools/build/generated-adapters/.opencode/AGENTS.md +1 -1
  139. package/tools/build/generated-adapters/AGENTS.md +1 -1
  140. package/tools/build/generated-adapters/CLAUDE.md +1 -1
  141. package/tools/build/generated-adapters/CONVENTIONS.md +1 -1
  142. package/tools/build/generated-adapters/GEMINI.md +1 -1
  143. package/tools/cli/commands/autoconfig.js +470 -470
  144. package/tools/cli/commands/doctor.js +233 -233
  145. package/tools/cli/commands/install.js +598 -501
  146. package/tools/cli/commands/memory-journal-cmd.js +311 -311
  147. package/tools/cli/commands/memory.js +195 -195
  148. package/tools/cli/commands/scan.js +348 -348
  149. package/tools/cli/commands/uninstall.js +101 -101
  150. package/tools/cli/commands/update.js +134 -134
  151. package/tools/cli/i18n.js +845 -845
  152. package/tools/cli/lib/README-memory-journal.md +125 -125
  153. package/tools/cli/lib/ide-config.js +267 -259
  154. package/tools/cli/lib/python-provision.js +508 -508
  155. package/tools/cli/lib/stack-detect.js +102 -102
  156. package/tools/cli/lib/validate.js +50 -50
@@ -1,482 +1,491 @@
1
- ---
2
- name: osint
3
- description: >
4
- Conduct deep OSINT research on individuals. Build full digital footprint, psychoprofile
5
- (MBTI/Big Five), career history, social graph with confidence scores. Recursive
6
- self-evaluation until completeness threshold is met. Includes internal intelligence
7
- (Telegram history, email, vault contacts) before going external.
8
- Use when: "osint", "досье", "research person", "find everything about", "пробей",
9
- "разведка", "due diligence", "background check", "digital footprint",
10
- "найди всё про", "собери информацию", "кто это", "профиль человека".
11
- NOT for: company/product research without a named person, competitive analysis,
12
- market research, content generation, or general web scraping tasks.
13
- ---
14
-
15
- # OSINT Skill v3.2
16
-
17
- Systematic intelligence gathering on individuals. From a name or handle to a scored
18
- dossier with psychoprofile, career map, and entry points.
19
-
20
- ## Phase −1 — Lawful Basis Gate (MANDATORY, runs before anything else)
21
-
22
- > Profiling a **named natural person** — especially building a psychoprofile — is
23
- > processing of personal data (and, for inferred personality/health/political traits,
24
- > **special-category data**) under the GDPR and analogous laws. This gate is not
25
- > optional. Governed by the Shield (GRC) pack. **Do NOT start Phase 0** on a named
26
- > person until it passes.
27
-
28
- Before any investigation of a named/identifiable person, establish and record:
29
-
30
- 1. **Lawful basis (GDPR Art. 6)** — which applies? (consent, contract, legal
31
- obligation, vital interests, public task, or **legitimate interest** with a
32
- documented balancing test). For psychoprofiling / special-category inferences,
33
- also an **Art. 9** condition (explicit consent, or another Art. 9(2) exemption).
34
- 2. **Purpose** — the specific, explicit, legitimate purpose (purpose limitation).
35
- Reject open-ended "find everything" with no purpose.
36
- 3. **Data-subject rights & retention** — who is the controller, retention period,
37
- and how a DSAR / erasure request will be honored.
38
-
39
- **Record it** using `assets/lawful-basis-record.md` and put the completed header at
40
- the top of the dossier (see `assets/dossier-template.md` governance block).
41
-
42
- **If the operator cannot state a lawful basis and purpose → DECLINE** the named-person
43
- profiling. Offer the non-personal alternatives instead (company/product/market
44
- research, or investigation of a public entity), which do not require this gate.
45
-
46
- This gate does not restrict lawful, authorized use (due diligence with a basis,
47
- security research, investigations you are mandated to run) — it makes that basis
48
- explicit and auditable. See `references/gdpr-osint.md`.
49
-
50
- ## Phase Router
51
-
52
- Determine entry point from context:
53
-
54
- - New name/handle/URL, "пробей", "find out about" Phase 0 (full cycle)
55
- - "Add LinkedIn/Instagram data" to existing dossier Phase 2 (extraction)
56
- - "Build psychoprofile" from existing data → Phase 4
57
- - "Rate completeness" of existing dossier → Phase 5
58
- - "Reformat" or "present" findings → Phase 6
59
-
60
- Default (full research request): Phase 0 → 1 → 1.5 2 → 3 → 4 → 5 → 6.
61
-
62
- ## Environment
63
-
64
- All API keys via environment variables. Never hardcode tokens.
65
-
66
- - `PERPLEXITY_API_KEY` Perplexity Sonar (fast answers + deep research)
67
- - `EXA_API_KEY` Exa AI (semantic search, company/people research, deep research)
68
- - `TAVILY_API_KEY` — Tavily (agent-optimized search + extract, $0.005/req basic)
69
- - `APIFY_API_TOKEN` Apify scraping (LinkedIn, Instagram, Facebook)
70
- - `JINA_API_KEY` — Jina reader/search/deepsearch
71
- - `PARALLEL_API_KEY` — Parallel AI search
72
- - `BRIGHTDATA_MCP_URL` — Bright Data MCP endpoint (full URL with token)
73
- - `MCPORTER_CONFIG` mcporter config path
74
-
75
- ## Scripts
76
-
77
- Run from skill dir: `bash scripts/<name>.sh`.
78
- Each validates env vars, exits with descriptive error + URL to get the key.
79
-
80
- **Search & Research:**
81
- - `diagnose.sh` — run FIRST. Capability map of all tools.
82
- - `perplexity.sh` — `search <query>` | `sonar <query>` (AI answer) | `deep <query>` (deep research)
83
- - `tavily.sh` — `search <query>` (basic $0.005) | `deep <query>` (advanced) | `extract <url>`
84
- - `exa.sh` — `search <query>` | `company <name>` | `people <name>` | `crawl <url>` | `deep <prompt>`
85
- - `first-volley.sh "Name" "context"` — parallel search, all engines at once.
86
- - `merge-volley.sh <outdir>` deduplicate and merge first-volley results.
87
-
88
- **Scraping:**
89
- - `apify.sh` — `linkedin <url>` | `instagram <handle>` | `run` | `results` | `store-search`
90
- - `run-actor.sh` — **universal Apify runner (55+ actors).** Embedded from [apify/agent-skills](https://github.com/apify/agent-skills).
91
- Quick answer: `bash scripts/run-actor.sh "actor/id" '{"input":"json"}'`
92
- Export: `bash scripts/run-actor.sh "actor/id" '{"input":"json"}' --output /tmp/out.csv`
93
- - `jina.sh` — `read <url>` | `search <query>` | `deepsearch <query>`
94
- - `parallel.sh` — `search <query>` | `extract <url>`
95
- - `brightdata.sh` — `scrape <url>` | `scrape-batch` | `search` | `search-geo <cc>` | `search-yandex`
96
-
97
- ## Research Escalation Flow
98
-
99
- **Принцип: от дешёвого к дорогому, от быстрого к глубокому.**
100
-
101
- ### Level 1: Quick Answers (секунды, ~$0.00)
102
- Начни ВСЕГДА с этого. Получи быстрый контекст прежде чем копать.
103
- Запускай ВСЕ параллельно:
104
- ```bash
105
- # Perplexity Sonar — AI ответ с цитатами
106
- bash skills/osint/scripts/perplexity.sh sonar "Who is <Name>, <context>"
107
- # Brave Search — классический поиск
108
- web_search "<Name> <company> <role>"
109
- # Tavily — agent-optimized search с AI answer
110
- bash skills/osint/scripts/tavily.sh search "<Name> <context>"
111
- # Exa семантический поиск + company/people research
112
- bash skills/osint/scripts/exa.sh search "<Name> <context>"
113
- bash skills/osint/scripts/exa.sh people "<Name>"
114
- ```
115
- Получаешь: быстрые факты, ссылки, контекст.
116
- Решение: достаточно? Phase 6. Нужно больше? → Level 2.
117
-
118
- ### Level 2: Source Verification (секунды-минуты, ~$0.01)
119
- Проверяй источники из Level 1 через fetch:
120
- ```bash
121
- # Читай найденные URL
122
- web_fetch "<url_from_perplexity>"
123
- bash skills/osint/scripts/jina.sh read "<url>"
124
- bash skills/osint/scripts/parallel.sh extract "<url>"
125
- ```
126
- → Получаешь: подтверждённые факты, cross-reference.
127
- Совпадает? дополняй досье. Нужно глубже? → Level 3.
128
-
129
- ### Level 3: Social Media Deep Dive (~$0.01-0.10)
130
- Подключай scraping для соцсетей:
131
- ```bash
132
- # LinkedIn
133
- bash skills/osint/scripts/apify.sh linkedin "<url>"
134
- # Instagram
135
- bash skills/osint/scripts/apify.sh instagram "<handle>"
136
- # Facebook, заблокированные сайты
137
- bash skills/osint/scripts/brightdata.sh scrape "<url>"
138
- ```
139
- Получаешь: структурированные профили, фото, связи.
140
-
141
- ### Level 4: Deep Research (~$0.05-0.50)
142
- Если нужно копать ещё глубже — формируй развёрнутый промпт и отправляй в deep research.
143
- Запускай ВСЕ параллельно (30-60 сек каждый):
144
- ```bash
145
- # Perplexity Deep Research
146
- bash skills/osint/scripts/perplexity.sh deep "<detailed research prompt about Name>"
147
- # Exa Deep Research
148
- bash skills/osint/scripts/exa.sh deep "<detailed prompt>"
149
- # Parallel AI Deep Search
150
- bash skills/osint/scripts/parallel.sh search "<detailed query>"
151
- # Jina DeepSearch
152
- bash skills/osint/scripts/jina.sh deepsearch "<query>"
153
- ```
154
-
155
- **Правило:** Level 4 промпт должен быть РАЗВЁРНУТЫМ — включай всё что уже знаешь
156
- из Level 1-3, чтобы deep research не повторял базовые факты, а копал дальше.
157
-
158
- ## Swarm Mode (DEFAULT)
159
-
160
- OSINT research runs as a **swarm of parallel sub-agents on Sonnet**.
161
- The main agent is the coordinator — it does NOT scrape itself.
162
-
163
- ### How it works:
164
- 1. Main agent runs Phase 0 (tooling check) and Phase 1 (seed collection) to get initial context
165
- 2. Main agent spawns 3-5 sub-agents via `sessions_spawn` with `model: sonnet`, `mode: run`
166
- 3. Each sub-agent gets a focused task + all known data from Phase 1
167
- 4. Sub-agents return results → main agent merges into dossier
168
-
169
- ### Task split pattern:
170
- - **Agent 1: YouTube/Content** extract transcripts via Apify (NOT yt-dlp, NOT BrightData — YouTube blocks them). 3-5 videos, speech style, topics. Use `streamers/youtube-channel-scraper` for channel data
171
- - **Agent 2: Facebook deep** — BrightData scrape: profile, posts, about, photos, friends (use m.facebook.com for more data). For public Pages: `apify/facebook-pages-scraper` + `apify/facebook-page-contact-information`
172
- - **Agent 3: Social platforms** — Instagram (Apify + tagged/comments scrapers), DOU, company websites, LinkedIn (BrightData). Contact enrichment: `vdrmota/contact-info-scraper` on found websites
173
- - **Agent 4: TikTok + Regional** TikTok profile/videos (`clockworks/tiktok-profile-scraper`), local registries, press, university records, Yandex search, Google Maps (`compass/crawler-google-places` if business owner)
174
- - **Agent 5: Deep research** Perplexity deep, Exa deep, Parallel deep (if needed)
175
-
176
- ### Rules:
177
- - Always pass ALL known data to each sub-agent (names, URLs, emails, phones, context)
178
- - Each sub-agent saves results to `/tmp/osint-<subject>-<task>.md`
179
- - Main agent waits for all results, then runs Phase 3-6 (cross-reference, psychoprofile, dossier)
180
- - Budget: each sub-agent ≤$0.15, total swarm ≤$0.50
181
- - YouTube transcripts: use **Apify** actors, NOT BrightData or yt-dlp (both blocked by YouTube)
182
-
183
- ### Why swarm:
184
- - 5 agents × 5 min = 10 min total (vs 30+ min sequential)
185
- - Sonnet is 5x cheaper than Opus
186
- - Parallel scraping avoids rate limit stacking on single IP
187
-
188
- ---
189
-
190
- ## Phase 0: Tooling Self-Check
191
-
192
- 1. Execute `bash skills/osint/scripts/diagnose.sh`.
193
- 2. Log available vs missing tools.
194
- 3. Check internal tools: `tg.py` (Telegram history), `himalaya` (email), vault contacts.
195
- 4. If Bright Data unavailable Facebook and LinkedIn deep scrape limited. Inform user.
196
- 5. If Apify unavailable → Instagram and LinkedIn structured data limited.
197
- 6. Proceed with available toolset.
198
-
199
- ## Phase 1: Seed Collection
200
-
201
- **Start with Level 1 (quick answers) ALWAYS before heavy scraping.**
202
-
203
- 1. Parse user input. Extract identifiers: names, handles, URLs, companies, locations.
204
- 2. **Perplexity fast pass:**
205
- ```bash
206
- bash skills/osint/scripts/perplexity.sh search "Who is <Name>, <context>"
207
- ```
208
- 3. **Brave + Parallel in parallel:**
209
- ```bash
210
- web_search "<Name> <company>"
211
- bash skills/osint/scripts/first-volley.sh "Full Name" "context"
212
- ```
213
- 4. **Review Perplexity citations** — fetch and verify top sources:
214
- ```bash
215
- web_fetch "<citation_url_1>"
216
- web_fetch "<citation_url_2>"
217
- ```
218
- 5. Parse & merge: `bash skills/osint/scripts/merge-volley.sh /tmp/osint-<timestamp>`.
219
- 6. Collect all identifiers into seed list. Deduplicate.
220
- 7. Flag name collisions (common names → verify with company/location cross-reference).
221
- 8. **Decision point:** enough context? → skip to Phase 4. Need social media? → Phase 2. Need deep dive? → Level 4 (deep research).
222
-
223
- **Rate limiting:** wait 1s between Brave queries, 2s between Jina calls.
224
- Do NOT hammer APIs in tight loops — stagger parallel launches.
225
-
226
- ## Phase 1.5: Internal Intelligence
227
-
228
- **Before going external, check what we already know.** This phase mines local sources
229
- that may contain gold prior conversations, emails, vault contacts.
230
-
231
- ### Telegram History
232
- If `tg.py` is available (check Phase 0):
233
- ```bash
234
- # Search by name/handle in Telegram
235
- python3 skills/telegram/scripts/tg.py search "Name" 20
236
- # If we have their username/id — read conversation history
237
- python3 skills/telegram/scripts/tg.py history <username_or_id> 50
238
- ```
239
-
240
- **What to extract from Telegram history:**
241
- - Communication style (formal/informal, language, emoji patterns)
242
- - Topics discussed — what they care about, what they ask for
243
- - Response patterns reply speed, active hours → timezone
244
- - Shared links/files projects they work on
245
- - How they address the userrelationship dynamics
246
- - Mentioned colleagues, partners, competitors → social graph seeds
247
- - Pricing discussions, deal terms (if business contact)
248
-
249
- ⚠️ **Telegram history is Grade A intelligence** — unfiltered, real-time, authentic.
250
- Weight it higher than curated LinkedIn/Instagram profiles.
251
- ⚠️ **Privacy:** internal intelligence stays in the dossier. Never quote DMs in public outputs.
252
-
253
- ### Email History
254
- If `himalaya` is available:
255
- ```bash
256
- # Search emails by name or domain
257
- ~/.local/bin/himalaya search "from:name@domain.com OR to:name@domain.com" -f INBOX
258
- # Or by name
259
- ~/.local/bin/himalaya search "Name Surname" -f INBOX
260
- ~/.local/bin/himalaya search "Name Surname" -f Sent
261
- ```
262
-
263
- **What to extract from email:**
264
- - Formal communication style vs Telegram style (contrast = insight)
265
- - Business proposals, invoices financial relationship
266
- - CC'd people organizational map
267
- - Signature block → title, phone, company, social links (often richer than LinkedIn)
268
-
269
- ### Vault / CRM Check
270
- ```bash
271
- # Check if we already have a card
272
- grep -rl "Name" vault/crm/ vault/contacts/ 2>/dev/null
273
- # Check MOC indexes (adjust paths to your vault structure)
274
- grep -i "name" vault/MOC/*.md 2>/dev/null
275
- ```
276
-
277
- **If vault card exists:** read it, note last_accessed, existing tags, prior interactions.
278
- Don't duplicate enrich the existing card after research completes.
279
-
280
- ### Node Camera/Location (if paired device available)
281
- If meeting in person and node is available, `nodes camera_snap` can capture context.
282
- Only with explicit user permission.
283
-
284
- ### Internal Intelligence Summary
285
- After Phase 1.5, you should know:
286
- - Do we have prior relationship? (cold/warm/hot contact)
287
- - What language do they prefer?
288
- - What's their communication style?
289
- - Any existing business context?
290
- - Social graph seeds from conversations
291
-
292
- This context shapes Phase 2 priorities — if we already know their career from emails,
293
- focus external research on psychoprofile and social media instead.
294
-
295
- ## Phase 2: Platform Extraction
296
-
297
- Read `references/platforms.md` ONLY when needing URL patterns or extraction signals.
298
-
299
- Tool priority (primary fallback). **If primary fails, switch immediately. Never retry same tool.**
300
-
301
- - LinkedIn: `apify.sh linkedin` `brightdata.sh scrape` `jina.sh read`
302
- - Instagram: `apify.sh instagram` `brightdata.sh scrape`
303
- - Instagram deep: `run-actor.sh "apify/instagram-tagged-scraper"` (who tags them), `apify/instagram-comment-scraper` (sentiment)
304
- - Facebook personal: `brightdata.sh scrape` → none (only Bright Data works)
305
- - Facebook pages/groups: `run-actor.sh "apify/facebook-pages-scraper"` → `brightdata.sh scrape`
306
- - TikTok: `run-actor.sh "clockworks/tiktok-profile-scraper"` `clockworks/tiktok-scraper` (comprehensive)
307
- - TikTok discovery: `run-actor.sh "clockworks/tiktok-user-search-scraper"` (find by keywords)
308
- - YouTube: `run-actor.sh "streamers/youtube-channel-scraper"` `jina.sh read` `brightdata.sh scrape`
309
- - Telegram channels: `web_fetch t.me/s/{channel}` → `jina.sh read`
310
- - Twitter/X: `python3 scripts/twitter.py tweet <url>` → `jina.sh read`
311
- - Google Maps (businesses): `run-actor.sh "compass/crawler-google-places"`
312
- - Contact enrichment: `run-actor.sh "vdrmota/contact-info-scraper"` (extract emails/phones from any URL)
313
- - Any site: `jina.sh read` → `brightdata.sh scrape`
314
-
315
- **run-actor.sh** = universal Apify runner (embedded, 55+ actors). See `references/tools.md` for full actor catalog.
316
-
317
- Read `references/tools.md` ONLY when troubleshooting a failed tool.
318
-
319
- ### ⚠️ Content Platform Rule (CRITICAL)
320
-
321
- When you find YouTube, podcast, blog, or conference talks — read `references/content-extraction.md` **immediately** and extract 3-5 pieces of content on the spot.
322
-
323
- Do NOT just note the URL. Extract transcripts/text NOW.
324
- A 20-minute YouTube video reveals more about a person than their entire LinkedIn.
325
- Content platforms are the #1 source for psychoprofile — skipping them = shallow dossier.
326
-
327
- ### OpSec-Aware Targets
328
-
329
- If initial searches return unusually little for someone who should have a footprint:
330
-
331
- 1. **Wayback Machine:** `web_fetch "https://web.archive.org/web/2024*/target-url"` — deleted profiles, old bios
332
- 2. **Google Cache:** `web_search "cache:domain.com/path"` recently removed pages
333
- 3. **Yandex Cache:** `brightdata.sh search-yandex "Name"` Yandex indexes CIS deeper and caches longer
334
- 4. **Username variations:** try transliteration (Иванов ivanov, ivanoff), birth year suffixes, company abbreviations
335
- 5. **Reverse image search:** if photo found, check for other profiles using same avatar
336
- 6. **Conference archives:** speaker bios often survive after profiles are deleted
337
-
338
- ## Phase 3: Cross-Reference & Confidence Scoring
339
-
340
- ### Step 1: Fact Table
341
- List every claim as a row: fact | source 1 | source 2 | grade.
342
-
343
- ### Step 2: Cross-check key facts
344
- For each critical fact (employer, role, location, education):
345
- - Compare LinkedIn title vs Telegram signature vs email signature vs company website
346
- - If 2+ match → Grade A
347
- - If only 1 source Grade B
348
- - If inferred (timezone from messages, geotag) → Grade C
349
- - If single unverified mention → Grade D
350
-
351
- ### Step 3: Resolve contradictions
352
- If LinkedIn says "CEO" but company site says "Co-founder" flag explicitly. Include both with sources. Do NOT silently pick one.
353
-
354
- ### Step 4: Name collision check
355
- If common name — verify at least 2 facts (company + city, or photo + company) link to same person. If unsure, split into separate entities.
356
-
357
- ### Confidence grades:
358
-
359
- - **A (confirmed)**: 2+ independent sources, or official/verified profile, or direct Telegram/email conversation
360
- - **B (probable)**: 1 credible source (LinkedIn, official media, company site)
361
- - **C (inferred)**: indirect evidence (photo geotag, timezone from message patterns, connections)
362
- - **D (unverified)**: single mention, could be wrong
363
-
364
- Internal intelligence (Phase 1.5) counts as an independent source.
365
-
366
- ## Phase 4: Psychoprofile
367
-
368
- Read `references/psychoprofile.md` ONLY at this phase.
369
-
370
- 1. Collect text samples: posts, bios, interviews, channel content, **Telegram messages** (highest signal).
371
- 2. Assess MBTI per dimension with cited behavioral evidence and confidence (high/medium/low).
372
- 3. Quantify writing style: sentence length, emoji density, self-reference rate.
373
- 4. **Compare formal (LinkedIn/email) vs informal (Telegram/Instagram) voice** the delta reveals the real person.
374
- 5. Deduce values from actions, not self-reported claims.
375
- 6. Zodiac ONLY if DOB confirmed (Grade A or B).
376
-
377
- ## Phase 5: Completeness Evaluation (Recursive)
378
-
379
- ### Axis 1: Data Coverage (pass/fail per dimension)
380
-
381
- 9 mandatory checks. If any fail, flag as critical gap:
382
-
383
- 1. Subject correctly identified? (not a namesake)
384
- 2. Current role/company confirmed?
385
- 3. At least 2 social platforms found?
386
- 4. At least 1 contact method (email/phone/messenger)?
387
- 5. Career history has 2+ verifiable positions?
388
- 6. Location (current) established?
389
- 7. At least 1 photo found?
390
- 8. No unresolved contradictions between sources?
391
- 9. Internal intelligence checked? (Telegram/email/vault — even if empty)
392
-
393
- ### Axis 2: Depth Score (8 weighted criteria)
394
-
395
- | Dimension | Weight | What to score (1-10) |
396
- |-----------|--------|---------------------|
397
- | Identity | 0.15 | Full name, DOB, location, education, photo |
398
- | Career | 0.20 | Completeness of work history, current role clarity |
399
- | Digital footprint | 0.15 | Number of platforms found, account activity level |
400
- | Psychoprofile | 0.15 | MBTI confidence, writing style quantified, values deduced |
401
- | Internal intel | 0.10 | Telegram/email history depth, vault data |
402
- | Personal life | 0.05 | Family, hobbies, lifestyle, pets |
403
- | Cross-reference | 0.10 | How many facts are A-grade, contradiction count |
404
- | Actionability | 0.10 | Entry points identified, approach strategy clear |
405
-
406
- Weighted sum (1-10) = **Depth Score**.
407
-
408
- ### Axis 3: Source Diversity
409
-
410
- Count unique source types used (max 12):
411
- LinkedIn, Instagram, Facebook, Telegram DM, Telegram channel, VK, Twitter/X,
412
- company website, press/media articles, conference profiles, government/business registries,
413
- email correspondence.
414
-
415
- - 8+ source types = Excellent
416
- - 5-7 = Good
417
- - 2-4 = Shallow
418
- - 1 = Insufficient
419
-
420
- ### Gap Analysis
421
-
422
- | Depth Score | Coverage | Diagnosis | Action |
423
- |------------|----------|-----------|--------|
424
- | 8+ | All pass | Strong dossier | Proceed to Phase 6 |
425
- | 8+ | Some fail | Deep but blind spots | Target failed checks, 1 more cycle |
426
- | <7 | All pass | Wide but shallow | Deepen via interviews/articles/deepsearch |
427
- | <7 | Some fail | Restart needed | Different search angle, new tool combination |
428
-
429
- ### Stopping Criteria
430
-
431
- **(a)** Depth Score 8.0 AND all coverage checks pass → exit to Phase 6
432
- **(b)** 3 cycles completed → deliver best available with honest assessment
433
- **(c)** Two cycles with delta < 0.5 plateau reached, deliver with note
434
-
435
- ### Calibration Benchmarks
436
-
437
- - **9-10**: full career timeline, 5+ platforms, confirmed DOB, psychoprofile with high confidence, family/hobbies known, multiple entry points, Telegram history analyzed. Equivalent to a professional PI report.
438
- - **7-8**: career outline, 3+ platforms, most facts B-grade or above, psychoprofile with medium confidence. Solid due diligence.
439
- - **5-6**: basic bio, 1-2 platforms, some gaps. Quick background check level.
440
- - **<5**: minimal data found. Name + current role at best. Flag as insufficient.
441
-
442
- ## Phase 6: Dossier Output
443
-
444
- Read `assets/dossier-template.md` before rendering. Follow the template structure exactly.
445
- No markdown tables in output (Telegram cannot render). Bullet lists only.
446
- Report Depth Score, source count, source types, and total API spend.
447
-
448
- If internal intelligence was used, add a separate **"из переписки"** section
449
- (marked as internal/confidential, not for sharing outside).
450
-
451
- ## Budget
452
-
453
- - ≤$0.50 per target: spend without asking.
454
- - >$0.50: ask user before proceeding.
455
- - Track cumulative spend per research session.
456
-
457
- ## Troubleshooting
458
-
459
- - **All tools return empty**: target has minimal digital presence. Try Bright Data Yandex search (better for CIS region), search by company + role instead of name.
460
- - **Wrong person keeps appearing**: add company name, city, or role to all queries. Use quotes around full name.
461
- - **LinkedIn blocked**: use `brightdata.sh scrape` as primary instead of Apify.
462
- - **Apify actor dead/changed**: check `apify.sh store-search "linkedin scraper"` for alternatives. Actors on Apify are volatile — always have a Bright Data fallback.
463
- - **Depth Score stuck at 6-7**: likely missing press/media articles or internal intel. Search industry publications (AdIndex, Sostav, Forbes, Kommersant for Russian market). Try `jina.sh deepsearch`. Check Telegram history.
464
- - **No social media found**: person may use pseudonyms. Search by email, phone, or company employee page. Search Apify store: `bash scripts/apify.sh store-search "people search"`. If `mcpc` installed: `APIFY_TOKEN=$APIFY_API_TOKEN mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="people search" limit:=10`. Check Telegram contacts by phone.
465
- - **TikTok scraper fails**: try `clockworks/free-tiktok-scraper` (free tier) as fallback. TikTok usernames often differ from other platforms — search by real name via `clockworks/tiktok-user-search-scraper`.
466
- - **Need emails from website**: use `vdrmota/contact-info-scraper` — it crawls the site and extracts all contact info.
467
- - **Rate limited (429)**: back off 5s, then 15s. Switch to fallback tool. Never retry immediately.
468
-
469
- ## Anti-Patterns
470
-
471
- 1. Never start with a single tool. Launch all available in parallel.
472
- 2. Never retry a failed tool more than once. Switch to fallback.
473
- 3. Never guess DOB, family, or zodiac.
474
- 4. Never attribute data without cross-referencing against namesakes.
475
- 5. Never include unsourced facts.
476
- 6. Never reveal OSINT methods in public messages.
477
- 7. Never exceed 3 recursive cycles. Diminishing returns.
478
- 8. Never rate Depth Score 9+ without justification.
479
- 9. Never skip psychoprofile. Without it, dossier = Wikipedia article.
480
- 10. Never skip Phase 1.5 (internal intel). Telegram history is often the richest source.
481
- 11. Never quote DMs verbatim in shareable outputs. Summarize and cite.
482
- 12. Never hammer APIs without rate limiting. Stagger requests.
1
+ ---
2
+ name: osint
3
+ description: >
4
+ Conduct deep OSINT research on individuals. Build full digital footprint, psychoprofile
5
+ (MBTI/Big Five), career history, social graph with confidence scores. Recursive
6
+ self-evaluation until completeness threshold is met. Includes internal intelligence
7
+ (Telegram history, email, vault contacts) before going external.
8
+ Use when: "osint", "досье", "research person", "find everything about", "пробей",
9
+ "разведка", "due diligence", "background check", "digital footprint",
10
+ "найди всё про", "собери информацию", "кто это", "профиль человека".
11
+ NOT for: company/product research without a named person, competitive analysis,
12
+ market research, content generation, or general web scraping tasks.
13
+ ---
14
+
15
+ # OSINT Skill v3.2
16
+
17
+ Systematic intelligence gathering on individuals. From a name or handle to a scored
18
+ dossier with psychoprofile, career map, and entry points.
19
+
20
+ ## Phase −1 — Lawful Basis Gate (MANDATORY, runs before anything else)
21
+
22
+ > Profiling a **named natural person** — especially building a psychoprofile — is
23
+ > processing of personal data (and, for inferred personality/health/political traits,
24
+ > **special-category data**) under the GDPR and analogous laws. This gate is not
25
+ > optional. Governed by the Shield (GRC) pack. **Do NOT start Phase 0** on a named
26
+ > person until it passes.
27
+
28
+ Before any investigation of a named/identifiable person, establish and record:
29
+
30
+ 1. **Lawful basis (GDPR Art. 6)** — which applies? (consent, contract, legal
31
+ obligation, vital interests, public task, or **legitimate interest** with a
32
+ documented balancing test). For psychoprofiling / special-category inferences,
33
+ also an **Art. 9** condition (explicit consent, or another Art. 9(2) exemption).
34
+ 2. **Purpose** — the specific, explicit, legitimate purpose (purpose limitation).
35
+ Reject open-ended "find everything" with no purpose.
36
+ 3. **Data-subject rights & retention** — who is the controller, retention period,
37
+ and how a DSAR / erasure request will be honored.
38
+
39
+ **Record it** using `assets/lawful-basis-record.md` and put the completed header at
40
+ the top of the dossier (see `assets/dossier-template.md` governance block).
41
+
42
+ **If the operator cannot state a lawful basis and purpose → DECLINE** the named-person
43
+ profiling. Offer the non-personal alternatives instead (company/product/market
44
+ research, or investigation of a public entity), which do not require this gate.
45
+
46
+ This gate does not restrict lawful, authorized use (due diligence with a basis,
47
+ security research, investigations you are mandated to run) — it makes that basis
48
+ explicit and auditable. See `references/gdpr-osint.md`.
49
+
50
+ ## Phase Router
51
+
52
+ Determine entry point from context. **Every entry point that touches a named or
53
+ identifiable natural person passes the Phase −1 Lawful Basis Gate FIRST — this
54
+ includes the direct extraction/profiling entries below. There is no route to
55
+ personal data that skips it, whether reached from the full pipeline or from a
56
+ single-purpose menu item (LinkedIn/Instagram/Facebook scrape, psychoprofile,
57
+ contact enrichment).**
58
+
59
+ - New name/handle/URL, "пробей", "find out about" → **Phase −1** → Phase 0 (full cycle)
60
+ - "Add LinkedIn/Instagram data" to existing dossier **Phase −1**Phase 2 (extraction)
61
+ - "Build psychoprofile" from existing data → **Phase −1** (Art. 9 condition required) → Phase 4
62
+ - "Contact enrichment" (email/phone) for a person → **Phase −1** → Phase 2
63
+ - "Rate completeness" of existing dossier → Phase 5
64
+ - "Reformat" or "present" findings Phase 6
65
+
66
+ Only operations on **non-personal** subjects (company/product/market research, public
67
+ entities) and non-processing utilities (tooling `diagnose`) skip the gate.
68
+
69
+ Default (full research request): Phase −1 0 → 1 → 1.5 → 2 → 3 → 4 → 5 → 6.
70
+
71
+ ## Environment
72
+
73
+ All API keys via environment variables. Never hardcode tokens.
74
+
75
+ - `PERPLEXITY_API_KEY` — Perplexity Sonar (fast answers + deep research)
76
+ - `EXA_API_KEY` — Exa AI (semantic search, company/people research, deep research)
77
+ - `TAVILY_API_KEY` Tavily (agent-optimized search + extract, $0.005/req basic)
78
+ - `APIFY_API_TOKEN` Apify scraping (LinkedIn, Instagram, Facebook)
79
+ - `JINA_API_KEY` — Jina reader/search/deepsearch
80
+ - `PARALLEL_API_KEY` — Parallel AI search
81
+ - `BRIGHTDATA_MCP_URL` — Bright Data MCP endpoint (full URL with token)
82
+ - `MCPORTER_CONFIG` — mcporter config path
83
+
84
+ ## Scripts
85
+
86
+ Run from skill dir: `bash scripts/<name>.sh`.
87
+ Each validates env vars, exits with descriptive error + URL to get the key.
88
+
89
+ **Search & Research:**
90
+ - `diagnose.sh` — run FIRST. Capability map of all tools.
91
+ - `perplexity.sh` — `search <query>` | `sonar <query>` (AI answer) | `deep <query>` (deep research)
92
+ - `tavily.sh` `search <query>` (basic $0.005) | `deep <query>` (advanced) | `extract <url>`
93
+ - `exa.sh` — `search <query>` | `company <name>` | `people <name>` | `crawl <url>` | `deep <prompt>`
94
+ - `first-volley.sh "Name" "context"` — parallel search, all engines at once.
95
+ - `merge-volley.sh <outdir>` deduplicate and merge first-volley results.
96
+
97
+ **Scraping:**
98
+ - `apify.sh` — `linkedin <url>` | `instagram <handle>` | `run` | `results` | `store-search`
99
+ - `run-actor.sh` **universal Apify runner (55+ actors).** Embedded from [apify/agent-skills](https://github.com/apify/agent-skills).
100
+ Quick answer: `bash scripts/run-actor.sh "actor/id" '{"input":"json"}'`
101
+ Export: `bash scripts/run-actor.sh "actor/id" '{"input":"json"}' --output /tmp/out.csv`
102
+ - `jina.sh` `read <url>` | `search <query>` | `deepsearch <query>`
103
+ - `parallel.sh` — `search <query>` | `extract <url>`
104
+ - `brightdata.sh` — `scrape <url>` | `scrape-batch` | `search` | `search-geo <cc>` | `search-yandex`
105
+
106
+ ## Research Escalation Flow
107
+
108
+ **Принцип: от дешёвого к дорогому, от быстрого к глубокому.**
109
+
110
+ ### Level 1: Quick Answers (секунды, ~$0.00)
111
+ Начни ВСЕГДА с этого. Получи быстрый контекст прежде чем копать.
112
+ Запускай ВСЕ параллельно:
113
+ ```bash
114
+ # Perplexity Sonar — AI ответ с цитатами
115
+ bash skills/osint/scripts/perplexity.sh sonar "Who is <Name>, <context>"
116
+ # Brave Search классический поиск
117
+ web_search "<Name> <company> <role>"
118
+ # Tavily agent-optimized search с AI answer
119
+ bash skills/osint/scripts/tavily.sh search "<Name> <context>"
120
+ # Exa — семантический поиск + company/people research
121
+ bash skills/osint/scripts/exa.sh search "<Name> <context>"
122
+ bash skills/osint/scripts/exa.sh people "<Name>"
123
+ ```
124
+ Получаешь: быстрые факты, ссылки, контекст.
125
+ → Решение: достаточно? → Phase 6. Нужно больше? → Level 2.
126
+
127
+ ### Level 2: Source Verification (секунды-минуты, ~$0.01)
128
+ Проверяй источники из Level 1 через fetch:
129
+ ```bash
130
+ # Читай найденные URL
131
+ web_fetch "<url_from_perplexity>"
132
+ bash skills/osint/scripts/jina.sh read "<url>"
133
+ bash skills/osint/scripts/parallel.sh extract "<url>"
134
+ ```
135
+ Получаешь: подтверждённые факты, cross-reference.
136
+ Совпадает? дополняй досье. Нужно глубже? → Level 3.
137
+
138
+ ### Level 3: Social Media Deep Dive (~$0.01-0.10)
139
+ Подключай scraping для соцсетей:
140
+ ```bash
141
+ # LinkedIn
142
+ bash skills/osint/scripts/apify.sh linkedin "<url>"
143
+ # Instagram
144
+ bash skills/osint/scripts/apify.sh instagram "<handle>"
145
+ # Facebook, заблокированные сайты
146
+ bash skills/osint/scripts/brightdata.sh scrape "<url>"
147
+ ```
148
+ Получаешь: структурированные профили, фото, связи.
149
+
150
+ ### Level 4: Deep Research (~$0.05-0.50)
151
+ Если нужно копать ещё глубже — формируй развёрнутый промпт и отправляй в deep research.
152
+ Запускай ВСЕ параллельно (30-60 сек каждый):
153
+ ```bash
154
+ # Perplexity Deep Research
155
+ bash skills/osint/scripts/perplexity.sh deep "<detailed research prompt about Name>"
156
+ # Exa Deep Research
157
+ bash skills/osint/scripts/exa.sh deep "<detailed prompt>"
158
+ # Parallel AI Deep Search
159
+ bash skills/osint/scripts/parallel.sh search "<detailed query>"
160
+ # Jina DeepSearch
161
+ bash skills/osint/scripts/jina.sh deepsearch "<query>"
162
+ ```
163
+
164
+ **Правило:** Level 4 промпт должен быть РАЗВЁРНУТЫМ включай всё что уже знаешь
165
+ из Level 1-3, чтобы deep research не повторял базовые факты, а копал дальше.
166
+
167
+ ## Swarm Mode (DEFAULT)
168
+
169
+ OSINT research runs as a **swarm of parallel sub-agents on Sonnet**.
170
+ The main agent is the coordinator it does NOT scrape itself.
171
+
172
+ ### How it works:
173
+ 1. Main agent runs Phase 0 (tooling check) and Phase 1 (seed collection) to get initial context
174
+ 2. Main agent spawns 3-5 sub-agents via `sessions_spawn` with `model: sonnet`, `mode: run`
175
+ 3. Each sub-agent gets a focused task + all known data from Phase 1
176
+ 4. Sub-agents return results → main agent merges into dossier
177
+
178
+ ### Task split pattern:
179
+ - **Agent 1: YouTube/Content** extract transcripts via Apify (NOT yt-dlp, NOT BrightData YouTube blocks them). 3-5 videos, speech style, topics. Use `streamers/youtube-channel-scraper` for channel data
180
+ - **Agent 2: Facebook deep** — BrightData scrape: profile, posts, about, photos, friends (use m.facebook.com for more data). For public Pages: `apify/facebook-pages-scraper` + `apify/facebook-page-contact-information`
181
+ - **Agent 3: Social platforms** — Instagram (Apify + tagged/comments scrapers), DOU, company websites, LinkedIn (BrightData). Contact enrichment: `vdrmota/contact-info-scraper` on found websites
182
+ - **Agent 4: TikTok + Regional** — TikTok profile/videos (`clockworks/tiktok-profile-scraper`), local registries, press, university records, Yandex search, Google Maps (`compass/crawler-google-places` if business owner)
183
+ - **Agent 5: Deep research** — Perplexity deep, Exa deep, Parallel deep (if needed)
184
+
185
+ ### Rules:
186
+ - Always pass ALL known data to each sub-agent (names, URLs, emails, phones, context)
187
+ - Each sub-agent saves results to `/tmp/osint-<subject>-<task>.md`
188
+ - Main agent waits for all results, then runs Phase 3-6 (cross-reference, psychoprofile, dossier)
189
+ - Budget: each sub-agent ≤$0.15, total swarm ≤$0.50
190
+ - YouTube transcripts: use **Apify** actors, NOT BrightData or yt-dlp (both blocked by YouTube)
191
+
192
+ ### Why swarm:
193
+ - 5 agents × 5 min = 10 min total (vs 30+ min sequential)
194
+ - Sonnet is 5x cheaper than Opus
195
+ - Parallel scraping avoids rate limit stacking on single IP
196
+
197
+ ---
198
+
199
+ ## Phase 0: Tooling Self-Check
200
+
201
+ 1. Execute `bash skills/osint/scripts/diagnose.sh`.
202
+ 2. Log available vs missing tools.
203
+ 3. Check internal tools: `tg.py` (Telegram history), `himalaya` (email), vault contacts.
204
+ 4. If Bright Data unavailable → Facebook and LinkedIn deep scrape limited. Inform user.
205
+ 5. If Apify unavailable → Instagram and LinkedIn structured data limited.
206
+ 6. Proceed with available toolset.
207
+
208
+ ## Phase 1: Seed Collection
209
+
210
+ **Start with Level 1 (quick answers) ALWAYS before heavy scraping.**
211
+
212
+ 1. Parse user input. Extract identifiers: names, handles, URLs, companies, locations.
213
+ 2. **Perplexity fast pass:**
214
+ ```bash
215
+ bash skills/osint/scripts/perplexity.sh search "Who is <Name>, <context>"
216
+ ```
217
+ 3. **Brave + Parallel in parallel:**
218
+ ```bash
219
+ web_search "<Name> <company>"
220
+ bash skills/osint/scripts/first-volley.sh "Full Name" "context"
221
+ ```
222
+ 4. **Review Perplexity citations** — fetch and verify top sources:
223
+ ```bash
224
+ web_fetch "<citation_url_1>"
225
+ web_fetch "<citation_url_2>"
226
+ ```
227
+ 5. Parse & merge: `bash skills/osint/scripts/merge-volley.sh /tmp/osint-<timestamp>`.
228
+ 6. Collect all identifiers into seed list. Deduplicate.
229
+ 7. Flag name collisions (common names verify with company/location cross-reference).
230
+ 8. **Decision point:** enough context? → skip to Phase 4. Need social media? → Phase 2. Need deep dive? → Level 4 (deep research).
231
+
232
+ **Rate limiting:** wait 1s between Brave queries, 2s between Jina calls.
233
+ Do NOT hammer APIs in tight loops — stagger parallel launches.
234
+
235
+ ## Phase 1.5: Internal Intelligence
236
+
237
+ **Before going external, check what we already know.** This phase mines local sources
238
+ that may contain gold — prior conversations, emails, vault contacts.
239
+
240
+ ### Telegram History
241
+ If `tg.py` is available (check Phase 0):
242
+ ```bash
243
+ # Search by name/handle in Telegram
244
+ python3 skills/telegram/scripts/tg.py search "Name" 20
245
+ # If we have their username/idread conversation history
246
+ python3 skills/telegram/scripts/tg.py history <username_or_id> 50
247
+ ```
248
+
249
+ **What to extract from Telegram history:**
250
+ - Communication style (formal/informal, language, emoji patterns)
251
+ - Topics discussed what they care about, what they ask for
252
+ - Response patterns — reply speed, active hours → timezone
253
+ - Shared links/files — projects they work on
254
+ - How they address the user — relationship dynamics
255
+ - Mentioned colleagues, partners, competitors → social graph seeds
256
+ - Pricing discussions, deal terms (if business contact)
257
+
258
+ ⚠️ **Telegram history is Grade A intelligence** — unfiltered, real-time, authentic.
259
+ Weight it higher than curated LinkedIn/Instagram profiles.
260
+ ⚠️ **Privacy:** internal intelligence stays in the dossier. Never quote DMs in public outputs.
261
+
262
+ ### Email History
263
+ If `himalaya` is available:
264
+ ```bash
265
+ # Search emails by name or domain
266
+ ~/.local/bin/himalaya search "from:name@domain.com OR to:name@domain.com" -f INBOX
267
+ # Or by name
268
+ ~/.local/bin/himalaya search "Name Surname" -f INBOX
269
+ ~/.local/bin/himalaya search "Name Surname" -f Sent
270
+ ```
271
+
272
+ **What to extract from email:**
273
+ - Formal communication style vs Telegram style (contrast = insight)
274
+ - Business proposals, invoices → financial relationship
275
+ - CC'd people → organizational map
276
+ - Signature block → title, phone, company, social links (often richer than LinkedIn)
277
+
278
+ ### Vault / CRM Check
279
+ ```bash
280
+ # Check if we already have a card
281
+ grep -rl "Name" vault/crm/ vault/contacts/ 2>/dev/null
282
+ # Check MOC indexes (adjust paths to your vault structure)
283
+ grep -i "name" vault/MOC/*.md 2>/dev/null
284
+ ```
285
+
286
+ **If vault card exists:** read it, note last_accessed, existing tags, prior interactions.
287
+ Don't duplicate enrich the existing card after research completes.
288
+
289
+ ### Node Camera/Location (if paired device available)
290
+ If meeting in person and node is available, `nodes camera_snap` can capture context.
291
+ Only with explicit user permission.
292
+
293
+ ### Internal Intelligence Summary
294
+ After Phase 1.5, you should know:
295
+ - Do we have prior relationship? (cold/warm/hot contact)
296
+ - What language do they prefer?
297
+ - What's their communication style?
298
+ - Any existing business context?
299
+ - Social graph seeds from conversations
300
+
301
+ This context shapes Phase 2 priorities if we already know their career from emails,
302
+ focus external research on psychoprofile and social media instead.
303
+
304
+ ## Phase 2: Platform Extraction
305
+
306
+ Read `references/platforms.md` ONLY when needing URL patterns or extraction signals.
307
+
308
+ Tool priority (primaryfallback). **If primary fails, switch immediately. Never retry same tool.**
309
+
310
+ - LinkedIn: `apify.sh linkedin` → `brightdata.sh scrape` → `jina.sh read`
311
+ - Instagram: `apify.sh instagram` `brightdata.sh scrape`
312
+ - Instagram deep: `run-actor.sh "apify/instagram-tagged-scraper"` (who tags them), `apify/instagram-comment-scraper` (sentiment)
313
+ - Facebook personal: `brightdata.sh scrape` → none (only Bright Data works)
314
+ - Facebook pages/groups: `run-actor.sh "apify/facebook-pages-scraper"` → `brightdata.sh scrape`
315
+ - TikTok: `run-actor.sh "clockworks/tiktok-profile-scraper"` `clockworks/tiktok-scraper` (comprehensive)
316
+ - TikTok discovery: `run-actor.sh "clockworks/tiktok-user-search-scraper"` (find by keywords)
317
+ - YouTube: `run-actor.sh "streamers/youtube-channel-scraper"` `jina.sh read` `brightdata.sh scrape`
318
+ - Telegram channels: `web_fetch t.me/s/{channel}` → `jina.sh read`
319
+ - Twitter/X: `python3 scripts/twitter.py tweet <url>` → `jina.sh read`
320
+ - Google Maps (businesses): `run-actor.sh "compass/crawler-google-places"`
321
+ - Contact enrichment: `run-actor.sh "vdrmota/contact-info-scraper"` (extract emails/phones from any URL)
322
+ - Any site: `jina.sh read` → `brightdata.sh scrape`
323
+
324
+ **run-actor.sh** = universal Apify runner (embedded, 55+ actors). See `references/tools.md` for full actor catalog.
325
+
326
+ Read `references/tools.md` ONLY when troubleshooting a failed tool.
327
+
328
+ ### ⚠️ Content Platform Rule (CRITICAL)
329
+
330
+ When you find YouTube, podcast, blog, or conference talks — read `references/content-extraction.md` **immediately** and extract 3-5 pieces of content on the spot.
331
+
332
+ Do NOT just note the URL. Extract transcripts/text NOW.
333
+ A 20-minute YouTube video reveals more about a person than their entire LinkedIn.
334
+ Content platforms are the #1 source for psychoprofile skipping them = shallow dossier.
335
+
336
+ ### OpSec-Aware Targets
337
+
338
+ If initial searches return unusually little for someone who should have a footprint:
339
+
340
+ 1. **Wayback Machine:** `web_fetch "https://web.archive.org/web/2024*/target-url"` — deleted profiles, old bios
341
+ 2. **Google Cache:** `web_search "cache:domain.com/path"` recently removed pages
342
+ 3. **Yandex Cache:** `brightdata.sh search-yandex "Name"` — Yandex indexes CIS deeper and caches longer
343
+ 4. **Username variations:** try transliteration (Иванов → ivanov, ivanoff), birth year suffixes, company abbreviations
344
+ 5. **Reverse image search:** if photo found, check for other profiles using same avatar
345
+ 6. **Conference archives:** speaker bios often survive after profiles are deleted
346
+
347
+ ## Phase 3: Cross-Reference & Confidence Scoring
348
+
349
+ ### Step 1: Fact Table
350
+ List every claim as a row: fact | source 1 | source 2 | grade.
351
+
352
+ ### Step 2: Cross-check key facts
353
+ For each critical fact (employer, role, location, education):
354
+ - Compare LinkedIn title vs Telegram signature vs email signature vs company website
355
+ - If 2+ match Grade A
356
+ - If only 1 source → Grade B
357
+ - If inferred (timezone from messages, geotag) → Grade C
358
+ - If single unverified mention → Grade D
359
+
360
+ ### Step 3: Resolve contradictions
361
+ If LinkedIn says "CEO" but company site says "Co-founder" flag explicitly. Include both with sources. Do NOT silently pick one.
362
+
363
+ ### Step 4: Name collision check
364
+ If common name — verify at least 2 facts (company + city, or photo + company) link to same person. If unsure, split into separate entities.
365
+
366
+ ### Confidence grades:
367
+
368
+ - **A (confirmed)**: 2+ independent sources, or official/verified profile, or direct Telegram/email conversation
369
+ - **B (probable)**: 1 credible source (LinkedIn, official media, company site)
370
+ - **C (inferred)**: indirect evidence (photo geotag, timezone from message patterns, connections)
371
+ - **D (unverified)**: single mention, could be wrong
372
+
373
+ Internal intelligence (Phase 1.5) counts as an independent source.
374
+
375
+ ## Phase 4: Psychoprofile
376
+
377
+ Read `references/psychoprofile.md` ONLY at this phase.
378
+
379
+ 1. Collect text samples: posts, bios, interviews, channel content, **Telegram messages** (highest signal).
380
+ 2. Assess MBTI per dimension with cited behavioral evidence and confidence (high/medium/low).
381
+ 3. Quantify writing style: sentence length, emoji density, self-reference rate.
382
+ 4. **Compare formal (LinkedIn/email) vs informal (Telegram/Instagram) voice** — the delta reveals the real person.
383
+ 5. Deduce values from actions, not self-reported claims.
384
+ 6. Zodiac ONLY if DOB confirmed (Grade A or B).
385
+
386
+ ## Phase 5: Completeness Evaluation (Recursive)
387
+
388
+ ### Axis 1: Data Coverage (pass/fail per dimension)
389
+
390
+ 9 mandatory checks. If any fail, flag as critical gap:
391
+
392
+ 1. Subject correctly identified? (not a namesake)
393
+ 2. Current role/company confirmed?
394
+ 3. At least 2 social platforms found?
395
+ 4. At least 1 contact method (email/phone/messenger)?
396
+ 5. Career history has 2+ verifiable positions?
397
+ 6. Location (current) established?
398
+ 7. At least 1 photo found?
399
+ 8. No unresolved contradictions between sources?
400
+ 9. Internal intelligence checked? (Telegram/email/vault even if empty)
401
+
402
+ ### Axis 2: Depth Score (8 weighted criteria)
403
+
404
+ | Dimension | Weight | What to score (1-10) |
405
+ |-----------|--------|---------------------|
406
+ | Identity | 0.15 | Full name, DOB, location, education, photo |
407
+ | Career | 0.20 | Completeness of work history, current role clarity |
408
+ | Digital footprint | 0.15 | Number of platforms found, account activity level |
409
+ | Psychoprofile | 0.15 | MBTI confidence, writing style quantified, values deduced |
410
+ | Internal intel | 0.10 | Telegram/email history depth, vault data |
411
+ | Personal life | 0.05 | Family, hobbies, lifestyle, pets |
412
+ | Cross-reference | 0.10 | How many facts are A-grade, contradiction count |
413
+ | Actionability | 0.10 | Entry points identified, approach strategy clear |
414
+
415
+ Weighted sum (1-10) = **Depth Score**.
416
+
417
+ ### Axis 3: Source Diversity
418
+
419
+ Count unique source types used (max 12):
420
+ LinkedIn, Instagram, Facebook, Telegram DM, Telegram channel, VK, Twitter/X,
421
+ company website, press/media articles, conference profiles, government/business registries,
422
+ email correspondence.
423
+
424
+ - 8+ source types = Excellent
425
+ - 5-7 = Good
426
+ - 2-4 = Shallow
427
+ - 1 = Insufficient
428
+
429
+ ### Gap Analysis
430
+
431
+ | Depth Score | Coverage | Diagnosis | Action |
432
+ |------------|----------|-----------|--------|
433
+ | 8+ | All pass | Strong dossier | Proceed to Phase 6 |
434
+ | 8+ | Some fail | Deep but blind spots | Target failed checks, 1 more cycle |
435
+ | <7 | All pass | Wide but shallow | Deepen via interviews/articles/deepsearch |
436
+ | <7 | Some fail | Restart needed | Different search angle, new tool combination |
437
+
438
+ ### Stopping Criteria
439
+
440
+ **(a)** Depth Score 8.0 AND all coverage checks pass exit to Phase 6
441
+ **(b)** 3 cycles completed → deliver best available with honest assessment
442
+ **(c)** Two cycles with delta < 0.5 → plateau reached, deliver with note
443
+
444
+ ### Calibration Benchmarks
445
+
446
+ - **9-10**: full career timeline, 5+ platforms, confirmed DOB, psychoprofile with high confidence, family/hobbies known, multiple entry points, Telegram history analyzed. Equivalent to a professional PI report.
447
+ - **7-8**: career outline, 3+ platforms, most facts B-grade or above, psychoprofile with medium confidence. Solid due diligence.
448
+ - **5-6**: basic bio, 1-2 platforms, some gaps. Quick background check level.
449
+ - **<5**: minimal data found. Name + current role at best. Flag as insufficient.
450
+
451
+ ## Phase 6: Dossier Output
452
+
453
+ Read `assets/dossier-template.md` before rendering. Follow the template structure exactly.
454
+ No markdown tables in output (Telegram cannot render). Bullet lists only.
455
+ Report Depth Score, source count, source types, and total API spend.
456
+
457
+ If internal intelligence was used, add a separate **"из переписки"** section
458
+ (marked as internal/confidential, not for sharing outside).
459
+
460
+ ## Budget
461
+
462
+ - ≤$0.50 per target: spend without asking.
463
+ - >$0.50: ask user before proceeding.
464
+ - Track cumulative spend per research session.
465
+
466
+ ## Troubleshooting
467
+
468
+ - **All tools return empty**: target has minimal digital presence. Try Bright Data Yandex search (better for CIS region), search by company + role instead of name.
469
+ - **Wrong person keeps appearing**: add company name, city, or role to all queries. Use quotes around full name.
470
+ - **LinkedIn blocked**: use `brightdata.sh scrape` as primary instead of Apify.
471
+ - **Apify actor dead/changed**: check `apify.sh store-search "linkedin scraper"` for alternatives. Actors on Apify are volatile — always have a Bright Data fallback.
472
+ - **Depth Score stuck at 6-7**: likely missing press/media articles or internal intel. Search industry publications (AdIndex, Sostav, Forbes, Kommersant for Russian market). Try `jina.sh deepsearch`. Check Telegram history.
473
+ - **No social media found**: person may use pseudonyms. Search by email, phone, or company employee page. Search Apify store: `bash scripts/apify.sh store-search "people search"`. If `mcpc` installed: `APIFY_TOKEN=$APIFY_API_TOKEN mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call search-actors keywords:="people search" limit:=10`. Check Telegram contacts by phone.
474
+ - **TikTok scraper fails**: try `clockworks/free-tiktok-scraper` (free tier) as fallback. TikTok usernames often differ from other platforms — search by real name via `clockworks/tiktok-user-search-scraper`.
475
+ - **Need emails from website**: use `vdrmota/contact-info-scraper` — it crawls the site and extracts all contact info.
476
+ - **Rate limited (429)**: back off 5s, then 15s. Switch to fallback tool. Never retry immediately.
477
+
478
+ ## Anti-Patterns
479
+
480
+ 1. Never start with a single tool. Launch all available in parallel.
481
+ 2. Never retry a failed tool more than once. Switch to fallback.
482
+ 3. Never guess DOB, family, or zodiac.
483
+ 4. Never attribute data without cross-referencing against namesakes.
484
+ 5. Never include unsourced facts.
485
+ 6. Never reveal OSINT methods in public messages.
486
+ 7. Never exceed 3 recursive cycles. Diminishing returns.
487
+ 8. Never rate Depth Score 9+ without justification.
488
+ 9. Never skip psychoprofile. Without it, dossier = Wikipedia article.
489
+ 10. Never skip Phase 1.5 (internal intel). Telegram history is often the richest source.
490
+ 11. Never quote DMs verbatim in shareable outputs. Summarize and cite.
491
+ 12. Never hammer APIs without rate limiting. Stagger requests.