threatwatch 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. threatwatch-1.0.0/CHANGELOG.md +46 -0
  2. threatwatch-1.0.0/LICENSE +21 -0
  3. threatwatch-1.0.0/MANIFEST.in +6 -0
  4. threatwatch-1.0.0/PKG-INFO +496 -0
  5. threatwatch-1.0.0/README.md +473 -0
  6. threatwatch-1.0.0/SECURITY.md +77 -0
  7. threatwatch-1.0.0/config.example.json +211 -0
  8. threatwatch-1.0.0/pyproject.toml +37 -0
  9. threatwatch-1.0.0/setup.cfg +4 -0
  10. threatwatch-1.0.0/tests/test_core.py +534 -0
  11. threatwatch-1.0.0/threatwatch/__init__.py +8 -0
  12. threatwatch-1.0.0/threatwatch/__main__.py +6 -0
  13. threatwatch-1.0.0/threatwatch/cli.py +592 -0
  14. threatwatch-1.0.0/threatwatch/config.py +228 -0
  15. threatwatch-1.0.0/threatwatch/correlate.py +478 -0
  16. threatwatch-1.0.0/threatwatch/extract.py +160 -0
  17. threatwatch-1.0.0/threatwatch/http.py +352 -0
  18. threatwatch-1.0.0/threatwatch/models.py +181 -0
  19. threatwatch-1.0.0/threatwatch/opsec.py +718 -0
  20. threatwatch-1.0.0/threatwatch/progress.py +323 -0
  21. threatwatch-1.0.0/threatwatch/report.py +971 -0
  22. threatwatch-1.0.0/threatwatch/scoring.py +264 -0
  23. threatwatch-1.0.0/threatwatch/sources.py +633 -0
  24. threatwatch-1.0.0/threatwatch/state.py +99 -0
  25. threatwatch-1.0.0/threatwatch/summarize.py +386 -0
  26. threatwatch-1.0.0/threatwatch/taxonomy.py +591 -0
  27. threatwatch-1.0.0/threatwatch/utils.py +193 -0
  28. threatwatch-1.0.0/threatwatch.egg-info/PKG-INFO +496 -0
  29. threatwatch-1.0.0/threatwatch.egg-info/SOURCES.txt +30 -0
  30. threatwatch-1.0.0/threatwatch.egg-info/dependency_links.txt +1 -0
  31. threatwatch-1.0.0/threatwatch.egg-info/entry_points.txt +2 -0
  32. threatwatch-1.0.0/threatwatch.egg-info/top_level.txt +1 -0
@@ -0,0 +1,46 @@
1
+ # Changelog
2
+
3
+ All notable changes to ThreatWatch are documented here.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [1.0.0] - 2026-11-14
9
+
10
+ Initial public release.
11
+
12
+ ### Added
13
+
14
+ - Live collection from CISA KEV, NVD 2.0, FIRST EPSS, GitHub Security Advisories,
15
+ and curated RSS/Atom feeds.
16
+ - Defensive-only scope: no exploit code, scanning, credential use, or offensive
17
+ capability. Extracted indicators are defanged and treated as unvalidated.
18
+ - Explainable additive risk scoring with CVSS, EPSS, KEV, ransomware, recency,
19
+ AI relevance, and asset exposure. KEV floors and tier-based remediation SLAs.
20
+ - Audience profiles for baseline, CISO, SOC, incident response, and AI platform
21
+ teams.
22
+ - Markdown, single-file HTML, JSON, and CSV reports with KPI dashboards, priority
23
+ actions, vulnerability detail, KEV watch, AI/LLM threat mapping, campaign
24
+ intelligence, OPSEC controls, detection priorities, and IOC appendices.
25
+ - AI/LLM taxonomy mapped to OWASP Top 10 for LLM Applications (2025) and MITRE
26
+ ATLAS.
27
+ - "Recently Patched / Fix Available" view with NVD patch references, explicit
28
+ fix language, and GitHub fixed-version metadata. Includes `--patched-only`
29
+ filtering and patch fields in JSON/CSV exports.
30
+ - Live weighted progress bar for interactive terminals, with plain milestone
31
+ output for cron/CI, `--quiet`, `--no-progress`, and `--no-color` controls.
32
+ - Persistent HTTP cache with offline fallback, retry/backoff, and polite rate
33
+ limiting for unauthenticated NVD and GitHub API access.
34
+ - Watch state and `--diff-only` reporting for new items since the previous run.
35
+ - CI gating with `--fail-on p1|p2|p3` and strict source-health checks.
36
+ - Optional LLM narrative polish with configurable endpoint and privacy controls.
37
+ - `enrich` command for single-CVE deep dives, and `opsec` command for a defensive
38
+ control baseline.
39
+ - Standard-library-only implementation for Python 3.11+.
40
+
41
+ ### Security
42
+
43
+ - API keys and tokens are read from environment variables and redacted by
44
+ `config show`.
45
+ - Generated reports may contain sensitive threat intelligence; the tool includes
46
+ classification markings and defanged indicators to support safe handling.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 ghoste12125
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,6 @@
1
+ include README.md
2
+ include CHANGELOG.md
3
+ include SECURITY.md
4
+ include LICENSE
5
+ include config.example.json
6
+ recursive-include tests *.py
@@ -0,0 +1,496 @@
1
+ Metadata-Version: 2.4
2
+ Name: threatwatch
3
+ Version: 1.0.0
4
+ Summary: Live defensive cyber and AI threat intelligence reports from public sources
5
+ Author-email: ghoste12125 <anonymosmauros@gmail.com>
6
+ Maintainer-email: ghoste12125 <anonymosmauros@gmail.com>
7
+ License-Expression: MIT
8
+ Keywords: threat-intelligence,cybersecurity,ai-security,kev,epss,opsec
9
+ Classifier: Environment :: Console
10
+ Classifier: Intended Audience :: Information Technology
11
+ Classifier: Intended Audience :: System Administrators
12
+ Classifier: Operating System :: OS Independent
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.11
15
+ Classifier: Programming Language :: Python :: 3.12
16
+ Classifier: Programming Language :: Python :: 3.13
17
+ Classifier: Programming Language :: Python :: 3.14
18
+ Classifier: Topic :: Security
19
+ Requires-Python: >=3.11
20
+ Description-Content-Type: text/markdown
21
+ License-File: LICENSE
22
+ Dynamic: license-file
23
+
24
+ # ThreatWatch
25
+
26
+ Live, defensive cyber and AI threat intelligence from public sources, rendered as a professional report for executives, SOC, incident response, and AI platform owners.
27
+
28
+ ThreatWatch collects exploitation evidence and reporting, scores what matters, explains every score, and produces Markdown, single-file HTML, JSON, and CSV. It is designed for repeatable use in a terminal, cron job, systemd timer, or CI pipeline.
29
+
30
+ > **Scope:** defensive intelligence only. ThreatWatch does not include exploit code, scanning, credential use, or offensive capability. All indicators are defanged and treated as unvalidated public reporting.
31
+
32
+ ---
33
+
34
+ ## Contents
35
+
36
+ - [Quickstart](#quickstart)
37
+ - [Commands](#commands)
38
+ - [Live progress](#live-progress)
39
+ - [Profiles](#profiles)
40
+ - [Report sections](#report-sections)
41
+ - [Recently patched / fix available](#recently-patched--fix-available)
42
+ - [How prioritisation works](#how-prioritisation-works)
43
+ - [AI and LLM taxonomy](#ai-and-llm-taxonomy)
44
+ - [Sources](#sources)
45
+ - [Configuration](#configuration)
46
+ - [CI gating](#ci-gating)
47
+ - [Scheduling](#scheduling)
48
+ - [LLM polish and privacy](#llm-polish-and-privacy)
49
+ - [Caching, state, and offline use](#caching-state-and-offline-use)
50
+ - [Limitations and responsible use](#limitations-and-responsible-use)
51
+ - [Security](#security)
52
+ - [Changelog](#changelog)
53
+ - [Development](#development)
54
+
55
+ ---
56
+
57
+ ## Quickstart
58
+
59
+ Requirements: Python **3.11+** (standard library only, no third-party packages).
60
+
61
+ ```bash
62
+ # Install the command from PyPI.
63
+ python3 -m pip install threatwatch
64
+
65
+ # Optional: create a local config and edit assets, thresholds, and org details.
66
+ threatwatch config init
67
+
68
+ # Collect, score, and write the default Markdown report.
69
+ threatwatch run
70
+
71
+ # Or write every format at once.
72
+ threatwatch run --format all --outdir reports
73
+
74
+ # See what was produced.
75
+ ls reports/
76
+ ```
77
+
78
+ Reports are written as `threatwatch-YYYYMMDD-HHMMSSZ.<ext>` plus `latest.<ext>` unless `report.write_latest` is disabled.
79
+
80
+ From a source checkout, use `python3 -m threatwatch` instead of `threatwatch`, or install the checkout in editable mode:
81
+
82
+ ```bash
83
+ cd threatwatch
84
+ python3 -m pip install -e .
85
+ threatwatch run
86
+ ```
87
+
88
+ No network access is required for `--offline` runs after a cache has been populated.
89
+
90
+ ---
91
+
92
+ ## Commands
93
+
94
+ | Command | Purpose |
95
+ |---|---|
96
+ | `run` (alias `report`) | Collect, correlate, score, and write a full report. |
97
+ | `watch` | Report only items new since the previous run; exits non-zero on new P1/P2 by default. |
98
+ | `feeds` | List configured feeds or check feed/API health. |
99
+ | `opsec` | Print the defensive OPSEC control baseline in Markdown, JSON, or CSV. |
100
+ | `enrich CVE-YYYY-NNNN` | Deep-dive one CVE with KEV, EPSS, AI mapping, and remediation context. |
101
+ | `config init\|show\|validate` | Create, inspect, or validate configuration. |
102
+ | `version` | Print the version. |
103
+
104
+ Running `python3 -m threatwatch` with no arguments is equivalent to `run`.
105
+
106
+ Common examples:
107
+
108
+ ```bash
109
+ # All formats, named output directory.
110
+ python3 -m threatwatch run --format all --outdir reports
111
+
112
+ # Executive view for the last 14 days.
113
+ python3 -m threatwatch run --profile ciso --since-days 14 --format md --format html
114
+
115
+ # Only AI/LLM-relevant items.
116
+ python3 -m threatwatch run --ai-only --profile ai-platform
117
+
118
+ # Only vulnerabilities with patch or fixed-version evidence.
119
+ python3 -m threatwatch run --patched-only --format md --format html
120
+
121
+ # New items only (the watch command does this automatically).
122
+ python3 -m threatwatch run --diff-only --format json --output -
123
+
124
+ # Single CVE deep dive.
125
+ python3 -m threatwatch enrich CVE-2024-3400
126
+
127
+ # OPSEC controls for a SOC.
128
+ python3 -m threatwatch opsec --profile soc --format md --output opsec-soc.md
129
+
130
+ # Check sources before relying on a scheduled run.
131
+ python3 -m threatwatch feeds --check-apis
132
+ ```
133
+
134
+ Useful run flags:
135
+
136
+ | Flag | Effect |
137
+ |---|---|
138
+ | `--format md,html,json,csv,all` | Output formats; repeatable. Default `md`. |
139
+ | `--outdir DIR` | Output directory. Default `reports`. |
140
+ | `--output FILE` / `--output -` | Single file, or stdout. |
141
+ | `--since-days N` | CVE window. |
142
+ | `--kev-days N` | KEV addition window. |
143
+ | `--news-hours N` | Reporting window. |
144
+ | `--top N` | Rows in vulnerability tables. |
145
+ | `--min-tier P1..P4` | Show this tier or higher. |
146
+ | `--ai-only` | Keep only AI/LLM-relevant vulnerabilities and reporting. |
147
+ | `--patched-only` | Keep only vulnerabilities with detected patch or fixed-version evidence. |
148
+ | `--diff-only` | Only items new since the previous run. |
149
+ | `--all-kev` | Include the full KEV catalog, not just recent/relevant entries. |
150
+ | `--no-feeds`, `--no-nvd`, `--no-ghsa` | Skip a collection source. |
151
+ | `--profile baseline\|ciso\|soc\|ir\|ai-platform` | Audience and OPSEC profile. |
152
+ | `--fail-on p1\|p2\|p3\|none` | CI gate; exits `2` when the threshold is met. |
153
+ | `--no-state`, `--state-file PATH` | Disable or relocate watch state. |
154
+ | `--offline`, `--refresh`, `--no-cache` | Cache behaviour. |
155
+ | `--org`, `--classification` | Override report metadata. |
156
+ | `--llm` / `--no-llm` | Opt in/out of configured LLM polish. |
157
+ | `--print` | Also print the report to stdout. |
158
+ | `--verbose` | Show fetch retries and diagnostics. |
159
+ | `--quiet` | Suppress all progress and status output (ideal for cron/CI logs). |
160
+ | `--no-progress` | Disable the live progress bar but keep normal status lines. |
161
+ | `--no-color` | Disable ANSI colour. `NO_COLOR=1` is also honoured. |
162
+
163
+ Exit codes: `0` success, `1` usage/runtime error, `2` CI gate or strict source-health failure.
164
+
165
+ ### Live progress
166
+
167
+ On an interactive terminal, collection shows a single animated bar with per-stage and sub-stage updates: KEV catalog, NVD pages, GitHub advisories, news feeds, EPSS batches, and scoring. The bar is drawn on `stderr`, so report output (`--output -`) stays clean.
168
+
169
+ When output is redirected (cron, CI, logs), ThreatWatch automatically falls back to plain stage and milestone lines instead of ANSI animation. Use `--quiet` to silence it entirely, or `--no-progress` to keep status messages but skip the bar.
170
+
171
+ ```bash
172
+ # Interactive: animated bar.
173
+ python3 -m threatwatch run --format all
174
+
175
+ # CI: plain, timestamp-friendly lines; no ANSI codes.
176
+ NO_COLOR=1 python3 -m threatwatch watch
177
+
178
+ # Cron: no progress at all.
179
+ python3 -m threatwatch watch --quiet
180
+ ```
181
+
182
+ ---
183
+
184
+ ## Profiles
185
+
186
+ Profiles change emphasis, narrative, and which OPSEC controls are included. They do not change the underlying scoring.
187
+
188
+ | Profile | Audience and emphasis |
189
+ |---|---|
190
+ | `baseline` | Balanced technical and management view. |
191
+ | `ciso` | Exposure, deadlines, accountability, and decision-ready summaries. |
192
+ | `soc` | Detection, KEV/EPSS triage, hunting leads, and operational queues. |
193
+ | `ir` | Exposure verification, evidence preservation, and containment for confirmed exploitation. |
194
+ | `ai-platform` | Model serving, agent permissions, retrieval boundaries, and artifact integrity. |
195
+
196
+ OPSEC controls are tagged `Baseline`, `Enhanced`, or `Advanced`. Profiles select the relevant subset; `--advanced` / `--no-advanced` override inclusion.
197
+
198
+ ---
199
+
200
+ ## Report sections
201
+
202
+ - Posture KPIs and bottom line
203
+ - Executive summary and key findings
204
+ - Priority actions with owners, tiers, and due dates
205
+ - Vulnerability priorities with explainable risk drivers
206
+ - Recently patched / fix available, sorted by most recent source modification
207
+ - CISA KEV watch, including ransomware and asset matches
208
+ - AI/LLM threat landscape mapped to OWASP and MITRE ATLAS
209
+ - Exploit and campaign intelligence
210
+ - OPSEC controls, quick wins, assurance questions, and metrics
211
+ - Detection priorities
212
+ - Defanged indicator appendix
213
+ - SLA and metrics table
214
+ - Methodology, confidence, and limitations
215
+ - Source health and collection warnings
216
+ - New-since-last-run delta when state is enabled
217
+
218
+ The HTML export is a single self-contained file with inline CSS/JS, filtering, a theme toggle, and print/PDF support.
219
+
220
+ ### Recently patched / fix available
221
+
222
+ ThreatWatch infers patch evidence conservatively from public metadata:
223
+
224
+ - NVD references tagged `Patch`.
225
+ - Explicit fix language in NVD descriptions, such as `fixed in`, `patched in`, `security update`, or `upgrade to`.
226
+ - GitHub Security Advisory `first_patched_version` and `patched_versions` fields.
227
+
228
+ The **Recently Patched / Fix Available** section shows these records sorted by the most recent source modification date (NVD `lastModified` or GHSA `updated_at`), with the fixed version, tier, risk, and KEV status. Use `--patched-only` to restrict the main vulnerability table and AI sections to the same records. CSV and JSON exports include `patch_available`, `patched_versions`, and `patch_urls`.
229
+
230
+ Patch detection is a metadata heuristic, not confirmation that a safe update exists for your deployment. Verify with the vendor advisory before relying on it.
231
+
232
+ ---
233
+
234
+ ## How prioritisation works
235
+
236
+ Every vulnerability gets an additive, explainable risk score (0–100):
237
+
238
+ ```text
239
+ risk = clamp(3.0 × CVSS
240
+ + 20 × EPSS
241
+ + 22 × KEV
242
+ + 6 × ransomware
243
+ + 8 × recency
244
+ + 12 × AI relevance
245
+ + 8 × asset exposure)
246
+ ```
247
+
248
+ - **Recency** decays with a 21-day half-life.
249
+ - **KEV floors** ensure confirmed exploitation is never buried: `72` baseline, `82` with known ransomware, `86` with strong AI relevance, `88` when a declared asset matches.
250
+ - **Tiers** come from configurable thresholds: P1 ≥ 80, P2 ≥ 62, P3 ≥ 42, otherwise P4.
251
+ - **SLA hours** are P1 24h, P2 72h, P3 14 days, P4 30 days, capped by the CISA KEV due date where one exists.
252
+ - **Explanations** are stored on each record and shown in tables, cards, JSON, and `enrich`.
253
+
254
+ Asset matching is keyword-based against the `assets` list in configuration. It is intentionally simple and should be tuned to your real inventory.
255
+
256
+ ---
257
+
258
+ ## AI and LLM taxonomy
259
+
260
+ AI relevance requires strong indicators, not generic words such as “AI”, “model”, or “agent”. Categories map to:
261
+
262
+ - **OWASP Top 10 for LLM Applications (2025)** — e.g. LLM01 Prompt Injection, LLM03 Supply Chain, LLM06 Excessive Agency, LLM07 System Prompt Leakage.
263
+ - **MITRE ATLAS** — verified technique IDs such as AML.T0051 (LLM Prompt Injection), AML.T0010 (AI Supply Chain Compromise), AML.T0040 (AI Model Inference API Access), and AML.T0110 (AI Agent Tool Poisoning).
264
+
265
+ The classifier combines indicator phrases, context guards (for example, avoiding “RAG” false positives), and AI-stack inventory terms.
266
+
267
+ ---
268
+
269
+ ## Sources
270
+
271
+ | Source | Role |
272
+ |---|---|
273
+ | CISA Known Exploited Vulnerabilities (KEV) | Authoritative confirmed exploitation and due dates. |
274
+ | NIST NVD 2.0 | CVSS, CWE, CPE, descriptions, references. |
275
+ | FIRST EPSS | 30-day exploitation probability and percentile. |
276
+ | GitHub Security Advisories | Open-source package advisories and GHSA identifiers. |
277
+ | Curated RSS/Atom feeds | Vendor research, incident reporting, and AI security commentary. |
278
+
279
+ `python3 -m threatwatch feeds --list` shows built-in feed IDs. Add your own under `feeds.custom`. The default enabled set favours established security publishers and AI-security sources; tune it to your sector.
280
+
281
+ NVD unauthenticated access is rate-limited. Set `NVD_API_KEY` to raise limits. Set `GITHUB_TOKEN` for higher GitHub API limits. Both are optional.
282
+
283
+ ---
284
+
285
+ ## Configuration
286
+
287
+ Configuration is JSON, merged over built-in defaults. Discovery order:
288
+
289
+ 1. `--config PATH`
290
+ 2. `THREATWATCH_CONFIG` environment variable
291
+ 3. `.threatwatch/config.json`
292
+ 4. `config.json` in the current directory
293
+
294
+ Start from the documented example:
295
+
296
+ ```bash
297
+ python3 -m threatwatch config init # writes .threatwatch/config.json
298
+ python3 -m threatwatch config show # effective config, secrets redacted
299
+ python3 -m threatwatch config validate # warnings and feed checks
300
+ ```
301
+
302
+ `config.example.json` in this repository mirrors the default configuration and can be copied manually.
303
+
304
+ Key sections:
305
+
306
+ | Section | Purpose |
307
+ |---|---|
308
+ | `org` | Name, sector, classification marking, timezone. |
309
+ | `assets` | Keyword-based inventory with criticality and exposure. |
310
+ | `ai_stack` | Providers, frameworks, servers, vector DBs, and AI infrastructure terms. |
311
+ | `feeds` | Enabled built-in feeds, custom feeds, per-feed item cap. |
312
+ | `windows` | Default CVE, KEV, and news windows. |
313
+ | `thresholds`, `sla_hours` | Tier boundaries and remediation SLAs. |
314
+ | `report` | Top-N, patched-section cap, IOC/OPSEC inclusion, theme, profile, latest-file behaviour. |
315
+ | `cache`, `state` | Cache TTLs, cache age, and watch-state location. |
316
+ | `nvd`, `epss`, `ghsa`, `http` | Rate limits, tokens, retries, timeout, user agent. |
317
+ | `llm` | Optional local/remote LLM endpoint and privacy controls. |
318
+ | `opsec` | Advanced-control inclusion and default profile. |
319
+
320
+ ---
321
+
322
+ ## CI gating
323
+
324
+ Use `--fail-on` to turn intelligence into a release or deployment gate. A gate failure exits `2`.
325
+
326
+ ```bash
327
+ # Fail if any P1 is present.
328
+ python3 -m threatwatch run --format json --output threatwatch.json --fail-on p1
329
+
330
+ # Watch mode defaults to failing on newly seen P1 or P2 items.
331
+ python3 -m threatwatch watch --format md --outdir reports
332
+ ```
333
+
334
+ Example GitHub Actions step:
335
+
336
+ ```yaml
337
+ - name: Threat intelligence gate
338
+ env:
339
+ NVD_API_KEY: ${{ secrets.NVD_API_KEY }}
340
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
341
+ run: |
342
+ python -m pip install .
343
+ threatwatch run --format json --output threatwatch.json --fail-on p1
344
+ ```
345
+
346
+ Recommended pattern: run `watch` on a schedule, archive JSON as an artifact, and keep the hard gate on P1. Avoid failing builds on P3/P4 noise unless your risk appetite justifies it.
347
+
348
+ ---
349
+
350
+ ## Scheduling
351
+
352
+ Cron example (every four hours, new items only):
353
+
354
+ ```cron
355
+ 0 */4 * * * cd /opt/threatwatch && /usr/bin/python3 -m threatwatch watch --quiet --outdir /var/lib/threatwatch/reports >> /var/log/threatwatch.log 2>&1
356
+ ```
357
+
358
+ A systemd timer is preferable for jitter, logging, and restart behaviour:
359
+
360
+ ```ini
361
+ # /etc/systemd/system/threatwatch.service
362
+ [Unit]
363
+ Description=ThreatWatch intelligence report
364
+ After=network-online.target
365
+
366
+ [Service]
367
+ Type=oneshot
368
+ WorkingDirectory=/opt/threatwatch
369
+ Environment=NVD_API_KEY=...
370
+ ExecStart=/usr/bin/python3 -m threatwatch watch --quiet --outdir /var/lib/threatwatch/reports
371
+ User=threatwatch
372
+ ```
373
+
374
+ ```ini
375
+ # /etc/systemd/system/threatwatch.timer
376
+ [Unit]
377
+ Description=Run ThreatWatch every four hours
378
+
379
+ [Timer]
380
+ OnCalendar=00/4:00
381
+ RandomizedDelaySec=10m
382
+ Persistent=true
383
+
384
+ [Install]
385
+ WantedBy=timers.target
386
+ ```
387
+
388
+ ```bash
389
+ sudo systemctl daemon-reload
390
+ sudo systemctl enable --now threatwatch.timer
391
+ ```
392
+
393
+ For a daily full report plus a frequent watch, use two units with different `--outdir` values.
394
+
395
+ ---
396
+
397
+ ## LLM polish and privacy
398
+
399
+ LLM polish is **off by default**. When enabled with `--llm`, ThreatWatch sends **aggregate metrics only** — counts, tiers, category names, and short risk summaries — not raw reporting text, indicators, or asset details. Organisation name can be redacted with `llm.redact_org`.
400
+
401
+ The default endpoint is local Ollama (`http://127.0.0.1:11434`). If you point it at a remote provider, review data-handling and residency requirements first. The report always has a deterministic fallback, so LLM failure does not block generation.
402
+
403
+ For sensitive environments, leave LLM polish disabled and use the deterministic narrative.
404
+
405
+ ---
406
+
407
+ ## Caching, state, and offline use
408
+
409
+ - HTTP responses are cached under `.threatwatch/cache` with per-source TTLs.
410
+ - `--refresh` bypasses TTLs; `--no-cache` disables reads and writes; `--offline` uses cache only.
411
+ - For time-windowed APIs (NVD, KEV, EPSS, GHSA), offline mode falls back to the most recent cached response for the same endpoint and query shape, and labels it stale rather than dropping the source.
412
+ - Stale cache is labelled in Source Health rather than silently presented as current.
413
+ - Watch state lives in `.threatwatch/state.json`. `--diff-only` / `watch` report new items; `--no-state` leaves state untouched.
414
+ - Failed fetches are retried with rate limiting and recorded as collection warnings.
415
+
416
+ The cache is not a substitute for a controlled archive. For audit purposes, retain generated JSON reports alongside their timestamps.
417
+
418
+ ---
419
+
420
+ ## Limitations and responsible use
421
+
422
+ - Public reporting can be incomplete, delayed, duplicated, or wrong. Validate before emergency change.
423
+ - KEV is authoritative for known exploitation; EPSS is probabilistic; NVD enrichment can lag.
424
+ - Asset matching is keyword-based and may miss or over-match. Tune `assets` and review results.
425
+ - AI tagging is a triage aid, not proof of exploitability or attribution. Attribution is not assessed.
426
+ - Indicators are automatically extracted, defanged, and unvalidated. Do not block on a single low-confidence indicator.
427
+ - The tool does not replace vendor advisories, EDR telemetry, vulnerability scanners, or incident response.
428
+ - Reports may contain security-sensitive content. Apply the classification marking, restrict distribution, and defang before external sharing.
429
+
430
+ ThreatWatch is intended for defensive security teams operating within their authorisation and applicable law.
431
+
432
+ ---
433
+
434
+ ## Security
435
+
436
+ Report suspected vulnerabilities privately to **anonymosmauros@gmail.com** with the subject `[SECURITY] ThreatWatch — <short summary>`. Do not open a public issue before coordinated disclosure.
437
+
438
+ Supported release: **1.0.x**. The full policy, scope, and reporting guidance are in `SECURITY.md`.
439
+
440
+ ThreatWatch is defensive-only, reads API credentials from environment variables, sends no telemetry, and redacts configured secrets in `config show`. Generated reports may still contain sensitive threat intelligence; store and share them according to your classification policy.
441
+
442
+ ---
443
+
444
+ ## Changelog
445
+
446
+ Release notes are maintained in `CHANGELOG.md`. This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
447
+
448
+ ---
449
+
450
+ ## Development
451
+
452
+ ```bash
453
+ cd threatwatch
454
+
455
+ # Offline test suite (stdlib unittest).
456
+ python3 -m unittest discover -s tests -v
457
+
458
+ # Byte-compile all modules.
459
+ python3 -m compileall -q threatwatch
460
+
461
+ # Live smoke test without GitHub advisories.
462
+ python3 -m threatwatch run --no-ghsa --since-days 3 --format md --format html --format json --outdir reports
463
+ ```
464
+
465
+ Project layout:
466
+
467
+ ```text
468
+ threatwatch/
469
+ ├── pyproject.toml
470
+ ├── MANIFEST.in
471
+ ├── config.example.json
472
+ ├── README.md
473
+ ├── CHANGELOG.md
474
+ ├── SECURITY.md
475
+ ├── LICENSE
476
+ ├── tests/
477
+ │ └── test_core.py
478
+ └── threatwatch/
479
+ ├── __main__.py
480
+ ├── cli.py # commands, argument wiring, output
481
+ ├── config.py # defaults, loading, validation
482
+ ├── models.py # data structures and serialisation
483
+ ├── taxonomy.py # OWASP LLM Top 10 + MITRE ATLAS mapping
484
+ ├── scoring.py # risk model, tiers, SLAs, asset matching
485
+ ├── sources.py # KEV, NVD, EPSS, GHSA, RSS/Atom collectors
486
+ ├── http.py # caching, retries, rate limiting
487
+ ├── extract.py # conservative IOC extraction
488
+ ├── state.py # watch state and diffs
489
+ ├── correlate.py # collection orchestration and KPIs
490
+ ├── summarize.py # deterministic narrative + optional LLM polish
491
+ ├── opsec.py # defensive control library
492
+ ├── report.py # Markdown, HTML, JSON, CSV renderers
493
+ └── utils.py # shared formatting helpers
494
+ ```
495
+
496
+ Contributions should preserve three properties: standard library only, defensive scope only, and explainable scoring.