scamscript 0.4.1__tar.gz → 0.6.0__tar.gz

{scamscript-0.4.1 → scamscript-0.6.0}/CHANGELOG.md

@@ -5,6 +5,33 @@ schema are the things that count as public API.
 
 ## Unreleased
 
+## 0.6.0 - 2026-11-02
+
+### Added
+
+- Closed-set attack type slugs, a per-type count and mean score table, and `--type` filtering with an
+  unknown-token warning. Raw descriptions are preserved.
+
+### Changed
+
+- Panels show the slug beneath the attack type.
+
+### Security
+
+- Log directories are created `0700` and log files `0600`, so captures are not world-readable.
+
+## 0.5.0 - 2026-11-02
+
+### Added
+
+- Content fingerprints and duplicate folding: reprints collapse to the strongest copy in reports,
+  with a folded count and `duplicate_of` in JSON.
+- `--dedupe/--no-dedupe` (on by default) and `--show-duplicates` to render every panel.
+
+### Changed
+
+- Duplicate panels are replaced by a one-line pointer unless `--show-duplicates` is set.
+
 ## 0.4.1 - 2026-11-02
 
 ### Fixed

{scamscript-0.4.1 → scamscript-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: scamscript
-Version: 0.4.1
+Version: 0.6.0
 Summary: Reverse-engineer scam scripts and vishing transcripts into structured counter-playbooks
 Project-URL: Homepage, https://github.com/sjdsjadjajdjsa/scamscript
 Project-URL: Repository, https://github.com/sjdsjadjajdjsa/scamscript
@@ -133,7 +133,8 @@ corpus is at [`docs/sample-report.md`](https://github.com/sjdsjadjajdjsa/scamscr
 | `prune` | Delete old logs |
 
 Useful flags: `--dry-run`, `--json`, `--redact`, `--top N`, `--recursive`, `--timeout`,
-`--retries`, `--max-tokens`, `--concurrency`, `--max-cost`, `--no-color`. `--json` puts one object per script on stdout and all
+`--retries`, `--max-tokens`, `--concurrency`, `--max-cost`, `--dedupe/--no-dedupe`,
+`--show-duplicates`, `--type`, `--no-color`. `--json` puts one object per script on stdout and all
 Rich output on stderr, so it pipes:
 
 ```bash
@@ -154,6 +155,33 @@ The cap reads `usage.cost` from the response, so it works only where the provide
 says so when none is sent. Spend is printed and logged per run. In-flight requests can overshoot by
 up to the concurrency, so keep headroom on a hard limit.
 
+## Attack types
+
+`attack_type` is free prose, which is useful to read and useless to group. Every result also carries a
+closed-set slug (`vishing`, `smishing`, `tech-support`, `refund`, `gov-impersonation`, `bec`,
+`romance`, `crypto`, `investment`, `utility`, `delivery`, `other`). Reports show counts and mean scores
+per slug above the ranking, and `--type` filters to the ones you ask for:
+
+```bash
+scamscript batch corpus/ --type tech-support,refund --output report.md
+```
+
+The raw phrase is always kept next to the slug. An unrecognised description lands in `other` and the
+CLI names any filter token it did not recognise.
+
+## Duplicates
+
+Corpora are full of reprints. Each script is fingerprinted on its body: headers, casing, punctuation
+and whitespace are ignored, so copies collapse. The strongest copy is ranked and the rest are listed
+under it; the report counts what was folded and JSON records carry `fingerprint` and `duplicate_of`.
+
+```bash
+scamscript batch corpus/ --output report.md          # folds by default
+scamscript batch corpus/ --no-dedupe --show-duplicates
+```
+
+Only byte-level twins fold. A copy with one changed account number stays separate, by design.
+
 ## How it works
 
 ```text
@@ -190,7 +218,7 @@ masking is best effort: names, case IDs and spelled-out digits survive it.
 Every request appends to a timestamped JSONL file under `~/.scamscript/logs/` (override with
 `--home` or `SCAMSCRIPT_HOME`). The API key is never written. Logs are the source for `report`, so
 a run replays without spending tokens, and `prune --keep 200 --keep-days 30` keeps the store
-bounded.
+bounded. Directories are `0700` and log files `0600`.
 
 ## Install and configure
 
@@ -262,7 +290,8 @@ docs/           calibration record, sample report, banner, demo
 - Breakdowns are model readings; a bad prompt or a weak model produces confident nonsense. Read
   `opening_hook` and `red_flags` against the source before quoting anything.
 - Redaction is regex-based and will miss names and identifiers without a fixed shape.
-- One script per request; no cross-script deduplication yet, so reprints inflate a ranking.
+- Duplicate folding ignores formatting only; a single changed digit keeps a copy separate.
+- Type slugs are keyword-matched, so an unusual description falls to `other` rather than inventing one.
 - The cost cap depends on provider-reported usage and can overshoot by the in-flight count.
 - Analysis is English-centric: the prompt and schema assume English artifacts.
 - Quote checking compares text, not meaning. A legitimate paraphrase left in quotes is flagged;

{scamscript-0.4.1 → scamscript-0.6.0}/README.md

@@ -97,7 +97,8 @@ corpus is at [`docs/sample-report.md`](https://github.com/sjdsjadjajdjsa/scamscr
 | `prune` | Delete old logs |
 
 Useful flags: `--dry-run`, `--json`, `--redact`, `--top N`, `--recursive`, `--timeout`,
-`--retries`, `--max-tokens`, `--concurrency`, `--max-cost`, `--no-color`. `--json` puts one object per script on stdout and all
+`--retries`, `--max-tokens`, `--concurrency`, `--max-cost`, `--dedupe/--no-dedupe`,
+`--show-duplicates`, `--type`, `--no-color`. `--json` puts one object per script on stdout and all
 Rich output on stderr, so it pipes:
 
 ```bash
@@ -118,6 +119,33 @@ The cap reads `usage.cost` from the response, so it works only where the provide
 says so when none is sent. Spend is printed and logged per run. In-flight requests can overshoot by
 up to the concurrency, so keep headroom on a hard limit.
 
+## Attack types
+
+`attack_type` is free prose, which is useful to read and useless to group. Every result also carries a
+closed-set slug (`vishing`, `smishing`, `tech-support`, `refund`, `gov-impersonation`, `bec`,
+`romance`, `crypto`, `investment`, `utility`, `delivery`, `other`). Reports show counts and mean scores
+per slug above the ranking, and `--type` filters to the ones you ask for:
+
+```bash
+scamscript batch corpus/ --type tech-support,refund --output report.md
+```
+
+The raw phrase is always kept next to the slug. An unrecognised description lands in `other` and the
+CLI names any filter token it did not recognise.
+
+## Duplicates
+
+Corpora are full of reprints. Each script is fingerprinted on its body: headers, casing, punctuation
+and whitespace are ignored, so copies collapse. The strongest copy is ranked and the rest are listed
+under it; the report counts what was folded and JSON records carry `fingerprint` and `duplicate_of`.
+
+```bash
+scamscript batch corpus/ --output report.md          # folds by default
+scamscript batch corpus/ --no-dedupe --show-duplicates
+```
+
+Only byte-level twins fold. A copy with one changed account number stays separate, by design.
+
 ## How it works
 
 ```text
@@ -154,7 +182,7 @@ masking is best effort: names, case IDs and spelled-out digits survive it.
 Every request appends to a timestamped JSONL file under `~/.scamscript/logs/` (override with
 `--home` or `SCAMSCRIPT_HOME`). The API key is never written. Logs are the source for `report`, so
 a run replays without spending tokens, and `prune --keep 200 --keep-days 30` keeps the store
-bounded.
+bounded. Directories are `0700` and log files `0600`.
 
 ## Install and configure
 
@@ -226,7 +254,8 @@ docs/           calibration record, sample report, banner, demo
 - Breakdowns are model readings; a bad prompt or a weak model produces confident nonsense. Read
   `opening_hook` and `red_flags` against the source before quoting anything.
 - Redaction is regex-based and will miss names and identifiers without a fixed shape.
-- One script per request; no cross-script deduplication yet, so reprints inflate a ranking.
+- Duplicate folding ignores formatting only; a single changed digit keeps a copy separate.
+- Type slugs are keyword-matched, so an unusual description falls to `other` rather than inventing one.
 - The cost cap depends on provider-reported usage and can overshoot by the in-flight count.
 - Analysis is English-centric: the prompt and schema assume English artifacts.
 - Quote checking compares text, not meaning. A legitimate paraphrase left in quotes is flagged;

{scamscript-0.4.1 → scamscript-0.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "scamscript"
-version = "0.4.1"
+version = "0.6.0"
 description = "Reverse-engineer scam scripts and vishing transcripts into structured counter-playbooks"
 readme = "README.md"
 requires-python = ">=3.12"

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/__init__.py

@@ -1,3 +1,3 @@
 """scamscript: reverse-engineering analyzer for scam call scripts."""
 
-__version__ = "0.4.1"
+__version__ = "0.6.0"

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/analysis.py

@@ -7,7 +7,7 @@ from dataclasses import dataclass
 from pathlib import Path
 from typing import Any
 
-from scamscript import prompts, redact, verify
+from scamscript import dedupe, prompts, redact, taxonomy, verify
 from scamscript.client import ChatClient, ClientError, TruncatedError
 from scamscript.ingest import ScriptFile
 from scamscript.models import BreakdownError, ScriptBreakdown, parse_breakdown
@@ -27,6 +27,9 @@ class Outcome:
     duration: float = 0.0
     redactions: dict[str, int] | None = None
     skipped: bool = False
+    fingerprint: str = ""
+    attack_slug: str = ""
+    duplicate_of: str | None = None
     issues: list[verify.Issue] | None = None
     verification: dict[str, int] | None = None
 
@@ -41,6 +44,9 @@ class Outcome:
             "name": self.name,
             "ok": self.ok,
             "skipped": self.skipped,
+            "fingerprint": self.fingerprint,
+            "attack_slug": self.attack_slug,
+            "duplicate_of": self.duplicate_of,
             "attempts": self.attempts,
             "duration_s": round(self.duration, 3),
             "breakdown": self.breakdown.model_dump() if self.breakdown else None,
@@ -63,7 +69,9 @@ def analyze_script(
     redact_identifiers: bool = False,
     verify_quotes: bool = True,
 ) -> Outcome:
-    outcome = Outcome(source=script.path, name=script.name)
+    outcome = Outcome(
+        source=script.path, name=script.name, fingerprint=dedupe.fingerprint(script.text)
+    )
     started = time.monotonic()
 
     text = script.text
@@ -113,6 +121,7 @@ def analyze_script(
         try:
             outcome.breakdown = parse_breakdown(visible)
             outcome.error = None
+            outcome.attack_slug = taxonomy.slugify(outcome.breakdown.attack_type)
             if verify_quotes:
                 outcome.issues = verify.verify_breakdown(outcome.breakdown, text)
                 outcome.verification = verify.summarize(outcome.issues)

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/cli.py

@@ -12,7 +12,7 @@ import typer
 from rich.console import Console
 from rich.table import Table
 
-from scamscript import __version__, prompts, redact, store
+from scamscript import __version__, prompts, redact, store, taxonomy
 from scamscript import report as reporting
 from scamscript.analysis import Outcome, analyze_script
 from scamscript.client import ChatClient
@@ -52,6 +52,8 @@ class Context:
     json_output: bool
     verify_quotes: bool
     strict_quotes: bool
+    dedupe: bool
+    show_duplicates: bool
     console: Console
     err_console: Console
 
@@ -135,6 +137,16 @@ def main(
         "--strict-quotes",
         help="Exit non-zero when a quoted claim is not found in the source.",
     ),
+    dedupe: bool = typer.Option(
+        True,
+        "--dedupe/--no-dedupe",
+        help="Fold identical scripts into the strongest copy in reports.",
+    ),
+    show_duplicates: bool = typer.Option(
+        False,
+        "--show-duplicates",
+        help="Render a full panel for duplicate copies too.",
+    ),
     dry_run: bool = typer.Option(
         False,
         "--dry-run",
@@ -173,6 +185,8 @@ def main(
         json_output=json_output,
         verify_quotes=verify_quotes,
         strict_quotes=strict_quotes,
+        dedupe=dedupe,
+        show_duplicates=show_duplicates,
         console=Console(
             no_color=no_color,
             highlight=False,
@@ -220,6 +234,7 @@ def _run(
     batch_mode: bool,
     output: Path | None,
     top: int | None,
+    types: list[str] | None = None,
 ) -> None:
     console = state.console
     settings = state.settings
@@ -278,6 +293,7 @@ def _run(
     spent = 0.0
     spent_seen = False
     lock = threading.Lock()
+    seen: dict[str, str] = {}
     client = ChatClient(
         api_key=api_key,
         base_url=settings.base_url,
@@ -308,6 +324,11 @@ def _run(
             redact_identifiers=settings.redact,
             verify_quotes=state.verify_quotes,
         )
+        with lock:
+            if state.dedupe and outcome.fingerprint:
+                outcome.duplicate_of = seen.setdefault(outcome.fingerprint, outcome.name)
+                if outcome.duplicate_of == outcome.name:
+                    outcome.duplicate_of = None
         cost = client.last_cost
         if cost:
             with lock:
@@ -320,6 +341,10 @@ def _run(
             _emit_json(outcome.as_record())
         elif outcome.skipped:
             console.print(f"[dim]skipped {outcome.name}: budget reached[/dim]")
+        elif outcome.duplicate_of and not state.show_duplicates:
+            console.print(
+                f"[dim]{outcome.name}: duplicate of {outcome.duplicate_of}[/dim]"
+            )
         elif outcome.breakdown is not None:
             console.print(
                 render_breakdown(
@@ -388,14 +413,24 @@ def _run(
         cost_usd=round(spent, 6),
     )
 
+    only_types = taxonomy.parse_filter(types)
+    if only_types:
+        unknown = taxonomy.unknown(types)
+        if unknown:
+            console.print(
+                f"[yellow]note[/yellow] unknown type(s): {', '.join(sorted(unknown))}"
+            )
+    collapsed = reporting.collapse(report_entries) if state.dedupe else report_entries
+
     if batch_mode and (successes or skipped):
         if not state.json_output:
-            console.print(
-                render_summary_table(
-                    [(o.name, o.breakdown, o.error, o.skipped) for o in finished]
-                )
+            rows = (
+                [(e.name, e.breakdown, e.error, e.skipped) for e in collapsed]
+                if state.dedupe
+                else [(o.name, o.breakdown, o.error, o.skipped) for o in finished]
             )
-        ranked = reporting.rank(report_entries)
+            console.print(render_summary_table(rows))
+        ranked = reporting.rank(collapsed)
         if top:
             ranked = ranked[:top]
         markdown = reporting.build_markdown(
@@ -403,6 +438,7 @@ def _run(
             run_id=logger.run_id,
             command=command,
             model=settings.model,
+            only_types=only_types,
         )
         written = _emit_report(console, markdown, output, json_mode=state.json_output)
         if state.json_output:
@@ -469,6 +505,9 @@ def analyze(
     ),
     recursive: bool = typer.Option(False, "--recursive", "-r", help="Recurse into subdirectories."),
     top: int | None = typer.Option(None, "--top", min=1, help="Keep only the N most sophisticated scripts."),
+    types: list[str] | None = typer.Option(
+        None, "--type", help="Only report these type slugs (repeatable or comma-separated)."
+    ),
 ) -> None:
     """Analyze a single script file, or a directory with --batch."""
     state = _context(ctx)
@@ -481,7 +520,15 @@ def analyze(
     except IngestError as exc:
         _fail(state.console, str(exc))
         return
-    _run(state, scripts, command="analyze", batch_mode=batch_mode, output=output, top=top)
+    _run(
+        state,
+        scripts,
+        command="analyze",
+        batch_mode=batch_mode,
+        output=output,
+        top=top,
+        types=types,
+    )
 
 
 @app.command()
@@ -493,6 +540,9 @@ def batch(
     ),
     recursive: bool = typer.Option(False, "--recursive", "-r", help="Recurse into subdirectories."),
     top: int | None = typer.Option(None, "--top", min=1, help="Keep only the N most sophisticated scripts."),
+    types: list[str] | None = typer.Option(
+        None, "--type", help="Only report these type slugs (repeatable or comma-separated)."
+    ),
 ) -> None:
     """Analyze a directory and rank every script by sophistication."""
     state = _context(ctx)
@@ -501,7 +551,7 @@ def batch(
     except IngestError as exc:
         _fail(state.console, str(exc))
         return
-    _run(state, scripts, command="batch", batch_mode=True, output=output, top=top)
+    _run(state, scripts, command="batch", batch_mode=True, output=output, top=top, types=types)
 
 
 @app.command()
@@ -512,6 +562,9 @@ def report(  # noqa: A001
     ),
     output: Path | None = typer.Option(None, "--output", "-o", help="Write the markdown here."),
     logs_dir: Path | None = typer.Option(None, "--logs-dir", help="Override the log directory."),
+    types: list[str] | None = typer.Option(
+        None, "--type", help="Only report these type slugs (repeatable or comma-separated)."
+    ),
 ) -> None:
     """Regenerate the aggregated report from a past run's log."""
     state = _context(ctx)
@@ -538,6 +591,7 @@ def report(  # noqa: A001
         run_id=str(run["run_id"]),
         command=str(run.get("command") or "report"),
         model=run.get("model"),
+        only_types=taxonomy.parse_filter(types),
     )
     _emit_report(state.console, markdown, output)
 

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/client.py

@@ -63,7 +63,7 @@ class ChatClient:
                 "Authorization": f"Bearer {api_key}",
                 "Content-Type": "application/json",
                 "Accept": "application/json",
-                "User-Agent": "scamscript/0.4.1",
+                "User-Agent": "scamscript/0.6.0",
             },
         )
 

scamscript-0.6.0/src/scamscript/dedupe.py

@@ -0,0 +1,29 @@
+"""Content fingerprints for collapsing reprinted scripts."""
+
+from __future__ import annotations
+
+import hashlib
+import re
+
+NON_WORD = re.compile(r"[^a-z0-9]+")
+HEADER_LINE = re.compile(r"^\s*#")
+
+
+def body(text: str) -> str:
+    """Drop the leading comment block a captured script usually carries."""
+    lines = text.splitlines()
+    index = 0
+    while index < len(lines) and (not lines[index].strip() or HEADER_LINE.match(lines[index])):
+        index += 1
+    return "\n".join(lines[index:])
+
+
+def normalize(text: str) -> str:
+    lowered = body(text).lower()
+    return " ".join(NON_WORD.sub(" ", lowered).split())
+
+
+def fingerprint(text: str) -> str:
+    """Short hash of the normalized body, stable across whitespace and punctuation."""
+    normalized = normalize(text)
+    return hashlib.sha256(normalized.encode("utf-8")).hexdigest()[:16]

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/render.py

@@ -9,6 +9,7 @@ from rich.table import Table
 from rich.text import Text
 
 from scamscript.ingest import ScriptFile
+from scamscript import taxonomy
 from scamscript.models import ScriptBreakdown
 from scamscript.verify import Issue, summarize
 
@@ -123,6 +124,7 @@ def render_breakdown(
         style=f"bold {border_for(breakdown)}",
     )
     renderables.append(attack)
+    renderables.append(Text(f"slug: {taxonomy.slugify(breakdown.attack_type)}", style="dim"))
     renderables.append(Text(""))
 
     for key in (

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/report.py

@@ -7,6 +7,7 @@ from typing import Any
 
 from pydantic import ValidationError
 
+from scamscript import taxonomy
 from scamscript.models import ScriptBreakdown
 from scamscript.store import results_of, utc_now
 
@@ -30,6 +31,9 @@ class ReportEntry:
     error: str | None = None
     duration: float = 0.0
     skipped: bool = False
+    fingerprint: str = ""
+    attack_slug: str = ""
+    duplicates: list[str] | None = None
     verification: dict[str, int] | None = None
     unverified: list[dict[str, str]] | None = None
 
@@ -44,6 +48,8 @@ def entry_from_outcome(outcome: Any) -> ReportEntry:
         source=str(getattr(outcome, "source", outcome.name)),
         breakdown=outcome.breakdown,
         skipped=getattr(outcome, "skipped", False),
+        fingerprint=getattr(outcome, "fingerprint", ""),
+        attack_slug=getattr(outcome, "attack_slug", ""),
         error=outcome.error,
         duration=getattr(outcome, "duration", 0.0),
         verification=getattr(outcome, "verification", None),
@@ -71,6 +77,9 @@ def entries_from_run(run: dict[str, Any]) -> list[ReportEntry]:
                 source=record.get("source") or "",
                 breakdown=breakdown,
                 skipped=bool(record.get("skipped")),
+                fingerprint=record.get("fingerprint") or "",
+                attack_slug=record.get("attack_slug") or "",
+                duplicates=record.get("duplicates") or None,
                 error=None if breakdown else record.get("error") or "no breakdown stored",
                 duration=float(record.get("duration_s") or 0.0),
                 verification=record.get("verification") or None,
@@ -80,6 +89,29 @@ def entries_from_run(run: dict[str, Any]) -> list[ReportEntry]:
     return entries
 
 
+def collapse(entries: list[ReportEntry]) -> list[ReportEntry]:
+    """Keep the strongest copy of each fingerprint and list the rest under it."""
+    groups: dict[str, list[ReportEntry]] = {}
+    for entry in entries:
+        if entry.ok and entry.fingerprint:
+            groups.setdefault(entry.fingerprint, []).append(entry)
+
+    primaries: dict[str, ReportEntry] = {}
+    for key, group in groups.items():
+        if len(group) > 1:
+            best = max(group, key=lambda item: (item.breakdown.sophistication_score, item.breakdown.flag_count))
+            best.duplicates = [item.name for item in group if item is not best]
+            primaries[key] = best
+        else:
+            primaries[key] = group[0]
+
+    return [
+        primaries[entry.fingerprint] if entry.ok and entry.fingerprint else entry
+        for entry in entries
+        if not (entry.ok and entry.fingerprint and primaries[entry.fingerprint] is not entry)
+    ]
+
+
 def rank(entries: list[ReportEntry]) -> list[ReportEntry]:
     return sorted(
         entries,
@@ -96,7 +128,14 @@ def build_markdown(
     run_id: str,
     command: str = "batch",
     model: str | None = None,
+    only_types: set[str] | None = None,
 ) -> str:
+    if only_types:
+        entries = [
+            entry
+            for entry in entries
+            if not entry.ok or (entry.attack_slug or taxonomy.slugify(entry.breakdown.attack_type)) in only_types
+        ]
     ranked = rank(entries)
     ok = [entry for entry in ranked if entry.ok]
     skipped = [entry for entry in ranked if entry.skipped and not entry.ok]
@@ -118,6 +157,9 @@ def build_markdown(
     lines.append(f"- Failed: {len(failed)}")
     if skipped:
         lines.append(f"- Skipped at budget: {len(skipped)}")
+    folded = sum(len(entry.duplicates or []) for entry in ok)
+    if folded:
+        lines.append(f"- Duplicate copies folded: {folded}")
     unverified_total = sum(len(entry.unverified or []) for entry in ok)
     if unverified_total:
         lines.append(f"- Unverified quotes: {unverified_total}")
@@ -126,15 +168,23 @@ def build_markdown(
     lines.append("")
 
     if ok:
+        lines.append("## Attack types")
+        lines.append("")
+        lines.append("| Type | Scripts | Mean score |")
+        lines.append("| --- | ---: | ---: |")
+        for slug, count, mean in _type_counts(ok):
+            lines.append(f"| {taxonomy.label(slug)} (`{slug}`) | {count} | {mean:.1f} |")
+        lines.append("")
         lines.append("## Ranking by sophistication")
         lines.append("")
-        lines.append("| Rank | Script | Attack type | Sophistication | Red flags | Unverified |")
-        lines.append("| ---: | --- | --- | ---: | ---: | ---: |")
+        lines.append("| Rank | Script | Type | Sophistication | Red flags | Unverified | Copies |")
+        lines.append("| ---: | --- | --- | ---: | ---: | ---: | ---: |")
         for position, entry in enumerate(ok, start=1):
             lines.append(
-                f"| {position} | {_cell(entry.name)} | {_cell(entry.breakdown.attack_type)} | "
+                f"| {position} | {_cell(entry.name)} | "
+                f"`{entry.attack_slug or taxonomy.slugify(entry.breakdown.attack_type)}` | "
                 f"{entry.breakdown.sophistication_score}/10 | {entry.breakdown.flag_count} | "
-                f"{len(entry.unverified or [])} |"
+                f"{len(entry.unverified or [])} | {1 + len(entry.duplicates or [])} |"
             )
         lines.append("")
         for position, entry in enumerate(ok, start=1):
@@ -172,8 +222,13 @@ def _detail_section(position: int, entry: ReportEntry) -> list[str]:
     assert breakdown is not None
     lines = [f"## {position}. {entry.name}", ""]
     lines.append(f"- **Attack type:** {breakdown.attack_type}")
+    lines.append(
+        f"- **Type slug:** `{entry.attack_slug or taxonomy.slugify(breakdown.attack_type)}`"
+    )
     lines.append(f"- **Sophistication:** {breakdown.sophistication_score}/10")
     lines.append(f"- **Red flags observed:** {breakdown.flag_count}")
+    if entry.duplicates:
+        lines.append(f"- **Identical copies folded:** {len(entry.duplicates)} ({', '.join(_cell(name) for name in entry.duplicates)})")
     counts = entry.verification or {}
     if counts.get("checked"):
         lines.append(
@@ -205,5 +260,19 @@ def _detail_section(position: int, entry: ReportEntry) -> list[str]:
     return lines
 
 
+def _type_counts(entries: list[ReportEntry]) -> list[tuple[str, int, float]]:
+    buckets: dict[str, list[int]] = {}
+    for entry in entries:
+        if not entry.breakdown:
+            continue
+        slug = entry.attack_slug or taxonomy.slugify(entry.breakdown.attack_type)
+        buckets.setdefault(slug, []).append(entry.breakdown.sophistication_score)
+    rows = [
+        (slug, len(scores), sum(scores) / len(scores))
+        for slug, scores in buckets.items()
+    ]
+    return sorted(rows, key=lambda row: (-row[1], row[0]))
+
+
 def _cell(value: str) -> str:
     return (value or "").replace("|", "\\|").replace("\n", " ").strip()

{scamscript-0.4.1 → scamscript-0.6.0}/src/scamscript/store.py

@@ -31,14 +31,23 @@ class RunLogger:
         self.command = command
         self.path = self.logs_dir / f"{self.run_id}.jsonl"
         self._started = False
+        self._locked_down = False
+
+    def _prepare(self) -> None:
+        self.logs_dir.mkdir(parents=True, exist_ok=True)
+        for directory in (self.logs_dir.parent, self.logs_dir):
+            _chmod(directory, 0o700)
 
     def _write(self, record: dict[str, Any]) -> None:
         record.setdefault("ts", utc_now())
         record.setdefault("run_id", self.run_id)
         try:
-            self.logs_dir.mkdir(parents=True, exist_ok=True)
+            self._prepare()
             with self.path.open("a", encoding="utf-8") as handle:
                 handle.write(json.dumps(record, ensure_ascii=False) + "\n")
+            if not self._locked_down:
+                self._locked_down = True
+                _chmod(self.path, 0o600)
         except OSError:
             pass
 
@@ -145,9 +154,18 @@ def _mtime(path: Path) -> float:
         return 0.0
 
 
+def _chmod(path: Path, mode: int) -> None:
+    try:
+        os.chmod(path, mode)
+    except OSError:
+        pass
+
+
 def ensure_home(home: str) -> None:
     try:
         os.makedirs(os.path.join(home, "logs"), exist_ok=True)
+        _chmod(Path(home), 0o700)
+        _chmod(Path(home) / "logs", 0o700)
     except OSError:
         pass
 

scamscript-0.6.0/src/scamscript/taxonomy.py

@@ -0,0 +1,77 @@
+"""Map free-text attack descriptions onto a fixed set of slugs."""
+
+from __future__ import annotations
+
+import re
+
+OTHER = "other"
+
+# Checked in order: the earlier, more specific family wins.
+RULES: tuple[tuple[str, tuple[str, ...]], ...] = (
+    ("bec", ("business email", "bec", "invoice fraud", "vendor payment", "mandate fraud", "ceo fraud")),
+    ("tech-support", ("tech support", "tech-support", "remote access", "windows security", "computer support")),
+    ("refund", ("refund", "overpayment", "rebate")),
+    ("gov-impersonation", ("government", "tax", "revenue", "irs", "hmrc", "dvla", "council", "police", "court")),
+    ("utility", ("utility", "water board", "energy", "electric", "gas board", "boiler", "home service")),
+    ("delivery", ("parcel", "delivery", "courier", "postage", "shipping")),
+    ("crypto", ("crypto", "bitcoin", "wallet", "token")),
+    ("investment", ("investment", "trading", "broker", "forex", "pig butchering")),
+    ("romance", ("romance", "dating", "long con", "long-con", "sweetheart")),
+    ("smishing", ("smishing", "sms", "text message", "text-message", "whatsapp")),
+    ("vishing", ("vishing", "phone call", "cold call", "robocall", "phone")),
+)
+
+SLUGS = tuple(slug for slug, _ in RULES) + (OTHER,)
+NON_WORD = re.compile(r"[^a-z0-9]+")
+
+
+def slugify(attack_type: str) -> str:
+    """Return a canonical slug for a model-written attack description."""
+    text = NON_WORD.sub(" ", (attack_type or "").lower())
+    for slug, needles in RULES:
+        if any(needle in text for needle in needles):
+            return slug
+    return OTHER
+
+
+def unknown(values: list[str] | None) -> set[str]:
+    """Raw filter tokens that are not a known slug."""
+    if not values:
+        return set()
+    out: set[str] = set()
+    for value in values:
+        for chunk in value.split(","):
+            token = chunk.strip().lower()
+            if token and token not in SLUGS:
+                out.add(token)
+    return out
+
+
+def label(slug: str) -> str:
+    return {
+        "bec": "Business email compromise",
+        "tech-support": "Tech support",
+        "refund": "Refund",
+        "gov-impersonation": "Government impersonation",
+        "utility": "Utility",
+        "delivery": "Delivery",
+        "crypto": "Crypto",
+        "investment": "Investment",
+        "romance": "Romance",
+        "smishing": "Smishing",
+        "vishing": "Vishing",
+        "other": "Other",
+    }.get(slug, slug)
+
+
+def parse_filter(values: list[str] | None) -> set[str]:
+    """Turn a comma-separated --type option into a slug set."""
+    if not values:
+        return set()
+    out: set[str] = set()
+    for value in values:
+        for chunk in value.split(","):
+            slug = chunk.strip().lower()
+            if slug:
+                out.add(slugify(slug))
+    return out

scamscript-0.6.0/tests/fixtures/reprints/copy-header.txt

@@ -0,0 +1,4 @@
+# third copy, header differs only
+CALLER: Good morning, this is the card security desk.
+CALLER: We have blocked two transactions on your card ending 4417.
+CALLER: Read me the last four digits and we will release the block.

scamscript-0.6.0/tests/fixtures/reprints/copy-punctuation.txt

@@ -0,0 +1,11 @@
+# second copy, reformatted
+
+# captured 2026-10-02, first copy
+
+Caller — Good morning, this is the card security desk.
+
+Caller — We have blocked two transactions on your card ending 4417!
+
+Caller — Read me the last four digits and we will release the block.
+
+

scamscript-0.6.0/tests/fixtures/reprints/near-twin.txt

@@ -0,0 +1,5 @@
+# near twin: one extra sentence, different card digits
+CALLER: Good morning, this is the card security desk.
+CALLER: We have blocked two transactions on your card ending 9902.
+CALLER: Read me the last four digits and we will release the block.
+CALLER: If we do not clear it now, the block stays until you visit a branch.

scamscript-0.6.0/tests/fixtures/reprints/original.txt

@@ -0,0 +1,4 @@
+# captured 2026-10-02, first copy
+CALLER: Good morning, this is the card security desk.
+CALLER: We have blocked two transactions on your card ending 4417.
+CALLER: Read me the last four digits and we will release the block.

{scamscript-0.4.1 → scamscript-0.6.0}/tests/test_cli.py

@@ -1,4 +1,6 @@
 import json
+import re
+from pathlib import Path
 
 import pytest
 from typer.testing import CliRunner
@@ -228,7 +230,7 @@ def test_dry_run_without_endpoint(home, tmp_path, monkeypatch):
 def test_version(home):
     result = runner.invoke(cli.app, ["--version"])
     assert result.exit_code == 0
-    assert result.stdout.strip() == "scamscript 0.4.1"
+    assert result.stdout.strip() == "scamscript 0.6.0"
 
 
 def test_json_output_is_one_object_per_script(home, tmp_path, fake):
@@ -409,3 +411,70 @@ def test_max_cost_warns_when_provider_reports_nothing(home, tmp_path, fake):
     )
     assert result.exit_code == 0
     assert "could not be enforced" in result.stdout
+
+
+def _reprints(tmp_path):
+    src = Path(__file__).parent / "fixtures" / "reprints"
+    for item in src.iterdir():
+        (tmp_path / item.name).write_text(item.read_text(), encoding="utf-8")
+
+
+def test_dedupe_folds_reprints(home, tmp_path, fake):
+    _reprints(tmp_path)
+    scores = {"original": 4, "copy-punctuation": 9, "copy-header": 6, "near-twin": 8}
+    for key, score in scores.items():
+        fake.replies[key] = breakdown(score=score)
+    out = tmp_path / "r.md"
+    result = runner.invoke(cli.app, ["--api-key", "k", "batch", str(tmp_path), "--output", str(out)])
+    assert result.exit_code == 0
+    markdown = out.read_text()
+    assert "Duplicate copies folded: 2" in markdown
+    assert len(re.findall(r"^## \d+\. ", markdown, re.M)) == 2
+    assert "| 1 | copy-punctuation.txt |" in markdown
+    assert "near-twin.txt" in markdown
+    assert "copy-header.txt" in markdown
+
+
+def test_no_dedupe_keeps_every_copy(home, tmp_path, fake):
+    _reprints(tmp_path)
+    out = tmp_path / "r.md"
+    result = runner.invoke(
+        cli.app, ["--api-key", "k", "--no-dedupe", "batch", str(tmp_path), "--output", str(out)]
+    )
+    assert result.exit_code == 0
+    text = out.read_text()
+    assert "Duplicate copies folded" not in text
+    assert len(re.findall(r"^\| \d+ \|", text, re.M)) == 4
+
+
+def test_json_marks_duplicates(home, tmp_path, fake):
+    _reprints(tmp_path)
+    result = runner.invoke(cli.app, ["--api-key", "k", "--json", "batch", str(tmp_path)])
+    assert result.exit_code == 0
+    records = [json.loads(line) for line in result.stdout.splitlines() if line.strip()]
+    results = [record for record in records if record.get("event") == "result"]
+    assert all(record["fingerprint"] for record in results)
+    assert sum(1 for record in results if record["duplicate_of"]) == 2
+
+
+def test_type_filter_limits_report(home, tmp_path, fake):
+    (tmp_path / "a.txt").write_text("CALLER: vish", encoding="utf-8")
+    (tmp_path / "b.txt").write_text("CALLER: romance", encoding="utf-8")
+    fake.replies = {"vish": breakdown(attack="vishing"), "romance": breakdown(attack="romance scam")}
+    out = tmp_path / "r.md"
+    result = runner.invoke(
+        cli.app, ["--api-key", "k", "batch", str(tmp_path), "--type", "vishing", "--output", str(out)]
+    )
+    assert result.exit_code == 0
+    text = out.read_text()
+    assert "a.txt" in text
+    assert "b.txt" not in text
+
+
+def test_unknown_type_warns(home, tmp_path, fake):
+    (tmp_path / "a.txt").write_text("CALLER: x", encoding="utf-8")
+    result = runner.invoke(
+        cli.app, ["--api-key", "k", "batch", str(tmp_path), "--type", "wibble"]
+    )
+    assert result.exit_code == 0
+    assert "unknown type" in result.stdout

scamscript-0.6.0/tests/test_dedupe.py

@@ -0,0 +1,60 @@
+from pathlib import Path
+
+from scamscript.dedupe import fingerprint, normalize
+from scamscript.report import ReportEntry, collapse
+
+FIXTURES = Path(__file__).parent / "fixtures" / "reprints"
+
+
+def read(name):
+    return (FIXTURES / name).read_text()
+
+
+def test_header_and_punctuation_ignored():
+    assert fingerprint(read("original.txt")) == fingerprint(read("copy-punctuation.txt"))
+    assert fingerprint(read("original.txt")) == fingerprint(read("copy-header.txt"))
+
+
+def test_near_twin_differs():
+    assert fingerprint(read("original.txt")) != fingerprint(read("near-twin.txt"))
+
+
+def test_normalize_collapses_whitespace():
+    assert normalize("a  b\n\tc") == "a b c"
+
+
+def test_empty_and_header_only_stable():
+    assert fingerprint("") == fingerprint("# only a header\n")
+
+
+def entry(name, score, fp="abc"):
+    from scamscript.models import ScriptBreakdown
+
+    breakdown = ScriptBreakdown(
+        attack_type="vishing",
+        persona="p",
+        opening_hook="h",
+        trust_building_moves=["a"],
+        urgency_tactics=["b"],
+        escalation_ladder=["c"],
+        target_demographics="d",
+        red_flags=["e"],
+        defense_summary="f",
+        sophistication_score=score,
+    )
+    return ReportEntry(name=name, source=name, breakdown=breakdown, fingerprint=fp)
+
+
+def test_collapse_keeps_strongest_and_lists_rest():
+    entries = [entry("a", 4), entry("b", 9), entry("c", 6)]
+    collapsed = collapse(entries)
+    assert len(collapsed) == 1
+    assert collapsed[0].name == "b"
+    assert collapsed[0].duplicates == ["a", "c"]
+
+
+def test_collapse_leaves_distinct_and_failed_alone():
+    entries = [entry("a", 5), entry("twin", 8, fp="zzz"), ReportEntry(name="bad", source="bad")]
+    collapsed = collapse(entries)
+    assert [item.name for item in collapsed] == ["a", "twin", "bad"]
+    assert collapsed[0].duplicates is None

{scamscript-0.4.1 → scamscript-0.6.0}/tests/test_report.py

@@ -31,10 +31,11 @@ def test_markdown_has_summary_and_details():
     ]
     markdown = build_markdown(entries, run_id="run-1", command="batch", model="m")
     assert "# Scam script breakdown report" in markdown
-    assert "| 1 | a.txt | vishing | 9/10 | 1 |" in markdown
+    assert "| 1 | a.txt | `vishing` | 9/10 | 1 | 0 | 1 |" in markdown
     assert "## Failures" in markdown
     assert "bad json" in markdown
     assert "### Defense summary" in markdown
+    assert "## Attack types" in markdown
     assert "security research and fraud-awareness" in markdown
 
 
@@ -77,3 +78,15 @@ def test_report_shows_unverified_quotes():
     assert "**Quote check:** 1/2 verbatim, 1 missing" in markdown
     assert "### Unverified quotes" in markdown
     assert "_missing_ [red flags] \"invented\"" in markdown
+
+
+def test_type_filter_keeps_only_allowed_and_failures():
+    entries = [
+        ReportEntry(name="a.txt", source="a", breakdown=breakdown(9, attack="vishing"), attack_slug="vishing"),
+        ReportEntry(name="b.txt", source="b", breakdown=breakdown(5, attack="romance scam"), attack_slug="romance"),
+        ReportEntry(name="bad.txt", source="bad", error="boom"),
+    ]
+    markdown = build_markdown(entries, run_id="r", only_types={"vishing"})
+    assert "a.txt" in markdown
+    assert "b.txt" not in markdown
+    assert "bad.txt" in markdown

{scamscript-0.4.1 → scamscript-0.6.0}/tests/test_store.py

@@ -38,3 +38,13 @@ def test_prune_by_age(tmp_path):
 
 def test_prune_missing_dir(tmp_path):
     assert prune(str(tmp_path / "ghost")) == (0, 0, 0)
+
+
+def test_log_dir_and_file_are_private(tmp_path):
+    from scamscript.store import RunLogger
+
+    log = RunLogger(str(tmp_path / "home" / "logs"), "analyze")
+    log.run_start(model="m")
+    assert oct((tmp_path / "home").stat().st_mode)[-3:] == "700"
+    assert oct((tmp_path / "home" / "logs").stat().st_mode)[-3:] == "700"
+    assert oct(log.path.stat().st_mode)[-3:] == "600"

scamscript-0.6.0/tests/test_taxonomy.py

@@ -0,0 +1,51 @@
+import pytest
+
+from scamscript.taxonomy import SLUGS, label, parse_filter, slugify
+
+
+@pytest.mark.parametrize(
+    "text,slug",
+    [
+        ("government impersonation vishing (tax/revenue)", "gov-impersonation"),
+        ("vishing (automated robocall payment scam)", "vishing"),
+        ("romance scam / smishing long con", "romance"),
+        ("parcel delivery fee scam", "delivery"),
+        ("tech support scam with remote access", "tech-support"),
+        ("business email compromise by phone", "bec"),
+        ("crypto investment", "crypto"),
+        ("pig butchering investment scam", "investment"),
+        ("smishing text message", "smishing"),
+        ("utility impersonation", "utility"),
+        ("something entirely new", "other"),
+        ("", "other"),
+    ],
+)
+def test_mapping(text, slug):
+    assert slugify(text) == slug
+
+
+def test_bec_wins_over_vishing():
+    assert slugify("vishing-led wire fraud, business email compromise") == "bec"
+
+
+def test_case_and_punctuation_insensitive():
+    assert slugify("TECH-SUPPORT, remote access!") == "tech-support"
+
+
+def test_slugs_are_closed_set():
+    assert "other" in SLUGS and "vishing" in SLUGS
+    assert slugify("nonsense") in SLUGS
+
+
+def test_parse_filter_accepts_comma_and_spaces():
+    assert parse_filter(["tech support,romance", "vishing"]) == {
+        "tech-support",
+        "romance",
+        "vishing",
+    }
+    assert parse_filter(None) == set()
+
+
+def test_label_known_and_fallback():
+    assert label("bec") == "Business email compromise"
+    assert label("odd") == "odd"