@chainpatrol/cli 0.3.3 → 0.3.4

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # @chainpatrol/cli
 
+## 0.3.4
+
+### Patch Changes
+
+- e30617f: Update the bundled Claude Code skill to document the `detections healthcheck`,
+  `queues snapshot`, `metrics summary | found | breakdown`, and `presets`
+  subcommands, plus the `--no-input` / `--no-color` / `--output` / `--quiet`
+  global flags. The existing soft / manual signals in the Organization
+  HealthCheck Guide are preserved verbatim; each subsection now also points at
+  the matching CLI command via a "Run via CLI" note, and a new "Quick Path"
+  block lists the commands an agent should reach for first when asked to run
+  a health check.
+
 ## 0.3.3
 
 ### Patch Changes

package/dist/{chunk-R7BPW32M.js → chunk-DOL35U2S.js}

@@ -264,6 +264,101 @@ You can also compare with the non-customer set using
 \`--no-reported-by-customer\` to gauge detection coverage on the same time
 window.
 
+### \`detections healthcheck\` \u2014 Validate enabled detection configs produce recent results
+
+Server-side check. The CLI calls the ChainPatrol API; the server fetches each
+enabled detection config for the org, counts results produced in the lookback
+window, and FAILs configs that fall under \`--min-results\` (or whose run
+errored when \`--run\` is set).
+
+\`\`\`bash
+chainpatrol --json detections healthcheck --org <slug>
+\`\`\`
+
+Flags:
+- \`--source <key>\` only validate one source (e.g. \`twitter_search\`)
+- \`--min-results <n>\` minimum results required in the window to pass
+- \`--lookback-hours <n>\` size of the lookback window
+- \`--run\` ask the server to run each config first, then validate the fresh output
+- \`--include-disabled\` also validate disabled configs
+
+What this command covers:
+- Configs that have gone silent (recentResultCount below threshold)
+- Configs that error when run (runOk=false) when \`--run\` is set
+
+What it does NOT cover:
+- Reviewing backlog / SLA breaches \u2192 use \`queues snapshot\`
+- Takedown ToDo / In Progress / Cancelled volumes \u2192 use \`queues snapshot\`
+- Spikes or drops in detection volume over time \u2192 use \`metrics breakdown\`
+- Customer-reported gaps \u2192 use \`reports list --reported-by-customer\`
+- Google Safe Browsing submission errors (not yet exposed in CLI)
+
+Use it as the first signal in the Detection part of an org healthcheck, then
+fall back to the manual checks in the HealthCheck Guide below for everything
+else.
+
+### \`queues snapshot\` \u2014 Operations review/takedown queue snapshot
+
+Server-side aggregation of the operations review queue (pending proposals,
+SLA breaches, age buckets) and the takedown queue (open, in-progress, stale).
+
+\`\`\`bash
+chainpatrol --json queues snapshot --org <slug>
+\`\`\`
+
+Useful for the **Reviewing** and **Takedowns** sections of the HealthCheck
+Guide. Key signals in the response:
+- \`reviewQueue.totalPendingProposals\` and \`reviewQueue.distinctReports\` \u2014
+  backlog size
+- \`reviewQueue.slaBuckets.breached\` \u2014 SLA breaches (treat any breach as a
+  finding)
+- \`reviewQueue.ageBuckets.gte168h\` \u2014 proposals older than 7 days (anything
+  >14 days from the manual guide should always be in here)
+- \`takedownQueue.totalOpen\` and \`takedownQueue.staleInProgress\` \u2014 open
+  and stuck takedowns
+
+Use \`--all\` to snapshot every org you have access to instead of a single slug.
+
+### \`metrics summary | found | breakdown\` \u2014 Org metrics for spike/drop analysis
+
+\`\`\`bash
+chainpatrol --json metrics summary  --org <slug> --this-week
+chainpatrol --json metrics breakdown --org <slug> --by day --this-week
+chainpatrol --json metrics found    --org <slug> --from <YYYY-MM-DD> --to <YYYY-MM-DD>
+\`\`\`
+
+\`breakdown\` is the one you usually want for healthchecks: it returns a
+time series (by day or week) of reports, new threats, watchlisted threats,
+and takedowns filed/completed. Compare the latest period against a prior
+window to spot the **spike** or **drop** signals described in the manual
+HealthCheck Guide. \`summary\` returns a single window total; \`found\` is
+oriented around when threats were first discovered.
+
+### \`presets list | run\` \u2014 Packaged workflows for common jobs
+
+\`\`\`bash
+chainpatrol presets list
+chainpatrol presets run cs-weekly-health --org <slug>
+\`\`\`
+
+Use \`presets list\` to discover packaged multi-step workflows. The bundled
+\`cs-weekly-health\` preset runs the standard customer-success weekly health
+sweep; prefer it over hand-rolling the same sequence of commands.
+
+## Headless / agent-mode tips
+
+When running these commands from an agent or CI:
+
+- Prefer \`--json\` (or \`--output json\`) so output is structured and the
+  update-check stderr nudge is suppressed automatically.
+- Pass \`--no-input\` to forbid any interactive prompt (the CLI will error
+  out instead of waiting on a TTY).
+- Pass \`--no-color\` or set \`NO_COLOR=1\` if your sink can't render ANSI.
+- Set \`CHAINPATROL_NO_UPDATE_CHECK=1\` to silence the skill/npm freshness
+  nudge entirely.
+- For \`login\` specifically, see the headless runbook in the login section
+  above \u2014 login is the one command that needs background + tail handling.
+
 ## Checking Auth Status
 
 To check if the user is logged in, read the credentials file:
@@ -303,12 +398,16 @@ machine-readable output is never polluted.
 
 ## Global Flags
 
-| Flag        | Description                          |
-|-------------|--------------------------------------|
-| \`--json\`    | Machine-readable JSON output         |
-| \`--org\`     | Organization slug (saved for later)  |
-| \`--help\`    | Show help                            |
-| \`--version\` | Show version                         |
+| Flag             | Description                                            |
+|------------------|--------------------------------------------------------|
+| \`--json\`         | Machine-readable JSON output (shortcut for \`--output json\`) |
+| \`--output <fmt>\` | Output format: \`human\` (default), \`json\`, \`markdown\`, \`csv\` |
+| \`--quiet\`, \`-q\`  | Suppress non-essential output and the update-check nudge |
+| \`--no-color\`     | Disable ANSI colors (also respects the \`NO_COLOR\` env var) |
+| \`--no-input\`     | Disable interactive prompts (use in scripts and agents) |
+| \`--org <slug>\`   | Organization slug (saved as the default for later commands) |
+| \`--help\`, \`-h\`   | Show help                                              |
+| \`--version\`      | Show version                                           |
 
 ## Workflow
 
@@ -336,7 +435,85 @@ When the user asks for a "health check", "audit", "what's wrong with org X",
 "review org X's setup", or similar, walk through each section below and surface
 findings.
 
-In each section there are things you can look for that may be wrong.
+In each section there are things you can look for that may be wrong. Some
+checks have a dedicated CLI command that does most of the work server-side;
+others are soft / qualitative signals that still need you to fetch data with
+\`configs list\` or \`reports list\` and reason about it manually.
+
+### Quick Path: CLI commands that automate parts of this guide
+
+Run these first to get cheap, structured signals before falling back to the
+manual checks in each subsection:
+
+\`\`\`bash
+# Detection: configs that have gone silent or are erroring
+chainpatrol --json detections healthcheck --org <slug>
+
+# Reviewing & Takedowns: backlog, SLA breaches, stuck takedowns
+chainpatrol --json queues snapshot --org <slug>
+
+# Spikes / drops in detection volume over time (compare windows)
+chainpatrol --json metrics breakdown --org <slug> --by day --this-week
+
+# Customer-reported threats \u2014 gaps in our own detection
+chainpatrol --json reports list --org <slug> --reported-by-customer
+
+# What's enabled vs disabled vs not configured for the org
+chainpatrol --json configs list --org <slug>
+
+# Packaged weekly customer-success sweep (preferred when it covers the ask)
+chainpatrol presets run cs-weekly-health --org <slug>
+\`\`\`
+
+Treat each command's output as one input to the healthcheck. The manual
+checks below still apply \u2014 especially for signals the CLI cannot infer on
+its own (e.g. "lots of Twitter assets are blocked but Twitter Post Search
+is disabled", or "this drop is fine because the config isn't relevant
+to this org").
+
+### Reporting progress while running a healthcheck
+
+When the user asks for a healthcheck, narrate each step in real time so they
+can watch the run unfold. Do NOT batch results and dump everything at the
+end. For every check you run (Quick Path CLI command, or a manual signal
+where you're fetching data with \`configs list\` / \`reports list\` to
+reason about):
+
+1. **Before** you call the command, emit ONE short sentence stating what
+   you're about to check, including the org slug. Examples:
+   - "Running detection config healthcheck for morpho\u2026"
+   - "Snapshotting review and takedown queues for morpho\u2026"
+   - "Fetching customer-reported reports for morpho to look for detection gaps\u2026"
+
+2. **As soon as** the command returns, emit ONE short result line that
+   starts with a status word and ends with a key number or finding:
+   - \`DONE\` \u2014 ran cleanly, nothing to flag
+   - \`WARN\` \u2014 soft signal worth surfacing (e.g. a borderline backlog,
+     mild spike or drop, a config you'd want a human to confirm)
+   - \`FAIL\` \u2014 concrete failure that the user should act on
+   - Examples:
+     - "DONE \u2014 14/14 detection configs passing."
+     - "WARN \u2014 23 proposals in the review queue; 4 are older than 7 days."
+     - "FAIL \u2014 twitter_post_search returned 0 results in the last 24h with --run."
+
+3. Run **independent** checks in **parallel** (single message, multiple
+   Bash tool calls) so progress lines arrive quickly. \`detections
+   healthcheck\`, \`queues snapshot\`, \`metrics breakdown\`,
+   \`reports list --reported-by-customer\`, and \`configs list\` are all
+   independent of each other and safe to fire concurrently. Dependent
+   follow-ups (e.g. paginating \`reports list\` with a returned cursor,
+   or fetching extra detail on a single failing config) run after the
+   first round.
+
+4. After every check is reported, emit a short final **Summary** section:
+   - A one-line status per check (\u2713 / \u26A0 / \u2717 + name + key number).
+   - A short "Top issues" list of the highest-priority FAIL / WARN
+     findings, in priority order, with the concrete next action for each.
+
+Keep each progress line to one sentence. The goal is for the user to see
+the healthcheck happening, not to read a wall of text mid-run \u2014 full
+detail belongs in the final Summary or in a follow-up when the user asks
+about a specific finding.
 
 ### Detection
 
@@ -347,18 +524,37 @@ turned off. For example: lots of Twitter assets are on the blocklist, but
 detection sources like "Twitter / X User Search" or "Twitter Post Search" are
 disabled. Those should be turned on.
 
+**Run via CLI:** there is no single command for this \u2014 it's a manual
+correlation. Fetch the org's enabled vs disabled configs with
+\`chainpatrol --json configs list --org <slug>\`, then look at recent reports
+(\`chainpatrol --json reports list --org <slug>\`) and the asset types of any
+blocked items. Flag any asset type where blocked items exist but the matching
+detection source has \`status: "disabled"\` (or appears in the "Not configured"
+group).
+
 #### Spike in Detections
 
 A spike in recent detections is worth investigating. It could be a bad config
 change, or it could be a legitimate new attack push in this area \u2014 useful
 intel to surface to the security team as a targeted spike.
 
+**Run via CLI:** \`chainpatrol --json metrics breakdown --org <slug> --by day --this-week\`
+(and a comparison window via \`--from\`/\`--to\`) to see daily detection
+volume. Anything notably above the recent baseline is a spike \u2014 cross-reference
+against recent config changes for that source.
+
 #### Drop in Detections
 
 A drop is also worth looking into. It may be a bad config change, or it may
 mean the config is not really relevant and can be safely turned off (not all
 default-on configs are relevant to every org).
 
+**Run via CLI:** the same \`metrics breakdown\` time series catches drops.
+Additionally, \`chainpatrol --json detections healthcheck --org <slug>\`
+fails configs whose \`recentResultCount\` is below \`--min-results\` in the
+\`--lookback-hours\` window \u2014 that's exactly a "this source went silent"
+signal. Use \`--run\` to also catch configs that error when executed.
+
 ### Reviewing
 
 #### Pile Up / Backlog of Unreviewed Proposals
@@ -368,6 +564,13 @@ reports, but really the threshold is relative to the average number of
 confirmed threats per week. Example: if an org only adds 5 blocked threats per
 week, then a 7-day backlog of even 10 proposals is a really big deal.
 
+**Run via CLI:** \`chainpatrol --json queues snapshot --org <slug>\` returns
+\`reviewQueue.totalPendingProposals\` and \`reviewQueue.distinctReports\`.
+Compare against the org's typical weekly throughput (use
+\`metrics summary --this-week\` for that baseline) \u2014 a backlog that exceeds
+a week of typical confirmed-threat volume is a finding regardless of the
+absolute number.
+
 #### Really Old Proposals
 
 Any proposal waiting for review longer than 14 days is a sign something has
@@ -375,6 +578,12 @@ gone wrong. Even complex investigations rarely take longer than this. Except
 for rare cases, these should be rejected or approved to prevent a backlog
 from building.
 
+**Run via CLI:** \`queues snapshot\` returns \`reviewQueue.ageBuckets.gte168h\`
+(proposals older than 7 days). Anything in that bucket is at least halfway
+to the 14-day threshold; investigate. \`reviewQueue.slaBuckets.breached\`
+captures the strictest SLA breaches separately \u2014 any non-zero value is
+worth raising.
+
 #### Spike in Auto Approved Reports
 
 If suddenly a lot of proposals in an org are being approved by automation,
@@ -385,6 +594,12 @@ cases, notify an engineer at ChainPatrol and check any detection configs you
 adjusted recently, since those may be the cause of spam combined with a weak
 rule.
 
+**Run via CLI:** there is no dedicated subcommand for the auto-approval
+rate yet. As a proxy, use \`chainpatrol --json metrics breakdown --org <slug> --by day --this-week\`
+and look for a sudden surge in \`newThreats\` / \`threatsWatchlisted\` that
+isn't matched by a parallel rise in reviewer activity \u2014 that gap usually
+points at automation doing the approving.
+
 ### Blocklisting
 
 #### Google Safe Browsing (Coming Soon)
@@ -398,6 +613,10 @@ also take a look at the org's custom detection sources \u2014 it's possible ther
 are too many false positives landing on the blocklist, indicating issues with
 detection and reviewing rules.
 
+**Run via CLI:** not yet \u2014 the CLI does not expose Google Safe Browsing
+submission state today. Until the new public API lands, this remains a
+manual / engineering-team check.
+
 ### Takedowns
 
 #### Too Many Takedowns in ToDo
@@ -406,12 +625,22 @@ Can mean a gap in automated takedowns not being implemented for some new area
 of threats. It can also mean the areas that require manual takedowns are
 being missed by the takedown team.
 
+**Run via CLI:** \`chainpatrol --json queues snapshot --org <slug>\` returns
+\`takedownQueue.totalOpen\` and breakdowns by status. A persistently large
+ToDo bucket relative to the org's typical takedown rate (use
+\`metrics summary\` for that baseline) is the finding.
+
 #### Too Many Takedowns In Progress
 
 Typically means something is wrong with the submission itself. The takedown
 may need to be resubmitted, the vendor asked for more evidence, or we may
 have submitted it in the wrong place.
 
+**Run via CLI:** \`queues snapshot\` returns \`takedownQueue.staleInProgress\`
+\u2014 takedowns that have sat in In Progress past the expected vendor turnaround.
+Any non-zero value is worth investigating; a growing number across snapshots
+strongly suggests a vendor-side or submission-format problem.
+
 #### Too Many Cancelled Takedowns
 
 Takedowns should rarely be cancelled. A cancelled takedown means "we will not
@@ -419,11 +648,21 @@ do this takedown" for some reason. Cases like adding an item to the blocklist
 when it's already taken down are treated as completed, not cancelled. So even
 3 cancelled takedowns in a 7-day period is too many.
 
+**Run via CLI:** no first-class command yet. As an interim signal, use
+\`chainpatrol --json metrics breakdown --org <slug> --by day --this-week\`
+and compare \`takedownsFiled\` vs \`takedownsCompleted\` for a sudden
+divergence \u2014 a widening gap with no In-Progress growth often shows up as
+cancellations.
+
 #### Automated Takedowns Turned Off for Over 30 Days
 
 Automated takedowns should be on by default for nearly every organization.
 Any issue that would make you want to turn off automated takedowns should be
 resolved within 30 days.
+
+**Run via CLI:** not yet exposed in a single command. Check this manually
+with the org's takedown automation settings; the CLI's role here is mostly
+to highlight stale state in \`queues snapshot\` so you know to ask.
 `;
 }
 function getBundledSkillVersion() {

package/dist/cli.js

@@ -13,7 +13,7 @@ import {
   getCliVersion,
   isSkillInstalled,
   readInstalledSkillVersion
-} from "./chunk-R7BPW32M.js";
+} from "./chunk-DOL35U2S.js";
 import "./chunk-IUZB3DQW.js";
 import {
   DateTime
@@ -1032,12 +1032,12 @@ async function main() {
     case "setup":
     case "install":
     case "i": {
-      const { setupSkill } = await import("./setup-skill-KKWETU4A.js");
+      const { setupSkill } = await import("./setup-skill-PCUBJJYU.js");
       setupSkill({ json: jsonMode });
       break;
     }
     case "uninstall": {
-      const { uninstallSkill } = await import("./setup-skill-KKWETU4A.js");
+      const { uninstallSkill } = await import("./setup-skill-PCUBJJYU.js");
       uninstallSkill({ json: jsonMode });
       break;
     }

package/dist/{setup-skill-KKWETU4A.js → setup-skill-PCUBJJYU.js}

@@ -6,7 +6,7 @@ import {
   readInstalledSkillVersion,
   setupSkill,
   uninstallSkill
-} from "./chunk-R7BPW32M.js";
+} from "./chunk-DOL35U2S.js";
 import "./chunk-IUZB3DQW.js";
 export {
   getBundledSkillContent,

package/package.json

@@ -2,7 +2,7 @@
   "name": "@chainpatrol/cli",
   "description": "The official ChainPatrol CLI — terminal interface for threat detection",
   "author": "Umar Ahmed <umar@chainpatrol.io>",
-  "version": "0.3.3",
+  "version": "0.3.4",
   "license": "UNLICENSED",
   "homepage": "https://chainpatrol.com/docs/cli",
   "keywords": [