mishkan-harness 0.2.0 → 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (113) hide show
  1. package/README.md +108 -172
  2. package/bin/mishkan.js +677 -16
  3. package/docs/design/MISHKAN_decisions.md +450 -8
  4. package/docs/design/MISHKAN_harness_design.md +1 -33
  5. package/docs/design/MISHKAN_observability.md +13 -9
  6. package/docs/research/graphify-token-saving-poc.md +108 -0
  7. package/docs/usage/09-workflows.md +74 -4
  8. package/docs/usage/10-observability.md +37 -13
  9. package/docs/usage/11-graphify.md +264 -0
  10. package/docs/usage/12-skill-discovery.md +294 -0
  11. package/docs/usage/README.md +5 -3
  12. package/package.json +1 -1
  13. package/payload/install/settings.hooks.json +24 -0
  14. package/payload/mishkan/AGENT_SPEC.md +25 -7
  15. package/payload/mishkan/agents/ahikam.md +4 -2
  16. package/payload/mishkan/agents/aholiab.md +4 -2
  17. package/payload/mishkan/agents/asaph.md +5 -2
  18. package/payload/mishkan/agents/baruch.md +19 -2
  19. package/payload/mishkan/agents/benaiah.md +4 -2
  20. package/payload/mishkan/agents/bezalel.md +4 -2
  21. package/payload/mishkan/agents/caleb.md +4 -2
  22. package/payload/mishkan/agents/deborah.md +4 -2
  23. package/payload/mishkan/agents/elasah.md +4 -2
  24. package/payload/mishkan/agents/eliashib.md +4 -2
  25. package/payload/mishkan/agents/ezra.md +4 -2
  26. package/payload/mishkan/agents/hanun.md +5 -2
  27. package/payload/mishkan/agents/hiram.md +5 -2
  28. package/payload/mishkan/agents/hizkiah.md +5 -2
  29. package/payload/mishkan/agents/huldah.md +4 -2
  30. package/payload/mishkan/agents/huram.md +4 -2
  31. package/payload/mishkan/agents/hushai.md +5 -2
  32. package/payload/mishkan/agents/igal.md +4 -2
  33. package/payload/mishkan/agents/ira.md +5 -2
  34. package/payload/mishkan/agents/jahaziel.md +5 -2
  35. package/payload/mishkan/agents/jakin.md +4 -2
  36. package/payload/mishkan/agents/jehonathan.md +5 -2
  37. package/payload/mishkan/agents/jehoshaphat.md +4 -2
  38. package/payload/mishkan/agents/joab.md +5 -2
  39. package/payload/mishkan/agents/joah.md +5 -2
  40. package/payload/mishkan/agents/maaseiah.md +4 -2
  41. package/payload/mishkan/agents/meremoth.md +5 -2
  42. package/payload/mishkan/agents/meshullam.md +5 -2
  43. package/payload/mishkan/agents/nathan.md +5 -2
  44. package/payload/mishkan/agents/nehemiah.md +4 -2
  45. package/payload/mishkan/agents/obed.md +4 -2
  46. package/payload/mishkan/agents/oholiab.md +5 -2
  47. package/payload/mishkan/agents/palal.md +5 -2
  48. package/payload/mishkan/agents/phinehas.md +4 -2
  49. package/payload/mishkan/agents/rehum.md +4 -2
  50. package/payload/mishkan/agents/salma.md +5 -2
  51. package/payload/mishkan/agents/seraiah.md +4 -2
  52. package/payload/mishkan/agents/shallum.md +5 -2
  53. package/payload/mishkan/agents/shaphan.md +4 -2
  54. package/payload/mishkan/agents/shemaiah.md +4 -2
  55. package/payload/mishkan/agents/shevna.md +5 -2
  56. package/payload/mishkan/agents/uriah.md +5 -2
  57. package/payload/mishkan/agents/zaccur.md +4 -2
  58. package/payload/mishkan/agents/zadok.md +5 -2
  59. package/payload/mishkan/agents/zerubbabel.md +4 -2
  60. package/payload/mishkan/commands/code-graph.md +35 -0
  61. package/payload/mishkan/commands/eval-baruch.md +57 -0
  62. package/payload/mishkan/commands/mishkan-org-reference.md +58 -0
  63. package/payload/mishkan/commands/mishkan-skills-misses.md +47 -0
  64. package/payload/mishkan/commands/mishkan-skills-reindex.md +33 -0
  65. package/payload/mishkan/commands/skills.md +35 -0
  66. package/payload/mishkan/commands/sprint-close.md +3 -1
  67. package/payload/mishkan/hooks/pre-tool-knowledge-route.sh +231 -0
  68. package/payload/mishkan/hooks/pre-tool-task-skill-route.sh +122 -0
  69. package/payload/mishkan/hooks/session-start-skill-index.sh +35 -0
  70. package/payload/mishkan/observability/schema.json +1 -1
  71. package/payload/mishkan/observability/watch/src/mishkan_watch/__main__.py +84 -2
  72. package/payload/mishkan/observability/watch/src/mishkan_watch/app.py +64 -3
  73. package/payload/mishkan/observability/watch/src/mishkan_watch/org_data.py +79 -0
  74. package/payload/mishkan/observability/watch/src/mishkan_watch/skills_data.py +267 -0
  75. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/agents.py +33 -1
  76. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/knowledge.py +118 -9
  77. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/live.py +102 -2
  78. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/org.py +139 -0
  79. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/skills.py +187 -0
  80. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/usage.py +377 -0
  81. package/payload/mishkan/observability/watch/src/mishkan_watch/tabs/workflows.py +199 -27
  82. package/payload/mishkan/observability/watch/src/mishkan_watch/theme.tcss +78 -4
  83. package/payload/mishkan/observability/watchd/src/mishkan_watchd/__main__.py +36 -6
  84. package/payload/mishkan/observability/watchd/src/mishkan_watchd/sources/bus_tail.py +16 -3
  85. package/payload/mishkan/observability/watchd/src/mishkan_watchd/sources/graphify_tail.py +220 -0
  86. package/payload/mishkan/observability/watchd/src/mishkan_watchd/sources/worktree_poll.py +13 -2
  87. package/payload/mishkan/observability/watchd/src/mishkan_watchd/state.py +157 -3
  88. package/payload/mishkan/observability/watchd/tests/test_state.py +6 -2
  89. package/payload/mishkan/org/org.json +141 -0
  90. package/payload/mishkan/scripts/pre-commit-eval-gate.sh +50 -0
  91. package/payload/mishkan/scripts/skill-discovery-indexer.py +462 -0
  92. package/payload/mishkan/scripts/skill-discovery-misses.py +173 -0
  93. package/payload/mishkan/scripts/skill-discovery-router.py +457 -0
  94. package/payload/mishkan/scripts/skill-discovery-smoke.sh +156 -0
  95. package/payload/mishkan/skills/graphify-query-craft/SKILL.md +162 -0
  96. package/payload/mishkan/skills/mishkan-init/SKILL.md +15 -0
  97. package/payload/mishkan/skills/skill-discovery/SKILL.md +120 -0
  98. package/payload/mishkan/templates/case-node.schema.json +1 -0
  99. package/payload/mishkan/templates/observability-log.schema.json +1 -0
  100. package/payload/mishkan/workflows/README.md +78 -13
  101. package/payload/mishkan/workflows/chosheb-feature-ship.js +75 -0
  102. package/payload/mishkan/workflows/migdal-dr-drill.js +94 -0
  103. package/payload/mishkan/workflows/migdal-infra-change.js +67 -0
  104. package/payload/mishkan/workflows/mishkan-blast-radius.js +144 -0
  105. package/payload/mishkan/workflows/mishkan-init.js +58 -0
  106. package/payload/mishkan/workflows/mishkan-knowledge-gap-discovery.js +121 -0
  107. package/payload/mishkan/workflows/mishkan-standards-rollout.js +153 -0
  108. package/payload/mishkan/workflows/mishmar-security-gate.js +83 -0
  109. package/payload/mishkan/workflows/panim-ds-rollout.js +83 -0
  110. package/payload/mishkan/workflows/sefer-release-notes.js +85 -0
  111. package/payload/mishkan/workflows/yasad-data-migration-wave.js +78 -0
  112. package/payload/mishkan/workflows/yasad-schema-evolution.js +76 -0
  113. package/payload/mishkan/templates/user-CLAUDE.md +0 -62
@@ -0,0 +1,108 @@
1
+ # Graphify token-saving POC — MISHKAN harness
2
+
3
+ > Status: **verified** — measured 2026-06-07 on the MISHKAN harness.
4
+ > Closes the D-008 "Out of Scope" item 1: the 71.5× third-party claim
5
+ > needed an instrumented MISHKAN measurement before any agent could
6
+ > cite it. This document IS that measurement.
7
+
8
+ ## Setup
9
+
10
+ ```
11
+ graphify CLI : v0.8.33 (PyPI package: graphifyy, binary: graphify)
12
+ install : uv tool install graphifyy
13
+ harness rev : c27d58c (main, package.json v0.2.0)
14
+ scan command : graphify update .
15
+ bench command: graphify benchmark
16
+ host : AiobiDev, 2026-06-07
17
+ ```
18
+
19
+ The wall clock for the initial AST extraction was **27.8 s** for 205 files
20
+ on a single host with 12 parallel workers. The semantic LLM-enrichment step
21
+ was skipped (no `GEMINI_API_KEY` / `GOOGLE_API_KEY`) because the deterministic
22
+ AST extraction is what produces the token-saving leverage — the LLM step
23
+ adds community naming, not structure.
24
+
25
+ ## Graph stats
26
+
27
+ ```
28
+ files scanned : 205 (.py, .ts, .js, .sh, .md, .yml, .json)
29
+ nodes : 2,370
30
+ edges : 2,526
31
+ communities : 268
32
+ artefacts : graphify-out/{graph.json, graph.html, GRAPH_REPORT.md, manifest.json}
33
+ ```
34
+
35
+ ## Corpus baseline
36
+
37
+ The "naive full-corpus" baseline is the token cost of feeding every file
38
+ of the harness as raw text into the model context (what an unaided agent
39
+ does when asked a structural question and grep-then-read takes it through
40
+ many files).
41
+
42
+ ```
43
+ total words : 118,500
44
+ token estimate : ~158,000 (graphify's tokenizer, comparable to Anthropic's)
45
+ ```
46
+
47
+ ## Results
48
+
49
+ Graphify's built-in `benchmark` command issues five canonical structural
50
+ questions and measures the average token cost via the graph-BFS query path
51
+ vs the naive baseline.
52
+
53
+ | Question | Tokens via Graphify | Reduction factor |
54
+ |---|---:|---:|
55
+ | how does authentication work | ~2,116 | **74.7×** |
56
+ | what is the main entry point | ~1,842 | **85.8×** |
57
+ | how are errors handled | ~1,115 | **141.8×** |
58
+ | what connects the data layer to the api | ~2,278 | **69.4×** |
59
+ | what are the core abstractions | ~1,619 | **97.6×** |
60
+ | **Average** | **~1,793** | **88.1×** |
61
+
62
+ Spread: **69.4×** (worst) to **141.8×** (best). Variance is driven by how
63
+ localised the answer is in the graph — "core abstractions" pulls many
64
+ small community labels; "error handling" hits a tight cluster.
65
+
66
+ ## Interpretation
67
+
68
+ The third-party 71.5× claim **lands inside the observed range** and the
69
+ MISHKAN-specific average is **+23 % higher** (88.1× vs 71.5×). The
70
+ harness has more Python and shell than the third-party benchmark corpora,
71
+ and Python's AST shape gives Graphify cleaner structural edges than
72
+ TS/JSX-heavy codebases — that's the plausible explanation.
73
+
74
+ For the five MISHKAN code-writing specialists (Hizkiah, Salma, Oholiab,
75
+ Nathan, Zadok), this means a single `graphify query` answers a
76
+ structural question at **~1.8k tokens** vs **~158k tokens** to lift the
77
+ full harness into context. The advisory hook D-009 (telemetry-only
78
+ phase 1, advisory phase 2) is justified by this margin even at the
79
+ worst-case 69×.
80
+
81
+ ## What this measurement does NOT cover
82
+
83
+ - **Production agent workload.** The five benchmark questions are
84
+ Graphify's own canonical set, chosen to be representative — they are
85
+ not the full distribution of MISHKAN-agent structural queries.
86
+ Phase 1 of D-009 (telemetry-only hook) will gather the real
87
+ production distribution; if it differs materially, the average above
88
+ needs re-measuring.
89
+ - **Token quality.** The 88.1× is a tokens-spent ratio, not an
90
+ answer-quality ratio. Graphify's BFS answer is structurally precise
91
+ but textually sparse; reading raw files gives more prose context.
92
+ The trade-off is intentional — agents that need prose still Read
93
+ after a Graphify query narrows the file set.
94
+ - **Re-scan cadence cost.** The 27.8 s initial scan is one-off; the
95
+ incremental `graphify update .` after a commit is sub-second on the
96
+ changed files only. POC didn't measure this — assumed negligible.
97
+ - **LLM-enriched semantic step.** Adding `GEMINI_API_KEY` re-runs the
98
+ community-naming step which produces nicer summaries but costs LLM
99
+ tokens. POC ran deterministic-only. The token ratios above are for
100
+ the deterministic path.
101
+
102
+ ## Reference
103
+
104
+ - D-008 — three-store knowledge surface (Graphify · Cognee work ·
105
+ Cognee curated). See `docs/design/MISHKAN_decisions.md`.
106
+ - D-009 — graph-first PreToolUse advisory hook for the five code-writing
107
+ specialists. Same file.
108
+ - Graphify upstream — https://github.com/safishamsi/graphify.
@@ -1,8 +1,41 @@
1
1
  # 09 — Dynamic Workflows
2
2
 
3
3
  > Goal: explain when MISHKAN reaches for a dynamic workflow vs ordinary
4
- > Task delegation, the seven workflows shipped, and the cost gate that
5
- > keeps the count from drifting up.
4
+ > Task delegation, the **18 workflows shipped** (10 org-level + 8 team-
5
+ > level), the cost gate, and how dynamic workflows relate to the 30+
6
+ > craft skills that drive each team's day-to-day work.
7
+
8
+ ## Three layers, not one
9
+
10
+ MISHKAN has **three** levels of orchestration. Conflating them produces
11
+ wrong tools at the wrong level:
12
+
13
+ | Layer | Lives in | Owns | Examples |
14
+ |---|---|---|---|
15
+ | **Dynamic workflows** (org-level) | `payload/mishkan/workflows/*.js` | The whole harness | `mishkan-sprint-close`, `mishkan-release-readiness`, `mishkan-codebase-audit` |
16
+ | **Craft skills** (team / agent-level) | `payload/mishkan/skills/<agent>-<role>-craft/` | One agent or one Team Lead | `hizkiah-backend-impl-craft`, `salma-frontend-impl-craft`, `team-lead-craft` |
17
+ | **Task delegations** (single shot) | inline `Task()` call from main session | The caller | "Bezalel, review this ADR" |
18
+
19
+ **Dynamic workflows** orchestrate dozens of agents in parallel with
20
+ typed contracts, adversarial verification, and judge panels. They are
21
+ expensive and bounded — hard cap at 10 in the harness. The gate to
22
+ add one is high (recurrence + parallelism + repeatable shape).
23
+
24
+ **Craft skills** are how each agent ships day-to-day work. They are
25
+ markdown documents loaded into the agent's context. There are 30+ of
26
+ them today — every specialist (Hizkiah, Salma, Nathan, etc.) has at
27
+ least one craft skill that codifies "this is how I do my job". Team
28
+ Leads additionally load `team-lead-craft` which captures the
29
+ orchestration shape they apply to their team's daily flow (handoff,
30
+ review, escalation, sprint contribution).
31
+
32
+ **Task delegations** are the one-off escape hatch — when neither a
33
+ workflow nor a craft skill captures the shape, the main session
34
+ spawns one or more subagents inline.
35
+
36
+ If you find yourself asking "is THIS a workflow?", apply the gate
37
+ below. If it doesn't pass all three, it's probably a craft skill
38
+ addition or a one-shot Task.
6
39
 
7
40
  ## What a workflow is, in one paragraph
8
41
 
@@ -30,7 +63,41 @@ The gate MISHKAN applies — **yes only if all three**:
30
63
  Anything that fails any of the three is better as Task fan-out from
31
64
  the main session.
32
65
 
33
- ## The seven workflows
66
+ ## Team-level workflows — the 8 shipped (ADR D-010)
67
+
68
+ Each team also has a small set of **team-level workflows** for the
69
+ recurring high-stakes orchestrations that *only that team* runs. Hard
70
+ cap: 4 per team. Co-owned by PM + CTO per ADR D-010. Day-to-day shipping
71
+ flows (feature ship, refactor, review) stay at the craft-skill layer —
72
+ the team-level workflows are reserved for the orchestrations where
73
+ parallelism + adversarial verification actually change the outcome.
74
+
75
+ | Team | Workflow | What it does |
76
+ |---|---|---|
77
+ | Chosheb | [`chosheb-feature-ship`](../../payload/mishkan/workflows/chosheb-feature-ship.js) | Design → handoff package (DS fit + a11y + assets + QA) |
78
+ | Panim | [`panim-ds-rollout`](../../payload/mishkan/workflows/panim-ds-rollout.js) | Token change propagated to all consumers (worktree + a11y + visual regression) |
79
+ | Yasad | [`yasad-data-migration-wave`](../../payload/mishkan/workflows/yasad-data-migration-wave.js) | Wave of DB migrations, per-table 4-lens review |
80
+ | Yasad | [`yasad-schema-evolution`](../../payload/mishkan/workflows/yasad-schema-evolution.js) | Zero-downtime phased schema change with rollback |
81
+ | Mishmar | [`mishmar-security-gate`](../../payload/mishkan/workflows/mishmar-security-gate.js) | Pre-merge security gate, 3 lenses + adversarial refute |
82
+ | Migdal | [`migdal-infra-change`](../../payload/mishkan/workflows/migdal-infra-change.js) | Infra change validated by 5 lenses |
83
+ | Migdal | [`migdal-dr-drill`](../../payload/mishkan/workflows/migdal-dr-drill.js) | DR drill — enumerate, simulate, verify, RTO/RPO measurement |
84
+ | Sefer | [`sefer-release-notes`](../../payload/mishkan/workflows/sefer-release-notes.js) | Release notes from git log per category, style-guided |
85
+
86
+ Day-to-day shipping flows that are *not* workflows (and why):
87
+
88
+ - **Feature ship cycle** (SRS → CONTRACT → impl → tests → review) — sequential
89
+ with a small parallel window during review only; Task delegation handles
90
+ it cheaper than a workflow.
91
+ - **PR review** — handled by craft skills (`huram-frontend-lead-craft`,
92
+ `zerubbabel-backend-lead-craft`) + Task fan-out when needed.
93
+ - **Component build per design** — shape varies per component too much for
94
+ a stable script (fail rule 3 of the gate).
95
+
96
+ Adding a team-level workflow requires PM + CTO joint review per ADR D-010
97
+ (`docs/design/MISHKAN_decisions.md#D-010`). Each team has spare slots
98
+ (cap 4) — candidates compete by recurrence + parallelism + repeatability.
99
+
100
+ ## The 10 org-level workflows
34
101
 
35
102
  | Workflow | Pattern | Invoked by | Args |
36
103
  |---|---|---|---|
@@ -41,6 +108,9 @@ the main session.
41
108
  | [`mishkan-architecture-panel`](../../payload/mishkan/workflows/mishkan-architecture-panel.js) | judge panel + impact-fanout + synthesis | Bezalel gates wide-answer architecture decisions | `{ decision, context, horizon? }` |
42
109
  | [`mishkan-release-readiness`](../../payload/mishkan/workflows/mishkan-release-readiness.js) | barrier + nested workflow | Nehemiah + Bezalel before every prod deploy | `{ project_root, release_tag, verify_commands, audit_security? }` |
43
110
  | [`mishkan-init`](../../payload/mishkan/workflows/mishkan-init.js) | pipeline with overlap | `/mishkan-init` | `{ project_name, project_root, raw_intent, stack_hint? }` |
111
+ | [`mishkan-blast-radius`](../../payload/mishkan/workflows/mishkan-blast-radius.js) | Graphify + 3-lens orthogonal | gated by `/plan` | `{ target, depth?, project, min_sites_to_verify? }` |
112
+ | [`mishkan-knowledge-gap-discovery`](../../payload/mishkan/workflows/mishkan-knowledge-gap-discovery.js) | parallel probe + loop-until-dry | sprint close (optional) | `{ concepts[], project }` |
113
+ | [`mishkan-standards-rollout`](../../payload/mishkan/workflows/mishkan-standards-rollout.js) | per-team translate + verify + ratify | new rule lands in `y4nn-standards.md` | `{ rule_text, rule_id?, scope_hint? }` |
44
114
 
45
115
  ## How invocation actually happens
46
116
 
@@ -59,7 +129,7 @@ If a subagent finds itself needing a workflow (e.g. Phinehas wants a
59
129
  codebase audit), the subagent's response surfaces the recommendation
60
130
  to the main session, which then decides whether to fire.
61
131
 
62
- ## Patterns the seven scripts use
132
+ ## Patterns the 10 scripts use
63
133
 
64
134
  From the [community patterns catalogue](https://github.com/ray-amjad/claude-code-workflow-creator/blob/main/references/patterns.md)
65
135
  and Anthropic's docs:
@@ -13,7 +13,7 @@ Two artefacts that ship as one stack:
13
13
  network sources into a single in-memory snapshot. Exposes it on a UNIX
14
14
  socket as a snapshot + delta + heartbeat NDJSON stream.
15
15
  - **`mishkan-watch`** — Textual TUI client. Connects to the daemon socket,
16
- renders 5 tabs and a permanent status bar.
16
+ renders 8 tabs and a permanent status bar, with a project filter on `p`.
17
17
 
18
18
  See the design doc [`docs/design/MISHKAN_observability.md`](../design/MISHKAN_observability.md)
19
19
  for the full event schema, daemon architecture, and TUI layout.
@@ -43,14 +43,30 @@ uv tool install --from ~/.claude/mishkan/observability/watch mishkan-watch
43
43
 
44
44
  ## Run
45
45
 
46
- Two processes, two terminals (or two tmux panes):
46
+ Single command the TUI auto-starts the daemon if its socket isn't
47
+ present, waits up to 8 s for the bind, then opens. The daemon survives
48
+ the TUI's exit so a second `mishkan-watch` (e.g. in another tmux pane)
49
+ connects instantly.
47
50
 
48
51
  ```bash
49
- # terminal 1 — daemon, foreground
52
+ mishkan-watch
53
+ ```
54
+
55
+ Power users who want to manage the daemon explicitly (separate logs,
56
+ custom socket path, attached over SSH from another host) can opt out:
57
+
58
+ ```bash
59
+ # terminal 1 — daemon, foreground (logs in your face)
50
60
  mishkan-watchd start
51
61
 
52
- # terminal 2 — TUI
53
- mishkan-watch
62
+ # terminal 2 — TUI; refuses to fork the daemon
63
+ mishkan-watch --no-autostart
64
+ ```
65
+
66
+ Stop the daemon when you're done:
67
+
68
+ ```bash
69
+ mishkan-watchd stop
54
70
  ```
55
71
 
56
72
  Daemon lifecycle is always manual. Auto-start follow-up:
@@ -67,15 +83,22 @@ mishkan-watchd status # connect, print current snapshot as JSON
67
83
  mishkan-watchd stop # SIGTERM via the PID file
68
84
  ```
69
85
 
70
- ## The 5 tabs
86
+ ## The 8 tabs
71
87
 
72
- | # | Tab | What it answers |
88
+ | Key | Tab | What it answers |
73
89
  |---|---|---|
74
- | 1 | **Live** | "What is happening *right now*?" Active agents, workflows in-flight, current worktrees, Cognee + MCP rollup, rolling feed of every event. Default tab on launch. |
75
- | 2 | **Agents** | "What did each agent do?" Sessions tree (left) × agent history `DataTable` (centre) × errors panel (right). Per-session current-agent tracking attributes subagent tool calls to the right agent (sourced via `subagent_tail`). |
76
- | 3 | **Workflows** | "What did each dynamic workflow run?" Cards list (left) × phase tree with fan-out + cost (right). Populated from `workflow_start` events; phase/agent detail lights up as the `Workflow` tool runs. |
77
- | 4 | **Knowledge** | "Are my Cognee stores up and growing?" Two cards (work + curated, real-time node counts via neo4j HTTP cypher) × recent ops `DataTable` × MCP server status table covering Cognee + every other configured MCP. |
78
- | 5 | **Activity** | Unified, filterable event stream. Regex filter + type/agent selects. Errors and blocked permissions break the visual rhythm with a separator line. |
90
+ | `1` | **Live** | "What is happening *right now*?" Active agents (with `alias · role` annotation), workflows in-flight, current worktrees, Cognee + Graphify + MCP rollup, rolling feed of every event. Default tab on launch. |
91
+ | `2` | **Agents** | "What did each agent do?" Sessions tree (left, project paths decoded) × agent history `DataTable` (centre) × errors panel (right). Phantom sessions filtered out at the daemon `_confirmed_alive` gate. |
92
+ | `3` | **Workflows** | "What did each dynamic workflow run, and what's *available* to run?" Recent runs (top) + static catalogue parsed from each script's `meta` block (description, when-to-use, phases). Click an entry for detail. |
93
+ | `4` | **Knowledge** | "Are my stores up and growing?" Three cards (work + curated + graphify, real-time counts) × recent ops `DataTable` × MCP server status table. |
94
+ | `5` | **Activity** | Unified, filterable event stream. Regex filter + type/agent selects. Errors and blocked permissions break the visual rhythm with a separator line. |
95
+ | `6` | **Org-Ref** | Read-only browser of the 45-agent org from `org.json`. Tree by team, click a group → mission / charter / Hebrew name; click an agent → role, source, description. |
96
+ | `7` | **Usage** | Harness-wide tokens in/out/cached, cost, context window estimate, request counts. Per-session table sorted by token volume. Detail panel with per-agent attribution + top tools. |
97
+ | `8` | **Skills** | All installed skills (MISHKAN craft + community + plugin) grouped by origin × category. Right panel cross-references each skill to the CTO decisions (ADRs) that mention it. |
98
+
99
+ **Project filter (`p`)** — default off; press `p` to scope Live's ACTIVE
100
+ and WORKTREES to the current project only. Current project picked from
101
+ `CLAUDE_PROJECT_DIR` env var or `pwd` at launch.
79
102
 
80
103
  ## The status bar
81
104
 
@@ -109,7 +132,8 @@ Six daemon sources, each independent and fail-open:
109
132
  ## Keybindings
110
133
 
111
134
  ```
112
- 1-5 switch tab (instantaneous)
135
+ 1-8 switch tab (instantaneous)
136
+ p toggle project filter (current ↔ all)
113
137
  q quit
114
138
  j/k ↑↓ move focus within current panel
115
139
  tab move focus between panels
@@ -0,0 +1,264 @@
1
+ # 11 — Graphify (code-structure graph)
2
+
3
+ > Goal: use Graphify as the **structural** layer of the MISHKAN
4
+ > three-store knowledge stack (Graphify · Cognee work · Cognee curated)
5
+ > so that "who calls X" / "what depends on Y" questions cost ~1.8k
6
+ > tokens instead of lifting the whole repo into context.
7
+
8
+ ## What it is
9
+
10
+ [Graphify](https://github.com/safishamsi/graphify) is a tree-sitter
11
+ based code-graph extractor with a query path that traverses the graph
12
+ to answer structural questions. MIT, Python, `uv tool install`-able.
13
+ Per D-008 of the MISHKAN decision log, Graphify is the third store of
14
+ the knowledge stack:
15
+
16
+ | Store | Question it answers | Source |
17
+ |---|---|---|
18
+ | **Graphify** | *How is the code structured?* | tree-sitter AST + optional LLM enrichment, deterministic, re-derivable |
19
+ | **Cognee work** | *Why does this code exist and what did we decide?* | agent-ingested ADRs, runbooks, resolved research |
20
+ | **Cognee curated** | *What did we learn on other projects?* | seeded, read-only from projects |
21
+
22
+ These are non-overlapping by design: one writer per store, one
23
+ question type per store, no semantic mixing. The crisp test:
24
+ **structure → Graphify, semantics → Cognee work.**
25
+
26
+ ## Install
27
+
28
+ ```bash
29
+ uv tool install "graphifyy>=0.8.33"
30
+ graphify --version
31
+ ```
32
+
33
+ The PyPI package is `graphifyy` (double-y), the binary is `graphify`.
34
+ The `>=0.8.33` pin matters — earlier 0.8.x had a test-file-orphan bug
35
+ (test imports didn't resolve to file-level edges, so test files looked
36
+ disconnected from the source they covered). Fixed in 0.8.33.
37
+
38
+ Useful soft-breaking change in 0.8.29 to be aware of: project-local
39
+ providers under `./.graphify/providers.json` no longer auto-load by
40
+ default. Set `GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1` if you use per-project
41
+ LLM providers (the deterministic AST path doesn't need this).
42
+
43
+ ## Bring up the graph for a project
44
+
45
+ From the project root:
46
+
47
+ ```bash
48
+ graphify update .
49
+ ```
50
+
51
+ First run on the MISHKAN harness: **205 files → 2,370 nodes → 27.8 s**.
52
+ Subsequent `graphify update .` runs are incremental and sub-second on
53
+ warm cache.
54
+
55
+ **Keep the graph fresh — install the git hooks.** The canonical
56
+ maintenance path (recommended by the upstream community, not cron):
57
+
58
+ ```bash
59
+ graphify hook install # registers post-commit / post-checkout / post-merge / post-rewrite
60
+ # + a `graphify-json` git merge driver so the graph
61
+ # union-merges on parallel branches (no conflicts)
62
+ graphify hook status
63
+ graphify hook uninstall # later, if you want to remove them
64
+ ```
65
+
66
+ Without the hooks the graph drifts vs HEAD silently — symptoms (from
67
+ upstream issues): `graphify query` returns stale `file:line` citations,
68
+ `GRAPHIFY_OUT/GRAPH_REPORT.md` stays at the old node count after a
69
+ refactor. The hooks rebuild incrementally in milliseconds; the merge
70
+ driver makes `graph.json` conflict-free across branches.
71
+
72
+ Output lives in `<project>/graphify-out/`:
73
+
74
+ ```
75
+ graphify-out/
76
+ ├── graph.json # the AST graph (consumed by `graphify query`)
77
+ ├── graph.html # interactive visualisation, open in a browser
78
+ ├── GRAPH_REPORT.md # god-nodes + anomalies + community summary
79
+ └── manifest.json # quick stats (nodes / edges / communities)
80
+ ```
81
+
82
+ > `graphify-out/` is gitignored in this repo (project-local artefact,
83
+ > re-derivable). On your own projects, decide per-project whether to
84
+ > commit it; the harness convention is to NOT commit it.
85
+
86
+ ## Querying
87
+
88
+ ```bash
89
+ graphify query "who calls process_payment"
90
+ graphify query "what depends on the User model" --budget 1500
91
+ graphify affected "process_payment" --depth 3
92
+ graphify path "ClassA" "ClassB"
93
+ graphify explain "MishkanWatch"
94
+ ```
95
+
96
+ Default budget is 2000 tokens; the answer is plain text with `file:line`
97
+ citations.
98
+
99
+ POC numbers on the MISHKAN harness (2026-06-07):
100
+
101
+ | Question | Tokens | Ratio vs naive baseline |
102
+ |---|---:|---:|
103
+ | how does authentication work | ~2,116 | 74.7× |
104
+ | what is the main entry point | ~1,842 | 85.8× |
105
+ | how are errors handled | ~1,115 | 141.8× |
106
+ | what connects the data layer to the api | ~2,278 | 69.4× |
107
+ | what are the core abstractions | ~1,619 | 97.6× |
108
+ | **average** | **~1,793** | **88.1×** |
109
+
110
+ Naive baseline = lifting all 118,500 words / ~158,000 tokens of source
111
+ into the model context (Graphify's own README methodology). Full
112
+ method: `docs/research/graphify-token-saving-poc.md`.
113
+
114
+ > **Important footnote on the 88.1×.** This is a *mixed-corpus,
115
+ > naive-baseline* ratio — the same methodology as Graphify's upstream
116
+ > 71.5× claim. Third-party measurements on *real Claude / Cursor
117
+ > sessions* (where the assistant already scopes) report much smaller
118
+ > reductions: roughly **7-8×** on small targeted sessions, **7-30×**
119
+ > on real codebases. Treat 71.5× / 88.1× as **directional**, not
120
+ > spec-grade: they prove "Graphify costs orders of magnitude less than
121
+ > reading everything" — they don't prove "your next chat session costs
122
+ > 88× fewer tokens." The advisory hook (D-009) keeps emitting telemetry
123
+ > alongside the advisory injection — the firing-vs-graphify-query ratio
124
+ > on real sessions is the data path to tighten this claim.
125
+
126
+ ## How MISHKAN agents use it
127
+
128
+ Twenty code-touching dev agents across all six teams (per D-009
129
+ amended scope, 2026-06-07) load **graphify-query-craft**: Yasad backend
130
+ (Hizkiah, Nathan, Zadok, Shallum, Uriah) · Panim frontend (Salma,
131
+ Oholiab, Asaph, Jahaziel) · Chosheb UI (Hiram) · Mishmar code-security
132
+ (Ira, Joab, Hushai) · Migdal infra-code (Palal, Meshullam, Meremoth,
133
+ Hanun) · Sefer code-documentation (Joah, Shevna, Jehonathan). Its core
134
+ rule:
135
+
136
+ > Structure question → `graphify query`.
137
+ > Semantic question → Cognee work.
138
+
139
+ The PreToolUse hook **`pre-tool-knowledge-route.sh`** (D-009 amendment
140
+ 2026-06-07 — Phase 2 shipped) routes the agent to the right knowledge
141
+ surface. On every structural Read / bare-identifier Grep, it does two
142
+ things:
143
+
144
+ - **Emits telemetry**: a `hook_fire` event on the bus. The Knowledge
145
+ tab's activity counter records it.
146
+ - **Injects a 4-surface palette** via
147
+ `hookSpecificOutput.additionalContext` listing every knowledge surface
148
+ MISHKAN exposes, with pre-formed commands and per-surface signals so
149
+ the agent picks the right one — not graphify by default.
150
+
151
+ The four surfaces (D-008 + this amendment):
152
+
153
+ | Surface | Question it answers |
154
+ |---|---|
155
+ | **CODE STRUCTURE** (Graphify) | "who calls X", "what depends on Y", "path between A and B", impact / blast-radius |
156
+ | **THIS project's MEMORY** (Cognee work) | "why did WE decide X", "what's OUR convention", "what was resolved last sprint" — ADRs, runbooks, past research |
157
+ | **CROSS-PROJECT REFERENCE** (Cognee curated) | "what does the spec say about X", "what did we learn on OTHER projects" — shared read-only library |
158
+ | **Literal file content / text match** (Read or Grep) | "what's in THIS specific file", "every line containing this exact string" |
159
+
160
+ The advisory carries **real signals** inlined per call, so the agent
161
+ doesn't pick blind:
162
+
163
+ - Graph node + edge count, last-scan age (with `(stale — /code-graph scan to refresh)` flag when > 1h)
164
+ - For Grep on a bare identifier: a `jq` check on `graph.json` says whether the target is actually a node in the graph. If not, the advisory explicitly warns "don't burn ~1.8k tokens on a seedless query"
165
+ - For Read: the file's line count and an approximate token cost (`~lines × 4`)
166
+ - Cognee work and curated node counts (from the daemon's poll cache at `~/.cache/mishkan/cognee-counts.json`) so "(0 nodes — nothing ingested yet)" replaces a misleading recommendation
167
+
168
+ The hook never sets `permissionDecision` — purely advisory, the
169
+ Read/Grep still proceeds. When the project has no `graphify-out/`
170
+ directory the CODE STRUCTURE line flips to an install hint; the cognee
171
+ lines stay since the MCP surface is independent of graphify state.
172
+
173
+ No pre-cooked verdict ("soft" vs "strong") and no invented thresholds.
174
+ Threshold tuning is the D-011 telemetry workflow's job (`hook_fire` vs
175
+ `graphify_query` / `cognee_op` rates, reviewed at `/sprint-close`).
176
+
177
+ Performance contract for the hook: ≤50 ms p95, fail-open everywhere.
178
+
179
+ ## Knowledge-tab observability
180
+
181
+ The TUI's **Knowledge** tab surfaces Graphify activity alongside Cognee
182
+ in the "Recent ops" panel:
183
+
184
+ - `graphify_scan` events fire when `graphify-out/graph.json` mtime
185
+ changes (i.e. after `graphify update .` completes). The event payload
186
+ carries `nodes`, `edges`, `communities` from the manifest.
187
+ - `graphify_query` events fire when `graphify save-result` lands a new
188
+ file under `graphify-out/memory/`. The payload carries the question
189
+ + a 200-char excerpt of the answer.
190
+
191
+ Both events come from the daemon source `graphify_tail` (Python
192
+ asyncio, polls every 5 s, fail-open).
193
+
194
+ ## When NOT to use Graphify
195
+
196
+ Per D-008's "what NOT to do" list:
197
+
198
+ - Don't use Graphify to answer SEMANTIC questions ("why is this
199
+ deprecated?"). Those go to Cognee work.
200
+ - Don't use it on a project that doesn't have a `graphify-out/` dir —
201
+ the cost of an initial scan (~28 s on a 200-file repo) is not paid
202
+ for a single one-off structural question. Fall back to grep / Read.
203
+ - Don't cite a graph answer without the `file:line` citations from the
204
+ query output — "according to the graph" without an id is fabrication.
205
+
206
+ ## Known limitations (community-sourced)
207
+
208
+ - **Dynamic dispatch / reflection** — `getattr`, `eval`,
209
+ dict-of-callables, runtime monkey-patching are invisible to AST.
210
+ Recent 0.8.x added cross-language type-reference edges that recover
211
+ *some* missing links via type context, but the gap stays.
212
+ - **Runtime configuration** (env vars, feature flags, DI containers,
213
+ Spring-like wiring) — not in the AST. Community recipe: feed the YAML/
214
+ JSON config files as docs into the same graph; Graphify's multi-modal
215
+ ingest will surface them as nodes.
216
+ - **`GRAPHIFY_OUT` env var ignored by `query` / `path` / `explain`** —
217
+ upstream issue still open at research time. If you relocate
218
+ `graphify-out/`, pass `--graph <path>` explicitly instead.
219
+ - **`./.graphify/providers.json` no longer auto-loads** (0.8.29
220
+ soft-break). Set `GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1` if you use
221
+ per-project provider configs.
222
+
223
+ ## Refreshing the graph
224
+
225
+ Graphify indexing is **manual** — there is no auto-update on file
226
+ changes today. Three equivalent ways to refresh:
227
+
228
+ ```bash
229
+ graphify update . # direct CLI
230
+ npx mishkan-harness code-graph scan # MISHKAN wrapper
231
+ /code-graph scan # slash command (same wrapper)
232
+ ```
233
+
234
+ A read-only inspection of the current graph (no rebuild):
235
+
236
+ ```bash
237
+ npx mishkan-harness code-graph status # nodes / edges / last scan timestamp
238
+ /code-graph status # same
239
+ npx mishkan-harness code-graph open # opens graph.html in the browser
240
+ ```
241
+
242
+ When the advisory hook fires and you suspect the graph is stale, the
243
+ hook's own advisory text reminds you to run `npx mishkan-harness
244
+ code-graph scan` first.
245
+
246
+ ## Project init
247
+
248
+ `/mishkan-init` (per the updated skill) optionally runs the initial
249
+ `graphify update .` after the spec chain lands, so the project has a
250
+ graph from sprint S0 onwards. Skipped automatically on projects that
251
+ opted out of the structural layer.
252
+
253
+ ## See also
254
+
255
+ - [D-008](../design/MISHKAN_decisions.md#d-008) — three-store knowledge
256
+ epistemology.
257
+ - [D-009](../design/MISHKAN_decisions.md#d-009) — graph-first PreToolUse
258
+ advisory hook (Phase 1 telemetry-only at v0.2.1, Phase 2 advisory
259
+ injection planned).
260
+ - [POC report](../research/graphify-token-saving-poc.md) — verified
261
+ 88.1× reduction on the MISHKAN harness, 2026-06-07.
262
+ - [Memory layer](./04-memory-layer.md) — the Cognee work and curated
263
+ stores that complete the three-store stack.
264
+ - Upstream — https://github.com/safishamsi/graphify.