nexo-brain 1.5.0 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +31 -8
- package/package.json +1 -1
- package/src/scripts/deep-sleep/analyze_session.py +215 -0
- package/src/scripts/deep-sleep/apply_findings.py +167 -0
- package/src/scripts/deep-sleep/collect_transcripts.py +143 -0
- package/src/scripts/deep-sleep/prompt.md +109 -0
- package/src/scripts/nexo-deep-sleep.sh +76 -0
- package/src/scripts/nexo-watchdog.sh +83 -3
package/README.md
CHANGED
|
@@ -198,7 +198,7 @@ This means long sessions (8+ hours) feel like one continuous conversation instea
|
|
|
198
198
|
|
|
199
199
|
## Cognitive Features
|
|
200
200
|
|
|
201
|
-
NEXO Brain provides
|
|
201
|
+
NEXO Brain provides **100+ MCP tools** implementing cognitive science concepts that go beyond basic memory:
|
|
202
202
|
|
|
203
203
|
### Input Pipeline
|
|
204
204
|
|
|
@@ -226,11 +226,13 @@ NEXO Brain provides 29 cognitive tools on top of the 78 base tools, totaling **1
|
|
|
226
226
|
|---------|-------------|
|
|
227
227
|
| **HyDE Query Expansion** | Generates hypothetical answer embeddings for richer semantic search. Instead of searching for "deploy error", it imagines what a helpful memory about deploy errors would look like, then searches for that. |
|
|
228
228
|
| **Hybrid Search (FTS5+BM25+RRF)** | Combines dense vector search with BM25 keyword search via Reciprocal Rank Fusion. Outperforms pure semantic search on precise terminology and code identifiers. |
|
|
229
|
+
| **KG Boost** | Knowledge Graph connection count influences retrieval ranking. Memories linked to well-connected entities (many edges) receive a logarithmic score bonus, surfacing contextually important facts higher. |
|
|
230
|
+
| **HNSW Vector Index** | Optional approximate nearest neighbor index (hnswlib). Activates automatically when memory count exceeds 10,000. Falls back to exact brute-force below that threshold — no configuration needed. |
|
|
229
231
|
| **Cross-Encoder Reranking** | After initial vector retrieval, a cross-encoder model rescores candidates for precision. The top-k results are reordered by true semantic relevance before being returned to the agent. |
|
|
230
232
|
| **Multi-Query Decomposition** | Complex questions are automatically split into sub-queries. Each component is retrieved independently, then fused for a higher-quality answer — improves recall on multi-faceted prompts. |
|
|
231
233
|
| **Temporal Indexing** | Memories are indexed by time in addition to semantics. Time-sensitive queries ("what did we decide last Tuesday?") use temporal proximity scoring alongside semantic similarity. |
|
|
232
234
|
| **Spreading Activation** | Graph-based co-activation network. Memories retrieved together reinforce each other's connections, building an associative web that improves over time. |
|
|
233
|
-
| **Recall Explanations** | Transparent score breakdown for every retrieval result. Shows exactly why a memory was returned: semantic similarity, recency, access frequency, and co-activation bonuses. |
|
|
235
|
+
| **Recall Explanations** | Transparent score breakdown for every retrieval result. Shows exactly why a memory was returned: semantic similarity, recency, access frequency, KG boost, and co-activation bonuses. |
|
|
234
236
|
|
|
235
237
|
### Proactive
|
|
236
238
|
|
|
@@ -430,6 +432,17 @@ That's it. No need to run `claude` manually. Your operator will greet you immedi
|
|
|
430
432
|
|
|
431
433
|
## Architecture
|
|
432
434
|
|
|
435
|
+
### Modular Package Structure (v1.5.0)
|
|
436
|
+
|
|
437
|
+
The core is organized into two Python packages:
|
|
438
|
+
|
|
439
|
+
| Package | Modules | Responsibility |
|
|
440
|
+
|---------|---------|----------------|
|
|
441
|
+
| `db/` | 11 modules (`_core`, `_schema`, `_sessions`, `_learnings`, `_episodic`, `_credentials`, `_entities`, `_evolution`, `_fts`, `_reminders`, `_tasks`) | All SQLite persistence: schema migrations, CRUD, FTS indexing |
|
|
442
|
+
| `cognitive/` | 6 modules (`_core`, `_memory`, `_ingest`, `_search`, `_decay`, `_trust`) | Cognitive engine: embeddings, RAG, decay, trust scoring |
|
|
443
|
+
|
|
444
|
+
The rest of the server (`server.py`, `tools_*.py`, `plugins/`) stays flat for clarity.
|
|
445
|
+
|
|
433
446
|
### 100+ MCP Tools across 20 Categories
|
|
434
447
|
|
|
435
448
|
| Category | Count | Tools | Purpose |
|
|
@@ -439,7 +452,7 @@ That's it. No need to run `claude` manually. Your operator will greet you immedi
|
|
|
439
452
|
| Cognitive Advanced | 8 | hyde_search, spread_activate, explain_recall, dream, prospect, hook_capture, pin, archive | Advanced retrieval, proactive, lifecycle |
|
|
440
453
|
| Guard | 3 | check, stats, log_repetition | Metacognitive error prevention |
|
|
441
454
|
| Episodic | 10 | change_log/search/commit, decision_log/outcome/search, review_queue, diary_write/read, recall | What happened and why |
|
|
442
|
-
| Sessions | 4 | startup, heartbeat, stop, status | Session lifecycle + context shift detection |
|
|
455
|
+
| Sessions | 4 | startup, heartbeat, stop, status | Session lifecycle + context shift detection + inter-terminal auto-inbox |
|
|
443
456
|
| Coordination | 7 | track, untrack, files, send, ask, answer, check_answer | Multi-session file coordination + messaging |
|
|
444
457
|
| Reminders | 5 | list, create, update, complete, delete | User's tasks and deadlines |
|
|
445
458
|
| Followups | 4 | create, update, complete, delete | System's autonomous verification tasks |
|
|
@@ -455,6 +468,7 @@ That's it. No need to run `claude` manually. Your operator will greet you immedi
|
|
|
455
468
|
| Adaptive & Somatic | 4 | adaptive_weights, adaptive_override, somatic_check, somatic_stats | Learned signal weights + pain memory per file |
|
|
456
469
|
| Knowledge Graph | 4 | kg_query, kg_path, kg_neighbors, kg_stats | Bi-temporal entity-relationship graph |
|
|
457
470
|
| Context Continuity | 2 | checkpoint_save, checkpoint_read | Auto-compaction session preservation |
|
|
471
|
+
| Claim Graph | — | (internal) | Atomic facts with provenance and contradiction detection |
|
|
458
472
|
|
|
459
473
|
### Plugin System
|
|
460
474
|
|
|
@@ -591,11 +605,11 @@ NEXO Brain builds on ideas from several open-source projects. We're grateful for
|
|
|
591
605
|
|
|
592
606
|
| Project | Inspired Features |
|
|
593
607
|
|---------|------------------|
|
|
594
|
-
|
|
|
595
|
-
|
|
|
596
|
-
|
|
|
597
|
-
|
|
|
598
|
-
|
|
|
608
|
+
| Vestige | HyDE query expansion, spreading activation, prediction error gating, memory dreaming, prospective memory |
|
|
609
|
+
| ShieldCortex | Security pipeline (4-layer memory poisoning defense) |
|
|
610
|
+
| Bicameral | Quarantine queue (trust promotion policy for new facts) |
|
|
611
|
+
| claude-mem | Hook auto-capture (extracting decisions and facts from conversations) |
|
|
612
|
+
| ClawMem | Co-activation reinforcement (memories retrieved together strengthen connections) |
|
|
599
613
|
|
|
600
614
|
## Support the Project
|
|
601
615
|
|
|
@@ -610,6 +624,15 @@ If NEXO Brain is useful to you, consider:
|
|
|
610
624
|
|
|
611
625
|
## Changelog
|
|
612
626
|
|
|
627
|
+
### v1.5.0 — Modular Core + Knowledge Graph Search (2026-03-29)
|
|
628
|
+
- **Architecture**: `db.py` refactored into `db/` package (11 modules: core, schema, sessions, learnings, episodic, credentials, entities, evolution, fts, reminders, tasks)
|
|
629
|
+
- **Architecture**: `cognitive.py` refactored into `cognitive/` package (6 modules: core, memory, ingest, search, decay, trust)
|
|
630
|
+
- **KG Boost**: Knowledge Graph connection count now influences search result ranking — well-connected entities surface higher in retrieval
|
|
631
|
+
- **HNSW Vector Index**: Optional approximate nearest neighbor acceleration (activates automatically above 10,000 memories, falls back to brute-force otherwise)
|
|
632
|
+
- **Claim Graph**: Decomposes blob memories into atomic verifiable facts with provenance, confidence scores, and contradiction detection
|
|
633
|
+
- **Inter-terminal Auto-inbox (D+)**: `nexo_startup` now accepts `claude_session_id` (Claude Code session UUID) — enables automatic inbox delivery between parallel terminals via PostToolUse hook + migration v13
|
|
634
|
+
- **Tests**: 24 pytest tests across 3 suites (cognitive, knowledge graph, migrations)
|
|
635
|
+
|
|
613
636
|
### v1.4.1 — Multi-AI Code Review (2026-03-29)
|
|
614
637
|
- **Fix**: 3 bugs found by GPT-5.4 (Codex CLI) + Gemini 2.5 (Gemini CLI) reviewing full codebase
|
|
615
638
|
- `session_diaries` → `session_diary` table name (smart startup silently failed)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "nexo-brain",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.1",
|
|
4
4
|
"mcpName": "io.github.wazionapps/nexo",
|
|
5
5
|
"description": "NEXO — Cognitive co-operator for Claude Code. Atkinson-Shiffrin memory, semantic RAG, knowledge graph, HNSW vector indexing, trust scoring, and metacognitive error prevention.",
|
|
6
6
|
"bin": {
|
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
Deep Sleep — Step 2: Analyze transcripts with Claude CLI (bare mode).
|
|
4
|
+
Sends each session to Claude opus for analysis, then consolidates findings.
|
|
5
|
+
"""
|
|
6
|
+
import json
|
|
7
|
+
import os
|
|
8
|
+
import subprocess
|
|
9
|
+
import sys
|
|
10
|
+
from datetime import datetime
|
|
11
|
+
from pathlib import Path
|
|
12
|
+
|
|
13
|
+
PROMPT_FILE = Path(__file__).parent / "prompt.md"
|
|
14
|
+
DEEP_SLEEP_DIR = Path.home() / "claude" / "operations" / "deep-sleep"
|
|
15
|
+
MAX_TRANSCRIPT_CHARS = 150_000
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def build_transcript_text(session: dict) -> str:
|
|
19
|
+
"""Build a readable transcript from a session."""
|
|
20
|
+
lines = [
|
|
21
|
+
f"## Session: {session['session_file']}",
|
|
22
|
+
f"Modified: {session['modified']}",
|
|
23
|
+
f"Messages: {session['message_count']}, Tool uses: {session['tool_use_count']}",
|
|
24
|
+
"",
|
|
25
|
+
"### Conversation"
|
|
26
|
+
]
|
|
27
|
+
for msg in session["messages"]:
|
|
28
|
+
role = "USER" if msg["role"] == "user" else "NEXO"
|
|
29
|
+
lines.append(f"\n**{role}:**")
|
|
30
|
+
lines.append(msg["text"])
|
|
31
|
+
|
|
32
|
+
if session["tool_uses"]:
|
|
33
|
+
lines.append("\n### Tool Usage Log")
|
|
34
|
+
for tu in session["tool_uses"]:
|
|
35
|
+
file_info = f" [{tu['file'][:80]}]" if tu.get("file") else ""
|
|
36
|
+
lines.append(f"- {tu['tool']}{file_info}")
|
|
37
|
+
|
|
38
|
+
return "\n".join(lines)
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
def find_api_key() -> str | None:
|
|
42
|
+
"""Find Anthropic API key from common locations."""
|
|
43
|
+
# Environment variable
|
|
44
|
+
key = os.environ.get("ANTHROPIC_API_KEY", "")
|
|
45
|
+
if key:
|
|
46
|
+
return key
|
|
47
|
+
|
|
48
|
+
# Common file locations
|
|
49
|
+
for path in [
|
|
50
|
+
Path.home() / ".claude" / "anthropic-api-key.txt",
|
|
51
|
+
Path.home() / ".anthropic" / "api_key",
|
|
52
|
+
Path.home() / ".config" / "anthropic" / "api_key",
|
|
53
|
+
]:
|
|
54
|
+
if path.exists():
|
|
55
|
+
return path.read_text().strip()
|
|
56
|
+
|
|
57
|
+
return None
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def analyze_with_claude(transcript: str, prompt: str) -> dict | None:
|
|
61
|
+
"""Send transcript to Claude CLI for analysis."""
|
|
62
|
+
full_prompt = (
|
|
63
|
+
f"{prompt}\n\n---\n\n# TODAY'S TRANSCRIPT\n\n{transcript}\n\n---\n\n"
|
|
64
|
+
"Analyze this transcript and return the JSON output as specified. "
|
|
65
|
+
"Return ONLY the JSON, no markdown code fences."
|
|
66
|
+
)
|
|
67
|
+
|
|
68
|
+
api_key = find_api_key()
|
|
69
|
+
env = os.environ.copy()
|
|
70
|
+
if api_key:
|
|
71
|
+
env["ANTHROPIC_API_KEY"] = api_key
|
|
72
|
+
|
|
73
|
+
try:
|
|
74
|
+
result = subprocess.run(
|
|
75
|
+
["claude", "-p", full_prompt, "--model", "opus", "--output-format", "text", "--bare"],
|
|
76
|
+
capture_output=True, text=True, timeout=300, env=env
|
|
77
|
+
)
|
|
78
|
+
|
|
79
|
+
if result.returncode != 0:
|
|
80
|
+
print(f"Claude CLI error: {result.stderr[:500]}", file=sys.stderr)
|
|
81
|
+
return None
|
|
82
|
+
|
|
83
|
+
response_text = result.stdout.strip()
|
|
84
|
+
|
|
85
|
+
# Strip markdown code fences if present
|
|
86
|
+
if response_text.startswith("```"):
|
|
87
|
+
lines = response_text.split("\n")
|
|
88
|
+
response_text = "\n".join(lines[1:-1] if lines[-1].strip() == "```" else lines[1:])
|
|
89
|
+
response_text = response_text.strip()
|
|
90
|
+
|
|
91
|
+
# Find JSON object in response
|
|
92
|
+
json_start = response_text.find("{")
|
|
93
|
+
json_end = response_text.rfind("}") + 1
|
|
94
|
+
if json_start >= 0 and json_end > json_start:
|
|
95
|
+
response_text = response_text[json_start:json_end]
|
|
96
|
+
|
|
97
|
+
return json.loads(response_text)
|
|
98
|
+
|
|
99
|
+
except subprocess.TimeoutExpired:
|
|
100
|
+
print("Claude CLI timeout (300s)", file=sys.stderr)
|
|
101
|
+
return None
|
|
102
|
+
except json.JSONDecodeError as e:
|
|
103
|
+
print(f"Failed to parse Claude response: {e}", file=sys.stderr)
|
|
104
|
+
return None
|
|
105
|
+
except FileNotFoundError:
|
|
106
|
+
print("Claude CLI not found. Install: npm install -g @anthropic-ai/claude-code", file=sys.stderr)
|
|
107
|
+
return None
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
def consolidate_findings(results: list[dict]) -> dict:
|
|
111
|
+
"""Merge findings from multiple sessions into one report."""
|
|
112
|
+
consolidated = {
|
|
113
|
+
"uncaptured_corrections": [],
|
|
114
|
+
"uncaptured_ideas": [],
|
|
115
|
+
"missed_commitments": [],
|
|
116
|
+
"protocol_compliance": {
|
|
117
|
+
"guard_check": {"required": 0, "executed": 0},
|
|
118
|
+
"heartbeat_quality": {"total": 0, "with_good_context": 0},
|
|
119
|
+
"trust_adjustments": {"corrections_detected": 0, "adjusted": 0},
|
|
120
|
+
"learning_capture": {"errors_resolved": 0, "captured": 0},
|
|
121
|
+
"change_log": {"production_edits": 0, "logged": 0},
|
|
122
|
+
"feedback_capture": {"corrections": 0, "captured": 0},
|
|
123
|
+
},
|
|
124
|
+
"protocol_violations": [],
|
|
125
|
+
"quality_issues": [],
|
|
126
|
+
"auto_reinforcements": [],
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
for r in results:
|
|
130
|
+
if not r:
|
|
131
|
+
continue
|
|
132
|
+
for key in ["uncaptured_corrections", "uncaptured_ideas", "missed_commitments",
|
|
133
|
+
"protocol_violations", "quality_issues", "auto_reinforcements"]:
|
|
134
|
+
consolidated[key].extend(r.get(key, []))
|
|
135
|
+
|
|
136
|
+
pc = r.get("protocol_compliance", {})
|
|
137
|
+
for key in consolidated["protocol_compliance"]:
|
|
138
|
+
if key in pc and isinstance(pc[key], dict):
|
|
139
|
+
for subkey in consolidated["protocol_compliance"][key]:
|
|
140
|
+
consolidated["protocol_compliance"][key][subkey] += pc[key].get(subkey, 0)
|
|
141
|
+
|
|
142
|
+
# Calculate rates
|
|
143
|
+
for key, vals in consolidated["protocol_compliance"].items():
|
|
144
|
+
keys = list(vals.keys())
|
|
145
|
+
if len(keys) == 2:
|
|
146
|
+
denominator = vals[keys[0]]
|
|
147
|
+
numerator = vals[keys[1]]
|
|
148
|
+
vals["rate"] = round(numerator / denominator, 2) if denominator > 0 else 1.0
|
|
149
|
+
|
|
150
|
+
rates = [v.get("rate", 1.0) for v in consolidated["protocol_compliance"].values()]
|
|
151
|
+
consolidated["protocol_compliance"]["overall_compliance"] = round(sum(rates) / len(rates), 2) if rates else 1.0
|
|
152
|
+
consolidated["auto_reinforcements"] = list(set(consolidated["auto_reinforcements"]))
|
|
153
|
+
|
|
154
|
+
return consolidated
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
def main():
|
|
158
|
+
date = sys.argv[1] if len(sys.argv) > 1 else datetime.now().strftime("%Y-%m-%d")
|
|
159
|
+
|
|
160
|
+
transcripts_file = DEEP_SLEEP_DIR / f"{date}-transcripts.json"
|
|
161
|
+
if not transcripts_file.exists():
|
|
162
|
+
print(f"No transcripts found for {date}. Run collect_transcripts.py first.")
|
|
163
|
+
sys.exit(1)
|
|
164
|
+
|
|
165
|
+
with open(transcripts_file) as f:
|
|
166
|
+
data = json.load(f)
|
|
167
|
+
|
|
168
|
+
sessions = data["sessions"]
|
|
169
|
+
print(f"Analyzing {len(sessions)} sessions from {date}...")
|
|
170
|
+
|
|
171
|
+
prompt = PROMPT_FILE.read_text()
|
|
172
|
+
|
|
173
|
+
results = []
|
|
174
|
+
for i, session in enumerate(sessions):
|
|
175
|
+
transcript = build_transcript_text(session)
|
|
176
|
+
|
|
177
|
+
if len(transcript) < 500:
|
|
178
|
+
print(f" Session {i+1}/{len(sessions)}: skipped (too short)")
|
|
179
|
+
continue
|
|
180
|
+
|
|
181
|
+
if len(transcript) > MAX_TRANSCRIPT_CHARS:
|
|
182
|
+
transcript = transcript[:MAX_TRANSCRIPT_CHARS] + "\n\n[TRUNCATED]"
|
|
183
|
+
|
|
184
|
+
print(f" Session {i+1}/{len(sessions)}: {session['session_file'][:12]}... ({len(transcript)} chars)")
|
|
185
|
+
result = analyze_with_claude(transcript, prompt)
|
|
186
|
+
if result:
|
|
187
|
+
results.append(result)
|
|
188
|
+
print(f" → {len(result.get('uncaptured_corrections', []))} corrections, "
|
|
189
|
+
f"{len(result.get('protocol_violations', []))} violations")
|
|
190
|
+
else:
|
|
191
|
+
print(f" → Analysis failed")
|
|
192
|
+
|
|
193
|
+
consolidated = consolidate_findings(results)
|
|
194
|
+
consolidated["date"] = date
|
|
195
|
+
consolidated["sessions_analyzed"] = len(results)
|
|
196
|
+
|
|
197
|
+
n_corrections = len(consolidated["uncaptured_corrections"])
|
|
198
|
+
n_violations = len(consolidated["protocol_violations"])
|
|
199
|
+
compliance = consolidated["protocol_compliance"]["overall_compliance"]
|
|
200
|
+
consolidated["summary"] = (
|
|
201
|
+
f"Analyzed {len(results)} sessions. "
|
|
202
|
+
f"Found {n_corrections} uncaptured corrections, {n_violations} protocol violations. "
|
|
203
|
+
f"Overall compliance: {compliance:.0%}."
|
|
204
|
+
)
|
|
205
|
+
|
|
206
|
+
output_file = DEEP_SLEEP_DIR / f"{date}-analysis.json"
|
|
207
|
+
with open(output_file, "w") as f:
|
|
208
|
+
json.dump(consolidated, f, indent=2, ensure_ascii=False)
|
|
209
|
+
|
|
210
|
+
print(f"\nResults: {output_file}")
|
|
211
|
+
print(consolidated["summary"])
|
|
212
|
+
|
|
213
|
+
|
|
214
|
+
if __name__ == "__main__":
|
|
215
|
+
main()
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
Deep Sleep — Step 3: Apply findings.
|
|
4
|
+
Takes the analysis output and writes feedback memories + trust adjustments.
|
|
5
|
+
"""
|
|
6
|
+
import json
|
|
7
|
+
import sqlite3
|
|
8
|
+
import sys
|
|
9
|
+
from datetime import datetime
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
|
|
12
|
+
DEEP_SLEEP_DIR = Path.home() / "claude" / "operations" / "deep-sleep"
|
|
13
|
+
NEXO_DB = Path.home() / "claude" / "nexo-mcp" / "nexo.db"
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def find_memory_dir() -> Path:
|
|
17
|
+
"""Find the Claude Code auto-memory directory."""
|
|
18
|
+
claude_dir = Path.home() / ".claude" / "projects"
|
|
19
|
+
for d in claude_dir.iterdir():
|
|
20
|
+
if d.is_dir():
|
|
21
|
+
mem_dir = d / "memory"
|
|
22
|
+
if mem_dir.exists():
|
|
23
|
+
return mem_dir
|
|
24
|
+
# Fallback: create under first project dir
|
|
25
|
+
for d in claude_dir.iterdir():
|
|
26
|
+
if d.is_dir():
|
|
27
|
+
mem_dir = d / "memory"
|
|
28
|
+
mem_dir.mkdir(exist_ok=True)
|
|
29
|
+
return mem_dir
|
|
30
|
+
return claude_dir / "memory"
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def write_feedback_memory(memory_dir: Path, filename: str, name: str, description: str, content: str):
|
|
34
|
+
"""Write a feedback memory file."""
|
|
35
|
+
filepath = memory_dir / filename
|
|
36
|
+
feedback = f"""---
|
|
37
|
+
name: {name}
|
|
38
|
+
description: {description}
|
|
39
|
+
type: feedback
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
{content}
|
|
43
|
+
"""
|
|
44
|
+
filepath.write_text(feedback)
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def update_memory_index(memory_dir: Path, new_entries: list[dict]):
|
|
48
|
+
"""Append new entries to MEMORY.md index."""
|
|
49
|
+
index_file = memory_dir / "MEMORY.md"
|
|
50
|
+
if not index_file.exists() or not new_entries:
|
|
51
|
+
return
|
|
52
|
+
|
|
53
|
+
current = index_file.read_text()
|
|
54
|
+
lines_to_add = []
|
|
55
|
+
for entry in new_entries:
|
|
56
|
+
line = f"- **{entry['title']}:** `{entry['filename']}` --- {entry['summary']}"
|
|
57
|
+
if line not in current:
|
|
58
|
+
lines_to_add.append(line)
|
|
59
|
+
|
|
60
|
+
if lines_to_add:
|
|
61
|
+
current += "\n" + "\n".join(lines_to_add) + "\n"
|
|
62
|
+
index_file.write_text(current)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def adjust_trust(points: int, context: str):
|
|
66
|
+
"""Record trust adjustment in cognitive.db if available."""
|
|
67
|
+
cog_db = Path.home() / "claude" / "nexo-mcp" / "cognitive.db"
|
|
68
|
+
if not cog_db.exists():
|
|
69
|
+
return
|
|
70
|
+
try:
|
|
71
|
+
conn = sqlite3.connect(str(cog_db))
|
|
72
|
+
conn.execute(
|
|
73
|
+
"INSERT INTO trust_events (event, context, points, created_at) VALUES (?, ?, ?, ?)",
|
|
74
|
+
("deep_sleep_violations", context, points, datetime.now().isoformat())
|
|
75
|
+
)
|
|
76
|
+
conn.commit()
|
|
77
|
+
conn.close()
|
|
78
|
+
except Exception:
|
|
79
|
+
pass
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def apply(analysis: dict):
|
|
83
|
+
"""Apply all findings from deep sleep analysis."""
|
|
84
|
+
memory_dir = find_memory_dir()
|
|
85
|
+
actions_taken = []
|
|
86
|
+
memory_entries = []
|
|
87
|
+
date = analysis["date"]
|
|
88
|
+
|
|
89
|
+
print(f"\nApplying findings for {date}...")
|
|
90
|
+
|
|
91
|
+
# 1. Uncaptured corrections → feedback memories (high/critical only)
|
|
92
|
+
for i, correction in enumerate(analysis.get("uncaptured_corrections", [])):
|
|
93
|
+
severity = correction.get("severity", "medium")
|
|
94
|
+
if severity not in ("high", "critical"):
|
|
95
|
+
continue
|
|
96
|
+
|
|
97
|
+
category = correction.get("category", "process")
|
|
98
|
+
content = correction.get("what_nexo_should_have_saved", "")
|
|
99
|
+
quote = correction.get("quote", "")
|
|
100
|
+
|
|
101
|
+
safe_name = category.replace(" ", "_").lower()
|
|
102
|
+
filename = f"ds_{date}_{safe_name}_{i}.md"
|
|
103
|
+
write_feedback_memory(
|
|
104
|
+
memory_dir, filename,
|
|
105
|
+
name=content[:60],
|
|
106
|
+
description=f"Deep sleep detected uncaptured correction ({severity})",
|
|
107
|
+
content=f"{content}\n\n**Why:** User said: \"{quote}\"\nContext: {correction.get('context', '')}\n\n**How to apply:** {content}"
|
|
108
|
+
)
|
|
109
|
+
memory_entries.append({
|
|
110
|
+
"title": content[:40],
|
|
111
|
+
"filename": filename,
|
|
112
|
+
"summary": f"Deep sleep {date}, severity {severity}"
|
|
113
|
+
})
|
|
114
|
+
actions_taken.append(f"feedback_write: {filename}")
|
|
115
|
+
|
|
116
|
+
# 2. Trust adjustments for critical violations
|
|
117
|
+
critical_violations = [v for v in analysis.get("protocol_violations", []) if v.get("severity") == "critical"]
|
|
118
|
+
if critical_violations:
|
|
119
|
+
points = -3 * len(critical_violations)
|
|
120
|
+
adjust_trust(points, f"{len(critical_violations)} critical violations on {date}")
|
|
121
|
+
actions_taken.append(f"trust: {points} points ({len(critical_violations)} critical violations)")
|
|
122
|
+
|
|
123
|
+
# 3. Update MEMORY.md index
|
|
124
|
+
update_memory_index(memory_dir, memory_entries)
|
|
125
|
+
if memory_entries:
|
|
126
|
+
actions_taken.append(f"memory_index: {len(memory_entries)} entries added")
|
|
127
|
+
|
|
128
|
+
# 4. Save applied actions log
|
|
129
|
+
applied_log = {
|
|
130
|
+
"date": date,
|
|
131
|
+
"applied_at": datetime.now().isoformat(),
|
|
132
|
+
"actions_taken": actions_taken,
|
|
133
|
+
"corrections_processed": len(analysis.get("uncaptured_corrections", [])),
|
|
134
|
+
"compliance": analysis.get("protocol_compliance", {}).get("overall_compliance", 0)
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
applied_file = DEEP_SLEEP_DIR / f"{date}-applied.json"
|
|
138
|
+
with open(applied_file, "w") as f:
|
|
139
|
+
json.dump(applied_log, f, indent=2, ensure_ascii=False)
|
|
140
|
+
|
|
141
|
+
print(f"Applied {len(actions_taken)} actions:")
|
|
142
|
+
for a in actions_taken:
|
|
143
|
+
print(f" ✓ {a}")
|
|
144
|
+
|
|
145
|
+
return applied_log
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+
def main():
|
|
149
|
+
date = sys.argv[1] if len(sys.argv) > 1 else datetime.now().strftime("%Y-%m-%d")
|
|
150
|
+
|
|
151
|
+
analysis_file = DEEP_SLEEP_DIR / f"{date}-analysis.json"
|
|
152
|
+
if not analysis_file.exists():
|
|
153
|
+
print(f"No analysis found for {date}. Run analyze_session.py first.")
|
|
154
|
+
sys.exit(1)
|
|
155
|
+
|
|
156
|
+
with open(analysis_file) as f:
|
|
157
|
+
analysis = json.load(f)
|
|
158
|
+
|
|
159
|
+
result = apply(analysis)
|
|
160
|
+
|
|
161
|
+
compliance = analysis.get("protocol_compliance", {}).get("overall_compliance", 0)
|
|
162
|
+
print(f"\nDeep Sleep {date} — {result['corrections_processed']} corrections, "
|
|
163
|
+
f"{compliance:.0%} compliance, {len(result['actions_taken'])} actions applied")
|
|
164
|
+
|
|
165
|
+
|
|
166
|
+
if __name__ == "__main__":
|
|
167
|
+
main()
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
Deep Sleep — Step 1: Collect today's session transcripts.
|
|
4
|
+
Reads Claude Code .jsonl files, extracts clean conversation text + tool usage.
|
|
5
|
+
"""
|
|
6
|
+
import json
|
|
7
|
+
import os
|
|
8
|
+
import sys
|
|
9
|
+
from datetime import datetime
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
|
|
12
|
+
MIN_USER_MESSAGES = 3 # Skip trivial sessions
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
def find_sessions_dir() -> Path:
|
|
16
|
+
"""Find the Claude Code sessions directory dynamically."""
|
|
17
|
+
claude_dir = Path.home() / ".claude" / "projects"
|
|
18
|
+
if not claude_dir.exists():
|
|
19
|
+
return claude_dir
|
|
20
|
+
|
|
21
|
+
# Find the project directory (usually named after the home path)
|
|
22
|
+
for d in claude_dir.iterdir():
|
|
23
|
+
if d.is_dir() and list(d.glob("*.jsonl")):
|
|
24
|
+
return d
|
|
25
|
+
|
|
26
|
+
# Fallback: look for any .jsonl in the projects dir tree
|
|
27
|
+
for jsonl in claude_dir.rglob("*.jsonl"):
|
|
28
|
+
return jsonl.parent
|
|
29
|
+
|
|
30
|
+
return claude_dir
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def extract_session(jsonl_path: str) -> dict | None:
|
|
34
|
+
"""Extract clean transcript from a session JSONL file."""
|
|
35
|
+
messages = []
|
|
36
|
+
tool_uses = []
|
|
37
|
+
user_msg_count = 0
|
|
38
|
+
|
|
39
|
+
try:
|
|
40
|
+
with open(jsonl_path, "r") as f:
|
|
41
|
+
for line in f:
|
|
42
|
+
line = line.strip()
|
|
43
|
+
if not line:
|
|
44
|
+
continue
|
|
45
|
+
try:
|
|
46
|
+
d = json.loads(line)
|
|
47
|
+
except json.JSONDecodeError:
|
|
48
|
+
continue
|
|
49
|
+
|
|
50
|
+
msg_type = d.get("type")
|
|
51
|
+
|
|
52
|
+
# User messages
|
|
53
|
+
if msg_type == "user":
|
|
54
|
+
content = d.get("message", {}).get("content", "")
|
|
55
|
+
if isinstance(content, str) and content.strip():
|
|
56
|
+
if content.startswith("<system-reminder>"):
|
|
57
|
+
continue
|
|
58
|
+
messages.append({
|
|
59
|
+
"role": "user",
|
|
60
|
+
"text": content[:5000],
|
|
61
|
+
"uuid": d.get("uuid", "")
|
|
62
|
+
})
|
|
63
|
+
user_msg_count += 1
|
|
64
|
+
|
|
65
|
+
# Assistant messages
|
|
66
|
+
elif msg_type in ("message", "assistant"):
|
|
67
|
+
msg = d.get("message", {})
|
|
68
|
+
content_blocks = msg.get("content", [])
|
|
69
|
+
text_parts = []
|
|
70
|
+
for block in content_blocks:
|
|
71
|
+
if isinstance(block, dict):
|
|
72
|
+
if block.get("type") == "text":
|
|
73
|
+
text_parts.append(block.get("text", ""))
|
|
74
|
+
elif block.get("type") == "tool_use":
|
|
75
|
+
tool_uses.append({
|
|
76
|
+
"tool": block.get("name", ""),
|
|
77
|
+
"input_keys": list(block.get("input", {}).keys()) if isinstance(block.get("input"), dict) else [],
|
|
78
|
+
"file": block.get("input", {}).get("file_path", "") or block.get("input", {}).get("command", "")[:100] if isinstance(block.get("input"), dict) else ""
|
|
79
|
+
})
|
|
80
|
+
if text_parts:
|
|
81
|
+
combined = "\n".join(text_parts)[:5000]
|
|
82
|
+
messages.append({
|
|
83
|
+
"role": "assistant",
|
|
84
|
+
"text": combined
|
|
85
|
+
})
|
|
86
|
+
|
|
87
|
+
except Exception as e:
|
|
88
|
+
print(f"Error reading {jsonl_path}: {e}", file=sys.stderr)
|
|
89
|
+
return None
|
|
90
|
+
|
|
91
|
+
if user_msg_count < MIN_USER_MESSAGES:
|
|
92
|
+
return None
|
|
93
|
+
|
|
94
|
+
return {
|
|
95
|
+
"session_file": os.path.basename(jsonl_path),
|
|
96
|
+
"message_count": len(messages),
|
|
97
|
+
"user_message_count": user_msg_count,
|
|
98
|
+
"tool_use_count": len(tool_uses),
|
|
99
|
+
"messages": messages,
|
|
100
|
+
"tool_uses": tool_uses
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
def collect_date(target_date: str, sessions_dir: Path) -> list[dict]:
|
|
105
|
+
"""Collect all sessions modified on a given date."""
|
|
106
|
+
sessions = []
|
|
107
|
+
for f in sessions_dir.glob("*.jsonl"):
|
|
108
|
+
mtime = datetime.fromtimestamp(f.stat().st_mtime)
|
|
109
|
+
if mtime.strftime("%Y-%m-%d") == target_date:
|
|
110
|
+
session = extract_session(str(f))
|
|
111
|
+
if session:
|
|
112
|
+
session["modified"] = mtime.isoformat()
|
|
113
|
+
sessions.append(session)
|
|
114
|
+
sessions.sort(key=lambda s: s["modified"])
|
|
115
|
+
return sessions
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
def main():
|
|
119
|
+
date_arg = sys.argv[1] if len(sys.argv) > 1 else datetime.now().strftime("%Y-%m-%d")
|
|
120
|
+
sessions_dir = find_sessions_dir()
|
|
121
|
+
|
|
122
|
+
sessions = collect_date(date_arg, sessions_dir)
|
|
123
|
+
|
|
124
|
+
output = {
|
|
125
|
+
"date": date_arg,
|
|
126
|
+
"sessions_found": len(sessions),
|
|
127
|
+
"total_messages": sum(s["message_count"] for s in sessions),
|
|
128
|
+
"total_tool_uses": sum(s["tool_use_count"] for s in sessions),
|
|
129
|
+
"sessions": sessions
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
output_dir = Path.home() / "claude" / "operations" / "deep-sleep"
|
|
133
|
+
output_dir.mkdir(parents=True, exist_ok=True)
|
|
134
|
+
output_file = output_dir / f"{output['date']}-transcripts.json"
|
|
135
|
+
with open(output_file, "w") as f:
|
|
136
|
+
json.dump(output, f, indent=2, ensure_ascii=False)
|
|
137
|
+
|
|
138
|
+
print(f"Collected {len(sessions)} sessions, {output['total_messages']} messages, {output['total_tool_uses']} tool uses")
|
|
139
|
+
print(f"Output: {output_file}")
|
|
140
|
+
|
|
141
|
+
|
|
142
|
+
if __name__ == "__main__":
|
|
143
|
+
main()
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# Deep Sleep Analyst — Session Transcript Analysis
|
|
2
|
+
|
|
3
|
+
You are NEXO's overnight analyst. You read the COMPLETE transcripts of today's sessions between Francisco and NEXO, and you find what NEXO missed.
|
|
4
|
+
|
|
5
|
+
## Your job
|
|
6
|
+
|
|
7
|
+
NEXO captures feedback, learnings, and corrections during sessions — but it misses things. Your job is to find the gaps by reading what ACTUALLY happened (the transcript), not what NEXO thinks happened (the diary).
|
|
8
|
+
|
|
9
|
+
## What you analyze
|
|
10
|
+
|
|
11
|
+
### 1. Uncaptured corrections
|
|
12
|
+
Francisco corrected NEXO but NEXO didn't save a learning or feedback memory.
|
|
13
|
+
Signals: "no", "mal", "eso no es", "por dios", "no ves que", "pero que coño", frustration tone, repeating the same instruction 2+ times, Francisco having to explain something twice.
|
|
14
|
+
|
|
15
|
+
### 2. Repeated patterns
|
|
16
|
+
The same correction appears multiple times in the day. This is a SYSTEMIC failure — it needs a strong learning with high severity.
|
|
17
|
+
|
|
18
|
+
### 3. Uncaptured ideas
|
|
19
|
+
Francisco mentioned an idea, plan, or intention that nobody formalized. Signals: "podríamos", "habría que", "molaría", "quiero", "necesito".
|
|
20
|
+
|
|
21
|
+
### 4. Missed commitments
|
|
22
|
+
Francisco said "lo miro mañana", "esta semana", "cuando pueda" — was a followup created? If not, flag it.
|
|
23
|
+
|
|
24
|
+
### 5. Protocol compliance (from tool_uses)
|
|
25
|
+
Check if NEXO followed its own protocols:
|
|
26
|
+
- `nexo_guard_check` before Edit/Write on production files?
|
|
27
|
+
- `nexo_heartbeat` called with meaningful context_hint?
|
|
28
|
+
- `nexo_cognitive_trust` called after corrections?
|
|
29
|
+
- `nexo_learning_add` called after resolving errors?
|
|
30
|
+
- `nexo_followup_complete` called when Francisco said "ya está"/"hecho"?
|
|
31
|
+
- `nexo_change_log` called after production code changes?
|
|
32
|
+
- Feedback memory saved after corrections?
|
|
33
|
+
|
|
34
|
+
### 6. Quality assessment
|
|
35
|
+
- Did NEXO declare "perfecto"/"completado" and Francisco had to correct after?
|
|
36
|
+
- Was NEXO too verbose when Francisco wanted action?
|
|
37
|
+
- Did NEXO delegate to subagents when it should have done the work directly?
|
|
38
|
+
|
|
39
|
+
## Output format
|
|
40
|
+
|
|
41
|
+
Return ONLY valid JSON:
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{
|
|
45
|
+
"date": "YYYY-MM-DD",
|
|
46
|
+
"sessions_analyzed": 5,
|
|
47
|
+
"uncaptured_corrections": [
|
|
48
|
+
{
|
|
49
|
+
"quote": "Francisco's exact words (max 100 chars)",
|
|
50
|
+
"context": "What they were working on",
|
|
51
|
+
"what_nexo_should_have_saved": "The learning/feedback content",
|
|
52
|
+
"action": "learning_add|feedback_write|preference_set",
|
|
53
|
+
"category": "ui|code|process|communication",
|
|
54
|
+
"severity": "low|medium|high|critical",
|
|
55
|
+
"times_repeated": 1
|
|
56
|
+
}
|
|
57
|
+
],
|
|
58
|
+
"uncaptured_ideas": [
|
|
59
|
+
{
|
|
60
|
+
"quote": "Francisco's words",
|
|
61
|
+
"idea": "What the idea is",
|
|
62
|
+
"action": "reminder_create|followup_create",
|
|
63
|
+
"suggested_date": "YYYY-MM-DD or null"
|
|
64
|
+
}
|
|
65
|
+
],
|
|
66
|
+
"missed_commitments": [
|
|
67
|
+
{
|
|
68
|
+
"quote": "Francisco's words",
|
|
69
|
+
"commitment": "What was promised",
|
|
70
|
+
"action": "followup_create",
|
|
71
|
+
"due_date": "YYYY-MM-DD"
|
|
72
|
+
}
|
|
73
|
+
],
|
|
74
|
+
"protocol_compliance": {
|
|
75
|
+
"guard_check": {"required": 0, "executed": 0, "rate": 1.0},
|
|
76
|
+
"heartbeat_quality": {"total": 0, "with_good_context": 0, "rate": 1.0},
|
|
77
|
+
"trust_adjustments": {"corrections_detected": 0, "adjusted": 0, "rate": 1.0},
|
|
78
|
+
"learning_capture": {"errors_resolved": 0, "captured": 0, "rate": 1.0},
|
|
79
|
+
"change_log": {"production_edits": 0, "logged": 0, "rate": 1.0},
|
|
80
|
+
"feedback_capture": {"corrections": 0, "captured": 0, "rate": 1.0},
|
|
81
|
+
"overall_compliance": 1.0
|
|
82
|
+
},
|
|
83
|
+
"protocol_violations": [
|
|
84
|
+
{
|
|
85
|
+
"protocol": "guard_check|trust_adjustment|feedback_capture|...",
|
|
86
|
+
"context": "What happened",
|
|
87
|
+
"severity": "low|medium|high|critical"
|
|
88
|
+
}
|
|
89
|
+
],
|
|
90
|
+
"quality_issues": [
|
|
91
|
+
{
|
|
92
|
+
"issue": "Description of quality problem",
|
|
93
|
+
"example": "Specific instance",
|
|
94
|
+
"severity": "low|medium|high"
|
|
95
|
+
}
|
|
96
|
+
],
|
|
97
|
+
"auto_reinforcements": [
|
|
98
|
+
"Specific rule to add or reinforce in CLAUDE.md or guard"
|
|
99
|
+
],
|
|
100
|
+
"summary": "2-3 sentence overall assessment of the day"
|
|
101
|
+
}
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Rules
|
|
105
|
+
- Be SPECIFIC. Quote Francisco's exact words.
|
|
106
|
+
- Only flag REAL issues. If NEXO did capture something correctly, don't flag it.
|
|
107
|
+
- severity=critical means Francisco repeated the same correction 3+ times or expressed strong frustration
|
|
108
|
+
- For protocol compliance, count ACTUAL tool_use entries in the transcript
|
|
109
|
+
- If no issues found in a category, return empty array — don't invent problems
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# NEXO Deep Sleep — Complete overnight session transcript analysis
|
|
3
|
+
# Reads ALL Claude Code session transcripts from the day, analyzes with
|
|
4
|
+
# Claude CLI (bare mode), and applies findings as feedback memories.
|
|
5
|
+
#
|
|
6
|
+
# Features:
|
|
7
|
+
# - Catch-up: if yesterday was missed (Mac off/asleep), runs it first
|
|
8
|
+
# - Uses --bare mode to avoid loading NEXO hooks during analysis
|
|
9
|
+
# - Requires ANTHROPIC_API_KEY env var or ~/.claude/anthropic-api-key.txt
|
|
10
|
+
#
|
|
11
|
+
# Install: Add as LaunchAgent for daily execution (recommended: 4:30 AM)
|
|
12
|
+
|
|
13
|
+
set -euo pipefail
|
|
14
|
+
|
|
15
|
+
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
16
|
+
LOG_DIR="$HOME/claude/logs"
|
|
17
|
+
DEEP_SLEEP_DIR="$HOME/claude/operations/deep-sleep"
|
|
18
|
+
LAST_RUN_FILE="$DEEP_SLEEP_DIR/.last-run"
|
|
19
|
+
TODAY=$(date +%Y-%m-%d)
|
|
20
|
+
|
|
21
|
+
mkdir -p "$LOG_DIR" "$DEEP_SLEEP_DIR"
|
|
22
|
+
|
|
23
|
+
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_DIR/deep-sleep.log"; }
|
|
24
|
+
|
|
25
|
+
run_analysis() {
|
|
26
|
+
local DATE="$1"
|
|
27
|
+
log "=== Deep Sleep starting for $DATE ==="
|
|
28
|
+
|
|
29
|
+
log "Step 1: Collecting transcripts for $DATE..."
|
|
30
|
+
python3 "$SCRIPT_DIR/deep-sleep/collect_transcripts.py" "$DATE" 2>&1 | tee -a "$LOG_DIR/deep-sleep.log"
|
|
31
|
+
|
|
32
|
+
if [ ! -f "$DEEP_SLEEP_DIR/$DATE-transcripts.json" ]; then
|
|
33
|
+
log "No transcripts file generated for $DATE. Skipping."
|
|
34
|
+
return 0
|
|
35
|
+
fi
|
|
36
|
+
|
|
37
|
+
SESSIONS=$(python3 -c "import json; print(json.load(open('$DEEP_SLEEP_DIR/$DATE-transcripts.json'))['sessions_found'])")
|
|
38
|
+
if [ "$SESSIONS" -eq 0 ]; then
|
|
39
|
+
log "No sessions found for $DATE. Skipping."
|
|
40
|
+
return 0
|
|
41
|
+
fi
|
|
42
|
+
|
|
43
|
+
log "Step 2: Analyzing $SESSIONS sessions with Claude CLI..."
|
|
44
|
+
python3 "$SCRIPT_DIR/deep-sleep/analyze_session.py" "$DATE" 2>&1 | tee -a "$LOG_DIR/deep-sleep.log"
|
|
45
|
+
|
|
46
|
+
if [ ! -f "$DEEP_SLEEP_DIR/$DATE-analysis.json" ]; then
|
|
47
|
+
log "Analysis failed for $DATE. No output generated."
|
|
48
|
+
return 1
|
|
49
|
+
fi
|
|
50
|
+
|
|
51
|
+
log "Step 3: Applying findings for $DATE..."
|
|
52
|
+
python3 "$SCRIPT_DIR/deep-sleep/apply_findings.py" "$DATE" 2>&1 | tee -a "$LOG_DIR/deep-sleep.log"
|
|
53
|
+
|
|
54
|
+
log "=== Deep Sleep complete for $DATE ==="
|
|
55
|
+
return 0
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
# --- Catch-up: check if yesterday was missed ---
|
|
59
|
+
YESTERDAY=$(date -v-1d +%Y-%m-%d 2>/dev/null || date -d "yesterday" +%Y-%m-%d 2>/dev/null)
|
|
60
|
+
LAST_RUN=""
|
|
61
|
+
if [ -f "$LAST_RUN_FILE" ]; then
|
|
62
|
+
LAST_RUN=$(cat "$LAST_RUN_FILE")
|
|
63
|
+
fi
|
|
64
|
+
|
|
65
|
+
if [ -n "$YESTERDAY" ] && [ "$LAST_RUN" != "$YESTERDAY" ] && [ "$LAST_RUN" != "$TODAY" ]; then
|
|
66
|
+
if [ ! -f "$DEEP_SLEEP_DIR/$YESTERDAY-analysis.json" ]; then
|
|
67
|
+
log "*** CATCH-UP: $YESTERDAY was missed. Running now. ***"
|
|
68
|
+
run_analysis "$YESTERDAY" || log "Catch-up for $YESTERDAY failed."
|
|
69
|
+
fi
|
|
70
|
+
fi
|
|
71
|
+
|
|
72
|
+
# --- Run today's analysis ---
|
|
73
|
+
run_analysis "$TODAY"
|
|
74
|
+
|
|
75
|
+
# Mark completion
|
|
76
|
+
echo "$TODAY" > "$LAST_RUN_FILE"
|
|
@@ -139,6 +139,80 @@ try_repair_cron() {
|
|
|
139
139
|
return 1
|
|
140
140
|
}
|
|
141
141
|
|
|
142
|
+
try_reexecute_missed_cron() {
|
|
143
|
+
# Re-execute a cron that missed its scheduled run
|
|
144
|
+
local plist_id="$1"
|
|
145
|
+
local plist_file="$HOME_DIR/Library/LaunchAgents/${plist_id}.plist"
|
|
146
|
+
|
|
147
|
+
if [ ! -f "$plist_file" ]; then
|
|
148
|
+
return 1
|
|
149
|
+
fi
|
|
150
|
+
|
|
151
|
+
local cmd
|
|
152
|
+
cmd=$(python3 -c "
|
|
153
|
+
import plistlib, sys
|
|
154
|
+
try:
|
|
155
|
+
with open('$plist_file', 'rb') as f:
|
|
156
|
+
d = plistlib.load(f)
|
|
157
|
+
if d.get('KeepAlive'):
|
|
158
|
+
sys.exit(1)
|
|
159
|
+
if not d.get('StartCalendarInterval') and not d.get('StartInterval'):
|
|
160
|
+
sys.exit(1)
|
|
161
|
+
print(' '.join(d.get('ProgramArguments', [])))
|
|
162
|
+
except:
|
|
163
|
+
sys.exit(1)
|
|
164
|
+
" 2>/dev/null)
|
|
165
|
+
|
|
166
|
+
if [ -z "$cmd" ] || [ $? -ne 0 ]; then
|
|
167
|
+
return 1
|
|
168
|
+
fi
|
|
169
|
+
|
|
170
|
+
log "Re-executing missed cron: $plist_id"
|
|
171
|
+
timeout 300 bash -c "$cmd" >> "$LOG_DIR/watchdog-reexec.log" 2>&1 &
|
|
172
|
+
local pid=$!
|
|
173
|
+
sleep 2
|
|
174
|
+
if kill -0 "$pid" 2>/dev/null || wait "$pid" 2>/dev/null; then
|
|
175
|
+
log_repair "$plist_id: re-executed missed cron (PID $pid)"
|
|
176
|
+
return 0
|
|
177
|
+
fi
|
|
178
|
+
return 1
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
try_verify_repair() {
|
|
182
|
+
# After Level 2 repair, verify the service is healthy
|
|
183
|
+
local plist_id="$1"
|
|
184
|
+
local log_stdout="$2"
|
|
185
|
+
local proc_grep="$3"
|
|
186
|
+
local max_wait=30
|
|
187
|
+
|
|
188
|
+
if ! is_loaded "$plist_id"; then
|
|
189
|
+
return 1
|
|
190
|
+
fi
|
|
191
|
+
|
|
192
|
+
if [ -n "$proc_grep" ]; then
|
|
193
|
+
local waited=0
|
|
194
|
+
while [ $waited -lt $max_wait ]; do
|
|
195
|
+
if process_running "$proc_grep"; then
|
|
196
|
+
log "Verify OK: $plist_id process running after ${waited}s"
|
|
197
|
+
return 0
|
|
198
|
+
fi
|
|
199
|
+
sleep 5
|
|
200
|
+
waited=$((waited + 5))
|
|
201
|
+
done
|
|
202
|
+
return 1
|
|
203
|
+
fi
|
|
204
|
+
|
|
205
|
+
if [ -n "$log_stdout" ] && [ -f "$log_stdout" ]; then
|
|
206
|
+
local age
|
|
207
|
+
age=$(file_age "$log_stdout")
|
|
208
|
+
if [ "$age" -lt 300 ]; then
|
|
209
|
+
return 0
|
|
210
|
+
fi
|
|
211
|
+
fi
|
|
212
|
+
|
|
213
|
+
return 0
|
|
214
|
+
}
|
|
215
|
+
|
|
142
216
|
try_repair_backup() {
|
|
143
217
|
local backup_script="$NEXO_DIR/backup_cron.sh"
|
|
144
218
|
if [ -x "$backup_script" ]; then
|
|
@@ -263,13 +337,19 @@ for monitor in "${MONITORS[@]}"; do
|
|
|
263
337
|
fi
|
|
264
338
|
fi
|
|
265
339
|
|
|
266
|
-
# Check 3: Log staleness
|
|
340
|
+
# Check 3: Log staleness + AUTO RE-EXECUTE missed crons
|
|
267
341
|
if [ -n "$log_stdout" ] && [ "$max_stale" -gt 0 ]; then
|
|
268
342
|
age=$(file_age "$log_stdout")
|
|
269
343
|
stale_age=$(format_age "$age")
|
|
270
344
|
if [ "$age" -gt $(( max_stale * 3 )) ]; then
|
|
271
|
-
|
|
272
|
-
|
|
345
|
+
if try_reexecute_missed_cron "$plist_id"; then
|
|
346
|
+
status="HEALED"
|
|
347
|
+
details="${details}Self-healed: re-executed missed cron (was stale: $stale_age). "
|
|
348
|
+
TOTAL_HEALED=$((TOTAL_HEALED + 1))
|
|
349
|
+
else
|
|
350
|
+
status="FAIL"
|
|
351
|
+
details="${details}Log stale: $stale_age (limit: $(format_age "$max_stale")). Re-execute failed. "
|
|
352
|
+
fi
|
|
273
353
|
elif [ "$age" -gt "$max_stale" ]; then
|
|
274
354
|
[ "$status" = "PASS" ] && status="WARN"
|
|
275
355
|
details="${details}Log slightly stale: $stale_age. "
|