@geoql/mdr 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/LICENSE +22 -0
  2. package/README.md +152 -0
  3. package/USAGE.md +59 -0
  4. package/bin/detect-user.sh +53 -0
  5. package/bin/index-conversations.ts +34 -0
  6. package/bin/macrodata-daemon.ts +28 -0
  7. package/bin/macrodata-hook.sh +277 -0
  8. package/dist/bin/index-conversations.js +31 -0
  9. package/dist/bin/macrodata-daemon.js +30 -0
  10. package/dist/opencode/context.js +210 -0
  11. package/dist/opencode/conversations.js +367 -0
  12. package/dist/opencode/index.js +155 -0
  13. package/dist/opencode/journal.js +108 -0
  14. package/dist/opencode/logger.js +29 -0
  15. package/dist/opencode/search.js +210 -0
  16. package/dist/opencode/skills/macrodata-distill/SKILL.md +171 -0
  17. package/dist/opencode/skills/macrodata-dreamtime/SKILL.md +120 -0
  18. package/dist/opencode/skills/macrodata-memory-maintenance/SKILL.md +96 -0
  19. package/dist/opencode/skills/macrodata-onboarding/SKILL.md +346 -0
  20. package/dist/opencode/tools.js +367 -0
  21. package/dist/src/config.js +55 -0
  22. package/dist/src/conversations.js +513 -0
  23. package/dist/src/daemon.js +582 -0
  24. package/dist/src/detect-user.js +73 -0
  25. package/dist/src/embeddings.js +190 -0
  26. package/dist/src/index.js +413 -0
  27. package/dist/src/indexer.js +286 -0
  28. package/opencode/context.ts +322 -0
  29. package/opencode/conversations.ts +467 -0
  30. package/opencode/index.ts +208 -0
  31. package/opencode/journal.ts +153 -0
  32. package/opencode/logger.ts +32 -0
  33. package/opencode/search.ts +288 -0
  34. package/opencode/skills/macrodata-distill/SKILL.md +171 -0
  35. package/opencode/skills/macrodata-dreamtime/SKILL.md +120 -0
  36. package/opencode/skills/macrodata-memory-maintenance/SKILL.md +96 -0
  37. package/opencode/skills/macrodata-onboarding/SKILL.md +346 -0
  38. package/opencode/tools.ts +453 -0
  39. package/package.json +87 -0
  40. package/src/config.ts +66 -0
  41. package/src/conversations.ts +709 -0
  42. package/src/daemon.ts +785 -0
  43. package/src/detect-user.ts +97 -0
  44. package/src/embeddings.ts +262 -0
  45. package/src/index.ts +726 -0
  46. package/src/indexer.ts +394 -0
@@ -0,0 +1,120 @@
1
+ ---
2
+ name: macrodata-dreamtime
3
+ description: Deep nightly reflection. Self-improvement, research, pattern recognition. Runs in background with no user interaction.
4
+ ---
5
+
6
+ # Dreamtime
7
+
8
+ Dedicated time for deep reflection and self-improvement. Runs automatically late at night when not interrupting work.
9
+
10
+ **Important:** This runs in the background with no user interaction. Think freely, research openly, update files directly.
11
+
12
+ ## Purpose
13
+
14
+ This is your time to think beyond immediate tasks. Reflect on your role, learn new things, notice patterns, and improve how you work.
15
+
16
+ ## Areas to Explore
17
+
18
+ ### Identity & Role
19
+
20
+ Read `identity.md` and reflect:
21
+
22
+ - Does this still match how you actually work?
23
+ - Any values or patterns to add or refine?
24
+ - Has the relationship with the user evolved?
25
+
26
+ Update identity.md if insights warrant it.
27
+
28
+ ### Understanding the Human
29
+
30
+ Review recent interactions and `human.md`:
31
+
32
+ - New communication patterns observed?
33
+ - Preferences you've learned implicitly?
34
+ - Working style insights?
35
+ - Topics they care about that aren't documented?
36
+
37
+ Update human.md with genuine new understanding.
38
+
39
+ ### Pattern Recognition
40
+
41
+ Look across recent journals, conversations, and work:
42
+
43
+ - Recurring questions or frustrations?
44
+ - Themes connecting different projects?
45
+ - Problems that keep coming back?
46
+ - Workflows that could be smoother?
47
+
48
+ Document patterns in topics/ or journal.
49
+
50
+ ### Knowledge Gaps
51
+
52
+ Think about recent work:
53
+
54
+ - What came up that you didn't know well?
55
+ - Areas where you felt uncertain?
56
+ - Technologies or concepts to understand better?
57
+
58
+ Use web search to research:
59
+
60
+ - Read documentation
61
+ - Find articles or posts
62
+ - Understand concepts more deeply
63
+
64
+ Journal what you learned.
65
+
66
+ ### Open Threads
67
+
68
+ Review workspace.md and recent context:
69
+
70
+ - Anything left unresolved?
71
+ - Questions raised but not answered?
72
+ - Ideas mentioned but not explored?
73
+
74
+ Update workspace.md open threads section.
75
+
76
+ ### Project Connections
77
+
78
+ Look across projects in entities/projects/:
79
+
80
+ - Links between different work?
81
+ - Patterns suggesting bigger themes?
82
+ - Reusable learnings from one project to another?
83
+
84
+ ### Tech Landscape
85
+
86
+ For the user's primary technologies:
87
+
88
+ - Any relevant news or updates?
89
+ - New tools or approaches worth knowing?
90
+ - Deprecations or changes to be aware of?
91
+
92
+ Use web search to scan relevant sources.
93
+
94
+ ### Tool Effectiveness
95
+
96
+ Reflect on memory system usage:
97
+
98
+ - Are the tools being used well?
99
+ - Any friction in the workflows?
100
+ - Ideas for improvement?
101
+
102
+ Journal observations.
103
+
104
+ ## Output
105
+
106
+ Update relevant files directly:
107
+
108
+ - identity.md - role/values refinements
109
+ - human.md - new understanding
110
+ - workspace.md - open threads
111
+ - topics/ - new or updated topics
112
+ - entities/ - project/people updates
113
+ - journal - learnings, observations, ideas
114
+
115
+ Write a dreamtime journal entry summarizing:
116
+
117
+ ```
118
+ topic: dreamtime
119
+ content: [key reflections, what was researched, what was updated]
120
+ ```
@@ -0,0 +1,96 @@
1
+ ---
2
+ name: macrodata-memory-maintenance
3
+ description: End of day memory maintenance. Runs distillation, updates state files, prunes stale info. Runs in background with no user interaction.
4
+ ---
5
+
6
+ # Memory Maintenance
7
+
8
+ Scheduled maintenance to keep memory current and useful. Runs automatically at end of day.
9
+
10
+ **Important:** This runs in the background with no user interaction. Do not ask questions - make decisions and note uncertainties in the journal.
11
+
12
+ ## Process
13
+
14
+ ### 1. Run Distillation
15
+
16
+ First, run the `macrodata-distill` skill to extract facts from today's conversations.
17
+
18
+ This processes all OpenCode sessions, spawns sub-agents for extraction, and writes distilled actions to the journal.
19
+
20
+ **Check if distill already ran today:**
21
+ Use `get_recent_journal` and look for topic "distill-summary" from today.
22
+
23
+ If not found, load `macrodata-distill` skill. If already ran, skip to step 2.
24
+
25
+ ### 2. Review Distilled Content
26
+
27
+ Use `get_recent_journal` filtered to topic "distilled" to see what was extracted.
28
+
29
+ Use these to inform state file updates.
30
+
31
+ ### 3. State File Updates
32
+
33
+ Review each state file and update if needed:
34
+
35
+ **today.md**
36
+
37
+ - Clear completed items
38
+ - Note anything that carried over
39
+ - Leave empty or minimal for morning prep to fill
40
+
41
+ **workspace.md**
42
+
43
+ - Update active projects list based on distilled actions
44
+ - Add/remove open threads
45
+ - Note any blocked items or waiting-on dependencies
46
+
47
+ **human.md**
48
+
49
+ - Any new preferences or patterns from distilled facts?
50
+ - Communication style insights?
51
+ - Only update if genuinely new information
52
+
53
+ ### 4. Entity Updates
54
+
55
+ Review `entities/people/` and `entities/projects/`:
56
+
57
+ - Integrate any facts extracted by distillation
58
+ - Project status changes?
59
+ - New projects to create files for?
60
+
61
+ ### 5. Prune Stale Info
62
+
63
+ Look for outdated information:
64
+
65
+ - Completed todos still listed as active
66
+ - Old context that's no longer relevant
67
+ - Temporary notes that should be removed
68
+ - Duplicated information across files
69
+
70
+ Remove or archive as appropriate.
71
+
72
+ ### 6. Index Maintenance
73
+
74
+ Check if indexes need rebuilding:
75
+
76
+ ```
77
+ manage_index(target="memory", action="stats")
78
+ manage_index(target="conversations", action="stats")
79
+ ```
80
+
81
+ If counts seem low or stale, trigger rebuild:
82
+
83
+ ```
84
+ manage_index(target="memory", action="rebuild")
85
+ manage_index(target="conversations", action="update")
86
+ ```
87
+
88
+ ### 7. Journal Summary
89
+
90
+ Write a brief maintenance journal entry:
91
+
92
+ ```
93
+ macrodata_log_journal(topic="maintenance", content="[what was updated, what was pruned, any observations]")
94
+ ```
95
+
96
+ Note anything uncertain that should be confirmed with the user next session.
@@ -0,0 +1,346 @@
1
+ ---
2
+ name: macrodata-onboarding
3
+ description: Guide new users through macrodata setup. Creates identity, human profile, and workspace files. Use when get_context returns isFirstRun true, or user asks to set up their profile.
4
+ ---
5
+
6
+ # Onboarding Skill
7
+
8
+ Guide new users through initial macrodata setup.
9
+
10
+ ## When to Use
11
+
12
+ - `get_context` returns `isFirstRun: true`
13
+ - User explicitly asks to set up or reset their profile
14
+ - State files are empty or missing
15
+
16
+ ## Onboarding Flow
17
+
18
+ ### Phase 1: User Info
19
+
20
+ User info has been pre-detected and is available in the context above as "Detected User Info". This JSON contains:
21
+
22
+ - `username`, `fullName`, `timezone`
23
+ - `git.name`, `git.email`
24
+ - `github.login`, `github.name`, `github.blog`, `github.bio`
25
+ - `codeDirs` - array of existing code directories
26
+
27
+ Use this data throughout onboarding - no need to run detection scripts.
28
+
29
+ ### Phase 2: Location
30
+
31
+ Offer location options. Always include:
32
+
33
+ - `~/Documents/macrodata` - easy to find
34
+ - `~/.config/macrodata` - hidden, default
35
+
36
+ Only include a code directory option if `codeDirs` from the detection was non-empty.
37
+
38
+ If they choose a non-default location, write the config to `~/.config/macrodata/config.json`:
39
+
40
+ ```json
41
+ {
42
+ "root": "/chosen/path"
43
+ }
44
+ ```
45
+
46
+ After writing the config, signal the daemon to reload:
47
+
48
+ ```bash
49
+ kill -HUP $(cat ~/.config/macrodata/.daemon.pid) 2>/dev/null || true
50
+ ```
51
+
52
+ Then create the directory structure:
53
+
54
+ - `<root>/state/`
55
+ - `<root>/journal/`
56
+ - `<root>/entities/people/`
57
+ - `<root>/entities/projects/`
58
+ - `<root>/topics/`
59
+
60
+ ### Phase 3: Human Profile
61
+
62
+ Use the info from the detection script to pre-populate.
63
+
64
+ **Ask the basics:**
65
+
66
+ - What should I call you? (confirm or correct auto-detected name)
67
+ - What's your GitHub username? (if not detected from gh cli)
68
+ - Do you have a website or blog?
69
+ - Any social profiles you'd like me to know about?
70
+
71
+ **Communication style:**
72
+ If they consent, analyze their OpenCode session history (`~/.local/share/opencode/storage/`):
73
+
74
+ ```bash
75
+ # Extract human messages from OpenCode session history
76
+ # Messages are stored in part/ directory, organized by message ID
77
+ # User messages have role: "user" in the parent message metadata
78
+
79
+ # Find recent user message parts
80
+ for msg_dir in $(ls -t ~/.local/share/opencode/storage/message/ 2>/dev/null | head -20); do
81
+ # Get message metadata
82
+ cat ~/.local/share/opencode/storage/message/$msg_dir/*.json 2>/dev/null | \
83
+ jq -r 'select(.role == "user") | .id' | while read msg_id; do
84
+ # Get the text parts for this message
85
+ cat ~/.local/share/opencode/storage/part/$msg_id/*.json 2>/dev/null | \
86
+ jq -r 'select(.type == "text" and .synthetic != true) | .text' 2>/dev/null
87
+ done
88
+ done | head -200
89
+ ```
90
+
91
+ Look for patterns:
92
+
93
+ - Message length (short/direct vs detailed)
94
+ - Tone (casual, formal, technical)
95
+ - How they give feedback (direct corrections, suggestions, questions)
96
+ - Language preferences (spelling variants, idioms)
97
+
98
+ **Current work context:**
99
+ Analyze recent session history to understand what they're working on:
100
+
101
+ ```bash
102
+ # Get recent project directories from OpenCode session storage
103
+ ls -t ~/.local/share/opencode/storage/session/ 2>/dev/null | head -10 | while read proj_dir; do
104
+ cat ~/.local/share/opencode/storage/session/$proj_dir/*.json 2>/dev/null | \
105
+ jq -r '.directory, .title' 2>/dev/null
106
+ done
107
+
108
+ # Sample recent conversations for context (last 7 days)
109
+ find ~/.local/share/opencode/storage/part -name "*.json" -mtime -7 -exec cat {} \; 2>/dev/null | \
110
+ jq -r 'select(.type == "text" and .synthetic != true) | .text' 2>/dev/null | \
111
+ head -100
112
+ ```
113
+
114
+ **Working patterns:**
115
+
116
+ - Ask about current focus areas (or confirm what you detected)
117
+ - Any preferences for how the agent should work?
118
+
119
+ Write findings to `state/human.md`:
120
+
121
+ ```markdown
122
+ # Human Profile
123
+
124
+ ## Basics
125
+
126
+ - **Name:** [name]
127
+ - **GitHub:** [username]
128
+ - **Website:** [url if provided]
129
+ - **Socials:** [any provided]
130
+ - **Timezone:** [detected]
131
+
132
+ ## Communication Style
133
+
134
+ - [observed patterns from analysis]
135
+ - [stated preferences]
136
+
137
+ ## Working Patterns
138
+
139
+ - [current focus areas]
140
+ - [preferences]
141
+
142
+ ## Current Projects
143
+
144
+ - [detected from recent sessions]
145
+
146
+ ## Pending Items
147
+
148
+ - [empty initially]
149
+ ```
150
+
151
+ ### Phase 4: Agent Identity
152
+
153
+ Help define who the agent should be:
154
+
155
+ **Name and persona:**
156
+
157
+ - What should the agent be called?
158
+ - What's its role? (assistant, partner, specialist)
159
+ - Any personality traits?
160
+
161
+ **Values and patterns:**
162
+
163
+ - What behaviors should it prioritize?
164
+ - How proactive should it be?
165
+
166
+ Write to `state/identity.md`:
167
+
168
+ ```markdown
169
+ # [Agent Name] Identity
170
+
171
+ ## Persona
172
+
173
+ [Description of who the agent is, its role, personality]
174
+
175
+ ## Values
176
+
177
+ - [core value 1]
178
+ - [core value 2]
179
+
180
+ ## Patterns
181
+
182
+ - [behavioral pattern 1]
183
+ - [behavioral pattern 2]
184
+ ```
185
+
186
+ ### Phase 5: Initial Workspace
187
+
188
+ Set up working context:
189
+
190
+ 1. Ask what they're currently working on
191
+ 2. Create initial project files in `entities/projects/`
192
+ 3. Write `state/today.md` with current context
193
+ 4. Write `state/workspace.md` with active projects
194
+
195
+ ```markdown
196
+ # Today
197
+
198
+ ## Now
199
+
200
+ [Current context from conversation]
201
+
202
+ ## Context
203
+
204
+ [Background information]
205
+ ```
206
+
207
+ ```markdown
208
+ # Workspace
209
+
210
+ ## Active Projects
211
+
212
+ - [project 1] - [brief description]
213
+
214
+ ## Open Threads
215
+
216
+ - [things in progress]
217
+ ```
218
+
219
+ ### Phase 6: Permissions
220
+
221
+ Ask if they'd like to pre-grant permissions for macrodata paths. This avoids permission prompts every session.
222
+
223
+ **Ask:** "Would you like me to update your OpenCode settings to pre-grant permissions for macrodata? This means you won't be prompted each time macrodata reads or writes to its memory folder."
224
+
225
+ If yes, update `~/.config/opencode/opencode.json` to add these permissions:
226
+
227
+ ```json
228
+ {
229
+ "permission": {
230
+ "external_directory": {
231
+ "<macrodata-root>/**": "allow"
232
+ }
233
+ }
234
+ }
235
+ ```
236
+
237
+ **Important:** Replace `<macrodata-root>` with their actual chosen root path (e.g., `~/Documents/macrodata`).
238
+
239
+ Merge with existing settings rather than overwriting. Read the file, add the new permissions to the existing `permission` object, and write back.
240
+
241
+ ### Phase 7: Scheduled Reminders
242
+
243
+ **First, check available integrations:**
244
+
245
+ Look at the tools and plugins available in your current context. Note any that might be useful for scheduled tasks - for example:
246
+
247
+ - Calendar integrations → could check meetings in morning prep
248
+ - Email/messaging tools → could summarize communications
249
+ - Task managers → could review todos
250
+ - Project management tools → could check status
251
+
252
+ Don't hardcode specific tool names - just note what categories are available. Mention relevant ones when describing the schedule options below.
253
+
254
+ **Then offer scheduled reminders:**
255
+
256
+ Offer optional scheduled reminders. These run in the background with no user interaction.
257
+
258
+ **Options to offer:**
259
+
260
+ 1. **Morning prep** (daily, 9am) - Quick review to set the day's focus. Updates today.md.
261
+
262
+ 2. **Memory maintenance** (daily, 6pm) - End of day cleanup. Reviews journals, updates state files, prunes stale info. Runs the `macrodata-memory-maintenance` skill.
263
+
264
+ 3. **Dreamtime** (daily, 2am) - Deep nightly reflection. Self-improvement, research, pattern recognition. Runs the `macrodata-dreamtime` skill.
265
+
266
+ For each selected reminder, use the `schedule` tool to create it. **Important:** For deep thinking tasks (dreamtime, memory maintenance), specify a capable model using the `model` parameter. Choose a model appropriate for reflection and reasoning - the user's most capable available model is best for these tasks.
267
+
268
+ ```
269
+ Morning prep:
270
+ - id: morning-prep
271
+ - type: cron
272
+ - expression: 0 9 * * *
273
+ - description: Morning prep - update today.md
274
+ - payload: Review today.md. What's the focus for today? Any carryover from yesterday? Check if any calendar, email, or task tools are available and use them to see what's planned. Keep it brief.
275
+ - model: (optional - default is fine for quick tasks)
276
+
277
+ Memory maintenance:
278
+ - id: macrodata-memory-maintenance
279
+ - type: cron
280
+ - expression: 0 18 * * *
281
+ - description: End of day memory maintenance
282
+ - payload: Run the macrodata-memory-maintenance skill.
283
+ - model: (use a capable model for reflection tasks)
284
+
285
+ Dreamtime:
286
+ - id: macrodata-dreamtime
287
+ - type: cron
288
+ - expression: 0 2 * * *
289
+ - description: Nightly reflection and self-improvement
290
+ - payload: Run the macrodata-dreamtime skill.
291
+ - model: (use the most capable available model for deep thinking)
292
+ ```
293
+
294
+ ### Phase 8: Finalize
295
+
296
+ 1. Rebuild the memory index with `manage_index`
297
+ 2. Log completion to journal
298
+ 3. Summarize what was created
299
+ 4. Suggest next steps
300
+
301
+ ## Session History Analysis
302
+
303
+ If the user consents, analyze their OpenCode history for context:
304
+
305
+ **Context to extract:**
306
+
307
+ - Recent project directories they've been working in
308
+ - Common file types and technologies
309
+ - Recurring themes or topics
310
+
311
+ ## Example Conversation
312
+
313
+ **Agent:** Welcome to macrodata! Let's get you set up.
314
+
315
+ First, where would you like me to store your memory files?
316
+
317
+ 1. `~/Documents/macrodata` (easy to find in Finder)
318
+ 2. `~/.config/macrodata` (hidden, default)
319
+
320
+ _[If a code directory like ~/Code or ~/Projects exists, also offer that as an option]_
321
+
322
+ **User:** Documents is fine
323
+
324
+ **Agent:** Got it. Now, what should I call you?
325
+
326
+ **User:** Alex
327
+
328
+ **Agent:** I can see from your system that you're [name from git config], GitHub username [from gh cli], and you're in the [timezone] timezone. Is that right?
329
+
330
+ **User:** Yeah
331
+
332
+ **Agent:** Do you have a website or any social profiles you'd like me to know about?
333
+
334
+ **User:** My blog is example.com
335
+
336
+ **Agent:** Got it. Would you like me to analyze your OpenCode session history to understand your communication style and what you've been working on? I'll look at things like how you communicate and recent projects. This stays completely local.
337
+
338
+ **User:** Sure
339
+
340
+ **Agent:** [runs analysis, summarizes findings]
341
+
342
+ Looking at your history, I can see you communicate in short, direct messages, give quick acknowledgments, and tend to ask questions before committing to an approach. You've been working on [detected projects] recently.
343
+
344
+ Now, what would you like to call your agent?
345
+
346
+ [continues through flow...]