@prmichaelsen/remember-mcp 3.0.0 → 3.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (208) hide show
  1. package/AGENT.md +296 -250
  2. package/CHANGELOG.md +358 -0
  3. package/README.md +68 -45
  4. package/agent/commands/acp.clarification-create.md +382 -0
  5. package/agent/commands/acp.project-info.md +309 -0
  6. package/agent/commands/acp.project-remove.md +379 -0
  7. package/agent/commands/acp.project-update.md +296 -0
  8. package/agent/commands/acp.task-create.md +17 -9
  9. package/agent/commands/git.commit.md +13 -1
  10. package/agent/design/comment-memory-type.md +2 -2
  11. package/agent/design/local.collaborative-memory-sync.md +265 -0
  12. package/agent/design/local.content-flags.md +210 -0
  13. package/agent/design/local.ghost-persona-system.md +273 -0
  14. package/agent/design/local.group-acl-integration.md +338 -0
  15. package/agent/design/local.memory-acl-schema.md +352 -0
  16. package/agent/design/local.memory-collection-pattern-v2.md +348 -0
  17. package/agent/design/local.moderation-and-space-config.md +257 -0
  18. package/agent/design/local.v2-api-reference.md +621 -0
  19. package/agent/design/local.v2-migration-guide.md +191 -0
  20. package/agent/design/local.v2-usage-examples.md +265 -0
  21. package/agent/design/permissions-storage-architecture.md +11 -3
  22. package/agent/design/trust-escalation-prevention.md +9 -2
  23. package/agent/design/trust-system-implementation.md +12 -3
  24. package/agent/milestones/milestone-14-memory-collection-v2.md +182 -0
  25. package/agent/milestones/milestone-15-moderation-space-config.md +126 -0
  26. package/agent/progress.yaml +628 -49
  27. package/agent/scripts/acp.common.sh +2 -0
  28. package/agent/scripts/acp.install.sh +11 -1
  29. package/agent/scripts/acp.package-install-optimized.sh +454 -0
  30. package/agent/scripts/acp.package-install.sh +247 -300
  31. package/agent/scripts/acp.project-info.sh +218 -0
  32. package/agent/scripts/acp.project-remove.sh +302 -0
  33. package/agent/scripts/acp.project-update.sh +296 -0
  34. package/agent/scripts/acp.yaml-parser.sh +128 -10
  35. package/agent/tasks/milestone-14-memory-collection-v2/task-165-core-infrastructure-setup.md +171 -0
  36. package/agent/tasks/milestone-14-memory-collection-v2/task-166-update-remember-publish.md +191 -0
  37. package/agent/tasks/milestone-14-memory-collection-v2/task-167-update-remember-retract.md +186 -0
  38. package/agent/tasks/milestone-14-memory-collection-v2/task-168-implement-remember-revise.md +184 -0
  39. package/agent/tasks/milestone-14-memory-collection-v2/task-169-update-remember-search-space.md +179 -0
  40. package/agent/tasks/milestone-14-memory-collection-v2/task-170-update-remember-create-update.md +139 -0
  41. package/agent/tasks/milestone-14-memory-collection-v2/task-172-performance-testing-optimization.md +161 -0
  42. package/agent/tasks/milestone-14-memory-collection-v2/task-173-documentation-examples.md +258 -0
  43. package/agent/tasks/milestone-15-moderation-space-config/task-174-add-moderation-schema-fields.md +57 -0
  44. package/agent/tasks/milestone-15-moderation-space-config/task-175-create-space-config-service.md +64 -0
  45. package/agent/tasks/milestone-15-moderation-space-config/task-176-wire-moderation-publish-flow.md +45 -0
  46. package/agent/tasks/milestone-15-moderation-space-config/task-177-add-moderation-search-filters.md +70 -0
  47. package/agent/tasks/milestone-15-moderation-space-config/task-178-create-remember-moderate-tool.md +69 -0
  48. package/agent/tasks/milestone-15-moderation-space-config/task-179-documentation-integration-tests.md +58 -0
  49. package/agent/tasks/milestone-16-ghost-system/task-187-ghost-config-firestore.md +41 -0
  50. package/agent/tasks/milestone-16-ghost-system/task-188-trust-filter-integration.md +44 -0
  51. package/agent/tasks/milestone-16-ghost-system/task-189-ghost-memory-filtering.md +43 -0
  52. package/agent/tasks/milestone-16-ghost-system/task-190-ghost-config-tools.md +45 -0
  53. package/agent/tasks/milestone-16-ghost-system/task-191-escalation-firestore.md +38 -0
  54. package/agent/tasks/milestone-16-ghost-system/task-192-documentation-verification.md +39 -0
  55. package/agent/tasks/milestone-7-trust-permissions/task-180-access-result-permission-types.md +69 -0
  56. package/agent/tasks/milestone-7-trust-permissions/task-181-firestore-permissions-access-logs.md +56 -0
  57. package/agent/tasks/milestone-7-trust-permissions/task-182-trust-enforcement-service.md +68 -0
  58. package/agent/tasks/milestone-7-trust-permissions/task-183-access-control-service.md +70 -0
  59. package/agent/tasks/milestone-7-trust-permissions/task-184-permission-tools.md +79 -0
  60. package/agent/tasks/milestone-7-trust-permissions/task-185-wire-trust-into-search-query.md +55 -0
  61. package/agent/tasks/milestone-7-trust-permissions/task-186-documentation-verification.md +56 -0
  62. package/agent/tasks/task-76-fix-indexnullstate-schema-bug.md +197 -0
  63. package/dist/collections/composite-ids.d.ts +106 -0
  64. package/dist/collections/core-infrastructure.spec.d.ts +11 -0
  65. package/dist/collections/dot-notation.d.ts +106 -0
  66. package/dist/collections/tracking-arrays.d.ts +176 -0
  67. package/dist/constants/content-types.d.ts +1 -0
  68. package/dist/schema/v2-collections-comments.spec.d.ts +8 -0
  69. package/dist/schema/v2-collections.d.ts +210 -0
  70. package/dist/server-factory.d.ts +15 -0
  71. package/dist/server-factory.js +2798 -1029
  72. package/dist/server.js +2526 -1012
  73. package/dist/services/access-control.d.ts +103 -0
  74. package/dist/services/access-control.spec.d.ts +2 -0
  75. package/dist/services/credentials-provider.d.ts +24 -0
  76. package/dist/services/credentials-provider.spec.d.ts +2 -0
  77. package/dist/services/escalation.service.d.ts +22 -0
  78. package/dist/services/escalation.service.spec.d.ts +2 -0
  79. package/dist/services/ghost-config.service.d.ts +55 -0
  80. package/dist/services/ghost-config.service.spec.d.ts +2 -0
  81. package/dist/services/space-config.service.d.ts +23 -0
  82. package/dist/services/space-config.service.spec.d.ts +2 -0
  83. package/dist/services/trust-enforcement.d.ts +83 -0
  84. package/dist/services/trust-enforcement.spec.d.ts +2 -0
  85. package/dist/services/trust-validator.d.ts +43 -0
  86. package/dist/services/trust-validator.spec.d.ts +2 -0
  87. package/dist/tools/confirm-publish-moderation.spec.d.ts +8 -0
  88. package/dist/tools/confirm.d.ts +8 -1
  89. package/dist/tools/create-memory.d.ts +2 -1
  90. package/dist/tools/create-memory.spec.d.ts +10 -0
  91. package/dist/tools/create-relationship.d.ts +2 -1
  92. package/dist/tools/delete-memory.d.ts +2 -1
  93. package/dist/tools/delete-relationship.d.ts +2 -1
  94. package/dist/tools/deny.d.ts +2 -1
  95. package/dist/tools/find-similar.d.ts +2 -1
  96. package/dist/tools/get-preferences.d.ts +2 -1
  97. package/dist/tools/ghost-config.d.ts +27 -0
  98. package/dist/tools/ghost-config.spec.d.ts +2 -0
  99. package/dist/tools/moderate.d.ts +20 -0
  100. package/dist/tools/moderate.spec.d.ts +5 -0
  101. package/dist/tools/publish.d.ts +11 -3
  102. package/dist/tools/query-memory.d.ts +3 -1
  103. package/dist/tools/query-space.d.ts +4 -1
  104. package/dist/tools/retract.d.ts +29 -0
  105. package/dist/tools/revise.d.ts +45 -0
  106. package/dist/tools/revise.spec.d.ts +8 -0
  107. package/dist/tools/search-memory.d.ts +2 -1
  108. package/dist/tools/search-relationship.d.ts +2 -1
  109. package/dist/tools/search-space.d.ts +25 -5
  110. package/dist/tools/search-space.spec.d.ts +9 -0
  111. package/dist/tools/set-preference.d.ts +2 -1
  112. package/dist/tools/update-memory.d.ts +2 -1
  113. package/dist/tools/update-relationship.d.ts +2 -1
  114. package/dist/types/access-result.d.ts +48 -0
  115. package/dist/types/access-result.spec.d.ts +2 -0
  116. package/dist/types/auth.d.ts +46 -0
  117. package/dist/types/ghost-config.d.ts +36 -0
  118. package/dist/types/memory.d.ts +3 -1
  119. package/dist/types/preferences.d.ts +1 -1
  120. package/dist/utils/auth-helpers.d.ts +14 -0
  121. package/dist/utils/auth-helpers.spec.d.ts +2 -0
  122. package/dist/utils/test-data-generator.d.ts +124 -0
  123. package/dist/utils/test-data-generator.spec.d.ts +12 -0
  124. package/dist/v2-performance.e2e.d.ts +17 -0
  125. package/dist/v2-smoke.e2e.d.ts +14 -0
  126. package/dist/weaviate/client.d.ts +5 -8
  127. package/dist/weaviate/space-schema.d.ts +2 -2
  128. package/docs/performance/v2-benchmarks.md +80 -0
  129. package/jest.e2e.config.js +14 -3
  130. package/package.json +1 -1
  131. package/scripts/.collection-recreation-state.yaml +16 -0
  132. package/scripts/.gitkeep +5 -0
  133. package/scripts/README-collection-recreation.md +224 -0
  134. package/scripts/README.md +51 -0
  135. package/scripts/backup-collections.ts +543 -0
  136. package/scripts/delete-collection.ts +137 -0
  137. package/scripts/migrate-recreate-collections.ts +578 -0
  138. package/scripts/migrate-v1-to-v2.ts +1094 -0
  139. package/scripts/package-lock.json +1113 -0
  140. package/scripts/package.json +27 -0
  141. package/src/collections/composite-ids.ts +193 -0
  142. package/src/collections/core-infrastructure.spec.ts +353 -0
  143. package/src/collections/dot-notation.ts +212 -0
  144. package/src/collections/tracking-arrays.ts +298 -0
  145. package/src/constants/content-types.ts +20 -0
  146. package/src/schema/v2-collections-comments.spec.ts +141 -0
  147. package/src/schema/v2-collections.ts +433 -0
  148. package/src/server-factory.ts +89 -20
  149. package/src/server.ts +45 -17
  150. package/src/services/access-control.spec.ts +383 -0
  151. package/src/services/access-control.ts +291 -0
  152. package/src/services/credentials-provider.spec.ts +22 -0
  153. package/src/services/credentials-provider.ts +34 -0
  154. package/src/services/escalation.service.spec.ts +183 -0
  155. package/src/services/escalation.service.ts +150 -0
  156. package/src/services/ghost-config.service.spec.ts +339 -0
  157. package/src/services/ghost-config.service.ts +219 -0
  158. package/src/services/space-config.service.spec.ts +102 -0
  159. package/src/services/space-config.service.ts +79 -0
  160. package/src/services/trust-enforcement.spec.ts +309 -0
  161. package/src/services/trust-enforcement.ts +197 -0
  162. package/src/services/trust-validator.spec.ts +108 -0
  163. package/src/services/trust-validator.ts +105 -0
  164. package/src/tools/confirm-publish-moderation.spec.ts +240 -0
  165. package/src/tools/confirm.ts +869 -135
  166. package/src/tools/create-memory.spec.ts +126 -0
  167. package/src/tools/create-memory.ts +20 -27
  168. package/src/tools/create-relationship.ts +17 -8
  169. package/src/tools/delete-memory.ts +13 -6
  170. package/src/tools/delete-relationship.ts +15 -6
  171. package/src/tools/deny.ts +8 -1
  172. package/src/tools/find-similar.ts +21 -8
  173. package/src/tools/get-preferences.ts +10 -1
  174. package/src/tools/ghost-config.spec.ts +180 -0
  175. package/src/tools/ghost-config.ts +230 -0
  176. package/src/tools/moderate.spec.ts +277 -0
  177. package/src/tools/moderate.ts +219 -0
  178. package/src/tools/publish.ts +99 -41
  179. package/src/tools/query-memory.ts +28 -6
  180. package/src/tools/query-space.ts +39 -4
  181. package/src/tools/retract.ts +292 -0
  182. package/src/tools/revise.spec.ts +146 -0
  183. package/src/tools/revise.ts +283 -0
  184. package/src/tools/search-memory.ts +30 -7
  185. package/src/tools/search-relationship.ts +11 -2
  186. package/src/tools/search-space.spec.ts +341 -0
  187. package/src/tools/search-space.ts +323 -99
  188. package/src/tools/set-preference.ts +10 -1
  189. package/src/tools/update-memory.ts +16 -5
  190. package/src/tools/update-relationship.ts +10 -1
  191. package/src/types/access-result.spec.ts +193 -0
  192. package/src/types/access-result.ts +62 -0
  193. package/src/types/auth.ts +52 -0
  194. package/src/types/ghost-config.ts +46 -0
  195. package/src/types/memory.ts +9 -1
  196. package/src/types/preferences.ts +2 -2
  197. package/src/utils/auth-helpers.spec.ts +75 -0
  198. package/src/utils/auth-helpers.ts +25 -0
  199. package/src/utils/test-data-generator.spec.ts +317 -0
  200. package/src/utils/test-data-generator.ts +292 -0
  201. package/src/utils/weaviate-filters.ts +4 -4
  202. package/src/v2-performance.e2e.ts +173 -0
  203. package/src/v2-smoke.e2e.ts +401 -0
  204. package/src/weaviate/client.spec.ts +5 -5
  205. package/src/weaviate/client.ts +51 -36
  206. package/src/weaviate/schema.ts +11 -256
  207. package/src/weaviate/space-schema.spec.ts +24 -24
  208. package/src/weaviate/space-schema.ts +18 -6
@@ -0,0 +1,179 @@
1
+ # Task 169: Implement remember_search_space Tool
2
+
3
+ **Milestone**: Milestone 14 - Memory Collection Pattern v2
4
+ **Estimated Time**: 4-6 hours
5
+ **Dependencies**: [Task 165: Core Infrastructure Setup](task-165-core-infrastructure-setup.md)
6
+ **Status**: Completed 2026-02-27
7
+
8
+ ---
9
+
10
+ ## Objective
11
+
12
+ Implement the `remember_search_space` tool to query memories using space_ids filtering and support group searches.
13
+
14
+ ---
15
+
16
+ ## Steps
17
+
18
+ ### 1. Create Tool Schema
19
+
20
+ **File**: `src/tools/remember-search-space.ts`
21
+
22
+ **Actions**:
23
+ - Update input schema to accept `spaces` array (filter by spaces)
24
+ - Add `groups` array parameter (filter by groups)
25
+ - Keep `query` parameter (search query)
26
+ - Add `search_type` parameter (hybrid, bm25, semantic)
27
+ - Update tool description
28
+
29
+ **Expected Schema**:
30
+ ```typescript
31
+ {
32
+ query: string,
33
+ spaces?: string[], // ["cooking", "recipes"]
34
+ groups?: string[], // ["{group-id}"]
35
+ search_type?: string, // "hybrid" | "bm25" | "semantic"
36
+ limit?: number
37
+ }
38
+ ```
39
+
40
+ ### 2. Implement Space Filtering
41
+
42
+ **Actions**:
43
+ - Query `Memory_spaces_public` collection
44
+ - Filter by `space_ids` array contains any of specified spaces
45
+ - Use Weaviate's array filtering capabilities
46
+ - Return memories with composite IDs
47
+
48
+ **Expected Behavior**:
49
+ ```typescript
50
+ // Search specific spaces
51
+ remember_search_space({
52
+ query: "recipe",
53
+ spaces: ["cooking", "recipes"]
54
+ })
55
+
56
+ // Queries: Memory_spaces_public WHERE space_ids contains "cooking" OR "recipes"
57
+ ```
58
+
59
+ ### 3. Implement Group Filtering
60
+
61
+ **Actions**:
62
+ - Query specific `Memory_groups_{groupId}` collections
63
+ - Combine results from multiple group collections
64
+ - Return memories with composite IDs
65
+
66
+ **Expected Behavior**:
67
+ ```typescript
68
+ // Search specific groups
69
+ remember_search_space({
70
+ query: "recipe",
71
+ groups: ["{foodie-group}", "{recipe-club}"]
72
+ })
73
+
74
+ // Queries: Memory_groups_{foodie-group} AND Memory_groups_{recipe-club}
75
+ ```
76
+
77
+ ### 4. Implement Combined Search
78
+
79
+ **Actions**:
80
+ - Support searching both spaces and groups simultaneously
81
+ - Merge results from both query types
82
+ - Deduplicate by composite ID
83
+ - Sort by relevance score
84
+
85
+ **Expected Behavior**:
86
+ ```typescript
87
+ // Search both spaces and groups
88
+ remember_search_space({
89
+ query: "recipe",
90
+ spaces: ["cooking"],
91
+ groups: ["{foodie-group}"]
92
+ })
93
+
94
+ // Queries both Memory_spaces_public and Memory_groups_{foodie-group}
95
+ // Returns merged, deduplicated results
96
+ ```
97
+
98
+ ### 5. Implement All-Public Search
99
+
100
+ **Actions**:
101
+ - When no spaces or groups specified, search all public memories
102
+ - Query entire `Memory_spaces_public` collection
103
+ - Return all matching memories regardless of space_ids
104
+
105
+ **Expected Behavior**:
106
+ ```typescript
107
+ // Search all public memories
108
+ remember_search_space({
109
+ query: "recipe"
110
+ })
111
+
112
+ // Queries: Memory_spaces_public (all memories)
113
+ ```
114
+
115
+ ### 6. Write Integration Tests
116
+
117
+ **File**: `tests/integration/remember-search-space.test.ts`
118
+
119
+ **Actions**:
120
+ - Test space filtering (single and multiple)
121
+ - Test group filtering (single and multiple)
122
+ - Test combined search (spaces + groups)
123
+ - Test all-public search
124
+ - Test deduplication
125
+ - Test search types (hybrid, bm25, semantic)
126
+ - Test error cases
127
+
128
+ **Expected Output**: All tests passing
129
+
130
+ ---
131
+
132
+ ## Verification
133
+
134
+ - [ ] Tool schema updated correctly
135
+ - [ ] Space filtering works correctly
136
+ - [ ] Group filtering works correctly
137
+ - [ ] Combined search works correctly
138
+ - [ ] All-public search works correctly
139
+ - [ ] Deduplication works correctly
140
+ - [ ] Search types work correctly
141
+ - [ ] Integration tests passing
142
+ - [ ] TypeScript compiles without errors
143
+ - [ ] Tool description updated in MCP manifest
144
+
145
+ ---
146
+
147
+ ## Expected Output
148
+
149
+ ### Files Modified
150
+ - `src/tools/remember-search-space-v2.ts` (~220 lines)
151
+
152
+ ### Files Created
153
+ - `tests/integration/remember-search-space.test.ts` (~180 lines)
154
+
155
+ ### Test Output
156
+ ```
157
+ ✓ Space filtering (4 tests)
158
+ ✓ Group filtering (4 tests)
159
+ ✓ Combined search (4 tests)
160
+ ✓ All-public search (3 tests)
161
+ ✓ Deduplication (3 tests)
162
+ ✓ Search types (3 tests)
163
+ ✓ Error handling (3 tests)
164
+
165
+ Total: 24 tests passing
166
+ ```
167
+
168
+ ---
169
+
170
+ ## Notes
171
+
172
+ - This is a breaking change from v1 (single space only)
173
+ - Deduplication critical for combined searches
174
+ - Consider caching frequently searched spaces
175
+ - Performance testing needed for large collections
176
+
177
+ ---
178
+
179
+ **Next Task**: [Task 170: Update remember_create_memory Tool](task-170-update-remember-create-memory.md)
@@ -0,0 +1,139 @@
1
+ # Task 170: Implement remember_create_memory and remember_update_memory Tools
2
+
3
+ **Milestone**: Milestone 14 - Memory Collection Pattern v2
4
+ **Estimated Time**: 3-4 hours
5
+ **Dependencies**: [Task 165: Core Infrastructure Setup](task-165-core-infrastructure-setup.md)
6
+ **Status**: Completed 2026-02-27
7
+
8
+ ---
9
+
10
+ ## Objective
11
+
12
+ Implement `remember_create_memory` and `remember_update_memory` tools to initialize and maintain tracking arrays (space_ids, group_ids) in user memories.
13
+
14
+ ---
15
+
16
+ ## Steps
17
+
18
+ ### 1. Implement remember_create_memory Tool
19
+
20
+ **File**: `src/tools/remember-create-memory.ts`
21
+
22
+ **Actions**:
23
+ - Add `space_ids: []` to new memory objects (empty array)
24
+ - Add `group_ids: []` to new memory objects (empty array)
25
+ - Ensure memories created in `Memory_users_{userId}` collection
26
+ - Keep all existing functionality
27
+
28
+ **Expected Behavior**:
29
+ ```typescript
30
+ // Create new memory
31
+ remember_create_memory({
32
+ content: "My recipe",
33
+ content_type: "note"
34
+ })
35
+
36
+ // Creates in Memory_users_{userId}:
37
+ {
38
+ id: "auto-generated-id",
39
+ content: "My recipe",
40
+ content_type: "note",
41
+ space_ids: [], // ← New field
42
+ group_ids: [], // ← New field
43
+ created_at: timestamp,
44
+ updated_at: timestamp
45
+ }
46
+ ```
47
+
48
+ ### 2. Implement remember_update_memory Tool
49
+
50
+ **File**: `src/tools/remember-update-memory.ts`
51
+
52
+ **Actions**:
53
+ - Preserve `space_ids` and `group_ids` arrays during updates
54
+ - Don't allow direct modification of tracking arrays (use publish/retract instead)
55
+ - Update `updated_at` timestamp
56
+ - Keep all existing functionality
57
+
58
+ **Expected Behavior**:
59
+ ```typescript
60
+ // Update memory content
61
+ remember_update_memory({
62
+ id: "my-recipe",
63
+ content: "Updated recipe"
64
+ })
65
+
66
+ // Updates in Memory_users_{userId}:
67
+ {
68
+ id: "my-recipe",
69
+ content: "Updated recipe", // ← Updated
70
+ space_ids: ["cooking"], // ← Preserved
71
+ group_ids: [], // ← Preserved
72
+ updated_at: timestamp // ← Updated
73
+ }
74
+ ```
75
+
76
+ ### 3. Add Validation
77
+
78
+ **Actions**:
79
+ - Validate `space_ids` and `group_ids` are arrays
80
+ - Prevent direct modification of tracking arrays
81
+ - Ensure arrays are initialized on create
82
+ - Handle migration of old memories without tracking arrays
83
+
84
+ ### 4. Write Unit Tests
85
+
86
+ **File**: `tests/unit/remember-create-update.test.ts`
87
+
88
+ **Actions**:
89
+ - Test memory creation with tracking arrays
90
+ - Test memory update preserves tracking arrays
91
+ - Test validation of tracking arrays
92
+ - Test error cases
93
+
94
+ **Expected Output**: All tests passing
95
+
96
+ ---
97
+
98
+ ## Verification
99
+
100
+ - [ ] remember_create_memory initializes tracking arrays
101
+ - [ ] remember_update_memory preserves tracking arrays
102
+ - [ ] Tracking arrays cannot be directly modified
103
+ - [ ] Validation works correctly
104
+ - [ ] Unit tests passing
105
+ - [ ] TypeScript compiles without errors
106
+ - [ ] Tool descriptions updated
107
+
108
+ ---
109
+
110
+ ## Expected Output
111
+
112
+ ### Files Modified
113
+ - `src/tools/remember-create-memory-v2.ts` (~150 lines)
114
+ - `src/tools/remember-update-memory-v2.ts` (~150 lines)
115
+
116
+ ### Files Created
117
+ - `tests/unit/remember-create-update.test.ts` (~100 lines)
118
+
119
+ ### Test Output
120
+ ```
121
+ ✓ Memory creation (5 tests)
122
+ ✓ Memory update (5 tests)
123
+ ✓ Tracking array validation (4 tests)
124
+ ✓ Error handling (3 tests)
125
+
126
+ Total: 17 tests passing
127
+ ```
128
+
129
+ ---
130
+
131
+ ## Notes
132
+
133
+ - Minimal changes to existing tools
134
+ - Tracking arrays managed by publish/retract/revise tools
135
+ - Backward compatibility for old memories without tracking arrays
136
+
137
+ ---
138
+
139
+ **Next Task**: [Task 171: Implement Migration Script](task-171-implement-migration-script.md)
@@ -0,0 +1,161 @@
1
+ # Task 172: Performance Testing and Optimization
2
+
3
+ **Milestone**: Milestone 14 - Memory Collection Pattern v2
4
+ **Estimated Time**: 6-8 hours
5
+ **Dependencies**: All previous tasks
6
+ **Status**: Not Started
7
+
8
+ ---
9
+
10
+ ## Objective
11
+
12
+ Conduct comprehensive performance testing of Memory Collection Pattern v2 and optimize query performance to meet targets (<200ms for typical queries).
13
+
14
+ ---
15
+
16
+ ## Steps
17
+
18
+ ### 1. Create Performance Test Suite
19
+
20
+ **File**: `tests/performance/v2-performance.test.ts`
21
+
22
+ **Actions**:
23
+ - Create test data generator (1K, 10K, 100K memories)
24
+ - Implement query performance tests
25
+ - Implement write performance tests
26
+ - Implement migration performance tests
27
+ - Set performance baselines
28
+
29
+ ### 2. Test Query Performance
30
+
31
+ **Scenarios**:
32
+ - Search single space (small: <100 memories)
33
+ - Search single space (medium: 1K-10K memories)
34
+ - Search single space (large: >10K memories)
35
+ - Search multiple spaces
36
+ - Search groups
37
+ - Search all public memories
38
+ - Composite ID lookups
39
+
40
+ **Target**: <200ms for typical queries (1K-10K memories)
41
+
42
+ ### 3. Test Write Performance
43
+
44
+ **Scenarios**:
45
+ - Create memory
46
+ - Update memory
47
+ - Publish to single space
48
+ - Publish to multiple spaces
49
+ - Publish to groups
50
+ - Retract from spaces
51
+ - Revise published memories
52
+
53
+ **Target**: <100ms for write operations
54
+
55
+ ### 4. Test Migration Performance
56
+
57
+ **Scenarios**:
58
+ - Migrate 1K memories
59
+ - Migrate 10K memories
60
+ - Migrate 100K memories
61
+ - Backup creation
62
+ - Rollback execution
63
+
64
+ **Target**: <10 minutes for 10K memories
65
+
66
+ ### 5. Identify Bottlenecks
67
+
68
+ **Actions**:
69
+ - Profile query execution
70
+ - Identify slow operations
71
+ - Analyze Weaviate query plans
72
+ - Check index usage
73
+ - Monitor memory usage
74
+
75
+ ### 6. Implement Optimizations
76
+
77
+ **Potential Optimizations**:
78
+ - Add Weaviate indexes on space_ids and group_ids
79
+ - Optimize composite ID generation
80
+ - Batch operations where possible
81
+ - Cache frequently accessed data
82
+ - Optimize tracking array operations
83
+ - Parallelize migration steps
84
+
85
+ ### 7. Benchmark Results
86
+
87
+ **Actions**:
88
+ - Run performance tests before optimization
89
+ - Run performance tests after optimization
90
+ - Compare results
91
+ - Document improvements
92
+ - Generate performance report
93
+
94
+ ---
95
+
96
+ ## Verification
97
+
98
+ - [ ] Performance test suite created
99
+ - [ ] Query performance meets targets (<200ms)
100
+ - [ ] Write performance meets targets (<100ms)
101
+ - [ ] Migration performance meets targets (<10 min for 10K)
102
+ - [ ] Bottlenecks identified and documented
103
+ - [ ] Optimizations implemented
104
+ - [ ] Benchmarks show improvement
105
+ - [ ] Performance report generated
106
+
107
+ ---
108
+
109
+ ## Expected Output
110
+
111
+ ### Files Created
112
+ - `tests/performance/v2-performance.test.ts` (~400 lines)
113
+ - `docs/performance/v2-benchmarks.md` (~50 lines)
114
+
115
+ ### Performance Report
116
+ ```
117
+ Memory Collection Pattern v2 - Performance Benchmarks
118
+ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
119
+
120
+ Query Performance (10K memories):
121
+ Single space search: 145ms ✓ (target: <200ms)
122
+ Multi-space search: 178ms ✓ (target: <200ms)
123
+ Group search: 132ms ✓ (target: <200ms)
124
+ All public search: 189ms ✓ (target: <200ms)
125
+ Composite ID lookup: 12ms ✓ (target: <50ms)
126
+
127
+ Write Performance:
128
+ Create memory: 45ms ✓ (target: <100ms)
129
+ Update memory: 38ms ✓ (target: <100ms)
130
+ Publish (single space): 67ms ✓ (target: <100ms)
131
+ Publish (multi-space): 89ms ✓ (target: <100ms)
132
+ Retract: 52ms ✓ (target: <100ms)
133
+ Revise: 78ms ✓ (target: <100ms)
134
+
135
+ Migration Performance:
136
+ 1K memories: 1.2 min ✓ (target: <2 min)
137
+ 10K memories: 8.5 min ✓ (target: <10 min)
138
+ 100K memories: 82.3 min ✓ (target: <120 min)
139
+
140
+ Optimizations Applied:
141
+ ✓ Added indexes on space_ids and group_ids
142
+ ✓ Optimized composite ID generation (30% faster)
143
+ ✓ Batched tracking array updates (40% faster)
144
+ ✓ Parallelized migration steps (2x faster)
145
+
146
+ Overall: All performance targets met ✓
147
+ ```
148
+
149
+ ---
150
+
151
+ ## Notes
152
+
153
+ - Performance testing should use realistic data volumes
154
+ - Test on hardware similar to production
155
+ - Consider network latency in benchmarks
156
+ - Document any performance regressions
157
+ - Optimization should not compromise correctness
158
+
159
+ ---
160
+
161
+ **Next Task**: [Task 173: Documentation and Examples](task-173-documentation-examples.md)