@prmichaelsen/remember-mcp 3.0.0 → 3.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT.md +296 -250
- package/CHANGELOG.md +358 -0
- package/README.md +68 -45
- package/agent/commands/acp.clarification-create.md +382 -0
- package/agent/commands/acp.project-info.md +309 -0
- package/agent/commands/acp.project-remove.md +379 -0
- package/agent/commands/acp.project-update.md +296 -0
- package/agent/commands/acp.task-create.md +17 -9
- package/agent/commands/git.commit.md +13 -1
- package/agent/design/comment-memory-type.md +2 -2
- package/agent/design/local.collaborative-memory-sync.md +265 -0
- package/agent/design/local.content-flags.md +210 -0
- package/agent/design/local.ghost-persona-system.md +273 -0
- package/agent/design/local.group-acl-integration.md +338 -0
- package/agent/design/local.memory-acl-schema.md +352 -0
- package/agent/design/local.memory-collection-pattern-v2.md +348 -0
- package/agent/design/local.moderation-and-space-config.md +257 -0
- package/agent/design/local.v2-api-reference.md +621 -0
- package/agent/design/local.v2-migration-guide.md +191 -0
- package/agent/design/local.v2-usage-examples.md +265 -0
- package/agent/design/permissions-storage-architecture.md +11 -3
- package/agent/design/trust-escalation-prevention.md +9 -2
- package/agent/design/trust-system-implementation.md +12 -3
- package/agent/milestones/milestone-14-memory-collection-v2.md +182 -0
- package/agent/milestones/milestone-15-moderation-space-config.md +126 -0
- package/agent/progress.yaml +628 -49
- package/agent/scripts/acp.common.sh +2 -0
- package/agent/scripts/acp.install.sh +11 -1
- package/agent/scripts/acp.package-install-optimized.sh +454 -0
- package/agent/scripts/acp.package-install.sh +247 -300
- package/agent/scripts/acp.project-info.sh +218 -0
- package/agent/scripts/acp.project-remove.sh +302 -0
- package/agent/scripts/acp.project-update.sh +296 -0
- package/agent/scripts/acp.yaml-parser.sh +128 -10
- package/agent/tasks/milestone-14-memory-collection-v2/task-165-core-infrastructure-setup.md +171 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-166-update-remember-publish.md +191 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-167-update-remember-retract.md +186 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-168-implement-remember-revise.md +184 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-169-update-remember-search-space.md +179 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-170-update-remember-create-update.md +139 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-172-performance-testing-optimization.md +161 -0
- package/agent/tasks/milestone-14-memory-collection-v2/task-173-documentation-examples.md +258 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-174-add-moderation-schema-fields.md +57 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-175-create-space-config-service.md +64 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-176-wire-moderation-publish-flow.md +45 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-177-add-moderation-search-filters.md +70 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-178-create-remember-moderate-tool.md +69 -0
- package/agent/tasks/milestone-15-moderation-space-config/task-179-documentation-integration-tests.md +58 -0
- package/agent/tasks/milestone-16-ghost-system/task-187-ghost-config-firestore.md +41 -0
- package/agent/tasks/milestone-16-ghost-system/task-188-trust-filter-integration.md +44 -0
- package/agent/tasks/milestone-16-ghost-system/task-189-ghost-memory-filtering.md +43 -0
- package/agent/tasks/milestone-16-ghost-system/task-190-ghost-config-tools.md +45 -0
- package/agent/tasks/milestone-16-ghost-system/task-191-escalation-firestore.md +38 -0
- package/agent/tasks/milestone-16-ghost-system/task-192-documentation-verification.md +39 -0
- package/agent/tasks/milestone-7-trust-permissions/task-180-access-result-permission-types.md +69 -0
- package/agent/tasks/milestone-7-trust-permissions/task-181-firestore-permissions-access-logs.md +56 -0
- package/agent/tasks/milestone-7-trust-permissions/task-182-trust-enforcement-service.md +68 -0
- package/agent/tasks/milestone-7-trust-permissions/task-183-access-control-service.md +70 -0
- package/agent/tasks/milestone-7-trust-permissions/task-184-permission-tools.md +79 -0
- package/agent/tasks/milestone-7-trust-permissions/task-185-wire-trust-into-search-query.md +55 -0
- package/agent/tasks/milestone-7-trust-permissions/task-186-documentation-verification.md +56 -0
- package/agent/tasks/task-76-fix-indexnullstate-schema-bug.md +197 -0
- package/dist/collections/composite-ids.d.ts +106 -0
- package/dist/collections/core-infrastructure.spec.d.ts +11 -0
- package/dist/collections/dot-notation.d.ts +106 -0
- package/dist/collections/tracking-arrays.d.ts +176 -0
- package/dist/constants/content-types.d.ts +1 -0
- package/dist/schema/v2-collections-comments.spec.d.ts +8 -0
- package/dist/schema/v2-collections.d.ts +210 -0
- package/dist/server-factory.d.ts +15 -0
- package/dist/server-factory.js +2798 -1029
- package/dist/server.js +2526 -1012
- package/dist/services/access-control.d.ts +103 -0
- package/dist/services/access-control.spec.d.ts +2 -0
- package/dist/services/credentials-provider.d.ts +24 -0
- package/dist/services/credentials-provider.spec.d.ts +2 -0
- package/dist/services/escalation.service.d.ts +22 -0
- package/dist/services/escalation.service.spec.d.ts +2 -0
- package/dist/services/ghost-config.service.d.ts +55 -0
- package/dist/services/ghost-config.service.spec.d.ts +2 -0
- package/dist/services/space-config.service.d.ts +23 -0
- package/dist/services/space-config.service.spec.d.ts +2 -0
- package/dist/services/trust-enforcement.d.ts +83 -0
- package/dist/services/trust-enforcement.spec.d.ts +2 -0
- package/dist/services/trust-validator.d.ts +43 -0
- package/dist/services/trust-validator.spec.d.ts +2 -0
- package/dist/tools/confirm-publish-moderation.spec.d.ts +8 -0
- package/dist/tools/confirm.d.ts +8 -1
- package/dist/tools/create-memory.d.ts +2 -1
- package/dist/tools/create-memory.spec.d.ts +10 -0
- package/dist/tools/create-relationship.d.ts +2 -1
- package/dist/tools/delete-memory.d.ts +2 -1
- package/dist/tools/delete-relationship.d.ts +2 -1
- package/dist/tools/deny.d.ts +2 -1
- package/dist/tools/find-similar.d.ts +2 -1
- package/dist/tools/get-preferences.d.ts +2 -1
- package/dist/tools/ghost-config.d.ts +27 -0
- package/dist/tools/ghost-config.spec.d.ts +2 -0
- package/dist/tools/moderate.d.ts +20 -0
- package/dist/tools/moderate.spec.d.ts +5 -0
- package/dist/tools/publish.d.ts +11 -3
- package/dist/tools/query-memory.d.ts +3 -1
- package/dist/tools/query-space.d.ts +4 -1
- package/dist/tools/retract.d.ts +29 -0
- package/dist/tools/revise.d.ts +45 -0
- package/dist/tools/revise.spec.d.ts +8 -0
- package/dist/tools/search-memory.d.ts +2 -1
- package/dist/tools/search-relationship.d.ts +2 -1
- package/dist/tools/search-space.d.ts +25 -5
- package/dist/tools/search-space.spec.d.ts +9 -0
- package/dist/tools/set-preference.d.ts +2 -1
- package/dist/tools/update-memory.d.ts +2 -1
- package/dist/tools/update-relationship.d.ts +2 -1
- package/dist/types/access-result.d.ts +48 -0
- package/dist/types/access-result.spec.d.ts +2 -0
- package/dist/types/auth.d.ts +46 -0
- package/dist/types/ghost-config.d.ts +36 -0
- package/dist/types/memory.d.ts +3 -1
- package/dist/types/preferences.d.ts +1 -1
- package/dist/utils/auth-helpers.d.ts +14 -0
- package/dist/utils/auth-helpers.spec.d.ts +2 -0
- package/dist/utils/test-data-generator.d.ts +124 -0
- package/dist/utils/test-data-generator.spec.d.ts +12 -0
- package/dist/v2-performance.e2e.d.ts +17 -0
- package/dist/v2-smoke.e2e.d.ts +14 -0
- package/dist/weaviate/client.d.ts +5 -8
- package/dist/weaviate/space-schema.d.ts +2 -2
- package/docs/performance/v2-benchmarks.md +80 -0
- package/jest.e2e.config.js +14 -3
- package/package.json +1 -1
- package/scripts/.collection-recreation-state.yaml +16 -0
- package/scripts/.gitkeep +5 -0
- package/scripts/README-collection-recreation.md +224 -0
- package/scripts/README.md +51 -0
- package/scripts/backup-collections.ts +543 -0
- package/scripts/delete-collection.ts +137 -0
- package/scripts/migrate-recreate-collections.ts +578 -0
- package/scripts/migrate-v1-to-v2.ts +1094 -0
- package/scripts/package-lock.json +1113 -0
- package/scripts/package.json +27 -0
- package/src/collections/composite-ids.ts +193 -0
- package/src/collections/core-infrastructure.spec.ts +353 -0
- package/src/collections/dot-notation.ts +212 -0
- package/src/collections/tracking-arrays.ts +298 -0
- package/src/constants/content-types.ts +20 -0
- package/src/schema/v2-collections-comments.spec.ts +141 -0
- package/src/schema/v2-collections.ts +433 -0
- package/src/server-factory.ts +89 -20
- package/src/server.ts +45 -17
- package/src/services/access-control.spec.ts +383 -0
- package/src/services/access-control.ts +291 -0
- package/src/services/credentials-provider.spec.ts +22 -0
- package/src/services/credentials-provider.ts +34 -0
- package/src/services/escalation.service.spec.ts +183 -0
- package/src/services/escalation.service.ts +150 -0
- package/src/services/ghost-config.service.spec.ts +339 -0
- package/src/services/ghost-config.service.ts +219 -0
- package/src/services/space-config.service.spec.ts +102 -0
- package/src/services/space-config.service.ts +79 -0
- package/src/services/trust-enforcement.spec.ts +309 -0
- package/src/services/trust-enforcement.ts +197 -0
- package/src/services/trust-validator.spec.ts +108 -0
- package/src/services/trust-validator.ts +105 -0
- package/src/tools/confirm-publish-moderation.spec.ts +240 -0
- package/src/tools/confirm.ts +869 -135
- package/src/tools/create-memory.spec.ts +126 -0
- package/src/tools/create-memory.ts +20 -27
- package/src/tools/create-relationship.ts +17 -8
- package/src/tools/delete-memory.ts +13 -6
- package/src/tools/delete-relationship.ts +15 -6
- package/src/tools/deny.ts +8 -1
- package/src/tools/find-similar.ts +21 -8
- package/src/tools/get-preferences.ts +10 -1
- package/src/tools/ghost-config.spec.ts +180 -0
- package/src/tools/ghost-config.ts +230 -0
- package/src/tools/moderate.spec.ts +277 -0
- package/src/tools/moderate.ts +219 -0
- package/src/tools/publish.ts +99 -41
- package/src/tools/query-memory.ts +28 -6
- package/src/tools/query-space.ts +39 -4
- package/src/tools/retract.ts +292 -0
- package/src/tools/revise.spec.ts +146 -0
- package/src/tools/revise.ts +283 -0
- package/src/tools/search-memory.ts +30 -7
- package/src/tools/search-relationship.ts +11 -2
- package/src/tools/search-space.spec.ts +341 -0
- package/src/tools/search-space.ts +323 -99
- package/src/tools/set-preference.ts +10 -1
- package/src/tools/update-memory.ts +16 -5
- package/src/tools/update-relationship.ts +10 -1
- package/src/types/access-result.spec.ts +193 -0
- package/src/types/access-result.ts +62 -0
- package/src/types/auth.ts +52 -0
- package/src/types/ghost-config.ts +46 -0
- package/src/types/memory.ts +9 -1
- package/src/types/preferences.ts +2 -2
- package/src/utils/auth-helpers.spec.ts +75 -0
- package/src/utils/auth-helpers.ts +25 -0
- package/src/utils/test-data-generator.spec.ts +317 -0
- package/src/utils/test-data-generator.ts +292 -0
- package/src/utils/weaviate-filters.ts +4 -4
- package/src/v2-performance.e2e.ts +173 -0
- package/src/v2-smoke.e2e.ts +401 -0
- package/src/weaviate/client.spec.ts +5 -5
- package/src/weaviate/client.ts +51 -36
- package/src/weaviate/schema.ts +11 -256
- package/src/weaviate/space-schema.spec.ts +24 -24
- package/src/weaviate/space-schema.ts +18 -6
package/agent/tasks/milestone-14-memory-collection-v2/task-169-update-remember-search-space.md
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Task 169: Implement remember_search_space Tool
|
|
2
|
+
|
|
3
|
+
**Milestone**: Milestone 14 - Memory Collection Pattern v2
|
|
4
|
+
**Estimated Time**: 4-6 hours
|
|
5
|
+
**Dependencies**: [Task 165: Core Infrastructure Setup](task-165-core-infrastructure-setup.md)
|
|
6
|
+
**Status**: Completed 2026-02-27
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Objective
|
|
11
|
+
|
|
12
|
+
Implement the `remember_search_space` tool to query memories using space_ids filtering and support group searches.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Steps
|
|
17
|
+
|
|
18
|
+
### 1. Create Tool Schema
|
|
19
|
+
|
|
20
|
+
**File**: `src/tools/remember-search-space.ts`
|
|
21
|
+
|
|
22
|
+
**Actions**:
|
|
23
|
+
- Update input schema to accept `spaces` array (filter by spaces)
|
|
24
|
+
- Add `groups` array parameter (filter by groups)
|
|
25
|
+
- Keep `query` parameter (search query)
|
|
26
|
+
- Add `search_type` parameter (hybrid, bm25, semantic)
|
|
27
|
+
- Update tool description
|
|
28
|
+
|
|
29
|
+
**Expected Schema**:
|
|
30
|
+
```typescript
|
|
31
|
+
{
|
|
32
|
+
query: string,
|
|
33
|
+
spaces?: string[], // ["cooking", "recipes"]
|
|
34
|
+
groups?: string[], // ["{group-id}"]
|
|
35
|
+
search_type?: string, // "hybrid" | "bm25" | "semantic"
|
|
36
|
+
limit?: number
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### 2. Implement Space Filtering
|
|
41
|
+
|
|
42
|
+
**Actions**:
|
|
43
|
+
- Query `Memory_spaces_public` collection
|
|
44
|
+
- Filter by `space_ids` array contains any of specified spaces
|
|
45
|
+
- Use Weaviate's array filtering capabilities
|
|
46
|
+
- Return memories with composite IDs
|
|
47
|
+
|
|
48
|
+
**Expected Behavior**:
|
|
49
|
+
```typescript
|
|
50
|
+
// Search specific spaces
|
|
51
|
+
remember_search_space({
|
|
52
|
+
query: "recipe",
|
|
53
|
+
spaces: ["cooking", "recipes"]
|
|
54
|
+
})
|
|
55
|
+
|
|
56
|
+
// Queries: Memory_spaces_public WHERE space_ids contains "cooking" OR "recipes"
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### 3. Implement Group Filtering
|
|
60
|
+
|
|
61
|
+
**Actions**:
|
|
62
|
+
- Query specific `Memory_groups_{groupId}` collections
|
|
63
|
+
- Combine results from multiple group collections
|
|
64
|
+
- Return memories with composite IDs
|
|
65
|
+
|
|
66
|
+
**Expected Behavior**:
|
|
67
|
+
```typescript
|
|
68
|
+
// Search specific groups
|
|
69
|
+
remember_search_space({
|
|
70
|
+
query: "recipe",
|
|
71
|
+
groups: ["{foodie-group}", "{recipe-club}"]
|
|
72
|
+
})
|
|
73
|
+
|
|
74
|
+
// Queries: Memory_groups_{foodie-group} AND Memory_groups_{recipe-club}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### 4. Implement Combined Search
|
|
78
|
+
|
|
79
|
+
**Actions**:
|
|
80
|
+
- Support searching both spaces and groups simultaneously
|
|
81
|
+
- Merge results from both query types
|
|
82
|
+
- Deduplicate by composite ID
|
|
83
|
+
- Sort by relevance score
|
|
84
|
+
|
|
85
|
+
**Expected Behavior**:
|
|
86
|
+
```typescript
|
|
87
|
+
// Search both spaces and groups
|
|
88
|
+
remember_search_space({
|
|
89
|
+
query: "recipe",
|
|
90
|
+
spaces: ["cooking"],
|
|
91
|
+
groups: ["{foodie-group}"]
|
|
92
|
+
})
|
|
93
|
+
|
|
94
|
+
// Queries both Memory_spaces_public and Memory_groups_{foodie-group}
|
|
95
|
+
// Returns merged, deduplicated results
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
### 5. Implement All-Public Search
|
|
99
|
+
|
|
100
|
+
**Actions**:
|
|
101
|
+
- When no spaces or groups specified, search all public memories
|
|
102
|
+
- Query entire `Memory_spaces_public` collection
|
|
103
|
+
- Return all matching memories regardless of space_ids
|
|
104
|
+
|
|
105
|
+
**Expected Behavior**:
|
|
106
|
+
```typescript
|
|
107
|
+
// Search all public memories
|
|
108
|
+
remember_search_space({
|
|
109
|
+
query: "recipe"
|
|
110
|
+
})
|
|
111
|
+
|
|
112
|
+
// Queries: Memory_spaces_public (all memories)
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
### 6. Write Integration Tests
|
|
116
|
+
|
|
117
|
+
**File**: `tests/integration/remember-search-space.test.ts`
|
|
118
|
+
|
|
119
|
+
**Actions**:
|
|
120
|
+
- Test space filtering (single and multiple)
|
|
121
|
+
- Test group filtering (single and multiple)
|
|
122
|
+
- Test combined search (spaces + groups)
|
|
123
|
+
- Test all-public search
|
|
124
|
+
- Test deduplication
|
|
125
|
+
- Test search types (hybrid, bm25, semantic)
|
|
126
|
+
- Test error cases
|
|
127
|
+
|
|
128
|
+
**Expected Output**: All tests passing
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## Verification
|
|
133
|
+
|
|
134
|
+
- [ ] Tool schema updated correctly
|
|
135
|
+
- [ ] Space filtering works correctly
|
|
136
|
+
- [ ] Group filtering works correctly
|
|
137
|
+
- [ ] Combined search works correctly
|
|
138
|
+
- [ ] All-public search works correctly
|
|
139
|
+
- [ ] Deduplication works correctly
|
|
140
|
+
- [ ] Search types work correctly
|
|
141
|
+
- [ ] Integration tests passing
|
|
142
|
+
- [ ] TypeScript compiles without errors
|
|
143
|
+
- [ ] Tool description updated in MCP manifest
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Expected Output
|
|
148
|
+
|
|
149
|
+
### Files Modified
|
|
150
|
+
- `src/tools/remember-search-space-v2.ts` (~220 lines)
|
|
151
|
+
|
|
152
|
+
### Files Created
|
|
153
|
+
- `tests/integration/remember-search-space.test.ts` (~180 lines)
|
|
154
|
+
|
|
155
|
+
### Test Output
|
|
156
|
+
```
|
|
157
|
+
✓ Space filtering (4 tests)
|
|
158
|
+
✓ Group filtering (4 tests)
|
|
159
|
+
✓ Combined search (4 tests)
|
|
160
|
+
✓ All-public search (3 tests)
|
|
161
|
+
✓ Deduplication (3 tests)
|
|
162
|
+
✓ Search types (3 tests)
|
|
163
|
+
✓ Error handling (3 tests)
|
|
164
|
+
|
|
165
|
+
Total: 24 tests passing
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
---
|
|
169
|
+
|
|
170
|
+
## Notes
|
|
171
|
+
|
|
172
|
+
- This is a breaking change from v1 (single space only)
|
|
173
|
+
- Deduplication critical for combined searches
|
|
174
|
+
- Consider caching frequently searched spaces
|
|
175
|
+
- Performance testing needed for large collections
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
**Next Task**: [Task 170: Update remember_create_memory Tool](task-170-update-remember-create-memory.md)
|
package/agent/tasks/milestone-14-memory-collection-v2/task-170-update-remember-create-update.md
ADDED
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# Task 170: Implement remember_create_memory and remember_update_memory Tools
|
|
2
|
+
|
|
3
|
+
**Milestone**: Milestone 14 - Memory Collection Pattern v2
|
|
4
|
+
**Estimated Time**: 3-4 hours
|
|
5
|
+
**Dependencies**: [Task 165: Core Infrastructure Setup](task-165-core-infrastructure-setup.md)
|
|
6
|
+
**Status**: Completed 2026-02-27
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Objective
|
|
11
|
+
|
|
12
|
+
Implement `remember_create_memory` and `remember_update_memory` tools to initialize and maintain tracking arrays (space_ids, group_ids) in user memories.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Steps
|
|
17
|
+
|
|
18
|
+
### 1. Implement remember_create_memory Tool
|
|
19
|
+
|
|
20
|
+
**File**: `src/tools/remember-create-memory.ts`
|
|
21
|
+
|
|
22
|
+
**Actions**:
|
|
23
|
+
- Add `space_ids: []` to new memory objects (empty array)
|
|
24
|
+
- Add `group_ids: []` to new memory objects (empty array)
|
|
25
|
+
- Ensure memories created in `Memory_users_{userId}` collection
|
|
26
|
+
- Keep all existing functionality
|
|
27
|
+
|
|
28
|
+
**Expected Behavior**:
|
|
29
|
+
```typescript
|
|
30
|
+
// Create new memory
|
|
31
|
+
remember_create_memory({
|
|
32
|
+
content: "My recipe",
|
|
33
|
+
content_type: "note"
|
|
34
|
+
})
|
|
35
|
+
|
|
36
|
+
// Creates in Memory_users_{userId}:
|
|
37
|
+
{
|
|
38
|
+
id: "auto-generated-id",
|
|
39
|
+
content: "My recipe",
|
|
40
|
+
content_type: "note",
|
|
41
|
+
space_ids: [], // ← New field
|
|
42
|
+
group_ids: [], // ← New field
|
|
43
|
+
created_at: timestamp,
|
|
44
|
+
updated_at: timestamp
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### 2. Implement remember_update_memory Tool
|
|
49
|
+
|
|
50
|
+
**File**: `src/tools/remember-update-memory.ts`
|
|
51
|
+
|
|
52
|
+
**Actions**:
|
|
53
|
+
- Preserve `space_ids` and `group_ids` arrays during updates
|
|
54
|
+
- Don't allow direct modification of tracking arrays (use publish/retract instead)
|
|
55
|
+
- Update `updated_at` timestamp
|
|
56
|
+
- Keep all existing functionality
|
|
57
|
+
|
|
58
|
+
**Expected Behavior**:
|
|
59
|
+
```typescript
|
|
60
|
+
// Update memory content
|
|
61
|
+
remember_update_memory({
|
|
62
|
+
id: "my-recipe",
|
|
63
|
+
content: "Updated recipe"
|
|
64
|
+
})
|
|
65
|
+
|
|
66
|
+
// Updates in Memory_users_{userId}:
|
|
67
|
+
{
|
|
68
|
+
id: "my-recipe",
|
|
69
|
+
content: "Updated recipe", // ← Updated
|
|
70
|
+
space_ids: ["cooking"], // ← Preserved
|
|
71
|
+
group_ids: [], // ← Preserved
|
|
72
|
+
updated_at: timestamp // ← Updated
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### 3. Add Validation
|
|
77
|
+
|
|
78
|
+
**Actions**:
|
|
79
|
+
- Validate `space_ids` and `group_ids` are arrays
|
|
80
|
+
- Prevent direct modification of tracking arrays
|
|
81
|
+
- Ensure arrays are initialized on create
|
|
82
|
+
- Handle migration of old memories without tracking arrays
|
|
83
|
+
|
|
84
|
+
### 4. Write Unit Tests
|
|
85
|
+
|
|
86
|
+
**File**: `tests/unit/remember-create-update.test.ts`
|
|
87
|
+
|
|
88
|
+
**Actions**:
|
|
89
|
+
- Test memory creation with tracking arrays
|
|
90
|
+
- Test memory update preserves tracking arrays
|
|
91
|
+
- Test validation of tracking arrays
|
|
92
|
+
- Test error cases
|
|
93
|
+
|
|
94
|
+
**Expected Output**: All tests passing
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## Verification
|
|
99
|
+
|
|
100
|
+
- [ ] remember_create_memory initializes tracking arrays
|
|
101
|
+
- [ ] remember_update_memory preserves tracking arrays
|
|
102
|
+
- [ ] Tracking arrays cannot be directly modified
|
|
103
|
+
- [ ] Validation works correctly
|
|
104
|
+
- [ ] Unit tests passing
|
|
105
|
+
- [ ] TypeScript compiles without errors
|
|
106
|
+
- [ ] Tool descriptions updated
|
|
107
|
+
|
|
108
|
+
---
|
|
109
|
+
|
|
110
|
+
## Expected Output
|
|
111
|
+
|
|
112
|
+
### Files Modified
|
|
113
|
+
- `src/tools/remember-create-memory-v2.ts` (~150 lines)
|
|
114
|
+
- `src/tools/remember-update-memory-v2.ts` (~150 lines)
|
|
115
|
+
|
|
116
|
+
### Files Created
|
|
117
|
+
- `tests/unit/remember-create-update.test.ts` (~100 lines)
|
|
118
|
+
|
|
119
|
+
### Test Output
|
|
120
|
+
```
|
|
121
|
+
✓ Memory creation (5 tests)
|
|
122
|
+
✓ Memory update (5 tests)
|
|
123
|
+
✓ Tracking array validation (4 tests)
|
|
124
|
+
✓ Error handling (3 tests)
|
|
125
|
+
|
|
126
|
+
Total: 17 tests passing
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Notes
|
|
132
|
+
|
|
133
|
+
- Minimal changes to existing tools
|
|
134
|
+
- Tracking arrays managed by publish/retract/revise tools
|
|
135
|
+
- Backward compatibility for old memories without tracking arrays
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
**Next Task**: [Task 171: Implement Migration Script](task-171-implement-migration-script.md)
|
package/agent/tasks/milestone-14-memory-collection-v2/task-172-performance-testing-optimization.md
ADDED
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Task 172: Performance Testing and Optimization
|
|
2
|
+
|
|
3
|
+
**Milestone**: Milestone 14 - Memory Collection Pattern v2
|
|
4
|
+
**Estimated Time**: 6-8 hours
|
|
5
|
+
**Dependencies**: All previous tasks
|
|
6
|
+
**Status**: Not Started
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Objective
|
|
11
|
+
|
|
12
|
+
Conduct comprehensive performance testing of Memory Collection Pattern v2 and optimize query performance to meet targets (<200ms for typical queries).
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Steps
|
|
17
|
+
|
|
18
|
+
### 1. Create Performance Test Suite
|
|
19
|
+
|
|
20
|
+
**File**: `tests/performance/v2-performance.test.ts`
|
|
21
|
+
|
|
22
|
+
**Actions**:
|
|
23
|
+
- Create test data generator (1K, 10K, 100K memories)
|
|
24
|
+
- Implement query performance tests
|
|
25
|
+
- Implement write performance tests
|
|
26
|
+
- Implement migration performance tests
|
|
27
|
+
- Set performance baselines
|
|
28
|
+
|
|
29
|
+
### 2. Test Query Performance
|
|
30
|
+
|
|
31
|
+
**Scenarios**:
|
|
32
|
+
- Search single space (small: <100 memories)
|
|
33
|
+
- Search single space (medium: 1K-10K memories)
|
|
34
|
+
- Search single space (large: >10K memories)
|
|
35
|
+
- Search multiple spaces
|
|
36
|
+
- Search groups
|
|
37
|
+
- Search all public memories
|
|
38
|
+
- Composite ID lookups
|
|
39
|
+
|
|
40
|
+
**Target**: <200ms for typical queries (1K-10K memories)
|
|
41
|
+
|
|
42
|
+
### 3. Test Write Performance
|
|
43
|
+
|
|
44
|
+
**Scenarios**:
|
|
45
|
+
- Create memory
|
|
46
|
+
- Update memory
|
|
47
|
+
- Publish to single space
|
|
48
|
+
- Publish to multiple spaces
|
|
49
|
+
- Publish to groups
|
|
50
|
+
- Retract from spaces
|
|
51
|
+
- Revise published memories
|
|
52
|
+
|
|
53
|
+
**Target**: <100ms for write operations
|
|
54
|
+
|
|
55
|
+
### 4. Test Migration Performance
|
|
56
|
+
|
|
57
|
+
**Scenarios**:
|
|
58
|
+
- Migrate 1K memories
|
|
59
|
+
- Migrate 10K memories
|
|
60
|
+
- Migrate 100K memories
|
|
61
|
+
- Backup creation
|
|
62
|
+
- Rollback execution
|
|
63
|
+
|
|
64
|
+
**Target**: <10 minutes for 10K memories
|
|
65
|
+
|
|
66
|
+
### 5. Identify Bottlenecks
|
|
67
|
+
|
|
68
|
+
**Actions**:
|
|
69
|
+
- Profile query execution
|
|
70
|
+
- Identify slow operations
|
|
71
|
+
- Analyze Weaviate query plans
|
|
72
|
+
- Check index usage
|
|
73
|
+
- Monitor memory usage
|
|
74
|
+
|
|
75
|
+
### 6. Implement Optimizations
|
|
76
|
+
|
|
77
|
+
**Potential Optimizations**:
|
|
78
|
+
- Add Weaviate indexes on space_ids and group_ids
|
|
79
|
+
- Optimize composite ID generation
|
|
80
|
+
- Batch operations where possible
|
|
81
|
+
- Cache frequently accessed data
|
|
82
|
+
- Optimize tracking array operations
|
|
83
|
+
- Parallelize migration steps
|
|
84
|
+
|
|
85
|
+
### 7. Benchmark Results
|
|
86
|
+
|
|
87
|
+
**Actions**:
|
|
88
|
+
- Run performance tests before optimization
|
|
89
|
+
- Run performance tests after optimization
|
|
90
|
+
- Compare results
|
|
91
|
+
- Document improvements
|
|
92
|
+
- Generate performance report
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Verification
|
|
97
|
+
|
|
98
|
+
- [ ] Performance test suite created
|
|
99
|
+
- [ ] Query performance meets targets (<200ms)
|
|
100
|
+
- [ ] Write performance meets targets (<100ms)
|
|
101
|
+
- [ ] Migration performance meets targets (<10 min for 10K)
|
|
102
|
+
- [ ] Bottlenecks identified and documented
|
|
103
|
+
- [ ] Optimizations implemented
|
|
104
|
+
- [ ] Benchmarks show improvement
|
|
105
|
+
- [ ] Performance report generated
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## Expected Output
|
|
110
|
+
|
|
111
|
+
### Files Created
|
|
112
|
+
- `tests/performance/v2-performance.test.ts` (~400 lines)
|
|
113
|
+
- `docs/performance/v2-benchmarks.md` (~50 lines)
|
|
114
|
+
|
|
115
|
+
### Performance Report
|
|
116
|
+
```
|
|
117
|
+
Memory Collection Pattern v2 - Performance Benchmarks
|
|
118
|
+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
|
119
|
+
|
|
120
|
+
Query Performance (10K memories):
|
|
121
|
+
Single space search: 145ms ✓ (target: <200ms)
|
|
122
|
+
Multi-space search: 178ms ✓ (target: <200ms)
|
|
123
|
+
Group search: 132ms ✓ (target: <200ms)
|
|
124
|
+
All public search: 189ms ✓ (target: <200ms)
|
|
125
|
+
Composite ID lookup: 12ms ✓ (target: <50ms)
|
|
126
|
+
|
|
127
|
+
Write Performance:
|
|
128
|
+
Create memory: 45ms ✓ (target: <100ms)
|
|
129
|
+
Update memory: 38ms ✓ (target: <100ms)
|
|
130
|
+
Publish (single space): 67ms ✓ (target: <100ms)
|
|
131
|
+
Publish (multi-space): 89ms ✓ (target: <100ms)
|
|
132
|
+
Retract: 52ms ✓ (target: <100ms)
|
|
133
|
+
Revise: 78ms ✓ (target: <100ms)
|
|
134
|
+
|
|
135
|
+
Migration Performance:
|
|
136
|
+
1K memories: 1.2 min ✓ (target: <2 min)
|
|
137
|
+
10K memories: 8.5 min ✓ (target: <10 min)
|
|
138
|
+
100K memories: 82.3 min ✓ (target: <120 min)
|
|
139
|
+
|
|
140
|
+
Optimizations Applied:
|
|
141
|
+
✓ Added indexes on space_ids and group_ids
|
|
142
|
+
✓ Optimized composite ID generation (30% faster)
|
|
143
|
+
✓ Batched tracking array updates (40% faster)
|
|
144
|
+
✓ Parallelized migration steps (2x faster)
|
|
145
|
+
|
|
146
|
+
Overall: All performance targets met ✓
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Notes
|
|
152
|
+
|
|
153
|
+
- Performance testing should use realistic data volumes
|
|
154
|
+
- Test on hardware similar to production
|
|
155
|
+
- Consider network latency in benchmarks
|
|
156
|
+
- Document any performance regressions
|
|
157
|
+
- Optimization should not compromise correctness
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
**Next Task**: [Task 173: Documentation and Examples](task-173-documentation-examples.md)
|