@drqedwards/pmll 2.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +26 -0
- package/agent_instructions.md +123 -0
- package/benchmarks/benchmark_retrieval.md +94 -0
- package/benchmarks/contextplus-standalone-speed.md +185 -0
- package/benchmarks/run_retrieval_stub.py +343 -0
- package/benchmarks/speed-test-results.md +163 -0
- package/benchmarks/three-way-speed-comparison.md +238 -0
- package/dist/embeddings.d.ts +32 -0
- package/dist/embeddings.d.ts.map +1 -0
- package/dist/embeddings.js +140 -0
- package/dist/embeddings.js.map +1 -0
- package/dist/graphql.d.ts +81 -0
- package/dist/graphql.d.ts.map +1 -0
- package/dist/graphql.js +389 -0
- package/dist/graphql.js.map +1 -0
- package/dist/index.d.ts +55 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +645 -0
- package/dist/index.js.map +1 -0
- package/dist/kv-store.d.ts +32 -0
- package/dist/kv-store.d.ts.map +1 -0
- package/dist/kv-store.js +103 -0
- package/dist/kv-store.js.map +1 -0
- package/dist/memory-graph.d.ts +122 -0
- package/dist/memory-graph.d.ts.map +1 -0
- package/dist/memory-graph.js +345 -0
- package/dist/memory-graph.js.map +1 -0
- package/dist/peek.d.ts +67 -0
- package/dist/peek.d.ts.map +1 -0
- package/dist/peek.js +61 -0
- package/dist/peek.js.map +1 -0
- package/dist/q-promise-bridge.d.ts +58 -0
- package/dist/q-promise-bridge.d.ts.map +1 -0
- package/dist/q-promise-bridge.js +88 -0
- package/dist/q-promise-bridge.js.map +1 -0
- package/dist/solution-engine.d.ts +54 -0
- package/dist/solution-engine.d.ts.map +1 -0
- package/dist/solution-engine.js +73 -0
- package/dist/solution-engine.js.map +1 -0
- package/package.json +69 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 drQedwards
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# pmll
|
|
2
|
+
|
|
3
|
+
Persistent Memory Logic Loop MCP server. Short name for [pmll-memory-mcp](https://www.npmjs.com/package/pmll-memory-mcp).
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npx pmll
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Same tools as pmll-memory-mcp v2.0.4: peek KV cache, Q-promise dedup, Context+ long-term graph, solution engine.
|
|
10
|
+
|
|
11
|
+
Combined Context+ + PMLL/peek benchmark (fastest config): 36ms TypeScript / 78ms Python.
|
|
12
|
+
|
|
13
|
+
Repo: https://github.com/drQedwards/PPM and https://github.com/drQedwards/pmll
|
|
14
|
+
|
|
15
|
+
## MCP config
|
|
16
|
+
|
|
17
|
+
```json
|
|
18
|
+
{
|
|
19
|
+
"mcpServers": {
|
|
20
|
+
"pmll": {
|
|
21
|
+
"command": "npx",
|
|
22
|
+
"args": ["-y", "pmll"]
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
}
|
|
26
|
+
```
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# Context+ MCP - Agent Workflow
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Context+ gives you structural awareness of the entire codebase without reading every file. These tools replace your default search and read operations — use them as your primary interface to the codebase.
|
|
6
|
+
|
|
7
|
+
## PMLL Short-Term KV Memory (5 Tools)
|
|
8
|
+
|
|
9
|
+
Before every expensive MCP tool invocation, agents MUST use the PMLL short-term KV memory tools to check the cache. This eliminates redundant calls and accelerates task execution. These tools are provided by the [`pmll-memory-mcp`](https://www.npmjs.com/package/pmll-memory-mcp) server (see [mcp/README.md](./README.md)).
|
|
10
|
+
|
|
11
|
+
| Tool | Input | Description |
|
|
12
|
+
|-----------|----------------------------------------------------|------------------------------------------------------------|
|
|
13
|
+
| `init` | `session_id: str`, `silo_size: int = 256` | Initialize a PMLL silo and Q-promise chain for the session |
|
|
14
|
+
| `peek` | `session_id: str`, `key: str` | Non-destructive cache lookup + Q-promise check (call before every expensive tool) |
|
|
15
|
+
| `set` | `session_id: str`, `key: str`, `value: str` | Store a key-value pair in the silo after a cache miss |
|
|
16
|
+
| `resolve` | `session_id: str`, `promise_id: str` | Check or resolve a Q-promise continuation |
|
|
17
|
+
| `flush` | `session_id: str` | Clear all silo slots at task completion |
|
|
18
|
+
|
|
19
|
+
### The `peek()` Pattern
|
|
20
|
+
|
|
21
|
+
Call `peek` before every expensive tool invocation:
|
|
22
|
+
|
|
23
|
+
1. **`init`** once at task start to set up the session silo
|
|
24
|
+
2. **`peek`** before each expensive call — if hit, use the cached value; if pending, wait on the Q-promise
|
|
25
|
+
3. **`set`** after a cache miss to populate the silo for future agents/subtasks
|
|
26
|
+
4. **`resolve`** to check or fulfill Q-promise continuations
|
|
27
|
+
5. **`flush`** at task end to clear all session slots
|
|
28
|
+
|
|
29
|
+
This pattern ensures that Context+ tool results, Playwright page contents, and other expensive outputs are cached and reused across subtasks rather than re-fetched.
|
|
30
|
+
|
|
31
|
+
## Tool Priority (Mandatory)
|
|
32
|
+
|
|
33
|
+
You MUST use Context+ tools instead of native equivalents. Only fall back to native tools when a Context+ tool cannot fulfill the specific need.
|
|
34
|
+
|
|
35
|
+
| Instead of… | MUST use… | Why |
|
|
36
|
+
|--------------------------|------------------------------|----------------------------------------------|
|
|
37
|
+
| `grep`, `rg`, `ripgrep` | `semantic_code_search` | Finds by meaning, not just string match |
|
|
38
|
+
| `find`, `ls`, `glob` | `get_context_tree` | Returns structure with symbols + line ranges |
|
|
39
|
+
| `cat`, `head`, read file | `get_file_skeleton` first | Signatures without wasting context on bodies |
|
|
40
|
+
| manual symbol tracing | `get_blast_radius` | Traces all usages across the entire codebase |
|
|
41
|
+
| keyword search | `semantic_identifier_search` | Ranked definitions + call chains |
|
|
42
|
+
| directory browsing | `semantic_navigate` | Browse by meaning, not file paths |
|
|
43
|
+
|
|
44
|
+
## Workflow
|
|
45
|
+
|
|
46
|
+
1. Start every task with `get_context_tree` or `get_file_skeleton` for structural overview
|
|
47
|
+
2. Use `semantic_code_search` or `semantic_identifier_search` to find code by meaning
|
|
48
|
+
3. Run `get_blast_radius` BEFORE modifying or deleting any symbol
|
|
49
|
+
4. Prefer structural tools over full-file reads — only read full files when signatures are insufficient
|
|
50
|
+
5. Run `run_static_analysis` after writing code
|
|
51
|
+
6. Use `search_memory_graph` at task start for prior context, `upsert_memory_node` after completing work
|
|
52
|
+
|
|
53
|
+
## Execution Rules
|
|
54
|
+
|
|
55
|
+
- Think less, execute sooner: make the smallest safe change that can be validated quickly
|
|
56
|
+
- Batch independent reads/searches in parallel — do not serialize them
|
|
57
|
+
- If a command fails, diagnose once, pivot strategy, continue — cap retries to 1-2
|
|
58
|
+
- Keep outputs concise: short status updates, no verbose reasoning
|
|
59
|
+
|
|
60
|
+
## Tool Reference
|
|
61
|
+
|
|
62
|
+
### PMLL Short-Term KV Memory
|
|
63
|
+
|
|
64
|
+
| Tool | When to Use |
|
|
65
|
+
|-----------|------------------------------------------------------------------------------|
|
|
66
|
+
| `init` | Once at task start. Set up the PMLL silo and Q-promise chain for the session.|
|
|
67
|
+
| `peek` | Before every expensive MCP tool call. Non-destructive cache + Q-promise check.|
|
|
68
|
+
| `set` | After a cache miss. Store the result so future agents/subtasks skip the call. |
|
|
69
|
+
| `resolve` | When a Q-promise is pending. Check or fulfill the continuation. |
|
|
70
|
+
| `flush` | At task end. Clear all silo slots for the session. |
|
|
71
|
+
|
|
72
|
+
### GraphQL
|
|
73
|
+
|
|
74
|
+
| Tool | When to Use |
|
|
75
|
+
|-----------|------------------------------------------------------------------------------|
|
|
76
|
+
| `graphql` | Execute GraphQL queries/mutations against the memory store with optional PMLL cache integration. |
|
|
77
|
+
|
|
78
|
+
### Context+ Structural Tools
|
|
79
|
+
|
|
80
|
+
| Tool | When to Use |
|
|
81
|
+
|-----------------------------|--------------------------------------------------------------|
|
|
82
|
+
| `get_context_tree` | Start of every task. Map files + symbols with line ranges. |
|
|
83
|
+
| `get_file_skeleton` | Before full reads. Get signatures + line ranges first. |
|
|
84
|
+
| `semantic_code_search` | Find relevant files by concept. |
|
|
85
|
+
| `semantic_identifier_search`| Find functions/classes/variables and their call chains. |
|
|
86
|
+
| `semantic_navigate` | Browse codebase by meaning, not directory structure. |
|
|
87
|
+
| `get_blast_radius` | Before deleting or modifying any symbol. |
|
|
88
|
+
| `get_feature_hub` | Browse feature graph hubs. Find orphaned files. |
|
|
89
|
+
| `run_static_analysis` | After writing code. Catch errors deterministically. |
|
|
90
|
+
| `propose_commit` | Validate and save file changes. |
|
|
91
|
+
| `list_restore_points` | See undo history. |
|
|
92
|
+
| `undo_change` | Revert a change without touching git. |
|
|
93
|
+
|
|
94
|
+
### Long-Term Memory Graph
|
|
95
|
+
|
|
96
|
+
| Tool | When to Use |
|
|
97
|
+
|-----------------------------|--------------------------------------------------------------|
|
|
98
|
+
| `upsert_memory_node` | Create/update memory nodes (concept, file, symbol, note). |
|
|
99
|
+
| `create_relation` | Create typed edges between memory nodes. |
|
|
100
|
+
| `search_memory_graph` | Semantic search + graph traversal across neighbors. |
|
|
101
|
+
| `prune_stale_links` | Remove decayed edges and orphan nodes. |
|
|
102
|
+
| `add_interlinked_context` | Bulk-add nodes with auto-similarity linking. |
|
|
103
|
+
| `retrieve_with_traversal` | Walk outward from a node, return scored neighbors. |
|
|
104
|
+
|
|
105
|
+
### Solution Engine
|
|
106
|
+
|
|
107
|
+
| Tool | When to Use |
|
|
108
|
+
|------------------------|-----------------------------------------------------------------------|
|
|
109
|
+
| `resolve_context` | Unified context lookup — checks short-term KV first, falls back to long-term semantic graph. |
|
|
110
|
+
| `promote_to_long_term` | Promote a frequently-accessed short-term KV entry to the long-term memory graph. |
|
|
111
|
+
| `memory_status` | Get a unified view of both short-term (KV cache) and long-term (semantic graph) memory layers. |
|
|
112
|
+
|
|
113
|
+
## Anti-Patterns
|
|
114
|
+
|
|
115
|
+
1. Reading entire files without checking the skeleton first
|
|
116
|
+
2. Deleting functions without checking blast radius
|
|
117
|
+
3. Running independent commands sequentially when they can be parallelized
|
|
118
|
+
4. Repeating failed commands without changing approach
|
|
119
|
+
5. Calling expensive MCP tools without calling `peek` first to check the cache
|
|
120
|
+
6. Forgetting to call `init` at task start or `flush` at task end, causing silent cache misses or stale data across sessions
|
|
121
|
+
7. Storing frequently-accessed payloads only in short-term KV instead of promoting them to long-term memory with `promote_to_long_term`
|
|
122
|
+
8. Calling `search_memory_graph` or `retrieve_with_traversal` directly instead of using `resolve_context`, which checks both memory layers in one call
|
|
123
|
+
9. Ignoring Q-promise `pending` status from `peek` and re-issuing the same expensive call instead of waiting with `resolve`
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Retrieval-quality benchmark harness (design)
|
|
2
|
+
|
|
3
|
+
This document defines how we will measure **memory-graph retrieval quality**
|
|
4
|
+
for `pmll_memory_mcp`. It is a unit-level retrieval bench — **not** an
|
|
5
|
+
end-to-end agent-success scorecard.
|
|
6
|
+
|
|
7
|
+
> **Accuracy (this harness)** = retrieval hit rate on labeled relevant nodes
|
|
8
|
+
> (precision@k / recall@k / MRR over synthetic queries). It is **not** agent
|
|
9
|
+
> task success, pass@k on coding tasks, or any product “X% accurate” claim.
|
|
10
|
+
> Do not cite this stub (or a future full run of this harness alone) as
|
|
11
|
+
> agent accuracy.
|
|
12
|
+
|
|
13
|
+
Speed benches already live beside this file (`*-speed*.md`). This harness
|
|
14
|
+
is the quality counterpart.
|
|
15
|
+
|
|
16
|
+
Runnable stub: [`run_retrieval_stub.py`](./run_retrieval_stub.py).
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Required fields for any future accuracy claim
|
|
21
|
+
|
|
22
|
+
Any published retrieval-quality number **must** record all of the following.
|
|
23
|
+
Claims missing a field are incomplete.
|
|
24
|
+
|
|
25
|
+
| Field | What to record |
|
|
26
|
+
|-------|----------------|
|
|
27
|
+
| **Dataset** | Synthetic coding-agent contexts: labeled `concept` / `file` / `symbol` / `note` nodes + edges. Version or hash of the seed set. |
|
|
28
|
+
| **Task defs** | (1) retrieve relevant node(s) given a natural-language query; (2) promote KV → graph then retrieve; (3) restart durability (write → new process / reload → retrieve). |
|
|
29
|
+
| **Baselines** | `no_graph` (empty / chance); `tfidf_legacy` if present; `hashing_only` (embed cosine, depth=0); `hashing+traversal` (embed + neighbor walk). Optional: vanilla KV peek / no memory. |
|
|
30
|
+
| **n trials** | Number of labeled queries (and bootstrap resamples if CI reported). |
|
|
31
|
+
| **Model / version** | **N/A** for pure retrieval unit bench (hashing embed + graph). Note separately if an agent E2E eval is attached — that is a different harness. |
|
|
32
|
+
| **Prompts** | **N/A** for retrieval unit. Record **retrieval config**: `top_k`, decay (`DECAY_LAMBDA`), `max_depth`, similarity / stale thresholds. |
|
|
33
|
+
| **Metrics** | `precision@k`, `recall@k`, `MRR` (mean reciprocal rank of first relevant hit). |
|
|
34
|
+
| **Confidence intervals** | Method (e.g. bootstrap percentile over queries, or Wilson for binary hit@k) + level (e.g. 95%). |
|
|
35
|
+
| **Comparison** | Same metrics vs vanilla KV peek / no-memory baseline on the same labeled queries. |
|
|
36
|
+
| **“Accuracy” definition** | Exactly: fraction of queries where a labeled relevant node appears in the top-k retrieved set (hit rate), plus the ranked metrics above — **not** agent task success. |
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Dataset sketch
|
|
41
|
+
|
|
42
|
+
Tiny seed (stub) and a future larger pack should share the same schema:
|
|
43
|
+
|
|
44
|
+
```text
|
|
45
|
+
node: {id_label, type, content, relevant_for: [query_ids...]}
|
|
46
|
+
edge: {source_label, target_label, relation}
|
|
47
|
+
query: {id, text, relevant_labels: [...], task: retrieve|promote_kv|restart}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Content themes: auth modules, config loaders, test helpers, API handlers —
|
|
51
|
+
the kinds of fragments a coding agent would store across a session.
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Baselines & configs
|
|
56
|
+
|
|
57
|
+
Default retrieval config for claims (override and record if changed):
|
|
58
|
+
|
|
59
|
+
- `top_k=5`
|
|
60
|
+
- `max_depth=0` (hashing-only) vs `max_depth=1..2` (hashing+traversal)
|
|
61
|
+
- `DECAY_LAMBDA` / `SIMILARITY_THRESHOLD` as in `memory_graph.py`
|
|
62
|
+
|
|
63
|
+
Baselines to report side-by-side on the same query set:
|
|
64
|
+
|
|
65
|
+
1. **no_graph** — empty store (sanity: metrics ≈ 0)
|
|
66
|
+
2. **hashing_only** — `search_graph(..., max_depth=0)`
|
|
67
|
+
3. **hashing+traversal** — `search_graph(..., max_depth≥1)`
|
|
68
|
+
4. **tfidf_legacy** — optional; only if a frozen TF-IDF path is exercised for comparison (not the production query path)
|
|
69
|
+
5. **kv_peek / no_memory** — exact-key peek or empty memory (shows graph value-add)
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Metrics formulas
|
|
74
|
+
|
|
75
|
+
For query \(q\) with relevant label set \(R_q\), retrieved top-\(k\) labels \(L_q\):
|
|
76
|
+
|
|
77
|
+
- \(\mathrm{precision@}k = |L_q \cap R_q| / k\) (or `/|L_q|` if fewer than \(k\) returned — record which)
|
|
78
|
+
- \(\mathrm{recall@}k = |L_q \cap R_q| / |R_q|\)
|
|
79
|
+
- \(\mathrm{hit@}k = 1\) if \(L_q \cap R_q \neq \emptyset\), else \(0\)
|
|
80
|
+
- \(\mathrm{MRR} = \mathrm{mean}_q\, 1/\mathrm{rank}\) of first relevant label (0 if none)
|
|
81
|
+
|
|
82
|
+
Aggregate means over queries; report CI via the method declared above.
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## Out of scope (separate harness)
|
|
87
|
+
|
|
88
|
+
- Agent E2E task success, tool-choice quality, latency SLOs (see speed benches)
|
|
89
|
+
- LLM judge scores, prompt ablations
|
|
90
|
+
- Any “99%” / product accuracy marketing copy — **not derived from this file**
|
|
91
|
+
|
|
92
|
+
When a full labeled pack lands, link results here and keep README claims
|
|
93
|
+
tied to this harness definition — never invent a percentage without the
|
|
94
|
+
table of required fields filled in.
|
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
# Context+ MCP Tools — Standalone Speed Test Results
|
|
2
|
+
|
|
3
|
+
> **Date**: 2026-04-04
|
|
4
|
+
> **Environment**: Linux (GitHub Actions runner), Node.js 18+, Python 3.12.3
|
|
5
|
+
> **Test runner**: Vitest 3.2.4 (TypeScript), pytest 9.0.2 (Python)
|
|
6
|
+
> **Reference repos**: [ForLoopCodes/contextplus](https://github.com/ForLoopCodes/contextplus), [drQedwards/PPM](https://github.com/drQedwards/PPM)
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Overview
|
|
11
|
+
|
|
12
|
+
This document benchmarks the **Context+ MCP tools in isolation** — the 6 long-term memory graph tools adapted from [Context+](https://github.com/ForLoopCodes/contextplus) — **without any `peek`/KV cache layer involved**. This shows how fast Context+'s graph operations are by themselves.
|
|
13
|
+
|
|
14
|
+
The tools measured:
|
|
15
|
+
- `upsert_memory_node` — create/update memory nodes with TF-IDF embeddings
|
|
16
|
+
- `create_relation` — create typed edges between nodes
|
|
17
|
+
- `search_memory_graph` — semantic search with graph traversal
|
|
18
|
+
- `prune_stale_links` — remove decayed edges and orphan nodes
|
|
19
|
+
- `add_interlinked_context` — bulk-add nodes with auto-similarity linking
|
|
20
|
+
- `retrieve_with_traversal` — walk outward from a node, return scored neighbors
|
|
21
|
+
|
|
22
|
+
Also benchmarks the underlying TF-IDF embedding engine (`tokenize`, `TfIdfVectorizer`, `cosineSimilarity`, `embed`).
|
|
23
|
+
|
|
24
|
+
**No KV store. No peek. No short-term cache. Pure Context+ graph speed.**
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## Test Suite
|
|
29
|
+
|
|
30
|
+
| Suite | Framework | Tests | File |
|
|
31
|
+
|-------|-----------|-------|------|
|
|
32
|
+
| TypeScript | Vitest 3.2.4 | 27 | `__tests__/contextplus-speed.test.ts` |
|
|
33
|
+
| Python | pytest 9.0.2 | 27 | `tests/test_contextplus_speed.py` |
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## TypeScript Results (5 runs)
|
|
38
|
+
|
|
39
|
+
### Overall Timing
|
|
40
|
+
|
|
41
|
+
| Run | Total Duration | Transform | Collect | Tests | Prepare |
|
|
42
|
+
|-----|---------------|-----------|---------|-------|---------|
|
|
43
|
+
| 1 | 357ms | 67ms | 69ms | 59ms | 62ms |
|
|
44
|
+
| 2 | 347ms | 64ms | 67ms | 59ms | 69ms |
|
|
45
|
+
| 3 | 337ms | 63ms | 65ms | 58ms | 71ms |
|
|
46
|
+
| 4 | 344ms | 63ms | 63ms | 58ms | 57ms |
|
|
47
|
+
| 5 | 347ms | 62ms | 65ms | 58ms | 57ms |
|
|
48
|
+
|
|
49
|
+
**Average**: 346ms total, **58ms test execution** (27 tests)
|
|
50
|
+
|
|
51
|
+
### Per-Test Breakdown (representative run)
|
|
52
|
+
|
|
53
|
+
#### Embedding Engine (no peek, no KV)
|
|
54
|
+
|
|
55
|
+
| Test | Time | What it measures |
|
|
56
|
+
|------|------|-----------------|
|
|
57
|
+
| tokenize: 100 strings | 2ms | Raw NLP tokenization throughput |
|
|
58
|
+
| TfIdfVectorizer: build vocab from 50 docs | 2ms | Corpus vocabulary construction |
|
|
59
|
+
| TfIdfVectorizer: vectorize 50 queries against 50-doc corpus | 3ms | Query vectorization throughput |
|
|
60
|
+
| cosineSimilarity: 1000 vector comparisons | 1ms | Raw similarity computation |
|
|
61
|
+
| embed: 100 documents through global vectorizer | 5ms | End-to-end embedding pipeline |
|
|
62
|
+
|
|
63
|
+
#### `upsert_memory_node` (no peek, no KV)
|
|
64
|
+
|
|
65
|
+
| Test | Time | Scale |
|
|
66
|
+
|------|------|-------|
|
|
67
|
+
| upsert 10 nodes (cold start) | 1ms | 10 nodes |
|
|
68
|
+
| upsert 50 nodes | 2ms | 50 nodes |
|
|
69
|
+
| upsert 100 nodes | 6–7ms | 100 nodes |
|
|
70
|
+
| update existing nodes (50 upserts on 10 labels) | 1–2ms | 50 operations, 10 unique |
|
|
71
|
+
| mixed node types (concept, file, symbol, note) | 1–2ms | 40 nodes, 4 types |
|
|
72
|
+
|
|
73
|
+
#### `create_relation` (no peek, no KV)
|
|
74
|
+
|
|
75
|
+
| Test | Time | Scale |
|
|
76
|
+
|------|------|-------|
|
|
77
|
+
| create 20 edges in a chain | 1–2ms | 21 nodes, 20 edges |
|
|
78
|
+
| create edges with all 6 relation types | 1ms | 7 nodes, 6 edges |
|
|
79
|
+
| create 45 edges in fully connected 10-node cluster | 1ms | 10 nodes, 45 edges |
|
|
80
|
+
|
|
81
|
+
#### `search_memory_graph` (no peek, no KV)
|
|
82
|
+
|
|
83
|
+
| Test | Time | Scale |
|
|
84
|
+
|------|------|-------|
|
|
85
|
+
| search empty graph | 1ms | 0 nodes |
|
|
86
|
+
| search across 10 nodes | 1ms | 10 nodes |
|
|
87
|
+
| search across 50 nodes | 2ms | 50 nodes |
|
|
88
|
+
| search across 100 nodes with depth-2 traversal | 7–8ms | 100 nodes, 99 edges |
|
|
89
|
+
| 10 successive searches on same graph | 1ms | 20 nodes, 10 queries |
|
|
90
|
+
|
|
91
|
+
#### `prune_stale_links` (no peek, no KV)
|
|
92
|
+
|
|
93
|
+
| Test | Time | Scale |
|
|
94
|
+
|------|------|-------|
|
|
95
|
+
| prune on graph with 50 fresh edges | 2ms | 51 nodes, 50 edges |
|
|
96
|
+
| prune on empty graph | 0ms | 0 nodes |
|
|
97
|
+
|
|
98
|
+
#### `add_interlinked_context` (no peek, no KV)
|
|
99
|
+
|
|
100
|
+
| Test | Time | Scale |
|
|
101
|
+
|------|------|-------|
|
|
102
|
+
| bulk add 5 nodes with auto-linking | 1ms | 5 nodes |
|
|
103
|
+
| bulk add 20 nodes with auto-linking | 3–9ms | 20 nodes + O(n²) similarity |
|
|
104
|
+
| bulk add 10 nodes without auto-linking | 0ms | 10 nodes, no edges |
|
|
105
|
+
|
|
106
|
+
#### `retrieve_with_traversal` (no peek, no KV)
|
|
107
|
+
|
|
108
|
+
| Test | Time | Scale |
|
|
109
|
+
|------|------|-------|
|
|
110
|
+
| traverse depth-1 from root with 10 children | 0–1ms | 11 nodes returned |
|
|
111
|
+
| traverse depth-2 from root through 3-level tree | 1ms | 26 nodes returned |
|
|
112
|
+
| traverse with edge filter | 0ms | 2 nodes returned (filtered) |
|
|
113
|
+
| traverse depth-3 chain of 30 nodes | 1ms | 4 nodes returned |
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## Python Results (5 runs)
|
|
118
|
+
|
|
119
|
+
| Run | Duration | Tests |
|
|
120
|
+
|-----|----------|-------|
|
|
121
|
+
| 1 | 0.06s | 27 |
|
|
122
|
+
| 2 | 0.06s | 27 |
|
|
123
|
+
| 3 | 0.06s | 27 |
|
|
124
|
+
| 4 | 0.06s | 27 |
|
|
125
|
+
| 5 | 0.06s | 27 |
|
|
126
|
+
|
|
127
|
+
**Average**: 60ms total (27 tests)
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Key Findings
|
|
132
|
+
|
|
133
|
+
### Context+ is blazing fast on its own
|
|
134
|
+
|
|
135
|
+
| Operation | Scale | Time | Per-op |
|
|
136
|
+
|-----------|-------|------|--------|
|
|
137
|
+
| `upsert_memory_node` | 100 nodes | 6–7ms | ~0.07ms/node |
|
|
138
|
+
| `create_relation` | 45 edges (fully connected) | 1ms | ~0.02ms/edge |
|
|
139
|
+
| `search_memory_graph` | 100 nodes + 99 edges, depth-2 | 7–8ms | Single query |
|
|
140
|
+
| `search_memory_graph` | 20 nodes, 10 queries | 1ms | ~0.1ms/query |
|
|
141
|
+
| `add_interlinked_context` | 20 nodes with auto-link | 3–9ms | Includes O(n²) similarity |
|
|
142
|
+
| `retrieve_with_traversal` | 26-node tree, depth-2 | 1ms | Single traversal |
|
|
143
|
+
| `prune_stale_links` | 51 nodes, 50 edges | 2ms | Full graph scan |
|
|
144
|
+
| `embed` | 100 documents | 5ms | ~0.05ms/doc |
|
|
145
|
+
| `cosineSimilarity` | 1000 comparisons | 1ms | ~0.001ms/comparison |
|
|
146
|
+
|
|
147
|
+
### What this means
|
|
148
|
+
|
|
149
|
+
1. **All 6 Context+ tools complete in single-digit milliseconds**, even at 100-node scale.
|
|
150
|
+
2. **Semantic search across 100 nodes with depth-2 traversal takes only 7–8ms** — fast enough for real-time agent decision-making.
|
|
151
|
+
3. **10 successive searches on a 20-node graph take 1ms total** — repeated queries are essentially free.
|
|
152
|
+
4. **The TF-IDF embedding engine handles 100 documents in 5ms** — no external API latency.
|
|
153
|
+
5. **1000 cosine similarity comparisons take 1ms** — vector math is negligible overhead.
|
|
154
|
+
|
|
155
|
+
Context+ graph operations are already sub-millisecond per operation. When combined with PMLL's `peek()` caching (see [speed-test-results.md](./speed-test-results.md)), even these fast operations can be eliminated entirely for repeated lookups.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## How to Reproduce
|
|
160
|
+
|
|
161
|
+
### TypeScript
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
cd mcp/
|
|
165
|
+
npm install
|
|
166
|
+
npx vitest run __tests__/contextplus-speed.test.ts --reporter=verbose
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
### Python
|
|
170
|
+
|
|
171
|
+
```bash
|
|
172
|
+
cd mcp/
|
|
173
|
+
pip install pytest
|
|
174
|
+
python3 -m pytest tests/test_contextplus_speed.py -v
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
### Benchmark loop (5 runs)
|
|
178
|
+
|
|
179
|
+
```bash
|
|
180
|
+
cd mcp/
|
|
181
|
+
for i in 1 2 3 4 5; do
|
|
182
|
+
echo "--- Run $i ---"
|
|
183
|
+
npx vitest run __tests__/contextplus-speed.test.ts --reporter=verbose 2>&1 | grep -E '(✓|Duration)'
|
|
184
|
+
done
|
|
185
|
+
```
|