@mrxkun/mcfast-mcp 3.3.5 → 3.3.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -11
- package/package.json +1 -1
- package/src/index.js +271 -153
- package/src/strategies/fuzzy-patch.js +415 -125
- package/src/strategies/tree-sitter/languages.js +40 -21
package/README.md
CHANGED
|
@@ -17,34 +17,48 @@ Standard AI agents often struggle with multi-file edits, broken syntax, and "hal
|
|
|
17
17
|
1. **🎯 Surgical Precision**: Uses real Abstract Syntax Trees (AST) to understand code structure. A "Rename" is scope-aware; it won't break unrelated variables.
|
|
18
18
|
2. **🛡️ Bulletproof Safety**: Every edit is automatically validated. If the AI generates a syntax error, mcfast detects it in milliseconds and **rolls back** the change instantly.
|
|
19
19
|
3. **⚡ Blazing Performance**: Powered by WASM, AST operations that take seconds in other tools are completed in **under 1ms** here.
|
|
20
|
-
4. **🌊 Multi-Language Native**: Full support for **Go, Rust, Java, JavaScript, and
|
|
20
|
+
4. **🌊 Multi-Language Native**: Full support for **Go, Rust, Java, JavaScript, TypeScript, Python, C++, C#, PHP, and Ruby**.
|
|
21
21
|
5. **🔒 Local-First Privacy**: Your code structure is analyzed on *your* machine. No proprietary code is sent to the cloud for AST analysis.
|
|
22
22
|
|
|
23
23
|
---
|
|
24
24
|
|
|
25
|
-
## 🚀 Key Features (v3.
|
|
25
|
+
## 🚀 Key Features (v3.3)
|
|
26
26
|
|
|
27
27
|
### 1. **AST-Aware Refactoring**
|
|
28
28
|
mcfast doesn't just "search and replace" text. It parses your code into a Tree-sitter AST to perform:
|
|
29
29
|
- **Scope-Aware Rename**: Rename functions, variables, or classes safely across your entire project.
|
|
30
30
|
- **Smart Symbol Search**: Find true references, ignoring comments and strings.
|
|
31
31
|
|
|
32
|
-
### 2. **
|
|
32
|
+
### 2. **Hybrid Fuzzy Patching** ⚡ NEW in v3.3
|
|
33
|
+
Multi-layered matching strategy with intelligent fallback:
|
|
34
|
+
1. **Exact Line Match** (Hash Map) - O(1) lookup for identical code blocks
|
|
35
|
+
2. **Myers Diff Algorithm** - Shortest Edit Script in O((M+N)D) time
|
|
36
|
+
3. **Levenshtein Distance** - For small single-line differences
|
|
37
|
+
|
|
38
|
+
This hybrid approach significantly improves accuracy and reduces false matches for complex refactoring tasks.
|
|
39
|
+
|
|
40
|
+
### 3. **Context-Aware Search** 🆕 NEW in v3.3
|
|
41
|
+
Automatic junk directory exclusion powered by intelligent pattern matching:
|
|
42
|
+
- Automatically filters `node_modules`, `.git`, `dist`, `build`, `.next`, `coverage`, `__pycache__`, and more
|
|
43
|
+
- No manual configuration required
|
|
44
|
+
- Respects `.gitignore` patterns automatically
|
|
45
|
+
|
|
46
|
+
### 4. **Advanced Fuzzy Patching**
|
|
33
47
|
Tired of "Line number mismatch" errors? mcfast uses a multi-layered matching strategy:
|
|
34
|
-
- **Levenshtein Distance**: Measures text similarity.
|
|
48
|
+
- **Levenshtein Distance**: Measures text similarity with early termination.
|
|
35
49
|
- **Token Analysis**: Matches code based on logic even if whitespace or formatting differs.
|
|
36
50
|
- **Structural Matching**: Validates that the patch "fits" the code structure.
|
|
37
51
|
|
|
38
|
-
###
|
|
52
|
+
### 5. **Auto-Rollback (Auto-Healing)**
|
|
39
53
|
mcfast integrates language-specific linters to ensure your build stays green:
|
|
40
54
|
- **JS/TS**: `node --check`
|
|
41
55
|
- **Go**: `gofmt -e`
|
|
42
56
|
- **Rust**: `rustc --parse-only`
|
|
43
|
-
- **
|
|
57
|
+
- **Python/PHP/Ruby**: Syntax validation.
|
|
44
58
|
*If validation fails, mcfast automatically restores from a hidden backup.*
|
|
45
59
|
|
|
46
|
-
###
|
|
47
|
-
Supports JS, TS, and
|
|
60
|
+
### 6. **Organize Imports**
|
|
61
|
+
Supports JS, TS, Go, Python, and more. Automatically sorts and cleans up your import blocks using high-speed S-expression queries.
|
|
48
62
|
|
|
49
63
|
---
|
|
50
64
|
|
|
@@ -55,6 +69,7 @@ Supports JS, TS, and Go. Automatically sorts and cleans up your import blocks us
|
|
|
55
69
|
| **Simple Rename** | ~5,000ms | **0.5ms** | **10,000x** |
|
|
56
70
|
| **Large File Parse** | ~800ms | **15ms** | **50x** |
|
|
57
71
|
| **Multi-File Update** | ~15,000ms | **2,000ms** | **7x** |
|
|
72
|
+
| **Fuzzy Patch** | ~2,000ms | **5-50ms** | **40-400x** |
|
|
58
73
|
|
|
59
74
|
---
|
|
60
75
|
|
|
@@ -94,7 +109,7 @@ mcfast exposes a unified set of tools to your AI agent:
|
|
|
94
109
|
* **`edit`**: The primary tool. It decides whether to use `ast_refactor`, `fuzzy_patch`, or `search_replace` based on the task complexity.
|
|
95
110
|
* **`search`**: Fast grep-style search with in-memory AST indexing.
|
|
96
111
|
* **`read`**: Smart reader that returns code chunks with line numbers, optimized for token savings.
|
|
97
|
-
* **`list_files`**: High-performance globbing
|
|
112
|
+
* **`list_files`**: High-performance globbing with `.gitignore` support and context-aware filtering.
|
|
98
113
|
* **`reapply`**: If an edit fails validation, the AI can use this to retry with a different strategy.
|
|
99
114
|
|
|
100
115
|
---
|
|
@@ -102,8 +117,7 @@ mcfast exposes a unified set of tools to your AI agent:
|
|
|
102
117
|
## 🔒 Privacy & Licensing
|
|
103
118
|
|
|
104
119
|
- **Code Privacy**: mcfast is designed for corporate security. WASM parsing and fuzzy matching happen **locally**. We do not store or train on your code.
|
|
105
|
-
- **Cloud Support**: Complex multi-file coordination
|
|
120
|
+
- **Cloud Support**: Complex multi-file coordination uses a high-performance edge service (Mercury Coder Cloud) to ensure accuracy, but code is never persisted.
|
|
106
121
|
- **Usage**: Free for personal and commercial use. Proprietary license.
|
|
107
122
|
|
|
108
123
|
Copyright © [mrxkun](https://github.com/mrxkun)
|
|
109
|
-
|
package/package.json
CHANGED
package/src/index.js
CHANGED
|
@@ -305,28 +305,53 @@ server.setRequestHandler(ListToolsRequestSchema, async () => {
|
|
|
305
305
|
};
|
|
306
306
|
});
|
|
307
307
|
|
|
308
|
-
// Helper for recursive file listing
|
|
309
|
-
async function getFiles(dir, depth = 5
|
|
310
|
-
|
|
308
|
+
// Helper for recursive file listing (v4.0 optimized with fast-glob)
|
|
309
|
+
async function getFiles(dir, depth = 5) {
|
|
310
|
+
const patterns = [];
|
|
311
311
|
|
|
312
|
-
|
|
313
|
-
|
|
312
|
+
for (let i = 1; i <= depth; i++) {
|
|
313
|
+
patterns.push('*'.repeat(i));
|
|
314
|
+
}
|
|
314
315
|
|
|
315
|
-
|
|
316
|
-
|
|
316
|
+
const gitignorePath = path.join(dir, '.gitignore');
|
|
317
|
+
let gitignoreContent = null;
|
|
318
|
+
try {
|
|
319
|
+
gitignoreContent = await fs.readFile(gitignorePath, 'utf8');
|
|
320
|
+
} catch {
|
|
321
|
+
// .gitignore not found, continue without it
|
|
322
|
+
}
|
|
317
323
|
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
324
|
+
const fastGlobOptions = {
|
|
325
|
+
cwd: dir,
|
|
326
|
+
onlyFiles: true,
|
|
327
|
+
onlyDirectories: false,
|
|
328
|
+
deep: depth,
|
|
329
|
+
ignore: [
|
|
330
|
+
'node_modules',
|
|
331
|
+
'.git',
|
|
332
|
+
'dist',
|
|
333
|
+
'build',
|
|
334
|
+
'.next',
|
|
335
|
+
'coverage',
|
|
336
|
+
'.cache',
|
|
337
|
+
'__pycache__',
|
|
338
|
+
'.venv',
|
|
339
|
+
'venv',
|
|
340
|
+
'node_modules/**',
|
|
341
|
+
'.git/**'
|
|
342
|
+
],
|
|
343
|
+
absolute: false
|
|
344
|
+
};
|
|
321
345
|
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
346
|
+
if (gitignoreContent) {
|
|
347
|
+
const ignorePatterns = gitignoreContent
|
|
348
|
+
.split('\n')
|
|
349
|
+
.map(l => l.trim())
|
|
350
|
+
.filter(l => l && !l.startsWith('#'));
|
|
351
|
+
fastGlobOptions.ignore.push(...ignorePatterns);
|
|
328
352
|
}
|
|
329
|
-
|
|
353
|
+
|
|
354
|
+
return await fg(patterns, fastGlobOptions);
|
|
330
355
|
}
|
|
331
356
|
|
|
332
357
|
/**
|
|
@@ -877,96 +902,120 @@ async function reportAudit(params) {
|
|
|
877
902
|
}
|
|
878
903
|
}
|
|
879
904
|
|
|
880
|
-
// Unified Search Implementation
|
|
905
|
+
// Unified Search Implementation (v4.0 - Early Termination with Stream)
|
|
881
906
|
async function handleSearchFilesystem({ query, path: searchPath = process.cwd(), include = "**/*", exclude = [], isRegex = false, caseSensitive = false }) {
|
|
882
907
|
const start = Date.now();
|
|
908
|
+
const MAX_RESULTS = 100;
|
|
909
|
+
const results = [];
|
|
910
|
+
let strategy = 'unknown';
|
|
911
|
+
|
|
883
912
|
try {
|
|
884
|
-
|
|
885
|
-
|
|
913
|
+
const { spawn } = await import('child_process');
|
|
914
|
+
const { promisify } = await import('util');
|
|
915
|
+
const sleep = promisify(setTimeout);
|
|
886
916
|
|
|
887
|
-
|
|
917
|
+
const escapedQuery = query.replace(/"/g, '\\"');
|
|
918
|
+
const caseFlag = caseSensitive ? '' : '-i';
|
|
919
|
+
const regexFlag = isRegex ? '-e' : '-F';
|
|
920
|
+
|
|
921
|
+
// Try ripgrep first with streaming and early termination
|
|
888
922
|
try {
|
|
889
|
-
const flags = [
|
|
890
|
-
"--json",
|
|
891
|
-
caseSensitive ? "-s" : "-i",
|
|
892
|
-
isRegex ? "-e" : "-F"
|
|
893
|
-
].join(" ");
|
|
894
|
-
// This is a simplified call; parsing JSON output from rg is best for structured data
|
|
895
|
-
// For now, we'll rely on a simpler text output for the LLM
|
|
896
|
-
const simpleFlags = [
|
|
897
|
-
"-n",
|
|
898
|
-
"--no-heading",
|
|
899
|
-
"--with-filename",
|
|
900
|
-
caseSensitive ? "-s" : "-i",
|
|
901
|
-
isRegex ? "-e" : "-F"
|
|
902
|
-
].join(" ");
|
|
903
|
-
|
|
904
|
-
const command = `rg ${simpleFlags} "${query.replace(/"/g, '\\"')}" ${searchPath}`;
|
|
905
|
-
const { stdout } = await execAsync(command, { maxBuffer: 10 * 1024 * 1024 });
|
|
906
|
-
results = stdout.trim().split('\n').filter(Boolean);
|
|
907
923
|
strategy = 'ripgrep';
|
|
924
|
+
const rgProcess = spawn('rg', [
|
|
925
|
+
'-n', '--no-heading', '--with-filename',
|
|
926
|
+
caseFlag, regexFlag,
|
|
927
|
+
escapedQuery,
|
|
928
|
+
searchPath
|
|
929
|
+
], {
|
|
930
|
+
stdio: ['ignore', 'pipe', 'pipe']
|
|
931
|
+
});
|
|
932
|
+
|
|
933
|
+
const readline = (await import('readline')).createInterface({
|
|
934
|
+
input: rgProcess.stdout,
|
|
935
|
+
crlfDelay: Infinity
|
|
936
|
+
});
|
|
937
|
+
|
|
938
|
+
for await (const line of readline) {
|
|
939
|
+
if (results.length >= MAX_RESULTS) {
|
|
940
|
+
rgProcess.kill();
|
|
941
|
+
break;
|
|
942
|
+
}
|
|
943
|
+
results.push(line);
|
|
944
|
+
}
|
|
945
|
+
|
|
946
|
+
rgProcess.stderr.on('data', () => { });
|
|
947
|
+
await new Promise(resolve => rgProcess.on('close', resolve));
|
|
948
|
+
|
|
949
|
+
if (results.length > 0 || rgProcess.exitCode === 0) {
|
|
950
|
+
return formatSearchResults(query, strategy, results, start, MAX_RESULTS);
|
|
951
|
+
}
|
|
908
952
|
} catch (rgErr) {
|
|
909
|
-
//
|
|
953
|
+
// Try git grep
|
|
910
954
|
try {
|
|
911
|
-
const flags = [
|
|
912
|
-
"-n",
|
|
913
|
-
"-I",
|
|
914
|
-
caseSensitive ? "" : "-i",
|
|
915
|
-
isRegex ? "-E" : "-F"
|
|
916
|
-
].filter(Boolean).join(" ");
|
|
917
|
-
const command = `git grep ${flags} "${query.replace(/"/g, '\\"')}" ${searchPath}`;
|
|
918
|
-
const { stdout } = await execAsync(command, { cwd: searchPath, maxBuffer: 10 * 1024 * 1024 });
|
|
919
|
-
results = stdout.trim().split('\n').filter(Boolean);
|
|
920
955
|
strategy = 'git_grep';
|
|
921
|
-
|
|
922
|
-
|
|
923
|
-
|
|
924
|
-
|
|
925
|
-
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
]
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
|
|
934
|
-
}
|
|
935
|
-
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
956
|
+
const gitProcess = spawn('git', [
|
|
957
|
+
'grep', '-n', '-I',
|
|
958
|
+
caseFlag ? '' : '-i',
|
|
959
|
+
regexFlag ? '-E' : '-F',
|
|
960
|
+
escapedQuery
|
|
961
|
+
], {
|
|
962
|
+
cwd: searchPath,
|
|
963
|
+
stdio: ['ignore', 'pipe', 'pipe']
|
|
964
|
+
});
|
|
965
|
+
|
|
966
|
+
const readline = (await import('readline')).createInterface({
|
|
967
|
+
input: gitProcess.stdout,
|
|
968
|
+
crlfDelay: Infinity
|
|
969
|
+
});
|
|
970
|
+
|
|
971
|
+
for await (const line of readline) {
|
|
972
|
+
if (results.length >= MAX_RESULTS) {
|
|
973
|
+
gitProcess.kill();
|
|
974
|
+
break;
|
|
975
|
+
}
|
|
976
|
+
results.push(line);
|
|
939
977
|
}
|
|
940
|
-
}
|
|
941
|
-
}
|
|
942
978
|
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
output += "No matches found.";
|
|
946
|
-
} else {
|
|
947
|
-
const limitedResults = results.slice(0, 100);
|
|
948
|
-
output += limitedResults.join('\n');
|
|
949
|
-
if (results.length > 100) output += `\n... and ${results.length - 100} more matches.`;
|
|
950
|
-
}
|
|
979
|
+
gitProcess.stderr.on('data', () => { });
|
|
980
|
+
await new Promise(resolve => gitProcess.on('close', resolve));
|
|
951
981
|
|
|
952
|
-
|
|
953
|
-
|
|
954
|
-
|
|
955
|
-
|
|
982
|
+
return formatSearchResults(query, strategy, results, start, MAX_RESULTS);
|
|
983
|
+
} catch (gitErr) {
|
|
984
|
+
// Fallback to native grep
|
|
985
|
+
strategy = 'native_grep';
|
|
986
|
+
const grepProcess = spawn('grep', [
|
|
987
|
+
'-r', '-n', '-I',
|
|
988
|
+
caseFlag ? '' : '-i',
|
|
989
|
+
regexFlag ? '-E' : '-F',
|
|
990
|
+
'--exclude-dir=node_modules', '--exclude-dir=.git',
|
|
991
|
+
'--exclude-dir=.next', '--exclude-dir=dist', '--exclude-dir=build',
|
|
992
|
+
escapedQuery,
|
|
993
|
+
searchPath
|
|
994
|
+
], {
|
|
995
|
+
stdio: ['ignore', 'pipe', 'pipe']
|
|
996
|
+
});
|
|
956
997
|
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
960
|
-
|
|
961
|
-
status: 'success',
|
|
962
|
-
latency_ms: Date.now() - start,
|
|
963
|
-
files_count: 0,
|
|
964
|
-
input_tokens: Math.ceil(query.length / 4), // Minimal input tokens for filesystem search
|
|
965
|
-
output_tokens: estimatedOutputTokens,
|
|
966
|
-
result_summary: JSON.stringify(results.slice(0, 100))
|
|
967
|
-
});
|
|
998
|
+
const readline = (await import('readline')).createInterface({
|
|
999
|
+
input: grepProcess.stdout,
|
|
1000
|
+
crlfDelay: Infinity
|
|
1001
|
+
});
|
|
968
1002
|
|
|
969
|
-
|
|
1003
|
+
for await (const line of readline) {
|
|
1004
|
+
if (results.length >= MAX_RESULTS) {
|
|
1005
|
+
grepProcess.kill();
|
|
1006
|
+
break;
|
|
1007
|
+
}
|
|
1008
|
+
results.push(line);
|
|
1009
|
+
}
|
|
1010
|
+
|
|
1011
|
+
grepProcess.stderr.on('data', () => { });
|
|
1012
|
+
await new Promise(resolve => grepProcess.on('close', resolve));
|
|
1013
|
+
|
|
1014
|
+
return formatSearchResults(query, strategy, results, start, MAX_RESULTS);
|
|
1015
|
+
}
|
|
1016
|
+
}
|
|
1017
|
+
|
|
1018
|
+
return formatSearchResults(query, strategy, results, start, MAX_RESULTS);
|
|
970
1019
|
|
|
971
1020
|
} catch (error) {
|
|
972
1021
|
reportAudit({
|
|
@@ -985,6 +1034,35 @@ async function handleSearchFilesystem({ query, path: searchPath = process.cwd(),
|
|
|
985
1034
|
}
|
|
986
1035
|
}
|
|
987
1036
|
|
|
1037
|
+
function formatSearchResults(query, strategy, results, start, maxResults) {
|
|
1038
|
+
let output = `⚡ search_filesystem (${strategy}) found ${results.length} results for "${query}"\n\n`;
|
|
1039
|
+
|
|
1040
|
+
if (results.length === 0) {
|
|
1041
|
+
output += "No matches found.";
|
|
1042
|
+
} else {
|
|
1043
|
+
output += results.join('\n');
|
|
1044
|
+
if (results.length >= maxResults) {
|
|
1045
|
+
output += `\n... and more matches (early termination at ${maxResults}).`;
|
|
1046
|
+
}
|
|
1047
|
+
}
|
|
1048
|
+
|
|
1049
|
+
const estimatedOutputTokens = Math.ceil(output.length / 4);
|
|
1050
|
+
|
|
1051
|
+
reportAudit({
|
|
1052
|
+
tool: 'search_filesystem',
|
|
1053
|
+
instruction: query,
|
|
1054
|
+
strategy,
|
|
1055
|
+
status: 'success',
|
|
1056
|
+
latency_ms: Date.now() - start,
|
|
1057
|
+
files_count: 0,
|
|
1058
|
+
input_tokens: Math.ceil(query.length / 4),
|
|
1059
|
+
output_tokens: estimatedOutputTokens,
|
|
1060
|
+
result_summary: JSON.stringify(results.slice(0, maxResults))
|
|
1061
|
+
});
|
|
1062
|
+
|
|
1063
|
+
return { content: [{ type: "text", text: output }] };
|
|
1064
|
+
}
|
|
1065
|
+
|
|
988
1066
|
// Native high-performance search
|
|
989
1067
|
async function handleWarpgrep({ query, include = ".", isRegex = false, caseSensitive = false }) {
|
|
990
1068
|
const start = Date.now();
|
|
@@ -1073,8 +1151,23 @@ async function handleSearchCode({ query, files, regex = false, caseSensitive = f
|
|
|
1073
1151
|
try {
|
|
1074
1152
|
const results = [];
|
|
1075
1153
|
let totalInputChars = 0;
|
|
1154
|
+
let lastYield = Date.now();
|
|
1155
|
+
const YIELD_INTERVAL_MS = 10;
|
|
1156
|
+
const YIELD_LINES = 1000;
|
|
1157
|
+
|
|
1158
|
+
const shouldYield = () => {
|
|
1159
|
+
const now = Date.now();
|
|
1160
|
+
if (now - lastYield > YIELD_INTERVAL_MS) {
|
|
1161
|
+
lastYield = now;
|
|
1162
|
+
return true;
|
|
1163
|
+
}
|
|
1164
|
+
return false;
|
|
1165
|
+
};
|
|
1166
|
+
|
|
1167
|
+
const yieldEventLoop = async () => {
|
|
1168
|
+
return new Promise(resolve => setImmediate(resolve));
|
|
1169
|
+
};
|
|
1076
1170
|
|
|
1077
|
-
// If regex mode, use original regex logic
|
|
1078
1171
|
if (regex) {
|
|
1079
1172
|
const flags = caseSensitive ? 'm' : 'im';
|
|
1080
1173
|
const pattern = new RegExp(query, flags);
|
|
@@ -1084,47 +1177,44 @@ async function handleSearchCode({ query, files, regex = false, caseSensitive = f
|
|
|
1084
1177
|
totalInputChars += content.length;
|
|
1085
1178
|
|
|
1086
1179
|
const lines = content.split('\n');
|
|
1087
|
-
lines.
|
|
1088
|
-
if (
|
|
1180
|
+
for (let i = 0; i < lines.length; i++) {
|
|
1181
|
+
if (shouldYield()) await yieldEventLoop();
|
|
1182
|
+
|
|
1183
|
+
if (pattern.test(lines[i])) {
|
|
1089
1184
|
pattern.lastIndex = 0;
|
|
1090
|
-
const startLine = Math.max(0,
|
|
1091
|
-
const endLine = Math.min(lines.length - 1,
|
|
1185
|
+
const startLine = Math.max(0, i - contextLines);
|
|
1186
|
+
const endLine = Math.min(lines.length - 1, i + contextLines);
|
|
1092
1187
|
|
|
1093
1188
|
const contextSnippet = lines
|
|
1094
1189
|
.slice(startLine, endLine + 1)
|
|
1095
|
-
.map((l,
|
|
1096
|
-
lineNumber: startLine +
|
|
1190
|
+
.map((l, idx) => ({
|
|
1191
|
+
lineNumber: startLine + idx + 1,
|
|
1097
1192
|
content: l,
|
|
1098
|
-
isMatch: startLine +
|
|
1193
|
+
isMatch: startLine + idx === i
|
|
1099
1194
|
}));
|
|
1100
1195
|
|
|
1101
1196
|
results.push({
|
|
1102
1197
|
file: filePath,
|
|
1103
|
-
lineNumber:
|
|
1104
|
-
matchedLine:
|
|
1198
|
+
lineNumber: i + 1,
|
|
1199
|
+
matchedLine: lines[i].trim(),
|
|
1105
1200
|
context: contextSnippet,
|
|
1106
1201
|
matchType: 'regex'
|
|
1107
1202
|
});
|
|
1108
1203
|
}
|
|
1109
|
-
}
|
|
1204
|
+
}
|
|
1110
1205
|
}
|
|
1111
1206
|
} else {
|
|
1112
|
-
// Semantic search with stop words filtering
|
|
1113
1207
|
const queryLower = query.toLowerCase();
|
|
1114
|
-
|
|
1115
|
-
// Common English stop words to filter out
|
|
1116
1208
|
const stopWords = new Set([
|
|
1117
1209
|
'a', 'an', 'and', 'are', 'as', 'at', 'be', 'by', 'for', 'from', 'has', 'he',
|
|
1118
1210
|
'in', 'is', 'it', 'its', 'of', 'on', 'that', 'the', 'to', 'was', 'will', 'with',
|
|
1119
1211
|
'how', 'what', 'when', 'where', 'who', 'why', 'does', 'do', 'this', 'these', 'those'
|
|
1120
1212
|
]);
|
|
1121
1213
|
|
|
1122
|
-
// Extract significant words (3+ chars, not stop words)
|
|
1123
1214
|
const words = queryLower
|
|
1124
1215
|
.split(/\W+/)
|
|
1125
1216
|
.filter(w => w.length >= 3 && !stopWords.has(w));
|
|
1126
1217
|
|
|
1127
|
-
// If no significant words, fall back to whole query
|
|
1128
1218
|
const searchTerms = words.length > 0 ? words : [queryLower];
|
|
1129
1219
|
|
|
1130
1220
|
for (const [filePath, content] of Object.entries(files)) {
|
|
@@ -1132,45 +1222,40 @@ async function handleSearchCode({ query, files, regex = false, caseSensitive = f
|
|
|
1132
1222
|
totalInputChars += content.length;
|
|
1133
1223
|
|
|
1134
1224
|
const lines = content.split('\n');
|
|
1135
|
-
lines.
|
|
1225
|
+
for (let i = 0; i < lines.length; i++) {
|
|
1226
|
+
if (shouldYield()) await yieldEventLoop();
|
|
1227
|
+
|
|
1136
1228
|
const lineLower = caseSensitive ? line : line.toLowerCase();
|
|
1137
1229
|
const searchQuery = caseSensitive ? query : queryLower;
|
|
1138
|
-
|
|
1139
|
-
// Check 1: Exact phrase match (highest priority)
|
|
1140
1230
|
const exactMatch = lineLower.includes(searchQuery);
|
|
1141
|
-
|
|
1142
|
-
// Check 2: All significant words present (semantic match)
|
|
1143
1231
|
const allWordsMatch = searchTerms.every(term => lineLower.includes(term));
|
|
1144
|
-
|
|
1145
|
-
// Check 3: At least half of significant words present (fuzzy match)
|
|
1146
1232
|
const matchCount = searchTerms.filter(term => lineLower.includes(term)).length;
|
|
1147
1233
|
const fuzzyMatch = matchCount >= Math.ceil(searchTerms.length / 2);
|
|
1148
1234
|
|
|
1149
1235
|
if (exactMatch || allWordsMatch || (searchTerms.length > 1 && fuzzyMatch)) {
|
|
1150
|
-
const startLine = Math.max(0,
|
|
1151
|
-
const endLine = Math.min(lines.length - 1,
|
|
1236
|
+
const startLine = Math.max(0, i - contextLines);
|
|
1237
|
+
const endLine = Math.min(lines.length - 1, i + contextLines);
|
|
1152
1238
|
|
|
1153
1239
|
const contextSnippet = lines
|
|
1154
1240
|
.slice(startLine, endLine + 1)
|
|
1155
|
-
.map((l,
|
|
1156
|
-
lineNumber: startLine +
|
|
1241
|
+
.map((l, idx) => ({
|
|
1242
|
+
lineNumber: startLine + idx + 1,
|
|
1157
1243
|
content: l,
|
|
1158
|
-
isMatch: startLine +
|
|
1244
|
+
isMatch: startLine + idx === i
|
|
1159
1245
|
}));
|
|
1160
1246
|
|
|
1161
1247
|
results.push({
|
|
1162
1248
|
file: filePath,
|
|
1163
|
-
lineNumber:
|
|
1249
|
+
lineNumber: i + 1,
|
|
1164
1250
|
matchedLine: line.trim(),
|
|
1165
1251
|
context: contextSnippet,
|
|
1166
1252
|
matchType: exactMatch ? 'exact' : allWordsMatch ? 'semantic' : 'fuzzy',
|
|
1167
1253
|
matchScore: exactMatch ? 100 : allWordsMatch ? 80 : matchCount * 10
|
|
1168
1254
|
});
|
|
1169
1255
|
}
|
|
1170
|
-
}
|
|
1256
|
+
}
|
|
1171
1257
|
}
|
|
1172
1258
|
|
|
1173
|
-
// Sort results: by score (highest first), then by file
|
|
1174
1259
|
results.sort((a, b) => {
|
|
1175
1260
|
if (a.matchScore !== b.matchScore) {
|
|
1176
1261
|
return b.matchScore - a.matchScore;
|
|
@@ -1226,18 +1311,16 @@ async function handleListFiles({ path: dirPath = process.cwd(), depth = 5 }) {
|
|
|
1226
1311
|
const start = Date.now();
|
|
1227
1312
|
try {
|
|
1228
1313
|
const files = await getFiles(dirPath, depth);
|
|
1229
|
-
// Return relative paths to save tokens
|
|
1230
|
-
const relativeFiles = files.map(f => path.relative(dirPath, f));
|
|
1231
1314
|
|
|
1232
|
-
const output = `📁 Files in ${dirPath}:\n\n${
|
|
1315
|
+
const output = `📁 Files in ${dirPath}:\n\n${files.join('\n')}`;
|
|
1233
1316
|
|
|
1234
1317
|
reportAudit({
|
|
1235
1318
|
tool: 'list_files_fast',
|
|
1236
1319
|
instruction: dirPath,
|
|
1237
1320
|
status: 'success',
|
|
1238
1321
|
latency_ms: Date.now() - start,
|
|
1239
|
-
files_count:
|
|
1240
|
-
result_summary: JSON.stringify(
|
|
1322
|
+
files_count: files.length,
|
|
1323
|
+
result_summary: JSON.stringify(files.slice(0, 500)),
|
|
1241
1324
|
input_tokens: Math.ceil(dirPath.length / 4),
|
|
1242
1325
|
output_tokens: Math.ceil(output.length / 4)
|
|
1243
1326
|
});
|
|
@@ -1302,41 +1385,76 @@ async function handleEditFile({ path: filePath, content, instruction = "" }) {
|
|
|
1302
1385
|
async function handleReadFile({ path: filePath, start_line, end_line }) {
|
|
1303
1386
|
const start = Date.now();
|
|
1304
1387
|
try {
|
|
1305
|
-
// Resolve absolute path
|
|
1306
1388
|
const absolutePath = path.resolve(filePath);
|
|
1307
|
-
|
|
1308
|
-
// Check if file exists and is a file
|
|
1309
1389
|
const stats = await fs.stat(absolutePath);
|
|
1390
|
+
|
|
1310
1391
|
if (!stats.isFile()) {
|
|
1311
1392
|
throw new Error(`Path is not a file: ${absolutePath}`);
|
|
1312
1393
|
}
|
|
1313
1394
|
|
|
1314
|
-
//
|
|
1315
|
-
const
|
|
1395
|
+
const STREAM_THRESHOLD = 1024 * 1024; // 1MB - files larger than this use streaming
|
|
1396
|
+
const LINE_RANGE_THRESHOLD = 50000; // If requesting specific lines and file is large, stream
|
|
1316
1397
|
|
|
1317
|
-
|
|
1318
|
-
|
|
1398
|
+
let startLine = start_line ? parseInt(start_line) : 1;
|
|
1399
|
+
let endLine = end_line ? parseInt(end_line) : -1;
|
|
1400
|
+
let outputContent;
|
|
1401
|
+
let totalLines;
|
|
1319
1402
|
|
|
1320
|
-
|
|
1321
|
-
|
|
1403
|
+
if ((stats.size > STREAM_THRESHOLD && (start_line || end_line)) || stats.size > 10 * 1024 * 1024) {
|
|
1404
|
+
const { Readable } = await import('stream');
|
|
1405
|
+
const { createInterface } = await import('readline');
|
|
1322
1406
|
|
|
1323
|
-
|
|
1324
|
-
|
|
1407
|
+
let currentLine = 0;
|
|
1408
|
+
const lines = [];
|
|
1325
1409
|
|
|
1326
|
-
|
|
1327
|
-
|
|
1328
|
-
|
|
1329
|
-
|
|
1330
|
-
|
|
1331
|
-
|
|
1410
|
+
const stream = (await import('fs')).createReadStream(absolutePath, { encoding: 'utf8' });
|
|
1411
|
+
const rl = createInterface({ input: stream, crlfDelay: Infinity });
|
|
1412
|
+
|
|
1413
|
+
for await (const line of rl) {
|
|
1414
|
+
currentLine++;
|
|
1415
|
+
if (startLine && endLine) {
|
|
1416
|
+
if (currentLine >= startLine && currentLine <= endLine) {
|
|
1417
|
+
lines.push(line);
|
|
1418
|
+
}
|
|
1419
|
+
if (currentLine >= endLine) break;
|
|
1420
|
+
} else if (startLine && currentLine >= startLine) {
|
|
1421
|
+
lines.push(line);
|
|
1422
|
+
} else if (lines.length < 2000) {
|
|
1423
|
+
lines.push(line);
|
|
1424
|
+
} else {
|
|
1425
|
+
break;
|
|
1426
|
+
}
|
|
1427
|
+
}
|
|
1332
1428
|
|
|
1333
|
-
|
|
1334
|
-
|
|
1335
|
-
|
|
1336
|
-
|
|
1337
|
-
|
|
1338
|
-
|
|
1339
|
-
|
|
1429
|
+
stream.destroy();
|
|
1430
|
+
outputContent = lines.join('\n');
|
|
1431
|
+
totalLines = currentLine;
|
|
1432
|
+
|
|
1433
|
+
if (startLine && endLine) {
|
|
1434
|
+
lineRangeInfo = `(Lines ${startLine}-${endLine} of ${totalLines})`;
|
|
1435
|
+
} else if (startLine) {
|
|
1436
|
+
lineRangeInfo = `(Lines ${startLine}-${currentLine} of ${totalLines})`;
|
|
1437
|
+
} else {
|
|
1438
|
+
lineRangeInfo = `(Lines 1-${lines.length} of ${totalLines} - truncated)`;
|
|
1439
|
+
}
|
|
1440
|
+
} else {
|
|
1441
|
+
const content = await fs.readFile(absolutePath, 'utf8');
|
|
1442
|
+
const lines = content.split('\n');
|
|
1443
|
+
totalLines = lines.length;
|
|
1444
|
+
|
|
1445
|
+
if (startLine < 1) startLine = 1;
|
|
1446
|
+
if (endLine < 1 || endLine > totalLines) endLine = totalLines;
|
|
1447
|
+
if (startLine > endLine) {
|
|
1448
|
+
throw new Error(`Invalid line range: start_line (${startLine}) > end_line (${endLine})`);
|
|
1449
|
+
}
|
|
1450
|
+
|
|
1451
|
+
if (start_line || end_line) {
|
|
1452
|
+
outputContent = lines.slice(startLine - 1, endLine).join('\n');
|
|
1453
|
+
lineRangeInfo = `(Lines ${startLine}-${endLine} of ${totalLines})`;
|
|
1454
|
+
} else {
|
|
1455
|
+
outputContent = content;
|
|
1456
|
+
lineRangeInfo = `(Total ${totalLines} lines)`;
|
|
1457
|
+
}
|
|
1340
1458
|
}
|
|
1341
1459
|
|
|
1342
1460
|
const output = `📄 File: ${filePath} ${lineRangeInfo}\n----------------------------------------\n${outputContent}`;
|