@crewx/search 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +60 -58
- package/dist/cli.d.ts +1 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +54 -55
- package/dist/cli.js.map +1 -1
- package/dist/src/engine.d.ts +4 -1
- package/dist/src/engine.d.ts.map +1 -1
- package/dist/src/engine.js +38 -17
- package/dist/src/engine.js.map +1 -1
- package/package.json +10 -2
package/SKILL.md
CHANGED
|
@@ -1,19 +1,21 @@
|
|
|
1
1
|
---
|
|
2
|
-
name: search
|
|
3
|
-
description: BM25
|
|
4
|
-
|
|
2
|
+
name: crewx/search
|
|
3
|
+
description: BM25-based document/code search skill. Smarter than grep -- compound keywords, relevance scoring, semantic matching.
|
|
4
|
+
metadata:
|
|
5
|
+
version: 0.1.0
|
|
6
|
+
built-in: true
|
|
5
7
|
---
|
|
6
8
|
|
|
7
9
|
# Search BM25 Skill
|
|
8
10
|
|
|
9
|
-
grep
|
|
11
|
+
Smarter than grep -- BM25-based search that understands compound keywords and sorts by relevance score.
|
|
10
12
|
|
|
11
13
|
## When to Use This Skill
|
|
12
14
|
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
- grep
|
|
16
|
-
-
|
|
15
|
+
- Finding specific topics in documents
|
|
16
|
+
- Finding related files in a codebase
|
|
17
|
+
- When grep can't find what you need
|
|
18
|
+
- Searching with multiple keyword combinations
|
|
17
19
|
|
|
18
20
|
## Setup
|
|
19
21
|
|
|
@@ -23,109 +25,109 @@ npm install
|
|
|
23
25
|
```
|
|
24
26
|
|
|
25
27
|
**Dependencies:**
|
|
26
|
-
- wink-bm25-text-search (BM25
|
|
27
|
-
- wink-nlp-utils (
|
|
28
|
-
- gray-matter (
|
|
29
|
-
- glob (
|
|
28
|
+
- wink-bm25-text-search (BM25 search engine)
|
|
29
|
+
- wink-nlp-utils (tokenizer, stemming)
|
|
30
|
+
- gray-matter (markdown frontmatter parsing)
|
|
31
|
+
- glob (file pattern matching)
|
|
30
32
|
|
|
31
33
|
## Quick Start
|
|
32
34
|
|
|
33
35
|
```bash
|
|
34
|
-
#
|
|
35
|
-
npx search "
|
|
36
|
+
# Search markdown files (default)
|
|
37
|
+
npx search "launcher architecture" .
|
|
36
38
|
|
|
37
|
-
#
|
|
39
|
+
# Specific directory
|
|
38
40
|
npx search "API endpoint" ./docs
|
|
39
41
|
|
|
40
|
-
# JS/TS
|
|
42
|
+
# JS/TS file search
|
|
41
43
|
npx search "authentication" ./src --glob="**/*.{js,ts}"
|
|
42
44
|
|
|
43
|
-
#
|
|
45
|
+
# Limit result count
|
|
44
46
|
npx search "error handling" . --limit=10
|
|
45
47
|
|
|
46
|
-
# JSON
|
|
48
|
+
# JSON output
|
|
47
49
|
npx search "database" . --json
|
|
48
50
|
```
|
|
49
51
|
|
|
50
52
|
## Options
|
|
51
53
|
|
|
52
|
-
|
|
|
53
|
-
|
|
54
|
-
| `--glob=<pattern>` |
|
|
55
|
-
| `--limit=<n>` |
|
|
56
|
-
| `--json` | JSON
|
|
54
|
+
| Option | Description | Default |
|
|
55
|
+
|--------|-------------|---------|
|
|
56
|
+
| `--glob=<pattern>` | File pattern | `**/*.md` |
|
|
57
|
+
| `--limit=<n>` | Limit result count | 20 |
|
|
58
|
+
| `--json` | JSON format output | false |
|
|
57
59
|
|
|
58
60
|
## grep vs search
|
|
59
61
|
|
|
60
62
|
| | grep | search |
|
|
61
63
|
|--|------|--------|
|
|
62
|
-
|
|
|
63
|
-
|
|
|
64
|
-
|
|
|
65
|
-
|
|
|
66
|
-
|
|
|
64
|
+
| Exact string | Yes | Yes |
|
|
65
|
+
| Compound keywords | No | Yes |
|
|
66
|
+
| Relevance scoring | No | Yes |
|
|
67
|
+
| Semantic similarity | No | Partial (BM25) |
|
|
68
|
+
| Korean search | Yes | Yes |
|
|
67
69
|
|
|
68
70
|
## Examples
|
|
69
71
|
|
|
70
|
-
###
|
|
72
|
+
### Document Search
|
|
71
73
|
```bash
|
|
72
|
-
#
|
|
74
|
+
# Find documents related to CrewX launcher
|
|
73
75
|
npx search "CrewX launcher electron architecture" ./docs
|
|
74
76
|
|
|
75
|
-
#
|
|
76
|
-
# [17.5
|
|
77
|
-
# [12.3
|
|
77
|
+
# Result:
|
|
78
|
+
# [17.5] docs/launcher/architecture.md
|
|
79
|
+
# [12.3] docs/decisions/2025-12-24-launcher.md
|
|
78
80
|
```
|
|
79
81
|
|
|
80
|
-
###
|
|
82
|
+
### Code Search
|
|
81
83
|
```bash
|
|
82
|
-
#
|
|
84
|
+
# Find authentication-related code
|
|
83
85
|
npx search "authentication JWT token" ./src --glob="**/*.ts"
|
|
84
86
|
|
|
85
|
-
#
|
|
86
|
-
# [15.2
|
|
87
|
-
# [10.8
|
|
87
|
+
# Result:
|
|
88
|
+
# [15.2] src/auth/jwt.service.ts
|
|
89
|
+
# [10.8] src/middleware/auth.middleware.ts
|
|
88
90
|
```
|
|
89
91
|
|
|
90
|
-
###
|
|
92
|
+
### Multi-language Search
|
|
91
93
|
```bash
|
|
92
|
-
#
|
|
93
|
-
npx search "
|
|
94
|
+
# Find error handling content
|
|
95
|
+
npx search "error handling exception" . --glob="**/*.md"
|
|
94
96
|
|
|
95
|
-
#
|
|
96
|
-
# [3
|
|
97
|
-
# [2
|
|
97
|
+
# Result:
|
|
98
|
+
# [3] docs/troubleshooting.md
|
|
99
|
+
# [2] docs/error-handling.md
|
|
98
100
|
```
|
|
99
101
|
|
|
100
102
|
## How It Works
|
|
101
103
|
|
|
102
|
-
1.
|
|
103
|
-
2.
|
|
104
|
-
3.
|
|
105
|
-
4.
|
|
104
|
+
1. **File collection**: Find target files using glob pattern
|
|
105
|
+
2. **Indexing**: Register each file's content in the BM25 engine
|
|
106
|
+
3. **Search**: Tokenize the query and calculate scores
|
|
107
|
+
4. **Sorting**: Output in descending order of relevance score
|
|
106
108
|
|
|
107
|
-
###
|
|
108
|
-
- **TF (Term Frequency)**:
|
|
109
|
-
- **IDF (Inverse Document Frequency)**:
|
|
110
|
-
-
|
|
109
|
+
### Score Calculation (BM25)
|
|
110
|
+
- **TF (Term Frequency)**: Word occurrence frequency
|
|
111
|
+
- **IDF (Inverse Document Frequency)**: Rarer words get higher scores
|
|
112
|
+
- **Document length normalization**: Matches in shorter documents are more significant
|
|
111
113
|
|
|
112
114
|
## Tips
|
|
113
115
|
|
|
114
|
-
1.
|
|
116
|
+
1. **Use multiple keywords**: BM25 also scores partial matches
|
|
115
117
|
```bash
|
|
116
|
-
#
|
|
118
|
+
# Good: multiple related keywords
|
|
117
119
|
search "API authentication JWT refresh token"
|
|
118
120
|
|
|
119
|
-
#
|
|
121
|
+
# Bad: single word only
|
|
120
122
|
search "API"
|
|
121
123
|
```
|
|
122
124
|
|
|
123
|
-
2.
|
|
125
|
+
2. **Mixed language search supported**
|
|
124
126
|
```bash
|
|
125
|
-
search "
|
|
127
|
+
search "launcher Electron Tauri architecture"
|
|
126
128
|
```
|
|
127
129
|
|
|
128
|
-
3.
|
|
130
|
+
3. **Adjust limit if too many results**
|
|
129
131
|
```bash
|
|
130
132
|
search "config" . --limit=5
|
|
131
133
|
```
|
package/dist/cli.d.ts
CHANGED
package/dist/cli.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../cli.ts"],"names":[],"mappings":""}
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../cli.ts"],"names":[],"mappings":";AAgDA,wBAAsB,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC,CAwE9D"}
|
package/dist/cli.js
CHANGED
|
@@ -34,6 +34,7 @@ var __importStar = (this && this.__importStar) || (function () {
|
|
|
34
34
|
};
|
|
35
35
|
})();
|
|
36
36
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
37
|
+
exports.main = main;
|
|
37
38
|
const fs = __importStar(require("fs"));
|
|
38
39
|
const path = __importStar(require("path"));
|
|
39
40
|
const engine_1 = require("./src/engine");
|
|
@@ -47,19 +48,13 @@ try {
|
|
|
47
48
|
PKG_VERSION = String(_pkg.version);
|
|
48
49
|
}
|
|
49
50
|
catch { }
|
|
50
|
-
function loadTracer() {
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
try {
|
|
57
|
-
if (fs.existsSync(p))
|
|
58
|
-
return require(p);
|
|
59
|
-
}
|
|
60
|
-
catch { }
|
|
51
|
+
async function loadTracer() {
|
|
52
|
+
try {
|
|
53
|
+
return await Promise.resolve().then(() => __importStar(require('@crewx/shared/skill-tracer')));
|
|
54
|
+
}
|
|
55
|
+
catch {
|
|
56
|
+
return null;
|
|
61
57
|
}
|
|
62
|
-
return null;
|
|
63
58
|
}
|
|
64
59
|
function showDetailedUsage() {
|
|
65
60
|
const skillMd = path.join(__dirname, '..', 'SKILL.md');
|
|
@@ -70,45 +65,48 @@ function showDetailedUsage() {
|
|
|
70
65
|
console.log('No SKILL.md found. Use --help for usage.');
|
|
71
66
|
}
|
|
72
67
|
}
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
if (args[0] === 'usage') {
|
|
79
|
-
showDetailedUsage();
|
|
80
|
-
process.exit(0);
|
|
81
|
-
}
|
|
82
|
-
if (args[0] === 'help' || args[0] === '--help' || args[0] === '-h') {
|
|
83
|
-
args.length = 0;
|
|
84
|
-
}
|
|
85
|
-
const options = {
|
|
86
|
-
query: '',
|
|
87
|
-
searchPath: '.',
|
|
88
|
-
glob: '**/*.md',
|
|
89
|
-
limit: 20,
|
|
90
|
-
json: false,
|
|
91
|
-
};
|
|
92
|
-
for (let i = 0; i < args.length; i++) {
|
|
93
|
-
const arg = args[i];
|
|
94
|
-
if (arg.startsWith('--glob=')) {
|
|
95
|
-
options.glob = arg.split('=').slice(1).join('=');
|
|
96
|
-
}
|
|
97
|
-
else if (arg.startsWith('--limit=')) {
|
|
98
|
-
options.limit = parseInt(arg.split('=')[1], 10);
|
|
68
|
+
async function main(inputArgs) {
|
|
69
|
+
const args = inputArgs ?? process.argv.slice(2);
|
|
70
|
+
if (args[0] === '--version') {
|
|
71
|
+
console.log(PKG_VERSION);
|
|
72
|
+
return;
|
|
99
73
|
}
|
|
100
|
-
|
|
101
|
-
|
|
74
|
+
if (args[0] === 'usage') {
|
|
75
|
+
showDetailedUsage();
|
|
76
|
+
return;
|
|
102
77
|
}
|
|
103
|
-
|
|
104
|
-
|
|
78
|
+
if (args[0] === 'help' || args[0] === '--help' || args[0] === '-h') {
|
|
79
|
+
args.length = 0;
|
|
105
80
|
}
|
|
106
|
-
|
|
107
|
-
|
|
81
|
+
const options = {
|
|
82
|
+
query: '',
|
|
83
|
+
searchPath: '.',
|
|
84
|
+
glob: '**/*.md',
|
|
85
|
+
limit: 20,
|
|
86
|
+
json: false,
|
|
87
|
+
};
|
|
88
|
+
for (let i = 0; i < args.length; i++) {
|
|
89
|
+
const arg = args[i];
|
|
90
|
+
if (!arg)
|
|
91
|
+
continue;
|
|
92
|
+
if (arg.startsWith('--glob=')) {
|
|
93
|
+
options.glob = arg.split('=').slice(1).join('=');
|
|
94
|
+
}
|
|
95
|
+
else if (arg.startsWith('--limit=')) {
|
|
96
|
+
options.limit = parseInt(arg.split('=')[1] ?? '20', 10);
|
|
97
|
+
}
|
|
98
|
+
else if (arg === '--json') {
|
|
99
|
+
options.json = true;
|
|
100
|
+
}
|
|
101
|
+
else if (!options.query) {
|
|
102
|
+
options.query = arg;
|
|
103
|
+
}
|
|
104
|
+
else if (options.searchPath === '.') {
|
|
105
|
+
options.searchPath = arg;
|
|
106
|
+
}
|
|
108
107
|
}
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
console.log(`
|
|
108
|
+
if (!options.query) {
|
|
109
|
+
console.log(`
|
|
112
110
|
@crewx/search@${PKG_VERSION} - BM25 full-text search engine
|
|
113
111
|
|
|
114
112
|
Usage:
|
|
@@ -132,14 +130,15 @@ Examples:
|
|
|
132
130
|
|
|
133
131
|
For detailed usage guide: search usage
|
|
134
132
|
`);
|
|
135
|
-
|
|
136
|
-
}
|
|
137
|
-
const engine = new engine_1.SearchEngine();
|
|
138
|
-
|
|
139
|
-
if (tracer) {
|
|
140
|
-
tracer.run('crewx/search', () => engine.search(options)).catch(console.error);
|
|
133
|
+
return;
|
|
134
|
+
}
|
|
135
|
+
const engine = new engine_1.SearchEngine();
|
|
136
|
+
await engine.search(options);
|
|
141
137
|
}
|
|
142
|
-
|
|
143
|
-
|
|
138
|
+
if (require.main === module) {
|
|
139
|
+
loadTracer().then(tracer => {
|
|
140
|
+
const opts = tracer ? { skillVersion: tracer.getOwnSkillVersion(__dirname) } : {};
|
|
141
|
+
return tracer ? tracer.run('crewx/search', () => main(), opts) : main();
|
|
142
|
+
}).catch(console.error);
|
|
144
143
|
}
|
|
145
144
|
//# sourceMappingURL=cli.js.map
|
package/dist/cli.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../cli.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
1
|
+
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../cli.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAgDA,oBAwEC;AArGD,uCAAyB;AACzB,2CAA6B;AAC7B,yCAA4C;AAI5C,IAAI,WAAW,GAAG,SAAS,CAAC;AAC5B,IAAI,CAAC;IACH,MAAM,QAAQ,GAAG,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,cAAc,CAAC,CAAC;QAClE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,cAAc,CAAC;QACtC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,IAAI,EAAE,cAAc,CAAC,CAAC;IAC/C,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,YAAY,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC,CAAC;IAC5D,IAAI,IAAI,EAAE,OAAO;QAAE,WAAW,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;AACxD,CAAC;AAAC,MAAM,CAAC,CAAA,CAAC;AAGV,KAAK,UAAU,UAAU;IACvB,IAAI,CAAC;QAAC,OAAO,wDAAa,4BAA4B,GAAC,CAAC;IAAC,CAAC;IAAC,MAAM,CAAC;QAAC,OAAO,IAAI,CAAC;IAAC,CAAC;AACnF,CAAC;AAED,SAAS,iBAAiB;IACxB,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,EAAE,IAAI,EAAE,UAAU,CAAC,CAAC;IACvD,IAAI,EAAE,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,YAAY,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC;IAC1D,CAAC;SAAM,CAAC;QACN,OAAO,CAAC,GAAG,CAAC,0CAA0C,CAAC,CAAC;IAC1D,CAAC;AACH,CAAC;AAEM,KAAK,UAAU,IAAI,CAAC,SAAoB;IAC7C,MAAM,IAAI,GAAG,SAAS,IAAI,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IAEhD,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,WAAW,EAAE,CAAC;QAC5B,OAAO,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC;QACzB,OAAO;IACT,CAAC;IAED,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,OAAO,EAAE,CAAC;QACxB,iBAAiB,EAAE,CAAC;QACpB,OAAO;IACT,CAAC;IAED,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,QAAQ,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QACnE,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC;IAClB,CAAC;IAED,MAAM,OAAO,GAAkB;QAC7B,KAAK,EAAE,EAAE;QACT,UAAU,EAAE,GAAG;QACf,IAAI,EAAE,SAAS;QACf,KAAK,EAAE,EAAE;QACT,IAAI,EAAE,KAAK;KACZ,CAAC;IAGF,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACpB,IAAI,CAAC,GAAG;YAAE,SAAS;QACnB,IAAI,GAAG,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC;YAC9B,OAAO,CAAC,IAAI,GAAG,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACnD,CAAC;aAAM,IAAI,GAAG,CAAC,UAAU,CAAC,UAAU,CAAC,EAAE,CAAC;YACtC,OAAO,CAAC,KAAK,GAAG,QAAQ,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,EAAE,CAAC,CAAC;QAC1D,CAAC;aAAM,IAAI,GAAG,KAAK,QAAQ,EAAE,CAAC;YAC5B,OAAO,CAAC,IAAI,GAAG,IAAI,CAAC;QACtB,CAAC;aAAM,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;YAC1B,OAAO,CAAC,KAAK,GAAG,GAAG,CAAC;QACtB,CAAC;aAAM,IAAI,OAAO,CAAC,UAAU,KAAK,GAAG,EAAE,CAAC;YACtC,OAAO,CAAC,UAAU,GAAG,GAAG,CAAC;QAC3B,CAAC;IACH,CAAC;IAED,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;QACnB,OAAO,CAAC,GAAG,CAAC;gBACA,WAAW;;;;;;;;;;;;;;;;;;;;;;CAsB1B,CAAC,CAAC;QACC,OAAO;IACT,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,qBAAY,EAAE,CAAC;IAClC,MAAM,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;AAC/B,CAAC;AAED,IAAI,OAAO,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;IAC5B,UAAU,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE;QACzB,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,MAAM,CAAC,kBAAkB,CAAC,SAAS,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAClF,OAAO,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,cAAc,EAAE,GAAG,EAAE,CAAC,IAAI,EAAE,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAC1E,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;AAC1B,CAAC"}
|
package/dist/src/engine.d.ts
CHANGED
|
@@ -1,5 +1,8 @@
|
|
|
1
|
-
import type { SearchOptions } from './types';
|
|
1
|
+
import type { SearchOptions, SearchResult } from './types';
|
|
2
2
|
export declare class SearchEngine {
|
|
3
|
+
private collectDocuments;
|
|
4
|
+
private runBm25Search;
|
|
3
5
|
search(opts: SearchOptions): Promise<void>;
|
|
6
|
+
searchAndReturn(opts: SearchOptions): Promise<SearchResult[]>;
|
|
4
7
|
}
|
|
5
8
|
//# sourceMappingURL=engine.d.ts.map
|
package/dist/src/engine.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../../src/engine.ts"],"names":[],"mappings":"AAaA,OAAO,KAAK,EAAY,aAAa,
|
|
1
|
+
{"version":3,"file":"engine.d.ts","sourceRoot":"","sources":["../../src/engine.ts"],"names":[],"mappings":"AAaA,OAAO,KAAK,EAAY,aAAa,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AAKrE,qBAAa,YAAY;YAKT,gBAAgB;IAkD9B,OAAO,CAAC,aAAa;IAwCf,MAAM,CAAC,IAAI,EAAE,aAAa,GAAG,OAAO,CAAC,IAAI,CAAC;IAoE1C,eAAe,CAAC,IAAI,EAAE,aAAa,GAAG,OAAO,CAAC,YAAY,EAAE,CAAC;CAiBpE"}
|
package/dist/src/engine.js
CHANGED
|
@@ -38,16 +38,13 @@ const fs = __importStar(require("fs"));
|
|
|
38
38
|
const path = __importStar(require("path"));
|
|
39
39
|
const glob_1 = require("glob");
|
|
40
40
|
const knowledge_core_1 = require("@crewx/knowledge-core");
|
|
41
|
+
const MAX_BODY_SIZE = 10 * 1024 * 1024;
|
|
41
42
|
class SearchEngine {
|
|
42
|
-
async
|
|
43
|
-
const { query, searchPath: rawPath = '.', glob: globPattern = '**/*.md', limit = 20, json = false, } = opts;
|
|
44
|
-
const searchPath = path.resolve(rawPath);
|
|
43
|
+
async collectDocuments(searchPath, globPattern) {
|
|
45
44
|
const pattern = path.join(searchPath, globPattern);
|
|
46
|
-
const
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
process.exit(1);
|
|
50
|
-
}
|
|
45
|
+
const config = (0, knowledge_core_1.loadDocsConfig)((0, knowledge_core_1.resolveProjectRoot)(searchPath));
|
|
46
|
+
const ignore = (0, knowledge_core_1.buildIgnorePatterns)(config.exclude);
|
|
47
|
+
const files = await (0, glob_1.glob)(pattern, { nodir: true, ignore });
|
|
51
48
|
const docs = [];
|
|
52
49
|
for (const file of files) {
|
|
53
50
|
try {
|
|
@@ -63,7 +60,7 @@ class SearchEngine {
|
|
|
63
60
|
}
|
|
64
61
|
catch {
|
|
65
62
|
const lines = content.split('\n');
|
|
66
|
-
title = lines[0].replace(/^#\s*/, '').substring(0, 100);
|
|
63
|
+
title = (lines[0] ?? '').replace(/^#\s*/, '').substring(0, 100);
|
|
67
64
|
}
|
|
68
65
|
}
|
|
69
66
|
else {
|
|
@@ -73,16 +70,17 @@ class SearchEngine {
|
|
|
73
70
|
file: relativePath,
|
|
74
71
|
fullPath: file,
|
|
75
72
|
title,
|
|
76
|
-
body: body.substring(0,
|
|
73
|
+
body: body.substring(0, MAX_BODY_SIZE),
|
|
77
74
|
});
|
|
78
75
|
}
|
|
79
76
|
catch {
|
|
80
77
|
}
|
|
81
78
|
}
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
79
|
+
return { docs, fileCount: files.length };
|
|
80
|
+
}
|
|
81
|
+
runBm25Search(docs, query, limit) {
|
|
82
|
+
if (docs.length < 3)
|
|
83
|
+
return [];
|
|
86
84
|
const kcEntries = docs.map((doc, idx) => ({
|
|
87
85
|
id: String(idx),
|
|
88
86
|
type: 'document',
|
|
@@ -96,14 +94,28 @@ class SearchEngine {
|
|
|
96
94
|
fullPath: doc.fullPath,
|
|
97
95
|
}));
|
|
98
96
|
const scored = (0, knowledge_core_1.searchEntries)(kcEntries, query, { title: 3, body: 1 });
|
|
99
|
-
|
|
97
|
+
const results = scored.map(entry => ({
|
|
100
98
|
file: entry['file'],
|
|
101
99
|
fullPath: entry['fullPath'],
|
|
102
100
|
title: entry['title'],
|
|
103
101
|
body: entry.body || '',
|
|
104
102
|
score: entry._score,
|
|
105
103
|
}));
|
|
106
|
-
|
|
104
|
+
return results.slice(0, limit);
|
|
105
|
+
}
|
|
106
|
+
async search(opts) {
|
|
107
|
+
const { query, searchPath: rawPath = '.', glob: globPattern = '**/*.md', limit = 20, json = false, } = opts;
|
|
108
|
+
const searchPath = path.resolve(rawPath);
|
|
109
|
+
const { docs, fileCount } = await this.collectDocuments(searchPath, globPattern);
|
|
110
|
+
if (fileCount === 0) {
|
|
111
|
+
console.log(`파일을 찾을 수 없습니다: ${path.join(searchPath, globPattern)}`);
|
|
112
|
+
process.exit(1);
|
|
113
|
+
}
|
|
114
|
+
if (docs.length === 0) {
|
|
115
|
+
console.log('읽을 수 있는 파일이 없습니다.');
|
|
116
|
+
process.exit(1);
|
|
117
|
+
}
|
|
118
|
+
const results = this.runBm25Search(docs, query, limit);
|
|
107
119
|
if (results.length === 0) {
|
|
108
120
|
console.log(`'${query}' 관련 결과를 찾을 수 없습니다.`);
|
|
109
121
|
process.exit(0);
|
|
@@ -116,7 +128,7 @@ class SearchEngine {
|
|
|
116
128
|
})), null, 2));
|
|
117
129
|
}
|
|
118
130
|
else {
|
|
119
|
-
console.log(`\n🔍 검색: "${query}" (${results.length}건, ${
|
|
131
|
+
console.log(`\n🔍 검색: "${query}" (${results.length}건, ${fileCount}개 파일 중)\n`);
|
|
120
132
|
for (const r of results) {
|
|
121
133
|
const title = r.title.substring(0, 60).replace(/\n/g, ' ');
|
|
122
134
|
console.log(`[${r.score}점] ${r.file}`);
|
|
@@ -134,6 +146,15 @@ class SearchEngine {
|
|
|
134
146
|
console.log(searchBlock);
|
|
135
147
|
}
|
|
136
148
|
}
|
|
149
|
+
async searchAndReturn(opts) {
|
|
150
|
+
const { query, searchPath: rawPath = '.', glob: globPattern = '**/*.md', limit = 20, } = opts;
|
|
151
|
+
const searchPath = path.resolve(rawPath);
|
|
152
|
+
const { docs } = await this.collectDocuments(searchPath, globPattern);
|
|
153
|
+
if (docs.length === 0) {
|
|
154
|
+
return [];
|
|
155
|
+
}
|
|
156
|
+
return this.runBm25Search(docs, query, limit);
|
|
157
|
+
}
|
|
137
158
|
}
|
|
138
159
|
exports.SearchEngine = SearchEngine;
|
|
139
160
|
//# sourceMappingURL=engine.js.map
|
package/dist/src/engine.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"engine.js","sourceRoot":"","sources":["../../src/engine.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAQA,uCAAyB;AACzB,2CAA6B;AAC7B,+BAA4B;AAC5B,
|
|
1
|
+
{"version":3,"file":"engine.js","sourceRoot":"","sources":["../../src/engine.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AAQA,uCAAyB;AACzB,2CAA6B;AAC7B,+BAA4B;AAC5B,0DAAqK;AAKrK,MAAM,aAAa,GAAG,EAAE,GAAG,IAAI,GAAG,IAAI,CAAC;AAEvC,MAAa,YAAY;IAKf,KAAK,CAAC,gBAAgB,CAC5B,UAAkB,EAClB,WAAmB;QAEnB,MAAM,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,UAAU,EAAE,WAAW,CAAC,CAAC;QACnD,MAAM,MAAM,GAAG,IAAA,+BAAc,EAAC,IAAA,mCAAkB,EAAC,UAAU,CAAC,CAAC,CAAC;QAC9D,MAAM,MAAM,GAAG,IAAA,oCAAmB,EAAC,MAAM,CAAC,OAAO,CAAC,CAAC;QACnD,MAAM,KAAK,GAAG,MAAM,IAAA,WAAI,EAAC,OAAO,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC;QAE3D,MAAM,IAAI,GAAe,EAAE,CAAC;QAC5B,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;YACzB,IAAI,CAAC;gBACH,MAAM,OAAO,GAAG,EAAE,CAAC,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;gBAC/C,MAAM,YAAY,GAAG,IAAI,CAAC,QAAQ,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;gBAErD,IAAI,KAAK,GAAG,EAAE,CAAC;gBACf,IAAI,IAAI,GAAG,OAAO,CAAC;gBAEnB,IAAI,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC;oBACzB,IAAI,CAAC;wBACH,MAAM,MAAM,GAAG,IAAA,iCAAgB,EAAC,OAAO,CAAC,CAAC;wBACzC,KAAK,GAAI,MAAM,CAAC,IAAI,CAAC,OAAO,CAAY,IAAK,MAAM,CAAC,IAAI,CAAC,SAAS,CAAY,IAAI,EAAE,CAAC;wBACrF,IAAI,GAAG,MAAM,CAAC,OAAO,CAAC;oBACxB,CAAC;oBAAC,MAAM,CAAC;wBACP,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;wBAClC,KAAK,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;oBAClE,CAAC;gBACH,CAAC;qBAAM,CAAC;oBACN,KAAK,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;gBAC9B,CAAC;gBAED,IAAI,CAAC,IAAI,CAAC;oBACR,IAAI,EAAE,YAAY;oBAClB,QAAQ,EAAE,IAAI;oBACd,KAAK;oBACL,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,aAAa,CAAC;iBACvC,CAAC,CAAC;YACL,CAAC;YAAC,MAAM,CAAC;YAET,CAAC;QACH,CAAC;QAED,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,CAAC,MAAM,EAAE,CAAC;IAC3C,CAAC;IAOO,aAAa,CACnB,IAAgB,EAChB,KAAa,EACb,KAAa;QAEb,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC;YAAE,OAAO,EAAE,CAAC;QAG/B,MAAM,SAAS,GAAqB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC;YAC1D,EAAE,EAAE,MAAM,CAAC,GAAG,CAAC;YACf,IAAI,EAAE,UAAmB;YACzB,QAAQ,EAAE,SAAS;YACnB,IAAI,EAAE,EAAE;YACR,IAAI,EAAE,EAAE;YACR,OAAO,EAAE,GAAG,CAAC,KAAK;YAClB,IAAI,EAAE,GAAG,CAAC,IAAI;YACd,KAAK,EAAE,GAAG,CAAC,KAAK;YAChB,IAAI,EAAE,GAAG,CAAC,IAAI;YACd,QAAQ,EAAE,GAAG,CAAC,QAAQ;SACvB,CAAC,CAAC,CAAC;QAGJ,MAAM,MAAM,GAAG,IAAA,8BAAa,EAAC,SAAS,EAAE,KAAK,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;QAGtE,MAAM,OAAO,GAAmB,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;YACnD,IAAI,EAAE,KAAK,CAAC,MAAM,CAAW;YAC7B,QAAQ,EAAE,KAAK,CAAC,UAAU,CAAW;YACrC,KAAK,EAAE,KAAK,CAAC,OAAO,CAAW;YAC/B,IAAI,EAAE,KAAK,CAAC,IAAI,IAAI,EAAE;YACtB,KAAK,EAAE,KAAK,CAAC,MAAM;SACpB,CAAC,CAAC,CAAC;QAEJ,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;IACjC,CAAC;IAMD,KAAK,CAAC,MAAM,CAAC,IAAmB;QAC9B,MAAM,EACJ,KAAK,EACL,UAAU,EAAE,OAAO,GAAG,GAAG,EACzB,IAAI,EAAE,WAAW,GAAG,SAAS,EAC7B,KAAK,GAAG,EAAE,EACV,IAAI,GAAG,KAAK,GACb,GAAG,IAAI,CAAC;QAET,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QAEzC,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,GAAG,MAAM,IAAI,CAAC,gBAAgB,CAAC,UAAU,EAAE,WAAW,CAAC,CAAC;QAEjF,IAAI,SAAS,KAAK,CAAC,EAAE,CAAC;YACpB,OAAO,CAAC,GAAG,CAAC,kBAAkB,IAAI,CAAC,IAAI,CAAC,UAAU,EAAE,WAAW,CAAC,EAAE,CAAC,CAAC;YACpE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAED,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACtB,OAAO,CAAC,GAAG,CAAC,mBAAmB,CAAC,CAAC;YACjC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAED,MAAM,OAAO,GAAG,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;QAEvD,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACzB,OAAO,CAAC,GAAG,CAAC,IAAI,KAAK,qBAAqB,CAAC,CAAC;YAC5C,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;QAGD,IAAI,IAAI,EAAE,CAAC;YACT,OAAO,CAAC,GAAG,CACT,IAAI,CAAC,SAAS,CACZ,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;gBAChB,IAAI,EAAE,CAAC,CAAC,IAAI;gBACZ,KAAK,EAAE,CAAC,CAAC,KAAK;gBACd,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC;aACjC,CAAC,CAAC,EACH,IAAI,EACJ,CAAC,CACF,CACF,CAAC;QACJ,CAAC;aAAM,CAAC;YACN,OAAO,CAAC,GAAG,CAAC,aAAa,KAAK,MAAM,OAAO,CAAC,MAAM,MAAM,SAAS,WAAW,CAAC,CAAC;YAC9E,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;gBACxB,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;gBAC3D,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,KAAK,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC;gBACvC,IAAI,KAAK;oBAAE,OAAO,CAAC,GAAG,CAAC,UAAU,KAAK,EAAE,CAAC,CAAC;YAC5C,CAAC;YAGD,MAAM,SAAS,GAAkB;gBAC/B,SAAS,EAAE,QAAQ;gBACnB,MAAM,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,UAAU,EAAE;gBACnC,WAAW,EAAE,OAAO,CAAC,MAAM;aAC5B,CAAC;YACF,MAAM,aAAa,GAAG,IAAA,oCAAmB,EAAC,SAAS,CAAC,CAAC;YACrD,MAAM,WAAW,GAAG,IAAA,8BAAa,EAAC,aAAa,CAAC,CAAC;YACjD,IAAI,WAAW;gBAAE,OAAO,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC;QAC5C,CAAC;IACH,CAAC;IAOD,KAAK,CAAC,eAAe,CAAC,IAAmB;QACvC,MAAM,EACJ,KAAK,EACL,UAAU,EAAE,OAAO,GAAG,GAAG,EACzB,IAAI,EAAE,WAAW,GAAG,SAAS,EAC7B,KAAK,GAAG,EAAE,GACX,GAAG,IAAI,CAAC;QAET,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;QACzC,MAAM,EAAE,IAAI,EAAE,GAAG,MAAM,IAAI,CAAC,gBAAgB,CAAC,UAAU,EAAE,WAAW,CAAC,CAAC;QAEtE,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACtB,OAAO,EAAE,CAAC;QACZ,CAAC;QAED,OAAO,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC;IAChD,CAAC;CACF;AApLD,oCAoLC"}
|
package/package.json
CHANGED
|
@@ -1,19 +1,27 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@crewx/search",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.6",
|
|
4
4
|
"description": "BM25 full-text search engine for CrewX documents and code",
|
|
5
5
|
"type": "commonjs",
|
|
6
6
|
"main": "dist/src/engine.js",
|
|
7
7
|
"types": "dist/src/engine.d.ts",
|
|
8
8
|
"exports": {
|
|
9
9
|
".": {
|
|
10
|
+
"types": "./dist/src/engine.d.ts",
|
|
11
|
+
"import": "./dist/src/engine.js",
|
|
10
12
|
"require": "./dist/src/engine.js",
|
|
11
|
-
"
|
|
13
|
+
"default": "./dist/src/engine.js"
|
|
14
|
+
},
|
|
15
|
+
"./cli": {
|
|
16
|
+
"import": "./dist/cli.js",
|
|
17
|
+
"require": "./dist/cli.js",
|
|
18
|
+
"default": "./dist/cli.js"
|
|
12
19
|
}
|
|
13
20
|
},
|
|
14
21
|
"bin": { "search": "./dist/cli.js" },
|
|
15
22
|
"files": ["dist", "SKILL.md"],
|
|
16
23
|
"scripts": {
|
|
24
|
+
"prepare": "npm run build",
|
|
17
25
|
"build": "tsc",
|
|
18
26
|
"dev": "ts-node cli.ts",
|
|
19
27
|
"clean": "rm -rf dist"
|