@maiyunnet/kebab 9.8.0 → 9.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/doc/kebab-rag.md +20 -20
- package/index.d.ts +1 -1
- package/index.js +1 -1
- package/lib/vector.d.ts +14 -17
- package/lib/vector.js +45 -70
- package/package.json +1 -2
- package/sys/master.js +109 -35
- package/www/example/ctr/test.js +1 -1
package/doc/kebab-rag.md
CHANGED
|
@@ -1360,7 +1360,7 @@ index/variables/VER.md
|
|
|
1360
1360
|
|
|
1361
1361
|
# Variable: VER
|
|
1362
1362
|
|
|
1363
|
-
> `const` **VER**: `"9.
|
|
1363
|
+
> `const` **VER**: `"9.10.0"` = `'9.10.0'`
|
|
1364
1364
|
|
|
1365
1365
|
Defined in: [index.ts:10](https://github.com/maiyunnet/kebab/blob/master/index.ts#L10)
|
|
1366
1366
|
|
|
@@ -19381,7 +19381,7 @@ lib/vector/classes/Vector.md
|
|
|
19381
19381
|
|
|
19382
19382
|
# Class: Vector
|
|
19383
19383
|
|
|
19384
|
-
Defined in: [lib/vector.ts:
|
|
19384
|
+
Defined in: [lib/vector.ts:25](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L25)
|
|
19385
19385
|
|
|
19386
19386
|
## Constructors
|
|
19387
19387
|
|
|
@@ -19389,7 +19389,7 @@ Defined in: [lib/vector.ts:31](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19389
19389
|
|
|
19390
19390
|
> **new Vector**(`etc`): `Vector`
|
|
19391
19391
|
|
|
19392
|
-
Defined in: [lib/vector.ts:
|
|
19392
|
+
Defined in: [lib/vector.ts:30](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L30)
|
|
19393
19393
|
|
|
19394
19394
|
#### Parameters
|
|
19395
19395
|
|
|
@@ -19405,9 +19405,9 @@ Defined in: [lib/vector.ts:36](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19405
19405
|
|
|
19406
19406
|
### delete()
|
|
19407
19407
|
|
|
19408
|
-
> **delete**(`data`): `Promise`\<`false` \| `
|
|
19408
|
+
> **delete**(`data`): `Promise`\<`false` \| \{ `deletedCount`: `number`; \}\>
|
|
19409
19409
|
|
|
19410
|
-
Defined in: [lib/vector.ts:
|
|
19410
|
+
Defined in: [lib/vector.ts:116](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L116)
|
|
19411
19411
|
|
|
19412
19412
|
删除数据
|
|
19413
19413
|
|
|
@@ -19429,15 +19429,15 @@ Defined in: [lib/vector.ts:104](https://github.com/maiyunnet/kebab/blob/master/l
|
|
|
19429
19429
|
|
|
19430
19430
|
#### Returns
|
|
19431
19431
|
|
|
19432
|
-
`Promise`\<`false` \| `
|
|
19432
|
+
`Promise`\<`false` \| \{ `deletedCount`: `number`; \}\>
|
|
19433
19433
|
|
|
19434
19434
|
***
|
|
19435
19435
|
|
|
19436
19436
|
### insert()
|
|
19437
19437
|
|
|
19438
|
-
> **insert**(`data`): `Promise`\<`false` \| `
|
|
19438
|
+
> **insert**(`data`): `Promise`\<`false` \| \{ `insertCount`: `number`; `insertIds`: (`string` \| `number`)[]; \}\>
|
|
19439
19439
|
|
|
19440
|
-
Defined in: [lib/vector.ts:
|
|
19440
|
+
Defined in: [lib/vector.ts:85](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L85)
|
|
19441
19441
|
|
|
19442
19442
|
插入数据
|
|
19443
19443
|
|
|
@@ -19453,21 +19453,21 @@ Defined in: [lib/vector.ts:79](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19453
19453
|
|
|
19454
19454
|
###### data
|
|
19455
19455
|
|
|
19456
|
-
`
|
|
19456
|
+
`Record`\<`string`, `any`\>[]
|
|
19457
19457
|
|
|
19458
19458
|
要插入的数据
|
|
19459
19459
|
|
|
19460
19460
|
#### Returns
|
|
19461
19461
|
|
|
19462
|
-
`Promise`\<`false` \| `
|
|
19462
|
+
`Promise`\<`false` \| \{ `insertCount`: `number`; `insertIds`: (`string` \| `number`)[]; \}\>
|
|
19463
19463
|
|
|
19464
19464
|
***
|
|
19465
19465
|
|
|
19466
19466
|
### seach()
|
|
19467
19467
|
|
|
19468
|
-
> **seach**(`data`): `Promise`\<`false` \| `
|
|
19468
|
+
> **seach**(`data`): `Promise`\<`false` \| `object`[]\>
|
|
19469
19469
|
|
|
19470
|
-
Defined in: [lib/vector.ts:
|
|
19470
|
+
Defined in: [lib/vector.ts:35](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L35)
|
|
19471
19471
|
|
|
19472
19472
|
搜索
|
|
19473
19473
|
|
|
@@ -19513,7 +19513,7 @@ Defined in: [lib/vector.ts:42](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19513
19513
|
|
|
19514
19514
|
#### Returns
|
|
19515
19515
|
|
|
19516
|
-
`Promise`\<`false` \| `
|
|
19516
|
+
`Promise`\<`false` \| `object`[]\>
|
|
19517
19517
|
|
|
19518
19518
|
lib/vector/functions/get.md
|
|
19519
19519
|
---
|
|
@@ -19528,7 +19528,7 @@ lib/vector/functions/get.md
|
|
|
19528
19528
|
|
|
19529
19529
|
> **get**(`ctrEtc`): [`Vector`](../classes/Vector.md)
|
|
19530
19530
|
|
|
19531
|
-
Defined in: [lib/vector.ts:
|
|
19531
|
+
Defined in: [lib/vector.ts:152](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L152)
|
|
19532
19532
|
|
|
19533
19533
|
创建一个 Vector 对象
|
|
19534
19534
|
|
|
@@ -19578,7 +19578,7 @@ lib/vector/interfaces/IOptions.md
|
|
|
19578
19578
|
|
|
19579
19579
|
# Interface: IOptions
|
|
19580
19580
|
|
|
19581
|
-
Defined in: [lib/vector.ts:
|
|
19581
|
+
Defined in: [lib/vector.ts:12](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L12)
|
|
19582
19582
|
|
|
19583
19583
|
选项
|
|
19584
19584
|
|
|
@@ -19588,7 +19588,7 @@ Defined in: [lib/vector.ts:18](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19588
19588
|
|
|
19589
19589
|
> `optional` **host?**: `string`
|
|
19590
19590
|
|
|
19591
|
-
Defined in: [lib/vector.ts:
|
|
19591
|
+
Defined in: [lib/vector.ts:14](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L14)
|
|
19592
19592
|
|
|
19593
19593
|
主机地址
|
|
19594
19594
|
|
|
@@ -19598,7 +19598,7 @@ Defined in: [lib/vector.ts:20](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19598
19598
|
|
|
19599
19599
|
> `optional` **name?**: `string`
|
|
19600
19600
|
|
|
19601
|
-
Defined in: [lib/vector.ts:
|
|
19601
|
+
Defined in: [lib/vector.ts:18](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L18)
|
|
19602
19602
|
|
|
19603
19603
|
数据库名称
|
|
19604
19604
|
|
|
@@ -19608,7 +19608,7 @@ Defined in: [lib/vector.ts:24](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19608
19608
|
|
|
19609
19609
|
> `optional` **port?**: `number`
|
|
19610
19610
|
|
|
19611
|
-
Defined in: [lib/vector.ts:
|
|
19611
|
+
Defined in: [lib/vector.ts:16](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L16)
|
|
19612
19612
|
|
|
19613
19613
|
端口号
|
|
19614
19614
|
|
|
@@ -19618,7 +19618,7 @@ Defined in: [lib/vector.ts:22](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19618
19618
|
|
|
19619
19619
|
> `optional` **pwd?**: `string`
|
|
19620
19620
|
|
|
19621
|
-
Defined in: [lib/vector.ts:
|
|
19621
|
+
Defined in: [lib/vector.ts:22](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L22)
|
|
19622
19622
|
|
|
19623
19623
|
密码
|
|
19624
19624
|
|
|
@@ -19628,7 +19628,7 @@ Defined in: [lib/vector.ts:28](https://github.com/maiyunnet/kebab/blob/master/li
|
|
|
19628
19628
|
|
|
19629
19629
|
> `optional` **user?**: `string`
|
|
19630
19630
|
|
|
19631
|
-
Defined in: [lib/vector.ts:
|
|
19631
|
+
Defined in: [lib/vector.ts:20](https://github.com/maiyunnet/kebab/blob/master/lib/vector.ts#L20)
|
|
19632
19632
|
|
|
19633
19633
|
用户名
|
|
19634
19634
|
|
package/index.d.ts
CHANGED
package/index.js
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
* --- 本文件用来定义每个目录实体地址的常量 ---
|
|
7
7
|
*/
|
|
8
8
|
/** --- 当前系统版本号 --- */
|
|
9
|
-
export const VER = '9.
|
|
9
|
+
export const VER = '9.10.0';
|
|
10
10
|
// --- 服务端用的路径 ---
|
|
11
11
|
const imu = decodeURIComponent(import.meta.url).replace('file://', '').replace(/^\/(\w:)/, '$1');
|
|
12
12
|
/** --- /xxx/xxx --- */
|
package/lib/vector.d.ts
CHANGED
|
@@ -1,9 +1,8 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Project: Kebab, User: JianSuoQiYue
|
|
3
3
|
* Date: 2025-10-28 15:18:41
|
|
4
|
-
* Last: 2025-10-28 15:18:44
|
|
4
|
+
* Last: 2025-10-28 15:18:44, 2026-5-20 23:31:45
|
|
5
5
|
*/
|
|
6
|
-
import * as milvus from '@zilliz/milvus2-sdk-node';
|
|
7
6
|
import * as kebab from '#kebab/index.js';
|
|
8
7
|
import * as sCtr from '#kebab/sys/ctr.js';
|
|
9
8
|
/** --- 选项 --- */
|
|
@@ -37,32 +36,30 @@ export declare class Vector {
|
|
|
37
36
|
'metric'?: 'L2' | 'IP' | 'COSINE';
|
|
38
37
|
/** --- 输出的字段,如 ['book_id', 'word_count'],默认全部 --- */
|
|
39
38
|
'fields'?: string[];
|
|
40
|
-
}): Promise<
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
metric_type: "L2" | "IP" | "COSINE";
|
|
46
|
-
output_fields: string[] | undefined;
|
|
47
|
-
}>>;
|
|
39
|
+
}): Promise<Array<{
|
|
40
|
+
'id': string | number;
|
|
41
|
+
'distance': number;
|
|
42
|
+
'entity': Record<string, kebab.Json>;
|
|
43
|
+
}> | false>;
|
|
48
44
|
/** --- 插入数据 --- */
|
|
49
45
|
insert(data: {
|
|
50
46
|
/** --- 表名 --- */
|
|
51
47
|
'collection': string;
|
|
52
48
|
/** --- 要插入的数据 --- */
|
|
53
|
-
'data':
|
|
54
|
-
}): Promise<
|
|
49
|
+
'data': Array<Record<string, kebab.Json>>;
|
|
50
|
+
}): Promise<{
|
|
51
|
+
'insertCount': number;
|
|
52
|
+
'insertIds': Array<string | number>;
|
|
53
|
+
} | false>;
|
|
55
54
|
/** --- 删除数据 --- */
|
|
56
55
|
delete(data: {
|
|
57
56
|
/** --- 表名 --- */
|
|
58
57
|
'collection': string;
|
|
59
58
|
/** --- 过滤器,如 word_count > 0 and book_id in [1, 2, 3] --- */
|
|
60
59
|
'filter': string;
|
|
61
|
-
}): Promise<
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
*/
|
|
65
|
-
private _getConnection;
|
|
60
|
+
}): Promise<{
|
|
61
|
+
'deletedCount': number;
|
|
62
|
+
} | false>;
|
|
66
63
|
}
|
|
67
64
|
/**
|
|
68
65
|
* --- 创建一个 Vector 对象 ---
|
package/lib/vector.js
CHANGED
|
@@ -1,13 +1,6 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Project: Kebab, User: JianSuoQiYue
|
|
3
|
-
* Date: 2025-10-28 15:18:41
|
|
4
|
-
* Last: 2025-10-28 15:18:44
|
|
5
|
-
*/
|
|
6
|
-
import * as milvus from '@zilliz/milvus2-sdk-node';
|
|
7
1
|
import * as lCore from '#kebab/lib/core.js';
|
|
2
|
+
import * as lUndici from '#kebab/lib/undici.js';
|
|
8
3
|
import * as sCtr from '#kebab/sys/ctr.js';
|
|
9
|
-
/** --- milvus 的连接对象 --- */
|
|
10
|
-
const links = [];
|
|
11
4
|
export class Vector {
|
|
12
5
|
/** --- 当前的 vector 连接信息 --- */
|
|
13
6
|
_etc;
|
|
@@ -15,97 +8,79 @@ export class Vector {
|
|
|
15
8
|
this._etc = etc;
|
|
16
9
|
}
|
|
17
10
|
/** --- 搜索 --- */
|
|
18
|
-
// eslint-disable-next-line @typescript-eslint/explicit-function-return-type
|
|
19
11
|
async seach(data) {
|
|
20
|
-
const
|
|
21
|
-
|
|
22
|
-
|
|
12
|
+
const body = {
|
|
13
|
+
'dbName': this._etc.name,
|
|
14
|
+
'collectionName': data.collection,
|
|
15
|
+
'data': [data.data],
|
|
16
|
+
'limit': data.limit ?? 3,
|
|
17
|
+
'searchParams': {
|
|
18
|
+
'metricType': data.metric ?? 'L2',
|
|
19
|
+
},
|
|
20
|
+
};
|
|
21
|
+
if (data.filter) {
|
|
22
|
+
body['filter'] = data.filter;
|
|
23
|
+
}
|
|
24
|
+
if (data.fields) {
|
|
25
|
+
body['outputFields'] = data.fields;
|
|
23
26
|
}
|
|
24
27
|
try {
|
|
25
|
-
const res = await
|
|
26
|
-
|
|
27
|
-
'
|
|
28
|
-
'
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
});
|
|
33
|
-
return res;
|
|
28
|
+
const res = await lUndici.postJsonResponseJson(`http://${this._etc.host}:${this._etc.port}/v2/vectordb/entities/search`, body, { 'headers': { 'Authorization': `Bearer ${this._etc.user}:${this._etc.pwd}` } });
|
|
29
|
+
if (res?.code !== 0) {
|
|
30
|
+
lCore.log({}, '[VECTOR][seach][error] ' + (res?.message ?? ''), '-error');
|
|
31
|
+
lCore.debug('[VECTOR][seach]', res);
|
|
32
|
+
return false;
|
|
33
|
+
}
|
|
34
|
+
return res.data;
|
|
34
35
|
}
|
|
35
36
|
catch (e) {
|
|
36
|
-
lCore.log({}, '[VECTOR][seach][error] ' +
|
|
37
|
+
lCore.log({}, '[VECTOR][seach][error] ' + e.message, '-error');
|
|
37
38
|
lCore.debug('[VECTOR][seach]', e);
|
|
38
39
|
return false;
|
|
39
40
|
}
|
|
40
41
|
}
|
|
41
42
|
/** --- 插入数据 --- */
|
|
42
43
|
async insert(data) {
|
|
43
|
-
const link = await this._getConnection();
|
|
44
|
-
if (!link) {
|
|
45
|
-
return false;
|
|
46
|
-
}
|
|
47
44
|
try {
|
|
48
|
-
const res = await
|
|
49
|
-
'
|
|
45
|
+
const res = await lUndici.postJsonResponseJson(`http://${this._etc.host}:${this._etc.port}/v2/vectordb/entities/insert`, {
|
|
46
|
+
'dbName': this._etc.name,
|
|
47
|
+
'collectionName': data.collection,
|
|
50
48
|
'data': data.data,
|
|
51
|
-
});
|
|
52
|
-
|
|
49
|
+
}, { 'headers': { 'Authorization': `Bearer ${this._etc.user}:${this._etc.pwd}` } });
|
|
50
|
+
if (res?.code !== 0) {
|
|
51
|
+
lCore.log({}, '[VECTOR][insert][error] ' + (res?.message ?? ''), '-error');
|
|
52
|
+
lCore.debug('[VECTOR][insert]', res);
|
|
53
|
+
return false;
|
|
54
|
+
}
|
|
55
|
+
return res.data;
|
|
53
56
|
}
|
|
54
57
|
catch (e) {
|
|
55
|
-
lCore.log({}, '[VECTOR][insert][error] ' +
|
|
58
|
+
lCore.log({}, '[VECTOR][insert][error] ' + e.message, '-error');
|
|
56
59
|
lCore.debug('[VECTOR][insert]', e);
|
|
57
60
|
return false;
|
|
58
61
|
}
|
|
59
62
|
}
|
|
60
63
|
/** --- 删除数据 --- */
|
|
61
64
|
async delete(data) {
|
|
62
|
-
const link = await this._getConnection();
|
|
63
|
-
if (!link) {
|
|
64
|
-
return false;
|
|
65
|
-
}
|
|
66
65
|
try {
|
|
67
|
-
const res = await
|
|
68
|
-
'
|
|
66
|
+
const res = await lUndici.postJson(`http://${this._etc.host}:${this._etc.port}/v2/vectordb/entities/delete`, {
|
|
67
|
+
'dbName': this._etc.name,
|
|
68
|
+
'collectionName': data.collection,
|
|
69
69
|
'filter': data.filter,
|
|
70
|
-
});
|
|
71
|
-
|
|
70
|
+
}, { 'headers': { 'Authorization': `Bearer ${this._etc.user}:${this._etc.pwd}` } });
|
|
71
|
+
const json = await res.getJson();
|
|
72
|
+
if (json?.code !== 0) {
|
|
73
|
+
lCore.log({}, '[VECTOR][delete][error] ' + (json?.message ?? ''), '-error');
|
|
74
|
+
return false;
|
|
75
|
+
}
|
|
76
|
+
return json.data;
|
|
72
77
|
}
|
|
73
78
|
catch (e) {
|
|
74
|
-
lCore.log({}, '[VECTOR][delete][error] ' +
|
|
79
|
+
lCore.log({}, '[VECTOR][delete][error] ' + e.message, '-error');
|
|
75
80
|
lCore.debug('[VECTOR][delete]', e);
|
|
76
81
|
return false;
|
|
77
82
|
}
|
|
78
83
|
}
|
|
79
|
-
/**
|
|
80
|
-
* --- 从连接池中获取一个符合要求的连接 ---
|
|
81
|
-
*/
|
|
82
|
-
async _getConnection() {
|
|
83
|
-
const token = `${this._etc.host}-${this._etc.port}-${this._etc.name}-${this._etc.user}`;
|
|
84
|
-
const item = links.find(item => item.token === token);
|
|
85
|
-
if (item) {
|
|
86
|
-
return item.link;
|
|
87
|
-
}
|
|
88
|
-
// --- 没有找到合适的连接,创建一个 ---
|
|
89
|
-
try {
|
|
90
|
-
const link = new milvus.MilvusClient({
|
|
91
|
-
'address': `${this._etc.host}:${this._etc.port}`,
|
|
92
|
-
'ssl': false,
|
|
93
|
-
'database': this._etc.name,
|
|
94
|
-
'username': this._etc.user,
|
|
95
|
-
'password': this._etc.pwd,
|
|
96
|
-
});
|
|
97
|
-
await link.connectPromise;
|
|
98
|
-
links.push({
|
|
99
|
-
'token': token,
|
|
100
|
-
link,
|
|
101
|
-
});
|
|
102
|
-
return link;
|
|
103
|
-
}
|
|
104
|
-
catch (e) {
|
|
105
|
-
lCore.debug('[VECTOR][_getConnection]', e.code, e.message);
|
|
106
|
-
return null;
|
|
107
|
-
}
|
|
108
|
-
}
|
|
109
84
|
}
|
|
110
85
|
/**
|
|
111
86
|
* --- 创建一个 Vector 对象 ---
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@maiyunnet/kebab",
|
|
3
|
-
"version": "9.
|
|
3
|
+
"version": "9.10.0",
|
|
4
4
|
"description": "Simple, easy-to-use, and fully-featured Node.js framework that is ready-to-use out of the box.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"keywords": [
|
|
@@ -34,7 +34,6 @@
|
|
|
34
34
|
"@radix-ui/react-switch": "^1.2.6",
|
|
35
35
|
"@tailwindcss/cli": "^4.3.0",
|
|
36
36
|
"@types/ssh2": "^1.15.5",
|
|
37
|
-
"@zilliz/milvus2-sdk-node": "^3.0.0",
|
|
38
37
|
"ajv": "^8.20.0",
|
|
39
38
|
"ajv-formats": "^3.0.1",
|
|
40
39
|
"class-variance-authority": "^0.7.1",
|
package/sys/master.js
CHANGED
|
@@ -519,8 +519,88 @@ function createRpcListener() {
|
|
|
519
519
|
}));
|
|
520
520
|
return;
|
|
521
521
|
}
|
|
522
|
-
|
|
522
|
+
let limit = msg.limit ?? 100;
|
|
523
|
+
const offset = msg.offset ?? 0;
|
|
523
524
|
let total = 0;
|
|
525
|
+
if (msg.search) {
|
|
526
|
+
// === 搜索模式:total 为匹配行数,offset/limit 均基于匹配行 ===
|
|
527
|
+
// --- shell 单引号安全转义(防止特殊字符破坏命令)---
|
|
528
|
+
const escaped = msg.search.replace(/'/g, "'\\''");
|
|
529
|
+
// --- 获取匹配行总数(grep -c 无匹配时退出码 1,|| echo 0 兜底)---
|
|
530
|
+
const countCmd = format === 'csv'
|
|
531
|
+
? `tail -n +2 "${path}" | grep -F -c '${escaped}' || echo 0`
|
|
532
|
+
: `grep -F -c '${escaped}' "${path}" || echo 0`;
|
|
533
|
+
const countRtn = await lCore.exec(countCmd);
|
|
534
|
+
if (countRtn !== false) {
|
|
535
|
+
total = parseInt(countRtn.trim()) || 0;
|
|
536
|
+
}
|
|
537
|
+
// --- 获取分页数据:第 offset+1 到 offset+limit 条匹配行(1-indexed)---
|
|
538
|
+
const from = offset + 1;
|
|
539
|
+
const to = offset + limit;
|
|
540
|
+
const dataCmd = format === 'csv'
|
|
541
|
+
? `tail -n +2 "${path}" | grep -F '${escaped}' | sed -n '${from},${to}p'`
|
|
542
|
+
: `grep -F '${escaped}' "${path}" | sed -n '${from},${to}p'`;
|
|
543
|
+
const dataRtn = await lCore.exec(dataCmd);
|
|
544
|
+
if (dataRtn === false) {
|
|
545
|
+
res.end(lText.stringifyJson({
|
|
546
|
+
'result': 1,
|
|
547
|
+
'list': [],
|
|
548
|
+
'total': total,
|
|
549
|
+
}));
|
|
550
|
+
return;
|
|
551
|
+
}
|
|
552
|
+
/**
|
|
553
|
+
* --- csv 格式:string[][] 每行是字段数组,顺序与表头一致 ---
|
|
554
|
+
* --- [['H:i:s', unix, url, cookie, session, userAgent, realIp, cfIp, xIp, osMem, procMem, message], ...] ---
|
|
555
|
+
* --- jsonl 格式:object[] 每行是解析后的 JSON 对象 ---
|
|
556
|
+
* --- [{ time, unix, url, cookie, session, userAgent, realIp, cfIp, xIp, osMem, procMem, message }, ...] ---
|
|
557
|
+
*/
|
|
558
|
+
const sList = [];
|
|
559
|
+
for (const rawLine of dataRtn.split('\n')) {
|
|
560
|
+
if (!rawLine) {
|
|
561
|
+
continue;
|
|
562
|
+
}
|
|
563
|
+
if (format === 'jsonl') {
|
|
564
|
+
const obj = lText.parseJson(rawLine);
|
|
565
|
+
if (obj) {
|
|
566
|
+
sList.push(obj);
|
|
567
|
+
}
|
|
568
|
+
}
|
|
569
|
+
else {
|
|
570
|
+
const result = [];
|
|
571
|
+
let currentField = '';
|
|
572
|
+
let inQuotes = false;
|
|
573
|
+
for (let i = 0; i < rawLine.length; ++i) {
|
|
574
|
+
const char = rawLine[i];
|
|
575
|
+
if (char === '"') {
|
|
576
|
+
if (inQuotes && rawLine[i + 1] === '"') {
|
|
577
|
+
currentField += '"';
|
|
578
|
+
++i;
|
|
579
|
+
}
|
|
580
|
+
else {
|
|
581
|
+
inQuotes = !inQuotes;
|
|
582
|
+
}
|
|
583
|
+
}
|
|
584
|
+
else if (char === ',' && !inQuotes) {
|
|
585
|
+
result.push(currentField);
|
|
586
|
+
currentField = '';
|
|
587
|
+
}
|
|
588
|
+
else {
|
|
589
|
+
currentField += char;
|
|
590
|
+
}
|
|
591
|
+
}
|
|
592
|
+
result.push(currentField);
|
|
593
|
+
sList.push(result);
|
|
594
|
+
}
|
|
595
|
+
}
|
|
596
|
+
res.end(lText.stringifyJson({
|
|
597
|
+
'result': 1,
|
|
598
|
+
'list': sList,
|
|
599
|
+
'total': total,
|
|
600
|
+
}));
|
|
601
|
+
return;
|
|
602
|
+
}
|
|
603
|
+
// === 无搜索模式:wc -l 获取总行数,grep -b '^' 定位字节偏移直接跳至 offset ===
|
|
524
604
|
const wclRtn = await lCore.exec(`wc -l "${path}"`);
|
|
525
605
|
if (wclRtn !== false) {
|
|
526
606
|
const wclMatch = /^\s*(\d+)/.exec(wclRtn);
|
|
@@ -532,10 +612,6 @@ function createRpcListener() {
|
|
|
532
612
|
}
|
|
533
613
|
}
|
|
534
614
|
}
|
|
535
|
-
/** --- 剩余 limit --- */
|
|
536
|
-
let limit = msg.limit ?? 100;
|
|
537
|
-
/** --- offset --- */
|
|
538
|
-
const offset = msg.offset ?? 0;
|
|
539
615
|
/**
|
|
540
616
|
* --- 用 grep -b '^' 获取目标行的字节偏移,直接传给 createReadStream start 跳过 offset ---
|
|
541
617
|
* --- jsonl 无表头:跳过 offset 行,从第 offset+1 行开始读 ---
|
|
@@ -560,7 +636,7 @@ function createRpcListener() {
|
|
|
560
636
|
// --- offset 超出文件范围,返回空列表 ---
|
|
561
637
|
res.end(lText.stringifyJson({
|
|
562
638
|
'result': 1,
|
|
563
|
-
'
|
|
639
|
+
'list': [],
|
|
564
640
|
'total': total,
|
|
565
641
|
}));
|
|
566
642
|
return;
|
|
@@ -602,41 +678,39 @@ function createRpcListener() {
|
|
|
602
678
|
// --- startByte > 0 时已跳过 offset(及 csv 表头),从第一行起均为数据行 ---
|
|
603
679
|
// --- startByte === 0 且 csv 时仍需跳过表头(line > 1)---
|
|
604
680
|
if (format === 'csv' && startByte === 0 ? line > 1 : line >= 1) {
|
|
605
|
-
if (
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
--limit;
|
|
611
|
-
}
|
|
681
|
+
if (format === 'jsonl') {
|
|
682
|
+
const obj = lText.parseJson(packet);
|
|
683
|
+
if (obj) {
|
|
684
|
+
list.push(obj);
|
|
685
|
+
--limit;
|
|
612
686
|
}
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
else {
|
|
625
|
-
inQuotes = !inQuotes;
|
|
626
|
-
}
|
|
627
|
-
}
|
|
628
|
-
else if (char === ',' && !inQuotes) {
|
|
629
|
-
result.push(currentField);
|
|
630
|
-
currentField = '';
|
|
687
|
+
}
|
|
688
|
+
else {
|
|
689
|
+
const result = [];
|
|
690
|
+
let currentField = '';
|
|
691
|
+
let inQuotes = false;
|
|
692
|
+
for (let i = 0; i < packet.length; ++i) {
|
|
693
|
+
const char = packet[i];
|
|
694
|
+
if (char === '"') {
|
|
695
|
+
if (inQuotes && packet[i + 1] === '"') {
|
|
696
|
+
currentField += '"';
|
|
697
|
+
++i;
|
|
631
698
|
}
|
|
632
699
|
else {
|
|
633
|
-
|
|
700
|
+
inQuotes = !inQuotes;
|
|
634
701
|
}
|
|
635
702
|
}
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
|
|
703
|
+
else if (char === ',' && !inQuotes) {
|
|
704
|
+
result.push(currentField);
|
|
705
|
+
currentField = '';
|
|
706
|
+
}
|
|
707
|
+
else {
|
|
708
|
+
currentField += char;
|
|
709
|
+
}
|
|
639
710
|
}
|
|
711
|
+
result.push(currentField);
|
|
712
|
+
list.push(result);
|
|
713
|
+
--limit;
|
|
640
714
|
}
|
|
641
715
|
}
|
|
642
716
|
// --- 处理结束 ---
|
package/www/example/ctr/test.js
CHANGED
|
@@ -1729,7 +1729,7 @@ exec: ${JSON.stringify(exec)}<br><br>`);
|
|
|
1729
1729
|
async vector() {
|
|
1730
1730
|
const vector = lVector.get(this);
|
|
1731
1731
|
const res = await vector.seach({
|
|
1732
|
-
'collection': '
|
|
1732
|
+
'collection': 'ai_wiki_content',
|
|
1733
1733
|
'metric': 'COSINE',
|
|
1734
1734
|
'data': [0.4100031323819555, 0.7188991736586672, 0.32890290245747833, 0.9187961849628559, 0.023142186415922916, 0.45239563148580797, 0.23537591588175988, 0.6848990771759962, 0.9865744633216178, 0.8239142304110896, 0.997768380245414, 0.3012929412655765, 0.13731236076734943, 0.5126131685642945, 0.9806290097015617, 0.1870468071764284, 0.17178642706602143, 0.4016660911946244, 0.15484433366942607, 0.29719222215610386, 0.3103748731148619, 0.6892950176658315, 0.01785695346759608, 0.08539564964986557, 0.2655959514380064, 0.9964617086704073, 0.2838371937684081, 0.14987010598298633, 0.6228479079557896, 0.8080833516756243, 0.15438562173495285, 0.5078008662168574, 0.27262149922940804, 0.23104911473240675, 0.6266656488428926, 0.8553958245035254, 0.11699454830789868, 0.2300336351244865, 0.3042943975707315, 0.20182231286864427, 0.14334825938359863, 0.5272057735967806, 0.6568390179350452, 0.11388912236667226, 0.4916225414998463, 0.4913046496135085, 0.6301252401085475, 0.26051075673874036, 0.17023441587522825, 0.371755148312803, 0.5831318953846139, 0.9756820442368113, 0.6818893859710211, 0.5188330204766185, 0.7868335112863099, 0.7263293211859105, 0.15777067513918275, 0.25071428052177835, 0.9055952502390328, 0.39066574162485046, 0.6694768856781026, 0.904527206408617, 0.5359679189729842, 0.42923033393177423, 0.6959492373261136, 0.4721197667950616, 0.6861043448179791, 0.05188999799541438, 0.7346607591003518, 0.1574483630625012, 0.9141756884547747, 0.9116789303557802, 0.7609450526832306, 0.7763073356227885, 0.7334996162535046, 0.8036447394427264, 0.21291521237560262, 0.4628513517221038, 0.005054981316060525, 0.6289756103407573, 0.9376033918403373, 0.79713161153396, 0.1795021378890418, 0.32421299215073773, 0.5585715705291334, 0.1964507020528392, 0.9170965358836494, 0.44199069294404825, 0.46924875851341463, 0.005915168915280544, 0.4519648598811945, 0.7817208441823769, 0.39630447274483926, 0.874803477246406, 0.02593537985516514, 0.03494415508402371, 0.6948556600612117, 0.02531235612016558, 0.3252859275909026, 0.6124602304575255, 0.6662991517754546, 0.054786469959723805, 0.09541281916656552, 0.047971618444194286, 0.9888851535738363, 0.09923862294731634, 0.37389883211204245, 0.0510449762242422, 0.5140187522339308, 0.9165108670272317, 0.473682888066292, 0.7553395418192084, 0.48450479101753663, 0.22080081972551513, 0.8461012374988566, 0.32711442729081885, 0.46740436391036244, 0.9330234279632854, 0.7456194346850014, 0.8575560505025164, 0.2617623984893922, 0.6418053194775086, 0.607749877895083, 0.2176063100000032, 0.5366633111896186, 0.9689538808794083, 0.8226310778269623, 0.4813948057818185]
|
|
1735
1735
|
});
|