pi-information-protecter 0.6.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/README.md +10 -2
- package/dist/scan-worker.mjs +1 -1
- package/package.json +1 -1
- package/protecter.example.json +6 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
|
1
1
|
# Changelog / 更新日志
|
|
2
2
|
|
|
3
|
+
## 0.6.1 — Valid address replacements / 合法的地址替换
|
|
4
|
+
|
|
5
|
+
### Fixed / 修复
|
|
6
|
+
|
|
7
|
+
- Keep replaced IPv4 addresses valid by bounding every octet to 0-255; per-character replacement produced octets such as `574.676.2.86`.
|
|
8
|
+
替换后的 IPv4 保持每段在 0-255 内;逐字符替换会产生如 `574.676.2.86` 的非法地址。
|
|
9
|
+
- Keep replaced IPv6 addresses valid by replacing hex digits within the hexadecimal alphabet and preserving `::` compression, group widths and any embedded IPv4 tail; per-character replacement produced non-hex characters.
|
|
10
|
+
替换后的 IPv6 保持合法:十六进制字符仍在十六进制范围内,并保留 `::` 压缩写法、分组宽度和末尾嵌入的 IPv4;逐字符替换会产生非十六进制字符。
|
|
11
|
+
|
|
12
|
+
### Added / 新增
|
|
13
|
+
|
|
14
|
+
- Add internal-domain, email, IPv4 and IPv6 rule examples to `protecter.example.json`, including a fixed-alias domain.
|
|
15
|
+
在示例配置中新增内网域名、邮箱、IPv4 和 IPv6 规则示例,包含一个固定别名域名。
|
|
16
|
+
- Add regressions asserting `net.isIPv4`/`net.isIPv6` acceptance, `::` preservation, no false match on version-like numbers, and that the shipped example file loads and protects its targets.
|
|
17
|
+
新增回归:验证替换结果通过 `net.isIPv4`/`net.isIPv6`、保留 `::`、不误匹配类版本号数字,以及发布的示例配置可加载并生效。
|
|
18
|
+
|
|
3
19
|
## 0.6.0 — Shape-preserving replacements and single-language messages / 同形替换与单语言消息
|
|
4
20
|
|
|
5
21
|
### Changed / 变更
|
package/README.md
CHANGED
|
@@ -53,7 +53,7 @@ Validated configuration is cached per extension instance. Outgoing scans and sta
|
|
|
53
53
|
}
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
More examples are in [`protecter.example.json`](protecter.example.json); they are not automatically enabled. Number patterns do not validate authenticity or check digits. Prefer exact values for names and passwords.
|
|
56
|
+
More examples are in [`protecter.example.json`](protecter.example.json), covering internal domains, email addresses, IPv4 and IPv6; they are not automatically enabled. Review the address and domain patterns before use: they match public values too, so a rule that is too broad can redact documentation examples, loopback addresses or dependency hosts. Number patterns do not validate authenticity or check digits. Prefer exact values for names and passwords.
|
|
57
57
|
|
|
58
58
|
- **Literals:** strings and `literal` rules match all case-sensitive occurrences, without regex interpretation.
|
|
59
59
|
- **Regex:** JavaScript regex replaces the entire match. Use lookbehind to match only a credential value. Flags `g i m s u` are supported; `g` is added automatically. `/pattern/flags` shorthand is not supported.
|
|
@@ -94,9 +94,13 @@ Since 0.6.0 a matched value is replaced by a random value of the **same shape**
|
|
|
94
94
|
| `张三` | `丅欕` | CJK script, character count |
|
|
95
95
|
| `Nguyễn Văn Tèo` | `Ufefṉg Zżt Sàb` | Latin with diacritics, word boundaries |
|
|
96
96
|
| `ひらがな` | `ゅせはっ` | Hiragana |
|
|
97
|
+
| `192.168.1.10` | `101.203.3.77` | valid IPv4, octets ≤ 255 |
|
|
98
|
+
| `fe80::1ff:fe23:4567` | `af44::6ea:bf19:2700` | valid IPv6, `::` and group widths |
|
|
97
99
|
|
|
98
100
|
This fixes provider errors such as `'9007199254740991' is not of type 'number'`, which happened when a numeric field was replaced by a token string. Numeric JSON values stay numbers, keep their digit count and remain safe integers. If a partial match cannot round-trip as a number, the request is blocked with `SCAN_NUMBER_UNSAFE` instead of sending an invalid type.
|
|
99
101
|
|
|
102
|
+
Addresses are format-aware: an IPv4 match keeps every octet within 0-255, and an IPv6 match keeps hex digits hexadecimal along with `::` compression, group widths and any embedded IPv4 tail, so `net.isIPv4`/`net.isIPv6` and provider validators still accept them. A plain per-character replacement would otherwise yield octets above 255 or non-hex characters. Addresses embedded inside a longer match, such as a URL matched as a whole, still use per-character replacement.
|
|
103
|
+
|
|
100
104
|
Letters and digits are replaced inside their own writing system: ASCII, Latin-1, Latin Extended-A, Latin Extended Additional (including Vietnamese), Greek, Cyrillic, Hebrew, Arabic, Thai, Hiragana, Katakana, Hangul and CJK. Punctuation, whitespace, emoji, symbols and code points outside these ranges are kept unchanged, which is what keeps URLs, emails and embedded JSON parseable.
|
|
101
105
|
|
|
102
106
|
**This is a deliberate trade-off.** An opaque token revealed only that a redaction happened; a shape-preserving value also reveals length, character classes and punctuation structure, and it looks like plausible real data to the model. Text in unsupported scripts is preserved as-is, so a rule matching such text may leave part of it visible.
|
|
@@ -249,7 +253,7 @@ pi -e ./src/index.ts
|
|
|
249
253
|
}
|
|
250
254
|
```
|
|
251
255
|
|
|
252
|
-
更多示例见 [`protecter.example.json`](protecter.example.json)
|
|
256
|
+
更多示例见 [`protecter.example.json`](protecter.example.json),涵盖内网域名、邮箱地址、IPv4 和 IPv6,不会自动启用。地址与域名规则请先核对:它们同样会匹配公开值,过于宽泛的规则可能脱敏文档示例、回环地址或依赖仓库域名。号码规则不验证真实性或校验位,姓名和密码建议使用精确值。
|
|
253
257
|
|
|
254
258
|
- **字面规则:**字符串和 `literal` 按大小写敏感方式匹配全部出现位置,不作正则解释。
|
|
255
259
|
- **正则:**JavaScript 正则替换整个匹配,可用后行断言仅匹配凭证值。支持 `g i m s u`,自动补 `g`,不支持 `/pattern/flags` 简写。
|
|
@@ -290,9 +294,13 @@ pi -e ./src/index.ts
|
|
|
290
294
|
| `张三` | `丅欕` | 汉字体系、字数 |
|
|
291
295
|
| `Nguyễn Văn Tèo` | `Ufefṉg Zżt Sàb` | 带重音拉丁字母、词边界 |
|
|
292
296
|
| `ひらがな` | `ゅせはっ` | 平假名 |
|
|
297
|
+
| `192.168.1.10` | `101.203.3.77` | 合法 IPv4,每段不超过 255 |
|
|
298
|
+
| `fe80::1ff:fe23:4567` | `af44::6ea:bf19:2700` | 合法 IPv6,保留 `::` 与分组宽度 |
|
|
293
299
|
|
|
294
300
|
这修复了类似 `'9007199254740991' is not of type 'number'` 的提供商错误:过去数值字段会被替换成字符串占位符。现在数值仍为数字,位数不变并保持安全整数;若部分匹配无法作为数字精确往返,则以 `SCAN_NUMBER_UNSAFE` 拦截,而不发送错误类型。
|
|
295
301
|
|
|
302
|
+
地址采用格式感知替换:IPv4 匹配保证每段在 0-255 内,IPv6 匹配保证十六进制字符仍为十六进制,并保留 `::` 压缩写法、分组宽度及末尾嵌入的 IPv4,因此 `net.isIPv4`/`net.isIPv6` 和提供商校验仍然通过。若仅逐字符替换,会产生超过 255 的段或非十六进制字符。嵌在更长匹配(如整条 URL)中的地址仍按字符替换。
|
|
303
|
+
|
|
296
304
|
字母和数字在各自文字体系内替换:ASCII、Latin-1、Latin Extended-A、Latin Extended Additional(含越南语)、希腊语、西里尔语、希伯来语、阿拉伯语、泰语、平假名、片假名、谚文及汉字。标点、空白、表情、符号以及上述范围之外的码位保持不变,这正是 URL、邮箱和内嵌 JSON 仍可解析的原因。
|
|
297
305
|
|
|
298
306
|
**这是有意为之的权衡。** 不透明占位符仅暴露“发生了脱敏”;同形值还会暴露长度、字符类别和标点结构,并且在模型看来像真实数据。不支持的文字体系会原样保留,因此匹配这类文本的规则可能残留部分可见内容。
|
package/dist/scan-worker.mjs
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
1
|
/*! SPI Protecter | MIT | see LICENSE / SPI 隐私保护插件 | MIT 许可 */
|
|
2
|
-
import{parentPort as
|
|
2
|
+
import{parentPort as $,workerData as Q}from"node:worker_threads";import{randomBytes as U}from"node:crypto";var V=[[48,57],[65,90],[97,122],[192,214],[216,246],[248,255],[256,383],[7680,7935],[913,937],[945,969],[940,974],[1040,1103],[1488,1514],[1569,1594],[1601,1610],[3585,3630],[3632,3635],[3636,3642],[3648,3652],[3655,3662],[12353,12438],[12449,12538],[44032,55203],[19968,40869]],F=BigInt(Number.MAX_SAFE_INTEGER);function g(n){if(n<=1n)return 0n;let i=n.toString(2).length,c=Math.ceil(i/8),p=(1n<<BigInt(i))-1n;for(let l=0;l<64;l++){let u=0n;for(let A of U(c))u=u<<8n|BigInt(A);if(u&=p,u<n)return u}return n-1n}function W(n){return V.find(([i,c])=>n>=i&&n<=c)}function k(n){let i="";for(let c of n){let p=c.codePointAt(0),l=W(p);if(!l){i+=c;continue}let[u,A]=l,T=BigInt(A-u+1);i+=String.fromCodePoint(u+Number(g(T)))}return i}var Y=/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/,z={1:[0n,9n],2:[10n,99n],3:[100n,255n]};function K(n){if(n.length>1&&n.startsWith("0")){let p="0";for(let l=1;l<n.length;l++)p+=String(g(10n));return p}let[i,c]=z[n.length];return String(i+g(c-i+1n))}function j(n){let i=Y.exec(n);if(!(!i||i.slice(1).some(c=>Number(c)>255)))return i.slice(1).map(K).join(".")}function P(n){let i="";for(let c of n)c>="0"&&c<="9"?i+=String(g(10n)):c>="a"&&c<="f"?i+=String.fromCharCode(97+Number(g(6n))):c>="A"&&c<="F"?i+=String.fromCharCode(65+Number(g(6n))):i+=c;return i}function Z(n){if(!n.includes(":"))return;let i=n.lastIndexOf(":"),c=n.slice(i+1);if(c.includes(".")){let p=j(c);return!p||!/^[0-9A-Fa-f:]*$/.test(n.slice(0,i+1))?void 0:P(n.slice(0,i+1))+p}if(/^[0-9A-Fa-f:]+$/.test(n))return P(n)}function B(n){return j(n)??Z(n)??k(n)}function J(n){if(!/^\d+$/.test(n))return;if(n.length===1)return n==="0"?void 0:String(1n+g(9n));if(n.startsWith("0")){let l="";for(let u of n)l+=u==="0"?"0":String(1n+g(9n));return l}let i=10n**BigInt(n.length-1),c=10n**BigInt(n.length)-1n,p=c>F?F:c;if(!(p<i))return String(i+g(p-i+1n))}function X(n){let i=Number(n);return Number.isFinite(i)&&String(i)===n}var N,M=0;function a(n,i){throw{code:n,rule:N,otherRule:i,node:M}}try{let L=function(e,o){for(let r=0;r<64;r++){let t=o&&/^\d+$/.test(e)?J(e):B(e);if(!(!t||t===e)&&!u.has(t)&&!l.has(t)&&!H.includes(t))return t}},D=function(e,o,r=!1){let t=l.get(e);return t?(o!==void 0&&t!==o&&a("FIXED_MAPPING_CONFLICT"),t):o!==void 0?(u.size>=5e4&&a("SCAN_MAPPING_LIMIT"),u.has(o)&&u.get(o)!==e&&a("FIXED_ALIAS_REUSED"),o.includes(e)&&a("FIXED_SENSITIVE_TARGET"),l.set(e,o),u.set(o,e),A.push([o,e]),o):(u.size>=5e4&&a("SCAN_MAPPING_LIMIT"),t=L(e,r),t||a("SCAN_UNIQUE_FAILED"),l.set(e,t),u.set(t,e),A.push([t,e]),t)},E=function(e,o=!1){let r=[];for(let f of C)if(N=f.rule,f.literal===void 0){f.regex.lastIndex=0;for(let s of e.matchAll(f.regex))s[0].length||a("SCAN_ZERO_WIDTH"),r.push([s.index,s.index+s[0].length,f.replacement,f.rule]),++O>1e5&&a("SCAN_MATCH_LIMIT")}else{if(!f.literal)continue;let s=0;for(;(s=e.indexOf(f.literal,s))!==-1;)r.push([s,s+f.literal.length,f.replacement,f.rule]),++O>1e5&&a("SCAN_MATCH_LIMIT"),s++}r.sort((f,s)=>f[0]-s[0]||s[1]-f[1]);let t=[];for(let f of r){let s=t.at(-1);s&&f[0]<s[1]?((s[2]!==void 0||f[2]!==void 0)&&((s[0]!==f[0]||s[1]!==f[1]||s[2]!==void 0&&f[2]!==void 0&&s[2]!==f[2])&&(N=s[3],a("FIXED_OVERLAP",f[3])),s[2]===void 0&&f[2]!==void 0&&(s[3]=f[3]),s[2]??=f[2]),s[1]=Math.max(s[1],f[1])):t.push([...f])}let x="",h=0;for(let[f,s,d,S]of t){N=S;let m=D(e.slice(f,s),d,o);T.add(m),x+=e.slice(h,f)+m,h=s}return x+e.slice(h)},I=function(e,o=!1){let r="",t=0,x=d=>d.replace(/[.*+?^${}()|[\]\\]/g,"\\$&"),h=[...u.keys()];if(!h.length)return E(e,o);let f=new RegExp(h.sort((d,S)=>S.length-d.length).map(x).join("|"),"g"),s=[...e.matchAll(f)].filter(d=>u.has(d[0]));for(let d of C)if(!(!d.regex||!h.length)){N=d.rule,d.regex.lastIndex=0;for(let S of e.matchAll(d.regex))++O>1e5&&a("SCAN_MATCH_LIMIT"),S[0].length||a("SCAN_ZERO_WIDTH"),s.some(m=>h.includes(m[0])&&S.index<m.index+m[0].length&&S.index+S[0].length>m.index)&&a("FIXED_ALIAS_CROSSING")}for(let d of s)u.has(d[0])&&(r+=E(e.slice(t,d.index),o)+d[0],t=d.index+d[0].length);return r+E(e.slice(t),o)},_=function(e,o=0){if(M=++R,N=void 0,(R>2e5||o>80)&&a("SCAN_COMPLEXITY"),typeof e=="string"){if(b.some(r=>e.includes(r)))return I(e);if(/^\s*[[{]/.test(e)){let r;try{r=JSON.parse(e)}catch{}if(r&&typeof r=="object"){let t=JSON.stringify(_(r,o+1)),x=I(t);if(x===t)return t;try{JSON.parse(x)}catch{a("SCAN_JSON_REWRITE")}return x}}return I(e)}if(typeof e=="number"){let r=String(e),t=I(r,!0);return t===r?e:(X(t)||a("SCAN_NUMBER_UNSAFE"),Number(t))}if(Array.isArray(e))return e.map(r=>_(r,o+1));if(e&&typeof e=="object"){typeof e.type=="string"&&/image|audio|video|document|(^|_)file($|_)/i.test(e.type)&&a("SCAN_ATTACHMENT"),Object.keys(e).some(t=>/^(inlineData|inline_data|fileData|file_data|image_url|input_audio)$/.test(t))&&a("SCAN_ATTACHMENT");let r=Object.create(null);for(let[t,x]of Object.entries(e)){let h=I(t);Object.hasOwn(r,h)&&a("SCAN_KEY_COLLISION"),r[h]=_(x,o+1)}return r}if(e===null||typeof e=="boolean")return e;a("PAYLOAD_JSON")},y=function(e,o,r=0){if(r>80&&a("SCAN_COMPLEXITY"),typeof e=="string"){if(e.includes(o))return!0;let t;try{t=JSON.parse(e)}catch{return!1}return t&&typeof t=="object"?y(t,o,r+1):!1}return Array.isArray(e)?e.some(t=>y(t,o,r+1)):e&&typeof e=="object"?Object.entries(e).some(([t,x])=>t.includes(o)||y(x,o,r+1)):!1};q=L,v=D,ee=E,ne=I,te=_,re=y;let{payload:n,rules:i,entries:c,configRaws:p}=Q,l=new Map(c.map(([e,o])=>[o,e])),u=new Map(c),A=[],T=new Set,C=i.map((e,o)=>{let r=o+1;return typeof e=="string"?{literal:e,rule:r}:e.type==="literal"?{literal:e.value,replacement:e.replacement,rule:r}:{rule:r,replacement:e.replacement,regex:new RegExp(e.pattern,[...new Set((e.flags??"")+"g")].join(""))}});for(let e of l.keys())C.push({literal:e});let b=[...new Set(p.flatMap(e=>[e,JSON.stringify(JSON.parse(e))]))];for(let e of b)C.push({literal:e});let H=JSON.stringify(n),O=0,R=0,w=_(n),G=[...T].filter(e=>y(w,e)).map(e=>[e,u.get(e)]);$.postMessage({payload:w,additions:A,hits:G})}catch(n){$.postMessage({failed:!0,diagnostic:{code:typeof n?.code=="string"?n.code:"SCAN_INTERNAL",rule:n?.rule??N,otherRule:n?.otherRule,node:n?.node??M}})}var q,v,ee,ne,te,re;
|
package/package.json
CHANGED
package/protecter.example.json
CHANGED
|
@@ -6,6 +6,12 @@
|
|
|
6
6
|
{ "type": "literal", "value": "example-private-key-123", "replacement": "MyPrivateKey" },
|
|
7
7
|
{ "type": "literal", "value": "replace-with-your-real-password" },
|
|
8
8
|
{ "type": "regex", "pattern": "https?://[^\\s\\\"<>]+", "flags": "i" },
|
|
9
|
+
{ "type": "literal", "value": "vpn.internal.example", "replacement": "gateway.internal.example" },
|
|
10
|
+
{ "type": "regex", "pattern": "(?<![\\w.-])(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\\.)+(?:corp|internal|intra|lan|local)(?![\\w.-])", "flags": "i" },
|
|
11
|
+
{ "type": "regex", "pattern": "(?<![\\w.-])[A-Za-z0-9._%+-]+@[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?(?:\\.[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?)*\\.[A-Za-z]{2,}(?![\\w.-])" },
|
|
12
|
+
{ "type": "regex", "pattern": "(?<![\\w.])(?:(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)\\.){3}(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)(?![\\w.])" },
|
|
13
|
+
{ "type": "regex", "pattern": "(?<![\\w:.])(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}(?![\\w:.])" },
|
|
14
|
+
{ "type": "regex", "pattern": "(?<![\\w:.])(?:[0-9A-Fa-f]{1,4}:){1,6}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,5})?(?![\\w:.])" },
|
|
9
15
|
{ "type": "regex", "pattern": "(?<!\\d)1[3-9]\\d{9}(?!\\d)" },
|
|
10
16
|
{ "type": "regex", "pattern": "(?<![0-9A-Za-z])\\d{17}[0-9Xx](?![0-9A-Za-z])" },
|
|
11
17
|
{ "type": "regex", "pattern": "(?<=Bearer )[A-Za-z0-9._~+/-]+=*", "flags": "i" }
|