ops-wiki-agent-kit 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/.github/agents/docs-target-catalog.agent.md +0 -40
- package/.github/agents/docs-target-queue-from-catalog.agent.md +0 -35
- package/.github/agents/source-code-to-spec-reviewer.agent.md +0 -53
- package/.github/prompts/00-generate-target-all-spec.prompt.md +0 -35
- package/.github/prompts/04-review-target-spec.prompt.md +0 -24
- package/.github/prompts/docs-target-catalog.prompt.md +0 -30
- package/.github/prompts/docs-target-queue-from-catalog.prompt.md +0 -28
- package/.github/prompts/generate-docs-index.prompt.md +0 -77
- package/.github/skills/database-query/SKILL.md +0 -142
- package/.github/skills/database-query/references/client-commands.md +0 -197
- package/.github/skills/database-query/references/query-safety.md +0 -109
- package/.github/skills/database-query/scripts/find_db_config.py +0 -273
- package/.github/skills/docs-target-catalog/SKILL.md +0 -194
- package/.github/skills/docs-target-catalog/references/docs-target-queue-conversion.md +0 -164
- package/.github/skills/docs-target-catalog/references/entrypoint-source-patterns.md +0 -83
- package/.github/skills/docs-target-catalog/references/output-templates.md +0 -168
- package/.github/skills/docs-target-queue-from-catalog/SKILL.md +0 -85
- package/.github/skills/docs-target-queue-from-catalog/references/docs-target-queue-contract.md +0 -172
- package/.github/skills/docs-target-queue-from-catalog/references/metadata-acquisition-patterns.md +0 -244
- package/.github/skills/docs-target-queue-from-catalog/scripts/write_documentation_target_queue.py +0 -544
|
@@ -1,197 +0,0 @@
|
|
|
1
|
-
# DB Client Commands
|
|
2
|
-
|
|
3
|
-
這份 reference 提供使用 native CLI 進行查詢的方式。實際使用時需依 repo 的 DB type、environment、schema、VPN/bastion/container 條件調整。
|
|
4
|
-
|
|
5
|
-
## General Handling
|
|
6
|
-
|
|
7
|
-
- 優先使用 password prompts、wallets、DSN 或 session environment variables,避免把 password 直接放在 command-line arguments。
|
|
8
|
-
- 查詢完成後,清除暫時性的 environment variables。
|
|
9
|
-
- 可以的話,優先使用 read-only users。
|
|
10
|
-
- query files 應維持為暫時用途,不要 commit。
|
|
11
|
-
- logs 與 final answers 中的 credentials 要遮蔽。
|
|
12
|
-
|
|
13
|
-
## Oracle
|
|
14
|
-
|
|
15
|
-
建議使用的 clients: `sqlplus`、Oracle SQLcl `sql`。
|
|
16
|
-
|
|
17
|
-
```powershell
|
|
18
|
-
sqlplus -L /nolog
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
如果 `sqlplus -V` 或 `sqlplus -L /nolog` 出現 `Error 46 initializing SQL*Plus`、`HTTP proxy setting has incorrect value` 或 `SP2-1502`,先用乾淨的 child process 移除 proxy env 後重試:
|
|
22
|
-
|
|
23
|
-
```powershell
|
|
24
|
-
cmd /c "set HTTP_PROXY=& set HTTPS_PROXY=& set ALL_PROXY=& set GIT_HTTP_PROXY=& set GIT_HTTPS_PROXY=& set NO_PROXY=& sqlplus -V"
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
若版本檢查成功,後續 Oracle read-only query 也應在同樣清掉 proxy env 的 child process 或 helper wrapper 內執行,並在 acquisition ledger 記錄 `sqlplus proxy env cleared before retry`。若清掉 proxy 後仍失敗,才將狀態記為 `connection_failed` 或 `missing_tool`。
|
|
28
|
-
|
|
29
|
-
接著在 client session 內連線:
|
|
30
|
-
|
|
31
|
-
```sql
|
|
32
|
-
connect <username>/<password>@//<host>:<port>/<service_name>
|
|
33
|
-
SELECT 1 FROM dual;
|
|
34
|
-
exit
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
實用的 inspection queries:
|
|
38
|
-
|
|
39
|
-
```sql
|
|
40
|
-
SELECT owner, table_name
|
|
41
|
-
FROM all_tables
|
|
42
|
-
WHERE owner = UPPER('<schema>')
|
|
43
|
-
FETCH FIRST 50 ROWS ONLY;
|
|
44
|
-
|
|
45
|
-
SELECT owner, table_name, column_name, data_type, nullable
|
|
46
|
-
FROM all_tab_columns
|
|
47
|
-
WHERE owner = UPPER('<schema>')
|
|
48
|
-
AND table_name = UPPER('<table_name>')
|
|
49
|
-
ORDER BY column_id;
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
較舊的 Oracle versions 可能需要使用 `WHERE ROWNUM <= 50`,而不是 `FETCH FIRST`。
|
|
53
|
-
|
|
54
|
-
## PostgreSQL
|
|
55
|
-
|
|
56
|
-
建議使用的 client: `psql`。
|
|
57
|
-
|
|
58
|
-
```powershell
|
|
59
|
-
$env:PGPASSWORD = "<password>"
|
|
60
|
-
psql -h <host> -p <port> -U <username> -d <database> -c "SELECT 1;"
|
|
61
|
-
Remove-Item Env:\PGPASSWORD
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
實用的 inspection queries:
|
|
65
|
-
|
|
66
|
-
```sql
|
|
67
|
-
SELECT table_schema, table_name
|
|
68
|
-
FROM information_schema.tables
|
|
69
|
-
WHERE table_schema NOT IN ('pg_catalog', 'information_schema')
|
|
70
|
-
ORDER BY table_schema, table_name
|
|
71
|
-
LIMIT 50;
|
|
72
|
-
|
|
73
|
-
SELECT column_name, data_type, is_nullable
|
|
74
|
-
FROM information_schema.columns
|
|
75
|
-
WHERE table_schema = '<schema>'
|
|
76
|
-
AND table_name = '<table_name>'
|
|
77
|
-
ORDER BY ordinal_position;
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
## SQL Server / Microsoft SQL Server / MS SQL / MSSQL
|
|
81
|
-
|
|
82
|
-
建議使用的 client: `sqlcmd`。
|
|
83
|
-
|
|
84
|
-
```powershell
|
|
85
|
-
$env:SQLCMDPASSWORD = "<password>"
|
|
86
|
-
sqlcmd -S <host>,<port> -d <database> -U <username> -Q "SELECT 1;"
|
|
87
|
-
Remove-Item Env:\SQLCMDPASSWORD
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
若使用 Windows authentication:
|
|
91
|
-
|
|
92
|
-
```powershell
|
|
93
|
-
sqlcmd -S <host>,<port> -d <database> -E -Q "SELECT 1;"
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
實用的 inspection queries:
|
|
97
|
-
|
|
98
|
-
```sql
|
|
99
|
-
SELECT TOP (50) TABLE_SCHEMA, TABLE_NAME
|
|
100
|
-
FROM INFORMATION_SCHEMA.TABLES
|
|
101
|
-
WHERE TABLE_TYPE = 'BASE TABLE'
|
|
102
|
-
ORDER BY TABLE_SCHEMA, TABLE_NAME;
|
|
103
|
-
|
|
104
|
-
SELECT COLUMN_NAME, DATA_TYPE, IS_NULLABLE
|
|
105
|
-
FROM INFORMATION_SCHEMA.COLUMNS
|
|
106
|
-
WHERE TABLE_SCHEMA = '<schema>'
|
|
107
|
-
AND TABLE_NAME = '<table_name>'
|
|
108
|
-
ORDER BY ORDINAL_POSITION;
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
## MySQL And MariaDB
|
|
112
|
-
|
|
113
|
-
建議使用的 clients: `mysql`、`mariadb`。
|
|
114
|
-
|
|
115
|
-
```powershell
|
|
116
|
-
$env:MYSQL_PWD = "<password>"
|
|
117
|
-
mysql --host=<host> --port=<port> --user=<username> --database=<database> --execute="SELECT 1;"
|
|
118
|
-
Remove-Item Env:\MYSQL_PWD
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
實用的 inspection queries:
|
|
122
|
-
|
|
123
|
-
```sql
|
|
124
|
-
SELECT TABLE_SCHEMA, TABLE_NAME
|
|
125
|
-
FROM information_schema.TABLES
|
|
126
|
-
WHERE TABLE_SCHEMA = '<database>'
|
|
127
|
-
ORDER BY TABLE_NAME
|
|
128
|
-
LIMIT 50;
|
|
129
|
-
|
|
130
|
-
SELECT COLUMN_NAME, DATA_TYPE, IS_NULLABLE
|
|
131
|
-
FROM information_schema.COLUMNS
|
|
132
|
-
WHERE TABLE_SCHEMA = '<database>'
|
|
133
|
-
AND TABLE_NAME = '<table_name>'
|
|
134
|
-
ORDER BY ORDINAL_POSITION;
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
## SQLite
|
|
138
|
-
|
|
139
|
-
建議使用的 client: `sqlite3`。
|
|
140
|
-
|
|
141
|
-
```powershell
|
|
142
|
-
sqlite3 <database_file> "SELECT 1;"
|
|
143
|
-
```
|
|
144
|
-
|
|
145
|
-
實用的 inspection commands:
|
|
146
|
-
|
|
147
|
-
```sql
|
|
148
|
-
.tables
|
|
149
|
-
PRAGMA table_info('<table_name>');
|
|
150
|
-
SELECT name, type FROM sqlite_master WHERE type IN ('table', 'view') ORDER BY name LIMIT 50;
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
## IBM Db2
|
|
154
|
-
|
|
155
|
-
建議使用的 client: `db2`。
|
|
156
|
-
|
|
157
|
-
```powershell
|
|
158
|
-
db2 connect to <database> user <username> using <password>
|
|
159
|
-
db2 -x "SELECT 1 FROM sysibm.sysdummy1"
|
|
160
|
-
db2 connect reset
|
|
161
|
-
```
|
|
162
|
-
|
|
163
|
-
實用的 inspection queries:
|
|
164
|
-
|
|
165
|
-
```sql
|
|
166
|
-
SELECT tabschema, tabname
|
|
167
|
-
FROM syscat.tables
|
|
168
|
-
WHERE type = 'T'
|
|
169
|
-
ORDER BY tabschema, tabname
|
|
170
|
-
FETCH FIRST 50 ROWS ONLY;
|
|
171
|
-
|
|
172
|
-
SELECT colname, typename, nulls
|
|
173
|
-
FROM syscat.columns
|
|
174
|
-
WHERE tabschema = UPPER('<schema>')
|
|
175
|
-
AND tabname = UPPER('<table_name>')
|
|
176
|
-
ORDER BY colno;
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
## H2
|
|
180
|
-
|
|
181
|
-
H2 常見於 tests 或 local development 中以 embedded 方式使用。優先使用 repo 的 test profile 或 app 提供的 console/tooling。
|
|
182
|
-
|
|
183
|
-
常見的 JDBC URL patterns:
|
|
184
|
-
|
|
185
|
-
```text
|
|
186
|
-
jdbc:h2:mem:<database>
|
|
187
|
-
jdbc:h2:file:<path>
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
實用的 inspection query:
|
|
191
|
-
|
|
192
|
-
```sql
|
|
193
|
-
SELECT TABLE_SCHEMA, TABLE_NAME
|
|
194
|
-
FROM INFORMATION_SCHEMA.TABLES
|
|
195
|
-
ORDER BY TABLE_SCHEMA, TABLE_NAME
|
|
196
|
-
LIMIT 50;
|
|
197
|
-
```
|
|
@@ -1,109 +0,0 @@
|
|
|
1
|
-
# Query Safety
|
|
2
|
-
|
|
3
|
-
## Read-Only Default
|
|
4
|
-
|
|
5
|
-
除非 user 明確要求資料修改,否則預設只允許 metadata 與 read-only `SELECT`。如果查詢內容涉及下列操作,必須先停下來確認:
|
|
6
|
-
|
|
7
|
-
- `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `TRUNCATE`
|
|
8
|
-
- `CREATE`, `ALTER`, `DROP`
|
|
9
|
-
- `GRANT`, `REVOKE`
|
|
10
|
-
- 可能修改資料的 stored procedure/function/package calls
|
|
11
|
-
- scheduler/job control、queue dequeue、lock/session kill
|
|
12
|
-
|
|
13
|
-
## Row Limits By Dialect
|
|
14
|
-
|
|
15
|
-
請使用有筆數上限的查詢:
|
|
16
|
-
|
|
17
|
-
```sql
|
|
18
|
-
-- PostgreSQL, MySQL, MariaDB, SQLite, H2
|
|
19
|
-
SELECT <columns>
|
|
20
|
-
FROM <table>
|
|
21
|
-
WHERE <condition>
|
|
22
|
-
LIMIT 50;
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
```sql
|
|
26
|
-
-- SQL Server
|
|
27
|
-
SELECT TOP (50) <columns>
|
|
28
|
-
FROM <schema>.<table>
|
|
29
|
-
WHERE <condition>;
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
```sql
|
|
33
|
-
-- Oracle 12c+, IBM Db2
|
|
34
|
-
SELECT <columns>
|
|
35
|
-
FROM <schema>.<table>
|
|
36
|
-
WHERE <condition>
|
|
37
|
-
FETCH FIRST 50 ROWS ONLY;
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
```sql
|
|
41
|
-
-- Older Oracle
|
|
42
|
-
SELECT <columns>
|
|
43
|
-
FROM <schema>.<table>
|
|
44
|
-
WHERE <condition>
|
|
45
|
-
AND ROWNUM <= 50;
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
## Oracle Empty String Semantics
|
|
49
|
-
|
|
50
|
-
這一節只在已確認 `db_type`、driver 或 client session 是 Oracle 時適用。不要把這組規則套用到 PostgreSQL、SQL Server、MySQL、MariaDB、SQLite、Db2 或其他 DB。
|
|
51
|
-
|
|
52
|
-
Oracle 會把 `''` 視為 `NULL`。因此 Oracle predicate 對空字串的處理要直接用 `NULL` 語意,不要把 `''` 當成一般字串比較。
|
|
53
|
-
|
|
54
|
-
- 不要用 `= ''`、`<> ''`,也不要用 `TRIM(NVL(<column>, '')) <> ''` 判斷空值或非空值。
|
|
55
|
-
- 判斷 `NULL` / 非 `NULL` 時,使用 `IS NULL` 或 `IS NOT NULL`。
|
|
56
|
-
- 判斷 trim 後是否有內容時,優先使用 `LENGTH(TRIM(NVL(<column>, ' '))) = 0` 或 `> 0`;若只需要確認 trim 後有值,可用 `TRIM(<column>) IS NOT NULL`。
|
|
57
|
-
- 若 query 結果意外為 0 rows,先檢查是否誤用了 Oracle empty-string 語意,再解讀資料是否真的為空。
|
|
58
|
-
|
|
59
|
-
範例:
|
|
60
|
-
|
|
61
|
-
```sql
|
|
62
|
-
-- Avoid in Oracle
|
|
63
|
-
WHERE path_program <> ''
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
```sql
|
|
67
|
-
-- Avoid in Oracle even after wrapping with TRIM/NVL
|
|
68
|
-
WHERE TRIM(NVL(path_program, '')) <> ''
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
```sql
|
|
72
|
-
-- Prefer when checking for the presence of a trimmed value
|
|
73
|
-
WHERE TRIM(path_program) IS NOT NULL
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
```sql
|
|
77
|
-
-- Prefer when checking for a non-empty value after trimming
|
|
78
|
-
WHERE LENGTH(TRIM(NVL(path_program, ' '))) > 0
|
|
79
|
-
```
|
|
80
|
-
|
|
81
|
-
## Safer Investigation Pattern
|
|
82
|
-
|
|
83
|
-
1. 先透過 metadata 確認 schema/table 存在。
|
|
84
|
-
2. 只有在安全的前提下,才檢查大致筆數或精確 count。
|
|
85
|
-
3. 先查 grouped summary,再看 row sample。
|
|
86
|
-
4. 以關鍵欄位查詢範圍較小的 sample。
|
|
87
|
-
5. 依照 user request 加上業務過濾條件。
|
|
88
|
-
|
|
89
|
-
範例:
|
|
90
|
-
|
|
91
|
-
```sql
|
|
92
|
-
SELECT status, COUNT(*) AS row_count
|
|
93
|
-
FROM <schema>.<table>
|
|
94
|
-
WHERE created_at >= <start_date>
|
|
95
|
-
GROUP BY status
|
|
96
|
-
ORDER BY row_count DESC;
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
## Reporting
|
|
100
|
-
|
|
101
|
-
最終回覆應提供足夠證據,讓結果可被重現,同時避免洩露敏感資訊:
|
|
102
|
-
|
|
103
|
-
- connection source:`src/main/resources/application-dev.yml`、`.env`、`docker-compose.yml`、user-provided session value
|
|
104
|
-
- DB type 與 target environment
|
|
105
|
-
- SQL statements;如果使用了敏感 literal,則改提供摘要化 SQL
|
|
106
|
-
- result rows 或 aggregate summary
|
|
107
|
-
- 不確定之處與 permission gaps
|
|
108
|
-
|
|
109
|
-
不可包含 plaintext password values、token values、帶有憑證資訊的 URL,或任何 private network secrets。
|
|
@@ -1,273 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env python3
|
|
2
|
-
import sys
|
|
3
|
-
|
|
4
|
-
sys.dont_write_bytecode = True
|
|
5
|
-
|
|
6
|
-
import argparse
|
|
7
|
-
import json
|
|
8
|
-
import os
|
|
9
|
-
import re
|
|
10
|
-
from pathlib import Path
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
SKIP_DIRS = {
|
|
14
|
-
".git",
|
|
15
|
-
".hg",
|
|
16
|
-
".svn",
|
|
17
|
-
".idea",
|
|
18
|
-
".vscode",
|
|
19
|
-
"__pycache__",
|
|
20
|
-
"node_modules",
|
|
21
|
-
"target",
|
|
22
|
-
"build",
|
|
23
|
-
"dist",
|
|
24
|
-
"out",
|
|
25
|
-
".gradle",
|
|
26
|
-
".mvn",
|
|
27
|
-
"vendor",
|
|
28
|
-
}
|
|
29
|
-
|
|
30
|
-
SCRIPT_SKILL_DIR = Path(__file__).resolve().parents[1]
|
|
31
|
-
|
|
32
|
-
TEXT_EXTENSIONS = {
|
|
33
|
-
".properties",
|
|
34
|
-
".yml",
|
|
35
|
-
".yaml",
|
|
36
|
-
".xml",
|
|
37
|
-
".json",
|
|
38
|
-
".env",
|
|
39
|
-
".ini",
|
|
40
|
-
".conf",
|
|
41
|
-
".cfg",
|
|
42
|
-
".toml",
|
|
43
|
-
".sql",
|
|
44
|
-
".java",
|
|
45
|
-
".kt",
|
|
46
|
-
".groovy",
|
|
47
|
-
".cs",
|
|
48
|
-
".js",
|
|
49
|
-
".ts",
|
|
50
|
-
".py",
|
|
51
|
-
".rb",
|
|
52
|
-
".php",
|
|
53
|
-
".sh",
|
|
54
|
-
".ps1",
|
|
55
|
-
".bat",
|
|
56
|
-
".cmd",
|
|
57
|
-
".txt",
|
|
58
|
-
}
|
|
59
|
-
|
|
60
|
-
CONFIG_EXTENSIONS = {
|
|
61
|
-
".properties",
|
|
62
|
-
".yml",
|
|
63
|
-
".yaml",
|
|
64
|
-
".xml",
|
|
65
|
-
".json",
|
|
66
|
-
".env",
|
|
67
|
-
".ini",
|
|
68
|
-
".conf",
|
|
69
|
-
".cfg",
|
|
70
|
-
".toml",
|
|
71
|
-
}
|
|
72
|
-
|
|
73
|
-
TEXT_NAMES = {
|
|
74
|
-
"Dockerfile",
|
|
75
|
-
"Jenkinsfile",
|
|
76
|
-
"docker-compose.yml",
|
|
77
|
-
"docker-compose.yaml",
|
|
78
|
-
"pom.xml",
|
|
79
|
-
"build.gradle",
|
|
80
|
-
"build.gradle.kts",
|
|
81
|
-
"package.json",
|
|
82
|
-
"requirements.txt",
|
|
83
|
-
"persistence.xml",
|
|
84
|
-
"context.xml",
|
|
85
|
-
"tnsnames.ora",
|
|
86
|
-
"pg_service.conf",
|
|
87
|
-
}
|
|
88
|
-
|
|
89
|
-
CONFIG_NAMES = TEXT_NAMES | {
|
|
90
|
-
".env",
|
|
91
|
-
".env.local",
|
|
92
|
-
".env.dev",
|
|
93
|
-
".env.test",
|
|
94
|
-
".env.prod",
|
|
95
|
-
"application.properties",
|
|
96
|
-
"application.yml",
|
|
97
|
-
"application.yaml",
|
|
98
|
-
}
|
|
99
|
-
|
|
100
|
-
DB_PATTERNS = [
|
|
101
|
-
("oracle", re.compile(r"jdbc:oracle:thin|oracle\.jdbc|tnsnames\.ora|SERVICE_NAME|ORACLE_SID", re.I)),
|
|
102
|
-
("postgresql", re.compile(r"jdbc:postgresql|org\.postgresql|PGHOST|PGDATABASE|postgres(?:ql)?://", re.I)),
|
|
103
|
-
("sqlserver", re.compile(r"jdbc:sqlserver|com\.microsoft\.sqlserver|sqlcmd|Data Source=|Initial Catalog=|Microsoft SQL Server|MS SQL|MSSQL", re.I)),
|
|
104
|
-
("mysql", re.compile(r"jdbc:mysql|com\.mysql|mysql://|MYSQL_", re.I)),
|
|
105
|
-
("mariadb", re.compile(r"jdbc:mariadb|org\.mariadb|mariadb://|MARIADB_", re.I)),
|
|
106
|
-
("db2", re.compile(r"jdbc:db2|com\.ibm\.db2|DB2", re.I)),
|
|
107
|
-
("sqlite", re.compile(r"jdbc:sqlite|sqlite3?|\.db\b|\.sqlite\b", re.I)),
|
|
108
|
-
("h2", re.compile(r"jdbc:h2|org\.h2|H2_", re.I)),
|
|
109
|
-
]
|
|
110
|
-
|
|
111
|
-
CONFIG_KEY_PATTERN = re.compile(
|
|
112
|
-
r"(?i)("
|
|
113
|
-
r"spring\.datasource\.[\w.-]+|"
|
|
114
|
-
r"datasource\.[\w.-]+|"
|
|
115
|
-
r"r2dbc\.[\w.-]+|"
|
|
116
|
-
r"hibernate\.connection\.[\w.-]+|"
|
|
117
|
-
r"database[_\-.]?\w*|"
|
|
118
|
-
r"db[_\-.]?(host|port|name|user|username|password|passwd|pwd|url|schema)|"
|
|
119
|
-
r"jdbc[_\-.]?\w*|"
|
|
120
|
-
r"connection(string)?|"
|
|
121
|
-
r"url|username|user|password|passwd|pwd|host|port|schema|service_name|sid|"
|
|
122
|
-
r"PGHOST|PGPORT|PGDATABASE|PGUSER|PGPASSWORD|"
|
|
123
|
-
r"MYSQL_HOST|MYSQL_PORT|MYSQL_DATABASE|MYSQL_USER|MYSQL_PASSWORD|"
|
|
124
|
-
r"MARIADB_HOST|MARIADB_PORT|MARIADB_DATABASE|MARIADB_USER|MARIADB_PASSWORD|"
|
|
125
|
-
r"ORACLE_HOST|ORACLE_PORT|ORACLE_SERVICE|ORACLE_SID|"
|
|
126
|
-
r"SQLSERVER_HOST|SQLSERVER_PORT|SQLSERVER_DATABASE|SQLSERVER_USER|SQLSERVER_PASSWORD"
|
|
127
|
-
r")"
|
|
128
|
-
)
|
|
129
|
-
|
|
130
|
-
NON_CONFIG_HINT_PATTERN = re.compile(
|
|
131
|
-
r"(?i)("
|
|
132
|
-
r"jdbc:[\w:]+|"
|
|
133
|
-
r"postgres(?:ql)?://|mysql://|mariadb://|"
|
|
134
|
-
r"Data Source=|Initial Catalog=|"
|
|
135
|
-
r"PGHOST|PGDATABASE|MYSQL_|MARIADB_|ORACLE_|SQLSERVER_|"
|
|
136
|
-
r"tnsnames\.ora|sqlplus|sqlcmd|sqlite3|db2 connect"
|
|
137
|
-
r")"
|
|
138
|
-
)
|
|
139
|
-
|
|
140
|
-
SECRET_KEY_PATTERN = re.compile(r"(?i)(password|passwd|pwd|secret|token|credential|private[_-]?key|access[_-]?key)")
|
|
141
|
-
USERNAME_KEY_PATTERN = re.compile(r"(?i)(^|[_.-])(user|username|uid)$|(^|[_.-])(user|username|uid)([_.-]|$)")
|
|
142
|
-
ASSIGNMENT_PATTERN = re.compile(r"^\s*([^#;!\s][^:=\s]*?)\s*[:=]\s*(.+?)\s*$")
|
|
143
|
-
MAX_FILE_BYTES = 2 * 1024 * 1024
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
def is_text_candidate(path):
|
|
147
|
-
return path.name in TEXT_NAMES or path.suffix in TEXT_EXTENSIONS or path.name.startswith(".env")
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
def is_config_file(path):
|
|
151
|
-
return path.name in CONFIG_NAMES or path.suffix in CONFIG_EXTENSIONS or path.name.startswith(".env")
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
def detect_db_type(text):
|
|
155
|
-
matches = [name for name, pattern in DB_PATTERNS if pattern.search(text)]
|
|
156
|
-
return ",".join(matches) if matches else ""
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
def redact_value(key, value):
|
|
160
|
-
value = value.strip().strip("\"'")
|
|
161
|
-
if SECRET_KEY_PATTERN.search(key):
|
|
162
|
-
return "<redacted>"
|
|
163
|
-
if USERNAME_KEY_PATTERN.search(key):
|
|
164
|
-
return "<redacted-user>"
|
|
165
|
-
|
|
166
|
-
redacted = value
|
|
167
|
-
redacted = re.sub(r"(?i)(password|passwd|pwd)=([^;&\s]+)", r"\1=<redacted>", redacted)
|
|
168
|
-
redacted = re.sub(r"(?i)(user(?:name)?\s*=\s*)([^;&\s]+)", r"\1<redacted-user>", redacted)
|
|
169
|
-
redacted = re.sub(r"(?i)(uid\s*=\s*)([^;&\s]+)", r"\1<redacted-user>", redacted)
|
|
170
|
-
redacted = re.sub(r"(?i)(User ID\s*=\s*)([^;&]+)", r"\1<redacted-user>", redacted)
|
|
171
|
-
redacted = re.sub(r"(?i)(Password\s*=\s*)([^;&]+)", r"\1<redacted>", redacted)
|
|
172
|
-
redacted = re.sub(r"://([^:/@\s]+):([^@\s]+)@", r"://<redacted-user>:<redacted>@", redacted)
|
|
173
|
-
return redacted
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
def iter_files(root):
|
|
177
|
-
for current_root, dirs, files in os.walk(root):
|
|
178
|
-
dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
|
|
179
|
-
for name in files:
|
|
180
|
-
path = Path(current_root) / name
|
|
181
|
-
try:
|
|
182
|
-
path.resolve().relative_to(SCRIPT_SKILL_DIR)
|
|
183
|
-
continue
|
|
184
|
-
except ValueError:
|
|
185
|
-
pass
|
|
186
|
-
if is_text_candidate(path):
|
|
187
|
-
yield path
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
def scan_file(path, root):
|
|
191
|
-
try:
|
|
192
|
-
if path.stat().st_size > MAX_FILE_BYTES:
|
|
193
|
-
return []
|
|
194
|
-
content = path.read_text(encoding="utf-8", errors="replace")
|
|
195
|
-
except OSError:
|
|
196
|
-
return []
|
|
197
|
-
|
|
198
|
-
results = []
|
|
199
|
-
rel_path = str(path.relative_to(root))
|
|
200
|
-
config_file = is_config_file(path)
|
|
201
|
-
for line_no, line in enumerate(content.splitlines(), start=1):
|
|
202
|
-
db_type = detect_db_type(line)
|
|
203
|
-
key = ""
|
|
204
|
-
value = line.strip()
|
|
205
|
-
match = ASSIGNMENT_PATTERN.match(line)
|
|
206
|
-
|
|
207
|
-
if match:
|
|
208
|
-
key = match.group(1).strip()
|
|
209
|
-
value = match.group(2).strip()
|
|
210
|
-
key_match = CONFIG_KEY_PATTERN.search(key)
|
|
211
|
-
if not db_type and not (config_file and key_match):
|
|
212
|
-
continue
|
|
213
|
-
else:
|
|
214
|
-
hint_match = NON_CONFIG_HINT_PATTERN.search(line)
|
|
215
|
-
if not db_type and not hint_match:
|
|
216
|
-
continue
|
|
217
|
-
if hint_match:
|
|
218
|
-
key = hint_match.group(1)
|
|
219
|
-
|
|
220
|
-
if not db_type:
|
|
221
|
-
db_type = detect_db_type(value)
|
|
222
|
-
|
|
223
|
-
results.append(
|
|
224
|
-
{
|
|
225
|
-
"file": rel_path,
|
|
226
|
-
"line": line_no,
|
|
227
|
-
"db_type": db_type or "unknown",
|
|
228
|
-
"key": key,
|
|
229
|
-
"value": redact_value(key, value),
|
|
230
|
-
}
|
|
231
|
-
)
|
|
232
|
-
return results
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
def print_markdown(results):
|
|
236
|
-
if not results:
|
|
237
|
-
print("No DB config candidates found.")
|
|
238
|
-
return
|
|
239
|
-
|
|
240
|
-
print("| file | line | db_type | key | redacted_value |")
|
|
241
|
-
print("| --- | ---: | --- | --- | --- |")
|
|
242
|
-
for item in results:
|
|
243
|
-
value = item["value"].replace("|", "\\|")
|
|
244
|
-
key = item["key"].replace("|", "\\|")
|
|
245
|
-
print(f"| {item['file']} | {item['line']} | {item['db_type']} | `{key}` | `{value}` |")
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
def main():
|
|
249
|
-
parser = argparse.ArgumentParser(description="Find DB connection config candidates with redacted output.")
|
|
250
|
-
parser.add_argument("repo", nargs="?", default=".", help="Repository path to scan.")
|
|
251
|
-
parser.add_argument("--json", action="store_true", help="Output JSON instead of markdown.")
|
|
252
|
-
args = parser.parse_args()
|
|
253
|
-
|
|
254
|
-
root = Path(args.repo).resolve()
|
|
255
|
-
if not root.exists():
|
|
256
|
-
print(f"Path does not exist: {root}", file=sys.stderr)
|
|
257
|
-
return 2
|
|
258
|
-
|
|
259
|
-
results = []
|
|
260
|
-
for path in iter_files(root):
|
|
261
|
-
results.extend(scan_file(path, root))
|
|
262
|
-
|
|
263
|
-
results.sort(key=lambda item: (item["file"], item["line"]))
|
|
264
|
-
|
|
265
|
-
if args.json:
|
|
266
|
-
print(json.dumps(results, indent=2, ensure_ascii=False))
|
|
267
|
-
else:
|
|
268
|
-
print_markdown(results)
|
|
269
|
-
return 0
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
if __name__ == "__main__":
|
|
273
|
-
raise SystemExit(main())
|