vaultkit 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +974 -726
- data/lib/vkit/cli/base_cli.rb +1 -1
- data/lib/vkit/cli/commands/base_command.rb +2 -0
- data/lib/vkit/cli/commands/policy_bundle_command.rb +44 -21
- data/lib/vkit/cli/commands/policy_deploy_command.rb +20 -1
- metadata +3 -3
data/README.md
CHANGED
|
@@ -1,611 +1,539 @@
|
|
|
1
|
-
# VaultKit
|
|
1
|
+
# VaultKit CLI (`vkit`)
|
|
2
2
|
|
|
3
|
+
[](https://rubygems.org/gems/vaultkitcli)
|
|
3
4
|
[](https://opensource.org/licenses/Apache-2.0)
|
|
4
|
-
[]()
|
|
5
|
-
[]()
|
|
6
|
-
[](https://golang.org)
|
|
7
5
|
[](https://www.ruby-lang.org)
|
|
8
6
|
|
|
9
|
-
>
|
|
7
|
+
> Command-line interface for VaultKit — secure, policy-driven data access control plane
|
|
10
8
|
|
|
11
|
-
|
|
9
|
+
The VaultKit CLI (`vkit`) is the primary interface for interacting with the VaultKit control plane. It enables users to request governed data access, track approvals, manage datasources, and deploy policy bundles—all while maintaining complete audit trails.
|
|
12
10
|
|
|
13
11
|
---
|
|
14
12
|
|
|
15
|
-
##
|
|
13
|
+
## 📋 Table of Contents
|
|
16
14
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
-
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
-
|
|
27
|
-
- **Approval Workflows**: Require explicit approval for accessing sensitive datasets
|
|
28
|
-
- **Audit Logging**: Complete audit trail of every data access request
|
|
29
|
-
- **Metadata Catalog**: Automated dataset classification and schema discovery
|
|
30
|
-
- **Session Tokens**: Short-lived, cryptographically-signed tokens for Zero-Trust security
|
|
31
|
-
- **Schema Governance**: Git-backed policy bundles with drift detection for safe schema evolution
|
|
15
|
+
- [What is VaultKit CLI?](#what-is-vaultkit-cli)
|
|
16
|
+
- [Installation](#installation)
|
|
17
|
+
- [Quick Start](#quick-start)
|
|
18
|
+
- [Authentication](#authentication)
|
|
19
|
+
- [Core Workflows](#core-workflows)
|
|
20
|
+
- [Command Reference](#command-reference)
|
|
21
|
+
- [Configuration](#configuration)
|
|
22
|
+
- [Advanced Usage](#advanced-usage)
|
|
23
|
+
- [Troubleshooting](#troubleshooting)
|
|
24
|
+
- [Contributing](#contributing)
|
|
32
25
|
|
|
33
26
|
---
|
|
34
27
|
|
|
35
|
-
##
|
|
28
|
+
## 🎯 What is VaultKit CLI?
|
|
36
29
|
|
|
37
|
-
|
|
30
|
+
The VaultKit CLI provides command-line access to:
|
|
38
31
|
|
|
39
|
-
|
|
32
|
+
| Capability | Description |
|
|
33
|
+
|------------|-------------|
|
|
34
|
+
| **Authentication** | Login, session management, identity verification |
|
|
35
|
+
| **Data Requests** | Submit AQL queries with automatic policy evaluation |
|
|
36
|
+
| **Approval Workflows** | Approve or deny access requests requiring authorization |
|
|
37
|
+
| **Grant Management** | Fetch data using approved, time-limited grants |
|
|
38
|
+
| **Datasource Admin** | Register and manage database connections (admin only) |
|
|
39
|
+
| **Schema Discovery** | Scan datasources and detect schema drift |
|
|
40
|
+
| **Policy Deployment** | Build, validate, and deploy policy bundles |
|
|
41
|
+
| **Audit Queries** | Search and export audit logs |
|
|
40
42
|
|
|
41
|
-
|
|
42
|
-
- **Multi-Engine Support**: Executes queries against PostgreSQL, MySQL, Snowflake, BigQuery, and more
|
|
43
|
-
- **SQL Masking Engine**: Applies column-level masking at the SQL execution layer
|
|
44
|
-
- **Query Execution**: Handles parameterized queries with injection prevention
|
|
45
|
-
- **Result Sanitization**: Returns masked and filtered results back to VaultKit
|
|
46
|
-
- **JWT Authentication**: Only accepts cryptographically-signed requests from VaultKit
|
|
43
|
+
### Mental Model
|
|
47
44
|
|
|
48
|
-
|
|
45
|
+
VaultKit uses a **request → policy → grant → fetch** workflow:
|
|
49
46
|
|
|
50
|
-
|
|
47
|
+
```
|
|
48
|
+
┌──────────┐ ┌──────────┐ ┌───────┐ ┌───────┐ ┌───────────┐
|
|
49
|
+
│ Request │ -> │ Policy │ -> │ Grant │ -> │ Fetch │ -> │ Audit Log │
|
|
50
|
+
│ (AQL) │ │ Eval │ │ (JWT) │ │ Data │ │ (Record) │
|
|
51
|
+
└──────────┘ └──────────┘ └───────┘ └───────┘ └───────────┘
|
|
52
|
+
```
|
|
51
53
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
cli["CLI Tools"]
|
|
59
|
-
agent["AI Agents"]
|
|
60
|
-
end
|
|
61
|
-
|
|
62
|
-
subgraph vaultkit["🛡️ VaultKit — Control Plane"]
|
|
63
|
-
direction TB
|
|
64
|
-
orch["Request Orchestrator"]
|
|
65
|
-
policy["Policy Engine<br/>(ABAC/RBAC)"]
|
|
66
|
-
auth["Authentication<br/>(JWT/SSO)"]
|
|
67
|
-
registry["Schema Registry"]
|
|
68
|
-
bundle["Active Policy Bundle"]
|
|
69
|
-
conn["Connection Manager"]
|
|
70
|
-
approval["Approval Workflow"]
|
|
71
|
-
audit["Audit Logger"]
|
|
72
|
-
end
|
|
73
|
-
|
|
74
|
-
subgraph funl["⚡ FUNL — Data Plane"]
|
|
75
|
-
direction TB
|
|
76
|
-
translator["AQL Translator"]
|
|
77
|
-
executor["Query Executor"]
|
|
78
|
-
masking["SQL Masking Layer"]
|
|
79
|
-
end
|
|
80
|
-
|
|
81
|
-
subgraph datasources["💾 Data Sources"]
|
|
82
|
-
pg[(PostgreSQL)]
|
|
83
|
-
mysql[(MySQL)]
|
|
84
|
-
snow[(Snowflake)]
|
|
85
|
-
bq[(BigQuery)]
|
|
86
|
-
end
|
|
87
|
-
|
|
88
|
-
subgraph governance["📋 Governance Layer"]
|
|
89
|
-
git["Git Repository<br/>(Policies & Registry)"]
|
|
90
|
-
scan["Schema Scans"]
|
|
91
|
-
cicd["CI/CD Pipeline"]
|
|
92
|
-
end
|
|
93
|
-
|
|
94
|
-
clients -->|AQL Request| orch
|
|
95
|
-
orch --> auth
|
|
96
|
-
orch --> policy
|
|
97
|
-
policy --> bundle
|
|
98
|
-
orch --> registry
|
|
99
|
-
orch --> approval
|
|
100
|
-
orch --> audit
|
|
101
|
-
orch -->|Signed Token + AQL| translator
|
|
102
|
-
|
|
103
|
-
translator --> executor
|
|
104
|
-
executor --> masking
|
|
105
|
-
masking --> datasources
|
|
106
|
-
datasources --> masking
|
|
107
|
-
masking --> executor
|
|
108
|
-
executor --> translator
|
|
109
|
-
translator -->|Masked Results| orch
|
|
110
|
-
orch -->|Response| clients
|
|
111
|
-
|
|
112
|
-
scan -.->|Discovers Schema| datasources
|
|
113
|
-
scan -->|Drift Detection| registry
|
|
114
|
-
git -->|Deploy Bundle| bundle
|
|
115
|
-
cicd -->|Build & Validate| git
|
|
116
|
-
|
|
117
|
-
style vaultkit fill:#e3f2fd
|
|
118
|
-
style funl fill:#fff3e0
|
|
119
|
-
style datasources fill:#f3e5f5
|
|
120
|
-
style governance fill:#f1f8e9
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
### Component Breakdown
|
|
124
|
-
|
|
125
|
-
#### VaultKit (Control Plane)
|
|
126
|
-
|
|
127
|
-
- **Request Orchestrator**: Routes and manages the lifecycle of data access requests
|
|
128
|
-
- **Policy Engine**: Evaluates ABAC/RBAC rules with support for sensitivity levels, regions, and time constraints
|
|
129
|
-
- **Policy Priority System**: `deny` > `require_approval` > `mask` > `allow`
|
|
130
|
-
- **Match Rules**: Dataset-based, field sensitivity, categories, specific field names
|
|
131
|
-
- **Context Awareness**: User role, clearance level, region, environment, time windows
|
|
132
|
-
- **Authentication Service**: Handles user identity via JWT, OAuth, or SSO integration
|
|
133
|
-
- **Schema Registry**: Runtime database storing discovered schemas and serving as drift baseline
|
|
134
|
-
- **Active Policy Bundle**: Immutable, Git-backed artifact containing all enforcement rules
|
|
135
|
-
- **Connection Manager**: Manages datasource credentials (supports VaultKit store, HashiCorp Vault, AWS Secrets Manager)
|
|
136
|
-
- **Approval Workflow**: Implements multi-stage approval for sensitive data access
|
|
137
|
-
- **Audit Logger**: Records every request to pluggable sinks (SQLite, PostgreSQL, S3, etc.)
|
|
138
|
-
|
|
139
|
-
#### FUNL (Data Plane)
|
|
140
|
-
- **AQL Translator**: Parses AQL and generates engine-specific SQL with proper escaping
|
|
141
|
-
- **Query Executor**: Maintains connection pools and executes parameterized queries
|
|
142
|
-
- **SQL Masking Layer**: Injects masking functions into SELECT statements based on policy
|
|
143
|
-
- **Engine Plugins**: Extensible architecture for adding new database engines
|
|
144
|
-
|
|
145
|
-
#### Governance Layer
|
|
146
|
-
- **Git Repository**: Source of truth for all policies and dataset registries
|
|
147
|
-
- **Schema Scans**: Observational tooling that discovers database changes
|
|
148
|
-
- **CI/CD Pipeline**: Automated validation, bundling, and deployment of policy changes
|
|
54
|
+
**Key Principles:**
|
|
55
|
+
- Users **request** data, not queries
|
|
56
|
+
- Policies **decide** what happens (allow, mask, deny, require approval)
|
|
57
|
+
- Grants **authorize** execution with TTLs
|
|
58
|
+
- FUNL **executes** queries safely
|
|
59
|
+
- Everything is **audited**
|
|
149
60
|
|
|
150
61
|
---
|
|
151
62
|
|
|
152
|
-
##
|
|
63
|
+
## 📦 Installation
|
|
153
64
|
|
|
154
|
-
|
|
65
|
+
### Prerequisites
|
|
155
66
|
|
|
156
|
-
|
|
67
|
+
- **Ruby** 3.1 or higher
|
|
68
|
+
- **VaultKit Control Plane** running and accessible
|
|
157
69
|
|
|
158
|
-
|
|
70
|
+
### Option 1: Install from RubyGems (Recommended)
|
|
159
71
|
|
|
160
|
-
|
|
161
|
-
|
|
72
|
+
```bash
|
|
73
|
+
gem install vaultkitcli
|
|
74
|
+
```
|
|
162
75
|
|
|
163
|
-
|
|
76
|
+
### Option 2: Install from Source
|
|
164
77
|
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
E -->|5. Deploy| F[Active Bundle]
|
|
172
|
-
F -->|Enforces| G[Access Control]
|
|
173
|
-
|
|
174
|
-
style A fill:#f3e5f5
|
|
175
|
-
style C fill:#fff3e0
|
|
176
|
-
style D fill:#e8f5e9
|
|
177
|
-
style F fill:#e3f2fd
|
|
178
|
-
style G fill:#fce4ec
|
|
78
|
+
```bash
|
|
79
|
+
git clone https://github.com/yourorg/vaultkit-cli.git
|
|
80
|
+
cd vaultkit-cli
|
|
81
|
+
bundle install
|
|
82
|
+
gem build vaultkitcli.gemspec
|
|
83
|
+
gem install ./vaultkitcli-0.1.0.gem
|
|
179
84
|
```
|
|
180
85
|
|
|
181
|
-
###
|
|
86
|
+
### Verify Installation
|
|
182
87
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
- Compiled policy rules
|
|
187
|
-
- Bundle metadata (version, checksum, source)
|
|
88
|
+
```bash
|
|
89
|
+
vkit --version
|
|
90
|
+
# => VaultKit CLI v0.1.0
|
|
188
91
|
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
2. Discover tables and columns
|
|
192
|
-
3. Classify fields (PII, financial, internal)
|
|
193
|
-
4. Compute diff against runtime registry
|
|
194
|
-
5. Surface changes for review
|
|
92
|
+
vkit --help
|
|
93
|
+
```
|
|
195
94
|
|
|
196
|
-
|
|
95
|
+
---
|
|
197
96
|
|
|
198
|
-
|
|
97
|
+
## ⚡ Quick Start
|
|
199
98
|
|
|
200
|
-
|
|
201
|
-
- No silent expansion of policy scope
|
|
202
|
-
- No auto-granting access to new sensitive data
|
|
203
|
-
- Full Git-based audit trail
|
|
204
|
-
- Compliance with SOC2, ISO 27001, GDPR, HIPAA
|
|
99
|
+
### 1. Set Environment Variables
|
|
205
100
|
|
|
206
|
-
|
|
101
|
+
```bash
|
|
102
|
+
export VKIT_API_URL=http://localhost:3000
|
|
103
|
+
```
|
|
207
104
|
|
|
105
|
+
For production:
|
|
208
106
|
```bash
|
|
209
|
-
|
|
210
|
-
|
|
107
|
+
export VKIT_API_URL=https://vaultkit.company.com
|
|
108
|
+
```
|
|
211
109
|
|
|
212
|
-
|
|
213
|
-
# + dataset: customers
|
|
214
|
-
# + field: ssn (string) [PII] ⚠️ NEW - requires policy
|
|
215
|
-
# ~ field: email (text → varchar) [PII] ✓ Already masked
|
|
110
|
+
### 2. Authenticate
|
|
216
111
|
|
|
217
|
-
|
|
218
|
-
vkit
|
|
112
|
+
```bash
|
|
113
|
+
vkit login --endpoint http://localhost:3000 --email analyst@company.com
|
|
114
|
+
```
|
|
219
115
|
|
|
220
|
-
|
|
221
|
-
vim policies/customer_data.yaml
|
|
116
|
+
You'll receive a prompt for your password or be redirected to SSO.
|
|
222
117
|
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
118
|
+
### 3. Submit Your First Request
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
vkit request --aql '{
|
|
122
|
+
"source_table": "customers",
|
|
123
|
+
"columns": ["id", "email", "total_spend"],
|
|
124
|
+
"limit": 10
|
|
125
|
+
}'
|
|
226
126
|
```
|
|
227
127
|
|
|
228
|
-
###
|
|
128
|
+
### 4. Fetch Results
|
|
229
129
|
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
jobs:
|
|
234
|
-
drift-detection:
|
|
235
|
-
runs-on: ubuntu-latest
|
|
236
|
-
steps:
|
|
237
|
-
- name: Scan all datasources
|
|
238
|
-
run: vkit scan --all --fail-on-drift
|
|
239
|
-
|
|
240
|
-
- name: Create PR if drift detected
|
|
241
|
-
if: failure()
|
|
242
|
-
run: |
|
|
243
|
-
vkit scan --all --diff > drift-report.md
|
|
244
|
-
gh pr create --title "Schema Drift Detected" --body-file drift-report.md
|
|
130
|
+
If granted immediately:
|
|
131
|
+
```bash
|
|
132
|
+
vkit fetch --grant g_customers_abc123xyz
|
|
245
133
|
```
|
|
246
134
|
|
|
247
|
-
|
|
135
|
+
If approval required:
|
|
136
|
+
```bash
|
|
137
|
+
# Check status
|
|
138
|
+
vkit requests list --state pending
|
|
139
|
+
|
|
140
|
+
# After approval
|
|
141
|
+
vkit fetch --grant g_customers_abc123xyz
|
|
142
|
+
```
|
|
248
143
|
|
|
249
144
|
---
|
|
250
145
|
|
|
251
|
-
##
|
|
146
|
+
## 🔐 Authentication
|
|
147
|
+
|
|
148
|
+
### `vkit login`
|
|
149
|
+
|
|
150
|
+
Authenticate with the VaultKit control plane.
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
vkit login --endpoint http://localhost:3000 --email analyst@acme.com
|
|
154
|
+
```
|
|
252
155
|
|
|
253
|
-
|
|
156
|
+
**Options:**
|
|
157
|
+
- `--endpoint` — VaultKit API endpoint (can also use `VKIT_API_URL`)
|
|
158
|
+
- `--email` — Email address for authentication
|
|
254
159
|
|
|
255
|
-
|
|
160
|
+
**SSO Integration:**
|
|
256
161
|
|
|
257
|
-
|
|
162
|
+
If your organization uses SSO, you'll be redirected to your identity provider:
|
|
258
163
|
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
"group_by": [],
|
|
267
|
-
"having": [],
|
|
268
|
-
"order_by": null,
|
|
269
|
-
"limit": 0,
|
|
270
|
-
"offset": 0
|
|
271
|
-
}
|
|
164
|
+
```bash
|
|
165
|
+
vkit login --endpoint https://vaultkit.company.com --email you@company.com
|
|
166
|
+
|
|
167
|
+
# Output:
|
|
168
|
+
# Opening browser for SSO authentication...
|
|
169
|
+
# ✓ Authentication successful
|
|
170
|
+
# Session expires: 2024-01-15 18:00:00 UTC
|
|
272
171
|
```
|
|
273
172
|
|
|
274
|
-
###
|
|
173
|
+
### `vkit whoami`
|
|
275
174
|
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
"columns": ["email", "country", "revenue"],
|
|
281
|
-
"filters": [
|
|
282
|
-
{ "field": "country", "operator": "eq", "value": "US" },
|
|
283
|
-
{ "field": "revenue", "operator": "gt", "value": 10000 }
|
|
284
|
-
],
|
|
285
|
-
"limit": 100
|
|
286
|
-
}
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
**FUNL Translation → PostgreSQL:**
|
|
290
|
-
```sql
|
|
291
|
-
SELECT email, country, revenue
|
|
292
|
-
FROM customers
|
|
293
|
-
WHERE country = $1 AND revenue > $2
|
|
294
|
-
LIMIT 100
|
|
295
|
-
```
|
|
296
|
-
|
|
297
|
-
**FUNL Translation → MySQL:**
|
|
298
|
-
```sql
|
|
299
|
-
SELECT `email`, `country`, `revenue`
|
|
300
|
-
FROM `customers`
|
|
301
|
-
WHERE `country` = ? AND `revenue` > ?
|
|
302
|
-
LIMIT 100
|
|
303
|
-
```
|
|
304
|
-
|
|
305
|
-
**Complex Query with JOINs and Aggregates:**
|
|
306
|
-
```json
|
|
307
|
-
{
|
|
308
|
-
"source_table": "users",
|
|
309
|
-
"joins": [
|
|
310
|
-
{
|
|
311
|
-
"type": "LEFT",
|
|
312
|
-
"table": "orders",
|
|
313
|
-
"left_field": "users.id",
|
|
314
|
-
"right_field": "orders.user_id"
|
|
315
|
-
}
|
|
316
|
-
],
|
|
317
|
-
"columns": ["users.email", "users.username"],
|
|
318
|
-
"aggregates": [
|
|
319
|
-
{
|
|
320
|
-
"func": "sum",
|
|
321
|
-
"field": "orders.amount",
|
|
322
|
-
"alias": "total_spent"
|
|
323
|
-
}
|
|
324
|
-
],
|
|
325
|
-
"group_by": ["users.email", "users.username"],
|
|
326
|
-
"having": [
|
|
327
|
-
{
|
|
328
|
-
"operator": "gt",
|
|
329
|
-
"field": "SUM(orders.amount)",
|
|
330
|
-
"value": 500
|
|
331
|
-
}
|
|
332
|
-
],
|
|
333
|
-
"order_by": {
|
|
334
|
-
"column": "total_spent",
|
|
335
|
-
"direction": "DESC"
|
|
336
|
-
},
|
|
337
|
-
"limit": 10
|
|
338
|
-
}
|
|
175
|
+
Display the currently authenticated user and session information.
|
|
176
|
+
|
|
177
|
+
```bash
|
|
178
|
+
vkit whoami
|
|
339
179
|
```
|
|
340
180
|
|
|
341
|
-
**
|
|
342
|
-
```
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
LIMIT 10
|
|
181
|
+
**Output:**
|
|
182
|
+
```
|
|
183
|
+
User ID: user_12345
|
|
184
|
+
Email: analyst@company.com
|
|
185
|
+
Role: analyst
|
|
186
|
+
Clearance Level: 2 (high)
|
|
187
|
+
Region: US
|
|
188
|
+
Session Expires: 2024-01-15 18:00:00 UTC
|
|
350
189
|
```
|
|
351
190
|
|
|
352
|
-
###
|
|
191
|
+
### `vkit logout`
|
|
192
|
+
|
|
193
|
+
Clear stored credentials and end session.
|
|
353
194
|
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
{
|
|
357
|
-
"source_table": "users",
|
|
358
|
-
"columns": ["id", "email", "role"],
|
|
359
|
-
"filters": [
|
|
360
|
-
{
|
|
361
|
-
"logic": "OR",
|
|
362
|
-
"conditions": [
|
|
363
|
-
{ "field": "users.role", "operator": "eq", "value": "admin" },
|
|
364
|
-
{ "field": "users.role", "operator": "eq", "value": "manager" }
|
|
365
|
-
]
|
|
366
|
-
},
|
|
367
|
-
{ "field": "orders.amount", "operator": "gt", "value": 100 }
|
|
368
|
-
]
|
|
369
|
-
}
|
|
195
|
+
```bash
|
|
196
|
+
vkit logout
|
|
370
197
|
```
|
|
371
198
|
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
199
|
+
---
|
|
200
|
+
|
|
201
|
+
## 🔄 Core Workflows
|
|
202
|
+
|
|
203
|
+
### Workflow 1: Immediate Access (Auto-Approved)
|
|
204
|
+
|
|
205
|
+
```bash
|
|
206
|
+
# 1. Submit request
|
|
207
|
+
vkit request --aql '{
|
|
208
|
+
"source_table": "public_reports",
|
|
209
|
+
"columns": ["report_id", "title", "published_date"],
|
|
210
|
+
"limit": 10
|
|
211
|
+
}'
|
|
212
|
+
|
|
213
|
+
# Output:
|
|
214
|
+
# ✓ Request granted immediately
|
|
215
|
+
# Grant ID: g_customers_abc123xyz
|
|
216
|
+
# TTL: 4 hours
|
|
217
|
+
# Use: vkit fetch --grant g_customers_abc123xyz
|
|
218
|
+
|
|
219
|
+
# 2. Fetch data
|
|
220
|
+
vkit fetch --grant g_customers_abc123xyz --format table
|
|
377
221
|
```
|
|
378
222
|
|
|
379
|
-
|
|
380
|
-
- Comparison: `eq`, `neq`, `gt`, `lt`, `gte`, `lte`
|
|
381
|
-
- Pattern matching: `like`
|
|
382
|
-
- Set operations: `in`
|
|
383
|
-
- Null checks: `is_null`, `is_not_null`
|
|
223
|
+
### Workflow 2: Approval Required
|
|
384
224
|
|
|
385
|
-
|
|
386
|
-
|
|
225
|
+
```bash
|
|
226
|
+
# 1. Submit request
|
|
227
|
+
vkit request --aql '{
|
|
228
|
+
"source_table": "financial_transactions",
|
|
229
|
+
"columns": ["transaction_id", "amount", "customer_id"],
|
|
230
|
+
"filters": [{"field": "amount", "operator": "gt", "value": 10000}]
|
|
231
|
+
}'
|
|
387
232
|
|
|
388
|
-
|
|
389
|
-
|
|
233
|
+
# Output:
|
|
234
|
+
# ⏳ Approval required
|
|
235
|
+
# Request ID: req_20240115_xyz789
|
|
236
|
+
# Approver: finance_manager
|
|
237
|
+
# Reason: Financial data requires manager approval
|
|
238
|
+
# Status: PENDING
|
|
390
239
|
|
|
391
|
-
|
|
240
|
+
# 2. Check status
|
|
241
|
+
vkit requests list --state pending
|
|
392
242
|
|
|
393
|
-
|
|
243
|
+
# 3. After approval, fetch data
|
|
244
|
+
vkit fetch --grant g_customers_approved_abc123
|
|
245
|
+
```
|
|
394
246
|
|
|
395
|
-
|
|
396
|
-
|-------------|------------|-------|-----------|
|
|
397
|
-
| **Full** | `'*****' AS email` | `'*****' AS email` | `'*****' AS email` |
|
|
398
|
-
| **Partial** | `CONCAT(LEFT(email, 3), '****')` | `CONCAT(LEFT(email, 3), '****')` | `CONCAT(LEFT(email, 3), '****')` |
|
|
399
|
-
| **Hash** | `ENCODE(SHA256(email::bytea), 'hex')` | `SHA2(email, 256)` | `SHA2(email, 256)` |
|
|
247
|
+
### Workflow 3: Denied Access
|
|
400
248
|
|
|
401
|
-
|
|
249
|
+
```bash
|
|
250
|
+
vkit request --aql '{
|
|
251
|
+
"source_table": "payment_cards",
|
|
252
|
+
"columns": ["card_number", "cvv"],
|
|
253
|
+
"limit": 10
|
|
254
|
+
}'
|
|
402
255
|
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
| **Policy Engine** | Attribute-based access control with support for clearance levels, regions, sensitivity tags, and time windows |
|
|
411
|
-
| **Policy Priority System** | Hierarchical decision making: `deny` > `require_approval` > `mask` > `allow` |
|
|
412
|
-
| **Field-Level Policies** | Match rules based on dataset name, field sensitivity, categories (pii, financial, etc.), or specific field names |
|
|
413
|
-
| **Context-Aware Rules** | Policies evaluate requester role, clearance level, region, environment, and time constraints |
|
|
414
|
-
| **Approval Workflows** | Require manager/security approval for accessing PII, financial data, or production environments |
|
|
415
|
-
| **Zero-Trust Sessions** | Short-lived, cryptographically-signed tokens with automatic expiration |
|
|
416
|
-
| **Credential Abstraction** | Never expose database credentials to users—supports multiple secret backends |
|
|
417
|
-
| **Git-Backed Governance** | All policies and registries versioned in Git with full audit trail |
|
|
418
|
-
| **Schema Drift Detection** | Automated discovery of database changes with manual approval workflow |
|
|
419
|
-
| **Comprehensive Auditing** | Every query logged with user identity, timestamp, policy decisions, and results metadata |
|
|
420
|
-
| **CLI & SDK** | Rich command-line tools (`vkit`) and programmatic SDK for integration |
|
|
421
|
-
|
|
422
|
-
### FUNL (Data Plane)
|
|
423
|
-
|
|
424
|
-
| Feature | Description |
|
|
425
|
-
|---------|-------------|
|
|
426
|
-
| **Multi-Engine Translation** | Supports PostgreSQL, MySQL, Snowflake, BigQuery with identical AQL interface |
|
|
427
|
-
| **SQL-Level Masking** | Masking applied during query execution—no post-processing overhead |
|
|
428
|
-
| **Injection Prevention** | All queries use parameterized execution with proper type binding |
|
|
429
|
-
| **Raw SQL Support** | Supports raw SQL for schema introspection and admin operations (policy-controlled) |
|
|
430
|
-
| **Horizontal Scaling** | Stateless design enables running multiple FUNL instances behind a load balancer |
|
|
431
|
-
| **JWT Verification** | Only executes queries signed by VaultKit's private key |
|
|
256
|
+
# Output:
|
|
257
|
+
# ❌ Request denied
|
|
258
|
+
# Reason: Direct payment card data access forbidden
|
|
259
|
+
# Policy: pci_dss_restrictions
|
|
260
|
+
# Alternative: Use tokenized_card_id field instead
|
|
261
|
+
# Audit ID: audit_20240115_denied_123
|
|
262
|
+
```
|
|
432
263
|
|
|
433
264
|
---
|
|
434
265
|
|
|
435
|
-
##
|
|
436
|
-
|
|
437
|
-
### Prerequisites Checklist
|
|
266
|
+
## 📚 Command Reference
|
|
438
267
|
|
|
439
|
-
|
|
268
|
+
### Authentication Commands
|
|
440
269
|
|
|
441
|
-
|
|
442
|
-
- [ ] **Go** 1.21+ installed (`go version`)
|
|
443
|
-
- [ ] **OpenSSL** for key generation
|
|
444
|
-
- [ ] **Git** for policy management
|
|
445
|
-
- [ ] Access credentials for at least one supported datasource
|
|
446
|
-
- [ ] (Optional) **Docker** for containerized FUNL deployment
|
|
270
|
+
#### `vkit login`
|
|
447
271
|
|
|
448
|
-
|
|
272
|
+
Authenticate with VaultKit.
|
|
449
273
|
|
|
450
|
-
|
|
274
|
+
```bash
|
|
275
|
+
vkit login --endpoint <URL> --email <EMAIL>
|
|
276
|
+
```
|
|
451
277
|
|
|
452
|
-
|
|
278
|
+
**Options:**
|
|
279
|
+
- `--endpoint, -e` — VaultKit API endpoint (default: `$VKIT_API_URL`)
|
|
280
|
+
- `--email` — User email address
|
|
281
|
+
- `--password` — Password (prompted if not provided)
|
|
453
282
|
|
|
283
|
+
**Examples:**
|
|
454
284
|
```bash
|
|
455
|
-
#
|
|
456
|
-
|
|
457
|
-
openssl rsa -pubout -in vaultkit_private.pem -out vaultkit_public.pem
|
|
285
|
+
# Interactive login
|
|
286
|
+
vkit login --endpoint http://localhost:3000 --email analyst@company.com
|
|
458
287
|
|
|
459
|
-
#
|
|
460
|
-
|
|
461
|
-
export VKIT_PUBLIC_KEY="$(pwd)/vaultkit_public.pem"
|
|
462
|
-
export FUNL_PUBLIC_KEY="$(pwd)/vaultkit_public.pem"
|
|
288
|
+
# Non-interactive (CI/CD)
|
|
289
|
+
vkit login --endpoint https://vaultkit.company.com --email bot@company.com --password $BOT_PASSWORD
|
|
463
290
|
```
|
|
464
291
|
|
|
465
|
-
|
|
292
|
+
#### `vkit whoami`
|
|
466
293
|
|
|
467
|
-
|
|
294
|
+
Display current user information.
|
|
468
295
|
|
|
469
296
|
```bash
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
docker compose up -d
|
|
297
|
+
vkit whoami [--format json|table]
|
|
298
|
+
```
|
|
473
299
|
|
|
474
|
-
|
|
475
|
-
|
|
300
|
+
**Examples:**
|
|
301
|
+
```bash
|
|
302
|
+
vkit whoami
|
|
303
|
+
vkit whoami --format json
|
|
476
304
|
```
|
|
477
305
|
|
|
478
|
-
####
|
|
306
|
+
#### `vkit logout`
|
|
307
|
+
|
|
308
|
+
End session and clear credentials.
|
|
479
309
|
|
|
480
310
|
```bash
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
gem build vaultkitcli.gemspec
|
|
484
|
-
gem install ./vaultkitcli-0.1.0.gem
|
|
311
|
+
vkit logout
|
|
312
|
+
```
|
|
485
313
|
|
|
486
|
-
|
|
487
|
-
|
|
314
|
+
---
|
|
315
|
+
|
|
316
|
+
### Data Request Commands
|
|
317
|
+
|
|
318
|
+
#### `vkit request`
|
|
319
|
+
|
|
320
|
+
Submit a governed data request using AQL.
|
|
321
|
+
|
|
322
|
+
```bash
|
|
323
|
+
vkit request --aql <AQL_JSON> [OPTIONS]
|
|
488
324
|
```
|
|
489
325
|
|
|
490
|
-
|
|
326
|
+
**Options:**
|
|
327
|
+
- `--aql` — AQL JSON payload (inline or via STDIN)
|
|
328
|
+
- `--env` — Environment (default: `production`)
|
|
329
|
+
- `--requester_region` — Your geographic region (e.g., `US`, `EU`)
|
|
330
|
+
- `--dataset_region` — Dataset's geographic region
|
|
331
|
+
- `--requester_clearance` — Clearance level: `low`, `high`, `admin`
|
|
332
|
+
- `--format` — Output format: `json`, `table` (default: `table`)
|
|
491
333
|
|
|
334
|
+
**Inline AQL:**
|
|
492
335
|
```bash
|
|
493
|
-
|
|
494
|
-
|
|
336
|
+
vkit request \
|
|
337
|
+
--aql '{"source_table":"customers","columns":["email","total_spend"],"limit":100}' \
|
|
338
|
+
--requester_region EU \
|
|
339
|
+
--dataset_region EU \
|
|
340
|
+
--requester_clearance high
|
|
341
|
+
```
|
|
495
342
|
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
"host": "localhost",
|
|
504
|
-
"port": 5432,
|
|
505
|
-
"database": "analytics"
|
|
506
|
-
}'
|
|
343
|
+
**AQL via STDIN:**
|
|
344
|
+
```bash
|
|
345
|
+
cat query.json | vkit request
|
|
346
|
+
|
|
347
|
+
# Or
|
|
348
|
+
echo '{"source_table":"orders","columns":["order_id","total"]}' | vkit request
|
|
349
|
+
```
|
|
507
350
|
|
|
508
|
-
|
|
509
|
-
|
|
351
|
+
**AQL from File:**
|
|
352
|
+
```bash
|
|
353
|
+
vkit request --aql "$(cat queries/customer_analysis.json)"
|
|
510
354
|
```
|
|
511
355
|
|
|
512
|
-
|
|
356
|
+
**Possible Outcomes:**
|
|
357
|
+
|
|
358
|
+
| Outcome | Description | Next Step |
|
|
359
|
+
|---------|-------------|-----------|
|
|
360
|
+
| ✅ **Granted** | Immediate access | `vkit fetch --grant <GRANT_ID>` |
|
|
361
|
+
| ⏳ **Queued** | Requires approval | Wait for approval, check with `vkit requests list` |
|
|
362
|
+
| ❌ **Denied** | Policy disallows | Review policy restrictions |
|
|
363
|
+
|
|
364
|
+
#### `vkit requests list`
|
|
365
|
+
|
|
366
|
+
View your request history.
|
|
513
367
|
|
|
514
368
|
```bash
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
369
|
+
vkit requests list [--state STATE] [--format FORMAT]
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
**Options:**
|
|
373
|
+
- `--state` — Filter by state: `all`, `pending`, `approved`, `denied` (default: `all`)
|
|
374
|
+
- `--format` — Output format: `json`, `table` (default: `table`)
|
|
375
|
+
- `--limit` — Number of results (default: 50)
|
|
376
|
+
|
|
377
|
+
**Examples:**
|
|
378
|
+
```bash
|
|
379
|
+
# All requests
|
|
380
|
+
vkit requests list
|
|
381
|
+
|
|
382
|
+
# Only pending requests
|
|
383
|
+
vkit requests list --state pending
|
|
384
|
+
|
|
385
|
+
# Approved requests in JSON
|
|
386
|
+
vkit requests list --state approved --format json
|
|
521
387
|
|
|
522
|
-
#
|
|
523
|
-
vkit
|
|
388
|
+
# Last 100 requests
|
|
389
|
+
vkit requests list --limit 100
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
#### `vkit requests show`
|
|
393
|
+
|
|
394
|
+
Show details for a specific request.
|
|
395
|
+
|
|
396
|
+
```bash
|
|
397
|
+
vkit requests show <REQUEST_ID>
|
|
524
398
|
```
|
|
525
399
|
|
|
526
|
-
|
|
400
|
+
**Example:**
|
|
401
|
+
```bash
|
|
402
|
+
vkit requests show req_20240115_xyz789
|
|
403
|
+
```
|
|
527
404
|
|
|
528
405
|
---
|
|
529
406
|
|
|
530
|
-
###
|
|
407
|
+
### Approval Commands
|
|
531
408
|
|
|
532
|
-
####
|
|
409
|
+
#### `vkit approval:list`
|
|
533
410
|
|
|
534
|
-
|
|
411
|
+
List approval requests you are authorized to act on.
|
|
535
412
|
|
|
536
413
|
```bash
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
cd vaultkitcli
|
|
540
|
-
bundle install
|
|
541
|
-
gem build vaultkitcli.gemspec
|
|
542
|
-
gem install ./vaultkitcli-0.1.0.gem
|
|
414
|
+
vkit approval:list [--state STATE] [--format FORMAT]
|
|
415
|
+
```
|
|
543
416
|
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
|
|
417
|
+
**Options:**
|
|
418
|
+
- `--state` — Filter by state: `all`, `pending`, `approved`, `denied` (default: `pending`)
|
|
419
|
+
- `--format` — Output format: `json`, `table` (default: `table`)
|
|
420
|
+
|
|
421
|
+
**Examples:**
|
|
422
|
+
```bash
|
|
423
|
+
# Pending approvals
|
|
424
|
+
vkit approval:list
|
|
425
|
+
|
|
426
|
+
# All approvals you've handled
|
|
427
|
+
vkit approval:list --state all
|
|
428
|
+
|
|
429
|
+
# JSON output for scripting
|
|
430
|
+
vkit approval:list --format json
|
|
549
431
|
```
|
|
550
432
|
|
|
551
|
-
|
|
433
|
+
#### `vkit approval:approve`
|
|
434
|
+
|
|
435
|
+
Approve a pending request.
|
|
552
436
|
|
|
553
437
|
```bash
|
|
554
|
-
|
|
555
|
-
version: '3.8'
|
|
556
|
-
services:
|
|
557
|
-
funl:
|
|
558
|
-
image: vaultkit/funl:0.1.0
|
|
559
|
-
ports:
|
|
560
|
-
- "8080:8080"
|
|
561
|
-
environment:
|
|
562
|
-
- FUNL_PUBLIC_KEY=/keys/vaultkit_public.pem
|
|
563
|
-
volumes:
|
|
564
|
-
- ./vaultkit_public.pem:/keys/vaultkit_public.pem:ro
|
|
565
|
-
healthcheck:
|
|
566
|
-
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
|
|
567
|
-
interval: 10s
|
|
568
|
-
timeout: 3s
|
|
569
|
-
retries: 3
|
|
438
|
+
vkit approval:approve <REQUEST_ID> [--ttl SECONDS] [--notes "..."]
|
|
570
439
|
```
|
|
571
440
|
|
|
572
|
-
|
|
441
|
+
**Options:**
|
|
442
|
+
- `--ttl` — Grant lifetime in seconds (default: 3600)
|
|
443
|
+
- `--notes` — Approval notes (optional)
|
|
573
444
|
|
|
574
|
-
|
|
445
|
+
**Examples:**
|
|
446
|
+
```bash
|
|
447
|
+
# Basic approval
|
|
448
|
+
vkit approval:approve 42
|
|
449
|
+
|
|
450
|
+
# With custom TTL and notes
|
|
451
|
+
vkit approval:approve 42 \
|
|
452
|
+
--ttl 7200 \
|
|
453
|
+
--notes "Approved for Q4 analysis. Limited to aggregate data only."
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
#### `vkit approval:deny`
|
|
457
|
+
|
|
458
|
+
Deny a pending request.
|
|
575
459
|
|
|
576
460
|
```bash
|
|
577
|
-
|
|
578
|
-
funl_url: "http://localhost:8080"
|
|
579
|
-
private_key_path: "/path/to/vaultkit_private.pem"
|
|
580
|
-
public_key_path: "/path/to/vaultkit_public.pem"
|
|
581
|
-
default_datasource: "production_db"
|
|
582
|
-
audit_log_path: "/var/log/vaultkit/audit.log"
|
|
461
|
+
vkit approval:deny <REQUEST_ID> --reason "..."
|
|
583
462
|
```
|
|
584
463
|
|
|
585
|
-
|
|
464
|
+
**Options:**
|
|
465
|
+
- `--reason` — Required explanation for denial
|
|
586
466
|
|
|
467
|
+
**Examples:**
|
|
587
468
|
```bash
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
export FUNL_PUBLIC_KEY="/path/to/vaultkit_public.pem"
|
|
592
|
-
export FUNL_URL="http://localhost:8080"
|
|
593
|
-
export VKIT_CONFIG_PATH="$HOME/.vkit/config.yaml"
|
|
469
|
+
vkit approval:deny 42 --reason "Insufficient business justification"
|
|
470
|
+
|
|
471
|
+
vkit approval:deny 42 --reason "Request violates data minimization policy"
|
|
594
472
|
```
|
|
595
473
|
|
|
596
|
-
|
|
474
|
+
---
|
|
597
475
|
|
|
598
|
-
|
|
476
|
+
### Data Fetch Commands
|
|
477
|
+
|
|
478
|
+
#### `vkit fetch`
|
|
479
|
+
|
|
480
|
+
Fetch data using a previously issued grant.
|
|
599
481
|
|
|
600
|
-
|
|
482
|
+
```bash
|
|
483
|
+
vkit fetch --grant <GRANT_REF> [--format FORMAT]
|
|
484
|
+
```
|
|
485
|
+
|
|
486
|
+
**Options:**
|
|
487
|
+
- `--grant, -g` — Grant reference or ID (required)
|
|
488
|
+
- `--format` — Output format: `json`, `table`, `csv` (default: `table`)
|
|
489
|
+
- `--output, -o` — Write to file instead of STDOUT
|
|
490
|
+
|
|
491
|
+
**Examples:**
|
|
492
|
+
```bash
|
|
493
|
+
# Fetch and display as table
|
|
494
|
+
vkit fetch --grant g_customers_abc123xyz
|
|
495
|
+
|
|
496
|
+
# Fetch as JSON
|
|
497
|
+
vkit fetch --grant g_customers_abc123xyz --format json
|
|
498
|
+
|
|
499
|
+
# Fetch and save to CSV
|
|
500
|
+
vkit fetch --grant g_customers_abc123xyz --format csv --output results.csv
|
|
501
|
+
|
|
502
|
+
# Pipe to jq for processing
|
|
503
|
+
vkit fetch --grant g_customers_abc123xyz --format json | jq '.data[] | select(.revenue > 1000)'
|
|
504
|
+
```
|
|
505
|
+
|
|
506
|
+
---
|
|
507
|
+
|
|
508
|
+
### Datasource Management (Admin Only)
|
|
509
|
+
|
|
510
|
+
#### `vkit datasource add`
|
|
511
|
+
|
|
512
|
+
Register a new datasource.
|
|
513
|
+
|
|
514
|
+
```bash
|
|
515
|
+
vkit datasource add --id <ID> --engine <ENGINE> [OPTIONS]
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
**Options:**
|
|
519
|
+
- `--id` — Unique datasource identifier (required)
|
|
520
|
+
- `--engine` — Database engine: `postgres`, `mysql`, `snowflake`, `bigquery` (required)
|
|
521
|
+
- `--username` — Database username
|
|
522
|
+
- `--password` — Database password
|
|
523
|
+
- `--config` — JSON configuration (host, port, database, etc.)
|
|
524
|
+
- `--credential-backend` — Secret backend: `vaultkit`, `vault`, `aws-secrets`
|
|
525
|
+
|
|
526
|
+
**Examples:**
|
|
527
|
+
|
|
528
|
+
**PostgreSQL:**
|
|
601
529
|
```bash
|
|
602
530
|
vkit datasource add \
|
|
603
|
-
--id
|
|
531
|
+
--id production_pg \
|
|
604
532
|
--engine postgres \
|
|
605
|
-
--username
|
|
533
|
+
--username vaultkit_ro \
|
|
606
534
|
--password $DB_PASSWORD \
|
|
607
535
|
--config '{
|
|
608
|
-
"host": "
|
|
536
|
+
"host": "db.production.internal",
|
|
609
537
|
"port": 5432,
|
|
610
538
|
"database": "analytics",
|
|
611
539
|
"ssl_mode": "require",
|
|
@@ -613,349 +541,669 @@ vkit datasource add \
|
|
|
613
541
|
}'
|
|
614
542
|
```
|
|
615
543
|
|
|
616
|
-
**HashiCorp Vault
|
|
544
|
+
**With HashiCorp Vault:**
|
|
617
545
|
```bash
|
|
618
546
|
vkit datasource add \
|
|
619
|
-
--id
|
|
547
|
+
--id production_pg \
|
|
620
548
|
--engine postgres \
|
|
621
549
|
--credential-backend vault \
|
|
622
550
|
--vault-path secret/data/databases/production \
|
|
623
551
|
--config '{
|
|
624
|
-
"host": "
|
|
552
|
+
"host": "db.production.internal",
|
|
625
553
|
"port": 5432,
|
|
626
554
|
"database": "analytics",
|
|
627
555
|
"ssl_mode": "verify-full"
|
|
628
556
|
}'
|
|
629
557
|
```
|
|
630
558
|
|
|
631
|
-
**
|
|
559
|
+
**Snowflake:**
|
|
632
560
|
```bash
|
|
633
561
|
vkit datasource add \
|
|
634
|
-
--id
|
|
635
|
-
--engine
|
|
636
|
-
--
|
|
637
|
-
--
|
|
562
|
+
--id snowflake_prod \
|
|
563
|
+
--engine snowflake \
|
|
564
|
+
--username analytics_user \
|
|
565
|
+
--password $SNOWFLAKE_PASSWORD \
|
|
638
566
|
--config '{
|
|
639
|
-
"
|
|
640
|
-
"
|
|
641
|
-
"database": "
|
|
567
|
+
"account": "xy12345.us-east-1",
|
|
568
|
+
"warehouse": "ANALYTICS_WH",
|
|
569
|
+
"database": "PRODUCTION",
|
|
570
|
+
"schema": "PUBLIC"
|
|
642
571
|
}'
|
|
643
572
|
```
|
|
644
573
|
|
|
645
|
-
####
|
|
574
|
+
#### `vkit datasource list`
|
|
646
575
|
|
|
647
|
-
|
|
576
|
+
List all registered datasources.
|
|
648
577
|
|
|
649
578
|
```bash
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
579
|
+
vkit datasource list [--format FORMAT]
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
**Examples:**
|
|
583
|
+
```bash
|
|
584
|
+
vkit datasource list
|
|
585
|
+
vkit datasource list --format json
|
|
586
|
+
```
|
|
653
587
|
|
|
654
|
-
|
|
655
|
-
git add .
|
|
656
|
-
git commit -m "Initialize VaultKit policies"
|
|
588
|
+
#### `vkit datasource get`
|
|
657
589
|
|
|
658
|
-
|
|
659
|
-
|
|
590
|
+
Get details for a specific datasource.
|
|
591
|
+
|
|
592
|
+
```bash
|
|
593
|
+
vkit datasource get <DATASOURCE_ID>
|
|
660
594
|
```
|
|
661
595
|
|
|
662
|
-
|
|
596
|
+
**Example:**
|
|
597
|
+
```bash
|
|
598
|
+
vkit datasource get production_pg
|
|
599
|
+
```
|
|
663
600
|
|
|
664
|
-
|
|
665
|
-
id: analyst_basic_access
|
|
666
|
-
description: Basic read access for analysts with PII masking
|
|
667
|
-
priority: 100
|
|
601
|
+
#### `vkit datasource test`
|
|
668
602
|
|
|
669
|
-
|
|
670
|
-
datasets:
|
|
671
|
-
- customers
|
|
672
|
-
- orders
|
|
673
|
-
|
|
674
|
-
context:
|
|
675
|
-
requester_role: analyst
|
|
676
|
-
environment: production
|
|
603
|
+
Test connectivity to a datasource.
|
|
677
604
|
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
action: mask
|
|
681
|
-
mask_type: partial
|
|
682
|
-
reason: "PII must be masked for analysts"
|
|
683
|
-
|
|
684
|
-
- field_category: financial
|
|
685
|
-
action: require_approval
|
|
686
|
-
approver_role: finance_manager
|
|
687
|
-
ttl: "2h"
|
|
688
|
-
reason: "Financial data requires approval"
|
|
689
|
-
|
|
690
|
-
- default:
|
|
691
|
-
action: allow
|
|
692
|
-
ttl: "8h"
|
|
605
|
+
```bash
|
|
606
|
+
vkit datasource test <DATASOURCE_ID>
|
|
693
607
|
```
|
|
694
608
|
|
|
695
|
-
|
|
609
|
+
**Example:**
|
|
610
|
+
```bash
|
|
611
|
+
vkit datasource test production_pg
|
|
612
|
+
|
|
613
|
+
# Output:
|
|
614
|
+
# Testing connection to production_pg...
|
|
615
|
+
# ✓ Connection successful
|
|
616
|
+
# Engine: PostgreSQL 15.3
|
|
617
|
+
# Latency: 45ms
|
|
618
|
+
```
|
|
619
|
+
|
|
620
|
+
#### `vkit datasource remove`
|
|
621
|
+
|
|
622
|
+
Remove a datasource (admin only).
|
|
696
623
|
|
|
697
624
|
```bash
|
|
698
|
-
|
|
699
|
-
|
|
625
|
+
vkit datasource remove <DATASOURCE_ID> [--force]
|
|
626
|
+
```
|
|
700
627
|
|
|
701
|
-
|
|
702
|
-
|
|
628
|
+
**Options:**
|
|
629
|
+
- `--force` — Skip confirmation prompt
|
|
703
630
|
|
|
704
|
-
|
|
705
|
-
|
|
631
|
+
**Example:**
|
|
632
|
+
```bash
|
|
633
|
+
vkit datasource remove old_staging_customers_db --force
|
|
706
634
|
```
|
|
707
635
|
|
|
708
636
|
---
|
|
709
637
|
|
|
710
|
-
|
|
638
|
+
### Schema Discovery Commands
|
|
711
639
|
|
|
712
|
-
|
|
713
|
-
Enable LLM-powered agents to query production databases without exposing credentials. VaultKit enforces policies that restrict which tables and columns AI agents can access, with automatic masking of sensitive fields.
|
|
640
|
+
#### `vkit scan`
|
|
714
641
|
|
|
715
|
-
|
|
716
|
-
- AI agents need data context for decision-making
|
|
717
|
-
- Direct database access creates security risks
|
|
718
|
-
- Credentials in AI systems can leak
|
|
719
|
-
- Uncontrolled queries can expose sensitive data
|
|
642
|
+
Scan a datasource and detect schema drift.
|
|
720
643
|
|
|
721
|
-
|
|
644
|
+
```bash
|
|
645
|
+
vkit scan <DATASOURCE_ID> [--mode MODE]
|
|
646
|
+
```
|
|
722
647
|
|
|
723
|
-
|
|
724
|
-
|
|
725
|
-
|
|
726
|
-
match:
|
|
727
|
-
fields:
|
|
728
|
-
category: pii
|
|
648
|
+
**Options:**
|
|
649
|
+
- `--mode` — Scan mode: `diff`, `apply` (default: `diff`)
|
|
650
|
+
- `--format` — Output format: `json`, `table` (default: `table`)
|
|
729
651
|
|
|
730
|
-
|
|
731
|
-
requester_role: ai_agent
|
|
732
|
-
environment: production
|
|
652
|
+
**Modes:**
|
|
733
653
|
|
|
734
|
-
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
654
|
+
| Mode | Description |
|
|
655
|
+
|------|-------------|
|
|
656
|
+
| `diff` | Show changes without applying (safe, read-only) |
|
|
657
|
+
| `apply` | Update baseline registry with discovered schema |
|
|
658
|
+
|
|
659
|
+
**Examples:**
|
|
660
|
+
|
|
661
|
+
**Discover Changes:**
|
|
662
|
+
```bash
|
|
663
|
+
vkit scan production_pg
|
|
664
|
+
|
|
665
|
+
# Output:
|
|
666
|
+
# Scanning datasource: production_pg
|
|
667
|
+
#
|
|
668
|
+
# Schema Drift Detected:
|
|
669
|
+
#
|
|
670
|
+
# + Dataset: customers
|
|
671
|
+
# + Field: phone_number (varchar) [PII] ⚠️ NEW - requires policy
|
|
672
|
+
# ~ Field: email (text → varchar) [PII] ✓ Already masked
|
|
673
|
+
#
|
|
674
|
+
# + Dataset: user_sessions
|
|
675
|
+
# + Field: ip_address (inet) [PII] ⚠️ NEW - requires policy
|
|
676
|
+
# + Field: user_agent (text) [INTERNAL] ⚠️ NEW - requires policy
|
|
677
|
+
#
|
|
678
|
+
# Summary: 2 new datasets, 3 new fields, 1 modified field
|
|
739
679
|
```
|
|
740
680
|
|
|
741
|
-
**
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
681
|
+
**Apply Changes:**
|
|
682
|
+
```bash
|
|
683
|
+
vkit scan production_pg --mode apply
|
|
684
|
+
|
|
685
|
+
# Output:
|
|
686
|
+
# ✓ Baseline registry updated
|
|
687
|
+
# Next steps:
|
|
688
|
+
# 1. Review new fields: customers.phone_number, user_sessions.ip_address
|
|
689
|
+
# 2. Update policies in Git: config/policies/
|
|
690
|
+
# 3. Build new bundle: vkit policy bundle
|
|
691
|
+
# 4. Deploy bundle: vkit policy deploy
|
|
692
|
+
```
|
|
746
693
|
|
|
747
|
-
**
|
|
694
|
+
**Scan All Datasources:**
|
|
748
695
|
```bash
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
753
|
-
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
696
|
+
for ds in $(vkit datasource list --format json | jq -r '.[].id'); do
|
|
697
|
+
echo "Scanning $ds..."
|
|
698
|
+
vkit scan $ds
|
|
699
|
+
done
|
|
700
|
+
```
|
|
701
|
+
|
|
702
|
+
---
|
|
703
|
+
|
|
704
|
+
### Policy Management Commands
|
|
705
|
+
|
|
706
|
+
#### `vkit policy bundle`
|
|
707
|
+
|
|
708
|
+
Compile policies and registry into a deployable bundle.
|
|
709
|
+
|
|
710
|
+
```bash
|
|
711
|
+
vkit policy bundle [OPTIONS]
|
|
712
|
+
```
|
|
713
|
+
|
|
714
|
+
**Options:**
|
|
715
|
+
- `--policies_dir` — Path to policies directory (default: `config/policies`)
|
|
716
|
+
- `--registry_dir` — Path to registry files (default: `config`)
|
|
717
|
+
- `--datasources_dir` — Path to datasource configs (default: `config/datasources`)
|
|
718
|
+
- `--out` — Output bundle file (default: `dist/policy_bundle.json`)
|
|
719
|
+
- `--org` — Organization identifier
|
|
720
|
+
|
|
721
|
+
**Example:**
|
|
722
|
+
```bash
|
|
723
|
+
vkit policy bundle \
|
|
724
|
+
--policies_dir config/policies \
|
|
725
|
+
--registry_dir config \
|
|
726
|
+
--datasources_dir config/datasources \
|
|
727
|
+
--out dist/policy_bundle.json \
|
|
728
|
+
--org acme
|
|
729
|
+
```
|
|
730
|
+
|
|
731
|
+
#### `vkit policy validate`
|
|
732
|
+
|
|
733
|
+
Validate a policy bundle without deploying.
|
|
734
|
+
|
|
735
|
+
```bash
|
|
736
|
+
vkit policy validate --bundle <BUNDLE_FILE>
|
|
737
|
+
```
|
|
738
|
+
|
|
739
|
+
**Example:**
|
|
740
|
+
```bash
|
|
741
|
+
vkit policy validate --bundle dist/policy_bundle.json
|
|
742
|
+
|
|
743
|
+
# Output:
|
|
744
|
+
# Validating policy bundle...
|
|
745
|
+
# ✓ Schema validation passed
|
|
746
|
+
# ✓ Policy syntax valid
|
|
747
|
+
# ✓ No circular dependencies
|
|
748
|
+
# ✓ All datasources referenced
|
|
749
|
+
# Bundle is valid and ready to deploy
|
|
750
|
+
```
|
|
758
751
|
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
|
|
752
|
+
#### `vkit policy deploy`
|
|
753
|
+
|
|
754
|
+
Deploy a policy bundle to the control plane.
|
|
755
|
+
|
|
756
|
+
```bash
|
|
757
|
+
vkit policy deploy --bundle <BUNDLE_FILE> [OPTIONS]
|
|
758
|
+
```
|
|
759
|
+
|
|
760
|
+
**Options:**
|
|
761
|
+
- `--bundle` — Path to bundle file (required)
|
|
762
|
+
- `--org` — Organization identifier
|
|
763
|
+
- `--activate` — Immediately activate bundle
|
|
764
|
+
- `--dry-run` — Validate deployment without activating
|
|
765
|
+
|
|
766
|
+
**Examples:**
|
|
767
|
+
```bash
|
|
768
|
+
# Deploy and activate
|
|
769
|
+
vkit policy deploy \
|
|
770
|
+
--bundle dist/policy_bundle.json \
|
|
771
|
+
--org acme \
|
|
772
|
+
--activate
|
|
773
|
+
|
|
774
|
+
# Dry run (test deployment)
|
|
775
|
+
vkit policy deploy \
|
|
776
|
+
--bundle dist/policy_bundle.json \
|
|
777
|
+
--org acme \
|
|
778
|
+
--dry-run
|
|
779
|
+
```
|
|
780
|
+
|
|
781
|
+
#### `vkit policy list`
|
|
782
|
+
|
|
783
|
+
List active policy bundles.
|
|
784
|
+
|
|
785
|
+
```bash
|
|
786
|
+
vkit policy list
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
#### `vkit policy show`
|
|
790
|
+
|
|
791
|
+
Show details of a specific policy.
|
|
792
|
+
|
|
793
|
+
```bash
|
|
794
|
+
vkit policy show <POLICY_ID>
|
|
765
795
|
```
|
|
766
796
|
|
|
767
797
|
---
|
|
768
798
|
|
|
769
|
-
###
|
|
770
|
-
Automatically mask or deny access to sensitive fields based on user location and data residency requirements.
|
|
799
|
+
### Audit Commands
|
|
771
800
|
|
|
772
|
-
|
|
773
|
-
- GDPR prohibits transferring EU citizen data outside EU
|
|
774
|
-
- CCPA requires disclosure of data access by California residents
|
|
775
|
-
- HIPAA restricts PHI access based on business need
|
|
776
|
-
- Manual compliance is error-prone and doesn't scale
|
|
801
|
+
#### `vkit audit query`
|
|
777
802
|
|
|
778
|
-
|
|
803
|
+
Search audit logs.
|
|
804
|
+
|
|
805
|
+
```bash
|
|
806
|
+
vkit audit query [OPTIONS]
|
|
807
|
+
```
|
|
808
|
+
|
|
809
|
+
**Options:**
|
|
810
|
+
- `--user` — Filter by user email
|
|
811
|
+
- `--action` — Filter by action: `granted`, `denied`, `pending`
|
|
812
|
+
- `--dataset` — Filter by dataset name
|
|
813
|
+
- `--days` — Look back N days (default: 7)
|
|
814
|
+
- `--limit` — Number of results (default: 100)
|
|
815
|
+
- `--format` — Output format: `json`, `table`, `csv`
|
|
816
|
+
|
|
817
|
+
**Examples:**
|
|
818
|
+
```bash
|
|
819
|
+
# All queries by a user
|
|
820
|
+
vkit audit query --user analyst@company.com --days 30
|
|
821
|
+
|
|
822
|
+
# Denied access attempts
|
|
823
|
+
vkit audit query --action denied --days 7
|
|
824
|
+
|
|
825
|
+
# Access to specific dataset
|
|
826
|
+
vkit audit query --dataset customers --action granted
|
|
827
|
+
|
|
828
|
+
# Export to CSV
|
|
829
|
+
vkit audit query --user analyst@company.com --format csv --output audit_report.csv
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
#### `vkit audit export`
|
|
833
|
+
|
|
834
|
+
Export audit logs for compliance reporting.
|
|
835
|
+
|
|
836
|
+
```bash
|
|
837
|
+
vkit audit export --start-date <DATE> --end-date <DATE> [OPTIONS]
|
|
838
|
+
```
|
|
839
|
+
|
|
840
|
+
**Options:**
|
|
841
|
+
- `--start-date` — Start date (YYYY-MM-DD)
|
|
842
|
+
- `--end-date` — End date (YYYY-MM-DD)
|
|
843
|
+
- `--format` — Output format: `json`, `csv` (default: `csv`)
|
|
844
|
+
- `--output` — Output file path
|
|
845
|
+
|
|
846
|
+
**Example:**
|
|
847
|
+
```bash
|
|
848
|
+
vkit audit export \
|
|
849
|
+
--start-date 2024-01-01 \
|
|
850
|
+
--end-date 2024-01-31 \
|
|
851
|
+
--format csv \
|
|
852
|
+
--output compliance_reports/january_2024.csv
|
|
853
|
+
```
|
|
854
|
+
|
|
855
|
+
---
|
|
856
|
+
|
|
857
|
+
## ⚙️ Configuration
|
|
858
|
+
|
|
859
|
+
### Configuration File
|
|
860
|
+
|
|
861
|
+
Create `~/.vkit/config.yaml` for persistent settings:
|
|
779
862
|
|
|
780
863
|
```yaml
|
|
781
|
-
#
|
|
782
|
-
|
|
783
|
-
|
|
784
|
-
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
864
|
+
# VaultKit Control Plane
|
|
865
|
+
api_url: "https://vaultkit.company.com"
|
|
866
|
+
|
|
867
|
+
# Authentication
|
|
868
|
+
auth:
|
|
869
|
+
method: "sso" # or "password"
|
|
870
|
+
sso_provider: "okta"
|
|
871
|
+
|
|
872
|
+
# Default Settings
|
|
873
|
+
defaults:
|
|
874
|
+
datasource: "production_pg"
|
|
875
|
+
environment: "production"
|
|
876
|
+
requester_region: "US"
|
|
877
|
+
clearance_level: "high"
|
|
878
|
+
|
|
879
|
+
# Output Preferences
|
|
880
|
+
output:
|
|
881
|
+
format: "table" # json, table, csv
|
|
882
|
+
color: true
|
|
883
|
+
|
|
884
|
+
# Audit Logging
|
|
885
|
+
audit:
|
|
886
|
+
local_log: true
|
|
887
|
+
log_customers_path: "~/.vkit/audit.log"
|
|
888
|
+
|
|
889
|
+
# Request Settings
|
|
890
|
+
request:
|
|
891
|
+
default_ttl: 3600 # 1 hour
|
|
892
|
+
auto_retry: true
|
|
893
|
+
retry_attempts: 3
|
|
894
|
+
```
|
|
895
|
+
|
|
896
|
+
### Environment Variables
|
|
897
|
+
|
|
898
|
+
Override config file with environment variables:
|
|
899
|
+
|
|
900
|
+
```bash
|
|
901
|
+
# Required
|
|
902
|
+
export VKIT_API_URL="https://vaultkit.company.com"
|
|
903
|
+
|
|
904
|
+
# Optional
|
|
905
|
+
export VKIT_CONFIg_customers_PATH="~/.vkit/config.yaml"
|
|
906
|
+
export VKIT_AUTH_TOKEN="eyJhbGciOiJIUzI1NiIs..."
|
|
907
|
+
export VKIT_DEFAULT_DATASOURCE="production_pg"
|
|
908
|
+
export VKIT_OUTPUT_FORMAT="json"
|
|
909
|
+
```
|
|
910
|
+
|
|
911
|
+
### Precedence Order
|
|
912
|
+
|
|
913
|
+
1. Command-line flags (highest priority)
|
|
914
|
+
2. Environment variables
|
|
915
|
+
3. Configuration file
|
|
916
|
+
4. Built-in defaults (lowest priority)
|
|
796
917
|
|
|
797
918
|
---
|
|
798
|
-
id: ccpa_disclosure_logging
|
|
799
|
-
match:
|
|
800
|
-
dataset: california_residents
|
|
801
919
|
|
|
802
|
-
|
|
803
|
-
|
|
920
|
+
## 🔬 Advanced Usage
|
|
921
|
+
|
|
922
|
+
### Scripting with vkit
|
|
923
|
+
|
|
924
|
+
**Batch Request Processing:**
|
|
804
925
|
|
|
805
|
-
|
|
806
|
-
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
926
|
+
```bash
|
|
927
|
+
#!/bin/bash
|
|
928
|
+
# process_requests.sh
|
|
929
|
+
|
|
930
|
+
REQUESTS=(
|
|
931
|
+
'{"source_table":"customers","columns":["id","email"],"limit":100}'
|
|
932
|
+
'{"source_table":"orders","columns":["order_id","total"],"limit":100}'
|
|
933
|
+
'{"source_table":"products","columns":["product_id","name"],"limit":100}'
|
|
934
|
+
)
|
|
935
|
+
|
|
936
|
+
for aql in "${REQUESTS[@]}"; do
|
|
937
|
+
echo "Processing: $aql"
|
|
938
|
+
vkit request --aql "$aql" --format json | jq -r '.grant_id' >> grants.txt
|
|
939
|
+
done
|
|
940
|
+
|
|
941
|
+
echo "Generated grants:"
|
|
942
|
+
cat grants.txt
|
|
811
943
|
```
|
|
812
944
|
|
|
813
|
-
**
|
|
814
|
-
- US-based analysts querying EU customer data are automatically denied
|
|
815
|
-
- Policy engine evaluates both requester and dataset regions
|
|
816
|
-
- Audit log captures denied attempts with compliance flags
|
|
817
|
-
- CCPA-flagged queries generate disclosure reports
|
|
945
|
+
**Automated Approvals (for testing):**
|
|
818
946
|
|
|
819
|
-
**Multi-Region Access Pattern:**
|
|
820
947
|
```bash
|
|
821
|
-
|
|
822
|
-
|
|
948
|
+
#!/bin/bash
|
|
949
|
+
# auto_approve_pending.sh
|
|
950
|
+
|
|
951
|
+
PENDING=$(vkit approval:list --state pending --format json)
|
|
823
952
|
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
953
|
+
echo "$PENDING" | jq -r '.[].id' | while read -r request_id; do
|
|
954
|
+
echo "Approving request: $request_id"
|
|
955
|
+
vkit approval:approve "$request_id" --ttl 3600 --notes "Auto-approved for testing"
|
|
956
|
+
done
|
|
957
|
+
```
|
|
958
|
+
|
|
959
|
+
### Integration with CI/CD
|
|
960
|
+
|
|
961
|
+
**GitHub Actions Example:**
|
|
962
|
+
|
|
963
|
+
```yaml
|
|
964
|
+
name: Deploy VaultKit Policies
|
|
965
|
+
|
|
966
|
+
on:
|
|
967
|
+
push:
|
|
968
|
+
branches: [main]
|
|
969
|
+
paths:
|
|
970
|
+
- 'config/policies/**'
|
|
971
|
+
- 'config/registry.yaml'
|
|
972
|
+
|
|
973
|
+
jobs:
|
|
974
|
+
deploy:
|
|
975
|
+
runs-on: ubuntu-latest
|
|
976
|
+
steps:
|
|
977
|
+
- uses: actions/checkout@v3
|
|
978
|
+
|
|
979
|
+
- name: Install vkit CLI
|
|
980
|
+
run: gem install vaultkitcli
|
|
981
|
+
|
|
982
|
+
- name: Authenticate
|
|
983
|
+
run: |
|
|
984
|
+
vkit login \
|
|
985
|
+
--endpoint ${{ secrets.VKIT_API_URL }} \
|
|
986
|
+
--email ${{ secrets.VKIT_BOT_EMAIL }} \
|
|
987
|
+
--password ${{ secrets.VKIT_BOT_PASSWORD }}
|
|
988
|
+
|
|
989
|
+
- name: Build Policy Bundle
|
|
990
|
+
run: |
|
|
991
|
+
vkit policy bundle \
|
|
992
|
+
--policies_dir config/policies \
|
|
993
|
+
--registry_dir config \
|
|
994
|
+
--out policy_bundle.json \
|
|
995
|
+
--org ${{ secrets.ORg_customers_ID }}
|
|
996
|
+
|
|
997
|
+
- name: Validate Bundle
|
|
998
|
+
run: vkit policy validate --bundle policy_bundle.json
|
|
999
|
+
|
|
1000
|
+
- name: Deploy Bundle
|
|
1001
|
+
run: |
|
|
1002
|
+
vkit policy deploy \
|
|
1003
|
+
--bundle policy_bundle.json \
|
|
1004
|
+
--org ${{ secrets.ORg_customers_ID }} \
|
|
1005
|
+
--activate
|
|
1006
|
+
```
|
|
1007
|
+
|
|
1008
|
+
### Using jq for Advanced Filtering
|
|
1009
|
+
|
|
1010
|
+
**Extract specific fields:**
|
|
1011
|
+
```bash
|
|
1012
|
+
vkit requests list --format json | jq '.[] | select(.state == "approved") | {id, dataset, requester}'
|
|
1013
|
+
```
|
|
1014
|
+
|
|
1015
|
+
**Count requests by state:**
|
|
1016
|
+
```bash
|
|
1017
|
+
vkit requests list --format json | jq 'group_by(.state) | map({state: .[0].state, count: length})'
|
|
1018
|
+
```
|
|
1019
|
+
|
|
1020
|
+
**Find high-value transactions:**
|
|
1021
|
+
```bash
|
|
1022
|
+
vkit fetch --grant g_customers_abc123 --format json | jq '.data[] | select(.amount > 10000)'
|
|
829
1023
|
```
|
|
830
1024
|
|
|
831
1025
|
---
|
|
832
1026
|
|
|
833
|
-
|
|
834
|
-
|
|
1027
|
+
## 🐛 Troubleshooting
|
|
1028
|
+
|
|
1029
|
+
### Common Issues
|
|
1030
|
+
|
|
1031
|
+
#### Authentication Failed
|
|
835
1032
|
|
|
836
1033
|
**Problem:**
|
|
837
|
-
|
|
838
|
-
|
|
839
|
-
|
|
840
|
-
- Audit trails are incomplete or manual
|
|
1034
|
+
```
|
|
1035
|
+
Error: Authentication failed (401 Unauthorized)
|
|
1036
|
+
```
|
|
841
1037
|
|
|
842
|
-
**
|
|
1038
|
+
**Solutions:**
|
|
1039
|
+
```bash
|
|
1040
|
+
# 1. Check endpoint
|
|
1041
|
+
echo $VKIT_API_URL
|
|
843
1042
|
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
match:
|
|
848
|
-
fields:
|
|
849
|
-
category: financial
|
|
850
|
-
|
|
851
|
-
context:
|
|
852
|
-
environment: production
|
|
853
|
-
requester_role: engineer
|
|
854
|
-
|
|
855
|
-
action:
|
|
856
|
-
require_approval: true
|
|
857
|
-
approver_role: finance_manager
|
|
858
|
-
approval_metadata_required:
|
|
859
|
-
- incident_ticket
|
|
860
|
-
- business_justification
|
|
861
|
-
ttl: "1h"
|
|
862
|
-
reason: "Financial data access requires approval"
|
|
863
|
-
auto_revoke: true
|
|
864
|
-
```
|
|
865
|
-
|
|
866
|
-
**CLI Usage:**
|
|
867
|
-
```bash
|
|
868
|
-
# Engineer requests emergency access
|
|
869
|
-
vkit request \
|
|
870
|
-
--datasource production_db \
|
|
871
|
-
--approval-required \
|
|
872
|
-
--reason "Investigating payment processing failure - Ticket INC-5432" \
|
|
873
|
-
--metadata '{"incident_ticket": "INC-5432", "business_justification": "Payment processor reporting transaction mismatch"}' \
|
|
874
|
-
--duration "2h" \
|
|
875
|
-
--aql '{
|
|
876
|
-
"source_table": "transactions",
|
|
877
|
-
"columns": ["transaction_id", "amount", "status"],
|
|
878
|
-
"filters": [{"field": "status", "operator": "eq", "value": "failed"}]
|
|
879
|
-
}'
|
|
1043
|
+
# 2. Re-authenticate
|
|
1044
|
+
vkit logout
|
|
1045
|
+
vkit login --endpoint http://localhost:3000 --email you@company.com
|
|
880
1046
|
|
|
881
|
-
#
|
|
882
|
-
|
|
883
|
-
# Request ID: req_20240115_xyz789
|
|
884
|
-
# Status: PENDING
|
|
885
|
-
# Approver: finance_manager
|
|
886
|
-
# Expires: 2024-01-15 18:30 UTC
|
|
1047
|
+
# 3. Verify credentials
|
|
1048
|
+
vkit whoami
|
|
887
1049
|
```
|
|
888
1050
|
|
|
889
|
-
|
|
1051
|
+
#### Connection Refused
|
|
1052
|
+
|
|
1053
|
+
**Problem:**
|
|
1054
|
+
```
|
|
1055
|
+
Error: Connection refused - connect(2) for "localhost" port 3000
|
|
1056
|
+
```
|
|
1057
|
+
|
|
1058
|
+
**Solutions:**
|
|
890
1059
|
```bash
|
|
891
|
-
#
|
|
892
|
-
|
|
1060
|
+
# 1. Check if control plane is running
|
|
1061
|
+
curl http://localhost:3000/health
|
|
893
1062
|
|
|
894
|
-
#
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
--notes "Approved for incident resolution. Access limited to failed transactions only."
|
|
1063
|
+
# 2. Verify Docker containers
|
|
1064
|
+
cd infra
|
|
1065
|
+
docker compose ps
|
|
898
1066
|
|
|
899
|
-
#
|
|
900
|
-
|
|
1067
|
+
# 3. Start services
|
|
1068
|
+
docker compose up
|
|
901
1069
|
```
|
|
902
1070
|
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
"approved_by": "finance.manager@company.com",
|
|
911
|
-
"approved_at": "2024-01-15T16:05:32Z",
|
|
912
|
-
"approval_reason": "Approved for incident resolution",
|
|
913
|
-
"access_granted": "2024-01-15T16:05:32Z",
|
|
914
|
-
"access_expires": "2024-01-15T18:05:32Z",
|
|
915
|
-
"queries_executed": 3,
|
|
916
|
-
"rows_accessed": 147,
|
|
917
|
-
"incident_ticket": "INC-5432",
|
|
918
|
-
"auto_revoked": true
|
|
919
|
-
}
|
|
1071
|
+
#### Grant Expired
|
|
1072
|
+
|
|
1073
|
+
**Problem:**
|
|
1074
|
+
```
|
|
1075
|
+
Error: Grant has expired
|
|
1076
|
+
Grant ID: g_customers_abc123xyz
|
|
1077
|
+
Expired at: 2024-01-15 12:00:00 UTC
|
|
920
1078
|
```
|
|
921
1079
|
|
|
922
|
-
|
|
1080
|
+
**Solutions:**
|
|
1081
|
+
```bash
|
|
1082
|
+
# 1. Request new grant
|
|
1083
|
+
vkit request --aql '{"source_table":"customers",...}'
|
|
1084
|
+
|
|
1085
|
+
# 2. Request longer TTL (if approval required)
|
|
1086
|
+
vkit request --aql '{...}' --ttl 7200
|
|
1087
|
+
```
|
|
923
1088
|
|
|
924
|
-
|
|
925
|
-
Enforce access controls based on business hours, maintenance windows, or compliance requirements.
|
|
1089
|
+
#### Policy Denial
|
|
926
1090
|
|
|
927
1091
|
**Problem:**
|
|
928
|
-
|
|
929
|
-
|
|
930
|
-
|
|
1092
|
+
```
|
|
1093
|
+
❌ Request denied
|
|
1094
|
+
Reason: Cross-region PII access forbidden
|
|
1095
|
+
Policy: gdpr_protection
|
|
1096
|
+
```
|
|
931
1097
|
|
|
932
|
-
**
|
|
1098
|
+
**Solutions:**
|
|
1099
|
+
1. Review policy restrictions
|
|
1100
|
+
2. Request approval if available
|
|
1101
|
+
3. Modify query to exclude restricted fields
|
|
1102
|
+
4. Contact admin to update policies
|
|
933
1103
|
|
|
934
|
-
|
|
935
|
-
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
|
|
941
|
-
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
|
|
951
|
-
|
|
1104
|
+
### Debug Mode
|
|
1105
|
+
|
|
1106
|
+
Enable verbose logging:
|
|
1107
|
+
|
|
1108
|
+
```bash
|
|
1109
|
+
# Set debug environment variable
|
|
1110
|
+
export VKIT_DEBUG=true
|
|
1111
|
+
|
|
1112
|
+
# Run command
|
|
1113
|
+
vkit request --aql '{...}'
|
|
1114
|
+
|
|
1115
|
+
# Output includes:
|
|
1116
|
+
# - HTTP request/response details
|
|
1117
|
+
# - Policy evaluation trace
|
|
1118
|
+
# - Timing information
|
|
1119
|
+
```
|
|
1120
|
+
|
|
1121
|
+
### Health Check
|
|
1122
|
+
|
|
1123
|
+
Verify CLI and control plane connectivity:
|
|
1124
|
+
|
|
1125
|
+
```bash
|
|
1126
|
+
# Check CLI version
|
|
1127
|
+
vkit --version
|
|
1128
|
+
|
|
1129
|
+
# Check control plane health
|
|
1130
|
+
curl $VKIT_API_URL/health
|
|
1131
|
+
|
|
1132
|
+
# Test authentication
|
|
1133
|
+
vkit whoami
|
|
1134
|
+
```
|
|
952
1135
|
|
|
953
1136
|
---
|
|
954
|
-
|
|
955
|
-
|
|
956
|
-
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
960
|
-
|
|
961
|
-
|
|
1137
|
+
|
|
1138
|
+
## 🤝 Contributing
|
|
1139
|
+
|
|
1140
|
+
We welcome contributions to the VaultKit CLI!
|
|
1141
|
+
|
|
1142
|
+
### Development Setup
|
|
1143
|
+
|
|
1144
|
+
```bash
|
|
1145
|
+
# Clone repository
|
|
1146
|
+
git clone https://github.com/yourorg/vaultkit-cli.git
|
|
1147
|
+
cd vaultkit-cli
|
|
1148
|
+
|
|
1149
|
+
# Install dependencies
|
|
1150
|
+
bundle install
|
|
1151
|
+
|
|
1152
|
+
# Run tests
|
|
1153
|
+
bundle exec rspec
|
|
1154
|
+
|
|
1155
|
+
# Run CLI locally
|
|
1156
|
+
bundle exec bin/vkit --help
|
|
1157
|
+
```
|
|
1158
|
+
|
|
1159
|
+
### Running Tests
|
|
1160
|
+
|
|
1161
|
+
```bash
|
|
1162
|
+
# Unit tests
|
|
1163
|
+
bundle exec rspec spec/unit
|
|
1164
|
+
|
|
1165
|
+
# Integration tests (requires running control plane)
|
|
1166
|
+
bundle exec rspec spec/integration
|
|
1167
|
+
|
|
1168
|
+
# All tests
|
|
1169
|
+
bundle exec rspec
|
|
1170
|
+
```
|
|
1171
|
+
|
|
1172
|
+
### Code Style
|
|
1173
|
+
|
|
1174
|
+
```bash
|
|
1175
|
+
# Run RuboCop
|
|
1176
|
+
bundle exec rubocop
|
|
1177
|
+
|
|
1178
|
+
# Auto-fix issues
|
|
1179
|
+
bundle exec rubocop -a
|
|
1180
|
+
```
|
|
1181
|
+
|
|
1182
|
+
---
|
|
1183
|
+
|
|
1184
|
+
## 📚 Additional Resources
|
|
1185
|
+
|
|
1186
|
+
- **Main Repository**: [github.com/yourorg/vaultkit](https://github.com/yourorg/vaultkit)
|
|
1187
|
+
- **Documentation**: [docs.vaultkit.io](https://docs.vaultkit.io)
|
|
1188
|
+
- **AQL Specification**: [docs.vaultkit.io/aql](https://docs.vaultkit.io/aql)
|
|
1189
|
+
- **Policy Reference**: [docs.vaultkit.io/policies](https://docs.vaultkit.io/policies)
|
|
1190
|
+
- **API Documentation**: [docs.vaultkit.io/api](https://docs.vaultkit.io/api)
|
|
1191
|
+
|
|
1192
|
+
---
|
|
1193
|
+
|
|
1194
|
+
## 📄 License
|
|
1195
|
+
|
|
1196
|
+
VaultKit CLI is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.
|
|
1197
|
+
|
|
1198
|
+
---
|
|
1199
|
+
|
|
1200
|
+
## 💬 Support
|
|
1201
|
+
|
|
1202
|
+
- **Issues**: [github.com/yourorg/vaultkit-cli/issues](https://github.com/yourorg/vaultkit-cli/issues)
|
|
1203
|
+
- **Discussions**: [github.com/yourorg/vaultkit-cli/discussions](https://github.com/yourorg/vaultkit-cli/discussions)
|
|
1204
|
+
- **Email**: support@vaultkit.io
|
|
1205
|
+
- **Slack**: [vaultkit.slack.com](https://vaultkit.slack.com)
|
|
1206
|
+
|
|
1207
|
+
---
|
|
1208
|
+
|
|
1209
|
+
**Built with ❤️ by the VaultKit team**
|