vaultkit 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,611 +1,543 @@
1
- # VaultKit
1
+ # VaultKit CLI (`vkit`)
2
2
 
3
+ [![Gem Version](https://img.shields.io/badge/gem-v0.1.0-blue.svg)](https://rubygems.org/gems/vaultkitcli)
3
4
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
4
- [![Build Status](https://img.shields.io/badge/build-passing-brightgreen.svg)]()
5
- [![Version](https://img.shields.io/badge/version-0.1.0-orange.svg)]()
6
- [![Go Version](https://img.shields.io/badge/go-1.21+-00ADD8.svg)](https://golang.org)
7
5
  [![Ruby Version](https://img.shields.io/badge/ruby-3.1+-CC342D.svg)](https://www.ruby-lang.org)
8
6
 
9
- > A secure, policy-driven control plane for unified data access across heterogeneous data sources.
7
+ > Command-line interface for VaultKit — secure, policy-driven data access control plane
10
8
 
11
- **TL;DR**: VaultKit prevents credential sprawl and enforces data access policies across databases. Write queries in AQL (vendor-neutral JSON), policies decide who sees what, FUNL executes safely. Schema changes require explicit reviewno silent data exposure.
9
+ The VaultKit CLI (`vkit`) is the primary interface for interacting with the VaultKit control plane. It enables users to request governed data access, track approvals, manage datasources, and deploy policy bundlesall while maintaining complete audit trails.
12
10
 
13
11
  ---
14
12
 
15
- ## 🎯 What is VaultKit?
13
+ ## 📋 Table of Contents
16
14
 
17
- VaultKit provides enterprise-grade governance and security for data access, enabling applications, engineers, and AI agents to query multiple data sources through a unified, policy-controlled interface.
15
+ - [What is VaultKit CLI?](#what-is-vaultkit-cli)
16
+ - [Installation](#installation)
17
+ - [Quick Start](#quick-start)
18
+ - [Authentication](#authentication)
19
+ - [Core Workflows](#core-workflows)
20
+ - [Command Reference](#command-reference)
21
+ - [Configuration](#configuration)
22
+ - [Advanced Usage](#advanced-usage)
23
+ - [Troubleshooting](#troubleshooting)
24
+ - [Contributing](#contributing)
18
25
 
19
- VaultKit is a **control plane** that governs how data is accessed across your organization. It centralizes authentication, authorization, policy evaluation, and access control—ensuring that every data request is properly authenticated, authorized, and audited before execution.
26
+ ---
20
27
 
21
- ### Core Responsibilities
28
+ ## 🎯 What is VaultKit CLI?
22
29
 
23
- - **Authentication & Authorization**: Identity verification and access control with RBAC/ABAC support
24
- - **Policy Evaluation**: Fine-grained policies based on user attributes, data sensitivity, geographic regions, and time constraints
25
- - **Data Masking**: Column-level masking rules (full, partial, hash) applied based on user clearance
26
- - **Connection Management**: Centralized datasource configuration and credential management
27
- - **Approval Workflows**: Require explicit approval for accessing sensitive datasets
28
- - **Audit Logging**: Complete audit trail of every data access request
29
- - **Metadata Catalog**: Automated dataset classification and schema discovery
30
- - **Session Tokens**: Short-lived, cryptographically-signed tokens for Zero-Trust security
31
- - **Schema Governance**: Git-backed policy bundles with drift detection for safe schema evolution
30
+ The VaultKit CLI provides command-line access to VaultKit for:
32
31
 
33
- ---
32
+ - **Users** - Data analysts, engineers, and administrators
33
+ - **AI Agents** - LLMs and autonomous systems requiring governed data access
34
+ - **Tools & Services** - CI/CD pipelines, data pipelines, and automated workflows
35
+ - **Applications** - Programmatic integration via scripting
34
36
 
35
- ## What is FUNL?
37
+ | Capability | Description |
38
+ |------------|-------------|
39
+ | **Authentication** | Login, session management, identity verification |
40
+ | **Data Requests** | Submit AQL queries with automatic policy evaluation |
41
+ | **Approval Workflows** | Approve or deny access requests requiring authorization |
42
+ | **Grant Management** | Fetch data using approved, time-limited grants |
43
+ | **Datasource Admin** | Register and manage database connections (admin only) |
44
+ | **Schema Discovery** | Scan datasources and detect schema drift |
45
+ | **Policy Deployment** | Build, validate, and deploy policy bundles |
46
+ | **Audit Queries** | Search and export audit logs |
36
47
 
37
- FUNL (Functional Universal Query Language) is the **data plane and execution engine** that powers VaultKit's query capabilities. While VaultKit decides *if* and *how* data can be accessed, FUNL handles the actual execution.
48
+ ### Mental Model
38
49
 
39
- ### Core Responsibilities
50
+ VaultKit uses a **request → policy → grant → fetch** workflow:
40
51
 
41
- - **AQL Translation**: Converts VaultKit's Abstract Query Language (AQL) into native query languages
42
- - **Multi-Engine Support**: Executes queries against PostgreSQL, MySQL, Snowflake, BigQuery, and more
43
- - **SQL Masking Engine**: Applies column-level masking at the SQL execution layer
44
- - **Query Execution**: Handles parameterized queries with injection prevention
45
- - **Result Sanitization**: Returns masked and filtered results back to VaultKit
46
- - **JWT Authentication**: Only accepts cryptographically-signed requests from VaultKit
52
+ ```
53
+ ┌──────────┐ ┌──────────┐ ┌───────┐ ┌───────┐ ┌───────────┐
54
+ Request │ -> Policy │ -> Grant -> Fetch │ -> │ Audit Log │
55
+ (AQL) │ │ Eval │ │ (JWT) │ │ Data │ │ (Record) │
56
+ └──────────┘ └──────────┘ └───────┘ └───────┘ └───────────┘
57
+ ```
47
58
 
48
- FUNL is designed to be lightweight, stateless, and horizontally scalable—making it ideal for high-throughput data access scenarios.
59
+ **Key Principles:**
60
+ - Users **request** data, not queries
61
+ - Policies **decide** what happens (allow, mask, deny, require approval)
62
+ - Grants **authorize** execution with TTLs
63
+ - FUNL **executes** queries safely
64
+ - Everything is **audited**
49
65
 
50
66
  ---
51
67
 
52
- ## 🏗️ Architecture
53
-
54
- ```mermaid
55
- flowchart TB
56
- subgraph clients["🖥️ Client Layer"]
57
- app["Applications"]
58
- cli["CLI Tools"]
59
- agent["AI Agents"]
60
- end
61
-
62
- subgraph vaultkit["🛡️ VaultKit — Control Plane"]
63
- direction TB
64
- orch["Request Orchestrator"]
65
- policy["Policy Engine<br/>(ABAC/RBAC)"]
66
- auth["Authentication<br/>(JWT/SSO)"]
67
- registry["Schema Registry"]
68
- bundle["Active Policy Bundle"]
69
- conn["Connection Manager"]
70
- approval["Approval Workflow"]
71
- audit["Audit Logger"]
72
- end
73
-
74
- subgraph funl["⚡ FUNL — Data Plane"]
75
- direction TB
76
- translator["AQL Translator"]
77
- executor["Query Executor"]
78
- masking["SQL Masking Layer"]
79
- end
80
-
81
- subgraph datasources["💾 Data Sources"]
82
- pg[(PostgreSQL)]
83
- mysql[(MySQL)]
84
- snow[(Snowflake)]
85
- bq[(BigQuery)]
86
- end
87
-
88
- subgraph governance["📋 Governance Layer"]
89
- git["Git Repository<br/>(Policies & Registry)"]
90
- scan["Schema Scans"]
91
- cicd["CI/CD Pipeline"]
92
- end
93
-
94
- clients -->|AQL Request| orch
95
- orch --> auth
96
- orch --> policy
97
- policy --> bundle
98
- orch --> registry
99
- orch --> approval
100
- orch --> audit
101
- orch -->|Signed Token + AQL| translator
102
-
103
- translator --> executor
104
- executor --> masking
105
- masking --> datasources
106
- datasources --> masking
107
- masking --> executor
108
- executor --> translator
109
- translator -->|Masked Results| orch
110
- orch -->|Response| clients
111
-
112
- scan -.->|Discovers Schema| datasources
113
- scan -->|Drift Detection| registry
114
- git -->|Deploy Bundle| bundle
115
- cicd -->|Build & Validate| git
116
-
117
- style vaultkit fill:#e3f2fd
118
- style funl fill:#fff3e0
119
- style datasources fill:#f3e5f5
120
- style governance fill:#f1f8e9
121
- ```
122
-
123
- ### Component Breakdown
124
-
125
- #### VaultKit (Control Plane)
126
-
127
- - **Request Orchestrator**: Routes and manages the lifecycle of data access requests
128
- - **Policy Engine**: Evaluates ABAC/RBAC rules with support for sensitivity levels, regions, and time constraints
129
- - **Policy Priority System**: `deny` > `require_approval` > `mask` > `allow`
130
- - **Match Rules**: Dataset-based, field sensitivity, categories, specific field names
131
- - **Context Awareness**: User role, clearance level, region, environment, time windows
132
- - **Authentication Service**: Handles user identity via JWT, OAuth, or SSO integration
133
- - **Schema Registry**: Runtime database storing discovered schemas and serving as drift baseline
134
- - **Active Policy Bundle**: Immutable, Git-backed artifact containing all enforcement rules
135
- - **Connection Manager**: Manages datasource credentials (supports VaultKit store, HashiCorp Vault, AWS Secrets Manager)
136
- - **Approval Workflow**: Implements multi-stage approval for sensitive data access
137
- - **Audit Logger**: Records every request to pluggable sinks (SQLite, PostgreSQL, S3, etc.)
138
-
139
- #### FUNL (Data Plane)
140
- - **AQL Translator**: Parses AQL and generates engine-specific SQL with proper escaping
141
- - **Query Executor**: Maintains connection pools and executes parameterized queries
142
- - **SQL Masking Layer**: Injects masking functions into SELECT statements based on policy
143
- - **Engine Plugins**: Extensible architecture for adding new database engines
144
-
145
- #### Governance Layer
146
- - **Git Repository**: Source of truth for all policies and dataset registries
147
- - **Schema Scans**: Observational tooling that discovers database changes
148
- - **CI/CD Pipeline**: Automated validation, bundling, and deployment of policy changes
149
-
150
- ---
68
+ ## 📦 Installation
151
69
 
152
- ## 📋 Schema Discovery & Policy Management
70
+ ### Prerequisites
153
71
 
154
- VaultKit's approach to schema evolution is designed for **security, auditability, and safe change management**.
72
+ - **Ruby** 3.1 or higher
73
+ - **VaultKit Control Plane** running and accessible
155
74
 
156
- ### The Core Philosophy: Intent vs. Reality
75
+ ### Option 1: Install from RubyGems (Recommended)
157
76
 
158
- VaultKit deliberately separates:
77
+ ```bash
78
+ gem install vaultkitcli
79
+ ```
159
80
 
160
- - **Policy Bundles** (Intent) — Git-backed, immutable definitions of *what should be allowed*
161
- - **Schema Scans** (Reality) — Observational discovery of *what actually exists*
81
+ ### Verify Installation
162
82
 
163
- This separation prevents silent data exposure. When new tables or sensitive columns appear, they require **explicit policy review** before access is granted.
83
+ ```bash
84
+ vkit --version
85
+ # => VaultKit CLI v0.1.0
164
86
 
165
- ```mermaid
166
- flowchart LR
167
- A[Live Database] -->|1. Scan Discovers| B[Schema Changes]
168
- B -->|2. Drift Detection| C[Schema Registry]
169
- C -->|3. Human Review| D[Policy Updates in Git]
170
- D -->|4. CI Validation| E[Build Bundle]
171
- E -->|5. Deploy| F[Active Bundle]
172
- F -->|Enforces| G[Access Control]
173
-
174
- style A fill:#f3e5f5
175
- style C fill:#fff3e0
176
- style D fill:#e8f5e9
177
- style F fill:#e3f2fd
178
- style G fill:#fce4ec
87
+ vkit --help
179
88
  ```
180
89
 
181
- ### How It Works
182
-
183
- **Policy Bundles** are versioned artifacts that contain:
184
- - Dataset registry (expected schema)
185
- - Datasource definitions
186
- - Compiled policy rules
187
- - Bundle metadata (version, checksum, source)
90
+ ---
188
91
 
189
- **Schema Scans** perform these steps:
190
- 1. Introspect datasources via FUNL
191
- 2. Discover tables and columns
192
- 3. Classify fields (PII, financial, internal)
193
- 4. Compute diff against runtime registry
194
- 5. Surface changes for review
92
+ ## Quick Start
195
93
 
196
- **Key Safety Principle:**
94
+ ### 1. Set Environment Variables
197
95
 
198
- > Scans inform. Bundles enforce. Humans decide.
96
+ ```bash
97
+ export VKIT_API_URL=http://localhost:3000
98
+ ```
199
99
 
200
- Scans **never** automatically update active policies. This ensures:
201
- - No silent expansion of policy scope
202
- - No auto-granting access to new sensitive data
203
- - Full Git-based audit trail
204
- - Compliance with SOC2, ISO 27001, GDPR, HIPAA
100
+ For production:
101
+ ```bash
102
+ export VKIT_API_URL=https://vaultkit.company.com
103
+ ```
205
104
 
206
- ### Quick Example: Handling Schema Drift
105
+ ### 2. Authenticate
207
106
 
208
107
  ```bash
209
- # 1. Discover schema changes (safe, read-only)
210
- vkit scan production_db
211
-
212
- # Output shows drift:
213
- # + dataset: customers
214
- # + field: ssn (string) [PII] ⚠️ NEW - requires policy
215
- # ~ field: email (text → varchar) [PII] ✓ Already masked
108
+ vkit login --endpoint http://localhost:3000 --email analyst@company.com
109
+ ```
216
110
 
217
- # 2. Review and update baseline
218
- vkit scan production_db --apply
111
+ You'll receive a prompt for your password or be redirected to SSO.
219
112
 
220
- # 3. Update policies in Git
221
- vim policies/customer_data.yaml
113
+ ### 3. Submit Your First Request
222
114
 
223
- # 4. Build and deploy new bundle
224
- vkit policy bundle
225
- vkit policy deploy
115
+ ```bash
116
+ vkit request --aql '{
117
+ "source_table": "customers",
118
+ "columns": ["id", "email", "total_spend"],
119
+ "limit": 10
120
+ }'
226
121
  ```
227
122
 
228
- ### CI/CD Integration (Recommended)
123
+ ### 4. Fetch Results
229
124
 
230
- ```yaml
231
- name: Schema Drift Check
232
- on: [schedule, push]
233
- jobs:
234
- drift-detection:
235
- runs-on: ubuntu-latest
236
- steps:
237
- - name: Scan all datasources
238
- run: vkit scan --all --fail-on-drift
239
-
240
- - name: Create PR if drift detected
241
- if: failure()
242
- run: |
243
- vkit scan --all --diff > drift-report.md
244
- gh pr create --title "Schema Drift Detected" --body-file drift-report.md
125
+ If granted immediately:
126
+ ```bash
127
+ vkit fetch --grant gr_abc123xyz
245
128
  ```
246
129
 
247
- 📘 **[Full Bundling & Scan System Documentation →](./docs/BUNDLING_SCAN_SYSTEM.md)**
130
+ If approval required:
131
+ ```bash
132
+ # Check status
133
+ vkit requests list --state pending
134
+
135
+ # After approval
136
+ vkit fetch --grant gr_abc123xyz
137
+ ```
248
138
 
249
139
  ---
250
140
 
251
- ## 🔄 AQL to Native Query Translation
141
+ ## 🔐 Authentication
142
+
143
+ ### `vkit login`
144
+
145
+ Authenticate with the VaultKit control plane.
252
146
 
253
- AQL (Access Query Language) is a structured JSON format that describes **what** to fetch, not **how**. This abstraction allows VaultKit to enforce policies consistently across different database engines.
147
+ ```bash
148
+ vkit login --endpoint http://localhost:3000 --email analyst@acme.com
149
+ ```
254
150
 
255
- ### AQL Structure
151
+ **Options:**
152
+ - `--endpoint` — VaultKit API endpoint (can also use `VKIT_API_URL`)
153
+ - `--email` — Email address for authentication
256
154
 
257
- AQL is a structured JSON specification with the following top-level fields:
155
+ **SSO Integration:**
258
156
 
259
- ```json
260
- {
261
- "source_table": "table_name",
262
- "columns": ["field1", "field2"],
263
- "joins": [],
264
- "aggregates": [],
265
- "filters": [],
266
- "group_by": [],
267
- "having": [],
268
- "order_by": null,
269
- "limit": 0,
270
- "offset": 0
271
- }
157
+ If your organization uses SSO, you'll be redirected to your identity provider:
158
+
159
+ ```bash
160
+ vkit login --endpoint https://vaultkit.company.com --email you@company.com
161
+
162
+ # Output:
163
+ # Opening browser for SSO authentication...
164
+ # ✓ Authentication successful
165
+ # Session expires: 2024-01-15 18:00:00 UTC
272
166
  ```
273
167
 
274
- ### Translation Examples
168
+ ### `vkit whoami`
275
169
 
276
- **Simple Query:**
277
- ```json
278
- {
279
- "source_table": "customers",
280
- "columns": ["email", "country", "revenue"],
281
- "filters": [
282
- { "field": "country", "operator": "eq", "value": "US" },
283
- { "field": "revenue", "operator": "gt", "value": 10000 }
284
- ],
285
- "limit": 100
286
- }
170
+ Display the currently authenticated user and session information.
171
+
172
+ ```bash
173
+ vkit whoami
287
174
  ```
288
175
 
289
- **FUNL Translation → PostgreSQL:**
290
- ```sql
291
- SELECT email, country, revenue
292
- FROM customers
293
- WHERE country = $1 AND revenue > $2
294
- LIMIT 100
176
+ **Output:**
177
+ ```
178
+ 👤 analyst@acme.com (role: analyst, org: acme)
295
179
  ```
296
180
 
297
- **FUNL Translation → MySQL:**
298
- ```sql
299
- SELECT `email`, `country`, `revenue`
300
- FROM `customers`
301
- WHERE `country` = ? AND `revenue` > ?
302
- LIMIT 100
181
+ **JSON Format:**
182
+ ```bash
183
+ vkit whoami --format json
303
184
  ```
304
185
 
305
- **Complex Query with JOINs and Aggregates:**
186
+ **Output:**
306
187
  ```json
307
188
  {
308
- "source_table": "users",
309
- "joins": [
310
- {
311
- "type": "LEFT",
312
- "table": "orders",
313
- "left_field": "users.id",
314
- "right_field": "orders.user_id"
315
- }
316
- ],
317
- "columns": ["users.email", "users.username"],
318
- "aggregates": [
319
- {
320
- "func": "sum",
321
- "field": "orders.amount",
322
- "alias": "total_spent"
323
- }
324
- ],
325
- "group_by": ["users.email", "users.username"],
326
- "having": [
327
- {
328
- "operator": "gt",
329
- "field": "SUM(orders.amount)",
330
- "value": 500
331
- }
332
- ],
333
- "order_by": {
334
- "column": "total_spent",
335
- "direction": "DESC"
336
- },
337
- "limit": 10
189
+ "email": "analyst@acme.com",
190
+ "role": "analyst",
191
+ "org": "acme"
338
192
  }
339
193
  ```
340
194
 
341
- **FUNL Translation → PostgreSQL:**
342
- ```sql
343
- SELECT users.email, users.username, SUM(orders.amount) AS total_spent
344
- FROM users
345
- LEFT JOIN orders ON users.id = orders.user_id
346
- GROUP BY users.email, users.username
347
- HAVING SUM(orders.amount) > $1
348
- ORDER BY total_spent DESC
349
- LIMIT 10
350
- ```
195
+ ### `vkit logout`
351
196
 
352
- ### Advanced Features
197
+ Clear stored credentials and end session.
353
198
 
354
- **Complex Filtering with OR Logic:**
355
- ```json
356
- {
357
- "source_table": "users",
358
- "columns": ["id", "email", "role"],
359
- "filters": [
360
- {
361
- "logic": "OR",
362
- "conditions": [
363
- { "field": "users.role", "operator": "eq", "value": "admin" },
364
- { "field": "users.role", "operator": "eq", "value": "manager" }
365
- ]
366
- },
367
- { "field": "orders.amount", "operator": "gt", "value": 100 }
368
- ]
369
- }
199
+ ```bash
200
+ vkit logout
370
201
  ```
371
202
 
372
- **Translation:**
373
- ```sql
374
- SELECT id, email, role
375
- FROM users
376
- WHERE (users.role = $1 OR users.role = $2) AND orders.amount > $3
203
+ ---
204
+
205
+ ## 🔄 Core Workflows
206
+
207
+ ### Workflow 1: Immediate Access (Auto-Approved)
208
+
209
+ ```bash
210
+ # 1. Submit request
211
+ vkit request --aql '{
212
+ "source_table": "public_reports",
213
+ "columns": ["report_id", "title", "published_date"],
214
+ "limit": 10
215
+ }'
216
+
217
+ # Output:
218
+ # ✓ Request granted immediately
219
+ # Grant ID: gr_abc123xyz
220
+ # TTL: 4 hours
221
+ # Use: vkit fetch --grant gr_abc123xyz
222
+
223
+ # 2. Fetch data
224
+ vkit fetch --grant gr_abc123xyz --format table
377
225
  ```
378
226
 
379
- **Supported Operators:**
380
- - Comparison: `eq`, `neq`, `gt`, `lt`, `gte`, `lte`
381
- - Pattern matching: `like`
382
- - Set operations: `in`
383
- - Null checks: `is_null`, `is_not_null`
227
+ ### Workflow 2: Approval Required
384
228
 
385
- **Supported Aggregations:**
386
- - `sum`, `count`, `avg`, `min`, `max`
229
+ ```bash
230
+ # 1. Submit request
231
+ vkit request --aql '{
232
+ "source_table": "financial_transactions",
233
+ "columns": ["transaction_id", "amount", "customer_id"],
234
+ "filters": [{"field": "amount", "operator": "gt", "value": 10000}]
235
+ }'
387
236
 
388
- **Supported JOINs:**
389
- - `INNER`, `LEFT`, `RIGHT`, `FULL`
237
+ # Output:
238
+ # Approval required
239
+ # Request ID: req_20240115_xyz789
240
+ # Approver: finance_manager
241
+ # Reason: Financial data requires manager approval
242
+ # Status: PENDING
390
243
 
391
- ### SQL-Level Masking
244
+ # 2. Check status
245
+ vkit requests list --state pending
392
246
 
393
- FUNL's **Masking Dialect System** applies transformations directly in SQL based on VaultKit's policy decisions:
247
+ # 3. After approval, fetch data
248
+ vkit fetch --grant gr_approved_abc123
249
+ ```
394
250
 
395
- | Masking Type | PostgreSQL | MySQL | Snowflake |
396
- |-------------|------------|-------|-----------|
397
- | **Full** | `'*****' AS email` | `'*****' AS email` | `'*****' AS email` |
398
- | **Partial** | `CONCAT(LEFT(email, 3), '****')` | `CONCAT(LEFT(email, 3), '****')` | `CONCAT(LEFT(email, 3), '****')` |
399
- | **Hash** | `ENCODE(SHA256(email::bytea), 'hex')` | `SHA2(email, 256)` | `SHA2(email, 256)` |
251
+ ### Workflow 3: Denied Access
400
252
 
401
- ---
253
+ ```bash
254
+ vkit request --aql '{
255
+ "source_table": "payment_cards",
256
+ "columns": ["card_number", "cvv"],
257
+ "limit": 10
258
+ }'
402
259
 
403
- ## ✨ Key Features
404
-
405
- ### VaultKit (Control Plane)
406
-
407
- | Feature | Description |
408
- |---------|-------------|
409
- | **AQL Orchestration** | Vendor-neutral query language prevents SQL injection and enables policy enforcement |
410
- | **Policy Engine** | Attribute-based access control with support for clearance levels, regions, sensitivity tags, and time windows |
411
- | **Policy Priority System** | Hierarchical decision making: `deny` > `require_approval` > `mask` > `allow` |
412
- | **Field-Level Policies** | Match rules based on dataset name, field sensitivity, categories (pii, financial, etc.), or specific field names |
413
- | **Context-Aware Rules** | Policies evaluate requester role, clearance level, region, environment, and time constraints |
414
- | **Approval Workflows** | Require manager/security approval for accessing PII, financial data, or production environments |
415
- | **Zero-Trust Sessions** | Short-lived, cryptographically-signed tokens with automatic expiration |
416
- | **Credential Abstraction** | Never expose database credentials to users—supports multiple secret backends |
417
- | **Git-Backed Governance** | All policies and registries versioned in Git with full audit trail |
418
- | **Schema Drift Detection** | Automated discovery of database changes with manual approval workflow |
419
- | **Comprehensive Auditing** | Every query logged with user identity, timestamp, policy decisions, and results metadata |
420
- | **CLI & SDK** | Rich command-line tools (`vkit`) and programmatic SDK for integration |
421
-
422
- ### FUNL (Data Plane)
423
-
424
- | Feature | Description |
425
- |---------|-------------|
426
- | **Multi-Engine Translation** | Supports PostgreSQL, MySQL, Snowflake, BigQuery with identical AQL interface |
427
- | **SQL-Level Masking** | Masking applied during query execution—no post-processing overhead |
428
- | **Injection Prevention** | All queries use parameterized execution with proper type binding |
429
- | **Raw SQL Support** | Supports raw SQL for schema introspection and admin operations (policy-controlled) |
430
- | **Horizontal Scaling** | Stateless design enables running multiple FUNL instances behind a load balancer |
431
- | **JWT Verification** | Only executes queries signed by VaultKit's private key |
260
+ # Output:
261
+ # ❌ Request denied
262
+ # Reason: Direct payment card data access forbidden
263
+ # Policy: pci_dss_restrictions
264
+ # Alternative: Use tokenized_card_id field instead
265
+ # Audit ID: audit_20240115_denied_123
266
+ ```
432
267
 
433
268
  ---
434
269
 
435
- ## 🚀 Getting Started
270
+ ## 📚 Command Reference
436
271
 
437
- ### Prerequisites Checklist
272
+ ### Authentication Commands
438
273
 
439
- Before installing VaultKit, ensure you have:
274
+ #### `vkit login`
440
275
 
441
- - [ ] **Ruby** 3.1+ installed (`ruby --version`)
442
- - [ ] **Go** 1.21+ installed (`go version`)
443
- - [ ] **OpenSSL** for key generation
444
- - [ ] **Git** for policy management
445
- - [ ] Access credentials for at least one supported datasource
446
- - [ ] (Optional) **Docker** for containerized FUNL deployment
276
+ Authenticate with VaultKit.
447
277
 
448
- ### Quick Start (5 Minutes)
449
-
450
- This streamlined path gets you querying data quickly. For production deployments, see [Detailed Setup](#detailed-setup) below.
278
+ ```bash
279
+ vkit login --endpoint <URL> --email <EMAIL>
280
+ ```
451
281
 
452
- #### 1. Generate Cryptographic Keys
282
+ **Options:**
283
+ - `--endpoint, -e` — VaultKit API endpoint (default: `$VKIT_API_URL`)
284
+ - `--email` — User email address
285
+ - `--password` — Password (prompted if not provided)
453
286
 
287
+ **Examples:**
454
288
  ```bash
455
- # Generate RSA key pair for token signing
456
- openssl genpkey -algorithm RSA -out vaultkit_private.pem -pkeyopt rsa_keygen_bits:2048
457
- openssl rsa -pubout -in vaultkit_private.pem -out vaultkit_public.pem
289
+ # Interactive login
290
+ vkit login --endpoint http://localhost:3000 --email analyst@company.com
458
291
 
459
- # Set environment variables
460
- export VKIT_PRIVATE_KEY="$(pwd)/vaultkit_private.pem"
461
- export VKIT_PUBLIC_KEY="$(pwd)/vaultkit_public.pem"
462
- export FUNL_PUBLIC_KEY="$(pwd)/vaultkit_public.pem"
292
+ # Non-interactive (CI/CD)
293
+ vkit login --endpoint https://vaultkit.company.com --email bot@company.com --password $BOT_PASSWORD
463
294
  ```
464
295
 
465
- ⚠️ **Security Note**: Keep your private key secure. Add `*.pem` to your `.gitignore`.
296
+ #### `vkit whoami`
466
297
 
467
- #### 2. Start FUNL
298
+ Display current user information.
468
299
 
469
300
  ```bash
470
- # Using Docker (recommended)
471
- export FUNL_URL="http://localhost:8080"
472
- docker compose up -d
301
+ vkit whoami [--format json|table]
302
+ ```
473
303
 
474
- # Verify FUNL is running
475
- curl http://localhost:8080/health
304
+ **Examples:**
305
+ ```bash
306
+ vkit whoami
307
+ vkit whoami --format json
476
308
  ```
477
309
 
478
- #### 3. Install VaultKit CLI
310
+ #### `vkit logout`
311
+
312
+ End session and clear credentials.
479
313
 
480
314
  ```bash
481
- git clone https://github.com/yourorg/vaultkitcli.git
482
- cd vaultkitcli
483
- gem build vaultkitcli.gemspec
484
- gem install ./vaultkitcli-0.1.0.gem
315
+ vkit logout
316
+ ```
485
317
 
486
- # Verify installation
487
- vkit --version
318
+ ---
319
+
320
+ ### Data Request Commands
321
+
322
+ #### `vkit request`
323
+
324
+ Submit a governed data request using AQL.
325
+
326
+ ```bash
327
+ vkit request --aql <AQL_JSON> [OPTIONS]
488
328
  ```
489
329
 
490
- #### 4. Initialize & Connect
330
+ **Options:**
331
+ - `--aql` — AQL JSON payload (inline or via STDIN)
332
+ - `--env` — Environment (default: `production`)
333
+ - `--requester_region` — Your geographic region (e.g., `US`, `EU`)
334
+ - `--dataset_region` — Dataset's geographic region
335
+ - `--requester_clearance` — Clearance level: `low`, `high`, `admin`
336
+ - `--format` — Output format: `json`, `table` (default: `table`)
491
337
 
338
+ **Inline AQL:**
492
339
  ```bash
493
- # Authenticate
494
- vkit login
340
+ vkit request \
341
+ --aql '{"source_table":"customers","columns":["email","total_spend"],"limit":100}' \
342
+ --requester_region EU \
343
+ --dataset_region EU \
344
+ --requester_clearance high
345
+ ```
495
346
 
496
- # Register your first datasource
497
- vkit datasource add \
498
- --id demo_db \
499
- --engine postgres \
500
- --username readonly_user \
501
- --password $DB_PASSWORD \
502
- --config '{
503
- "host": "localhost",
504
- "port": 5432,
505
- "database": "analytics"
506
- }'
347
+ **AQL via STDIN:**
348
+ ```bash
349
+ cat query.json | vkit request
350
+
351
+ # Or
352
+ echo '{"source_table":"orders","columns":["order_id","total"]}' | vkit request
353
+ ```
507
354
 
508
- # Discover schema
509
- vkit scan demo_db --apply
355
+ **AQL from File:**
356
+ ```bash
357
+ vkit request --aql "$(cat queries/customer_analysis.json)"
510
358
  ```
511
359
 
512
- #### 5. Query Data
360
+ **Possible Outcomes:**
361
+
362
+ | Outcome | Description | Next Step |
363
+ |---------|-------------|-----------|
364
+ | ✅ **Granted** | Immediate access | `vkit fetch --grant <GRANT_ID>` |
365
+ | ⏳ **Queued** | Requires approval | Wait for approval, check with `vkit requests list` |
366
+ | ❌ **Denied** | Policy disallows | Review policy restrictions |
367
+
368
+ #### `vkit requests list`
369
+
370
+ View your request history.
513
371
 
514
372
  ```bash
515
- # Submit AQL query
516
- vkit request --datasource demo_db --aql '{
517
- "source_table": "users",
518
- "columns": ["id", "email", "created_at"],
519
- "limit": 10
520
- }'
373
+ vkit requests list [--state STATE] [--format FORMAT]
374
+ ```
375
+
376
+ **Options:**
377
+ - `--state` — Filter by state: `all`, `pending`, `approved`, `denied` (default: `all`)
378
+ - `--format` — Output format: `json`, `table` (default: `table`)
379
+ - `--limit` — Number of results (default: 50)
380
+
381
+ **Examples:**
382
+ ```bash
383
+ # All requests
384
+ vkit requests list
521
385
 
522
- # Fetch results (use grant ID from previous command)
523
- vkit fetch --grant grant_abc123xyz
386
+ # Only pending requests
387
+ vkit requests list --state pending
388
+
389
+ # Approved requests in JSON
390
+ vkit requests list --state approved --format json
391
+
392
+ # Last 100 requests
393
+ vkit requests list --limit 100
524
394
  ```
525
395
 
526
- 🎉 **You're now querying data through VaultKit!** Continue to [Detailed Setup](#detailed-setup) for production configuration.
396
+ #### `vkit requests show`
397
+
398
+ Show details for a specific request.
399
+
400
+ ```bash
401
+ vkit requests show <REQUEST_ID>
402
+ ```
403
+
404
+ **Example:**
405
+ ```bash
406
+ vkit requests show req_20240115_xyz789
407
+ ```
527
408
 
528
409
  ---
529
410
 
530
- ### Detailed Setup
411
+ ### Approval Commands
531
412
 
532
- #### Installation Options
413
+ #### `vkit approval:list`
533
414
 
534
- **Option A: From Source (Recommended for Development)**
415
+ List approval requests you are authorized to act on.
535
416
 
536
417
  ```bash
537
- # Clone and install CLI
538
- git clone https://github.com/yourorg/vaultkitcli.git
539
- cd vaultkitcli
540
- bundle install
541
- gem build vaultkitcli.gemspec
542
- gem install ./vaultkitcli-0.1.0.gem
418
+ vkit approval:list [--state STATE] [--format FORMAT]
419
+ ```
420
+
421
+ **Options:**
422
+ - `--state` — Filter by state: `all`, `pending`, `approved`, `denied` (default: `pending`)
423
+ - `--format` — Output format: `json`, `table` (default: `table`)
424
+
425
+ **Examples:**
426
+ ```bash
427
+ # Pending approvals
428
+ vkit approval:list
429
+
430
+ # All approvals you've handled
431
+ vkit approval:list --state all
432
+
433
+ # JSON output for scripting
434
+ vkit approval:list --format json
435
+ ```
436
+
437
+ #### `vkit approval:approve`
438
+
439
+ Approve a pending request.
440
+
441
+ ```bash
442
+ vkit approval:approve <REQUEST_ID> [--ttl SECONDS] [--notes "..."]
443
+ ```
543
444
 
544
- # Clone and build FUNL
545
- git clone https://github.com/yourorg/funl.git
546
- cd funl
547
- go build -o funl ./cmd/funl
548
- ./funl serve --port 8080
445
+ **Options:**
446
+ - `--ttl` — Grant lifetime in seconds (default: 3600)
447
+ - `--notes` — Approval notes (optional)
448
+
449
+ **Examples:**
450
+ ```bash
451
+ # Basic approval
452
+ vkit approval:approve 42
453
+
454
+ # With custom TTL and notes
455
+ vkit approval:approve 42 \
456
+ --ttl 7200 \
457
+ --notes "Approved for Q4 analysis. Limited to aggregate data only."
549
458
  ```
550
459
 
551
- **Option B: Using Docker (Recommended for Production)**
460
+ #### `vkit approval:deny`
461
+
462
+ Deny a pending request.
552
463
 
553
464
  ```bash
554
- # docker-compose.yml
555
- version: '3.8'
556
- services:
557
- funl:
558
- image: vaultkit/funl:0.1.0
559
- ports:
560
- - "8080:8080"
561
- environment:
562
- - FUNL_PUBLIC_KEY=/keys/vaultkit_public.pem
563
- volumes:
564
- - ./vaultkit_public.pem:/keys/vaultkit_public.pem:ro
565
- healthcheck:
566
- test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
567
- interval: 10s
568
- timeout: 3s
569
- retries: 3
465
+ vkit approval:deny <REQUEST_ID> --reason "..."
570
466
  ```
571
467
 
572
- #### Environment Configuration
468
+ **Options:**
469
+ - `--reason` — Required explanation for denial
470
+
471
+ **Examples:**
472
+ ```bash
473
+ vkit approval:deny 42 --reason "Insufficient business justification"
474
+
475
+ vkit approval:deny 42 --reason "Request violates data minimization policy"
476
+ ```
477
+
478
+ ---
479
+
480
+ ### Data Fetch Commands
573
481
 
574
- Create a configuration file for persistent settings:
482
+ #### `vkit fetch`
483
+
484
+ Fetch data using a previously issued grant.
575
485
 
576
486
  ```bash
577
- # ~/.vkit/config.yaml
578
- funl_url: "http://localhost:8080"
579
- private_key_path: "/path/to/vaultkit_private.pem"
580
- public_key_path: "/path/to/vaultkit_public.pem"
581
- default_datasource: "production_db"
582
- audit_log_path: "/var/log/vaultkit/audit.log"
487
+ vkit fetch --grant <GRANT_REF> [--format FORMAT]
583
488
  ```
584
489
 
585
- Add environment variables to your shell profile:
490
+ **Options:**
491
+ - `--grant, -g` — Grant reference or ID (required)
492
+ - `--format` — Output format: `json`, `table`, `csv` (default: `table`)
493
+ - `--output, -o` — Write to file instead of STDOUT
494
+
495
+ **Examples:**
496
+ ```bash
497
+ # Fetch and display as table
498
+ vkit fetch --grant gr_abc123xyz
499
+
500
+ # Fetch as JSON
501
+ vkit fetch --grant gr_abc123xyz --format json
502
+
503
+ # Fetch and save to CSV
504
+ vkit fetch --grant gr_abc123xyz --format csv --output results.csv
505
+
506
+ # Pipe to jq for processing
507
+ vkit fetch --grant gr_abc123xyz --format json | jq '.data[] | select(.revenue > 1000)'
508
+ ```
509
+
510
+ ---
511
+
512
+ ### Datasource Management (Admin Only)
513
+
514
+ #### `vkit datasource add`
515
+
516
+ Register a new datasource.
586
517
 
587
518
  ```bash
588
- # ~/.bashrc or ~/.zshrc
589
- export VKIT_PRIVATE_KEY="/path/to/vaultkit_private.pem"
590
- export VKIT_PUBLIC_KEY="/path/to/vaultkit_public.pem"
591
- export FUNL_PUBLIC_KEY="/path/to/vaultkit_public.pem"
592
- export FUNL_URL="http://localhost:8080"
593
- export VKIT_CONFIG_PATH="$HOME/.vkit/config.yaml"
519
+ vkit datasource add --id <ID> --engine <ENGINE> [OPTIONS]
594
520
  ```
595
521
 
596
- #### Datasource Configuration
522
+ **Options:**
523
+ - `--id` — Unique datasource identifier (required)
524
+ - `--engine` — Database engine: `postgres`, `mysql`, `snowflake`, `bigquery` (required)
525
+ - `--username` — Database username
526
+ - `--password` — Database password
527
+ - `--config` — JSON configuration (host, port, database, etc.)
528
+ - `--credential-backend` — Secret backend: `vaultkit`, `vault`, `aws-secrets`
597
529
 
598
- VaultKit supports multiple credential storage backends:
530
+ **Examples:**
599
531
 
600
- **Built-in Storage (Development):**
532
+ **PostgreSQL (Built-in credential storage):**
601
533
  ```bash
602
534
  vkit datasource add \
603
- --id staging_db \
535
+ --id production_pg \
604
536
  --engine postgres \
605
- --username app_user \
537
+ --username vaultkit_ro \
606
538
  --password $DB_PASSWORD \
607
539
  --config '{
608
- "host": "staging.db.internal",
540
+ "host": "db.production.internal",
609
541
  "port": 5432,
610
542
  "database": "analytics",
611
543
  "ssl_mode": "require",
@@ -613,349 +545,594 @@ vkit datasource add \
613
545
  }'
614
546
  ```
615
547
 
616
- **HashiCorp Vault (Production):**
548
+ **MySQL:**
617
549
  ```bash
618
550
  vkit datasource add \
619
- --id production_db \
620
- --engine postgres \
621
- --credential-backend vault \
622
- --vault-path secret/data/databases/production \
551
+ --id production_mysql \
552
+ --engine mysql \
553
+ --username app_reader \
554
+ --password $MYSQL_PASSWORD \
623
555
  --config '{
624
- "host": "prod.db.internal",
625
- "port": 5432,
626
- "database": "analytics",
627
- "ssl_mode": "verify-full"
556
+ "host": "mysql.production.internal",
557
+ "port": 3306,
558
+ "database": "ecommerce",
559
+ "ssl_mode": "REQUIRED"
628
560
  }'
629
561
  ```
630
562
 
631
- **AWS Secrets Manager:**
563
+ #### `vkit datasource list`
564
+
565
+ List all registered datasources.
566
+
632
567
  ```bash
633
- vkit datasource add \
634
- --id cloud_db \
635
- --engine postgres \
636
- --credential-backend aws-secrets \
637
- --secret-arn arn:aws:secretsmanager:us-east-1:123456789:secret:db-creds \
638
- --config '{
639
- "host": "rds.amazonaws.com",
640
- "port": 5432,
641
- "database": "production"
642
- }'
568
+ vkit datasource list [--format FORMAT]
643
569
  ```
644
570
 
645
- #### Policy Setup
571
+ **Examples:**
572
+ ```bash
573
+ vkit datasource list
574
+ vkit datasource list --format json
575
+ ```
576
+
577
+ #### `vkit datasource get`
646
578
 
647
- Initialize your policy repository:
579
+ Get details for a specific datasource.
648
580
 
649
581
  ```bash
650
- # Create policy repository structure
651
- mkdir -p vaultkit-policies/{policies,registries,datasources}
652
- cd vaultkit-policies
582
+ vkit datasource get <DATASOURCE_ID>
583
+ ```
653
584
 
654
- git init
655
- git add .
656
- git commit -m "Initialize VaultKit policies"
585
+ **Example:**
586
+ ```bash
587
+ vkit datasource get production_pg
588
+ ```
657
589
 
658
- # Link VaultKit to your policy repo
659
- vkit policy init --repo $(pwd)
590
+ #### `vkit datasource test`
591
+
592
+ Test connectivity to a datasource.
593
+
594
+ ```bash
595
+ vkit datasource test <DATASOURCE_ID>
660
596
  ```
661
597
 
662
- Create your first policy (`policies/base_access.yaml`):
598
+ **Example:**
599
+ ```bash
600
+ vkit datasource test production_pg
663
601
 
664
- ```yaml
665
- id: analyst_basic_access
666
- description: Basic read access for analysts with PII masking
667
- priority: 100
602
+ # Output:
603
+ # Testing connection to production_pg...
604
+ # Connection successful
605
+ # Engine: PostgreSQL 15.3
606
+ # Latency: 45ms
607
+ ```
608
+
609
+ #### `vkit datasource remove`
668
610
 
669
- match:
670
- datasets:
671
- - customers
672
- - orders
673
-
674
- context:
675
- requester_role: analyst
676
- environment: production
611
+ Remove a datasource (admin only).
677
612
 
678
- rules:
679
- - field_category: pii
680
- action: mask
681
- mask_type: partial
682
- reason: "PII must be masked for analysts"
683
-
684
- - field_category: financial
685
- action: require_approval
686
- approver_role: finance_manager
687
- ttl: "2h"
688
- reason: "Financial data requires approval"
689
-
690
- - default:
691
- action: allow
692
- ttl: "8h"
613
+ ```bash
614
+ vkit datasource remove <DATASOURCE_ID> [--force]
693
615
  ```
694
616
 
695
- Build and deploy the policy bundle:
617
+ **Options:**
618
+ - `--force` — Skip confirmation prompt
696
619
 
620
+ **Example:**
697
621
  ```bash
698
- # Validate policies
699
- vkit policy validate
622
+ vkit datasource remove old_staging_db --force
623
+ ```
624
+
625
+ ---
700
626
 
701
- # Build bundle
702
- vkit policy bundle --output bundle.tar.gz
627
+ ### Schema Discovery Commands
703
628
 
704
- # Deploy to active control plane
705
- vkit policy deploy --bundle bundle.tar.gz
629
+ #### `vkit scan`
630
+
631
+ Scan a datasource and detect schema drift.
632
+
633
+ ```bash
634
+ vkit scan <DATASOURCE_ID> [--mode MODE]
635
+ ```
636
+
637
+ **Options:**
638
+ - `--mode` — Scan mode: `diff`, `apply` (default: `diff`)
639
+ - `--format` — Output format: `json`, `table` (default: `table`)
640
+
641
+ **Modes:**
642
+
643
+ | Mode | Description |
644
+ |------|-------------|
645
+ | `diff` | Show changes without applying (safe, read-only) |
646
+ | `apply` | Update baseline registry with discovered schema |
647
+
648
+ **Examples:**
649
+
650
+ **Discover Changes:**
651
+ ```bash
652
+ vkit scan production_pg
653
+
654
+ # Output:
655
+ # Scanning datasource: production_pg
656
+ #
657
+ # Schema Drift Detected:
658
+ #
659
+ # + Dataset: customers
660
+ # + Field: phone_number (varchar) [PII] ⚠️ NEW - requires policy
661
+ # ~ Field: email (text → varchar) [PII] ✓ Already masked
662
+ #
663
+ # + Dataset: user_sessions
664
+ # + Field: ip_address (inet) [PII] ⚠️ NEW - requires policy
665
+ # + Field: user_agent (text) [INTERNAL] ⚠️ NEW - requires policy
666
+ #
667
+ # Summary: 2 new datasets, 3 new fields, 1 modified field
668
+ ```
669
+
670
+ **Apply Changes:**
671
+ ```bash
672
+ vkit scan production_pg --mode apply
673
+
674
+ # Output:
675
+ # ✓ Baseline registry updated
676
+ # Next steps:
677
+ # 1. Review new fields: customers.phone_number, user_sessions.ip_address
678
+ # 2. Update policies in Git: config/policies/
679
+ # 3. Build new bundle: vkit policy bundle
680
+ # 4. Deploy bundle: vkit policy deploy
681
+ ```
682
+
683
+ **Scan All Datasources:**
684
+ ```bash
685
+ for ds in $(vkit datasource list --format json | jq -r '.[].id'); do
686
+ echo "Scanning $ds..."
687
+ vkit scan $ds
688
+ done
706
689
  ```
707
690
 
708
691
  ---
709
692
 
710
- ## 🎯 Use Cases
693
+ ### Policy Management Commands
711
694
 
712
- ### 1. **Secure AI Agent Access**
713
- Enable LLM-powered agents to query production databases without exposing credentials. VaultKit enforces policies that restrict which tables and columns AI agents can access, with automatic masking of sensitive fields.
695
+ #### `vkit policy bundle`
714
696
 
715
- **Problem:**
716
- - AI agents need data context for decision-making
717
- - Direct database access creates security risks
718
- - Credentials in AI systems can leak
719
- - Uncontrolled queries can expose sensitive data
697
+ Compile policies and registry into a deployable bundle.
720
698
 
721
- **VaultKit Solution:**
699
+ ```bash
700
+ vkit policy bundle [OPTIONS]
701
+ ```
722
702
 
723
- ```yaml
724
- # policies/ai_agent_restrictions.yaml
725
- id: ai_agent_restrictions
726
- match:
727
- fields:
728
- category: pii
703
+ **Options:**
704
+ - `--policies_dir` — Path to policies directory (default: `config/policies`)
705
+ - `--registry_dir` — Path to registry files (default: `config`)
706
+ - `--datasources_dir` — Path to datasource configs (default: `config/datasources`)
707
+ - `--out` — Output bundle file (default: `dist/policy_bundle.json`)
708
+ - `--org` — Organization identifier (optional, defaults to logged-in user's org)
709
+
710
+ **Example:**
711
+ ```bash
712
+ # Using logged-in user's org
713
+ vkit policy bundle \
714
+ --policies_dir config/policies \
715
+ --registry_dir config \
716
+ --datasources_dir config/datasources \
717
+ --out dist/policy_bundle.json
718
+
719
+ # Specifying org explicitly
720
+ vkit policy bundle \
721
+ --policies_dir config/policies \
722
+ --registry_dir config \
723
+ --datasources_dir config/datasources \
724
+ --out dist/policy_bundle.json \
725
+ --org acme
726
+ ```
727
+
728
+ #### `vkit policy validate`
729
+
730
+ Validate a policy bundle without deploying.
731
+
732
+ ```bash
733
+ vkit policy validate --bundle <BUNDLE_FILE>
734
+ ```
729
735
 
730
- context:
731
- requester_role: ai_agent
732
- environment: production
736
+ **Example:**
737
+ ```bash
738
+ vkit policy validate --bundle dist/policy_bundle.json
739
+
740
+ # Output:
741
+ # Validating policy bundle...
742
+ # ✓ Schema validation passed
743
+ # ✓ Policy syntax valid
744
+ # ✓ No circular dependencies
745
+ # ✓ All datasources referenced
746
+ # Bundle is valid and ready to deploy
747
+ ```
748
+
749
+ #### `vkit policy deploy`
733
750
 
734
- action:
735
- mask: true
736
- mask_type: hash
737
- reason: "PII must be hashed for AI agents"
738
- ttl: "15m"
751
+ Deploy a policy bundle to the control plane.
752
+
753
+ ```bash
754
+ vkit policy deploy --bundle <BUNDLE_FILE> [OPTIONS]
739
755
  ```
740
756
 
741
- **How it works:**
742
- 1. AI agents receive short-lived session tokens (15 minutes)
743
- 2. All PII fields automatically hashed before results return
744
- 3. Agents can analyze patterns without accessing raw sensitive data
745
- 4. Every query logged with AI agent identity
757
+ **Options:**
758
+ - `--bundle` Path to bundle file (required)
759
+ - `--org` Organization identifier (optional, defaults to logged-in user's org)
760
+ - `--activate` Immediately activate bundle
761
+ - `--dry-run` Validate deployment without activating
746
762
 
747
- **CLI Usage:**
763
+ **Examples:**
748
764
  ```bash
749
- # AI agent requests data
750
- vkit request \
751
- --datasource production_db \
752
- --role ai_agent \
753
- --aql '{
754
- "source_table": "customer_behavior",
755
- "columns": ["user_id", "email", "purchase_amount"],
756
- "filters": [{"field": "purchase_amount", "operator": "gt", "value": 1000}]
757
- }'
765
+ # Deploy and activate (using logged-in user's org)
766
+ vkit policy deploy \
767
+ --bundle dist/policy_bundle.json \
768
+ --activate
769
+
770
+ # Deploy with explicit org
771
+ vkit policy deploy \
772
+ --bundle dist/policy_bundle.json \
773
+ --org acme \
774
+ --activate
775
+
776
+ # Dry run (test deployment)
777
+ vkit policy deploy \
778
+ --bundle dist/policy_bundle.json \
779
+ --dry-run
780
+ ```
781
+
782
+ #### `vkit policy list`
758
783
 
759
- # Result: email field is hashed
760
- # {
761
- # "user_id": 12345,
762
- # "email": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
763
- # "purchase_amount": 1250.00
764
- # }
784
+ List active policy bundles.
785
+
786
+ ```bash
787
+ vkit policy list
788
+ ```
789
+
790
+ #### `vkit policy show`
791
+
792
+ Show details of a specific policy.
793
+
794
+ ```bash
795
+ vkit policy show <POLICY_ID>
765
796
  ```
766
797
 
767
798
  ---
768
799
 
769
- ### 2. **Cross-Region Compliance (GDPR, CCPA, HIPAA)**
770
- Automatically mask or deny access to sensitive fields based on user location and data residency requirements.
800
+ ### Audit Commands
771
801
 
772
- **Problem:**
773
- - GDPR prohibits transferring EU citizen data outside EU
774
- - CCPA requires disclosure of data access by California residents
775
- - HIPAA restricts PHI access based on business need
776
- - Manual compliance is error-prone and doesn't scale
802
+ #### `vkit audit query`
777
803
 
778
- **VaultKit Solution:**
804
+ Search audit logs.
779
805
 
780
- ```yaml
781
- # policies/gdpr_protection.yaml
782
- id: gdpr_cross_region_deny
783
- match:
784
- dataset: eu_customers
785
- fields:
786
- category: pii
787
-
788
- context:
789
- requester_region: US
790
- dataset_region: EU
791
-
792
- action:
793
- deny: true
794
- reason: "Cross-region PII access forbidden due to GDPR Article 44"
795
- audit_flag: compliance_violation
806
+ ```bash
807
+ vkit audit query [OPTIONS]
808
+ ```
809
+
810
+ **Options:**
811
+ - `--user` — Filter by user email
812
+ - `--action` — Filter by action: `granted`, `denied`, `pending`
813
+ - `--dataset` — Filter by dataset name
814
+ - `--days` — Look back N days (default: 7)
815
+ - `--limit` — Number of results (default: 100)
816
+ - `--format` — Output format: `json`, `table`, `csv`
817
+
818
+ **Examples:**
819
+ ```bash
820
+ # All queries by a user
821
+ vkit audit query --user analyst@company.com --days 30
822
+
823
+ # Denied access attempts
824
+ vkit audit query --action denied --days 7
825
+
826
+ # Access to specific dataset
827
+ vkit audit query --dataset customers --action granted
828
+
829
+ # Export to CSV
830
+ vkit audit query --user analyst@company.com --format csv --output audit_report.csv
831
+ ```
832
+
833
+ #### `vkit audit export`
834
+
835
+ Export audit logs for compliance reporting.
836
+
837
+ ```bash
838
+ vkit audit export --start-date <DATE> --end-date <DATE> [OPTIONS]
839
+ ```
840
+
841
+ **Options:**
842
+ - `--start-date` — Start date (YYYY-MM-DD)
843
+ - `--end-date` — End date (YYYY-MM-DD)
844
+ - `--format` — Output format: `json`, `csv` (default: `csv`)
845
+ - `--output` — Output file path
846
+
847
+ **Example:**
848
+ ```bash
849
+ vkit audit export \
850
+ --start-date 2024-01-01 \
851
+ --end-date 2024-01-31 \
852
+ --format csv \
853
+ --output compliance_reports/january_2024.csv
854
+ ```
796
855
 
797
856
  ---
798
- id: ccpa_disclosure_logging
799
- match:
800
- dataset: california_residents
801
857
 
802
- context:
803
- requester_region: ANY
858
+ ## 🔬 Advanced Usage
804
859
 
805
- action:
806
- allow: true
807
- ttl: "1h"
808
- audit_metadata:
809
- ccpa_disclosure: true
810
- data_subject_rights: "User has right to know about this access"
860
+ ### Scripting with vkit
861
+
862
+ **Batch Request Processing:**
863
+
864
+ ```bash
865
+ #!/bin/bash
866
+ # process_requests.sh
867
+
868
+ REQUESTS=(
869
+ '{"source_table":"customers","columns":["id","email"],"limit":100}'
870
+ '{"source_table":"orders","columns":["order_id","total"],"limit":100}'
871
+ '{"source_table":"products","columns":["product_id","name"],"limit":100}'
872
+ )
873
+
874
+ for aql in "${REQUESTS[@]}"; do
875
+ echo "Processing: $aql"
876
+ vkit request --aql "$aql" --format json | jq -r '.grant_id' >> grants.txt
877
+ done
878
+
879
+ echo "Generated grants:"
880
+ cat grants.txt
811
881
  ```
812
882
 
813
- **How it works:**
814
- - US-based analysts querying EU customer data are automatically denied
815
- - Policy engine evaluates both requester and dataset regions
816
- - Audit log captures denied attempts with compliance flags
817
- - CCPA-flagged queries generate disclosure reports
883
+ **Automated Approvals (for testing):**
818
884
 
819
- **Multi-Region Access Pattern:**
820
885
  ```bash
821
- # US analyst attempts to query EU data
822
- vkit request --datasource eu_customers_db --aql '{...}'
886
+ #!/bin/bash
887
+ # auto_approve_pending.sh
823
888
 
824
- # Response:
825
- # ❌ Access Denied
826
- # Reason: Cross-region PII access forbidden due to GDPR Article 44
827
- # Policy: gdpr_cross_region_deny
828
- # Audit ID: audit_20240115_abc123
889
+ PENDING=$(vkit approval:list --state pending --format json)
890
+
891
+ echo "$PENDING" | jq -r '.[].id' | while read -r request_id; do
892
+ echo "Approving request: $request_id"
893
+ vkit approval:approve "$request_id" --ttl 3600 --notes "Auto-approved for testing"
894
+ done
895
+ ```
896
+
897
+ ### Integration with CI/CD
898
+
899
+ **GitHub Actions Example:**
900
+
901
+ ```yaml
902
+ name: Deploy VaultKit Policies
903
+
904
+ on:
905
+ push:
906
+ branches: [main]
907
+ paths:
908
+ - 'config/policies/**'
909
+ - 'config/registry.yaml'
910
+
911
+ jobs:
912
+ deploy:
913
+ runs-on: ubuntu-latest
914
+ steps:
915
+ - uses: actions/checkout@v3
916
+
917
+ - name: Install vkit CLI
918
+ run: gem install vaultkitcli
919
+
920
+ - name: Authenticate
921
+ run: |
922
+ vkit login \
923
+ --endpoint ${{ secrets.VKIT_API_URL }} \
924
+ --email ${{ secrets.VKIT_BOT_EMAIL }} \
925
+ --password ${{ secrets.VKIT_BOT_PASSWORD }}
926
+
927
+ - name: Build Policy Bundle
928
+ run: |
929
+ vkit policy bundle \
930
+ --policies_dir config/policies \
931
+ --registry_dir config \
932
+ --out policy_bundle.json \
933
+ --org ${{ secrets.ORG_ID }}
934
+
935
+ - name: Validate Bundle
936
+ run: vkit policy validate --bundle policy_bundle.json
937
+
938
+ - name: Deploy Bundle
939
+ run: |
940
+ vkit policy deploy \
941
+ --bundle policy_bundle.json \
942
+ --org ${{ secrets.ORG_ID }} \
943
+ --activate
944
+ ```
945
+
946
+ ### Using jq for Advanced Filtering
947
+
948
+ **Extract specific fields:**
949
+ ```bash
950
+ vkit requests list --format json | jq '.[] | select(.state == "approved") | {id, dataset, requester}'
951
+ ```
952
+
953
+ **Count requests by state:**
954
+ ```bash
955
+ vkit requests list --format json | jq 'group_by(.state) | map({state: .[0].state, count: length})'
956
+ ```
957
+
958
+ **Find high-value transactions:**
959
+ ```bash
960
+ vkit fetch --grant gr_abc123 --format json | jq '.data[] | select(.amount > 10000)'
829
961
  ```
830
962
 
831
963
  ---
832
964
 
833
- ### 3. **Just-In-Time Access (Break-Glass)**
834
- Developers can request temporary elevated access to production data with automatic approval workflows and audit trails.
965
+ ## 🐛 Troubleshooting
966
+
967
+ ### Common Issues
968
+
969
+ #### Authentication Failed
835
970
 
836
971
  **Problem:**
837
- - Engineers need emergency production access for incident resolution
838
- - Standing privileges violate principle of least privilege
839
- - Manual approval processes slow incident response
840
- - Audit trails are incomplete or manual
972
+ ```
973
+ Error: Authentication failed (401 Unauthorized)
974
+ ```
841
975
 
842
- **VaultKit Solution:**
976
+ **Solutions:**
977
+ ```bash
978
+ # 1. Check endpoint
979
+ echo $VKIT_API_URL
843
980
 
844
- ```yaml
845
- # policies/financial_requires_approval.yaml
846
- id: financial_break_glass
847
- match:
848
- fields:
849
- category: financial
850
-
851
- context:
852
- environment: production
853
- requester_role: engineer
854
-
855
- action:
856
- require_approval: true
857
- approver_role: finance_manager
858
- approval_metadata_required:
859
- - incident_ticket
860
- - business_justification
861
- ttl: "1h"
862
- reason: "Financial data access requires approval"
863
- auto_revoke: true
864
- ```
865
-
866
- **CLI Usage:**
867
- ```bash
868
- # Engineer requests emergency access
869
- vkit request \
870
- --datasource production_db \
871
- --approval-required \
872
- --reason "Investigating payment processing failure - Ticket INC-5432" \
873
- --metadata '{"incident_ticket": "INC-5432", "business_justification": "Payment processor reporting transaction mismatch"}' \
874
- --duration "2h" \
875
- --aql '{
876
- "source_table": "transactions",
877
- "columns": ["transaction_id", "amount", "status"],
878
- "filters": [{"field": "status", "operator": "eq", "value": "failed"}]
879
- }'
981
+ # 2. Re-authenticate
982
+ vkit logout
983
+ vkit login --endpoint http://localhost:3000 --email you@company.com
880
984
 
881
- # Response:
882
- # ⏳ Approval Required
883
- # Request ID: req_20240115_xyz789
884
- # Status: PENDING
885
- # Approver: finance_manager
886
- # Expires: 2024-01-15 18:30 UTC
985
+ # 3. Verify credentials
986
+ vkit whoami
887
987
  ```
888
988
 
889
- **Approval Workflow:**
989
+ #### Connection Refused
990
+
991
+ **Problem:**
992
+ ```
993
+ Error: Connection refused - connect(2) for "localhost" port 3000
994
+ ```
995
+
996
+ **Solutions:**
890
997
  ```bash
891
- # Finance manager reviews request
892
- vkit approval list --pending
998
+ # 1. Check if control plane is running
999
+ curl http://localhost:3000/health
893
1000
 
894
- # Approve with notes
895
- vkit approval grant \
896
- --request-id req_20240115_xyz789 \
897
- --notes "Approved for incident resolution. Access limited to failed transactions only."
1001
+ # 2. Verify Docker containers
1002
+ cd infra
1003
+ docker compose ps
898
1004
 
899
- # Engineer notified and can fetch data
900
- vkit fetch --grant grant_approved_abc123
1005
+ # 3. Start services
1006
+ docker compose up
901
1007
  ```
902
1008
 
903
- **Audit Trail Output:**
904
- ```json
905
- {
906
- "request_id": "req_20240115_xyz789",
907
- "requester": "engineer@company.com",
908
- "requester_role": "engineer",
909
- "requested_at": "2024-01-15T16:00:00Z",
910
- "approved_by": "finance.manager@company.com",
911
- "approved_at": "2024-01-15T16:05:32Z",
912
- "approval_reason": "Approved for incident resolution",
913
- "access_granted": "2024-01-15T16:05:32Z",
914
- "access_expires": "2024-01-15T18:05:32Z",
915
- "queries_executed": 3,
916
- "rows_accessed": 147,
917
- "incident_ticket": "INC-5432",
918
- "auto_revoked": true
919
- }
1009
+ #### Grant Expired
1010
+
1011
+ **Problem:**
1012
+ ```
1013
+ Error: Grant has expired
1014
+ Grant ID: gr_abc123xyz
1015
+ Expired at: 2024-01-15 12:00:00 UTC
920
1016
  ```
921
1017
 
922
- ---
1018
+ **Solutions:**
1019
+ ```bash
1020
+ # 1. Request new grant
1021
+ vkit request --aql '{"source_table":"customers",...}'
1022
+
1023
+ # 2. Request longer TTL (if approval required)
1024
+ vkit request --aql '{...}' --ttl 7200
1025
+ ```
923
1026
 
924
- ### 4. **Time-Restricted Access Windows**
925
- Enforce access controls based on business hours, maintenance windows, or compliance requirements.
1027
+ #### Policy Denial
926
1028
 
927
1029
  **Problem:**
928
- - Audit logs should only be accessible during business hours
929
- - Production data access outside work hours indicates potential misuse
930
- - Compliance requires time-based controls for certain datasets
1030
+ ```
1031
+ Request denied
1032
+ Reason: Cross-region PII access forbidden
1033
+ Policy: gdpr_protection
1034
+ ```
931
1035
 
932
- **VaultKit Solution:**
1036
+ **Solutions:**
1037
+ 1. Review policy restrictions
1038
+ 2. Request approval if available
1039
+ 3. Modify query to exclude restricted fields
1040
+ 4. Contact admin to update policies
933
1041
 
934
- ```yaml
935
- # policies/time_restricted_access.yaml
936
- id: business_hours_only
937
- match:
938
- dataset: audit_logs
939
- environment: production
940
-
941
- context:
942
- time:
943
- start: "08:00"
944
- end: "18:00"
945
- timezone: "America/Toronto"
946
- days: ["monday", "tuesday", "wednesday", "thursday", "friday"]
947
-
948
- action:
949
- allow: true
950
- reason: "Access allowed during business hours"
951
- ttl: "1h"
1042
+ ### Debug Mode
1043
+
1044
+ Enable verbose logging:
1045
+
1046
+ ```bash
1047
+ # Set debug environment variable
1048
+ export VKIT_DEBUG=true
1049
+
1050
+ # Run command
1051
+ vkit request --aql '{...}'
1052
+
1053
+ # Output includes:
1054
+ # - HTTP request/response details
1055
+ # - Policy evaluation trace
1056
+ # - Timing information
1057
+ ```
1058
+
1059
+ ### Health Check
1060
+
1061
+ Verify CLI and control plane connectivity:
1062
+
1063
+ ```bash
1064
+ # Check CLI version
1065
+ vkit --version
1066
+
1067
+ # Check control plane health
1068
+ curl $VKIT_API_URL/health
1069
+
1070
+ # Test authentication
1071
+ vkit whoami
1072
+ ```
1073
+
1074
+ ---
1075
+
1076
+ ## 🤝 Contributing
1077
+
1078
+ We welcome contributions to the VaultKit CLI!
1079
+
1080
+ ### Development Setup
1081
+
1082
+ ```bash
1083
+ # Clone repository
1084
+ git clone https://github.com/yourorg/vaultkit-cli.git
1085
+ cd vaultkit-cli
1086
+
1087
+ # Install dependencies
1088
+ bundle install
1089
+
1090
+ # Run tests
1091
+ bundle exec rspec
1092
+
1093
+ # Run CLI locally
1094
+ bundle exec bin/vkit --help
1095
+ ```
1096
+
1097
+ ### Running Tests
1098
+
1099
+ ```bash
1100
+ # Unit tests
1101
+ bundle exec rspec spec/unit
1102
+
1103
+ # Integration tests (requires running control plane)
1104
+ bundle exec rspec spec/integration
1105
+
1106
+ # All tests
1107
+ bundle exec rspec
1108
+ ```
1109
+
1110
+ ### Code Style
1111
+
1112
+ ```bash
1113
+ # Run RuboCop
1114
+ bundle exec rubocop
1115
+
1116
+ # Auto-fix issues
1117
+ bundle exec rubocop -a
1118
+ ```
1119
+
1120
+ ---
1121
+
1122
+ ## 📚 Additional Resources
1123
+
1124
+ - **Main Repository**: [github.com/ndbaba1/vaultkitcli.git](https://github.com/ndbaba1/vaultkitcli.git)
1125
+ - **Documentation**: [docs.vaultkit.io](https://docs.vaultkit.io)
1126
+ - **AQL Specification**: [docs.vaultkit.io/aql](https://docs.vaultkit.io/aql)
1127
+ - **Policy Reference**: [docs.vaultkit.io/policies](https://docs.vaultkit.io/policies)
1128
+ - **API Documentation**: [docs.vaultkit.io/api](https://docs.vaultkit.io/api)
952
1129
 
953
1130
  ---
954
- id: after_hours_deny
955
- match:
956
- dataset: audit_logs
957
- environment: production
958
-
959
- context:
960
- time:
961
- outside_window
1131
+
1132
+ ## 📄 License
1133
+
1134
+ VaultKit CLI is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.
1135
+
1136
+ ---
1137
+
1138
+ **Built with ❤️ by the VaultKit team**