atabey-shared 0.0.37 → 0.1.0

package/README.md

@@ -1,26 +1,24 @@
 # [GOV] Atabey Shared — Core Utility Library for AI Governance Platform
 
-[![Version](https://img.shields.io/badge/Version-v0.0.23-blue.svg)](https://github.com/ysf-bkr/atabey)
+[![Version](https://img.shields.io/badge/Version-v0.1.0-blue.svg)](https://github.com/ysf-bkr/atabey)
 [![npm](https://img.shields.io/npm/v/atabey-shared)](https://www.npmjs.com/package/atabey-shared)
 [![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
 
-**Atabey Shared** is the foundational utility package of the Agent Atabey framework. It provides the shared codebase, data structures, constants, security filters, database holders, logging layers, and locking mechanisms used by both `atabey` (Core CLI/Orchestrator) and `atabey-mcp` (MCP Server).
+**Atabey Shared** is the foundational utility package of the Atabey framework. It provides the shared codebase, data structures, constants, security filters, database holders, logging layers, and locking mechanisms used by both `atabey` (Core CLI/Orchestrator) and `atabey-mcp` (MCP Server).
 
 ---
 
 ## 📋 Package Overview
 
-| Module | Files | Core Responsibilities |
-|--------|-------|-----------------------|
-| **PII Masking** | `pii.ts` | Regex-based sanitization of 20+ sensitive patterns (emails, phones, credit cards, IP addresses, JWT tokens, API keys) |
-| **Distributed Locking** | `lock.ts` | Multi-process file-based lock implementation with stale lock detection (10s TTL) and retry backoff |
-| **Enterprise Logger** | `logger.ts` | Multi-destination structured logging (Stdout/Stderr + local files) supporting min-level and JSON formatting |
-| **Data Retention** | `retention.ts` | KVKK/GDPR-compliant telemetry and log pruning based on 30-day and 90-day retention policies |
-| **Audit Logs** | `audit.ts` | Unified JSON audit trail capturing agent actions, timestamps, trace IDs, and results |
-| **Error Handling** | `errors.ts` | Custom error classes (`AtabeyBaseError`, `LockError`, `ComplianceError`) preserving stack traces |
-| **Constants & Types** | `constants.ts`, `types.ts` | Centralized constants (Phase paths, candidate directories, MCP constants) and strictly branded types |
-| **File System** | `fs.ts` | Safe file write wrapper with automatic parent directory creation and encoding validation |
-| **Database Holder** | `database.ts` | Singleton SQLite database holder for shared workspace storage |
+| Module group | Files | Responsibility |
+|--------------|-------|----------------|
+| **primitives** | `constants`, `types`, `errors`, `fs`, `string`, `time`, `outbound-http` | Framework constants, branded types, error classes, safe fs, TLS-aware outbound HTTP |
+| **storage** | `database`, `storage`, `migrations`, `memory-prune` | SQLite holder (singleton), schema migrations, memory pruning |
+| **security** | `pii`, `secrets`, `tls`, `policy-gate`, `retention` | PII masking, AES-256-GCM vault, TLS/mTLS, fail-closed PolicyGate, retention |
+| **concurrency** | `lock`, `file-lock`, `job-queue`, `worker-pool`, `worker-thread`, `resilience` | Distributed/file locks, DB-backed job queue, worker pool, circuit breaker |
+| **sandbox** | `sandbox`, `sandbox-runtime`, `sandbox-fs` | OS sandbox (seatbelt/uid), fail-closed sandboxed file writes |
+| **observability** | `logger`, `otel`, `audit`, `audit-chain` | Structured logging, OTel spans, HMAC-signed audit chain |
+| **agent** | `agent-identity`, `api-catalog`, `tokenizer` | Signed agent identity, API endpoint catalog, `cl100k_base` tokenizer |
 
 ---
 
@@ -34,6 +32,44 @@ npm install atabey-shared
 
 ---
 
+## 🧩 Module Structure
+
+The package is organized into focused **module groups**. `src/index.ts` is a barrel
+that re-exports everything flat **plus** a `Modules.<Group>` namespace per group;
+the flat `atabey-shared/*.js` entry points are kept as thin re-export shims for
+backward compatibility.
+
+```
+packages/shared/
+├── src/
+│   ├── index.ts                # Barrel: flat exports + `Modules.<Group>`
+│   └── modules/
+│       ├── primitives/         # constants, types, errors, fs, string, time, outbound-http
+│       ├── storage/            # database, storage, migrations, memory-prune
+│       ├── security/           # pii, secrets, tls, policy-gate, retention
+│       ├── concurrency/        # lock, file-lock, job-queue, worker-pool, worker-thread, resilience
+│       ├── sandbox/            # sandbox, sandbox-runtime, sandbox-fs
+│       ├── observability/      # logger, otel, audit, audit-chain
+│       └── agent/              # agent-identity, api-catalog, tokenizer
+```
+
+Three import styles are supported:
+
+```typescript
+// 1. Flat — stable public API
+import { maskText, logger } from "atabey-shared";
+import { maskText } from "atabey-shared/pii.js";
+
+// 2. Namespaced by module group
+import { Modules } from "atabey-shared";
+Modules.Security.maskText("user@example.com");
+
+// 3. Deep module subpath
+import { maskText } from "atabey-shared/security/pii.js";
+```
+
+---
+
 ## ⚙️ Core Modules Detail
 
 ### 1. PII Protection (KVKK/GDPR Compliance)
@@ -48,23 +84,26 @@ console.log(sanitizedText);
 ```
 
 ### 2. Distributed Resource Locking
-The `lock.ts` module prevents race conditions between parallel running agents writing to shared resources (e.g. `PROJECT_MEMORY.md`).
+The `lock.ts` module provides a SQLite-backed `DistributedLock` (server-authoritative,
+with stale-lock reclaim) that prevents race conditions between parallel agents writing
+to shared resources (e.g. `PROJECT_MEMORY.md`). For a lightweight file-based lock, use
+`AtomicFileLock` from `file-lock.ts`.
 ```typescript
-import { AtabeyLock } from "atabey-shared";
+import { DistributedLock } from "atabey-shared";
 
-const lock = new AtabeyLock("project-memory", { ttlMs: 10000, retryCount: 3 });
+DistributedLock.initialize(); // once at startup
 
-try {
-    const acquired = await lock.acquire();
-    if (acquired) {
-        // Safe region: write to file
+const owner = DistributedLock.getLocalOwnerId();
+if (DistributedLock.acquire("PROJECT_MEMORY.md", owner, { ttlMs: 10000, reason: "memory sync" })) {
+    try {
+        // safe region: write the shared resource
+    } finally {
+        DistributedLock.release("PROJECT_MEMORY.md", owner);
     }
-} finally {
-    await lock.release();
 }
 ```
 
-### 3. Enterprise Logger
+### 3. Structured Logger
 Structured logger wrapping `process.stdout` and `process.stderr` supporting log file logging and JSON output format.
 ```typescript
 import { logger, LogLevel } from "atabey-shared";
@@ -91,8 +130,8 @@ const task: TaskID = asTaskID("TASK-102");
 
 ## 📦 Dependencies
 
-- **better-sqlite3**: SQLite database engine for storage shims
-- **typescript**: Type safety enforcement
+- **js-tiktoken**: `cl100k_base` BPE tokenizer for token/cost accounting
+- **better-sqlite3**, **chalk**, **zod** _(peer)_: storage, terminal styling and runtime validation
 
 ---
 

package/dist/agent-identity.d.ts

@@ -0,0 +1 @@
+export * from "./modules/agent/agent-identity.js";

package/dist/agent-identity.js

@@ -0,0 +1,2 @@
+export * from "./modules/agent/agent-identity.js";
+//# sourceMappingURL=agent-identity.js.map

package/dist/agent-identity.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"agent-identity.js","sourceRoot":"","sources":["../src/agent-identity.ts"],"names":[],"mappings":"AAAA,cAAc,mCAAmC,CAAC"}

package/dist/api-catalog.d.ts

@@ -0,0 +1 @@
+export * from "./modules/agent/api-catalog.js";

package/dist/api-catalog.js

@@ -0,0 +1,2 @@
+export * from "./modules/agent/api-catalog.js";
+//# sourceMappingURL=api-catalog.js.map

package/dist/api-catalog.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"api-catalog.js","sourceRoot":"","sources":["../src/api-catalog.ts"],"names":[],"mappings":"AAAA,cAAc,gCAAgC,CAAC"}

package/dist/audit-chain.d.ts

@@ -1,51 +1 @@
-/**
- * Tamper-evident audit hash chain helpers (Phase 1.4).
- *
- * Chain: each entry stores prev_hash (previous entry hash or GENESIS) and hash = SHA-256(payload).
- * Any mutation of historical rows fails verification.
- */
-export declare const AUDIT_CHAIN_GENESIS = "GENESIS";
-export declare function sha256Hex(payload: string): string;
-/**
- * Canonical payload for agent execution logs (Storage.saveLog).
- * MUST stay stable — changing it breaks verification of existing DBs.
- */
-export declare function buildAgentLogChainPayload(fields: {
-    prevHash: string;
-    agent: string;
-    action: string;
-    traceId: string;
-    status: string;
-    summary: string;
-    timestamp: string;
-}): string;
-/**
- * Canonical payload for structured Audit.log entries.
- */
-export declare function buildStructuredAuditChainPayload(fields: {
-    prevHash: string;
-    agent: string;
-    action: string;
-    status: string;
-    traceId: string;
-    timestamp: string;
-    detailsJson: string;
-    errorMessage: string;
-}): string;
-export interface ChainRow {
-    id: number;
-    prev_hash: string | null;
-    hash: string | null;
-}
-export interface ChainVerifyResult {
-    valid: boolean;
-    checked: number;
-    failedLogId?: number;
-    reason?: string;
-    tip?: string;
-}
-/**
- * Verify a sequence of rows already ordered by id ASC.
- * `computePayload(row, prevHash)` must rebuild the exact string used at insert time.
- */
-export declare function verifyHashChain<T extends ChainRow>(rows: T[], computePayload: (row: T, prevHash: string) => string): ChainVerifyResult;
+export * from "./modules/observability/audit-chain.js";

package/dist/audit-chain.js

@@ -1,85 +1,2 @@
-/**
- * Tamper-evident audit hash chain helpers (Phase 1.4).
- *
- * Chain: each entry stores prev_hash (previous entry hash or GENESIS) and hash = SHA-256(payload).
- * Any mutation of historical rows fails verification.
- */
-import crypto from "crypto";
-export const AUDIT_CHAIN_GENESIS = "GENESIS";
-export function sha256Hex(payload) {
-    return crypto.createHash("sha256").update(payload, "utf8").digest("hex");
-}
-/**
- * Canonical payload for agent execution logs (Storage.saveLog).
- * MUST stay stable — changing it breaks verification of existing DBs.
- */
-export function buildAgentLogChainPayload(fields) {
-    return [
-        fields.prevHash,
-        fields.agent,
-        fields.action,
-        fields.traceId,
-        fields.status,
-        fields.summary,
-        fields.timestamp,
-    ].join("|");
-}
-/**
- * Canonical payload for structured Audit.log entries.
- */
-export function buildStructuredAuditChainPayload(fields) {
-    return [
-        fields.prevHash,
-        fields.agent,
-        fields.action,
-        fields.status,
-        fields.traceId,
-        fields.timestamp,
-        fields.detailsJson,
-        fields.errorMessage,
-    ].join("|");
-}
-/**
- * Verify a sequence of rows already ordered by id ASC.
- * `computePayload(row, prevHash)` must rebuild the exact string used at insert time.
- */
-export function verifyHashChain(rows, computePayload) {
-    let expectedPrev = rows.length > 0 ? (rows[0].prev_hash || AUDIT_CHAIN_GENESIS) : AUDIT_CHAIN_GENESIS;
-    let checked = 0;
-    for (const row of rows) {
-        checked++;
-        const prev = row.prev_hash || "";
-        if (prev !== expectedPrev) {
-            return {
-                valid: false,
-                checked,
-                failedLogId: row.id,
-                reason: `prev_hash mismatch at row ${row.id}`,
-                tip: "A log was deleted, reordered, or inserted out of chain order.",
-            };
-        }
-        if (!row.hash) {
-            return {
-                valid: false,
-                checked,
-                failedLogId: row.id,
-                reason: `Missing hash at row ${row.id}`,
-                tip: "Legacy row without hash; re-seed or accept and re-hash from tip.",
-            };
-        }
-        const payload = computePayload(row, prev);
-        const calculated = sha256Hex(payload);
-        if (calculated !== row.hash) {
-            return {
-                valid: false,
-                checked,
-                failedLogId: row.id,
-                reason: `Hash corruption at row ${row.id}`,
-                tip: "Row content was modified after append.",
-            };
-        }
-        expectedPrev = row.hash;
-    }
-    return { valid: true, checked };
-}
+export * from "./modules/observability/audit-chain.js";
 //# sourceMappingURL=audit-chain.js.map

package/dist/audit-chain.js.map

@@ -1 +1 @@
-{"version":3,"file":"audit-chain.js","sourceRoot":"","sources":["../src/audit-chain.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,MAAM,MAAM,QAAQ,CAAC;AAE5B,MAAM,CAAC,MAAM,mBAAmB,GAAG,SAAS,CAAC;AAE7C,MAAM,UAAU,SAAS,CAAC,OAAe;IACrC,OAAO,MAAM,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;AAC7E,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,yBAAyB,CAAC,MAQzC;IACG,OAAO;QACH,MAAM,CAAC,QAAQ;QACf,MAAM,CAAC,KAAK;QACZ,MAAM,CAAC,MAAM;QACb,MAAM,CAAC,OAAO;QACd,MAAM,CAAC,MAAM;QACb,MAAM,CAAC,OAAO;QACd,MAAM,CAAC,SAAS;KACnB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAChB,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,gCAAgC,CAAC,MAShD;IACG,OAAO;QACH,MAAM,CAAC,QAAQ;QACf,MAAM,CAAC,KAAK;QACZ,MAAM,CAAC,MAAM;QACb,MAAM,CAAC,MAAM;QACb,MAAM,CAAC,OAAO;QACd,MAAM,CAAC,SAAS;QAChB,MAAM,CAAC,WAAW;QAClB,MAAM,CAAC,YAAY;KACtB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;AAChB,CAAC;AAgBD;;;GAGG;AACH,MAAM,UAAU,eAAe,CAC3B,IAAS,EACT,cAAoD;IAEpD,IAAI,YAAY,GAAG,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,IAAI,mBAAmB,CAAC,CAAC,CAAC,CAAC,mBAAmB,CAAC;IACtG,IAAI,OAAO,GAAG,CAAC,CAAC;IAEhB,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;QACrB,OAAO,EAAE,CAAC;QACV,MAAM,IAAI,GAAG,GAAG,CAAC,SAAS,IAAI,EAAE,CAAC;QACjC,IAAI,IAAI,KAAK,YAAY,EAAE,CAAC;YACxB,OAAO;gBACH,KAAK,EAAE,KAAK;gBACZ,OAAO;gBACP,WAAW,EAAE,GAAG,CAAC,EAAE;gBACnB,MAAM,EAAE,6BAA6B,GAAG,CAAC,EAAE,EAAE;gBAC7C,GAAG,EAAE,+DAA+D;aACvE,CAAC;QACN,CAAC;QACD,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC;YACZ,OAAO;gBACH,KAAK,EAAE,KAAK;gBACZ,OAAO;gBACP,WAAW,EAAE,GAAG,CAAC,EAAE;gBACnB,MAAM,EAAE,uBAAuB,GAAG,CAAC,EAAE,EAAE;gBACvC,GAAG,EAAE,kEAAkE;aAC1E,CAAC;QACN,CAAC;QACD,MAAM,OAAO,GAAG,cAAc,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;QAC1C,MAAM,UAAU,GAAG,SAAS,CAAC,OAAO,CAAC,CAAC;QACtC,IAAI,UAAU,KAAK,GAAG,CAAC,IAAI,EAAE,CAAC;YAC1B,OAAO;gBACH,KAAK,EAAE,KAAK;gBACZ,OAAO;gBACP,WAAW,EAAE,GAAG,CAAC,EAAE;gBACnB,MAAM,EAAE,0BAA0B,GAAG,CAAC,EAAE,EAAE;gBAC1C,GAAG,EAAE,wCAAwC;aAChD,CAAC;QACN,CAAC;QACD,YAAY,GAAG,GAAG,CAAC,IAAI,CAAC;IAC5B,CAAC;IAED,OAAO,EAAE,KAAK,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC;AACpC,CAAC"}
+{"version":3,"file":"audit-chain.js","sourceRoot":"","sources":["../src/audit-chain.ts"],"names":[],"mappings":"AAAA,cAAc,wCAAwC,CAAC"}

package/dist/audit.d.ts

@@ -1,113 +1 @@
-/**
- * [AUDIT] Structured Audit Log Service
- *
- * Records all critical operations in structured JSON format to SQLite.
- * Queryable, filterable, and traceable.
- *
- * [KVKK/GDPR] GDPR and KVKK Art. 4, 5, 11, 12 compliant:
- * - Sensitive data automatically masked (PII masking)
- * - Data retention limited (TTL)
- * - Data subject rights supported (erasure, query)
- *
- * Usage:
- *   Audit.log("user.create", "SUCCESS", { userId: "usr_123" });
- *   Audit.query({ action: "user.create", status: "FAILED" });
- */
-import { type ChainVerifyResult } from "./audit-chain.js";
-export type AuditStatus = "SUCCESS" | "FAILED" | "BLOCKED" | "APPROVED" | "REJECTED";
-export type DataCategory = "OPERATIONAL" | "USER_DATA" | "API_CALL" | "SECURITY" | "COMPLIANCE";
-export interface AuditEntry {
-    id?: number;
-    timestamp: string;
-    action: string;
-    status: AuditStatus;
-    agent: string;
-    traceId?: string;
-    details?: Record<string, unknown>;
-    errorMessage?: string;
-    durationMs?: number;
-    dataCategory?: DataCategory;
-    /** [KVKK] Data retention period (days). null = unlimited (not recommended) */
-    retentionDays?: number;
-    /** Tamper-evident chain (Phase 1.4) */
-    prev_hash?: string;
-    hash?: string;
-}
-export declare class Audit {
-    private static TABLE;
-    /** [KVKK] Default data retention period: 30 days */
-    private static DEFAULT_RETENTION_DAYS;
-    /** [KVKK] Maximum retention for sensitive categories: 90 days */
-    private static MAX_RETENTION_DAYS;
-    static initialize(): void;
-    /**
-     * Logs a structured audit entry.
-     * [KVKK] Sensitive data is automatically masked.
-     */
-    static log(action: string, status: AuditStatus, options?: {
-        agent?: string;
-        traceId?: string;
-        details?: Record<string, unknown>;
-        errorMessage?: string;
-        durationMs?: number;
-        dataCategory?: DataCategory;
-        /** [KVKK] Custom retention period (days). Default: 30 */
-        retentionDays?: number;
-    }): void;
-    /**
-     * Verify tamper-evident chain for structured audit_log table.
-     */
-    static verifyIntegrity(): ChainVerifyResult;
-    /**
-     * Queries audit logs with filters.
-     */
-    static query(options?: {
-        action?: string;
-        status?: AuditStatus;
-        agent?: string;
-        traceId?: string;
-        dataCategory?: DataCategory;
-        limit?: number;
-        offset?: number;
-        since?: string;
-        until?: string;
-    }): AuditEntry[];
-    /**
-   * [KVKK Art. 11] Deletes records upon data subject request.
-   * Clears all records matching a given traceId or agent.
-   *
-   * @param filter Which records to delete
-   * @returns Number of deleted records
-   */
-    static deleteByFilter(filter: {
-        traceId?: string;
-        agent?: string;
-        action?: string;
-        olderThanDays?: number;
-    }): number;
-    /**
-     * [KVKK/GDPR] Cleans expired records.
-     * Operates based on each audit record's retentionDays field.
-     * Can be auto-triggered by cron job or each Audit.log() call.
-     *
-     * @returns Number of cleaned records
-     */
-    static cleanExpired(): number;
-    /**
-     * Returns audit statistics.
-     */
-    static getStats(): {
-        total: number;
-        byStatus: Record<string, number>;
-        byAction: Record<string, number>;
-        byCategory: Record<string, number>;
-        recentErrors: AuditEntry[];
-        expiredCount: number;
-    };
-    /**
-     * [KVKK Art. 7] Clears all audit records.
-     * Used under the "Right to Erasure" provision.
-     * Only authorized users may call this.
-     */
-    static clearAll(confirmationCode: string): number;
-}
+export * from "./modules/observability/audit.js";