typesecure 0.1.0 → 0.2.3

package/README.md

@@ -1,21 +1,34 @@
 # typesecure
 
-A focused TypeScript cryptography package that provides secure encryption and hashing utilities with strong typing and runtime validation using Zod.
+`typesecure` is a **classification-first security core** for TypeScript projects.
+
+Instead of starting with crypto primitives, it starts with what actually causes most security incidents in web apps: **data leaving the boundary it should never cross** (logs, analytics, error trackers, headers, client bundles, etc).
+
+You “type” your data as `public | pii | secret | token | credential`, and `typesecure` helps you **enforce** safe handling using TypeScript + runtime checks.
 
 ## Features
 
-- 🔐 **Strong Typing**: Built with TypeScript for complete type safety.
-- ✅ **Runtime Validation**: Uses Zod to validate inputs and ensure security.
-- 🔍 **Advanced Encryption**: AES encryption with multiple modes (CBC, CTR, GCM, ECB).
-- 🛡️ **Authenticated Encryption**: GCM mode for authenticated encryption with additional data (AAD).
-- 🔏 **Cryptographic Hashing**: SHA-256, SHA-512, SHA-3, and more.
-- 📝 **HMAC Signatures**: Create and verify message authentication codes.
-- ⏱️ **Timing-Safe Comparison**: Prevent timing attacks with constant-time string comparison.
-- 🚦 **Security Level Assessment**: Analyze and report the security level of encryption configurations.
-- 🔑 **Password Hashing**: PBKDF2 for secure password hashing with salt and configurable iterations.
+- **Classification types**: `PublicString`, `PIIString`, `SecretString`, `TokenString`, `CredentialString`.
+- **Runtime validation**: Zod-backed constructors (`secretText()`, `piiText()`, ...).
+- **Redaction**: `redact()` and `safeJsonStringify()` prevent secret/PII leakage.
+- **Policy enforcement**: `defaultPolicy()`, `assertAllowed()`, `audit()` help block unsafe crossings.
+
+## Good for / Use when
+
+- **You need to stop leaks early**: preventing secrets/PII from ending up in logs, analytics, error trackers, or client bundles.
+- **You want safe defaults**: making insecure behavior harder than secure behavior.
+- **You want guardrails at the boundary**: before logging, emitting telemetry, making network calls, or writing to storage.
+
+## Not a fit / Don’t use when
+
+- **You need a full security platform** (hosted policy registry, enterprise controls). `typesecure` is a library.
+- **You need production-grade crypto primitives**. Use well-reviewed, purpose-built libraries and treat crypto carefully.
+- **You only want compile-time types with zero runtime behavior**. `typesecure` deliberately includes runtime checks/redaction.
 
 ## Installation
 
+Requires Node.js `>=18.18.0`.
+
 ```bash
 # Using npm
 npm install typesecure
@@ -29,113 +42,125 @@ pnpm add typesecure
 
 ## Usage
 
-### Encryption with Security Assessment
+### Classification-first data handling
 
 ```typescript
-import { encrypt, decrypt, generateKey, getSecurityLevel, SecurityLevel } from 'typesecure';
-
-// Generate a secure key
-const key = generateKey();
-
-// Encrypt data with GCM (authenticated encryption)
-const encrypted = encrypt('Sensitive information', key, {
-  mode: 'aes-gcm',
-  aad: 'Additional authenticated data' // Optional
-});
-
-// Decrypt data
-const decrypted = decrypt(encrypted, key, {
-  mode: 'aes-gcm',
-  aad: 'Additional authenticated data' // Must match encryption
-});
-
-// Assess security level of encryption options
-const securityLevel = getSecurityLevel({ mode: 'aes-cbc', padding: 'Pkcs7' });
-if (securityLevel === SecurityLevel.HIGH) {
-  console.log('Using high security encryption configuration');
-}
+import {
+  piiText,
+  secretText,
+  token,
+  publicText,
+  redact,
+  safeJsonStringify,
+  defaultPolicy,
+  assertAllowed,
+  policyLog,
+} from "typesecure";
+
+const userEmail = piiText("user@example.com");
+const sessionToken = token("abc.def.ghi");
+const dbPassword = secretText(process.env.DB_PASSWORD ?? "");
+
+// Redact before logging / serialization
+console.log(redact({ userEmail, sessionToken, dbPassword }));
+console.log(
+  safeJsonStringify({ userEmail, sessionToken, dbPassword }, undefined, 2),
+);
+
+// Enforce policy before a boundary crossing
+const policy = defaultPolicy();
+assertAllowed(policy, "network", { sessionToken }); // allowed
+// assertAllowed(policy, 'log', { dbPassword }); // throws
+
+// Safe logging helper with enforcement
+policyLog(policy, console, "info", publicText("login_ok"), { userEmail });
 ```
 
-### Secure Password Storage
+### Express / Next.js examples
 
 ```typescript
-import { hashPassword, verifyPassword } from 'typesecure';
-
-// Hash a password with PBKDF2
-const { hash, salt, params } = hashPassword('userPassword123', {
-  algorithm: 'pbkdf2',
-  iterations: 10000,
-  saltLength: 32,
-  keyLength: 64
+// Express middleware example
+import {
+  safeLoggerAdapter,
+  defaultPolicy,
+  assertAllowed,
+  token,
+} from "typesecure";
+
+const log = safeLoggerAdapter(console);
+const policy = defaultPolicy();
+
+app.use((req, _res, next) => {
+  const auth = req.headers.authorization?.replace(/^Bearer\s+/i, "");
+  if (auth) {
+    const t = token(auth);
+    assertAllowed(policy, "network", { t });
+    log.info({ route: req.path, auth: t }); // will be redacted
+  }
+  next();
 });
-
-// Store hash, salt, and params in your database
-
-// Later, verify the password
-const isValid = verifyPassword('userPassword123', hash, salt, params);
 ```
 
-### Timing-Safe Comparison and Random Bytes Generation
-
-```typescript
-import { timingSafeEqual, generateRandomBytes } from 'typesecure';
-
-// Compare strings in constant time to prevent timing attacks
-const isEqual = timingSafeEqual(userProvidedToken, storedToken);
-
-// Generate cryptographically secure random bytes
-const randomBytes = generateRandomBytes(32, 'hex');
-```
+## API Reference
 
-### Hashing and HMAC
+### Classification
+
+- `publicText(value: string): PublicString`
+- `piiText(value: string): PIIString`
+- `secretText(value: string): SecretString`
+- `token(value: string): TokenString`
+- `credential(value: string): CredentialString`
+- `reveal(value): string` (intentionally explicit)
+
+### Redaction
+
+- `redact(value): value` (deep traversal)
+- `redactText(value): string` (mask sensitive fragments in plain text)
+- `detectText(value): StringDetection[]` (return ranges/kinds for audit workflows)
+- `safeJsonStringify(value): string`
+- `safeLoggerAdapter(consoleLike)`
+- Redaction options:
+  - `guessByKey` (default `true`): redact suspicious keys like `password`, `token`, `apiKey`.
+  - `guessByValue` (default `true`): auto-detect and redact sensitive-looking values.
+  - `useDefaultValueDetector` (default `true`): keep built-in rule-based detectors on/off.
+  - `stringDetectors`: add custom detectors (for NER/ML or domain-specific logic).
+  - `minDetectionConfidence` (default `0`): ignore low-confidence custom detections.
+- Value detection masks only the sensitive fragments inside a larger string (instead of replacing the whole text), including:
+  - PII: email, phone, SSN, date of birth (`YYYY-MM-DD`), IPv4 address, payment card numbers (Luhn-validated).
+  - Secrets/tokens: JWTs, private key PEM blocks, GitHub tokens, AWS access keys, Stripe secret keys, OpenAI-style `sk-...` keys, credential pairs (`user:pass`), high-entropy token-like strings.
+
+Example custom detector (NER/ML-style integration):
 
 ```typescript
-import { hash, verifyHash, hmac } from 'typesecure';
-
-// Create a hash
-const hashedValue = hash('data to hash', {
-  algorithm: 'sha256',
-  encoding: 'hex'
-});
-
-// Verify a hash
-const isMatch = verifyHash('data to hash', hashedValue, {
-  algorithm: 'sha256',
-  encoding: 'hex'
-});
-
-// Create an HMAC
-const signature = hmac('message', 'secret key', {
-  algorithm: 'sha256',
-  encoding: 'base64'
-});
+const out = redact(
+  { text: "Customer Jane Doe uses jane@example.com" },
+  {
+    stringDetectors: [
+      (value) => {
+        const name = "Jane Doe";
+        const idx = value.indexOf(name);
+        return idx >= 0
+          ? [{ start: idx, end: idx + name.length, kind: "pii", confidence: 0.92, source: "ml.ner" }]
+          : [];
+      },
+    ],
+    minDetectionConfidence: 0.8,
+  },
+);
 ```
 
-## API Reference
-
-### Encryption
-
-- `encrypt(text: string, key: string, options?: Partial<EncryptionOptions>): string`
-- `decrypt(encryptedText: string, key: string, options?: Partial<EncryptionOptions>): string`
-- `generateKey(length?: number): string`
-- `getSecurityLevel(options: EncryptionOptions): SecurityLevel`
-
-### Secure Password Storage
+### Policy
 
-- `hashPassword(password: string, options?: Partial<PasswordHashOptions>): { hash: string; salt: string; params: PasswordHashOptions }`
-- `verifyPassword(password: string, hash: string, salt: string, options?: Partial<PasswordHashOptions>): boolean`
-- `timingSafeEqual(a: string, b: string, options?: Partial<TimingSafeOptions>): boolean`
-- `generateRandomBytes(length?: number, encoding?: 'hex' | 'base64'): string`
-
-### Hashing
-
-- `hash(input: string, options?: Partial<HashOptions>): string`
-- `verifyHash(input: string, hashedValue: string, options?: Partial<HashOptions>): boolean`
-- `hmac(input: string, key: string, options?: Partial<HashOptions>): string`
+- `defaultPolicy(): Policy`
+- `assertAllowed(policy, action, data): void`
+- `audit(policy, action, data): AuditEvent`
+- `policyLog(policy, logger, level, ...args): void`
 
 ## Security Considerations
 
-This package implements best practices for cryptographic operations, but remember that cryptography is complex. For production applications with high security requirements, consider:
+Security is as much about **preventing leaks** as it is about cryptographic correctness. `typesecure` focuses on preventing accidental secret/PII exposure across common boundaries.
+
+If you need cryptography for production-grade requirements, prefer well-reviewed primitives and consult a security professional. For production applications with high security requirements, consider:
 
 1. Consulting a security professional
 2. Using specialized security libraries
@@ -153,13 +178,51 @@ To contribute to this project:
 2. Install dependencies with `pnpm install`
 3. Run tests with `pnpm test`
 4. Build the package with `pnpm build`
+5. Run Enron dataset integration tests with `pnpm test:data`
+
+### Optional: external dataset setup
+
+For larger redaction/policy experiments (Enron + Synthea FHIR), fetch datasets locally:
+
+```bash
+pnpm data:setup
+```
+
+This command downloads and extracts to:
+
+- `data/enron-maildir`
+- `data/synthea_sample_data_fhir_latest`
+
+Notes:
+
+- `data/` is gitignored and not published to npm.
+- `pnpm test` excludes dataset suites by default.
+- `pnpm test:data` runs Enron dataset tests with verbose output.
+- `pnpm test:data:synthea` runs Synthea-specific dataset tests.
+- `pnpm test:data:all` runs all dataset suites.
+- You can override source URLs with `ENRON_URL=...` and/or `SYNTHEA_FHIR_URL=...`.
+- You can change destination with `DATA_DIR=/path/to/data`.
+
+Dataset sources:
+
+- Enron: [https://www.cs.cmu.edu/~enron/](https://www.cs.cmu.edu/~enron/)
+- Synthea: [https://github.com/synthetichealth/synthea-sample-data/](https://github.com/synthetichealth/synthea-sample-data/)
 
 This project uses TypeScript for type safety, Jest for testing, and ESLint for code quality.
 
+## Dataset Acknowledgements
+
+We use these public datasets for redaction and policy testing:
+
+- [CMU Enron Email Dataset](https://www.cs.cmu.edu/~enron/)
+- [Synthea Sample Data](https://github.com/synthetichealth/synthea-sample-data/)
+
+Personal note: I am especially interested in the historical context around Enron, including how it was able to happen and the improvements in governance and controls that followed.
+
 ## License
 
 MIT © [Arvid Berndtsson](https://github.com/arvid-berndtsson)
 
 ## Contributing
 
-Contributions are welcome! Please feel free to submit a Pull Request. 
+Contributions are welcome! Please feel free to submit a Pull Request.