s3db.js 11.0.5 → 11.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -2
- package/SECURITY.md +76 -0
- package/dist/s3db.cjs.js +446 -86
- package/dist/s3db.cjs.js.map +1 -1
- package/dist/s3db.es.js +446 -86
- package/dist/s3db.es.js.map +1 -1
- package/package.json +3 -1
- package/src/concerns/crypto.js +7 -14
- package/src/plugins/eventual-consistency/analytics.js +164 -2
- package/src/plugins/eventual-consistency/consolidation.js +228 -80
- package/src/plugins/eventual-consistency/helpers.js +24 -8
- package/src/plugins/eventual-consistency/install.js +2 -1
- package/src/plugins/eventual-consistency/utils.js +218 -4
- package/src/concerns/advanced-metadata-encoding.js +0 -440
package/README.md
CHANGED
|
@@ -794,7 +794,7 @@ await users.insert({ name: "John", email: "john@example.com" });
|
|
|
794
794
|
- **📝 [Audit Plugin](./docs/plugins/audit.md)** - Comprehensive audit logging for compliance
|
|
795
795
|
- **📬 [Queue Consumer Plugin](./docs/plugins/queue-consumer.md)** - Message consumption from SQS/RabbitMQ
|
|
796
796
|
- **🔒 [S3Queue Plugin](./docs/plugins/s3-queue.md)** - Distributed queue processing with zero race conditions
|
|
797
|
-
- **📈 [Eventual Consistency Plugin](./docs/plugins/eventual-consistency.md)** -
|
|
797
|
+
- **📈 [Eventual Consistency Plugin](./docs/plugins/eventual-consistency.md)** - Transactional counters with pre-computed analytics (15 functions for time-series data)
|
|
798
798
|
- **📅 [Scheduler Plugin](./docs/plugins/scheduler.md)** - Task scheduling and automation
|
|
799
799
|
- **🔄 [State Machine Plugin](./docs/plugins/state-machine.md)** - State management and transitions
|
|
800
800
|
- **💾 [Backup Plugin](./docs/plugins/backup.md)** - Backup and restore functionality
|
|
@@ -1868,6 +1868,12 @@ await users.insert({ name: 'John' });
|
|
|
1868
1868
|
|
|
1869
1869
|
## 📖 API Reference
|
|
1870
1870
|
|
|
1871
|
+
### 📚 Core Classes Documentation
|
|
1872
|
+
|
|
1873
|
+
- **[Client Class](./docs/client.md)** - Low-level S3 operations, HTTP client configuration, and advanced object management
|
|
1874
|
+
- **[Database Class](./docs/database.md)** - High-level database interface (coming soon)
|
|
1875
|
+
- **[Resource Class](./docs/resource.md)** - Resource operations and methods (coming soon)
|
|
1876
|
+
|
|
1871
1877
|
### 🔌 Database Operations
|
|
1872
1878
|
|
|
1873
1879
|
| Method | Description | Example |
|
|
@@ -1971,4 +1977,55 @@ console.log(`Total users: ${allUsers.length}`);
|
|
|
1971
1977
|
| Method | Description | Example |
|
|
1972
1978
|
|--------|-------------|---------|
|
|
1973
1979
|
| `readable(options?)` | Create readable stream | `await users.readable()` |
|
|
1974
|
-
| `writable(options?)` | Create writable stream | `await users.writable()` |
|
|
1980
|
+
| `writable(options?)` | Create writable stream | `await users.writable()` |
|
|
1981
|
+
|
|
1982
|
+
---
|
|
1983
|
+
|
|
1984
|
+
## 📊 Performance Benchmarks
|
|
1985
|
+
|
|
1986
|
+
> **⚠️ Important**: All benchmark results documented below were generated using **Node.js v22.6.0**. Performance results may vary with different Node.js versions.
|
|
1987
|
+
|
|
1988
|
+
s3db.js includes comprehensive benchmarks demonstrating real-world performance optimizations. Key areas tested:
|
|
1989
|
+
|
|
1990
|
+
### 🎯 Data Encoding & Compression
|
|
1991
|
+
|
|
1992
|
+
**[Base62 Encoding](./docs/benchmarks/base62.md)** - Number compression for S3 metadata
|
|
1993
|
+
- **40-46% space savings** for large numbers
|
|
1994
|
+
- **5x faster encoding** vs Base36
|
|
1995
|
+
- **Real-world impact**: More data fits in 2KB S3 metadata limit
|
|
1996
|
+
|
|
1997
|
+
**[Advanced Encoding](./docs/benchmarks/advanced-encoding.md)** - Multi-technique compression
|
|
1998
|
+
- **67% savings** on ISO timestamps (Unix Base62)
|
|
1999
|
+
- **33% savings** on UUIDs (Binary Base64)
|
|
2000
|
+
- **95% savings** on common values (Dictionary encoding)
|
|
2001
|
+
- **Overall**: 40-50% metadata reduction on typical datasets
|
|
2002
|
+
|
|
2003
|
+
**[Smart Encoding](./docs/benchmarks/smart-encoding.md)** - Intelligent encoding selection
|
|
2004
|
+
- **Automatic type detection** and optimal encoding selection
|
|
2005
|
+
- **2-3x faster** UTF-8 byte calculations with caching
|
|
2006
|
+
- **Lazy evaluation** for performance-critical paths
|
|
2007
|
+
|
|
2008
|
+
### 🔌 Plugin Performance
|
|
2009
|
+
|
|
2010
|
+
**[EventualConsistency Plugin](./docs/benchmarks/eventual-consistency.md)** - Transaction processing & analytics
|
|
2011
|
+
- **70-100% faster writes** with async partitions
|
|
2012
|
+
- **Parallel analytics updates** for high-throughput scenarios
|
|
2013
|
+
- **O(1) partition queries** vs O(n) full scans
|
|
2014
|
+
|
|
2015
|
+
### 🗂️ Partitioning Performance
|
|
2016
|
+
|
|
2017
|
+
**[Partitions Matrix Benchmark](./docs/benchmarks/partitions.md)** - Performance testing across partition configurations
|
|
2018
|
+
- **Test matrix**: 0-10 partitions × 1-10 attributes (110 combinations)
|
|
2019
|
+
- **Measurements**: Create, insert, query (partition & full scan)
|
|
2020
|
+
- **Insights**: Find optimal partition configuration for your use case
|
|
2021
|
+
- Run with: `pnpm run benchmark:partitions`
|
|
2022
|
+
|
|
2023
|
+
### 📖 Benchmark Documentation
|
|
2024
|
+
|
|
2025
|
+
All benchmarks include:
|
|
2026
|
+
- ✅ **TL;DR summary** - Quick results and recommendations
|
|
2027
|
+
- ✅ **Code examples** - Runnable benchmark scripts
|
|
2028
|
+
- ✅ **Performance metrics** - Real numbers with explanations
|
|
2029
|
+
- ✅ **Use cases** - When to apply each optimization
|
|
2030
|
+
|
|
2031
|
+
**[📋 Complete Benchmark Index](./docs/benchmarks/README.md)**
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
| ------- | ------------------ |
|
|
7
|
+
| 11.x.x | :white_check_mark: |
|
|
8
|
+
| < 11.0 | :x: |
|
|
9
|
+
|
|
10
|
+
## Known Security Advisories
|
|
11
|
+
|
|
12
|
+
### Development Dependencies
|
|
13
|
+
|
|
14
|
+
The following vulnerabilities exist in **development-only** dependencies and **do not affect** the published npm package or runtime security:
|
|
15
|
+
|
|
16
|
+
#### pkg (GHSA-22r3-9w55-cj54) - MODERATE
|
|
17
|
+
- **Status**: Acknowledged, monitored
|
|
18
|
+
- **Impact**: Local privilege escalation
|
|
19
|
+
- **Scope**: Only affects developers running `pnpm run build:binaries`
|
|
20
|
+
- **Mitigation**: pkg is deprecated and archived. No patched version available (`<0.0.0`).
|
|
21
|
+
- **Risk Assessment**: LOW - Only used for creating standalone binaries during release process
|
|
22
|
+
- **Future Plans**: Migrate to Node.js Single Executable Applications (SEA) when stable
|
|
23
|
+
|
|
24
|
+
#### tar-fs - HIGH
|
|
25
|
+
- **Status**: RESOLVED in v11.1.1+
|
|
26
|
+
- **Fix**: Updated to patched version 2.1.4+
|
|
27
|
+
|
|
28
|
+
## Reporting a Vulnerability
|
|
29
|
+
|
|
30
|
+
If you discover a security vulnerability in the **runtime code** (not dev dependencies), please report it by:
|
|
31
|
+
|
|
32
|
+
1. **DO NOT** open a public issue
|
|
33
|
+
2. Email: [security contact - update this]
|
|
34
|
+
3. Include:
|
|
35
|
+
- Description of the vulnerability
|
|
36
|
+
- Steps to reproduce
|
|
37
|
+
- Potential impact
|
|
38
|
+
- Suggested fix (if any)
|
|
39
|
+
|
|
40
|
+
### Response Timeline
|
|
41
|
+
|
|
42
|
+
- **Initial Response**: Within 48 hours
|
|
43
|
+
- **Status Update**: Within 7 days
|
|
44
|
+
- **Fix Timeline**: Depends on severity
|
|
45
|
+
- Critical: 7 days
|
|
46
|
+
- High: 14 days
|
|
47
|
+
- Medium: 30 days
|
|
48
|
+
- Low: 60 days
|
|
49
|
+
|
|
50
|
+
## Security Best Practices
|
|
51
|
+
|
|
52
|
+
### For Users
|
|
53
|
+
|
|
54
|
+
1. **Always encrypt sensitive data**: Use `secret` field type for passwords, tokens, etc.
|
|
55
|
+
2. **Validate credentials**: Never commit AWS credentials to version control
|
|
56
|
+
3. **Use IAM policies**: Implement least-privilege access for S3 buckets
|
|
57
|
+
4. **Enable paranoid mode**: For production, use `paranoid: true` for soft deletes
|
|
58
|
+
5. **Audit hooks**: Review serialized functions before deploying to production
|
|
59
|
+
|
|
60
|
+
### For Contributors
|
|
61
|
+
|
|
62
|
+
1. **No secrets in tests**: Use environment variables or LocalStack
|
|
63
|
+
2. **Validate input**: All user input should be validated before S3 operations
|
|
64
|
+
3. **Handle errors safely**: Never expose AWS error details to end users
|
|
65
|
+
4. **Review dependencies**: Run `pnpm audit` before submitting PRs
|
|
66
|
+
5. **Test encryption**: Verify `secret` fields are actually encrypted in S3
|
|
67
|
+
|
|
68
|
+
## Audit Configuration
|
|
69
|
+
|
|
70
|
+
This project uses `audit-level=high` in `.npmrc` to focus on critical vulnerabilities affecting production. Moderate/low severity issues in dev-only dependencies are monitored but may not block releases if:
|
|
71
|
+
|
|
72
|
+
- They only affect development tools
|
|
73
|
+
- No patch is available
|
|
74
|
+
- The risk is assessed as acceptable
|
|
75
|
+
|
|
76
|
+
Current audit threshold: **HIGH** (ignores moderate/low in dev dependencies)
|