@photostructure/fs-metadata 0.6.0 → 0.6.1

package/CHANGELOG.md

@@ -14,7 +14,7 @@ Fixed for any bug fixes.
 Security in case of vulnerabilities.
 -->
 
-## [0.6.0] - 2025-06-03
+## [0.6.0] - 2025-06-09
 
 ### Added
 
@@ -24,6 +24,8 @@ Security in case of vulnerabilities.
 - Clang-tidy integration for C++ code analysis
 - Worker thread helper for volume metadata operations
 - NAPI_VERSION=9 definition for improved compatibility
+- Dynamic test timeout configuration based on environment (CI, platform, architecture)
+- Prebuild script for Linux GLIBC compatibility in Docker environments
 
 ### Breaking
 
@@ -36,6 +38,7 @@ Security in case of vulnerabilities.
 - Updated all imports to use `node:` prefix for built-in modules
 - Reorganized build and test scripts for better clarity
 - Improved memory test workflow for cross-platform compatibility
+- Renamed native module target from `node_fs_meta` to `fs_metadata` for consistency
 
 ### Fixed
 
@@ -44,13 +47,9 @@ Security in case of vulnerabilities.
 - Fixed buffer allocation issues in Windows networking and volume operations
 - Enhanced resource management with better validation for empty mount points
 - Made `SystemPathPatternsDefault` values visible in TypeScript typings
-- Improved test reliability across different CI environments
 - Added Napi::HandleScope to OnOK and OnError methods for proper scope management
 - Removed unnecessary std::move operations in worker implementations
-- Updated filesystem metadata tests to verify type and existence instead of specific values
-- Improved error handling and messages in various components
-- Enhanced error message validation for file access in worker thread tests
-- Updated tests to use healthy mount points for volume metadata retrieval
+- Resolved CI test reliability issues across different environments, particularly Alpine ARM64 emulation timeouts
 
 ## [0.4.0] - 2025-01-09
 

package/CLAUDE.md

@@ -22,86 +22,6 @@ This is @photostructure/fs-metadata - a cross-platform native Node.js module for
 - **Windows**: Uses separate threads per mountpoint for health checks to handle blocked system calls
 - **System Volumes**: Each platform handles system volumes differently; the library uses heuristics to identify them
 
-## Common Commands
-
-### Build and Development
-```bash
-# Install dependencies and build native modules
-npm install
-
-# Configure platform-specific build settings
-npm run configure:native
-
-# Build native bindings
-npm run node-gyp-rebuild
-
-# Create prebuilds for distribution  
-npm run prebuild
-
-# Assemble TypeScript to dist/
-npm run build:dist
-```
-
-### Testing
-```bash
-# Run all tests with coverage (includes memory tests on Linux)
-npm run test:all
-
-# Run CommonJS tests
-npm test:cjs
-
-# Run ESM tests
-npm test:esm
-
-# Test memory leaks (JavaScript)
-npm run test:memory
-
-# Run valgrind memory analysis (Linux only)
-bash scripts/valgrind-test.sh
-
-# Run AddressSanitizer and LeakSanitizer (Linux only)
-bash scripts/sanitizers-test.sh
-
-# Run comprehensive memory tests (JavaScript + valgrind + sanitizers)
-npm run test:memory
-
-# Run a specific test file (no coverage)
-npm test volume_metadata
-```
-
-### Code Quality
-```bash
-# Run ESLint
-npm run lint
-
-# Fix ESLint issues
-npm run lint:fix
-
-# Format code (all formats)
-npm run fmt
-
-# Format C++ code only
-npm run fmt:cpp
-
-# Format TypeScript only
-npm run fmt:ts
-
-# Type checking
-npm run build:ts
-```
-
-### Pre-commit
-```bash
-# Full precommit check (fmt, clean, prebuild, tests)
-npm run precommit
-```
-
-### Documentation
-```bash
-# Generate API documentation
-npm run docs
-```
-
 ## Architecture Overview
 
 ### Core Structure
@@ -157,6 +77,34 @@ npm run docs
 - Debug messages from both JavaScript and native code are sent to `stderr`
 - Uses native Node.js debuglog for determining if logging is enabled
 
+## Release Process
+
+The project uses a vanilla npm/git release workflow with GPG signed commits for security:
+
+### Prerequisites
+- Repository secrets must be configured:
+  - `NPM_TOKEN`: Authentication token for npm publishing
+  - `GPG_PRIVATE_KEY`: ASCII-armored GPG private key for signing commits
+  - `GPG_PASSPHRASE`: Passphrase for the GPG key (if applicable)
+
+### Automated Release
+1. Trigger via GitHub Actions workflow dispatch with version type (patch/minor/major)
+2. Builds all prebuilds for supported platforms
+3. Runs comprehensive test suite across platforms
+4. Uses `npm version` to bump version and create signed git tags
+5. Publishes to npm registry
+6. Creates GitHub release with auto-generated notes
+7. All commits and tags are GPG signed for verification
+
+### Manual Release (if needed)
+```bash
+npm run prepare-release
+git config commit.gpgsign true
+npm version patch|minor|major
+npm publish
+git push origin main --follow-tags
+```
+
 ## Example Usage
 
 ```typescript
@@ -173,4 +121,254 @@ console.dir({ volumeMetadata });
 // Check if a file is hidden
 import { isHidden } from "@photostructure/fs-metadata";
 const hidden = await isHidden("/path/to/file");
-```
+```
+
+## CI/CD Test Reliability Guidelines
+
+Based on analysis of recent test failures, here are critical patterns to avoid flaky tests:
+
+### 1. Benchmark and Performance Tests
+- **Problem**: "Cannot log after tests are done" errors in worker_threads.test.ts
+- **Solution**: 
+  - Always await all async operations before test completion
+  - Use proper test lifecycle hooks (afterEach/afterAll) for cleanup
+  - Avoid console.log in async contexts without proper synchronization
+  - Consider using test.concurrent with explicit done() callbacks
+
+### 2. Alpine Linux ARM64 Issues
+- **Problem**: Tests timeout on emulated Alpine ARM64 environments
+- **Solution**:
+  - Skip process-spawning tests on Alpine ARM64 (`if (isAlpine && isARM64)`)
+  - Use increased timeout multipliers (20x) for emulated environments
+  - Detect emulation via `/proc/cpuinfo` or environment checks
+  - Consider separate test suites for native vs emulated environments
+
+### 3. Worker Thread Management
+- **Problem**: Race conditions in concurrent worker operations
+- **Solution**:
+  - Implement proper worker pool management with size limits
+  - Use Promise.allSettled() instead of Promise.all() for parallel operations
+  - Add explicit cleanup in test teardown to terminate all workers
+  - Set reasonable concurrency limits based on environment (CPU cores)
+
+### 4. Timeout Test Reliability
+- **Problem**: Timeout tests fail due to timing precision issues
+- **Solution**:
+  - Never use exact timing assertions (e.g., expect 100ms)
+  - Use ranges with adequate margins (e.g., 90-110ms)
+  - Account for CI environment variability (slower machines)
+  - Consider mocking timers for deterministic behavior
+
+### 5. File System Operations
+- **Problem**: ENOENT errors for test directories, permission issues
+- **Solution**:
+  - Always use unique temporary directories per test
+  - Clean up test artifacts in afterEach hooks
+  - Check directory existence before operations
+  - Handle platform-specific path separators
+
+### 6. Memory and Resource Leaks
+- **Problem**: Tests don't properly clean up resources
+- **Solution**:
+  - Explicitly close all file handles, network connections
+  - Use try-finally blocks for resource cleanup
+  - Monitor memory usage in long-running tests
+  - Implement proper garbage collection triggers
+
+### 7. Platform-Specific Failures
+- **Problem**: Different behavior across Windows/macOS/Linux
+- **Solution**:
+  - Use platform detection helpers consistently
+  - Skip platform-specific tests appropriately
+  - Account for filesystem differences (case sensitivity, path formats)
+  - Test with platform-specific CI matrices
+
+### 8. Jest Configuration
+- **Problem**: Tests interfere with each other
+- **Solution**:
+  - Use `--runInBand` for tests with shared resources
+  - Clear module cache between tests when needed
+  - Isolate tests that spawn processes
+  - Configure proper test timeouts per environment
+
+### Async Cleanup Anti-Patterns
+
+**IMPORTANT**: The following approaches are NOT valid solutions for async cleanup issues:
+
+```javascript
+// BAD: Arbitrary timeouts in tests
+await new Promise((resolve) => setTimeout(resolve, 100));
+
+// BAD: Forcing garbage collection
+if (global.gc) {
+  global.gc();
+}
+
+// BAD: Adding setImmediate in afterAll to "fix" hanging tests
+afterAll(async () => {
+  await new Promise((resolve) => setImmediate(resolve));
+});
+```
+
+**Why these are problematic:**
+
+1. **Arbitrary timeouts** are race conditions waiting to happen. They might work on fast machines but fail on slower CI runners.
+2. **Forcing GC** should never be required for correct behavior. If your code depends on GC for correctness, it has a fundamental design flaw.
+3. **setImmediate/nextTick delays** in cleanup hooks don't fix the root cause - they just paper over the real issue.
+4. These approaches mask the real problem instead of fixing it.
+
+**Note**: This is different from legitimate uses of timeouts, such as:
+
+- Waiting for time to pass to test timestamp changes
+- Rate limiting or throttling tests
+- Testing timeout behavior itself
+
+The anti-pattern is using timeouts or GC to "fix" async cleanup issues.
+
+**What to do instead:**
+
+1. Find the actual resource that's keeping the process alive (use `--detectOpenHandles`)
+2. Ensure all database connections are properly closed
+3. Ensure all file handles are closed
+4. Cancel or await all pending async operations
+5. Use proper resource management patterns (RAII, try-finally, using statements)
+
+### Windows-Compatible Directory Cleanup
+
+**IMPORTANT**: Never use `fs.rmSync()` or `fs.rm()` without proper Windows retry logic for directory cleanup in tests.
+
+**Problem**: On Windows, file handles and directory locks can remain active longer than on Unix systems, causing `EBUSY` errors during cleanup.
+
+**Proper Solution**: Use `fsp.rm()` (async) with retry options:
+
+```typescript
+await fsp.rm(tempDir, {
+  recursive: true,
+  force: true,
+  maxRetries: process.platform === "win32" ? 3 : 1,
+  retryDelay: process.platform === "win32" ? 100 : 0,
+});
+```
+
+**Best Practice**: Use existing test utilities that handle Windows-compatible cleanup patterns. Don't manually clean up temp directories - let the test framework handle it with proper retry logic.
+
+### Adaptive Timeout Testing
+
+**Problem**: Fixed timeouts don't account for varying CI environment performance.
+
+**Root Causes**:
+
+- Alpine Linux (musl libc) is 2x slower than glibc
+- ARM64 emulation on x64 runners is 5x slower
+- Windows process operations are 4x slower
+- macOS VMs are 4x slower
+- CI environments have resource constraints
+
+**Solutions**:
+
+```typescript
+// DON'T: Use fixed timeouts
+test("my test", async () => {
+  // Test code
+}, 10000);
+
+// DO: Use adaptive timeouts based on environment
+import { getTestTimeout } from "./test-utils/test-timeout-config";
+
+test(
+  "my test",
+  async () => {
+    // Test code
+  },
+  getTestTimeout(10000),
+);
+
+// DO: Account for platform timing differences
+const timingMultiplier = process.platform === "win32" ? 4 :
+                        process.platform === "darwin" ? 4 :
+                        process.env.CI ? 2 : 1;
+```
+
+### Multi-Process Test Synchronization
+
+**Problem**: Multi-process tests failing due to race conditions between process startup and test assertions.
+
+**Root Cause**: The timing between process startup and resource acquisition varies significantly by platform.
+
+**Solutions**:
+
+```typescript
+// DON'T: Assume immediate process readiness
+const proc = spawn(nodeCmd, [script]);
+const result = await waitForProcessResult(proc);
+expect(result).toBe("expected_outcome"); // May fail due to timing
+
+// DO: Use explicit synchronization signals
+const script = `
+  // Setup code
+  console.log("READY");  // Signal readiness
+  // Main test logic
+  console.log("RESULT:" + outcome);
+`;
+
+const proc = spawn(process.execPath, ["-e", script]);
+await waitForOutput(proc, "READY"); // Wait for process to be ready
+const result = await waitForOutput(proc, "RESULT:");
+expect(result.split(":")[1]).toBe("expected_outcome");
+```
+
+### Wait-for-Condition Pattern
+
+**Problem**: Tests failing because they don't wait for asynchronous conditions to be met.
+
+**Solution**: Implement robust condition waiting with platform-aware timing:
+
+```typescript
+async function waitForCondition(
+  check: () => boolean | Promise<boolean>,
+  options: {
+    maxAttempts?: number;
+    delay?: number;
+    timeoutMs?: number;
+  } = {}
+) {
+  const {
+    maxAttempts = 50,
+    delay = 100,
+    timeoutMs = 30000
+  } = options;
+  
+  const startTime = Date.now();
+  
+  for (let i = 0; i < maxAttempts; i++) {
+    if (Date.now() - startTime > timeoutMs) {
+      throw new Error(`Condition not met within ${timeoutMs}ms`);
+    }
+    
+    if (await check()) return true;
+    await new Promise((resolve) => setTimeout(resolve, delay));
+  }
+  
+  return false;
+}
+
+// Usage example
+await waitForCondition(
+  () => fs.existsSync(expectedFile),
+  { timeoutMs: getTestTimeout(10000) }
+);
+```
+
+### Best Practices Summary
+1. **Always clean up**: Resources, timers, workers, file handles
+2. **Never assume timing**: Use ranges and adaptive timeouts, not exact values
+3. **Isolate tests**: Each test should be independent
+4. **Platform awareness**: Skip tests that can't work on certain platforms
+5. **Proper async handling**: Always await or return promises
+6. **Resource limits**: Don't spawn unlimited workers/processes
+7. **Environment detection**: Adjust behavior for CI vs local
+8. **Deterministic tests**: Mock external dependencies when possible
+9. **Explicit synchronization**: Use signals for multi-process coordination
+10. **Robust waiting**: Use condition-based waiting instead of arbitrary timeouts
+11. **Windows compatibility**: Use retry logic for file operations on Windows
+12. **Anti-pattern awareness**: Avoid masking problems with timeouts or forced GC

package/CONTRIBUTING.md

@@ -106,7 +106,7 @@ Actions that have multiple targets can be run in parallel using wildcards:
 
 ### Naming Guidelines
 
-- Use explicit names to avoid ambiguity (e.g., `configure:native` instead of just `configure`)
+- Use explicit names to avoid ambiguity (e.g., `setup:native` instead of just `setup`)
 - Group related scripts by action prefix for easy wildcard execution
 - Avoid names that could cause npm lifecycle conflicts (e.g., `prebuild` vs `build`)
 - Use descriptive suffixes that clearly indicate the target or purpose

package/README.md

@@ -16,7 +16,8 @@ Built and supported by [PhotoStructure](https://photostructure.com).
 - Cross-platform support:
   - Windows 10+ (x64)
   - macOS 14+
-  - Ubuntu 22+ (x64, arm64) (with Gnome GIO/`GVfs` mount support where available)
+  - Debian 11/Ubuntu 20.04+ (x64, arm64) (with Gnome GIO/`GVfs` mount support where available)
+  - Alpine 3.21+ (x64, arm64)
 
 - [List all mounted volumes/drives](https://photostructure.github.io/fs-metadata/functions/getVolumeMountPoints.html)
 
@@ -41,6 +42,24 @@ Built and supported by [PhotoStructure](https://photostructure.com).
 
 - Comprehensive test coverage
 
+## Supported Platforms
+
+Prebuilt binaries are provided for the following platforms:
+
+| Platform | Architecture | Node.js | Minimum OS Version |
+|----------|--------------|---------|-------------------|
+| Windows | x64 | 20+ | Windows 10 |
+| macOS | x64 | 20+ | macOS 14 (Sonoma) |
+| macOS | arm64 | 20+ | macOS 14 (Sonoma) |
+| Linux (glibc) | x64 | 20+ | Debian 11 (Bullseye), Ubuntu 20.04, GLIBC 2.31+ |
+| Linux (glibc) | arm64 | 20+ | Debian 11 (Bullseye), Ubuntu 20.04, GLIBC 2.31+ |
+| Linux (musl) | x64 | 20+ | Alpine 3.21 |
+| Linux (musl) | arm64 | 20+ | Alpine 3.21 |
+
+Notes:
+- Linux binaries require GLIBC 2.31+ (Debian 11 Bullseye or newer). **Note**: this means the `node:20` docker image is **not** supported, due to the last several major versions of `node-gyp-build` requiring both a newer python and newer C++ version.
+- Electron is supported via [Node-API](https://nodejs.org/api/n-api.html) compatibility
+
 ## Installation
 
 ```bash

package/binding.gyp

@@ -4,7 +4,7 @@
   },
   "targets": [
     {
-      "target_name": "node_fs_meta",
+      "target_name": "fs_metadata",
       "sources": [
         "src/binding.cpp"
       ],

package/dist/index.cjs

@@ -97,6 +97,7 @@ function defer2(thunk) {
 var import_node_path = require("path");
 
 // src/platform.ts
+var import_node_fs = require("fs");
 var import_node_process = require("process");
 var isLinux = import_node_process.platform === "linux";
 var isWindows = import_node_process.platform === "win32";
@@ -196,7 +197,7 @@ function _dirname() {
 }
 
 // src/fs.ts
-var import_node_fs = require("fs");
+var import_node_fs2 = require("fs");
 var import_promises = require("fs/promises");
 var import_node_path2 = require("path");