@photostructure/fs-metadata 0.6.1 → 0.7.0

package/CHANGELOG.md

@@ -14,7 +14,13 @@ Fixed for any bug fixes.
 Security in case of vulnerabilities.
 -->
 
-## [0.6.0] - 2025-06-09
+## 0.7.0 - 2025-10-24
+
+- Audit and address [several resource handling issues](./doc/SECURITY_AUDIT_2025.md)
+- Added support for Node.js v25
+- Updated dev dependencies
+
+## 0.6.0 - 2025-06-09
 
 ### Added
 

package/CLAUDE.md

@@ -4,242 +4,151 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## Project Overview
 
-This is @photostructure/fs-metadata - a cross-platform native Node.js module for retrieving filesystem metadata, including mount points, volume information, and space utilization statistics.
-
-### Key Features
-- Cross-platform support: Windows 10+ (x64), macOS 14+, Ubuntu 22+ (x64, arm64)
-- Lists all mounted volumes/drives
-- Gets detailed volume metadata (size, usage, filesystem type, etc.)
-- Hidden file/directory attribute support (get/set)
-- Non-blocking async native implementations
-- Timeout handling for unresponsive network volumes
-- ESM and CJS module support
-- Full TypeScript type definitions
-- Worker threads support with proper context isolation
-
-### Platform-Specific Notes
-- **Linux**: Optional GIO/GVfs mount support via Gnome libraries
-- **Windows**: Uses separate threads per mountpoint for health checks to handle blocked system calls
-- **System Volumes**: Each platform handles system volumes differently; the library uses heuristics to identify them
-
-## Architecture Overview
-
-### Core Structure
-- **Native binding layer** (`src/binding.cpp`): Node-API v9 bridge between JavaScript and platform-specific implementations
-- **Platform abstractions** in `src/common/`: Shared C++ interfaces for cross-platform functionality
-- **Platform implementations**:
-  - `src/darwin/`: macOS-specific code using Core Foundation APIs
-  - `src/linux/`: Linux-specific code with optional GIO support for GNOME/GVfs
-  - `src/windows/`: Windows-specific code using Win32 APIs
-
-### Key Features
-- **Volume Metadata**: Retrieves filesystem information (mount points, disk usage, health status)
-- **Hidden File Support**: Cross-platform hidden file detection and manipulation
-- **Async Operations**: All native operations use Worker threads to avoid blocking
-
-### Build System
-- Uses `node-gyp` for native compilation
-- Conditionally enables GIO support on Linux via `scripts/configure.mjs`
-- Provides prebuilt binaries via `prebuildify` for common platforms
-- Supports both ESM and CJS module formats
+@photostructure/fs-metadata - Cross-platform native Node.js module for filesystem metadata retrieval.
 
-### Cross-Module Compatibility
-- **Directory Path Resolution**: Use `_dirname()` from `./dirname` instead of `__dirname`
-  - Works in both CommonJS and ESM contexts
-  - Handles Jest test environments correctly
-  - Example: `const dir = _dirname()` instead of `const dir = __dirname`
+### Directory Structure
+
+- `src/` - Source code (TypeScript and C++)
+- `dist/` - Compiled JavaScript output (gitignored)
+- `doc/` - Static documentation (manually written, checked into git)
+- `build/` - All build artifacts (gitignored)
+  - `build/docs/` - Generated API documentation from TypeDoc (deployed to GitHub Pages)
+- `scripts/` - Build and utility scripts
+- `prebuilds/` - Prebuilt native binaries for different platforms
+
+### Script Preferences
+
+**Always** use TypeScript (`.ts`) scripts executed with `tsx` instead of:
+
+- `.js` scripts (require compilation or older Node.js syntax)
+- `.mjs` scripts (ESM-only, compatibility issues)
+- `.cjs` scripts (CommonJS-only, less type safety)
 
-### Testing Strategy
-- Jest for both CJS and ESM test suites
-- Platform-specific test expectations handled via `isWindows`, `isMacOS`, `isLinux` helpers
-- Memory leak testing with garbage collection monitoring
-- Coverage thresholds: 80% for all metrics
+TypeScript with tsx provides type safety, modern syntax, and seamless execution.
+
+## Critical Knowledge
 
 ### Testing File System Metadata
-- **Important**: File system metadata like `available` space, `used` space, and other dynamic properties change continuously as other processes run on the machine
-- Tests should **never** expect exact equality for these values between multiple calls
-- **Do not** make range assertions (e.g., `available > 0` or `used < size`) because:
-  - Files can be created or deleted between calls (potentially gigabytes)
-  - The changes can be dramatic and unpredictable
-- Instead, for dynamic metadata:
-  - Only verify the value exists and has the correct type
-  - Use `typeof result.available === 'number'` rather than range checks
-  - Focus on testing static properties (e.g., `size`, `mountFrom`, `fstype`) for exact equality
-  - Consider using snapshot testing only for stable properties
-
-### Timeout Handling
-- Default timeout for volume operations to handle unresponsive network mounts
-- Windows uses separate threads per mountpoint for health checks
-- Configurable via `Options` interface
-
-### Debug Logging
-- Enable with `NODE_DEBUG=fs-meta` or `NODE_DEBUG=photostructure:fs-metadata`
-- Debug messages from both JavaScript and native code are sent to `stderr`
-- Uses native Node.js debuglog for determining if logging is enabled
 
-## Release Process
+**Never** expect exact equality for dynamic values (`available`, `used`) between calls. Only verify:
 
-The project uses a vanilla npm/git release workflow with GPG signed commits for security:
+- Value exists and has correct type: `typeof result.available === 'number'`
+- Test static properties (`size`, `mountFrom`, `fstype`) for exact equality
+- Avoid range assertions (`available > 0`) - file changes can be dramatic
 
-### Prerequisites
-- Repository secrets must be configured:
-  - `NPM_TOKEN`: Authentication token for npm publishing
-  - `GPG_PRIVATE_KEY`: ASCII-armored GPG private key for signing commits
-  - `GPG_PASSPHRASE`: Passphrase for the GPG key (if applicable)
+### Cross-Module Compatibility
 
-### Automated Release
-1. Trigger via GitHub Actions workflow dispatch with version type (patch/minor/major)
-2. Builds all prebuilds for supported platforms
-3. Runs comprehensive test suite across platforms
-4. Uses `npm version` to bump version and create signed git tags
-5. Publishes to npm registry
-6. Creates GitHub release with auto-generated notes
-7. All commits and tags are GPG signed for verification
+Use `_dirname()` from `./dirname` instead of `__dirname` - works in both CommonJS and ESM contexts.
 
-### Manual Release (if needed)
-```bash
-npm run prepare-release
-git config commit.gpgsign true
-npm version patch|minor|major
-npm publish
-git push origin main --follow-tags
-```
+### Node.js Version Compatibility
 
-## Example Usage
+Jest 30 doesn't support Node.js 23. Use Node.js 20, 22, or 24.
 
-```typescript
-import { getVolumeMountPoints, getVolumeMetadata } from "@photostructure/fs-metadata";
+## Windows-Specific Issues
 
-// List all mounted volumes
-const mountPoints = await getVolumeMountPoints();
-console.dir({ mountPoints });
+### Windows CI Jest Worker Failures
 
-// Get metadata for a specific volume
-const volumeMetadata = await getVolumeMetadata(mountPoints[0]);
-console.dir({ volumeMetadata });
+**Problem**: Jest worker processes fail on Windows CI environments (both x64 and ARM64) with "Jest worker encountered 4 child process exceptions".
 
-// Check if a file is hidden
-import { isHidden } from "@photostructure/fs-metadata";
-const hidden = await isHidden("/path/to/file");
-```
+**Solution for Memory Tests**:
 
-## CI/CD Test Reliability Guidelines
-
-Based on analysis of recent test failures, here are critical patterns to avoid flaky tests:
-
-### 1. Benchmark and Performance Tests
-- **Problem**: "Cannot log after tests are done" errors in worker_threads.test.ts
-- **Solution**: 
-  - Always await all async operations before test completion
-  - Use proper test lifecycle hooks (afterEach/afterAll) for cleanup
-  - Avoid console.log in async contexts without proper synchronization
-  - Consider using test.concurrent with explicit done() callbacks
-
-### 2. Alpine Linux ARM64 Issues
-- **Problem**: Tests timeout on emulated Alpine ARM64 environments
-- **Solution**:
-  - Skip process-spawning tests on Alpine ARM64 (`if (isAlpine && isARM64)`)
-  - Use increased timeout multipliers (20x) for emulated environments
-  - Detect emulation via `/proc/cpuinfo` or environment checks
-  - Consider separate test suites for native vs emulated environments
-
-### 3. Worker Thread Management
-- **Problem**: Race conditions in concurrent worker operations
-- **Solution**:
-  - Implement proper worker pool management with size limits
-  - Use Promise.allSettled() instead of Promise.all() for parallel operations
-  - Add explicit cleanup in test teardown to terminate all workers
-  - Set reasonable concurrency limits based on environment (CPU cores)
-
-### 4. Timeout Test Reliability
-- **Problem**: Timeout tests fail due to timing precision issues
-- **Solution**:
-  - Never use exact timing assertions (e.g., expect 100ms)
-  - Use ranges with adequate margins (e.g., 90-110ms)
-  - Account for CI environment variability (slower machines)
-  - Consider mocking timers for deterministic behavior
-
-### 5. File System Operations
-- **Problem**: ENOENT errors for test directories, permission issues
-- **Solution**:
-  - Always use unique temporary directories per test
-  - Clean up test artifacts in afterEach hooks
-  - Check directory existence before operations
-  - Handle platform-specific path separators
-
-### 6. Memory and Resource Leaks
-- **Problem**: Tests don't properly clean up resources
-- **Solution**:
-  - Explicitly close all file handles, network connections
-  - Use try-finally blocks for resource cleanup
-  - Monitor memory usage in long-running tests
-  - Implement proper garbage collection triggers
-
-### 7. Platform-Specific Failures
-- **Problem**: Different behavior across Windows/macOS/Linux
-- **Solution**:
-  - Use platform detection helpers consistently
-  - Skip platform-specific tests appropriately
-  - Account for filesystem differences (case sensitivity, path formats)
-  - Test with platform-specific CI matrices
-
-### 8. Jest Configuration
-- **Problem**: Tests interfere with each other
-- **Solution**:
-  - Use `--runInBand` for tests with shared resources
-  - Clear module cache between tests when needed
-  - Isolate tests that spawn processes
-  - Configure proper test timeouts per environment
-
-### Async Cleanup Anti-Patterns
-
-**IMPORTANT**: The following approaches are NOT valid solutions for async cleanup issues:
+Memory tests now use a standalone TypeScript runner (`src/test-utils/memory-test-runner.ts`) that bypasses Jest entirely on all platforms. This provides more accurate memory measurements without Jest overhead and avoids worker process issues.
 
-```javascript
-// BAD: Arbitrary timeouts in tests
-await new Promise((resolve) => setTimeout(resolve, 100));
+- Run full memory check suite (includes native tools): `npm run check:memory`
+- Memory test logic is in `src/test-utils/memory-test-core.ts`
 
-// BAD: Forcing garbage collection
-if (global.gc) {
-  global.gc();
-}
+**Workaround for Other Tests**:
 
-// BAD: Adding setImmediate in afterAll to "fix" hanging tests
-afterAll(async () => {
-  await new Promise((resolve) => setImmediate(resolve));
-});
-```
+1. Jest is configured to use single worker mode (`maxWorkers: 1`) for all Windows CI environments
+2. Tests that stress worker threads or concurrency are skipped on Windows CI using `describeSkipWindowsCI` or `describePlatformStable`:
+
+- `worker_threads.test.ts` - Worker thread integration tests
+- `thread_safety.test.ts` - Concurrent operations stress tests
+- `windows-memory-check.test.ts` - Memory leak detection (Windows only)
+- `windows-resource-security.test.ts` - Resource handle leak tests (Windows only)
+
+**Note**: These tests pass locally but fail in CI. The native module loads correctly, but Jest's worker process management has fundamental incompatibilities with these specific tests on GitHub Actions Windows runners.
 
-**Why these are problematic:**
+### Build Architecture Issue
 
-1. **Arbitrary timeouts** are race conditions waiting to happen. They might work on fast machines but fail on slower CI runners.
-2. **Forcing GC** should never be required for correct behavior. If your code depends on GC for correctness, it has a fundamental design flaw.
-3. **setImmediate/nextTick delays** in cleanup hooks don't fix the root cause - they just paper over the real issue.
-4. These approaches mask the real problem instead of fixing it.
+**Problem**: "No Target Architecture" error from Windows SDK headers when building with node-gyp/prebuildify.
 
-**Note**: This is different from legitimate uses of timeouts, such as:
+**Solution**: Use `scripts/prebuildify-wrapper.ts` which sets the `CL` environment variable with architecture defines:
 
-- Waiting for time to pass to test timestamp changes
-- Rate limiting or throttling tests
-- Testing timeout behavior itself
+- For x64: `CL=/D_M_X64 /D_WIN64 /D_AMD64_`
+- For ARM64: `CL=/D_M_ARM64 /D_WIN64`
 
-The anti-pattern is using timeouts or GC to "fix" async cleanup issues.
+**Why This is Necessary**:
 
-**What to do instead:**
+- Prebuildify doesn't properly pass architecture defines from binding.gyp conditions
+- The Windows SDK requires these macros before including `<windows.h>`
+- Projects like node-sqlite avoid this by not using Windows headers directly
 
-1. Find the actual resource that's keeping the process alive (use `--detectOpenHandles`)
-2. Ensure all database connections are properly closed
-3. Ensure all file handles are closed
-4. Cancel or await all pending async operations
-5. Use proper resource management patterns (RAII, try-finally, using statements)
+**Why Other Approaches Failed**:
 
-### Windows-Compatible Directory Cleanup
+- **Source file defines**: Would hardcode x64 defines, breaking ARM64 builds
+- **windows_compat.h wrapper**: Can't distinguish x64 from ARM64 at compile time
+- **binding.gyp conditions**: Not evaluated properly by prebuildify
+- **msvs_settings defines**: Not passed through to the compiler
 
-**IMPORTANT**: Never use `fs.rmSync()` or `fs.rm()` without proper Windows retry logic for directory cleanup in tests.
+### Memory Testing Limitations
 
-**Problem**: On Windows, file handles and directory locks can remain active longer than on Unix systems, causing `EBUSY` errors during cleanup.
+Traditional Windows tools **do not work** with Node.js native modules:
 
-**Proper Solution**: Use `fsp.rm()` (async) with retry options:
+- **Dr. Memory**: Fails with "Unable to load client library: ucrtbase.dll"
+- **Debug CRT builds**: Cannot be loaded by Node.js (missing debug runtime + UNC path issues)
+- **Visual Leak Detector**: Requires debug builds which don't work
+- **Application Verifier**: Cannot hook into Node.js memory management
+
+Use JavaScript-based memory testing (`src/windows-memory-check.test.ts`) instead.
+
+### Static Analysis (clang-tidy) Limitations
+
+**clang-tidy on Windows** has limited effectiveness due to MSVC header incompatibility:
+
+- Generates many false errors about missing std namespace members
+- Still provides valuable warnings about your code
+- See `doc/windows-clang-tidy.md` for details
+- Consider using Visual Studio Code Analysis as an alternative
+
+### WSL Development
+
+Run Windows commands from WSL:
+
+```bash
+cmd.exe /c "cd C:\\Users\\matth\\src\\fs-metadata && npm test"
+# Or create helper: echo 'cmd.exe /c "cd C:\\Users\\matth\\src\\fs-metadata && $@"' > ~/bin/win-run
+```
+
+## Memory Leak Detection
+
+Run `npm run check:memory` for comprehensive platform-specific testing:
+
+- **All platforms**: JavaScript memory tests with GC triggers
+- **Windows**: Handle count monitoring via `process.report`
+- **Linux**: Valgrind + AddressSanitizer/LeakSanitizer
+- **macOS**: AddressSanitizer (may fail due to SIP - expected)
+
+## CI/CD Test Reliability
+
+### Critical Anti-Patterns
+
+**Never** use these to "fix" async issues:
+
+```javascript
+// BAD: Arbitrary timeouts
+await new Promise((resolve) => setTimeout(resolve, 100));
+// BAD: Forcing GC
+if (global.gc) global.gc();
+// BAD: setImmediate in afterAll
+afterAll(async () => {
+  await new Promise((resolve) => setImmediate(resolve));
+});
+```
+
+### Windows Directory Cleanup
+
+Always use retry logic:
 
 ```typescript
 await fsp.rm(tempDir, {
@@ -250,125 +159,42 @@ await fsp.rm(tempDir, {
 });
 ```
 
-**Best Practice**: Use existing test utilities that handle Windows-compatible cleanup patterns. Don't manually clean up temp directories - let the test framework handle it with proper retry logic.
+### Platform Performance Multipliers
 
-### Adaptive Timeout Testing
+- Alpine Linux (musl): 2x slower
+- ARM64 emulation: 5x slower
+- Windows processes: 4x slower
+- macOS VMs: 4x slower
 
-**Problem**: Fixed timeouts don't account for varying CI environment performance.
+### Multi-Process Synchronization
 
-**Root Causes**:
+Use explicit signals:
 
-- Alpine Linux (musl libc) is 2x slower than glibc
-- ARM64 emulation on x64 runners is 5x slower
-- Windows process operations are 4x slower
-- macOS VMs are 4x slower
-- CI environments have resource constraints
-
-**Solutions**:
-
-```typescript
-// DON'T: Use fixed timeouts
-test("my test", async () => {
-  // Test code
-}, 10000);
-
-// DO: Use adaptive timeouts based on environment
-import { getTestTimeout } from "./test-utils/test-timeout-config";
-
-test(
-  "my test",
-  async () => {
-    // Test code
-  },
-  getTestTimeout(10000),
-);
-
-// DO: Account for platform timing differences
-const timingMultiplier = process.platform === "win32" ? 4 :
-                        process.platform === "darwin" ? 4 :
-                        process.env.CI ? 2 : 1;
+```javascript
+console.log("READY"); // Signal readiness
+console.log("RESULT:" + outcome); // Signal result
 ```
 
-### Multi-Process Test Synchronization
+## Release Process
 
-**Problem**: Multi-process tests failing due to race conditions between process startup and test assertions.
+Requires repository secrets:
 
-**Root Cause**: The timing between process startup and resource acquisition varies significantly by platform.
+- `NPM_TOKEN`: npm authentication
+- `GPG_PRIVATE_KEY`: ASCII-armored GPG key
+- `GPG_PASSPHRASE`: GPG passphrase
 
-**Solutions**:
+Automated via GitHub Actions workflow dispatch or manual:
 
-```typescript
-// DON'T: Assume immediate process readiness
-const proc = spawn(nodeCmd, [script]);
-const result = await waitForProcessResult(proc);
-expect(result).toBe("expected_outcome"); // May fail due to timing
-
-// DO: Use explicit synchronization signals
-const script = `
-  // Setup code
-  console.log("READY");  // Signal readiness
-  // Main test logic
-  console.log("RESULT:" + outcome);
-`;
-
-const proc = spawn(process.execPath, ["-e", script]);
-await waitForOutput(proc, "READY"); // Wait for process to be ready
-const result = await waitForOutput(proc, "RESULT:");
-expect(result.split(":")[1]).toBe("expected_outcome");
+```bash
+npm run prepare-release
+git config commit.gpgsign true
+npm version patch|minor|major
+npm publish
+git push origin main --follow-tags
 ```
 
-### Wait-for-Condition Pattern
+## General guidance
 
-**Problem**: Tests failing because they don't wait for asynchronous conditions to be met.
-
-**Solution**: Implement robust condition waiting with platform-aware timing:
-
-```typescript
-async function waitForCondition(
-  check: () => boolean | Promise<boolean>,
-  options: {
-    maxAttempts?: number;
-    delay?: number;
-    timeoutMs?: number;
-  } = {}
-) {
-  const {
-    maxAttempts = 50,
-    delay = 100,
-    timeoutMs = 30000
-  } = options;
-  
-  const startTime = Date.now();
-  
-  for (let i = 0; i < maxAttempts; i++) {
-    if (Date.now() - startTime > timeoutMs) {
-      throw new Error(`Condition not met within ${timeoutMs}ms`);
-    }
-    
-    if (await check()) return true;
-    await new Promise((resolve) => setTimeout(resolve, delay));
-  }
-  
-  return false;
-}
-
-// Usage example
-await waitForCondition(
-  () => fs.existsSync(expectedFile),
-  { timeoutMs: getTestTimeout(10000) }
-);
-```
+Never do inline imports like `const { mkdirSync } = await import("node:fs");` -- just use standard imports.
 
-### Best Practices Summary
-1. **Always clean up**: Resources, timers, workers, file handles
-2. **Never assume timing**: Use ranges and adaptive timeouts, not exact values
-3. **Isolate tests**: Each test should be independent
-4. **Platform awareness**: Skip tests that can't work on certain platforms
-5. **Proper async handling**: Always await or return promises
-6. **Resource limits**: Don't spawn unlimited workers/processes
-7. **Environment detection**: Adjust behavior for CI vs local
-8. **Deterministic tests**: Mock external dependencies when possible
-9. **Explicit synchronization**: Use signals for multi-process coordination
-10. **Robust waiting**: Use condition-based waiting instead of arbitrary timeouts
-11. **Windows compatibility**: Use retry logic for file operations on Windows
-12. **Anti-pattern awareness**: Avoid masking problems with timeouts or forced GC
+Never include claude code messages in git commits

package/CODE_OF_CONDUCT.md

@@ -17,23 +17,23 @@ diverse, inclusive, and healthy community.
 Examples of behavior that contributes to a positive environment for our
 community include:
 
-* Demonstrating empathy and kindness toward other people
-* Being respectful of differing opinions, viewpoints, and experiences
-* Giving and gracefully accepting constructive feedback
-* Accepting responsibility and apologizing to those affected by our mistakes,
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes,
   and learning from the experience
-* Focusing on what is best not just for us as individuals, but for the
+- Focusing on what is best not just for us as individuals, but for the
   overall community
 
 Examples of unacceptable behavior include:
 
-* The use of sexualized language or imagery, and sexual attention or
+- The use of sexualized language or imagery, and sexual attention or
   advances of any kind
-* Trolling, insulting or derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or email
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email
   address, without their explicit permission
-* Other conduct which could reasonably be considered inappropriate in a
+- Other conduct which could reasonably be considered inappropriate in a
   professional setting
 
 ## Enforcement Responsibilities
@@ -106,7 +106,7 @@ Violating these terms may lead to a permanent ban.
 ### 4. Permanent Ban
 
 **Community Impact**: Demonstrating a pattern of violation of community
-standards, including sustained inappropriate behavior,  harassment of an
+standards, including sustained inappropriate behavior, harassment of an
 individual, or aggression toward or disparagement of classes of individuals.
 
 **Consequence**: A permanent ban from any sort of public interaction within

package/CONTRIBUTING.md

@@ -26,7 +26,7 @@ Install the Xcode Command Line Tools, and then
 
 ### On Ubuntu/Debian
 
-    sudo apt-get install build-essential clang-format libglib2.0-dev libblkid-dev uuid-dev
+    sudo apt-get install bear build-essential clang-format libglib2.0-dev libblkid-dev uuid-dev
 
 ## Before submitting your PR