xrootd 0.2.2 → 1.0.0-beta.1

package/README.md

@@ -1,146 +1,383 @@
-# XRootD TypeScript Support
+# xrootd
 
-A high-performance, enterprise-grade Node.js binding for the [XRootD](https://github.com/xrootd/xrootd) client library. Designed to bring seamless, extreme-throughput data access to the TypeScript ecosystem.
+A TypeScript client library for the [XRootD](https://xrootd.org) protocol.
 
-Github Repo: [**Wu-Yijun/xrootd**](https://github.com/Wu-Yijun/xrootd); NPM package: [**XRootD**](https://www.npmjs.com/package/xrootd)
+XRootD (eXtended ROOT Daemon) is a high-performance, fault-tolerant protocol for accessing and managing large-scale distributed storage systems. It is widely used in High Energy Physics (HEP) for data access at facilities like CERN's LHC experiments.
 
-> [!NOTE]
-> **Work in Progress**: This package is currently under active development with some features not supported yet. While core features are highly functional, the API may undergo refinements. Api interfaces are not tests thoroughly so it may be unstable.
+> **🚀 v2.0.0 is Here: A Modern, Native Rewrite!**
+> ⚠️ **Breaking Changes**: Because of the fundamental architecture shift, **v2 is not API-compatible with v1**.
+> Please read our [Migration Guide (MIGRATING.md)](MIGRATING.md) to upgrade.
 
+## Installation
 
-## Supported Capabilities
+```sh
+npm install xrootd
+```
 
-* **`FileSystem`**: Cluster-level operations (`stat`, `dirList`, `rm`, `mkdir`, `locate`).
-* **`File`**: High-performance I/O (`open`, `read`, `write`, `vectorRead`, server-side `clone`).
-* **`CopyProcess`**: Asynchronous, highly-parallel data transfers with real-time progress callbacks.
-* **`Env`**: Type-safe configuration management and OS-level auth protocol routing.
-* **`Url`**: XRootD-specific URI scheme parsing and validation.
+## Quick Start
 
+```ts
+import { XRootDClient, OpenFlags } from 'xrootd'
 
-## Prerequisites
+const client = new XRootDClient('root://server.example.com')
+await client.connect()
 
-To ensure compatibility with the pre-compiled native bindings, your environment must meet the following requirements:
+// List directory contents
+const list = await client.readdir('/data')
+for (const entry of list.entries) {
+  console.log(entry.name, entry.size)
+}
 
-* **Node.js**: Version 20.x or higher.
-* **System Dependencies (Linux)**: Requires `libstdc++` providing `GLIBCXX_3.4.26` or newer (typically GCC 9+, e.g., CentOS 9 / Ubuntu 20.04+).
-* **System Dependencies (Macos)**: Requires macos 14 or later for arm64 architecture, macos 15 or later for intel architecture.
-* **Authentication**: Ensure your host machine has valid tickets (e.g., `kinit` for Kerberos) if accessing a secured cluster.
+// Read a file
+const file = await client.open('/data/file.dat')
+const data = await file.read(0, 1024)
+await file.close()
 
+// Write a file
+const file2 = await client.open('/data/output.dat', { flags: OpenFlags.Write | OpenFlags.New })
+await file2.write(0, new Uint8Array([72, 101, 108, 108, 111]))
+await file2.close()
 
-## Installation
+await client.close()
+```
 
-```bash
-npm install xrootd
-# or
-yarn add xrootd
+## API
+
+### XRootDClient
+
+High-level client that manages connection, authentication, and automatic redirect handling.
+
+```ts
+import { XRootDClient, OpenFlags } from 'xrootd'
+
+const client = new XRootDClient('root://server.example.com', {
+  credentials: { username: 'user', password: 'pass' },
+  timeout: 30000,
+  maxRedirects: 16,
+})
+
+await client.connect()
+
+// Connection state
+console.log(client.isConnected)  // boolean
+console.log(client.location)     // "root://server.example.com:1094/"
+
+// File operations
+const file = await client.open('/data/file.dat', { flags: OpenFlags.Read })
+const data = await file.read(0, 1024)
+await file.close()
+
+// Filesystem operations
+await client.mkdir('/new/dir')
+const list = await client.readdir('/data')
+await client.mv('/old/path', '/new/path')
+await client.rm('/old/file')
+await client.rmdir('/old/dir')
+
+// Metadata — two approaches
+const info1 = await client.stat('/data/file.dat')          // opens + stats + closes
+const info2 = await client.statFilesystem('/data/file.dat') // filesystem protocol, no file open
+
+await client.close()
+```
+
+### File
+
+File operations for reading, writing, and managing files on XRootD servers.
+
+Obtained via `client.open()` — do not construct directly.
+
+```ts
+// Open a file
+const file = await client.open('/data/file.dat', { flags: OpenFlags.Read })
+
+// Core operations
+const data: Uint8Array = await file.read(offset, size)
+const written: number = await file.write(offset, data)
+const info: StatInfo = await file.stat()
+await file.sync()
+await file.truncate(size)
+await file.close()
+```
+
+### FileSystem
+
+Stateless filesystem metadata operations:
+
+```ts
+import { FileSystem } from 'xrootd'
+
+// Obtained internally by XRootDClient; shown here for reference
+const fs = new FileSystem(mux)
+
+// Metadata
+const info = await fs.stat('/data/file.dat')
+console.log(info.size, info.mtime, info.isDirectory)
+
+// Directory operations
+await fs.mkdir('/new/dir')
+const list = await fs.readdir('/data')
+await fs.mv('/old/path', '/new/path')
+await fs.rm('/old/file')
+await fs.rmdir('/old/dir')
+```
+
+### Authentication
+
+XRootD supports multiple authentication mechanisms. The library automatically negotiates the best available protocol during handshake.
+
+#### Supported Protocols
 
+| Protocol | Description |
+|----------|-------------|
+| `host` | Host-based trust authentication (IP whitelist) |
+| `sss` | Simple Shared Secret (Blowfish-ECB encrypted password) |
+
+#### Configuration
+
+```ts
+import { XRootDClient } from 'xrootd'
+
+// host authentication (server trusts client by IP)
+const client = new XRootDClient('root://server.example.com', {
+  credentials: { username: 'user' },
+})
+
+// SSS authentication (shared secret)
+const client = new XRootDClient('root://server.example.com', {
+  credentials: {
+    username: 'user',
+    password: 'shared-secret',
+  },
+})
+
+await client.connect()
 ```
 
----
+When connecting to a server that requires authentication, the client will automatically:
 
-## Usage Examples
+1. Receive the server's supported authentication protocols (`secReqs`)
+2. Select the first matching protocol from the list
+3. Execute the authentication handshake (including multi-round `kXR_authmore` challenges)
 
-### 1. Configuration & Basic FileSystem
+### Error Handling
 
-```typescript
-import { Env, FileSystem } from 'xrootd';
+All errors are thrown as `XRootDError` instances:
 
-// Safely configure underlying XRootD environment
-Env.configure({
-    RequestTimeout: 30,       // Prevent 30-min infinite hangs (Node.js friendly)
-    WorkerThreads: 4,         // Boost underlying multiplexing
-    SecProtocol: 'krb5,unix'  // Force Kerberos or Unix socket auth
-});
+```ts
+import { XRootDClient, OpenFlags, XRootDError } from 'xrootd'
 
-const fs = new FileSystem('root://eos01.ihep.ac.cn/');
+const client = new XRootDClient('root://server.example.com')
+await client.connect()
 
-async function checkCluster() {
-    try {
-        const stat = await fs.stat('/eos/lhaaso/data');
-        console.log(`Directory size: ${stat.size} bytes`);
-        console.log(`Modified at: ${stat.modTimeAsString}`);
-    } catch (err: any) {
-        if (err.code === 'ENOENT') {
-            console.error('File not found in cluster.');
-        } else {
-            console.error(err.message);
-        }
-    }
+try {
+  const file = await client.open('/nonexistent', { flags: OpenFlags.Read })
+  await file.read(0, 1024)
+  await file.close()
+} catch (err) {
+  if (err instanceof XRootDError) {
+    console.log(err.code)     // 3011 (NotFound)
+    console.log(err.message)  // "File not found"
+    console.log(err.errno)    // POSIX errno (if applicable)
+  }
+}
+```
+
+#### Common Error Codes
+
+| Code | Constant | Description |
+|------|----------|-------------|
+| 3010 | `NotAuthorized` | Permission denied |
+| 3011 | `NotFound` | File or directory not found |
+| 3016 | `IsDirectory` | Expected file, got directory |
+| 3018 | `ItExists` | File already exists (e.g. mkdir on existing path) |
+| 3030 | `AuthFailed` | Authentication failed |
+
+#### Client-side Error Codes
+
+| Code | Constant | Description |
+|------|----------|-------------|
+| 309 | `Timeout` | Request timed out |
+| 312 | `Disconnected` | Connection closed unexpectedly |
+| 315 | `TooManyRedirs` | Exceeded maximum redirect count |
+
+### Types
+
+```ts
+interface StatInfo {
+  id: string        // opaque device id (string to avoid precision loss)
+  size: bigint      // file size in bytes (bigint for >4GB files)
+  flags: number     // XRootD server flags
+  mtime: number     // modification time (epoch seconds)
+  ctime: number     // change time (epoch seconds)
+  atime: number     // access time (epoch seconds)
+  mode: number      // POSIX mode (e.g. 0o100644)
+  owner: string     // file owner
+  group: string     // file group
+  get isDirectory(): boolean
+  get isLink(): boolean
+  get isOffline(): boolean
+  get isCached(): boolean
 }
 
+interface DirectoryEntry {
+  name: string
+  size: number
+  flags: number
+  mtime: number
+}
+
+interface DirectoryList {
+  name: string
+  entries: DirectoryEntry[]
+}
+
+const enum OpenFlags {
+  Compress  = 0x0001,
+  Delete    = 0x0002,
+  Force     = 0x0004,
+  New       = 0x0008,
+  Read      = 0x0010,
+  Write     = 0x0020,
+  Async     = 0x0040,
+  Refresh   = 0x0080,
+  Mkpath    = 0x0100,
+  Append    = 0x0200,
+  Retstat   = 0x0400,
+  Replica   = 0x0800,
+  Posc      = 0x1000,
+  Nowait    = 0x2000,
+  Seqio     = 0x4000,
+  Wrto      = 0x8000,
+}
 ```
 
-### 2. Zero-Copy File Reading
+#### OpenOptions
 
-```typescript
-import { File, OpenFlags } from 'xrootd';
+```ts
+interface OpenOptions {
+  flags?: OpenFlags   // file open flags (default: OpenFlags.Read)
+  mode?: number       // POSIX mode (default: 0)
+  signal?: AbortSignal  // cancellation signal
+}
+```
 
-async function readHeader() {
-    const file = new File();
-    
-    // Auto-translates and handles C++ lifecycle
-    await file.open('root://eos01.ihep.ac.cn//eos/test.dat', OpenFlags.Read);
-    
-    // The returned buffer is a zero-copy mapping of C++ allocated memory
-    const buffer = await file.read(0n, 1024);
-    console.log(buffer.toString('utf-8'));
-    
-    await file.close();
+#### StatFlags
+
+```ts
+const StatFlags = {
+  XBitSet: 1,
+  IsDir: 2,
+  Other: 4,
+  Offline: 8,
+  Readable: 16,
+  Writable: 32,
+  POSCPending: 64,
+  BackUpExists: 128,
+  CacheResp: 512,
 }
+```
+
+### Redirect Handling
 
+The client automatically handles server redirects. When a server responds with `kXR_redirect`, the client:
+
+1. Closes the current connection
+2. Connects to the new target server
+3. Re-executes the handshake and authentication
+4. Retries the original request
+
+```ts
+const client = new XRootDClient('root://server.example.com', {
+  maxRedirects: 16, // default
+})
 ```
 
-### 3. High-Performance CopyProcess with Progress
+## Architecture
 
-```typescript
-import { CopyProcess } from 'xrootd';
+The library uses a three-layer architecture, simplified from the original C++ XrdCl's five-layer design:
 
-async function syncData() {
-    const cp = new CopyProcess();
-    
-    cp.addJob({
-        source: 'root://eos01.ihep.ac.cn//eos/data.raw',
-        target: '/local/disk/data.raw',
-        force: true,
-        parallelChunks: 4
-    });
+```
+┌──────────────────────────────────────────────────┐
+│  Layer 3: Multiplexer                            │
+│  streamId → Promise mapping, timeout management  │
+│  + automatic redirect interception               │
+├──────────────────────────────────────────────────┤
+│  Layer 2: Framer                                 │
+│  TCP packet reassembly, Header+Body framing      │
+├──────────────────────────────────────────────────┤
+│  Layer 1: Transport                              │
+│  net.Socket wrapper, binary data I/O             │
+└──────────────────────────────────────────────────┘
+```
 
-    await cp.prepare();
-    
-    const results = await cp.run((jobNum, processed, total) => {
-        const percent = ((processed / total) * 100).toFixed(2);
-        console.log(`Job ${jobNum} Progress: ${percent}%`);
-    });
+| C++ Original | TypeScript |
+|--------------|-----------|
+| 138 files, ~15,000 lines | ~25 files, ~3,000 lines |
+| Callbacks + state machines | async/await |
+| Error code checking | throw XRootDError |
+| 5-layer abstraction | 3-layer streamlined architecture |
+| Multiple auth plugins | Pluggable SecurityProtocol interface |
 
-    console.log('Copy completed:', results);
-}
+## Development
 
+### Prerequisites
+
+- Node.js >= 22
+- pnpm
+
+### Setup
+
+```sh
+pnpm install
 ```
 
-## ✨ Why `xrootd`? (Core Architecture)
+### Mock Server
 
-Unlike traditional Node.js C++ addons, `xrootd` is architected for **High Energy Physics (HEP)** and **Big Data** workloads:
+A Docker-based XRootD mock server is available for integration testing:
 
-*  **True Native Async**: Bypasses the notoriously limited Node.js `libuv` thread pool (default 4 threads). It hooks directly into XRootD's underlying C++ event loop via N-API `ThreadSafeFunction`, allowing thousands of concurrent requests without blocking the V8 engine.
-*  **Absolute Zero-Copy I/O**: Implements direct memory handoffs. Data read from the EOS cluster is mounted directly as V8 `Buffer` objects without a single byte of internal memory copying, completely eliminating GC (Garbage Collection) pauses during heavy I/O.
-*  **Idiomatic Node.js Experience**: "Thin C++, Thick TS". Complex XRootD protocol errors (e.g., `3011`) are smartly translated into standard Node.js exceptions (e.g., `ENOENT`, `EACCES`), complete with system call contexts and retryable flags.
-*  **Zero-Config Authentication**: Bundles essential security plugins (Kerberos, SSS, Unix, Token) natively. Path injection is handled automatically under the hood—no more `[FATAL] Auth failed` or missing `.so` nightmares.
+```sh
+# Start mock server
+pnpm mock-server:up
 
----
+# Verify it's running
+pnpm mock-server:verify
 
-##  Licensing
+# View logs
+pnpm mock-server:logs
 
-This project is released under a **Dual License** strategy to balance open-source compatibility and developer freedom:
+# Stop
+pnpm mock-server:down
+```
 
-1. **[GNU GPLv3](LICENSE-GPLv3)**: The native binding codebase adheres to the GPLv3 license to remain fully compatible with the upstream C++ XRootD project.
-2. **[MIT](LICENSE-MIT)**: The core TypeScript APIs, interface definitions, and glue layers are provided under the MIT license, allowing you to integrate the TS components into your own software architectures without viral restrictive requirements.
+### Build
+
+```sh
+pnpm build
+```
+
+### Type Check
+
+```sh
+pnpm typecheck
+```
+
+### Test
+
+```sh
+pnpm test
+```
 
-*Disclaimer: This project is a third-party community initiative built for modern web ecosystems and is not affiliated with, officially endorsed by, or sponsored by the core XRootD project.*
+## Documentation
 
----
+- [Migration Design](docs/migration.md) — Full protocol analysis and architecture design
+- [TypeScript API Design](docs/typescript-design.md) — Detailed API specifications
+- [Work Plan](docs/work-plan.md) — Implementation roadmap
+- [Phase 2 Plan](docs/phase2.md) — Complete API, fault tolerance, and authentication
+- [Rust Future Optimizations](docs/rust-future-optimizations.md) — Performance optimizations deferred to Rust version
 
-## Contributing
+## License
 
-We welcome contributions from the high-energy physics, astrophysics, and Node.js communities! Whether it's reporting bugs, improving documentation, or adding support for advanced XRootD features.
+This project is licensed under the [GNU Lesser General Public License v3.0 or later](LICENSE).
 
-*Contribution guidelines coming soon.*
+This is an independent TypeScript implementation of the XRootD protocol. The original XRootD project is developed by the Board of Trustees of the Leland Stanford, Jr. University and is licensed under LGPL-3.0-or-later.