arcane-os 0.29.0 → 0.29.1

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.29.1
+
+- Preserve complete long and non-ASCII filenames in application release bundles
+  through standard per-file PAX path extensions. Public bundle creation and
+  verification retain logical payload paths and complete file content without
+  application-side renaming. Ordinary USTAR representation remains supported,
+  and the existing SDK-version metadata contract is unchanged. Bundles using
+  extended paths require the updated SDK reader or another PAX-capable reader.
+- Preserve application-authored resource URL queries, including `v`, encoded
+  and repeated fields, empty query segments, and fragments. Import-map refresh,
+  serving, packaging, and PWA delivery now change only SDK-owned `arcaneVersion`
+  fields through the existing shared transformer.
+
 ## 0.29.0
 
 - Add `legacyAppPaths: false` to `arcane-packager.json` for root applications

package/README.md

@@ -19,7 +19,7 @@ version-locked SDK runtime, while an integrated Arcane checkout uses its live
 `arcane/` runtime. Both profiles preserve the same app URLs, theme, packaging,
 event, cancellation, and browser run contracts.
 
-This checkout defines the `0.29.0` SDK contract. Applications pin one exact npm
+This checkout defines the `0.29.1` SDK contract. Applications pin one exact npm
 version and lockfile; registry state is deliberately not baked into application
 artifacts.
 
@@ -604,9 +604,13 @@ prior file until the replacement is complete. Cancellation or failure before
 commit restores the prior output when that can be done without overwriting a
 concurrent change. A conflicting or uncertain path is preserved for inspection.
 
-The archive uses the documented USTAR+gzip structure and publishes its v1 JSON
-contract at `arcane-os/schemas/arcane-app-bundle.json`. When the user explicitly
-selects bundle verification, it rejects malformed archives, links, devices,
+The archive uses USTAR+gzip with per-file PAX path extensions for long or
+non-ASCII filenames and publishes its v1 JSON contract at
+`arcane-os/schemas/arcane-app-bundle.json`. Logical filenames and content remain
+complete; ordinary USTAR paths retain their existing representation, and bundle
+metadata keeps its existing SDK-version matching contract. Readers must support
+PAX path extensions to consume bundles that need them. When the
+user explicitly selects bundle verification, it rejects malformed archives, links, devices,
 unsafe or colliding paths, unsupported archive members, trailing data, and
 inconsistent descriptor or inventory structure. These checks reject corrupt
 selected artifacts; they do not impose byte-count, hash, provenance, or

package/docs/architecture.md

@@ -273,16 +273,31 @@ External repository delivery adds a distinct schema-1
 `arcane-app-release-bundle` envelope. Bundle creation uses an authored schema-2
 `arcane-app.json`; a synthesized package or registry projection remains valid for integrated
 packaging but is not used for an external bundle. The
-archive contains exactly `ARCANE_APP_BUNDLE.json`, canonical `arcane-app.json`,
+logical file inventory contains exactly `ARCANE_APP_BUNDLE.json`, canonical `arcane-app.json`,
 `payload/ARCANE_APP_RELEASE.json`, and the release inventory beneath `payload/`
 in that order. The envelope adds no repository-only source or build tooling
 beyond that selected release inventory. Individual apps remain responsible for
 their authored source policy.
 
-The bundle contract uses the documented USTAR+gzip structure. Explicit bundle
-verification parses the selected archive without extraction and rejects
-genuinely malformed structures, unsafe or colliding paths, unsupported members,
-trailing data, and incompatible bundle generations. These corrupt-artifact
+The bundle contract retains schema 1 and the `ustar+gzip` format identifier.
+Ordinary representable ASCII paths use the existing USTAR headers. Long paths
+and non-ASCII names use a [POSIX PAX](https://docs.oracle.com/cd/E86824_01/html/E54763/pax-1.html)
+per-file `x` header containing the complete
+UTF-8 `path`, immediately before that file's regular header and content. This
+transport framing is not a payload member and never appears in the logical
+inventory or `readFile()` results. Path spelling and file content remain unchanged;
+the existing portable-path rules still apply. The public
+`createCanonicalUstarHeader()` helper remains a single USTAR header and retains
+its format-local field limits; complete bundles use `createAppReleaseBundle()`.
+
+The bundle reader consumes these `path` extensions for the following file only,
+as well as ordinary USTAR entries. Other PAX attributes and archive member
+types remain outside this bundle profile. Older SDK readers without PAX support
+cannot read bundles that require these extensions; update the consuming SDK
+before importing one. The existing same-SDK-version bundle condition is unchanged.
+Explicit bundle verification parses the selected archive without extraction and
+rejects genuinely malformed structures, unsafe or colliding paths, unsupported
+members, and incompatible bundle generations. These corrupt-artifact
 checks do not create byte-count, content-hash, provenance, or admission gates
 for ordinary development, packaging, serving, or running.
 

package/docs/reference/asset-versioning.md

@@ -3,7 +3,7 @@
 Applications that enable [PWA delivery](pwa.md) use clean local resource URLs.
 Their generated offline manifest and service worker own the selected application
 and SDK release information. In that mode, the delivery transformer removes
-both `v` and `arcaneVersion`, preserving other query fields and fragments.
+only the SDK-owned `arcaneVersion` field, preserving authored query fields and fragments.
 The behavior below continues to apply when PWA delivery is disabled and to native
 packages. Workspace runtime materialization remains usable by either target;
 the selected browser delivery applies its PWA URL policy.
@@ -15,11 +15,12 @@ SDK package metadata, not a timestamp, content measurement, or application
 constant.
 
 For example, an existing `./arcane/modules/HTMLImport.js?v=6#module` reference
-becomes `./arcane/modules/HTMLImport.js?arcaneVersion=${version}#module`.
-`arcaneVersion` is the sole resource version field: transformation removes `v`,
-updates the first existing `arcaneVersion`, and removes duplicate version fields.
-Regenerating for another SDK release replaces that version value. Unrelated
-query fields, their spelling, and fragments remain intact.
+becomes `./arcane/modules/HTMLImport.js?v=6&arcaneVersion=${version}#module`.
+`arcaneVersion` is the SDK's resource version field: transformation updates its
+first existing value and removes duplicate `arcaneVersion` fields, or appends it
+when absent. Regenerating for another SDK release replaces only that SDK value.
+Authored fields, including `v`, encoded keys and values, repeated or empty query
+segments, their source spelling, and fragments remain intact.
 
 ## Public tooling
 

package/docs/reference/inventory/package-api.json

@@ -464,14 +464,14 @@
       "name": "createAppReleaseBundle",
       "displayName": "createAppReleaseBundle()",
       "kind": "function",
-      "signature": "async createAppReleaseBundle({ receipt, releaseRoot, outputPath, overwrite=false, signal, onEvent }={})",
+      "signature": "async createAppReleaseBundle({releaseRoot, appDescriptor, outputPath, overwrite=false, signal, onEvent}={})",
       "entrypoints": [
         "arcane-os",
         "arcane-os/release-bundle"
       ],
       "primaryImport": "arcane-os",
       "group": "Packaging and release bundles",
-      "summary": "Writes one deterministic USTAR+gzip external application bundle from the selected authored release state.",
+      "summary": "Writes one deterministic USTAR+gzip external application bundle, using per-file POSIX PAX path extensions for complete long or non-ASCII filenames.",
       "availability": "Node",
       "protocol": "SDK packager and deterministic bundle contract",
       "normalization": "Normalized SDK validation with complete canonical archive and release content"
@@ -2164,7 +2164,7 @@
       ],
       "primaryImport": "arcane-os",
       "group": "Packaging and release bundles",
-      "summary": "Parses and authenticates one deterministic app bundle without extraction.",
+      "summary": "Parses one deterministic app bundle without extraction, supporting ordinary USTAR entries and complete per-file POSIX PAX paths within the existing same-SDK-version contract.",
       "availability": "Node",
       "protocol": "SDK packager and deterministic bundle contract",
       "normalization": "Normalized SDK validation with complete canonical archive and release content"

package/docs/reference/pwa.md

@@ -150,9 +150,10 @@ The same bootstrap starts one initially hidden `pwa-install.html` component with
 the generated manifest's app name. Component loading and worker registration
 proceed independently.
 
-The selected PWA browser delivery removes `v` and `arcaneVersion` from actual
-local resource references, including the managed import map. Other query fields,
-fragments, source spelling and unrelated payloads are preserved. The offline
+The selected PWA browser delivery removes only SDK-owned `arcaneVersion` fields
+from actual local resource references, including the managed import map. Authored
+fields such as `v`, encoded and repeated fields, empty query segments, fragments,
+source spelling and unrelated payloads are preserved. The offline
 manifest now carries release information. Non-PWA and native delivery retain
 the [existing asset version contract](asset-versioning.md).
 

package/docs/reference/sdk-api.md

@@ -629,12 +629,17 @@ async function previewNextPrerelease(workspaceRoot) {
 
 ### Overview
 
-Writes one deterministic USTAR+gzip external application bundle from the selected authored release state.
+Writes one deterministic USTAR+gzip external application bundle from the selected
+authored release state. Long or non-ASCII filenames use per-file POSIX PAX `path`
+extensions, preserving their complete UTF-8 spelling and file content. Ordinary
+USTAR entries remain unchanged. Metadata framing is excluded from the manifest's
+logical file inventory. Existing portable-path rules, schema 1, and the
+`ustar+gzip` format identifier remain unchanged.
 
 ### Signature and result
 
 ```text
-async createAppReleaseBundle({ receipt, releaseRoot, outputPath, overwrite=false, signal, onEvent }={})
+async createAppReleaseBundle({releaseRoot, appDescriptor, outputPath, overwrite=false, signal, onEvent}={})
 ```
 
 Import it from `arcane-os` or `arcane-os/release-bundle`. The signature above states whether settlement is synchronous or promise-based. The overview and owning group define result authority, side effects, callbacks, events, cancellation, and lifecycle.
@@ -658,6 +663,9 @@ async function usecreateAppReleaseBundle(...arguments_) {
 ### Overview
 
 Builds the exact 512-byte canonical USTAR header for one validated bundle entry.
+This low-level helper remains a single regular-file header with the USTAR field
+limits. Use `createAppReleaseBundle()` to encode complete long or Unicode paths;
+the bundle writer supplies PAX framing where needed.
 
 ### Signature and result
 
@@ -1104,7 +1112,14 @@ async function useverifyApp(...arguments_) {
 
 ### Overview
 
-Parses and authenticates one deterministic app bundle without extraction.
+Parses one deterministic app bundle without extraction and checks its structural
+manifest and complete logical file inventory. It supports ordinary USTAR entries
+and per-file POSIX PAX `path` extensions. The complete extended path is used by
+`files` and `readFile()`; the metadata and regular-header placeholder are not
+payload files. Malformed or orphaned PAX records produce `ARCANE_BUNDLE_INVALID`.
+Other PAX attributes and archive member types remain unsupported. The existing
+same-SDK-version condition remains unchanged. An older reader without PAX support
+must be updated before importing a bundle that requires these path extensions.
 
 ### Signature and result
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "arcane-os",
-  "version": "0.29.0",
+  "version": "0.29.1",
   "description": "Arcane OS JavaScript SDK, project-local CLI, browser runtime, and repository-portable application packager.",
   "type": "module",
   "main": "./src/index.mjs",

package/schemas/arcane-app-bundle.schema.json

@@ -2,7 +2,7 @@
   "$schema": "https://json-schema.org/draft/2020-12/schema",
   "$id": "https://github.com/TheWizardNexus/arcane-os-sdk/schemas/arcane-app-bundle.schema.json",
   "title": "Arcane external application release bundle",
-  "description": "Structural manifest for one complete USTAR+gzip application bundle.",
+  "description": "Structural manifest for one complete USTAR+gzip application bundle, with per-file POSIX PAX path extensions for long or non-ASCII filenames. PAX framing is excluded from the logical file inventory.",
   "type": "object",
   "required": [
     "schemaVersion",

package/src/import-map.mjs

@@ -847,7 +847,7 @@ function assetUrlVersionEdits(value,version){
         let decodedKey=key;
         try{decodedKey=decodeURIComponent(key.replaceAll('+',' '));}
         catch{decodedKey=key;}
-        const remove=decodedKey==='v'||(decodedKey==='arcaneVersion'&&(clean||versionFound));
+        const remove=decodedKey==='arcaneVersion'&&(clean||versionFound);
         parameters.push({start:offset,end:offset+parameter.length,remove});
         if(decodedKey==='arcaneVersion'&&!versionFound&&!clean){
             versionFound=true;
@@ -859,10 +859,10 @@ function assetUrlVersionEdits(value,version){
         }
         offset+=parameter.length+1;
     }
-    if(clean&&!parameters.some(function hasRemainingField(parameter){
-        return !parameter.remove&&parameter.end>parameter.start;
+    if(clean&&parameters.every(function removesSdkField(parameter){
+        return parameter.remove;
     }))return [{start:queryStart,end:address.length,value:''}];
-    // Remove adjacent obsolete fields together, including only their separator.
+    // Remove adjacent SDK version fields together, including only their separator.
     // Other field spelling and source-level escapes remain untouched.
     for(let index=0;index<parameters.length;index+=1){
         if(!parameters[index].remove)continue;
@@ -874,13 +874,10 @@ function assetUrlVersionEdits(value,version){
             value:''
         });
     }
-    const lastRetained=parameters.findLast(function retainedParameter(parameter){
-        return !parameter.remove;
-    });
     if(!clean&&!versionFound)edits.push({
         start:address.length,
         end:address.length,
-        value:`${lastRetained&&lastRetained.end>lastRetained.start?'&':''}arcaneVersion=${versionValue}`
+        value:`&arcaneVersion=${versionValue}`
     });
     return edits;
 }

package/src/release-bundle.mjs

@@ -93,7 +93,7 @@ function splitUstarPath(archivePath){
             return {name,prefix};
         }
     }
-    fail(`Archive path cannot be represented by ustar: ${normalized}.`);
+    return null;
 }
 
 function writeTextField(header,offset,length,value,label){
@@ -112,8 +112,10 @@ function writeOctalField(header,offset,length,value,label,{trailingSpace=false}=
     header.write(rendered,offset,length,'ascii');
 }
 
-export function createCanonicalUstarHeader(archivePath,size){
-    const {name,prefix}=splitUstarPath(archivePath);
+function createTarHeader(archivePath,size,type){
+    const fields=splitUstarPath(archivePath);
+    if(!fields)fail(`Archive path cannot be represented by ustar: ${archivePath}.`);
+    const {name,prefix}=fields;
     const header=Buffer.alloc(TAR_BLOCK_SIZE);
     writeTextField(header,0,100,name,'ustar name');
     writeOctalField(header,100,8,ARCHIVE_MODE,'ustar mode');
@@ -122,7 +124,7 @@ export function createCanonicalUstarHeader(archivePath,size){
     writeOctalField(header,124,12,size,'ustar size');
     writeOctalField(header,136,12,0,'ustar mtime');
     header.fill(0x20,148,156);
-    header[156]=0x30;
+    header[156]=type;
     header.write('ustar\0',257,6,'ascii');
     header.write('00',263,2,'ascii');
     writeTextField(header,265,32,'root','ustar owner');
@@ -134,13 +136,45 @@ export function createCanonicalUstarHeader(archivePath,size){
     return header;
 }
 
-function tarEntry(archivePath,content){
-    const header=createCanonicalUstarHeader(archivePath,content.length);
+export function createCanonicalUstarHeader(archivePath,size){
+    return createTarHeader(archivePath,size,0x30);
+}
+
+function tarRecord(archivePath,content,type=0x30){
+    const header=createTarHeader(archivePath,content.length,type);
     const remainder=content.length%TAR_BLOCK_SIZE;
     const padding=remainder===0?Buffer.alloc(0):Buffer.alloc(TAR_BLOCK_SIZE-remainder);
     return Buffer.concat([header,content,padding]);
 }
 
+function paxPathRecord(archivePath) {
+    const value = Buffer.from(` path=${archivePath}\n`, 'utf8');
+    // POSIX PAX framing includes the decimal prefix itself in the record length.
+    let length = value.length + 1;
+    while(String(length).length + value.length !== length) {
+        length = String(length).length + value.length;
+    }
+    const prefix = Buffer.from(String(length), 'ascii');
+    return Buffer.concat(
+        [prefix, value]
+    );
+}
+
+function tarEntry(archivePath, content) {
+    const fields = splitUstarPath(archivePath);
+    if(fields && !/[^\u0000-\u007f]/u.test(archivePath)) {
+        return tarRecord(archivePath, content);
+    }
+    // The following regular header is only a placeholder; PAX owns the complete path.
+    const extendedPath = paxPathRecord(archivePath);
+    return Buffer.concat(
+        [
+            tarRecord('PaxHeaders/entry', extendedPath, 0x78),
+            tarRecord('PaxPayload/entry', content)
+        ]
+    );
+}
+
 async function realDirectory(location,label){
     const requested=path.resolve(location);
     let info;
@@ -308,9 +342,53 @@ function readOctalField(header,offset,length,label){
     return value;
 }
 
+function readPaxPath(content) {
+    let offset = 0;
+    let archivePath;
+    while(offset < content.length) {
+        const separator = content.indexOf(0x20, offset);
+        if(separator < 0) {
+            fail('Bundle contains malformed PAX record framing.');
+        }
+        const lengthText = content.subarray(offset, separator).toString('latin1');
+        if(!/^[1-9][0-9]*$/u.test(lengthText)) {
+            fail('Bundle contains malformed PAX record framing.');
+        }
+        const length = Number(lengthText);
+        const end = offset + length;
+        if(!is.safeInteger(length) || end > content.length || end <= separator + 1
+            || content[end - 1] !== 0x0a) {
+            fail('Bundle contains an incomplete or malformed PAX record.');
+        }
+        const assignment = content.indexOf(0x3d, separator + 1);
+        if(assignment <= separator + 1 || assignment >= end - 1) {
+            fail('Bundle contains a malformed PAX assignment.');
+        }
+        const keyword = content.subarray(separator + 1, assignment).toString('utf8');
+        if(keyword !== 'path') {
+            fail(`Bundle contains an unsupported PAX field: ${keyword}.`);
+        }
+        try {
+            const decoder = new TextDecoder(
+                'utf-8',
+                {fatal: true, ignoreBOM: true}
+            );
+            archivePath = decoder.decode(content.subarray(assignment + 1, end - 1));
+        } catch (error) {
+            fail(`Bundle PAX path is not valid UTF-8: ${error.message}.`);
+        }
+        offset = end;
+    }
+    if(archivePath === undefined) {
+        fail('Bundle PAX header is missing its path.');
+    }
+    return archivePath;
+}
+
 function readTarEntries(archive){
     const entries=new Map();
     let offset=0;
+    let pendingPath;
     while(offset+TAR_BLOCK_SIZE<=archive.length){
         const header=archive.subarray(offset,offset+TAR_BLOCK_SIZE);
         if(header.every(value=>value===0))break;
@@ -321,21 +399,31 @@ function readTarEntries(archive){
         let actualChecksum=0;
         for(const value of checksumHeader)actualChecksum+=value;
         if(actualChecksum!==expectedChecksum)fail('Bundle contains a malformed ustar header.');
-        if(header[156]!==0&&header[156]!==0x30)fail('Bundle contains a non-file archive entry.');
+        const type=header[156];
+        if(type!==0&&type!==0x30&&type!==0x78)fail('Bundle contains a non-file archive entry.');
         const name=readStringField(header,0,100);
         const prefix=readStringField(header,345,155);
-        const archivePath=validateAppBundlePath(prefix?`${prefix}/${name}`:name,'archive path');
-        if(entries.has(pathKey(archivePath)))fail(`Bundle contains a duplicate path: ${archivePath}.`);
+        const headerPath=prefix?`${prefix}/${name}`:name;
         const size=readOctalField(header,124,12,'ustar size');
         const contentStart=offset+TAR_BLOCK_SIZE;
         const contentEnd=contentStart+size;
-        if(contentEnd>archive.length)fail(`Bundle entry is incomplete: ${archivePath}.`);
+        if(contentEnd>archive.length)fail(`Bundle entry is incomplete: ${pendingPath??headerPath}.`);
+        const content=archive.subarray(contentStart,contentEnd);
+        offset=contentStart+Math.ceil(size/TAR_BLOCK_SIZE)*TAR_BLOCK_SIZE;
+        if(type===0x78){
+            if(pendingPath!==undefined)fail('Bundle PAX path is missing its following file.');
+            pendingPath=readPaxPath(content);
+            continue;
+        }
+        const archivePath=validateAppBundlePath(pendingPath??headerPath,'archive path');
+        pendingPath=undefined;
+        if(entries.has(pathKey(archivePath)))fail(`Bundle contains a duplicate path: ${archivePath}.`);
         entries.set(pathKey(archivePath),{
             path:archivePath,
-            content:Buffer.from(archive.subarray(contentStart,contentEnd))
+            content:Buffer.from(content)
         });
-        offset=contentStart+Math.ceil(size/TAR_BLOCK_SIZE)*TAR_BLOCK_SIZE;
     }
+    if(pendingPath!==undefined)fail('Bundle PAX path is missing its following file.');
     return entries;
 }