json-as 1.3.0 → 1.3.1

package/ARCHITECTURE.md

@@ -1,323 +0,0 @@
-# Architecture
-
-This document describes the architecture of json-as, a high-performance JSON serialization library for AssemblyScript.
-
-## Overview
-
-json-as uses a two-tier architecture:
-
-1. **Compile-time Transform**: A TypeScript-based AST transformer that generates optimized serialization/deserialization code
-2. **Runtime Library**: AssemblyScript implementations for JSON processing with multiple optimization levels
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                        Compile Time                             │
-│   ┌─────────────┐    ┌──────────────┐    ┌──────────────────┐   │
-│   │ Source Code │ -> │  Transform   │ -> │ Generated Code   │   │
-│   │ with @json  │    │ (TypeScript) │    │ __SERIALIZE etc  │   │
-│   └─────────────┘    └──────────────┘    └──────────────────┘   │
-└─────────────────────────────────────────────────────────────────┘
-                              │
-                              V
-┌─────────────────────────────────────────────────────────────────┐
-│                      Runtime                                    │
-│  ┌──────────────────────────────────────────────────────────┐   │
-│  │                    JSON Namespace                        │   │
-│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐   │   │
-│  │  │  stringify  │  │    parse    │  │  Dynamic Types  │   │   │
-│  │  └──────┬──────┘  └──────┬──────┘  │  Value, Obj,    │   │   │
-│  │         │                │         │  Box, Raw       │   │   │
-│  │         V                V         └─────────────────┘   │   │
-│  │  ┌───────────────────────────────────────────────────┐   │   │
-│  │  │               Optimization Modes                  │   │   │
-│  │  │         ┌───────┐  ┌────────┐  ┌────────┐         │   │   │
-│  │  │         │ NAIVE │  │  SWAR  │  │  SIMD  │         │   │   │
-│  │  │         └───────┘  └────────┘  └────────┘         │   │   │
-│  │  └───────────────────────────────────────────────────┘   │   │
-│  └──────────────────────────────────────────────────────────┘   │
-│                              │                                  │
-│                              V                                  │
-│  ┌──────────────────────────────────────────────────────────┐   │
-│  │              Memory Allocator (bs namespace)             │   │
-│  │  - Dynamic buffer management                             │   │
-│  │  - EMA-based adaptive sizing                             │   │
-│  │  - Optional string caching (sc namespace)                │   │
-│  └──────────────────────────────────────────────────────────┘   │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-## Transform System
-
-The transform (`transform/src/`) is an AssemblyScript compiler plugin that runs during compilation.
-
-### How It Works
-
-1. **Discovery**: Scans source files for classes decorated with `@json` or `@serializable`
-2. **Schema Building**: Creates a schema for each decorated class including:
-   - Field names and types
-   - Decorator metadata (@alias, @omit, @omitnull, @omitif)
-   - Inheritance relationships
-   - Type dependencies
-3. **Code Generation**: Generates methods for each class:
-   - `__SERIALIZE(ptr: usize): void` - Writes JSON to the buffer
-   - `__DESERIALIZE<T>(srcStart, srcEnd, out): usize` - Parses JSON into `out` and returns the advanced source pointer
-
-### Key Files
-
-- `transform/src/index.ts` - Main transform entry point (JSONTransform class)
-- `transform/src/visitor.ts` - AST visitor for traversing source code
-- `transform/src/builder.ts` - AST builder for code generation
-- `transform/src/types.ts` - Type definitions (Property, Schema, Src)
-
-### Generated Code Example
-
-For this class:
-```typescript
-@json
-class Player {
-  name: string = "";
-  score: i32 = 0;
-}
-```
-
-The transform generates:
-```typescript
-__SERIALIZE(ptr: usize): void {
-  store<u16>(bs.offset, 123); // {
-  bs.offset += 2;
-  // ... "name": serialize string ...
-  // ... "score": serialize integer ...
-  store<u16>(bs.offset, 125); // }
-  bs.offset += 2;
-}
-
-__DESERIALIZE<T>(srcStart: usize, srcEnd: usize, out: T): usize {
-  // Key matching and value parsing logic
-  // Uses switch statements on key length for efficiency
-  return srcStart;
-}
-```
-
-## Optimization Modes
-
-json-as provides three optimization levels, selected via the `JSON_MODE` environment variable:
-
-### NAIVE Mode
-
-The baseline implementation with character-by-character processing.
-
-- **Best for**: Debugging, compatibility testing
-- **Performance**: Slowest but most readable
-- **String escaping**: Checks each character individually
-
-### SWAR Mode (Default)
-
-Single Instruction, Multiple Data processing at the word level.
-
-- **Best for**: General use, good balance of speed and compatibility
-- **Performance**: Processes 4 characters at once
-- **String escaping**: Uses bit manipulation to detect escape characters in parallel:
-  ```
-  // Check if any byte in a 64-bit word needs escaping
-  const hasEscape = (word ^ 0x2222...) - 0x0101... & 0x8080...
-  ```
-
-### SIMD Mode
-
-Uses WebAssembly SIMD instructions for 128-bit parallel processing.
-
-- **Best for**: Maximum performance when SIMD is available
-- **Performance**: Processes 8 characters at once
-- **Requirement**: `--enable simd` flag during compilation
-- **String escaping**: Uses `v128` operations for parallel character checking
-
-## Buffer System
-
-Located in `lib/as-bs.ts`, the buffer system (`bs` namespace) manages memory for serialization output.
-
-### Key Features
-
-1. **Dynamic Growth**: Buffer grows as needed during serialization
-2. **Adaptive Sizing**: Uses exponential moving average (EMA) to track typical output sizes
-3. **Automatic Shrinking**: Periodically shrinks if buffer is oversized for typical usage
-
-### Memory Layout
-
-```
-┌───────────────────────────────────────────┐
-│              ArrayBuffer                  │
-│  ┌───────────────────┬─────────────────┐  │
-│  │   Written Data    │   Free Space    │  │
-│  └───────────────────┴─────────────────┘  │
-│  ^ buffer            ^ offset             │
-└───────────────────────────────────────────┘
-```
-
-### Key Functions
-
-- `proposeSize(size)` - Ensure buffer can hold additional bytes
-- `ensureSize(size)` - Grow buffer if necessary
-- `out<T>()` - Copy buffer contents to new string, reset for next use
-- `resize(size)` - Explicitly resize buffer
-
-### String Caching (`sc` namespace)
-
-Optional feature enabled via `JSON_CACHE=1` for repeated string serialization.
-
-```
-┌───────────────────────────────────────────────────────────────┐
-│                    Cache Structure                            │
-│  ┌─────────────────────────────────────────────────────────┐  │
-│  │  Entry Table (4096 slots)                               │  │
-│  │  ┌────────┬────────┬────────┬────────┐                  │  │
-│  │  │ Entry 0│ Entry 1│  ...   │Entry N │                  │  │
-│  │  │key,ptr,│key,ptr,│        │key,ptr,│                  │  │
-│  │  │  len   │  len   │        │  len   │                  │  │
-│  │  └────────┴────────┴────────┴────────┘                  │  │
-│  └─────────────────────────────────────────────────────────┘  │
-│  ┌─────────────────────────────────────────────────────────┐  │
-│  │  Arena (1MB circular buffer)                            │  │
-│  │  ┌──────────────────────────────────────────────────┐   │  │
-│  │  │  Cached serialized string data...                │   │  │
-│  │  └──────────────────────────────────────────────────┘   │  │
-│  │                                          ^              │  │
-│  │                                       arenaPtr          │  │
-│  └─────────────────────────────────────────────────────────┘  │
-└───────────────────────────────────────────────────────────────┘
-```
-
-## Type System
-
-### Static Types
-
-Regular AssemblyScript types handled directly:
-- Primitives: `i8`, `i16`, `i32`, `i64`, `u8`, `u16`, `u32`, `u64`, `f32`, `f64`, `bool`
-- Strings: `string`
-- Collections: `Array<T>`, `StaticArray<T>`, `Map<K, V>`
-- Classes decorated with `@json`
-
-### Dynamic Types
-
-For runtime type flexibility:
-
-- **`JSON.Value`**: Can hold any JSON type
-  ```typescript
-  const v = JSON.Value.from<i32>(42);
-  v.set<string>("hello");  // Can change type
-  ```
-
-- **`JSON.Obj`**: Dynamic object with string keys
-  ```typescript
-  const obj = new JSON.Obj();
-  obj.set("key", 123);
-  ```
-
-- **`JSON.Box<T>`**: Nullable wrapper for primitives
-  ```typescript
-  let maybeInt: JSON.Box<i32> | null = null;
-  ```
-
-- **`JSON.Raw`**: Pre-formatted JSON string (no re-serialization)
-  ```typescript
-  map.set("data", new JSON.Raw('{"already":"json"}'));
-  ```
-
-## Serialization Flow
-
-```
-JSON.stringify<T>(data)
-        │
-        V
-┌───────────────────┐
-│  Type Dispatch    │
-│  (compile-time)   │
-└─────────┬─────────┘
-          │
-    ┌─────┴─────┬─────────────┬──────────────┐
-    V           V             V              V
-┌───────┐  ┌────────┐   ┌──────────┐   ┌──────────┐
-│Boolean│  │Integer │   │  String  │   │  Struct  │
-│ Float │  │        │   │          │   │ (@json)  │
-└───┬───┘  └────┬───┘   └────┬─────┘   └────┬─────┘
-    │           │            │              │
-    │           │            │              │
-    └───────────┴─────┬──────┴──────────────┘
-                      V
-              ┌───────────────┐
-              │ Buffer System │
-              └───────┬───────┘
-                      V
-              ┌───────────────┐
-              │     Heap      │
-              └───────┬───────┘
-                      V
-                   String
-```
-
-## Deserialization Flow
-
-```
-JSON.parse<T>(jsonString)
-        │
-        V
-┌───────────────────┐
-│  Type Dispatch    │
-│  (compile-time)   │
-└─────────┬─────────┘
-          │
-    ┌─────┴─────┬─────────────┬──────────────┐
-    V           V             V              V
-┌───────┐  ┌────────┐   ┌──────────┐   ┌──────────┐
-│Boolean│  │Integer │   │  String  │   │  Struct  │
-└───┬───┘  └────┬───┘   └────┬─────┘   └────┬─────┘
-    │           │            │              │
-    └───────────┴─────┬──────┴──────────────┘
-                      │
-                      V
-                      T
-```
-
-### Struct Deserialization
-
-For `@json` decorated classes, the generated `__DESERIALIZE` method:
-
-1. Scans for opening `{`
-2. Iterates through key-value pairs
-3. Uses switch on key length for fast dispatch
-4. Compares key bytes directly (often as `u32` or `u64` for short keys)
-5. Deserializes value to appropriate type
-6. Stores in output object at correct offset
-7. Returns the updated source pointer after the parsed value
-
-## Environment Variables
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `JSON_MODE` | SWAR | Optimization mode: NAIVE, SWAR, SIMD |
-| `JSON_USE_FAST_PATH` | 0 | When set to `1`, emit the fast struct `__DESERIALIZE` body and helper; otherwise emit only the slow path |
-| `JSON_DEBUG` | 0 | Debug level 0-3 (prints generated code) |
-| `JSON_WRITE` | "" | Comma-separated files to output after transform |
-| `JSON_CACHE` | 0 | Enable string caching (set to 1) |
-| `JSON_STRICT` | false | Enable strict JSON validation |
-
-## Performance Considerations
-
-### Serialization
-
-- Pre-computes static key bytes at compile time
-- Uses direct memory stores for known strings
-- SIMD/SWAR for escape character detection
-- Optional caching for repeated strings
-
-### Deserialization
-
-- Groups fields by key length for switch optimization
-- Uses direct memory loads for key comparison
-- Avoids string allocation during key matching
-- Tracks depth for nested structures
-
-### Memory
-
-- Single reusable buffer reduces allocations
-- EMA-based sizing prevents memory waste
-- Circular arena for cache prevents unbounded growth

package/CONTRIBUTING.md

@@ -1,244 +0,0 @@
-# Contributing to json-as
-
-Thank you for your interest in contributing to json-as! This document provides guidelines and instructions for contributing.
-
-## Table of Contents
-
-- [Getting Started](#getting-started)
-- [Development Setup](#development-setup)
-- [Project Structure](#project-structure)
-- [Making Changes](#making-changes)
-- [Testing](#testing)
-- [Code Style](#code-style)
-- [Pull Request Process](#pull-request-process)
-- [Reporting Issues](#reporting-issues)
-
-## Getting Started
-
-1. Fork the repository on GitHub
-2. Clone your fork locally:
-   ```bash
-   git clone https://github.com/YOUR_USERNAME/json-as.git
-   cd json-as
-   ```
-3. Add the upstream remote:
-   ```bash
-   git remote add upstream https://github.com/JairusSW/json-as.git
-   ```
-
-## Development Setup
-
-### Prerequisites
-
-- Node.js 18+ or Bun
-- Wasmtime (for running tests)
-
-### Install Dependencies
-
-```bash
-npm install
-# or
-bun install
-```
-
-### Build the Transform
-
-The transform is written in TypeScript and needs to be compiled:
-
-```bash
-npm run build:transform
-```
-
-### Running Tests
-
-Run the full test suite across all modes (NAIVE, SWAR, SIMD):
-
-```bash
-npm test
-```
-
-Run a specific test file:
-
-```bash
-npm test string # Runs string.spec.ts
-```
-
-### Running Benchmarks
-
-AssemblyScript benchmarks:
-```bash
-npm run bench:as
-```
-
-JavaScript comparison benchmarks:
-```bash
-npm run bench:js
-```
-
-## Project Structure
-
-```
-json-as/
-├── .as-test/          # As-test configuration and runners
-├── assembly/          # AssemblyScript runtime implementation
-│   ├── index.ts       # Main entry point (JSON namespace)
-│   ├── serialize/     # Serialization implementations
-│   │   ├── index/     # Indexer of all methods
-│   │   ├── simple/    # Naive implementation
-│   │   ├── swar/      # SWAR-optimized
-│   │   └── simd/      # SIMD-optimized
-│   ├── deserialize/   # Deserialization implementations
-│   │   ├── index/     # Indexer of all methods
-│   │   ├── simple/    # Naive implementation
-│   │   ├── swar/      # SWAR-optimized
-│   │   └── simd/      # SIMD-optimized
-│   ├── util/          # Utility functions
-│   ├── custom/        # Constants and character codes
-│   └── __tests__/     # Test files
-├── transform/         # TypeScript compiler transform
-│   └── src/           # Transform source code
-├── lib/               # Shared utilities (buffer system)
-├── bench/             # Benchmark suite
-└── .github/           # CI/CD workflows
-```
-
-## Making Changes
-
-### Branching Strategy
-
-1. Create a feature branch from `main`:
-   ```bash
-   git checkout -b feature/your-feature-name
-   ```
-
-2. Make your changes with clear, atomic commits
-
-3. Keep your branch up to date:
-   ```bash
-   git fetch upstream
-   git rebase upstream/main
-   ```
-
-### Commit Messages
-
-Use clear, descriptive commit messages:
-
-- `feat: add support for BigInt serialization`
-- `fix: handle escaped unicode in strings`
-- `perf: optimize SIMD string escaping`
-- `docs: update README examples`
-- `test: add edge case tests for nested arrays`
-- `chore: update dependencies`
-
-## Testing
-
-### Writing Tests
-
-Tests are located in `assembly/__tests__/`. Each test file follows the pattern `*.spec.ts`.
-
-Example test structure:
-
-```typescript
-import { JSON } from "..";
-
-describe("Feature Name", () => {
-  test("should serialize correctly", () => {
-    const result = JSON.stringify<string>("hello");
-    expect(result).toBe('"hello"');
-  });
-
-  test("should deserialize correctly", () => {
-    const result = JSON.parse<string>('"hello"');
-    expect(result).toBe("hello");
-  });
-});
-```
-
-### Test Coverage
-
-Ensure your changes include tests for:
-- Happy path scenarios
-- Edge cases
-- Error conditions
-- All three modes (NAIVE, SWAR, SIMD) if applicable
-
-## Code Style
-
-### Formatting
-
-The project uses Prettier for formatting:
-
-```bash
-npm run format
-```
-
-### AssemblyScript Guidelines
-
-- Use `@inline` decorator for small, frequently-called functions
-- Prefer `store<T>` and `load<T>` for direct memory operations
-- Use typed arrays and explicit types
-- Add `// @ts-ignore` comments with explanations when necessary
-
-### TypeScript Guidelines (Transform)
-
-- Use strict TypeScript settings
-- Document complex logic with comments
-- Keep functions focused and small
-
-## Pull Request Process
-
-1. **Before submitting:**
-   - Run the full test suite: `npm test`
-   - Run the formatter: `npm run format`
-   - Ensure your branch is up to date with `main`
-
-2. **PR Description:**
-   - Clearly describe the changes
-   - Reference any related issues
-   - Include before/after benchmarks for performance changes
-
-3. **Review Process:**
-   - PRs require at least one approval
-   - Address review feedback promptly
-   - Keep the PR focused on a single concern
-
-4. **After Merge:**
-   - Delete your feature branch
-   - Update any related issues
-
-## Reporting Issues
-
-### Bug Reports
-
-Include:
-- json-as version
-- AssemblyScript version
-- Minimal reproduction case
-- Expected vs actual behavior
-- Error messages (if any)
-
-### Feature Requests
-
-Include:
-- Use case description
-- Proposed API (if applicable)
-- Alternatives considered
-
-## Performance Contributions
-
-If your change affects performance:
-
-1. Run benchmarks before and after
-2. Include benchmark results in the PR
-3. Test across all three modes (NAIVE, SWAR, SIMD)
-4. Consider memory usage implications
-
-## Questions?
-
-- Open a GitHub Discussion for general questions
-- Join the [AssemblyScript Discord](https://discord.gg/assemblyscript)
-- Email the maintainer at [me@jairus.dev](mailto:me@jairus.dev)
-
-## License
-
-By contributing, you agree that your contributions will be licensed under the MIT License.

package/TODO

@@ -1 +0,0 @@
-get staticarrays working within classes

package/as-test.config.json

@@ -1,40 +0,0 @@
-{
-  "$schema": "./node_modules/as-test/as-test.config.schema.json",
-  "input": ["./assembly/__tests__/*.spec.ts"],
-  "outDir": "./.as-test/build",
-  "logs": "./.as-test/logs",
-  "coverageDir": "./.as-test/coverage",
-  "snapshotDir": "./.as-test/snapshots",
-  "config": "none",
-  "coverage": true,
-  "buildOptions": {
-    "args": ["--transform ./transform", "--debug"],
-    "target": "wasi"
-  },
-  "runOptions": {
-    "runtime": {
-      "cmd": "wasmtime <file>"
-    },
-    "reporter": "default"
-  },
-  "modes": {
-    "naive": {
-      "env": {
-        "JSON_MODE": "NAIVE"
-      }
-    },
-    "swar": {
-      "env": {
-        "JSON_MODE": "SWAR"
-      }
-    },
-    "simd": {
-      "env": {
-        "JSON_MODE": "SIMD"
-      },
-      "buildOptions": {
-        "args": ["--enable simd"]
-      }
-    }
-  }
-}

package/assembly/test.ts

@@ -1,24 +0,0 @@
-function get_mask_chars(src: string, mask: u64): void {
-  const srcPtr = changetype<usize>(src);
-  while (mask != 0) {
-    const laneIdx = usize(ctz(mask) >> 3);
-    mask &= ~(0xffff << (laneIdx << 3));
-    // mask &= mask - 1;
-    console.log("Lane:  " + laneIdx.toString());
-    const srcIdx = srcPtr + laneIdx;
-    const chunk = load<u32>(srcIdx);
-    const code = <u16>(chunk >> 16);
-
-    console.log("Chunk: " + String.fromCharCode(chunk & 0xffff) + String.fromCharCode(chunk >> 16));
-    console.log("Code:  " + String.fromCharCode(load<u16>(srcIdx)));
-  }
-}
-
-function backslash_or_quote_mask(block: u64): u64 {
-  const b = block ^ 0x005c_005c_005c_005c;
-  const q = block ^ 0x0022_0022_0022_0022;
-  return (((q - 0x0001_0001_0001_0001) & ~q) | ((b - 0x0001_0001_0001_0001) & ~b)) & 0x0080_0080_0080_0080;
-}
-
-const input = '\\"a"';
-get_mask_chars(input, backslash_or_quote_mask(load<u64>(changetype<usize>(input))));