@casadi/casadi-reader 0.0.0-poc

package/LICENSES/LGPL-3.0.txt

@@ -0,0 +1,165 @@
+                   GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+
+  This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+  0. Additional Definitions.
+
+  As used herein, "this License" refers to version 3 of the GNU Lesser
+General Public License, and the "GNU GPL" refers to version 3 of the GNU
+General Public License.
+
+  "The Library" refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+  An "Application" is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+  A "Combined Work" is a work produced by combining or linking an
+Application with the Library.  The particular version of the Library
+with which the Combined Work was made is also called the "Linked
+Version".
+
+  The "Minimal Corresponding Source" for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+  The "Corresponding Application Code" for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+  1. Exception to Section 3 of the GNU GPL.
+
+  You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+  2. Conveying Modified Versions.
+
+  If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+   a) under this License, provided that you make a good faith effort to
+   ensure that, in the event an Application does not supply the
+   function or data, the facility still operates, and performs
+   whatever part of its purpose remains meaningful, or
+
+   b) under the GNU GPL, with none of the additional permissions of
+   this License applicable to that copy.
+
+  3. Object Code Incorporating Material from Library Header Files.
+
+  The object code form of an Application may incorporate material from
+a header file that is part of the Library.  You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+   a) Give prominent notice with each copy of the object code that the
+   Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the object code with a copy of the GNU GPL and this license
+   document.
+
+  4. Combined Works.
+
+  You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+   a) Give prominent notice with each copy of the Combined Work that
+   the Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the Combined Work with a copy of the GNU GPL and this license
+   document.
+
+   c) For a Combined Work that displays copyright notices during
+   execution, include the copyright notice for the Library among
+   these notices, as well as a reference directing the user to the
+   copies of the GNU GPL and this license document.
+
+   d) Do one of the following:
+
+       0) Convey the Minimal Corresponding Source under the terms of this
+       License, and the Corresponding Application Code in a form
+       suitable for, and under terms that permit, the user to
+       recombine or relink the Application with a modified version of
+       the Linked Version to produce a modified Combined Work, in the
+       manner specified by section 6 of the GNU GPL for conveying
+       Corresponding Source.
+
+       1) Use a suitable shared library mechanism for linking with the
+       Library.  A suitable mechanism is one that (a) uses at run time
+       a copy of the Library already present on the user's computer
+       system, and (b) will operate properly with a modified version
+       of the Library that is interface-compatible with the Linked
+       Version.
+
+   e) Provide Installation Information, but only if you would otherwise
+   be required to provide such information under section 6 of the
+   GNU GPL, and only to the extent that such information is
+   necessary to install and execute a modified version of the
+   Combined Work produced by recombining or relinking the
+   Application with a modified version of the Linked Version. (If
+   you use option 4d0, the Installation Information must accompany
+   the Minimal Corresponding Source and Corresponding Application
+   Code. If you use option 4d1, you must provide the Installation
+   Information in the manner specified by section 6 of the GNU GPL
+   for conveying Corresponding Source.)
+
+  5. Combined Libraries.
+
+  You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+   a) Accompany the combined library with a copy of the same work based
+   on the Library, uncombined with any other library facilities,
+   conveyed under the terms of this License.
+
+   b) Give prominent notice with the combined library that part of it
+   is a work based on the Library, and explaining where to find the
+   accompanying uncombined form of the same work.
+
+  6. Revised Versions of the GNU Lesser General Public License.
+
+  The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+  Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License "or any later version"
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+  If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.

package/NOTICE

@@ -0,0 +1,17 @@
+casadi-reader's implementation is licensed under the MIT License (LICENSE).
+
+The vendored schemes/serialization_scheme.json contains serializer source
+excerpts from CasADi 3.8.1, https://github.com/casadi/casadi. Those excerpts
+retain CasADi's LGPL-3.0-or-later license; they are not relicensed by this
+repository's MIT License. License texts are in LICENSES/.
+
+CasADi source attribution:
+Copyright (C) 2010-2023 Joel Andersson, Joris Gillis, Moritz Diehl,
+                        KU Leuven. All rights reserved.
+Copyright (C) 2011-2014 Greg Horn
+Additional source-file notices remain available in the referenced CasADi tree.
+
+The published reader runtime assets contain generated protocol metadata,
+not the scheme's serializer source excerpts. The full scheme is included in
+this source repository and Python source distributions as a generation/test
+input; it is not needed by the installed readers.

package/README.md

@@ -0,0 +1,273 @@
+# casadi-reader
+
+Inspect `.casadi` files without installing or loading CasADi. One repository
+provides JavaScript/npm, Python/PyPI, C++, C, MATLAB and Julia interfaces.
+The current development focus is the **npm package**. The other language
+interfaces are experimental and deferred. This is not yet a published npm release.
+Files are inspected, never evaluated; no archives are extracted.
+
+| Interface | Implementation | Runtime dependencies |
+| --- | --- | --- |
+| `@casadi/casadi-reader` (npm) | Standalone JavaScript | None; Node 22+ or browser ESM |
+| `casadi-reader` (PyPI) | Standalone Python | None; Python 3.9+ |
+| C++ | Native reader, RAII `Document` | C++ standard library |
+| C | C ABI over the native reader | Same native library |
+| MATLAB | MATLAB package and MEX | Native reader compiled into the MEX |
+| Julia `CasadiReader` | Julia package calling the C ABI | Standalone native library, JSON.jl |
+
+All interfaces use the same object-table JSON contract and fixtures. The three
+reader implementations use source assets generated from the vendored scheme.
+MATLAB and Julia do not launch Python, Node or a subprocess to decode files.
+
+## Python
+
+Install locally with `pip install .` (the distribution declares no runtime
+requirements):
+
+```python
+from casadi_reader import read_casadi, read_resource, to_json
+
+graph = read_casadi('model.casadi')
+print(to_json(graph, indent=2))
+
+resource = read_resource('resource.casadi', lazy=True)
+blob = resource['resource']['blob']
+try:
+    first_bytes = blob.read(0, 64)
+finally:
+    blob.close()
+```
+
+File input uses a read-only memory map. Lazy payloads keep that mapping alive;
+metadata and requested slices are decoded, without allocating the whole archive.
+`loads(text)` and `loads_resource(text, lazy=True)` also accept encoded strings.
+The CLI is `casadi-reader [--resource] [--lazy] input.casadi [output.json]`, or
+`python -m casadi_reader ...`.
+
+## JavaScript
+
+Install locally with `npm install /path/to/casadi-reader`:
+
+```js
+import {readCasadi, decodeResource, openResource} from '@casadi/casadi-reader';
+const graph = readCasadi(await file.text());
+const {resource} = decodeResource(resourceStreamText, {lazy: true});
+const firstBytes = resource.blob.read(0, 64); // Uint8Array
+
+// File/Blob input also defers reading the encoded payload:
+const opened = await openResource(resourceFile, {lazy: true});
+const bytes = await opened.resource.blob.read(0, 64);
+```
+
+The npm tarball contains bundled ESM in `dist/`, with serialization metadata
+compiled into the JavaScript. It does not fetch or import the scheme JSON.
+After publication, a browser can import the versioned
+`https://unpkg.com/@casadi/casadi-reader@VERSION/dist/index.js` directly from a
+`<script type="module">`; no import map or bundler is required. `VERSION` is a
+placeholder: this prototype has not been published.
+
+The Node CLI is `node bin/casadi-reader.js [--resource] [--lazy] input.casadi`.
+Text mode retains the encoded string. File/Blob mode reads a bounded metadata
+prefix, then requested slices. File/Blob Resource input must have no surrounding
+whitespace. An application installing both CLIs should use the explicit module
+or Node entry point to avoid the common `casadi-reader` executable name.
+
+## C++ and C
+
+Build with CMake and a C++11 compiler; no third-party library is needed:
+
+```sh
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
+cmake --build build
+ctest --test-dir build --output-on-failure
+cmake --install build --prefix /your/prefix
+```
+
+Installed CMake consumers can use `find_package(casadi_reader CONFIG REQUIRED)`
+and link `casadi_reader::casadi_reader`. Headers are
+`<casadi_reader/reader.hpp>` and `<casadi_reader/reader.h>`.
+
+```cpp
+casadi_reader::Document graph("model.casadi");
+std::string json = graph.json();
+casadi_reader::Document resource("resource.casadi", true, true);
+auto bytes = resource.read_blob(0, 0, 64);
+```
+
+```c
+char error[1024];
+cr_document *doc = cr_open("model.casadi", 0, 0, error, sizeof(error));
+if (doc) {
+  puts(cr_json(doc)); /* pointer valid until cr_close */
+  cr_close(doc);
+} else {
+  fprintf(stderr, "%s\n", error);
+}
+```
+
+The C ABI catches exceptions and reports errors through caller-owned buffers.
+Handles own their source streams; lazy blob slices seek directly to encoded
+file ranges. C/C++ blob indices and byte offsets are zero-based. Do not share
+one handle between concurrent operations. Native eager archive-to-JSON expansion
+is limited to 1,000,000 bytes; use lazy mode for larger archives.
+
+## MATLAB
+
+```sh
+cmake -S . -B build -DCASADI_READER_MATLAB=ON
+cmake --build build
+```
+
+```matlab
+addpath('matlab');
+addpath('build/matlab');
+graph = casadi_reader.read('model.casadi');
+document = casadi_reader.Document('resource.casadi', true, true);
+cleanup = onCleanup(@() delete(document));
+metadata = document.data();
+bytes = document.readBlob(1, 0, 64);
+```
+
+`Document.json()` returns JSON text; `data()` uses MATLAB's `jsondecode`.
+The package ZIP can include the built MEX. MEX binaries are platform-specific;
+this prototype has been tested with MATLAB R2024b on Linux.
+
+## Julia
+
+From this checkout, build the library and instantiate the Julia environment:
+
+```sh
+julia --project=julia -e 'using Pkg; Pkg.instantiate(); include("julia/deps/build.jl")'
+```
+
+```julia
+using CasadiReader
+graph = read_casadi("model.casadi")
+document = read_resource("resource.casadi"; lazy=true)
+try
+    metadata = data(document)
+    bytes = read_blob(document, 1, 0, 64)
+finally
+    close(document)
+end
+```
+
+The source archive is a self-contained Julia project, including the native
+sources in `deps/reader`. `Pkg.build("CasadiReader")` needs CMake and a C++
+compiler. Alternatively set `CASADI_READER_LIBRARY` to an existing standalone
+reader library before starting Julia. Tested with Julia 1.6.2 on Linux.
+
+MATLAB and Julia blob indices are one-based; byte offsets remain zero-based.
+**Object references inside the JSON contract remain zero-based in every language.**
+For example, a Julia consumer accesses the root as `graph["objects"][graph["root"]+1]`.
+
+## JSON contract
+
+Graph documents have `format: "casadi_json"`, `version: 1`,
+`serializationProtocol: 3`, a `root` object index, and an `objects` array.
+References are indices; shared nodes are stored once; null references stay null.
+
+- `kind: "function"`: name, type, input/output sparsities and names, input nodes,
+  and instructions containing node references and argument/result work slots.
+- `kind: "mx"`: operation ID/name, dependency and sparsity references, constants
+  and node-specific `info`. Indexing nodes also have a normalized `mapping`.
+  Constants are numeric strings, including nonfinite values; decimal spelling
+  may differ between implementations.
+- `kind: "sparsity"`: shape and compressed-column `colind` / `row` arrays.
+
+`info` is reconstructed from serialized members and tested against native
+`MX.info()`. It does not prescribe viewer layout or styling. The representation
+is not lossless and cannot be written back to `.casadi` by this prototype.
+
+Lazy Resource payloads serialize as small descriptors with `offset`, `byteLength`
+and `encoding`. Retrieving bytes requires the live document/blob and its original
+source; descriptors alone are not portable archive copies. Lazy mode defers
+opaque payloads, not all numerical arrays or graph nodes. Eager mode is default.
+Payload encoding is checked on access; declared extents are checked on opening.
+
+## Supported subset
+
+The fixtures come from CasADi 3.8.1 default `Function.save()` files: MX arithmetic,
+sparse matrices, gathers, slices and entry assignments. Supported layouts are
+ProtoFunction 2, FunctionInternal 8, XFunction 1, MXFunction 3, little-endian
+protocol 3. Unsupported operations/layouts fail explicitly. SX functions, nested
+function calls, plugins, JIT, nonempty option/cache dictionaries and debug
+serialization are outside this prototype. It is not a general reader for every
+CasADi file or version, nor a hardened untrusted-file service.
+
+Resource entry points accept a **raw SerializingStream containing one Resource**.
+This tests the embedded ZIP representation used by FMUs, but **does not yet read
+an entire saved FmuFunction**. No FMU is loaded or executed.
+
+## Scheme, tests and packaging
+
+`schemes/serialization_scheme.json` vendors CasADi's checked-in
+`misc/serialization_scheme.json`. It is a source-derived serializer index and
+named-field contract, **not a complete executable deserialization grammar**.
+`scripts/generate-reader-assets.py` consumes this JSON and emits three checked-in
+source assets: `src/scheme.js`, `python/casadi_reader/_scheme.py`, and
+`native/src/scheme.hpp`. These contain runtime protocol constants, operation IDs
+and class versions, omitting the C++ source index. Supported positional layouts
+are still implemented explicitly in the readers.
+
+`npm run build` regenerates the assets and bundles the JavaScript. `npm pack`
+runs that build automatically. Python packages contain the generated Python
+module, so neither Python nor JavaScript parses the scheme JSON at runtime.
+Build tools are development dependencies only.
+
+CI checks `npm run check:generated` **before** rebuilding, rejecting stale
+committed assets. The initial CI builds and tests the npm package and produces its tarball. A browser test serves the extracted npm tarball
+over HTTP and checks that decoding needs one JavaScript request and no scheme
+JSON request. CI uploads build artifacts; it does not publish packages.
+
+```sh
+node scripts/vendor-scheme.mjs ../serialization-scheme/misc/serialization_scheme.json
+python scripts/generate-reader-assets.py
+npm ci
+npm run check:generated
+npm test
+npm run test:browser
+PYTHONPATH=python python -m unittest discover -s python/tests -v
+CASADI_READER_LIBRARY="$PWD/build/libcasadi_reader.so" julia --project=julia julia/test/runtests.jl
+```
+
+Set `CASADI_READER_NATIVE` to the native CLI to enable Python/native agreement
+tests. Python tests also compare JavaScript when Node is available. Native CasADi
+is used only to regenerate fixtures (`scripts/generate-fixtures.py` and
+`scripts/generate-resource-fixture.cpp`), never to run the readers. Tests check
+native graph/metadata oracles and deferred 16 MiB payload access.
+
+Create unpublished artifacts with:
+
+```sh
+python -m build
+npm pack
+python scripts/package-bindings.py --output dist --mex build/matlab/casadi_reader_mex.mexa64
+```
+
+This produces Python wheel/sdist, npm tarball, native and Julia source archives,
+and a MATLAB ZIP. It does not register packages or upload releases. The adjacent
+`casadi-viz` proof of concept consumes the npm package through a local dependency.
+
+The reader implementation is MIT licensed. The vendored scheme includes
+upstream CasADi source excerpts that retain their original license; see NOTICE.
+
+## npm publishing
+
+`.github/workflows/publish.yml` publishes on a published GitHub release, using
+OIDC Trusted Publishing and provenance. Its release tag must equal `v` plus the
+version in `package.json`. Prerelease versions use npm's `next` tag; stable
+versions use `latest`. It runs the tests and packed-browser check before publishing.
+
+Once the package exists on npm, open its Settings → Trusted publishing and add:
+
+- Provider: GitHub Actions
+- Organization: `casadi`
+- Repository: `casadi-reader`
+- Workflow filename: `publish.yml`
+- Environment: leave blank (the workflow does not use a GitHub environment)
+- Allowed actions: enable direct `npm publish`
+
+No npm token secret is needed. See the current
+[npm Trusted Publishing documentation](https://docs.npmjs.com/trusted-publishers/).
+The workflow is prepared; the package has not yet been published to npm.

package/bin/casadi-reader.js

@@ -0,0 +1,11 @@
+#!/usr/bin/env node
+import {readFile,writeFile} from 'node:fs/promises';
+import {decodeCasadi,decodeResource} from '../dist/index.js';
+try {
+  const args=process.argv.slice(2),options={lazy:args.includes('--lazy')},resource=args.includes('--resource');
+  const [input,output,...extra]=args.filter(a=>a!=='--lazy'&&a!=='--resource');
+  if(!input||extra.length)throw Error('Usage: casadi-reader [--lazy] [--resource] INPUT [OUTPUT.json]');
+  const read=resource?decodeResource:decodeCasadi;
+  const json=JSON.stringify(read(await readFile(input,'utf8'),options),null,2)+'\n';
+  if(output)await writeFile(output,json);else process.stdout.write(json);
+}catch(error){console.error(error.message);process.exitCode=1;}