space-data-module-sdk 0.8.14 → 0.8.16

package/docs/publication-submission.md

@@ -0,0 +1,78 @@
+# Publication and listing
+
+The last step of the integrator path: turn a signed artifact into something a
+consumer can discover, fetch and load. Part of this is implemented and part of
+it is still being built, and this page separates the two explicitly.
+
+## What is implemented today
+
+**The published artifact layout.** A protected artifact carries its records in a
+trailer:
+
+```
+protected-payload-bytes || REC-flatbuffer-bytes || uint32le(REC length) || "$REC"
+```
+
+`REC`, `MBL`, `PNM` and `ENC` are Space Data Standards FlatBuffer record types.
+The SDK consumes the generated message classes; the schemas are ratified at
+`spacedatastandards.org`. The full contract — which record carries the manifest,
+which carries the bundle metadata, which carries the publication notice, and
+which carries the encryption envelope — is the
+[module publication standard](module-publication-standard.html).
+
+**The artifact primitives.** `protect`, `sign` and `verify` are real commands
+with real flags. See [Protect and sign](protect-and-sign.html).
+
+**Post-publication loadability is gated.** The tri-runtime parity gate carries a
+fixture for the *published* form of a module — the same artifact after its
+signature and manifest records have been appended as a trailing custom section —
+so the record-stripping and instantiation path is exercised by the gate, not
+assumed. See the [parity gate](tri-runtime-parity-gate.html).
+
+**Package distribution.** The publication standard also covers delivery through
+conventional package ecosystems (npm, PyPI, Maven Central, NuGet, Go modules,
+crates.io, Swift Package Manager) for the language bindings that accompany a
+module.
+
+## What is still being built
+
+These are checked absences in the SDK today, not roadmap decoration:
+
+- **No listing-submission command.** There is no `publish`, `submit` or `list`
+  subcommand. The CLI offers artifact-level primitives only.
+- **No storefront or marketplace workflow in this repository.** A module
+  marketplace is a program in the wider stack; no listing code lives here.
+- **No manifest injection for foreign-compiled binaries.** A BYO vendor's
+  artifact cannot yet be stamped with a manifest by a tool. See the
+  [BYO-wasm quickstart](byo-wasm-quickstart.html).
+
+## The escorted path, which works now
+
+Until self-serve submission lands, a vendor can complete the whole path with
+help:
+
+1. The vendor builds `module.wasm` on the pinned toolchain and supplies the
+   manifest JSON alongside it.
+2. Conformance and the parity gate are run against the artifact — the vendor can
+   run both themselves and attach the output, or hand over the artifact and have
+   them run.
+3. The artifact is protected and signed, and `verify` is run against the
+   published bytes with the intended trusted key set.
+4. The publication records are attached and the module is listed by hand.
+
+Nothing in that sequence requires the vendor to have access to a private
+repository, and every gate in it is a real command whose output is the evidence.
+
+## Submission checklist
+
+Before handing over an artifact, confirm each of these:
+
+| Check | Command or evidence |
+| --- | --- |
+| Built on the pinned toolchain | Link flags include `-Wl,--shared-memory` and `-Wl,--max-memory=2147483648` |
+| No Emscripten anywhere in the build | Build log; the parity gate's import classifier will catch it |
+| Family ABI conformance passes | `space-data-module conformance <family> --artifact <path>` |
+| Tri-runtime parity passes | `space-data-module parity --wasm <path> --fixture <path> --lanes browser,wasmedge,docker-wasmedge` |
+| Manifest declares family, invoke surface, runtime targets, record types | The manifest JSON |
+| Signed with the intended key | `space-data-module verify --wasm <path> --trusted <hex> --require-signature` |
+| Teardown does not leak | The conformance leak cycles reach steady state |

package/docs/secrets-capability.html

@@ -0,0 +1,206 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <meta name="description" content="A module often needs an operator's third-party credential: a Space-Track login," />
+    <title>Credential lanes — Space Data Module SDK</title>
+    <link rel="stylesheet" href="./styles.css" />
+    <!-- SDN_CONSUMER_ASSETS_START -->
+<link rel="stylesheet" href="https://static.spacedatanetwork.org/assets/hd-wallet-ui/2.0.28/sdn-wallet-public-client.c3f68d1cfd88478f10d836a5e829d1dfc6a10157972cf0f7d4d319d0636f2cc4.css" integrity="sha384-fICuhN4I9xqOK1F5vGGzl26opuO+xQIvTZXMSV76lWTBiQ6AfzztMGLLDw8yRT8i" crossorigin="anonymous">
+<script defer src="https://static.spacedatanetwork.org/assets/hd-wallet-ui/2.0.28/sdn-wallet-public-client.f611e7e151a3b5c38384272f8894a4c6634f0a29bf925b0e02e0e27f8db0bfff.js" integrity="sha384-R+QqsKoWJIS7iWrtPmgJ49DVnb1hSGjZokupbLInJd1PLp8RwQhtF/hnaerc33ci" crossorigin="anonymous" data-sdn-wallet-public-client="v1"></script>
+<script defer src="https://static.spacedatanetwork.org/assets/sdn-stack-nav/1.0.0/sdn-stack-nav.52fde607eee38ffa116188201f50258ed50bd18c9f06c1af9678f763147a8fe5.js" integrity="sha384-dgeojhJ8vTszHXIbv7O7nZcEzqD10oUYYJjBmPrV7+kLQEUlXLQD4ek5Q7HOFiO5" crossorigin="anonymous" data-nav-style-url="https://static.spacedatanetwork.org/assets/sdn-stack-nav/1.0.0/sdn-stack-nav.36a36359ce18322185e9ff179f88175bab67d5ad84a14d9c08a54f2ff27267e7.css" data-nav-style-integrity="sha384-c4M8Fg+kYaeOYYtJVr7jJsde24IhSWkWaRyAOHEZU9jozhRck089aw+mnHwxymds" data-wallet-client-url="https://static.spacedatanetwork.org/assets/hd-wallet-ui/2.0.28/sdn-wallet-public-client.f611e7e151a3b5c38384272f8894a4c6634f0a29bf925b0e02e0e27f8db0bfff.js" data-wallet-client-integrity="sha384-R+QqsKoWJIS7iWrtPmgJ49DVnb1hSGjZokupbLInJd1PLp8RwQhtF/hnaerc33ci" data-wallet-style-url="https://static.spacedatanetwork.org/assets/hd-wallet-ui/2.0.28/sdn-wallet-public-client.c3f68d1cfd88478f10d836a5e829d1dfc6a10157972cf0f7d4d319d0636f2cc4.css" data-wallet-style-integrity="sha384-fICuhN4I9xqOK1F5vGGzl26opuO+xQIvTZXMSV76lWTBiQ6AfzztMGLLDw8yRT8i" data-wallet-client-id="sdn-module-sdk-pages-v1" data-wallet-callback-uri="https://digitalarsenal.github.io/space-data-module-sdk/wallet-callback.html" data-wallet-registry-sha256="e1ce6fe903c9700484a8a87d96581c8cad97063dabf63030b4518a31a3bdaa93"></script>
+<!-- SDN_CONSUMER_ASSETS_END -->
+  </head>
+  <body>
+    <a class="skip-link" href="#main">Skip to content</a>
+    <sdn-stack-nav active="module-sdk"></sdn-stack-nav>
+    <header class="site-header">
+      <a class="site-brand" href="./">Space Data Module SDK</a>
+      <nav class="header-links" aria-label="Site">
+        <a href="./byo-wasm-quickstart.html">Quickstart</a>
+        <a href="./conformance.html">Conformance</a>
+        <a href="https://github.com/DigitalArsenal/space-data-module-sdk">GitHub</a>
+        <a class="stack-button" href="#stack">Stack</a>
+      </nav>
+    </header>
+    <div class="shell">
+      <nav class="sidebar" id="sidebar" aria-label="Documentation">
+<h2>Start here</h2><ul>
+<li><a href="./index.html"><span>Harness family matrix</span></a></li>
+<li><a href="./byo-wasm-quickstart.html"><span>BYO-wasm quickstart</span></a></li>
+<li><a href="./conformance.html"><span>Conformance kit</span></a></li>
+<li><a href="./protect-and-sign.html"><span>Protect and sign</span></a></li>
+<li><a href="./publication-submission.html"><span>Publication and listing</span></a></li>
+</ul>
+<h2>Dynamics</h2><ul>
+<li><a href="./families/propagator.html"><span>Propagator</span><span class="tag tag-shipped">shipped</span></a></li>
+<li><a href="./families/maneuver.html"><span>Maneuver</span><span class="tag tag-experimental">experimental</span></a></li>
+<li><a href="./families/propulsion.html"><span>Propulsion</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/attitude.html"><span>Attitude</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/gnc.html"><span>GNC</span><span class="tag tag-planned">planned</span></a></li>
+</ul>
+<h2>Environment and interaction</h2><ul>
+<li><a href="./families/rf.html"><span>RF</span><span class="tag tag-designed">designed</span></a></li>
+<li><a href="./families/sensor.html"><span>Sensor</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/signature.html"><span>Signature</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/environment.html"><span>Environment</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/obstruction.html"><span>Obstruction</span><span class="tag tag-designed">designed</span></a></li>
+</ul>
+<h2>Event physics</h2><ul>
+<li><a href="./families/breakup.html"><span>Breakup</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/reentry.html"><span>Reentry</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/conjunction.html"><span>Conjunction</span><span class="tag tag-designed">designed</span></a></li>
+<li><a href="./families/effects.html"><span>Effects</span><span class="tag tag-planned">planned</span></a></li>
+</ul>
+<h2>Estimation, data and logic</h2><ul>
+<li><a href="./families/estimation.html"><span>Estimation</span><span class="tag tag-experimental">experimental</span></a></li>
+<li><a href="./families/data-source.html"><span>Data source</span><span class="tag tag-shipped">shipped</span></a></li>
+<li><a href="./families/analytics.html"><span>Analytics</span><span class="tag tag-experimental">experimental</span></a></li>
+<li><a href="./families/scheduler.html"><span>Scheduler</span><span class="tag tag-planned">planned</span></a></li>
+<li><a href="./families/behavior.html"><span>Behavior</span><span class="tag tag-planned">planned</span></a></li>
+</ul>
+<h2>Runtime contract</h2><ul>
+<li><a href="./events-abi.html"><span>Event locator ABI</span></a></li>
+<li><a href="./module-publication-standard.html"><span>Module publication standard</span></a></li>
+<li><a href="./browser-wasmedge-isomorphic.html"><span>Browser / WasmEdge isomorphism</span></a></li>
+<li><a href="./isomorphic-pthreads.html"><span>Isomorphic pthreads</span></a></li>
+<li><a href="./tri-runtime-parity.html"><span>Tri-runtime parity</span></a></li>
+<li><a href="./tri-runtime-parity-gate.html"><span>Tri-runtime parity gate</span></a></li>
+<li><a href="./testing-harness.html"><span>Testing harness</span></a></li>
+<li><a href="./language-runtime-matrix.html"><span>Language and runtime matrix</span></a></li>
+</ul>
+<h2>Host surfaces</h2><ul>
+<li><a href="./flatsql-host-contract.html"><span>FlatSQL host contract</span></a></li>
+<li><a href="./flatsql-streaming-standard.html"><span>FlatSQL streaming standard</span></a></li>
+<li><a href="./secrets-capability.html" aria-current="page"><span>Credential lanes</span></a></li>
+<li><a href="./protocol-installation.html"><span>Protocol installation</span></a></li>
+<li><a href="./gpu-module-abi.html"><span>GPU module ABI</span></a></li>
+<li><a href="./module-bundle-runtime-plan.html"><span>Module bundle runtime</span></a></li>
+<li><a href="./isomorphic-sdn-runtime-plan.html"><span>Isomorphic SDN runtime</span></a></li>
+<li><a href="./node-red-default-node-parity.html"><span>Node-RED node parity</span></a></li>
+</ul>
+      </nav>
+      <main class="doc" id="main">
+        <h1>Credential lanes</h1>
+<p>A module often needs an operator's third-party credential: a Space-Track login, a data-provider API password, an account at whatever service the operator uses. The <code>secrets</code> capability is how that credential reaches the module <strong>without the credential ever living in the module, the manifest, the flow, or a record</strong>.</p>
+<p>The operator enters the credential once on their node. The node seals it at rest under its own key material. A module the operator has explicitly approved can then read exactly that one credential at runtime.</p>
+<p>This document is normative for module authors. The guest-side helper is <code>src/host/cpp/secretsClient.hpp</code>; the reference host implementation is <code>sdn-server/internal/modulert/caps/secrets.go</code>.</p>
+<h2 id="lanes"><a class="anchor" href="#lanes" aria-hidden="true">#</a>Lanes</h2>
+<p>A <strong>lane</strong> is a named credential slot on the node — <code>spacetrack</code>, <code>acme-weather</code>, <code>zephyr_billing</code>. Lane ids are lowercase <code>[a-z0-9_-]{2,64}</code>.</p>
+<p><strong>Lanes are operator-defined.</strong> A few are well-known (<code>spacetrack</code>, <code>edc_cpf</code>, <code>myintelsat</code>) because the node ships a verifier that can probe them, but an operator may create a lane for any service at all. Do not hard-code an assumption that a fixed set exists:</p>
+<ul>
+<li>Take the lane id as <strong>module configuration</strong>, not as a compile-time constant, so an operator can point your module at their own lane.</li>
+<li>Treat &quot;not configured&quot; as a <strong>normal runtime state</strong>, not an error condition that should crash a run.</li>
+<li>A lane may be permanently <strong>unverified</strong>. The node cannot probe a service it knows nothing about, so <code>verified_at</code> stays empty forever for most operator-defined lanes. That is honest, not broken.</li>
+</ul>
+<h2 id="the-approval-contract"><a class="anchor" href="#the-approval-contract" aria-hidden="true">#</a>The approval contract</h2>
+<p>Reading a credential requires <strong>two</strong> things, and neither is something your module can arrange for itself.</p>
+<ol>
+<li><strong>Declare the capability.</strong> Add <code>secrets:&lt;lane&gt;</code> to your manifest's <code>capabilities</code> for each lane you read. <code>secrets:spacetrack</code> conveys nothing about <code>secrets:edc_cpf</code>: the host re-checks the exact lane on <strong>every call</strong>, so a module holding one lane's grant is refused on every other lane.</li>
+</ol>
+<ol>
+<li><strong>The operator approves your module's content hash.</strong> Every <code>secrets:*</code> capability is <em>sensitive</em>. The operator records an approval for your module's exact SHA-256 content hash, for that exact lane, in the node's <code>capability_policy.json</code>. Without it the module is <strong>denied at load</strong> — the whole module fails to load, not just the call.</li>
+</ol>
+<p>Recompiling changes the hash and therefore <strong>revokes</strong> the approval. That is deliberate: the approval is for the bytes the operator reviewed, not for a name.</p>
+<p>There is <strong>no enumeration</strong>. There is no <code>secrets.list</code> and no <code>secrets.export</code>. A guest can ask for a lane it is already approved for and nothing else; it can never discover which credentials the node holds. Do not probe lane names.</p>
+<h2 id="handling-the-plaintext"><a class="anchor" href="#handling-the-plaintext" aria-hidden="true">#</a>Handling the plaintext</h2>
+<p><code>secrets.get</code> returns the credential <strong>in the clear</strong>, in your linear memory. There is no way around that — the module's job is to present the password to the provider's own login, so the secret <em>is</em> the message. Once it crosses the boundary the host cannot protect it further, and these rules are yours to keep:</p>
+<ul>
+<li><strong>Never log it.</strong> Not in an error message, not in a trace, not in a progress event, not at debug level.</li>
+<li><strong>Never persist it.</strong> Not through <code>storage.write</code>, not to the filesystem, not into a record, not into a cache that outlives the call.</li>
+<li><strong>Never forward it</strong> anywhere except the provider it belongs to.</li>
+<li><strong>Wipe it</strong> as soon as the provider call is done, on success <em>and</em> failure paths, and keep its lifetime as short as you can.</li>
+</ul>
+<p>If your module also holds <code>http</code>, it is technically capable of exfiltrating the credential. The operator's approval is precisely the decision to trust it not to. Honor that.</p>
+<h2 id="minimal-usage"><a class="anchor" href="#minimal-usage" aria-hidden="true">#</a>Minimal usage</h2>
+<div class="codeblock"><div class="codeblock-head">cpp</div><pre><code>#include &quot;sdm_hostcall_wire.hpp&quot;   // must be included first
+#include &quot;secretsClient.hpp&quot;
+
+// The lane comes from module configuration — never assume &quot;spacetrack&quot;.
+bool fetch_provider_data(const std::string&amp; lane) {
+  sdm_secrets::Credential cred;
+  if (!sdm_secrets::secrets_get(lane, &amp;cred)) {
+    // Not approved for this lane, or the operator has not entered it yet.
+    // Report the CONDITION, never the lane's contents.
+    log_error(&quot;provider credential unavailable&quot;);
+    return false;
+  }
+
+  const bool ok = provider_login(cred.username, cred.secret);
+  sdm_secrets::wipe(&amp;cred);   // on every path, including this failure one
+  return ok;
+}</code></pre></div>
+<p>Manifest:</p>
+<div class="codeblock"><div class="codeblock-head">json</div><pre><code>{
+  &quot;pluginId&quot;: &quot;com.example.provider-fetch&quot;,
+  &quot;capabilities&quot;: [&quot;http&quot;, &quot;secrets:acme-weather&quot;]
+}</code></pre></div>
+<p>To decide whether a fetch is worth attempting without touching the plaintext:</p>
+<div class="codeblock"><div class="codeblock-head">cpp</div><pre><code>sdm_secrets::Status status;
+if (sdm_secrets::secrets_status(lane, &amp;status) &amp;&amp; status.configured) {
+  // A credential is present. status.username_masked is safe to show an
+  // operator (&quot;o***@acme.example&quot;); status.verified_at is EMPTY when the node
+  // has never probed it, which is normal for an operator-defined lane.
+}</code></pre></div>
+<p><code>secrets_status</code> is gated by the <strong>same</strong> per-lane capability, so an unapproved module cannot use it to probe which credentials a node holds.</p>
+<h2 id="wire-shapes"><a class="anchor" href="#wire-shapes" aria-hidden="true">#</a>Wire shapes</h2>
+<p>Both operations are plain JSON meta documents; neither uses binary segments.</p>
+<div class="codeblock"><pre><code>secrets.get     -&gt; {&quot;id&quot;:&quot;&lt;lane&gt;&quot;}
+                &lt;- {&quot;ok&quot;:true,&quot;result&quot;:{&quot;id&quot;:&quot;...&quot;,&quot;username&quot;:&quot;...&quot;,&quot;secret&quot;:&quot;...&quot;}}
+
+secrets.status  -&gt; {&quot;id&quot;:&quot;&lt;lane&gt;&quot;}
+                &lt;- {&quot;ok&quot;:true,&quot;result&quot;:{&quot;id&quot;:&quot;...&quot;,&quot;configured&quot;:true,
+                     &quot;username_masked&quot;:&quot;o***@example.com&quot;,
+                     &quot;updated_at&quot;:&quot;...&quot;,&quot;verified_at&quot;:&quot;...&quot;}}</code></pre></div>
+<p>A refusal — unapproved lane, unconfigured credential, no keystore on this node — comes back as <code>{&quot;ok&quot;:false,...}</code> and <strong>never</strong> carries credential material. Both helpers return <code>false</code> in that case and leave their output parameter untouched. A <code>false</code> return never means &quot;empty credential&quot;; it means no credential was obtained.</p>
+<h2 id="failure-modes-worth-expecting"><a class="anchor" href="#failure-modes-worth-expecting" aria-hidden="true">#</a>Failure modes worth expecting</h2>
+<div class="table-wrap"><table><thead><tr><th>Situation</th><th>What you see</th></tr></thead><tbody><tr><td>Capability not declared in the manifest</td><td>Refused per call (<code>ok:false</code>)</td></tr><tr><td>Declared but not approved by the operator</td><td><strong>Module denied at load</strong></td></tr><tr><td>Approved for a different lane</td><td>Refused per call (<code>ok:false</code>)</td></tr><tr><td>Lane exists but the operator has not filled it in</td><td><code>secrets_get</code> false; <code>secrets_status</code> <code>configured:false</code></td></tr><tr><td>Lane has no verifier on this node</td><td><code>verified_at</code> empty — normal, not an error</td></tr><tr><td>Node has no credential keystore</td><td>Refused per call (<code>ok:false</code>)</td></tr></tbody></table></div>
+
+        <p class="doc-footer">
+          Source of truth for this page:
+          <code>docs/secrets-capability.md</code>. Regenerate with
+          <code>npm run build:docs</code>.
+        </p>
+      </main>
+<nav class="rail" aria-label="On this page"><h2>On this page</h2><ul><li><a class="depth-2" href="#lanes">Lanes</a></li>
+<li><a class="depth-2" href="#the-approval-contract">The approval contract</a></li>
+<li><a class="depth-2" href="#handling-the-plaintext">Handling the plaintext</a></li>
+<li><a class="depth-2" href="#minimal-usage">Minimal usage</a></li>
+<li><a class="depth-2" href="#wire-shapes">Wire shapes</a></li>
+<li><a class="depth-2" href="#failure-modes-worth-expecting">Failure modes worth expecting</a></li></ul></nav>
+    </div>
+    <section id="stack" class="section" aria-labelledby="stack-title">
+      <div class="section-head">
+        <p class="eyebrow">SDN Stack</p>
+        <h2 id="stack-title">Connected sites</h2>
+      </div>
+      <div class="link-list">
+        <a href="https://spacedatastandards.org/">
+          <span>Standards</span>
+          <small>Canonical schemas, record contracts, and generated bindings.</small>
+        </a>
+        <a href="https://digitalarsenal.github.io/flatbuffers/">
+          <span>FlatBuffers</span>
+          <small>Binary encoding, schema tooling, and runtime documentation.</small>
+        </a>
+        <a href="https://digitalarsenal.github.io/flatsql/">
+          <span>FlatSQL</span>
+          <small>SQL-style queries over FlatBuffer-backed datasets and streams.</small>
+        </a>
+        <a href="https://spacedatanetwork.org/">
+          <span>SDN</span>
+          <small>Distributed publication, discovery, delivery, and marketplace infrastructure.</small>
+        </a>
+        <a href="https://digitalarsenal.github.io/space-data-module-sdk/" class="active">
+          <span>Module SDK</span>
+          <small>WASM module packaging, validation, and host compatibility tooling.</small>
+        </a>
+      </div>
+    </section>
+    <footer class="site-footer">
+      Space Data Module SDK — Apache-2.0. Every ABI page renders a Markdown
+      source in <code>docs/</code>; edit the Markdown, never the HTML.
+    </footer>
+  </body>
+</html>