@grainulation/silo 1.0.0

package/CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+## 1.0.0
+
+Initial release.
+
+- 11 built-in knowledge packs (compliance, architecture, security, testing, and more)
+- Web pack browser with search and claim table
+- Store, Search, ImportExport, Templates, and Packs library classes
+- `silo pull`, `silo store`, `silo search`, `silo publish` CLI commands
+- File-watching with automatic state refresh
+- SSE live-reload
+- Zero runtime dependencies

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 grainulation contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,111 @@
+# @grainulation/silo
+
+Reusable knowledge for research sprints. Grab a starter pack in one command:
+
+```bash
+npx @grainulation/silo pull compliance --into ./claims.json
+```
+
+## What is Silo?
+
+Silo stores and shares research knowledge across projects and teams. Instead of starting every sprint from scratch, pull in battle-tested constraint sets, risk patterns, and decision templates.
+
+- **Shared claim libraries** -- reusable constraint sets (HIPAA, SOC 2, GDPR, etc.)
+- **Community sprint templates** -- pre-built research questions with seeded claims
+- **Claim import/export** -- pull claims from one sprint into another
+- **Knowledge packs** -- version-controlled bundles teams can subscribe to
+- **Full-text search** -- find relevant claims across everything you've stored
+
+Zero dependencies. Filesystem-based storage. Works offline.
+
+## Built-in Packs
+
+| Pack | Claims | What's inside |
+|------|--------|---------------|
+| `api-design` | 13 | REST conventions, versioning, pagination, error formats, GraphQL tradeoffs |
+| `architecture` | 12 | Monolith vs micro, build vs buy, SQL vs NoSQL decision claims |
+| `ci-cd` | 12 | CI/CD pipeline patterns, caching, rollback strategies |
+| `compliance` | 14 | HIPAA, SOC 2, GDPR constraint sets with regulatory citations |
+| `data-engineering` | 12 | ETL patterns, data quality, warehouse design |
+| `frontend` | 12 | Frontend architecture, performance, accessibility patterns |
+| `migration` | 10 | Database/cloud/framework migration risks and patterns |
+| `observability` | 12 | Logging, metrics, tracing, alerting patterns |
+| `security` | 12 | Security constraints, threat models, authentication patterns |
+| `team-process` | 12 | Team workflow, code review, incident response patterns |
+| `testing` | 10 | Testing strategies, coverage, test architecture |
+
+## Quick Start
+
+```bash
+# See available packs
+silo packs
+
+# Pull compliance constraints into your sprint
+silo pull compliance --into ./claims.json
+
+# Pull only GDPR constraints
+silo pull compliance --into ./claims.json --type constraint
+
+# Search across stored claims
+silo search "encryption at rest"
+
+# Store your sprint's findings for reuse
+silo store "q4-migration-findings" --from ./claims.json
+
+# List everything in your silo
+silo list
+```
+
+## CLI Reference
+
+| Command | Description |
+|---------|-------------|
+| `silo list` | List all stored collections |
+| `silo pull <pack> --into <file>` | Pull claims into a claims file |
+| `silo store <name> --from <file>` | Store claims from a sprint |
+| `silo search <query>` | Full-text search across claims |
+| `silo packs` | List available knowledge packs |
+| `silo templates` | List saved sprint templates |
+| `silo publish <name> --collections <ids>` | Bundle collections into a pack |
+| `silo install <file>` | Install a pack from a JSON file |
+
+## How It Works
+
+Silo uses your filesystem for storage (`~/.silo` by default). No database, no network calls, no accounts. Claims are JSON files organized by collection, indexed for search.
+
+When you `pull`, Silo:
+1. Resolves the source (built-in pack or stored collection)
+2. Re-prefixes claim IDs to avoid collisions
+3. Deduplicates against existing claims in the target
+4. Merges into your claims.json
+
+When you `store`, Silo:
+1. Reads your sprint's claims.json
+2. Hashes the content for versioning
+3. Saves to `~/.silo/claims/` with metadata
+4. Updates the search index
+
+## Programmatic API
+
+```js
+const { Store } = require('@grainulation/silo/lib/store');
+const { Search } = require('@grainulation/silo/lib/search');
+const { ImportExport } = require('@grainulation/silo/lib/import-export');
+
+const store = new Store().init();
+const search = new Search(store);
+const io = new ImportExport(store);
+
+// Search for encryption-related claims
+const results = search.query('encryption', { type: 'constraint' });
+
+// Pull a pack programmatically
+io.pull('compliance', './claims.json');
+
+// Store findings
+io.push('./claims.json', 'my-sprint-findings');
+```
+
+## License
+
+MIT

package/bin/silo.js

@@ -0,0 +1,327 @@
+#!/usr/bin/env node
+
+/**
+ * silo — CLI for storing and sharing reusable research knowledge
+ *
+ * Usage:
+ *   silo list                          List all stored collections and packs
+ *   silo pull <pack> --into <file>     Pull claims from a pack into a claims file
+ *   silo store <name> --from <file>    Store claims from a file into the silo
+ *   silo search <query>                Search across all stored claims
+ *   silo publish <name> --collections <ids...>  Bundle collections into a pack
+ *   silo packs                         List available knowledge packs
+ *   silo templates                     List available sprint templates
+ *   silo serve [--port 9095]           Start the knowledge browser UI
+ *   silo analyze                        Run cross-library analytics (requires harvest)
+ *   silo serve-mcp                     Start the MCP server on stdio
+ */
+
+const { Store } = require('../lib/store.js');
+const { Search } = require('../lib/search.js');
+const { ImportExport } = require('../lib/import-export.js');
+const { Templates } = require('../lib/templates.js');
+const { Packs } = require('../lib/packs.js');
+
+// ── --version / -v (before verbose check) ──
+if (process.argv.includes('--version') || (process.argv.includes('-v') && process.argv.length === 3)) {
+  const pkg = require('../package.json');
+  process.stdout.write(pkg.version + '\n');
+  process.exit(0);
+}
+
+const verbose = process.argv.includes('--verbose') || process.argv.includes('-v');
+function vlog(...a) {
+  if (!verbose) return;
+  const ts = new Date().toISOString();
+  process.stderr.write(`[${ts}] silo: ${a.join(' ')}\n`);
+}
+
+const store = new Store();
+const search = new Search(store);
+const io = new ImportExport(store);
+const templates = new Templates(store);
+const packs = new Packs(store);
+
+const args = process.argv.slice(2);
+const command = args[0];
+
+vlog('startup', `command=${command || '(none)'}`, `cwd=${process.cwd()}`);
+
+const jsonMode = args.includes('--json');
+
+function flag(name) {
+  const idx = args.indexOf(`--${name}`);
+  if (idx === -1) return null;
+  return args[idx + 1] || true;
+}
+
+function flagList(name) {
+  const idx = args.indexOf(`--${name}`);
+  if (idx === -1) return [];
+  const values = [];
+  for (let i = idx + 1; i < args.length; i++) {
+    if (args[i].startsWith('--')) break;
+    values.push(args[i]);
+  }
+  return values;
+}
+
+function print(obj) {
+  if (typeof obj === 'string') {
+    process.stdout.write(obj + '\n');
+  } else {
+    process.stdout.write(JSON.stringify(obj, null, 2) + '\n');
+  }
+}
+
+function usage() {
+  print(`silo -- reusable knowledge for research sprints
+
+Commands:
+  list                              List all stored collections and packs
+  pull <pack> --into <file> [--filter <ids>]  Pull claims (optionally filter by ID)
+  store <name> --from <file>        Store claims from a file into the silo
+  search <query> [--type <type>]    Search across all stored claims
+  publish <name> --collections <ids...>  Bundle collections into a pack
+  packs                             List available knowledge packs
+  templates                         List available sprint templates
+  install <file>                    Install a pack from a file
+  analyze                            Run cross-library analytics (requires harvest)
+  serve [--port 9095]               Start the knowledge browser UI
+  serve-mcp                         Start the MCP server on stdio
+
+Examples:
+  silo pull compliance --into ./claims.json
+  silo search "encryption" --type constraint
+  silo store "my-findings" --from ./claims.json
+  silo serve --port 9095
+  silo packs`);
+}
+
+try {
+  switch (command) {
+    case 'list': {
+      const collections = store.list();
+      if (jsonMode) {
+        print(JSON.stringify(collections));
+        break;
+      }
+      if (collections.length === 0) {
+        print('No stored collections. Use "silo store" to save claims or "silo packs" to see built-in packs.');
+      } else {
+        print('Stored collections:\n');
+        for (const c of collections) {
+          print(`  ${c.id}  (${c.type || 'claims'}, ${c.claimCount} claims)  ${c.storedAt || ''}`);
+        }
+      }
+      break;
+    }
+
+    case 'pull': {
+      const source = args[1];
+      const into = flag('into');
+      if (!source || !into) {
+        print('Usage: silo pull <pack> --into <file>');
+        process.exit(1);
+      }
+      const dryRun = args.includes('--dry-run');
+      const types = flagList('type');
+      const filterIds = flag('filter');
+      const ids = filterIds ? filterIds.split(',').map((s) => s.trim()) : undefined;
+      const result = io.pull(source, into, { types: types.length ? types : undefined, ids, dryRun });
+      if (dryRun) {
+        print(`Dry run: would import ${result.wouldImport} claims`);
+        print(result.claims);
+      } else {
+        print(`Imported ${result.imported} claims into ${into} (${result.skippedDuplicates} duplicates skipped, ${result.totalClaims} total)`);
+      }
+      break;
+    }
+
+    case 'store': {
+      const name = args[1];
+      const from = flag('from');
+      if (!name || !from) {
+        print('Usage: silo store <name> --from <file>');
+        process.exit(1);
+      }
+      const result = io.push(from, name);
+      print(`Stored "${name}" (${result.claimCount} claims, hash: ${result.hash})`);
+      break;
+    }
+
+    case 'search': {
+      const query = args.slice(1).filter((a) => !a.startsWith('--')).join(' ');
+      if (!query) {
+        print('Usage: silo search <query> [--type <type>] [--evidence <tier>]');
+        process.exit(1);
+      }
+      const type = flag('type');
+      const tier = flag('tier');
+      const results = search.query(query, { type, tier });
+      if (jsonMode) {
+        print(JSON.stringify(results));
+        break;
+      }
+      if (results.length === 0) {
+        print('No matches found.');
+      } else {
+        print(`${results.length} result(s):\n`);
+        for (const r of results) {
+          const text = r.claim.content || r.claim.text || '';
+          const tier = r.claim.evidence || r.claim.tier || 'unknown';
+          print(`  [${r.claim.id}] (${r.claim.type}, ${tier}) ${text.slice(0, 120)}${text.length > 120 ? '...' : ''}`);
+          print(`    from: ${r.collection}  score: ${r.score}\n`);
+        }
+      }
+      break;
+    }
+
+    case 'publish': {
+      const name = args[1];
+      const collections = flagList('collections');
+      if (!name || collections.length === 0) {
+        print('Usage: silo publish <name> --collections <id1> <id2> ...');
+        process.exit(1);
+      }
+      const desc = flag('description') || '';
+      const result = packs.bundle(name, collections, { description: desc });
+      print(`Published pack "${name}" (${result.claimCount} claims) -> ${result.path}`);
+      break;
+    }
+
+    case 'packs': {
+      const allPacks = packs.list();
+      if (jsonMode) {
+        print(JSON.stringify(allPacks));
+        break;
+      }
+      if (allPacks.length === 0) {
+        print('No packs available.');
+      } else {
+        print('Available packs:\n');
+        for (const p of allPacks) {
+          print(`  ${p.id}  ${p.name}  (${p.claimCount} claims, v${p.version}, ${p.source})`);
+          if (p.description) print(`    ${p.description.slice(0, 100)}`);
+          print('');
+        }
+      }
+      break;
+    }
+
+    case 'templates': {
+      const allTemplates = templates.list();
+      if (jsonMode) {
+        print(JSON.stringify(allTemplates));
+        break;
+      }
+      if (allTemplates.length === 0) {
+        print('No templates saved yet. Use the Templates API to create them.');
+      } else {
+        for (const t of allTemplates) {
+          print(`  ${t.id}  "${t.question || t.name}"  (${t.seedClaims} seed claims)  [${t.tags.join(', ')}]`);
+        }
+      }
+      break;
+    }
+
+    case 'install': {
+      const filePath = args[1];
+      if (!filePath) {
+        print('Usage: silo install <pack-file.json>');
+        process.exit(1);
+      }
+      const result = packs.install(filePath);
+      print(`Installed pack "${result.id}" (${result.claimCount} claims)`);
+      break;
+    }
+
+    case 'analyze': {
+      const { analyzeLibrary } = require('../lib/analytics.js');
+      const result = analyzeLibrary(store);
+      if (jsonMode) {
+        print(JSON.stringify(result));
+        break;
+      }
+      if (!result.available) {
+        print(`silo analyze: harvest not found.\n\nThe analyze command requires @grainulation/harvest in a sibling directory.\nExpected locations:\n  ../harvest/lib/analyzer.js\n\nInstall harvest alongside silo and try again.`);
+        process.exit(1);
+      }
+      if (!result.analysis) {
+        print(`silo analyze: ${result.reason || 'no data to analyze'}`);
+        break;
+      }
+      print(`Cross-library analytics (${result.collectionCount} collection(s)):\n`);
+      const analysis = result.analysis;
+      if (analysis.typeDistribution) {
+        print('Type distribution:');
+        for (const [type, count] of Object.entries(analysis.typeDistribution)) {
+          print(`  ${type}: ${count}`);
+        }
+        print('');
+      }
+      if (analysis.evidenceQuality) {
+        print('Evidence quality:');
+        for (const [tier, count] of Object.entries(analysis.evidenceQuality)) {
+          print(`  ${tier}: ${count}`);
+        }
+        print('');
+      }
+      // Print any other top-level keys from the analysis
+      for (const [key, value] of Object.entries(analysis)) {
+        if (key === 'typeDistribution' || key === 'evidenceQuality') continue;
+        if (typeof value === 'object') {
+          print(`${key}: ${JSON.stringify(value, null, 2)}`);
+        } else {
+          print(`${key}: ${value}`);
+        }
+      }
+      break;
+    }
+
+    case 'serve-mcp': {
+      const serveMcp = require('../lib/serve-mcp.js');
+      serveMcp.run(process.cwd());
+      break;
+    }
+
+    case 'serve': {
+      // Dynamic import for ESM server module -- use fork() for proper stdio
+      const port = flag('port') || '9095';
+      const root = flag('root') || process.cwd();
+      const serverArgs = [];
+      if (flag('port')) { serverArgs.push('--port', port); }
+      if (flag('root')) { serverArgs.push('--root', root); }
+      const { fork } = require('node:child_process');
+      const path = require('node:path');
+      const serverPath = path.join(__dirname, '..', 'lib', 'server.js');
+      const child = fork(serverPath, serverArgs, {
+        stdio: 'inherit',
+        env: process.env,
+      });
+      child.on('error', (err) => {
+        process.stderr.write(`silo: error starting server: ${err.message}\n`);
+        process.exit(1);
+      });
+      child.on('exit', (code) => process.exit(code || 0));
+      process.on('SIGTERM', () => child.kill('SIGTERM'));
+      process.on('SIGINT', () => child.kill('SIGINT'));
+      break;
+    }
+
+    case 'help':
+    case '--help':
+    case '-h':
+    case undefined:
+      usage();
+      break;
+
+    default:
+      print(`silo: unknown command: ${command}\n`);
+      usage();
+      process.exit(1);
+  }
+} catch (err) {
+  process.stderr.write(`silo: ${err.message}\n`);
+  process.exit(1);
+}

package/lib/analytics.js

@@ -0,0 +1,76 @@
+'use strict';
+
+/**
+ * silo -> harvest edge: cross-library analytics.
+ *
+ * Feeds silo's stored claim collections to harvest's analyzer
+ * to surface patterns, evidence quality trends, and type distribution
+ * across the entire knowledge base. Graceful fallback if harvest
+ * is not available.
+ */
+
+const fs = require('node:fs');
+const path = require('node:path');
+
+const HARVEST_SIBLINGS = [
+  path.join(__dirname, '..', '..', 'harvest'),
+  path.join(__dirname, '..', '..', '..', 'harvest'),
+];
+
+/**
+ * Try to load harvest's analyzer module from a sibling checkout.
+ * Returns the analyze function or null.
+ */
+function loadHarvestAnalyzer() {
+  for (const dir of HARVEST_SIBLINGS) {
+    const analyzerPath = path.join(dir, 'lib', 'analyzer.js');
+    if (fs.existsSync(analyzerPath)) {
+      try {
+        const mod = require(analyzerPath);
+        if (typeof mod.analyze === 'function') return mod.analyze;
+      } catch { continue; }
+    }
+  }
+  return null;
+}
+
+/**
+ * Run cross-library analytics on all stored claim collections.
+ * @param {import('./store').Store} store — an initialized silo Store instance
+ * @returns {{ available: boolean, analysis?: object, collectionCount?: number }}
+ */
+function analyzeLibrary(store) {
+  const analyze = loadHarvestAnalyzer();
+  if (!analyze) {
+    return { available: false, reason: 'harvest analyzer not found in sibling directories' };
+  }
+
+  const collections = store.list();
+  if (collections.length === 0) {
+    return { available: true, analysis: null, collectionCount: 0, reason: 'no collections stored' };
+  }
+
+  // Convert silo collections into the sprint format harvest expects
+  const sprints = [];
+  for (const entry of collections) {
+    const data = store.getClaims(entry.id);
+    if (!data || !data.claims) continue;
+    sprints.push({
+      name: entry.name || entry.id,
+      claims: data.claims,
+    });
+  }
+
+  if (sprints.length === 0) {
+    return { available: true, analysis: null, collectionCount: 0, reason: 'no valid claim data' };
+  }
+
+  const analysis = analyze(sprints);
+  return {
+    available: true,
+    collectionCount: sprints.length,
+    analysis,
+  };
+}
+
+module.exports = { analyzeLibrary, loadHarvestAnalyzer };