hostdb 0.10.0

package/LICENSE

@@ -0,0 +1,131 @@
+# PolyForm Noncommercial License 1.0.0
+
+<https://polyformproject.org/licenses/noncommercial/1.0.0>
+
+## Acceptance
+
+In order to get any license under these terms, you must agree
+to them as both strict obligations and conditions to all
+your licenses.
+
+## Copyright License
+
+The licensor grants you a copyright license for the
+software to do everything you might do with the software
+that would otherwise infringe the licensor's copyright
+in it for any permitted purpose.  However, you may
+only distribute the software according to [Distribution
+License](#distribution-license) and make changes or new works
+based on the software according to [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Distribution License
+
+The licensor grants you an additional copyright license
+to distribute copies of the software.  Your license
+to distribute covers distributing the software with
+changes and new works permitted by [Changes and New Works
+License](#changes-and-new-works-license).
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of
+the software from you also gets a copy of these terms or the
+URL for them above, as well as copies of any plain-text lines
+beginning with `Required Notice:` that the licensor provided
+with the software.  For example:
+
+> Required Notice: Copyright Bob Bass (https://github.com/robertjbass)
+
+## Changes and New Works License
+
+The licensor grants you an additional copyright license to
+make changes and new works based on the software for any
+permitted purpose.
+
+## Patent License
+
+The licensor grants you a patent license for the software that
+covers patent claims the licensor can license, or becomes able
+to license, that you would infringe by using the software.
+
+## Noncommercial Purposes
+
+Any noncommercial purpose is a permitted purpose.
+
+## Personal Uses
+
+Personal use for research, experiment, and testing for
+the benefit of public knowledge, personal study, private
+entertainment, hobby projects, amateur pursuits, or religious
+observance, without any anticipated commercial application,
+is use for a permitted purpose.
+
+## Noncommercial Organizations
+
+Use by any charitable organization, educational institution,
+public research organization, public safety or health
+organization, environmental protection organization,
+or government institution is use for a permitted purpose
+regardless of the source of funding or obligations resulting
+from the funding.
+
+## Fair Use
+
+You may have "fair use" rights for the software under the
+law. These terms do not limit them.
+
+## No Other Rights
+
+These terms do not allow you to sublicense or transfer any of
+your licenses to anyone else, or prevent the licensor from
+granting licenses to anyone else.  These terms do not imply
+any other licenses.
+
+## Patent Defense
+
+If you make any written claim that the software infringes or
+contributes to infringement of any patent, your patent license
+for the software granted under these terms ends immediately. If
+your company makes such a claim, your patent license ends
+immediately for work on behalf of your company.
+
+## Violations
+
+The first time you are notified in writing that you have
+violated any of these terms, or done anything with the software
+not covered by your licenses, your licenses can nonetheless
+continue if you come into full compliance with these terms,
+and take practical steps to correct past violations, within
+32 days of receiving notice.  Otherwise, all your licenses
+end immediately.
+
+## No Liability
+
+***As far as the law allows, the software comes as is, without
+any warranty or condition, and the licensor will not be liable
+to you for any damages arising out of these terms or the use
+or nature of the software, under any kind of legal claim.***
+
+## Definitions
+
+The **licensor** is the individual or entity offering these
+terms, and the **software** is the software the licensor makes
+available under these terms.
+
+**You** refers to the individual or entity agreeing to these
+terms.
+
+**Your company** is any legal entity, sole proprietorship,
+or other kind of organization that you work for, plus all
+organizations that have control over, are under the control of,
+or are under common control with that organization.  **Control**
+means ownership of substantially all the assets of an entity,
+or the power to direct its management and policies by vote,
+contract, or otherwise.  Control can be direct or indirect.
+
+**Your licenses** are all the licenses granted to you for the
+software under these terms.
+
+**Use** means anything you do with the software requiring one
+of your licenses.

package/README.md

@@ -0,0 +1,228 @@
+# hostdb
+
+Pre-built database binaries for all major platforms, distributed via GitHub Releases.
+
+**Primary consumer:** [SpinDB](https://github.com/robertjbass/spindb) - a CLI tool for spinning up local database instances
+
+## Philosophy
+
+This repository exists to solve one problem: **database binaries should be available for download on every major platform, for every supported version, without relying on third-party sources that may disappear.**
+
+### Binary Sourcing Priority
+
+When adding a database, we source binaries in this order:
+
+1. **Official binaries** - Direct from vendor CDNs (Oracle for MySQL, MariaDB Foundation, EnterpriseDB for PostgreSQL Windows, etc.)
+2. **Third-party repositories** - Trusted sources like [MariaDB4j](https://github.com/MariaDB4j/MariaDB4j) Maven JARs
+3. **Build from source** - Docker builds for Linux, native GitHub Actions builds for macOS
+
+### What This Means
+
+- Every database version we support has binaries for all 5 platforms
+- Binaries are built once and hosted forever on GitHub Releases
+- `releases.json` provides a queryable manifest of all available downloads
+- CLI tools (like SpinDB) query this manifest to find and download binaries
+
+## Supported Platforms
+
+| Platform | Description |
+|----------|-------------|
+| `linux-x64` | Linux x86_64 (glibc 2.28+) |
+| `linux-arm64` | Linux ARM64 (glibc 2.28+) |
+| `darwin-x64` | macOS Intel |
+| `darwin-arm64` | macOS Apple Silicon |
+| `win32-x64` | Windows x64 |
+
+## Quick Start
+
+```bash
+# Download MySQL 8.4.3 for current platform
+pnpm download:mysql
+
+# Download for all platforms
+pnpm download:mysql -- --all-platforms
+
+# Build from source if no binary available
+pnpm download:mariadb -- --version 11.8.5 --platform linux-arm64 --build-fallback
+
+# List supported databases
+pnpm dbs
+```
+
+## Querying Available Binaries
+
+SpinDB (or any consumer) can fetch `releases.json` for available binaries:
+
+```bash
+curl https://raw.githubusercontent.com/robertjbass/hostdb/main/releases.json
+```
+
+**Download URL pattern:**
+```
+https://github.com/robertjbass/hostdb/releases/download/{tag}/{filename}
+
+# Example:
+https://github.com/robertjbass/hostdb/releases/download/mysql-8.4.3/mysql-8.4.3-darwin-arm64.tar.gz
+```
+
+## Configuration Files
+
+| File | Purpose |
+|------|---------|
+| `databases.json` | **Single source of truth** for all databases, versions, and platforms |
+| `releases.json` | Queryable manifest of all GitHub Releases (auto-updated) |
+| `builds/*/sources.json` | URL mappings for each database's binaries |
+
+### databases.json
+
+The central configuration that **drives all automation**. GitHub Actions workflows validate against this file before building.
+
+```json
+{
+  "mysql": {
+    "displayName": "MySQL",
+    "status": "in-progress",
+    "versions": { "8.4.7": true, "8.0.40": true },
+    "platforms": { "linux-x64": true, "darwin-arm64": true, ... }
+  }
+}
+```
+
+**To enable a new version:**
+1. Add it to `databases.json` with `true`
+2. Add URLs to `builds/<database>/sources.json`
+3. Run the workflow - it validates against databases.json automatically
+
+**Status values:**
+- `completed` - Fully built and released
+- `in-progress` - Currently being implemented
+- `pending` - Planned, not yet started
+- `unsupported` - Not planned (licensing, niche use case, etc.)
+
+### releases.json
+
+Auto-generated manifest updated after each GitHub Release. Structure:
+
+```json
+{
+  "repository": "robertjbass/hostdb",
+  "lastUpdated": "2024-01-15T10:30:00Z",
+  "databases": {
+    "mysql": {
+      "8.4.3": {
+        "releaseTag": "mysql-8.4.3",
+        "platforms": {
+          "darwin-arm64": {
+            "url": "https://github.com/.../mysql-8.4.3-darwin-arm64.tar.gz",
+            "sha256": "abc123...",
+            "size": 165000000
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+## Current Status
+
+| Database | Status | Versions | Notes |
+|----------|--------|----------|-------|
+| MySQL | Completed | 8.4.7, 8.0.40 | Official binaries for all platforms |
+| PostgreSQL | In Progress | 18.1.0, 17.7.0, 16.11.0, 15.15.0 | Source builds + EDB (Windows) |
+| MariaDB | Completed | 11.8.5, 11.4.5, 10.11.15 | Official + source builds |
+| Redis | Completed | 8.4.0, 7.4.7 | Source builds |
+| SQLite | In Progress | 3.51.1 | Official amalgamation |
+| Valkey | Completed | 9.0.1, 8.0.6 | Redis-compatible, permissive license (includes bespoke Windows builds via Cygwin) |
+| MongoDB | Completed | 8.0.17, 8.2.3, 7.0.28 | Official binaries (SSPL license) |
+
+See `pnpm dbs` for the full list.
+
+### Licensing Notes
+
+Some databases have restrictive licenses that limit commercial and closed-source use:
+
+| Database | License | Commercial Use | Open-Source Alternative |
+|----------|---------|----------------|------------------------|
+| MongoDB | SSPL | ❌ Restricted | [FerretDB](https://www.ferretdb.com/) (Apache 2.0) |
+| Redis | RSALv2 + SSPLv1 | ❌ Restricted | [Valkey](https://valkey.io/) (BSD-3-Clause) |
+
+**FerretDB** is a MongoDB-compatible database built on PostgreSQL. **Valkey** is a Redis fork maintained by the Linux Foundation after Redis changed to a non-open-source license.
+
+If you need MongoDB or Redis compatibility for commercial/closed-source projects, use FerretDB or Valkey instead.
+
+## GitHub Actions
+
+Each database has a release workflow triggered via `workflow_dispatch`:
+
+1. Go to Actions → "Release [Database]" → Run workflow
+2. **Select the version** from dropdown (synced from `databases.json`)
+3. Select platforms (default: all)
+4. Workflow **validates against databases.json** before building
+5. Downloads/builds binaries for all platforms in parallel
+6. Creates GitHub Release with artifacts
+7. Updates `releases.json` manifest
+
+**Validation:** The workflow validates the selected version exists in `databases.json` and `sources.json` before building.
+
+**Sync dropdowns:** Run `pnpm sync:versions` after adding new versions to databases.json.
+
+## Project Structure
+
+```
+hostdb/
+├── databases.json          # Source of truth for all databases
+├── releases.json           # Queryable manifest of GitHub Releases
+├── schemas/                # JSON schemas for validation
+├── builds/
+│   ├── mysql/
+│   │   ├── download.ts     # Download script
+│   │   ├── sources.json    # Version → URL mappings
+│   │   ├── Dockerfile      # Source build fallback
+│   │   └── README.md
+│   ├── postgresql/
+│   ├── mariadb/
+│   └── ...
+├── scripts/
+│   ├── add-engine.ts         # pnpm add:engine - scaffold new database
+│   ├── fetch-edb-fileids.ts  # pnpm edb:fileids - fetch PostgreSQL Windows file IDs
+│   ├── list-databases.ts     # pnpm dbs
+│   ├── sync-versions.ts      # pnpm sync:versions - sync workflow dropdowns
+│   └── update-releases.ts    # Updates releases.json after release
+└── .github/workflows/
+    ├── release-mysql.yml
+    ├── release-postgresql.yml
+    └── ...
+```
+
+## Adding a New Database
+
+Use the scaffolding script:
+
+```bash
+pnpm add:engine redis    # Creates builds/redis/, workflow, and package.json script
+pnpm add:engine sqlite   # Then follow printed instructions
+```
+
+## Documentation
+
+- [ARCHITECTURE.md](./ARCHITECTURE.md) - Visual representation of how this repo works
+- [CHECKLIST.md](./CHECKLIST.md) - Checklist for adding a new database
+
+## TODO
+
+- [ ] Add Windows filesystem support to download scripts:
+  - [ ] Check `process.platform === 'win32'` for platform-specific logic
+  - [ ] Use Node's `path` utilities (`path.sep`, `path.join`, `path.normalize`) instead of manual string concatenation
+  - [ ] Use `os.tmpdir()` for temp file locations
+  - [ ] Use `fs.mkdtemp()` for safe cross-platform temp directory creation
+  - [ ] Normalize/escape backslashes when constructing download paths or invoking shell commands
+
+## Inspiration
+
+- [MariaDB4j](https://github.com/MariaDB4j/MariaDB4j) - Embedded MariaDB for Java
+- [embedded-postgres-binaries](https://github.com/zonkyio/embedded-postgres-binaries) - PostgreSQL binaries (we previously used this, now build from source)
+
+## License
+
+[PolyForm Noncommercial 1.0.0](./LICENSE)

package/bin/cli.js

@@ -0,0 +1,75 @@
+#!/usr/bin/env node
+
+import { fileURLToPath, pathToFileURL } from 'node:url'
+import { dirname, join } from 'node:path'
+import { spawn } from 'node:child_process'
+import { existsSync } from 'node:fs'
+import { createRequire } from 'node:module'
+
+const __filename = fileURLToPath(import.meta.url)
+const __dirname = dirname(__filename)
+
+const packageRoot = join(__dirname, '..')
+const mainScript = join(packageRoot, 'cli', 'bin.ts')
+
+// Find tsx ESM loader using Node's module resolution
+let tsxLoader = null
+
+try {
+  const require = createRequire(import.meta.url)
+  const tsxDir = dirname(require.resolve('tsx/package.json'))
+  const loaderPaths = [
+    join(tsxDir, 'dist', 'esm', 'index.mjs'),
+    join(tsxDir, 'dist', 'loader.mjs'),
+  ]
+  tsxLoader = loaderPaths.find((p) => existsSync(p))
+} catch {
+  // tsx not found via module resolution
+}
+
+if (!tsxLoader) {
+  console.error('Error: tsx loader not found.')
+  console.error('\nTry running: pnpm install')
+  process.exit(1)
+}
+
+const tsxLoaderUrl = pathToFileURL(tsxLoader).href
+
+const child = spawn(
+  process.execPath,
+  ['--import', tsxLoaderUrl, mainScript, ...process.argv.slice(2)],
+  {
+    stdio: 'inherit',
+    shell: false,
+    cwd: packageRoot,
+  },
+)
+
+// Forward termination signals to child process
+const forwardSignal = (signal) => {
+  if (child.pid && !child.killed) {
+    child.kill(signal)
+  }
+}
+
+process.on('SIGINT', () => forwardSignal('SIGINT'))
+process.on('SIGTERM', () => forwardSignal('SIGTERM'))
+process.on('SIGHUP', () => forwardSignal('SIGHUP'))
+
+child.on('exit', (code, signal) => {
+  // Clean up signal handlers
+  process.removeAllListeners('SIGINT')
+  process.removeAllListeners('SIGTERM')
+  process.removeAllListeners('SIGHUP')
+
+  // Exit with same code or signal-based exit code
+  if (signal) {
+    process.exit(128 + (signal === 'SIGINT' ? 2 : signal === 'SIGTERM' ? 15 : 1))
+  }
+  process.exit(code ?? 0)
+})
+
+child.on('error', (err) => {
+  console.error('Failed to start hostdb:', err.message)
+  process.exit(1)
+})