mdshelf 1.1.0 → 1.1.2

package/README.md

@@ -2,7 +2,9 @@
 
 Turn any folder of Markdown files into a fast, beautiful, browsable site. Point mdshelf at your OpenClaw, Hermes, GBrain, wikis, notes, docs, etc. — mount each one under its own URL (e.g. `/agent`, `/hermes`, `gbrain`, `/docs`, `/notes`, `/blog`) — and read them anywhere with hot reload, clean typography, and a mobile-first theme out of the box.
 
-Pairs beautifully with [Tailscale](https://tailscale.com): one command serves your shelf to every device on your tailnet — phone, iPad, laptop. Sharing with Tailscale is fully private by default: no port forwarding, no public DNS, nothing exposed to the open internet.
+Pairs beautifully with [Tailscale](https://tailscale.com): one command serves your shelf to every device on your tailnet — phone, iPad, laptop. Serving over Tailscale is fully private by default: no port forwarding, no public DNS, nothing exposed to the open internet.
+
+Putting a shelf on the open internet? `mdshelf serve --auth google` signs readers in with Google and shows each one only the pages their address is allowed to read — the rules live in each file's frontmatter. Signed-in readers can hand anyone a link to a single page that expires on its own and can be revoked at any time.
 
 One command installs it as a native system service (launchd, systemd, or Windows SCM) that runs quietly in the background and survives reboots.
 
@@ -25,6 +27,7 @@ Ships as a single static Rust binary with zero runtime dependencies.
 - Auto-generated index pages for folders that have no `index.md`
 - Home page listing all mounted sites with page counts
 - Copy or download any page's Markdown source straight from its header
+- **Search** — keyword and semantic search across every mounted site, with [qmd](https://github.com/tobi/qmd) ranking and results limited to pages the reader may open
 
 **Theme**
 - Bundled `mdshelf-theme` — mobile-first, responsive, light/dark aware
@@ -33,6 +36,13 @@ Ships as a single static Rust binary with zero runtime dependencies.
 - Per-site accent colors — auto-assigned from a palette or configured explicitly
 - MiniJinja templates for layouts and partials — override any file without forking the whole theme
 
+**Access**
+- **Google sign-in** — `--auth google` puts every page behind sign-in with your own OAuth client
+- Per-page access rules in frontmatter (`allow` / `deny`), inherited through folder `index.md` files; anything not granted is denied
+- **Share links** — one page, no sign-in, expiring and revocable, from a Share button or `mdshelf share`
+- Access log with `mdshelf audit`; rule debugging with `mdshelf acl explain` and `mdshelf acl doctor`
+- Built-in HTTPS: Let's Encrypt, your own certificate, or TLS terminated by a proxy in front
+
 **Server**
 - **Hot reload** — WebSocket-based live reload; the browser refreshes automatically the moment you save a file
 - Response compression (gzip / brotli / zstd via tower-http)
@@ -113,7 +123,7 @@ Open `http://127.0.0.1:4444/` for the site index.
 | Command | Description |
 |---|---|
 | `mdshelf init [dir]` | Scaffold config and sample content |
-| `mdshelf serve` | Start the web server |
+| `mdshelf serve` | Start the web server (`--auth google` for a private site) |
 | `mdshelf check` | Validate config and scan content |
 | `mdshelf export` | Export a static bundle of HTML and CSS files |
 | `mdshelf auth setup` | Walk through creating a Google OAuth client, then verify it |
@@ -124,7 +134,7 @@ Open `http://127.0.0.1:4444/` for the site index.
 | `mdshelf share <path>` | Create a link that shares one page with anyone holding the URL |
 | `mdshelf share list` | List share links (`--all` for dead ones, `--json` for an inventory) |
 | `mdshelf share revoke <id>` | Revoke one link, or `--all` to revoke every live link |
-| `mdshelf install` | Register as a native system service |
+| `mdshelf install` | Register `serve`, with any of its options, as a native system service |
 | `mdshelf start` | Start the installed service |
 | `mdshelf stop` | Stop the installed service |
 | `mdshelf restart` | Restart the installed service |
@@ -300,7 +310,7 @@ mid-page. Sessions expire absolutely after `session_max_age` (default 30 days).
 > qualification. Each such event is logged at WARN with the cause, so an outage is
 > visible in the logs rather than presenting as mysterious sign-outs.
 
-### Static export
+### Exporting a private vault
 
 A static bundle has no authentication, so exporting a vault that declares rules
 requires saying whose view it is:
@@ -430,12 +440,69 @@ Keep `mdshelf share list --json` if you want an inventory of what a deletion wou
 before creating a folder's `index.md`. Serving, checking, exporting, and explaining are
 strictly read-only.
 
+## Search
+
+With a `[search]` section, every page gets a search field in its header (`/` or
+<kbd>Cmd</kbd>/<kbd>Ctrl</kbd>+<kbd>K</kbd> opens it). Results update as you type, with the
+matching words highlighted; <kbd>Enter</kbd> opens the selected page, and **All results**
+opens the full results page at `/__search?q=…`. It finds pages both by the words they
+contain and by meaning — "how do we handle incidents" finds an incident response page
+that never uses those words.
+
+mdshelf does not rank pages itself. It works with a [qmd](https://github.com/tobi/qmd)
+server (`npm install -g @tobilu/qmd`), and splits the work so that access rules stay
+mdshelf's:
+
+1. **mdshelf writes a corpus.** At startup and after every content change, mdshelf writes
+   each published page's body — frontmatter removed, drafts left out — to
+   `<corpus>/<collection>/<page path>`. The collection is the site's mount without its
+   leading `/` (`/docs/v2` → `docs-v2`). A page whose file does not end in `.md` gets `.md`
+   appended, because qmd indexes `*.md`.
+2. **qmd indexes the corpus** and answers queries over HTTP. It never sees the vault, so
+   `allow` and `deny` lists are not searchable text.
+3. **mdshelf filters what qmd ranks.** Each ranked page is looked up exactly as a page
+   request would be, for the reader asking. Pages they may not open and drafts are left
+   out, so a query that matches only restricted pages answers exactly like one that
+   matches nothing. Titles and snippets come from mdshelf's own copy of the page.
+
+```toml
+[search]
+qmd_url = "http://127.0.0.1:4600"   # the qmd server holding this site's index
+corpus  = "/var/lib/mdshelf/search" # a directory of its own: mdshelf deletes stale files here
+```
+
+Set up qmd once, with one collection per site, pointing at the corpus:
+
+```bash
+qmd --index mysite collection add /var/lib/mdshelf/search/docs --name docs
+qmd --index mysite embed                           # downloads the ~300 MB embedding model once
+qmd --index mysite mcp --http --host 127.0.0.1 --port 4600
+```
+
+Keep the index current by running `qmd --index mysite update && qmd --index mysite embed`
+on a timer; both only process what changed. Restart the qmd server after adding a
+collection, since it reads the collection list at startup.
+
+mdshelf asks qmd for keyword and vector matches fused by rank, without qmd's language-model
+reranker, so only the embedding model is loaded: about 800 MB of memory per qmd server,
+with queries in tens of milliseconds once warm. The first query after qmd starts loads
+the model and takes several seconds. If qmd is unreachable, the results page says search
+is unavailable rather than showing no matches.
+
+Notes:
+
+- Run qmd with `CI` unset: qmd refuses to load models when `CI` is set.
+- qmd always skips folders named `node_modules`, `.git`, `.cache`, `vendor`, `dist` and
+  `build`, so pages under such folders are not searchable.
+- Share-link readers get no search; anonymous visitors get the sign-in page, as for any
+  other path.
+
 ## Theme customisation
 
 Override or extend templates by pointing `[theme].directory` at a local directory:
 
-- `layouts/` — `base.html`, `doc.html`, `home.html`, `error.html`, `link.html`
-- `partials/` — `header.html`, `sidebar.html`, `toc.html`, `breadcrumbs.html`, `footer.html`
+- `layouts/` — `base.html`, `doc.html`, `home.html`, `error.html`, `link.html`, `search.html`
+- `partials/` — `header.html`, `sidebar.html`, `toc.html`, `breadcrumbs.html`, `footer.html`, `search.html`
 - `assets/` — static files served from `/__assets/...`
 
 ### Theme capabilities
@@ -468,6 +535,35 @@ sudo mdshelf uninstall
 
 Use `--user` for a per-user service where the platform supports it.
 
+`mdshelf install` takes every `serve` option and the service runs with exactly those, so a
+private site installs the same way it serves:
+
+```bash
+sudo mdshelf auth setup        # store Google credentials for the user installing the service
+sudo mdshelf install --auth google --behind-proxy --public-url https://docs.acme.com
+sudo mdshelf start
+```
+
+Install runs the checks `serve` runs before it binds — TLS flags, the `[auth]` and `[links]`
+settings, and the stored credentials — and refuses to register a service that could not
+start. It prints the redirect URI to register with Google.
+
+- **Credentials come from the file, not your shell.** A service does not inherit the
+  installing shell's `MDSHELF_GOOGLE_*` variables, so install requires both in
+  `~/.config/mdshelf/credentials.env` (what `mdshelf auth setup` writes). Run `auth setup`
+  as the same user that runs `install` — with `sudo`, that is root.
+- **The service gets the installer's `HOME`.** With `--auth` or `--domain`, the service is
+  given the installing user's `HOME`, so the credentials, the at-rest key
+  (`~/.config/mdshelf/secret.key`) and the ACME cache resolve to the paths install checked.
+- **Relative paths are pinned** to the directory you ran install in: `--tls-cert`,
+  `--tls-key` and `--acme-cache` are stored as absolute paths.
+- **Paths with spaces** work with launchd, systemd and Windows: install quotes each
+  argument for systemd, so `%` and `$` stay literal too.
+- **OpenRC, rc.d and Windows** services are not given environment variables, so `install`
+  refuses `--auth`, and `--domain` without `--acme-cache`, there. OpenRC and rc.d also
+  split their command on spaces with no way to quote, so install refuses paths or values
+  containing spaces or shell characters on those systems.
+
 ## Serving on port 80
 
 Set `host` and `port` in your config:
@@ -504,7 +600,7 @@ Tailscale provisions a certificate automatically and makes the site available **
 tailscale serve status   # confirm the URL
 ```
 
-Pair it with `sudo mdshelf install && sudo mdshelf start` (see [System service](#system-service)) so both services survive reboots.
+Pair it with `sudo mdshelf install && sudo mdshelf start` (see [System service](#system-service)) so both services survive reboots; pass the same `--auth` and TLS flags to `install` as to `serve`.
 
 To stop serving:
 
@@ -514,7 +610,16 @@ tailscale serve off
 
 ### Public access via Tailscale Funnel
 
-To make the site reachable from the **public internet** (not just your tailnet), use Funnel instead:
+To make the site reachable from the **public internet** (not just your tailnet), use Funnel instead.
+Funnel publishes everything mdshelf serves to anyone, so serve a private site with sign-in on
+and Tailscale's hostname as the public URL:
+
+```bash
+mdshelf serve --auth google --behind-proxy --public-url https://<machine>.<tailnet>.ts.net
+```
+
+Register `https://<machine>.<tailnet>.ts.net/auth/callback` as a redirect URI on your Google
+OAuth client, then:
 
 ```bash
 tailscale funnel --bg 4444
@@ -556,7 +661,7 @@ iPadOS Safari mishandle that, so there they go through `GET /__mdshelf/md/<page>
 enforces exactly the same access rules as viewing the page — a page you may not read
 returns the same response as one that does not exist.
 
-### Static export
+### Exporting a static site
 
 ```bash
 mdshelf export                          # all sites → ./dist

package/npm-shrinkwrap.json

@@ -23,7 +23,7 @@
       "hasInstallScript": true,
       "license": "MIT OR Apache-2.0",
       "name": "mdshelf",
-      "version": "1.1.0"
+      "version": "1.1.2"
     },
     "node_modules/@isaacs/cliui": {
       "engines": {
@@ -542,5 +542,5 @@
     }
   },
   "requires": true,
-  "version": "1.1.0"
+  "version": "1.1.2"
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "artifactDownloadUrls": [
-    "https://github.com/prbski/mdshelf/releases/download/v1.1.0"
+    "https://github.com/prbski/mdshelf/releases/download/v1.1.2"
   ],
   "author": "Patryk Porebski",
   "bin": {
@@ -100,7 +100,7 @@
       "zipExt": ".tar.xz"
     }
   },
-  "version": "1.1.0",
+  "version": "1.1.2",
   "volta": {
     "node": "18.14.1",
     "npm": "9.5.0"