upcontent 0.0.0-stage → 0.1.1

package/README.md

@@ -1,3 +1,133 @@
-# Temporary Holding Version
+---
+heading: Upcontent
+description: Turn a documentation repository into a fast, searchable, customizable static portal.
+---
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+<div align="center">
+
+# Upcontent
+
+**Turn the documentation repository you already have into a fast, searchable, customizable portal.**
+
+[![CI](https://github.com/lumamontes/upcontent/actions/workflows/ci.yml/badge.svg)](https://github.com/lumamontes/upcontent/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/upcontent?color=0f766e&label=npm)](https://www.npmjs.com/package/upcontent)
+
+<a href="https://lumamontes.github.io/upcontent/">See the live showcase</a> · <a href="#get-started">Get started in under a minute</a>
+
+</div>
+
+## See the result
+
+This is the real Upcontent portal generated from this repository and deployed to GitHub Pages.
+
+<p align="center">
+  <a href="https://lumamontes.github.io/upcontent/"><img src="assets/readme/portal-home.png" alt="Upcontent documentation portal home page" width="900"></a>
+</p>
+
+The same build includes a capability showcase for technical content, structured data, callouts, diagrams, navigation, and search.
+
+<p align="center">
+  <a href="https://lumamontes.github.io/upcontent/showcase/"><img src="assets/readme/portal-showcase.png" alt="Upcontent capability showcase page" width="900"></a>
+</p>
+
+## From repository to portal
+
+Upcontent keeps your source of truth in Git and adds the publishing layer: clear navigation, full-text search, technical content rendering, consumer-owned branding, and a repeatable static build.
+
+If your Markdown already lives in a GitHub repository, the flow is:
+
+```sh
+pnpm dlx upcontent init
+pnpm dlx upcontent dev
+pnpm dlx upcontent check
+```
+
+The first command creates `.upcontent/` and a GitHub Pages workflow. The second starts a local portal using the repository you are already in. The third builds the consumer portal and validates the generated site. You do not need to clone the Upcontent renderer into your documentation repository.
+
+Push the generated workflow and your documentation is published as a static site. The source repository remains the source of truth; Upcontent does not require a database, a companion server, or a new authoring system.
+
+## What you get
+
+- A responsive [Starlight](https://starlight.astro.build/) portal with navigation, search, themes, and table of contents.
+- A `.upcontent/config.json` file for identity, navigation, rendering, and content boundaries.
+- Consumer-owned logos, favicon, CSS, site identity, and navigation.
+- Rendered callouts, Mermaid diagrams, JSON, YAML, CSV, and wiki links.
+- A static `dist/` directory deployable to GitHub Pages or any static host.
+- Build checks that catch broken wiki links and excluded content before publication.
+
+## Is it a good fit?
+
+Upcontent is a good fit when:
+
+- Your documentation already lives in Markdown or MDX.
+- Your team wants to keep writing in Git.
+- You want the portal to be owned and deployed by the documentation repository.
+- You need more than raw Markdown, but do not need a dynamic application.
+- You want branding and navigation without maintaining a custom docs frontend.
+
+It is not the right fit when your site needs runtime authentication, server-rendered personalization, or review comments inside the published portal.
+
+## Explore this repository locally
+
+Clone this repository, then start the included golden consumer:
+
+```sh
+git clone https://github.com/lumamontes/upcontent.git
+cd upcontent
+pnpm install
+make dev CONTENT_PATH=.
+```
+
+Then open the local URL printed by Astro. The [capability showcase](showcase/) is the fastest way to see the complete rendering and customization surface.
+
+## Configure your portal
+
+The consumer configuration is deliberately small:
+
+```json
+{
+  "site": {
+    "title": "Engineering Docs",
+    "description": "Documentation for the engineering team.",
+    "socialImage": "https://docs.example.com/social-card.png",
+    "locale": "en-US",
+    "logo": {
+      "src": ".upcontent/logo.svg",
+      "alt": "Engineering Docs"
+    },
+    "favicon": ".upcontent/favicon.svg"
+  },
+  "seo": {
+    "enabled": true
+  },
+  "repo": {
+    "url": "https://github.com/acme/engineering-docs"
+  },
+  "theme": {
+    "customCss": [".upcontent/theme.css"]
+  }
+}
+```
+
+Follow [Set up a consumer repository](getting-started/consumer-repository/) for the complete setup, then use [Customization](customization/) to shape the portal.
+
+## Build for production
+
+```sh
+make build CONTENT_PATH=/path/to/your-consumer-repo
+```
+
+The generated site is written to `dist/` and includes the Pagefind search index. Before publishing, run:
+
+```sh
+pnpm test
+pnpm check
+make build CONTENT_PATH=.
+make check-external
+```
+
+Read the [deployment guide](deployment/) for GitHub Pages and other static hosts.
+
+## Built on Astro and Starlight
+
+Upcontent uses the official [Astro](https://astro.build/) framework and [Starlight](https://starlight.astro.build/) documentation theme as its rendering foundation. Upcontent adds the consumer-repository contract, content integrity checks, configuration surface, and deployment workflow around them.

package/SHOWCASE.mdx

@@ -0,0 +1,163 @@
+---
+heading: Capability showcase
+description: One practical page showing what a consumer repository can render, configure, and publish with Upcontent.
+sidebar:
+  order: 1
+---
+
+import PaletteShowcase from './src/components/PaletteShowcase.astro';
+
+Upcontent turns Markdown into a searchable static documentation site. Configure the consumer repository, add the content formats your team needs, and publish the generated `dist/` directory to a static host.
+
+## Rendered content
+
+The portal accepts ordinary Markdown and adds a small set of useful content features.
+
+### Callouts
+
+:::tip[Use callouts for decisions]
+Keep callouts short. Use them for a warning, a recommendation, or a next action that should not disappear in the surrounding text.
+:::
+
+:::caution
+Static output is public wherever the selected host is public. A private source repository does not automatically make the published site private.
+:::
+
+### Code and diagrams
+
+```ts
+export function buildPortal(contentPath: string) {
+  return `make build CONTENT_PATH=${contentPath}`
+}
+```
+
+```mermaid
+flowchart LR
+  Content[Consumer repository] --> Config[.upcontent/config.json]
+  Config --> Build[Static build]
+  Content --> Build
+  Build --> Output[Searchable portal]
+```
+
+Mermaid diagrams support theme synchronization, fullscreen, zoom, pan, source copying, and an invalid-source fallback.
+
+### Structured examples
+
+JSON and YAML blocks remain readable as collapsible data:
+
+```json
+{
+  "portal": "static",
+  "search": "pagefind",
+  "themes": ["light", "dark"]
+}
+```
+
+```yaml
+contentPath: ./docs
+publish: github-pages
+sourceOfTruth: consumer-repository
+```
+
+CSV becomes a table when the content is tabular:
+
+```csv
+capability,status
+custom-css,available
+blocklist,available
+pagefind,available
+```
+
+### Links and navigation
+
+Regular Markdown links are best for stable routes, such as [site validation](../guides/validate-your-site/). Wiki links are useful when the source repository refers to files by name, such as `[[README]]`. Missing wiki-link targets fail the build instead of producing broken navigation.
+
+## Consumer customization
+
+The consumer changes identity and presentation in `.upcontent/`, without editing the portal renderer:
+
+```text
+.upcontent/
+├── config.json
+├── favicon.svg
+├── logo.svg
+└── theme.css
+```
+
+The configuration controls:
+
+- Site title, description, URL, logo, and favicon
+- Repository links and local custom CSS
+- Social links, table of contents, pagination, last-updated metadata, and code styling
+- Sidebar roots, labels, and content blocklists
+- The frontmatter field used as a page title
+
+Example identity configuration:
+
+```json
+{
+  "site": {
+    "title": "Engineering Docs",
+    "description": "Documentation for the engineering team.",
+    "logo": {
+      "src": ".upcontent/logo.svg",
+      "alt": "Engineering Docs"
+    },
+    "favicon": ".upcontent/favicon.svg"
+  },
+  "repo": {
+    "url": "https://github.com/acme/engineering-docs"
+  },
+  "theme": {
+    "customCss": [".upcontent/theme.css"]
+  }
+}
+```
+
+The [configuration reference](../customization/config-json/) explains every supported field and its fallback behavior.
+
+### Palette variations
+
+The same Starlight surface can carry different consumer identities through tokens and custom CSS:
+
+<PaletteShowcase />
+
+These are examples for a consumer repository, not a runtime theme picker that every published portal needs to expose.
+
+## Content boundaries
+
+The build excludes protected internal paths and consumer blocklists before parsing. This keeps excluded files out of both the content collection and the sidebar.
+
+```json
+{
+  "navigation": {
+    "blocklist": {
+      "exact": ["notes.md"],
+      "prefixes": ["drafts/", "internal/"]
+    }
+  }
+}
+```
+
+Broken wiki links and traversal attempts fail with the source file and invalid reference. The same pipeline also works when `CONTENT_PATH` points to a separate consumer repository.
+
+## Validation and publishing
+
+The repeatable local contract is:
+
+```sh
+pnpm test
+pnpm check
+make build CONTENT_PATH=.
+make check-external
+```
+
+The reference CI workflow validates the site and an external consumer fixture. The deployment workflow publishes `dist/` to a static host such as GitHub Pages.
+
+## Explore the product
+
+- [Your first build](../getting-started/first-build/)
+- [Set up a consumer repository](../getting-started/consumer-repository/)
+- [Authoring content](../guides/authoring-content/)
+- [Validate your site](../guides/validate-your-site/)
+- [Deployment](../deployment/)

package/astro.config.mjs

@@ -0,0 +1,167 @@
+import { fileURLToPath } from 'node:url'
+import { cpSync, existsSync, mkdirSync, rmSync, statSync } from 'node:fs'
+import { basename, resolve, sep } from 'node:path'
+import { unified } from '@astrojs/markdown-remark'
+import sitemap from '@astrojs/sitemap'
+import starlight from '@astrojs/starlight'
+import { defineConfig } from 'astro/config'
+import { visit } from 'unist-util-visit'
+import { rehypeCallouts } from './src/lib/rehype-callouts.ts'
+import { remarkStripDuplicateTitle } from './src/lib/remark-strip-duplicate-title.ts'
+import { remarkStructuredDataPreview } from './src/lib/remark-structured-data-preview.ts'
+import { remarkWikiLinks } from './src/lib/remark-wiki-links.ts'
+import { remarkDocumentLinks } from './src/lib/remark-doc-links.ts'
+import { getPortalConfig } from './src/lib/portal-config.ts'
+import { buildSidebar } from './src/lib/sidebar.ts'
+import { getNoindexRoutes } from './src/lib/seo-sitemap.ts'
+import { PRODUCT_NAME, PRODUCT_TAGLINE } from './src/lib/product-identity.ts'
+
+const docsRoot = fileURLToPath(new URL('./src/content/docs', import.meta.url))
+const portalConfig = getPortalConfig()
+
+function copyContentAssetDirectory(assetPath) {
+  const source = resolve(docsRoot, assetPath)
+  const target = resolve(process.cwd(), 'public', assetPath)
+  if (!existsSync(source) || !statSync(source).isDirectory()) {
+    rmSync(target, { force: true, recursive: true })
+    return
+  }
+
+  rmSync(target, { force: true, recursive: true })
+  mkdirSync(resolve(target, '..'), { recursive: true })
+  cpSync(source, target, { recursive: true })
+}
+
+copyContentAssetDirectory('assets/readme')
+
+function resolvePortalAsset(assetPath) {
+  if (!assetPath || assetPath.startsWith('http')) return assetPath
+  if (assetPath.startsWith('/')) return assetPath
+
+  const source = resolve(docsRoot, assetPath)
+  if (!source.startsWith(`${resolve(docsRoot)}${sep}`) || !existsSync(source) || !statSync(source).isFile()) {
+    console.warn(`[${PRODUCT_NAME}] Portal asset not found: ${source}`)
+    return undefined
+  }
+
+  const targetDir = resolve(process.cwd(), 'public/upcontent-assets')
+  mkdirSync(targetDir, { recursive: true })
+  const targetName = basename(source)
+  cpSync(source, resolve(targetDir, targetName))
+  return `/upcontent-assets/${targetName}`
+}
+
+function resolvePortalLogo(logo) {
+  if (!logo) return undefined
+  if (logo.src.startsWith('http') || logo.src.startsWith('/')) return logo
+  const source = resolve(docsRoot, logo.src)
+  if (!source.startsWith(`${resolve(docsRoot)}${sep}`) || !existsSync(source) || !statSync(source).isFile()) {
+    console.warn(`[${PRODUCT_NAME}] Portal logo not found: ${source}`)
+    return undefined
+  }
+  const targetDir = resolve(process.cwd(), 'src/assets')
+  mkdirSync(targetDir, { recursive: true })
+  const target = resolve(targetDir, 'consumer-logo.svg')
+  cpSync(source, target)
+  return { ...logo, src: './src/assets/consumer-logo.svg' }
+}
+
+const consumerCss = (portalConfig.theme?.customCss ?? [])
+  .map(cssPath => resolve(docsRoot, cssPath))
+  .filter(cssPath => {
+    if (existsSync(cssPath)) return true
+    console.warn(`[${PRODUCT_NAME}] Custom CSS file not found: ${cssPath}`)
+    return false
+  })
+
+const customCss = ['./src/styles/callouts.css', './src/styles/structured-data-preview.css', ...consumerCss]
+const starlightOptions = portalConfig.starlight ?? {}
+const noindexRoutes = getNoindexRoutes(docsRoot)
+const configuredSiteUrl = process.env.SITE_URL || portalConfig.site?.url
+let site
+let base = process.env.BASE_PATH || undefined
+if (portalConfig.seo?.enabled === true && configuredSiteUrl) {
+  try {
+    const parsedSiteUrl = new URL(configuredSiteUrl)
+    if (parsedSiteUrl.protocol !== 'http:' && parsedSiteUrl.protocol !== 'https:') throw new Error('unsupported protocol')
+    if (parsedSiteUrl.username || parsedSiteUrl.password) throw new Error('userinfo is not allowed')
+    site = parsedSiteUrl.origin
+    if (!process.env.BASE_PATH) base = parsedSiteUrl.pathname === '/' ? undefined : parsedSiteUrl.pathname
+  } catch {
+    console.warn(`[${PRODUCT_NAME}] SEO site URL must be an absolute URL: ${configuredSiteUrl}`)
+  }
+}
+
+function includeInSitemap(page) {
+  const pathname = new URL(page).pathname.replace(/\/+$/, '') || '/'
+  const basePath = (base || '').replace(/\/+$/, '')
+  const route = pathname === basePath
+    ? '/'
+    : basePath && pathname.startsWith(`${basePath}/`)
+      ? pathname.slice(basePath.length)
+      : pathname
+  return !noindexRoutes.has(route)
+}
+
+function serializeSitemapEntry(entry) {
+  if (!site) return entry
+  const homepageUrl = new URL(`${(base || '').replace(/\/+$/, '')}/`, site).href
+  if (entry.url === homepageUrl.replace(/\/$/, '') || entry.url === homepageUrl) {
+    return { ...entry, url: homepageUrl }
+  }
+  return entry
+}
+
+// Remark plugin: converts ```mermaid blocks to <div class="mermaid"> BEFORE Shiki runs
+function remarkMermaid() {
+  return (tree) => {
+    visit(tree, 'code', (node, index, parent) => {
+      if (node.lang !== 'mermaid') return
+      // <br/> inside a div becomes a DOM element before Mermaid parses the text, breaking the parser
+      const safe = node.value.replace(/<br\s*\/?>/gi, ' ')
+      parent.children.splice(index, 1, {
+        type: 'html',
+        value: `<div class="mermaid">${safe}</div>`,
+      })
+      return index + 1
+    })
+  }
+}
+
+export default defineConfig({
+  output: 'static',
+  site,
+  base,
+  integrations: [
+    sitemap({ filter: includeInSitemap, serialize: serializeSitemapEntry }),
+    starlight({
+      title: portalConfig.site?.title ?? PRODUCT_NAME,
+      description: portalConfig.site?.description ?? PRODUCT_TAGLINE,
+      logo: resolvePortalLogo(portalConfig.site?.logo),
+      favicon: resolvePortalAsset(portalConfig.site?.favicon),
+      components: {
+        Head: './src/overrides/Head.astro',
+        Footer: './src/overrides/Footer.astro',
+      },
+      customCss,
+      social: starlightOptions.social,
+      tableOfContents: starlightOptions.tableOfContents,
+      lastUpdated: starlightOptions.lastUpdated,
+      pagination: starlightOptions.pagination,
+      expressiveCode: starlightOptions.expressiveCode,
+      sidebar: buildSidebar(docsRoot),
+    }),
+  ],
+  markdown: {
+    processor: unified({
+      remarkPlugins: [
+        remarkStripDuplicateTitle,
+        [remarkWikiLinks, { contentRoot: docsRoot, basePath: import.meta.env.BASE_URL, failOnBrokenLinks: true }],
+        [remarkDocumentLinks, { contentRoot: docsRoot, basePath: import.meta.env.BASE_URL }],
+        remarkMermaid,
+        remarkStructuredDataPreview,
+      ],
+      rehypePlugins: [rehypeCallouts],
+    }),
+  },
+})

package/customization/config-json.md

@@ -0,0 +1,100 @@
+---
+title: JSON configuration reference
+description: Complete reference for the consumer-owned .upcontent/config.json file.
+sidebar:
+  order: 6
+---
+
+The `.upcontent/config.json` file is optional. An omitted field uses the portal or Starlight default.
+
+## Start with the minimum
+
+You only need a title and repository link to get started. Add a logo and custom CSS when you are ready to make the portal yours:
+
+```json
+{
+  "site": {
+    "title": "Engineering Docs",
+    "description": "Documentation for the engineering team."
+  },
+  "repo": {
+    "url": "https://github.com/acme/engineering-docs"
+  }
+}
+```
+
+Relative asset paths resolve from the consumer repository. Absolute URLs are also supported for externally hosted assets.
+
+## Complete example
+
+The optional fields below cover navigation, Starlight presentation, and content conventions. You can add them one group at a time.
+
+<details>
+<summary>Show the complete configuration</summary>
+
+```json
+{
+  "site": {
+    "title": "Engineering Docs",
+    "description": "Documentation for the engineering team.",
+    "url": "https://docs.example.com",
+    "socialImage": "https://docs.example.com/social-card.png",
+    "locale": "en-US",
+    "logo": {
+      "src": ".upcontent/logo.svg",
+      "alt": "Engineering Docs",
+      "replacesTitle": false
+    },
+    "favicon": ".upcontent/favicon.svg"
+  },
+  "seo": {
+    "enabled": true
+  },
+  "repo": {
+    "url": "https://github.com/acme/engineering-docs"
+  },
+  "theme": {
+    "customCss": [".upcontent/theme.css"]
+  },
+  "starlight": {
+    "social": [
+      { "icon": "github", "label": "GitHub", "href": "https://github.com/acme/engineering-docs" }
+    ],
+    "tableOfContents": { "minHeadingLevel": 2, "maxHeadingLevel": 3 },
+    "lastUpdated": true,
+    "pagination": true,
+    "expressiveCode": {
+      "styleOverrides": { "borderRadius": "0.6rem" }
+    }
+  },
+  "navigation": {
+    "roots": ["README.md", "guides"],
+    "labelOverrides": { "api": "API reference" },
+    "blocklist": {
+      "exact": ["notes.md"],
+      "prefixes": ["drafts/"]
+    }
+  },
+  "content": {
+    "titleField": "title"
+  }
+}
+```
+
+</details>
+
+## Field groups
+
+| Group | Controls |
+| --- | --- |
+| `site` | Name, description, URL, social image, locale, logo, and favicon. |
+| `seo` | Explicitly enables search-engine discoverability. Defaults to disabled. |
+| `repo` | Source repository links. |
+| `theme` | Consumer-owned local CSS. |
+| `starlight` | Safe layout, social, table of contents, pagination, and code options. |
+| `navigation` | Sidebar roots, labels, and blocklists. |
+| `content` | Frontmatter title field selection. |
+
+Invalid curated values are ignored or fall back safely. Heading levels must be integers from 1 through 6, and the minimum cannot exceed the maximum.
+
+SEO is opt-in. With `seo.enabled` omitted or set to `false`, the portal emits `noindex, nofollow` and `robots.txt` disallows crawling. This does not protect the site: use access-controlled hosting for a private portal.

package/customization/content.md

@@ -0,0 +1,62 @@
+---
+title: Content behavior
+description: Configure page titles, metadata, and the Markdown features available to authors.
+sidebar:
+  order: 5
+---
+
+Upcontent accepts Markdown and MDX files from the consumer repository. Starlight provides the page layout and frontmatter schema; Upcontent adds title fallback, wiki links, callouts, Mermaid, and structured previews.
+
+## Choose a title field
+
+If an existing repository uses a field such as `heading` instead of `title`, configure it once:
+
+```json
+{
+  "content": {
+    "titleField": "heading"
+  }
+}
+```
+
+When the configured field is missing, the portal falls back to the filename. A file named `first-build.md` becomes `First Build`.
+
+## Page metadata
+
+Use frontmatter at the top of a page:
+
+```md
+---
+heading: Configure a consumer repository
+sidebar:
+  order: 2
+---
+```
+
+Use `##` for body sections because the page title is rendered as the top-level heading.
+
+SEO-specific frontmatter is optional:
+
+```md
+---
+title: Configure a consumer repository
+description: Set up a documentation repository and publish it as a searchable portal.
+canonical: https://docs.example.com/getting-started/consumer-repository/
+image: https://docs.example.com/social-card.png
+noindex: false
+---
+```
+
+`description` is used in search and social metadata. `canonical`, `image`, and `noindex` override the generated defaults for that page. The site-level `socialImage` in `.upcontent/config.json` is used when a page does not define its own image.
+
+## Supported content features
+
+- Obsidian-style callouts for notes, tips, cautions, and dangers
+- Mermaid diagrams with theme-aware rendering and interaction controls
+- JSON and YAML data previews
+- CSV tables
+- `[[wiki links]]` with build-time target validation
+- Syntax-highlighted code blocks
+- Standard Markdown links, tables, lists, and images
+
+See [Authoring content](../guides/authoring-content/) for writing rules and the [capability showcase](../showcase/) for rendered examples.

package/customization/environment.md

@@ -0,0 +1,43 @@
+---
+title: Environment variables
+description: Configure repository links and hosting paths at build time.
+sidebar:
+  order: 7
+---
+
+The portal is static, but a few values belong to the build environment rather than consumer content.
+
+## Supported variables
+
+| Variable | Purpose | Example |
+| --- | --- | --- |
+| `CONTENT_PATH` | Filesystem path passed to `make dev` or `make build`. | `./` |
+| `REPO_URL` | Overrides the configured source repository URL for generated GitHub links. | `https://github.com/acme/docs` |
+| `BASE_PATH` | URL prefix for a project site hosted below the domain root. | `/engineering-docs` |
+| `SITE_URL` | Canonical site origin used by Astro integrations such as the sitemap. | `https://docs.example.com` |
+
+## Local development
+
+Pass the content path to Make:
+
+```sh
+make dev CONTENT_PATH=.
+```
+
+`CONTENT_PATH` is a Make variable, not a value read from `.upcontent/config.json`.
+
+## Static hosting
+
+For GitHub Pages project sites, set a base path and site URL in the deployment workflow:
+
+```yaml
+env:
+  BASE_PATH: /engineering-docs
+  SITE_URL: https://acme.github.io/engineering-docs
+```
+
+For a custom domain or a host serving the site at `/`, leave `BASE_PATH` empty and set `SITE_URL` to the canonical origin.
+
+## Precedence
+
+`REPO_URL` takes precedence over `repo.url` for generated source links. `BASE_PATH` and `SITE_URL` are build inputs; they do not change the consumer configuration file.