@rmdes/indiekit-endpoint-microsub 1.0.28 → 1.0.30

package/README.md

@@ -0,0 +1,246 @@
+# @rmdes/indiekit-endpoint-microsub
+
+A comprehensive Microsub social reader plugin for Indiekit. Subscribe to feeds (RSS, Atom, JSON Feed, h-feed), organize them into channels, and read posts in a unified timeline interface with a built-in web reader UI.
+
+## Features
+
+- **Microsub Protocol**: Full implementation of the [Microsub spec](https://indieweb.org/Microsub)
+- **Multi-Format Feeds**: RSS, Atom, JSON Feed, h-feed (microformats)
+- **Smart Polling**: Adaptive tiered polling (2 minutes to 17+ hours) based on update frequency
+- **Real-Time Updates**: WebSub (PubSubHubbub) support for instant notifications
+- **Web Reader UI**: Built-in reader interface with channel navigation and timeline view
+- **Feed Discovery**: Automatic discovery of feeds from website URLs
+- **Read State**: Per-user read tracking with automatic cleanup
+- **Compose Interface**: Post replies, likes, reposts, and bookmarks via Micropub
+- **Webmention Support**: Receive webmentions in your notifications channel
+- **Media Proxy**: Privacy-friendly image proxying
+- **OPML Export**: Export your subscriptions as OPML
+
+## Installation
+
+```bash
+npm install @rmdes/indiekit-endpoint-microsub
+```
+
+## Configuration
+
+Add to your Indiekit config:
+
+```javascript
+import MicrosubEndpoint from "@rmdes/indiekit-endpoint-microsub";
+
+export default {
+  plugins: [
+    new MicrosubEndpoint({
+      mountPath: "/microsub", // Default mount path
+    }),
+  ],
+};
+```
+
+## Usage
+
+### Web Reader UI
+
+Navigate to `/microsub/reader` in your Indiekit installation to access the web interface.
+
+**Channels**: Organize feeds into channels (Technology, News, Friends, etc.)
+- Create new channels
+- Configure content filters (exclude types, regex patterns)
+- Reorder channels
+
+**Feeds**: Manage subscriptions within each channel
+- Subscribe to feeds by URL
+- Search and discover feeds from websites
+- Edit or rediscover feed URLs
+- Force refresh feeds
+- View feed health status
+
+**Timeline**: Read posts from subscribed feeds
+- Paginated timeline view
+- Mark individual items or all items as read
+- View read items separately
+- Click through to original posts
+
+**Compose**: Create posts via Micropub
+- Reply to posts
+- Like posts
+- Repost posts
+- Bookmark posts
+- Include syndication targets
+
+### Microsub API
+
+Compatible with Microsub clients like [Indigenous](https://indigenous.realize.be/) and [Monocle](https://monocle.p3k.io/).
+
+**Endpoint:** Your Indiekit URL + `/microsub`
+
+**Supported Actions:**
+- `channels` - List, create, update, delete, reorder channels
+- `timeline` - Get timeline items (paginated)
+- `follow` - Subscribe to a feed
+- `unfollow` - Unsubscribe from a feed
+- `mute` - Mute URLs
+- `unmute` - Unmute URLs
+- `block` - Block authors
+- `unblock` - Unblock authors
+- `search` - Discover feeds from URL
+- `preview` - Preview feed before subscribing
+
+**Example:**
+
+```bash
+# List channels
+curl "https://your-site.example/microsub?action=channels" \
+  -H "Authorization: Bearer YOUR_TOKEN"
+
+# Get timeline for channel
+curl "https://your-site.example/microsub?action=timeline&channel=CHANNEL_UID" \
+  -H "Authorization: Bearer YOUR_TOKEN"
+
+# Subscribe to feed
+curl "https://your-site.example/microsub" \
+  -X POST \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -d "action=follow&channel=CHANNEL_UID&url=https://example.com/feed"
+```
+
+## Feed Polling
+
+Feeds are polled using an adaptive tiered system:
+
+- **Tier 0**: 1 minute (very active feeds)
+- **Tier 1**: 2 minutes (active feeds)
+- **Tier 2**: 4 minutes
+- **Tier 3**: 8 minutes
+- ...
+- **Tier 10**: ~17 hours (inactive feeds)
+
+Tiers adjust automatically:
+- Feed updates → decrease tier (faster polling)
+- No changes for 2+ fetches → increase tier (slower polling)
+
+WebSub-enabled feeds receive instant updates when available.
+
+## Read State Management
+
+Read items are tracked per user. To prevent database bloat, only the last 30 read items per channel are kept. Unread items are never deleted.
+
+Cleanup runs automatically:
+- On server startup
+- After marking items read
+
+## Integration with Other Plugins
+
+### Blogroll Plugin
+
+If `@rmdes/indiekit-endpoint-blogroll` is installed, Microsub will automatically sync feed subscriptions:
+- Subscribe to feed → adds to blogroll
+- Unsubscribe → soft-deletes from blogroll
+
+### Micropub Plugin
+
+The compose interface posts via Micropub. Ensure `@indiekit/endpoint-micropub` is configured.
+
+## OPML Export
+
+Export your subscriptions:
+
+```
+GET /microsub/reader/opml
+```
+
+Returns OPML XML with all subscribed feeds organized by channel.
+
+## Webmentions
+
+The plugin accepts webmentions at `/microsub/webmention`. Received webmentions appear in the special "Notifications" channel.
+
+To advertise your webmention endpoint, add to your site's `<head>`:
+
+```html
+<link rel="webmention" href="https://your-site.example/microsub/webmention" />
+```
+
+## Media Proxy
+
+External images are proxied through `/microsub/media/:hash` for privacy and caching. This prevents your IP address from being sent to third-party image hosts.
+
+## API Response Format
+
+All API responses follow the Microsub spec. Timeline items use the [jf2 format](https://jf2.spec.indieweb.org/).
+
+**Example timeline response:**
+
+```json
+{
+  "items": [
+    {
+      "type": "entry",
+      "uid": "https://example.com/post/123",
+      "url": "https://example.com/post/123",
+      "published": "2026-02-13T12:00:00.000Z",
+      "name": "Post Title",
+      "content": {
+        "text": "Plain text content",
+        "html": "<p>HTML content</p>"
+      },
+      "author": {
+        "name": "Author Name",
+        "url": "https://author.example/",
+        "photo": "https://author.example/photo.jpg"
+      },
+      "_id": "507f1f77bcf86cd799439011",
+      "_is_read": false
+    }
+  ],
+  "paging": {
+    "after": "cursor-string"
+  }
+}
+```
+
+## Database Collections
+
+The plugin creates these MongoDB collections:
+
+- `microsub_channels` - User channels
+- `microsub_feeds` - Feed subscriptions with polling metadata
+- `microsub_items` - Timeline items (posts)
+- `microsub_notifications` - Notifications channel items
+- `microsub_muted` - Muted URLs
+- `microsub_blocked` - Blocked authors
+
+## Troubleshooting
+
+### Feeds not updating
+
+- Check the feed's `nextFetchAt` time in the admin UI
+- Use "Force Refresh" button to poll immediately
+- Try "Rediscover" to find the correct feed URL
+
+### "Unable to detect feed type" error
+
+- The URL may not be a valid feed
+- Try using the search feature to discover feeds from the homepage
+- Check if the feed requires authentication
+
+### Items disappearing after marking read
+
+This is normal behavior - only the last 30 read items per channel are kept to prevent database bloat. Unread items are never deleted.
+
+### Duplicate items
+
+Deduplication is based on the feed's GUID/URL. If a feed doesn't provide stable GUIDs, duplicates may appear.
+
+## Contributing
+
+Issues and pull requests welcome at [github.com/rmdes/indiekit-endpoint-microsub](https://github.com/rmdes/indiekit-endpoint-microsub)
+
+## License
+
+MIT
+
+## Credits
+
+Built by [Ricardo Mendes](https://rmendes.net) for the IndieWeb community.

package/index.js

@@ -1,4 +1,5 @@
 import path from "node:path";
+import { fileURLToPath } from "node:url";
 
 import express from "express";
 
@@ -29,6 +30,14 @@ export default class MicrosubEndpoint {
     this.mountPath = this.options.mountPath;
   }
 
+  /**
+   * Locales directory path
+   * @returns {string} Path to locales directory
+   */
+  get localesDirectory() {
+    return path.join(path.dirname(fileURLToPath(import.meta.url)), "locales");
+  }
+
   /**
    * Navigation items for Indiekit admin
    * @returns {object} Navigation item configuration

package/lib/media/proxy.js

@@ -7,6 +7,60 @@ import crypto from "node:crypto";
 
 import { getCache, setCache } from "../cache/redis.js";
 
+/**
+ * Private/internal IP ranges that should never be fetched (SSRF protection)
+ */
+const BLOCKED_HOSTNAMES = new Set(["localhost", "0.0.0.0"]);
+const BLOCKED_IP_PREFIXES = [
+  "127.", // Loopback
+  "10.", // Private Class A
+  "192.168.", // Private Class C
+  "169.254.", // Link-local
+  "0.", // Current network
+];
+
+/**
+ * Check if a hostname resolves to a private/internal address
+ * @param {string} urlString - URL to check
+ * @returns {boolean} True if the URL targets a private/internal address
+ */
+export function isPrivateUrl(urlString) {
+  try {
+    const parsed = new URL(urlString);
+    const hostname = parsed.hostname;
+
+    // Block known private hostnames
+    if (BLOCKED_HOSTNAMES.has(hostname)) {
+      return true;
+    }
+
+    // Block IPv6 loopback
+    if (hostname === "::1" || hostname === "[::1]") {
+      return true;
+    }
+
+    // Block private IPv4 ranges
+    for (const prefix of BLOCKED_IP_PREFIXES) {
+      if (hostname.startsWith(prefix)) {
+        return true;
+      }
+    }
+
+    // Block 172.16.0.0/12 (172.16.x.x - 172.31.x.x)
+    const match172 = hostname.match(/^172\.(\d+)\./);
+    if (match172) {
+      const second = Number.parseInt(match172[1], 10);
+      if (second >= 16 && second <= 31) {
+        return true;
+      }
+    }
+
+    return false;
+  } catch {
+    return true; // Invalid URLs are blocked
+  }
+}
+
 const MAX_SIZE = 2 * 1024 * 1024; // 2MB max image size
 const CACHE_TTL = 4 * 60 * 60; // 4 hours
 const ALLOWED_TYPES = new Set([
@@ -99,6 +153,12 @@ export function proxyItemImages(item, baseUrl) {
  * @returns {Promise<object|null>} Cached image data or null
  */
 export async function fetchImage(redis, url) {
+  // Block private/internal URLs (defense-in-depth)
+  if (isPrivateUrl(url)) {
+    console.error(`[Microsub] Media proxy blocked private URL: ${url}`);
+    return;
+  }
+
   const cacheKey = `media:${hashUrl(url)}`;
 
   // Try cache first
@@ -194,6 +254,11 @@ export async function handleMediaProxy(request, response) {
     return response.status(400).send("Invalid URL");
   }
 
+  // Block requests to private/internal networks (SSRF protection)
+  if (isPrivateUrl(url)) {
+    return response.status(403).send("URL not allowed");
+  }
+
   // Get Redis client from application
   const { application } = request.app.locals;
   const redis = application.redis;
@@ -202,8 +267,7 @@ export async function handleMediaProxy(request, response) {
   const imageData = await fetchImage(redis, url);
 
   if (!imageData) {
-    // Redirect to original URL as fallback
-    return response.redirect(url);
+    return response.status(404).send("Image not available");
   }
 
   // Set cache headers

package/lib/storage/items.js

@@ -602,7 +602,11 @@ export async function searchItems(application, channelId, query, limit = 20) {
     typeof channelId === "string" ? new ObjectId(channelId) : channelId;
 
   // Use regex search (consider adding text index for better performance)
-  const regex = new RegExp(query, "i");
+  const escapedQuery = query.replaceAll(
+    /[$()*+.?[\\\]^{|}]/g,
+    String.raw`\$&`,
+  );
+  const regex = new RegExp(escapedQuery, "i");
   const items = await collection
     .find({
       channelId: objectId,

package/lib/webmention/verifier.js

@@ -4,6 +4,29 @@
  */
 
 import { mf2 } from "microformats-parser";
+import sanitizeHtml from "sanitize-html";
+
+/**
+ * Sanitize HTML options (matches normalizer.js)
+ */
+const SANITIZE_OPTIONS = {
+  allowedTags: [
+    "a", "abbr", "b", "blockquote", "br", "code", "em", "figcaption",
+    "figure", "h1", "h2", "h3", "h4", "h5", "h6", "hr", "i", "img",
+    "li", "ol", "p", "pre", "s", "span", "strike", "strong", "sub",
+    "sup", "table", "tbody", "td", "th", "thead", "tr", "u", "ul",
+    "video", "audio", "source",
+  ],
+  allowedAttributes: {
+    a: ["href", "title", "rel"],
+    img: ["src", "alt", "title", "width", "height"],
+    video: ["src", "poster", "controls", "width", "height"],
+    audio: ["src", "controls"],
+    source: ["src", "type"],
+    "*": ["class"],
+  },
+  allowedSchemes: ["http", "https", "mailto"],
+};
 
 /**
  * Verify a webmention
@@ -276,7 +299,7 @@ function extractContent(entry) {
 
   return {
     text: content.value,
-    html: content.html,
+    html: content.html ? sanitizeHtml(content.html, SANITIZE_OPTIONS) : undefined,
   };
 }
 

package/locales/de.json

@@ -0,0 +1,104 @@
+{
+  "microsub": {
+    "reader": {
+      "title": "Leser",
+      "empty": "Keine Elemente zum Anzeigen",
+      "markAllRead": "Alle als gelesen markieren",
+      "showRead": "Gelesene anzeigen ({{count}})",
+      "hideRead": "Gelesene Elemente ausblenden",
+      "allRead": "Alles aufgeholt!",
+      "newer": "Neuer",
+      "older": "Älter"
+    },
+    "channels": {
+      "title": "Kanäle",
+      "name": "Kanalname",
+      "new": "Neuer Kanal",
+      "create": "Kanal erstellen",
+      "delete": "Kanal löschen",
+      "settings": "Kanaleinstellungen",
+      "empty": "Noch keine Kanäle. Erstellen Sie einen, um zu beginnen.",
+      "notifications": "Benachrichtigungen"
+    },
+    "timeline": {
+      "title": "Zeitleiste",
+      "empty": "Keine Elemente in diesem Kanal",
+      "markRead": "Als gelesen markieren",
+      "markUnread": "Als ungelesen markieren",
+      "remove": "Entfernen"
+    },
+    "feeds": {
+      "title": "Feeds",
+      "follow": "Folgen",
+      "subscribe": "Einen Feed abonnieren",
+      "unfollow": "Entfolgen",
+      "empty": "Keine Feeds in diesem Kanal abonniert",
+      "url": "Feed-URL",
+      "urlPlaceholder": "https://beispiel.de/feed.xml",
+      "edit": "Feed bearbeiten",
+      "rediscover": "Feed neu entdecken",
+      "refresh": "Jetzt aktualisieren",
+      "status": {
+        "active": "Aktiv",
+        "error": "Fehler",
+        "stale": "Veraltet"
+      }
+    },
+    "item": {
+      "reply": "Antworten",
+      "like": "Gefällt mir",
+      "repost": "Teilen",
+      "bookmark": "Lesezeichen",
+      "viewOriginal": "Original anzeigen"
+    },
+    "compose": {
+      "title": "Verfassen",
+      "content": "Was beschäftigt Sie?",
+      "comment": "Kommentar hinzufügen (optional)",
+      "commentHint": "Ihr Kommentar wird bei der Syndizierung mit einbezogen",
+      "syndicateTo": "Syndizieren an",
+      "syndicateHint": "Wählen Sie aus, wo dies quergepostet werden soll",
+      "submit": "Veröffentlichen",
+      "cancel": "Abbrechen",
+      "replyTo": "Antworten an",
+      "likeOf": "Gefällt mir",
+      "repostOf": "Teilen",
+      "bookmarkOf": "Lesezeichen setzen"
+    },
+    "settings": {
+      "title": "{{channel}}-Einstellungen",
+      "excludeTypes": "Interaktionstypen ausschließen",
+      "excludeTypesHelp": "Wählen Sie Beitragstypen aus, die in diesem Kanal ausgeblendet werden sollen",
+      "excludeRegex": "Ausschlussmuster",
+      "excludeRegexHelp": "Regulärer Ausdruck zum Herausfiltern übereinstimmender Inhalte",
+      "save": "Einstellungen speichern",
+      "dangerZone": "Gefahrenzone",
+      "deleteWarning": "Das Löschen dieses Kanals entfernt dauerhaft alle Feeds und Elemente. Diese Aktion kann nicht rückgängig gemacht werden.",
+      "deleteConfirm": "Sind Sie sicher, dass Sie diesen Kanal und alle seine Inhalte löschen möchten?",
+      "delete": "Kanal löschen",
+      "types": {
+        "like": "Gefällt mir",
+        "repost": "Geteilte Beiträge",
+        "bookmark": "Lesezeichen",
+        "reply": "Antworten",
+        "checkin": "Check-ins"
+      }
+    },
+    "search": {
+      "title": "Suchen",
+      "placeholder": "URL oder Suchbegriff eingeben",
+      "submit": "Suchen",
+      "noResults": "Keine Ergebnisse gefunden"
+    },
+    "preview": {
+      "title": "Vorschau",
+      "subscribe": "Diesen Feed abonnieren"
+    },
+    "error": {
+      "channelNotFound": "Kanal nicht gefunden",
+      "feedNotFound": "Feed nicht gefunden",
+      "invalidUrl": "Ungültige URL",
+      "invalidAction": "Ungültige Aktion"
+    }
+  }
+}

package/locales/es-419.json

@@ -0,0 +1,104 @@
+{
+  "microsub": {
+    "reader": {
+      "title": "Lector",
+      "empty": "No hay elementos para mostrar",
+      "markAllRead": "Marcar todo como leído",
+      "showRead": "Mostrar leídos ({{count}})",
+      "hideRead": "Ocultar elementos leídos",
+      "allRead": "¡Todo al día!",
+      "newer": "Más reciente",
+      "older": "Más antiguo"
+    },
+    "channels": {
+      "title": "Canales",
+      "name": "Nombre del canal",
+      "new": "Nuevo canal",
+      "create": "Crear canal",
+      "delete": "Eliminar canal",
+      "settings": "Configuración del canal",
+      "empty": "Todavía no hay canales. Creá uno para empezar.",
+      "notifications": "Notificaciones"
+    },
+    "timeline": {
+      "title": "Línea de tiempo",
+      "empty": "No hay elementos en este canal",
+      "markRead": "Marcar como leído",
+      "markUnread": "Marcar como no leído",
+      "remove": "Eliminar"
+    },
+    "feeds": {
+      "title": "Feeds",
+      "follow": "Seguir",
+      "subscribe": "Suscribirse a un feed",
+      "unfollow": "Dejar de seguir",
+      "empty": "No se sigue ningún feed en este canal",
+      "url": "URL del feed",
+      "urlPlaceholder": "https://ejemplo.com/feed.xml",
+      "edit": "Editar feed",
+      "rediscover": "Redescubrir feed",
+      "refresh": "Actualizar ahora",
+      "status": {
+        "active": "Activo",
+        "error": "Error",
+        "stale": "Obsoleto"
+      }
+    },
+    "item": {
+      "reply": "Responder",
+      "like": "Me gusta",
+      "repost": "Repostear",
+      "bookmark": "Marcador",
+      "viewOriginal": "Ver original"
+    },
+    "compose": {
+      "title": "Redactar",
+      "content": "¿Qué estás pensando?",
+      "comment": "Agregar un comentario (opcional)",
+      "commentHint": "Tu comentario se incluirá cuando esto se sindique",
+      "syndicateTo": "Sindicar a",
+      "syndicateHint": "Seleccioná dónde publicar esto",
+      "submit": "Publicar",
+      "cancel": "Cancelar",
+      "replyTo": "Respondiendo a",
+      "likeOf": "Me gusta",
+      "repostOf": "Reposteando",
+      "bookmarkOf": "Marcando"
+    },
+    "settings": {
+      "title": "Configuración de {{channel}}",
+      "excludeTypes": "Excluir tipos de interacción",
+      "excludeTypesHelp": "Seleccioná los tipos de publicaciones que querés ocultar de este canal",
+      "excludeRegex": "Patrón de exclusión",
+      "excludeRegexHelp": "Expresión regular para filtrar contenido coincidente",
+      "save": "Guardar configuración",
+      "dangerZone": "Zona de peligro",
+      "deleteWarning": "Eliminar este canal eliminará permanentemente todos los feeds y elementos. Esta acción no se puede deshacer.",
+      "deleteConfirm": "¿Estás seguro de que querés eliminar este canal y todo su contenido?",
+      "delete": "Eliminar canal",
+      "types": {
+        "like": "Me gusta",
+        "repost": "Reposteos",
+        "bookmark": "Marcadores",
+        "reply": "Respuestas",
+        "checkin": "Registros"
+      }
+    },
+    "search": {
+      "title": "Buscar",
+      "placeholder": "Ingresá URL o término de búsqueda",
+      "submit": "Buscar",
+      "noResults": "No se encontraron resultados"
+    },
+    "preview": {
+      "title": "Vista previa",
+      "subscribe": "Suscribirse a este feed"
+    },
+    "error": {
+      "channelNotFound": "Canal no encontrado",
+      "feedNotFound": "Feed no encontrado",
+      "invalidUrl": "URL no válida",
+      "invalidAction": "Acción no válida"
+    }
+  }
+}