@rmdes/indiekit-endpoint-microsub 1.0.27 → 1.0.29

package/README.md

@@ -0,0 +1,246 @@
+# @rmdes/indiekit-endpoint-microsub
+
+A comprehensive Microsub social reader plugin for Indiekit. Subscribe to feeds (RSS, Atom, JSON Feed, h-feed), organize them into channels, and read posts in a unified timeline interface with a built-in web reader UI.
+
+## Features
+
+- **Microsub Protocol**: Full implementation of the [Microsub spec](https://indieweb.org/Microsub)
+- **Multi-Format Feeds**: RSS, Atom, JSON Feed, h-feed (microformats)
+- **Smart Polling**: Adaptive tiered polling (2 minutes to 17+ hours) based on update frequency
+- **Real-Time Updates**: WebSub (PubSubHubbub) support for instant notifications
+- **Web Reader UI**: Built-in reader interface with channel navigation and timeline view
+- **Feed Discovery**: Automatic discovery of feeds from website URLs
+- **Read State**: Per-user read tracking with automatic cleanup
+- **Compose Interface**: Post replies, likes, reposts, and bookmarks via Micropub
+- **Webmention Support**: Receive webmentions in your notifications channel
+- **Media Proxy**: Privacy-friendly image proxying
+- **OPML Export**: Export your subscriptions as OPML
+
+## Installation
+
+```bash
+npm install @rmdes/indiekit-endpoint-microsub
+```
+
+## Configuration
+
+Add to your Indiekit config:
+
+```javascript
+import MicrosubEndpoint from "@rmdes/indiekit-endpoint-microsub";
+
+export default {
+  plugins: [
+    new MicrosubEndpoint({
+      mountPath: "/microsub", // Default mount path
+    }),
+  ],
+};
+```
+
+## Usage
+
+### Web Reader UI
+
+Navigate to `/microsub/reader` in your Indiekit installation to access the web interface.
+
+**Channels**: Organize feeds into channels (Technology, News, Friends, etc.)
+- Create new channels
+- Configure content filters (exclude types, regex patterns)
+- Reorder channels
+
+**Feeds**: Manage subscriptions within each channel
+- Subscribe to feeds by URL
+- Search and discover feeds from websites
+- Edit or rediscover feed URLs
+- Force refresh feeds
+- View feed health status
+
+**Timeline**: Read posts from subscribed feeds
+- Paginated timeline view
+- Mark individual items or all items as read
+- View read items separately
+- Click through to original posts
+
+**Compose**: Create posts via Micropub
+- Reply to posts
+- Like posts
+- Repost posts
+- Bookmark posts
+- Include syndication targets
+
+### Microsub API
+
+Compatible with Microsub clients like [Indigenous](https://indigenous.realize.be/) and [Monocle](https://monocle.p3k.io/).
+
+**Endpoint:** Your Indiekit URL + `/microsub`
+
+**Supported Actions:**
+- `channels` - List, create, update, delete, reorder channels
+- `timeline` - Get timeline items (paginated)
+- `follow` - Subscribe to a feed
+- `unfollow` - Unsubscribe from a feed
+- `mute` - Mute URLs
+- `unmute` - Unmute URLs
+- `block` - Block authors
+- `unblock` - Unblock authors
+- `search` - Discover feeds from URL
+- `preview` - Preview feed before subscribing
+
+**Example:**
+
+```bash
+# List channels
+curl "https://your-site.example/microsub?action=channels" \
+  -H "Authorization: Bearer YOUR_TOKEN"
+
+# Get timeline for channel
+curl "https://your-site.example/microsub?action=timeline&channel=CHANNEL_UID" \
+  -H "Authorization: Bearer YOUR_TOKEN"
+
+# Subscribe to feed
+curl "https://your-site.example/microsub" \
+  -X POST \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -d "action=follow&channel=CHANNEL_UID&url=https://example.com/feed"
+```
+
+## Feed Polling
+
+Feeds are polled using an adaptive tiered system:
+
+- **Tier 0**: 1 minute (very active feeds)
+- **Tier 1**: 2 minutes (active feeds)
+- **Tier 2**: 4 minutes
+- **Tier 3**: 8 minutes
+- ...
+- **Tier 10**: ~17 hours (inactive feeds)
+
+Tiers adjust automatically:
+- Feed updates → decrease tier (faster polling)
+- No changes for 2+ fetches → increase tier (slower polling)
+
+WebSub-enabled feeds receive instant updates when available.
+
+## Read State Management
+
+Read items are tracked per user. To prevent database bloat, only the last 30 read items per channel are kept. Unread items are never deleted.
+
+Cleanup runs automatically:
+- On server startup
+- After marking items read
+
+## Integration with Other Plugins
+
+### Blogroll Plugin
+
+If `@rmdes/indiekit-endpoint-blogroll` is installed, Microsub will automatically sync feed subscriptions:
+- Subscribe to feed → adds to blogroll
+- Unsubscribe → soft-deletes from blogroll
+
+### Micropub Plugin
+
+The compose interface posts via Micropub. Ensure `@indiekit/endpoint-micropub` is configured.
+
+## OPML Export
+
+Export your subscriptions:
+
+```
+GET /microsub/reader/opml
+```
+
+Returns OPML XML with all subscribed feeds organized by channel.
+
+## Webmentions
+
+The plugin accepts webmentions at `/microsub/webmention`. Received webmentions appear in the special "Notifications" channel.
+
+To advertise your webmention endpoint, add to your site's `<head>`:
+
+```html
+<link rel="webmention" href="https://your-site.example/microsub/webmention" />
+```
+
+## Media Proxy
+
+External images are proxied through `/microsub/media/:hash` for privacy and caching. This prevents your IP address from being sent to third-party image hosts.
+
+## API Response Format
+
+All API responses follow the Microsub spec. Timeline items use the [jf2 format](https://jf2.spec.indieweb.org/).
+
+**Example timeline response:**
+
+```json
+{
+  "items": [
+    {
+      "type": "entry",
+      "uid": "https://example.com/post/123",
+      "url": "https://example.com/post/123",
+      "published": "2026-02-13T12:00:00.000Z",
+      "name": "Post Title",
+      "content": {
+        "text": "Plain text content",
+        "html": "<p>HTML content</p>"
+      },
+      "author": {
+        "name": "Author Name",
+        "url": "https://author.example/",
+        "photo": "https://author.example/photo.jpg"
+      },
+      "_id": "507f1f77bcf86cd799439011",
+      "_is_read": false
+    }
+  ],
+  "paging": {
+    "after": "cursor-string"
+  }
+}
+```
+
+## Database Collections
+
+The plugin creates these MongoDB collections:
+
+- `microsub_channels` - User channels
+- `microsub_feeds` - Feed subscriptions with polling metadata
+- `microsub_items` - Timeline items (posts)
+- `microsub_notifications` - Notifications channel items
+- `microsub_muted` - Muted URLs
+- `microsub_blocked` - Blocked authors
+
+## Troubleshooting
+
+### Feeds not updating
+
+- Check the feed's `nextFetchAt` time in the admin UI
+- Use "Force Refresh" button to poll immediately
+- Try "Rediscover" to find the correct feed URL
+
+### "Unable to detect feed type" error
+
+- The URL may not be a valid feed
+- Try using the search feature to discover feeds from the homepage
+- Check if the feed requires authentication
+
+### Items disappearing after marking read
+
+This is normal behavior - only the last 30 read items per channel are kept to prevent database bloat. Unread items are never deleted.
+
+### Duplicate items
+
+Deduplication is based on the feed's GUID/URL. If a feed doesn't provide stable GUIDs, duplicates may appear.
+
+## Contributing
+
+Issues and pull requests welcome at [github.com/rmdes/indiekit-endpoint-microsub](https://github.com/rmdes/indiekit-endpoint-microsub)
+
+## License
+
+MIT
+
+## Credits
+
+Built by [Ricardo Mendes](https://rmendes.net) for the IndieWeb community.

package/index.js

@@ -1,4 +1,5 @@
 import path from "node:path";
+import { fileURLToPath } from "node:url";
 
 import express from "express";
 
@@ -29,6 +30,14 @@ export default class MicrosubEndpoint {
     this.mountPath = this.options.mountPath;
   }
 
+  /**
+   * Locales directory path
+   * @returns {string} Path to locales directory
+   */
+  get localesDirectory() {
+    return path.join(path.dirname(fileURLToPath(import.meta.url)), "locales");
+  }
+
   /**
    * Navigation items for Indiekit admin
    * @returns {object} Navigation item configuration

package/lib/controllers/block.js

@@ -54,7 +54,7 @@ export async function block(request, response) {
     await collection.insertOne({
       userId,
       url,
-      createdAt: new Date(),
+      createdAt: new Date().toISOString(),
     });
   }
 

package/lib/controllers/mute.js

@@ -85,7 +85,7 @@ export async function mute(request, response) {
       userId,
       channelId,
       url,
-      createdAt: new Date(),
+      createdAt: new Date().toISOString(),
     });
   }
 

package/lib/storage/channels.js

@@ -74,8 +74,8 @@ export async function createChannel(application, { name, userId }) {
       excludeTypes: [],
       excludeRegex: undefined,
     },
-    createdAt: new Date(),
-    updatedAt: new Date(),
+    createdAt: new Date().toISOString(),
+    updatedAt: new Date().toISOString(),
   };
 
   await collection.insertOne(channel);
@@ -185,7 +185,7 @@ export async function updateChannel(application, uid, updates, userId) {
     {
       $set: {
         ...updates,
-        updatedAt: new Date(),
+        updatedAt: new Date().toISOString(),
       },
     },
     { returnDocument: "after" },
@@ -242,7 +242,7 @@ export async function reorderChannels(application, channelUids, userId) {
   const operations = channelUids.map((uid, index) => ({
     updateOne: {
       filter: userId ? { uid, userId } : { uid },
-      update: { $set: { order: index, updatedAt: new Date() } },
+      update: { $set: { order: index, updatedAt: new Date().toISOString() } },
     },
   }));
 
@@ -298,8 +298,8 @@ export async function ensureNotificationsChannel(application, userId) {
       excludeTypes: [],
       excludeRegex: undefined,
     },
-    createdAt: new Date(),
-    updatedAt: new Date(),
+    createdAt: new Date().toISOString(),
+    updatedAt: new Date().toISOString(),
   };
 
   await collection.insertOne(channel);

package/lib/storage/feeds.js

@@ -45,11 +45,11 @@ export async function createFeed(
     photo: photo || undefined,
     tier: 1, // Start at tier 1 (2 minutes)
     unmodified: 0,
-    nextFetchAt: new Date(), // Fetch immediately
+    nextFetchAt: new Date(), // Fetch immediately (kept as Date for query compatibility)
     lastFetchedAt: undefined,
     websub: undefined, // Will be populated if hub is discovered
-    createdAt: new Date(),
-    updatedAt: new Date(),
+    createdAt: new Date().toISOString(),
+    updatedAt: new Date().toISOString(),
   };
 
   await collection.insertOne(feed);
@@ -114,7 +114,7 @@ export async function updateFeed(application, id, updates) {
     {
       $set: {
         ...updates,
-        updatedAt: new Date(),
+        updatedAt: new Date().toISOString(),
       },
     },
     { returnDocument: "after" },
@@ -227,15 +227,15 @@ export async function updateFeedAfterFetch(
     updateData = {
       tier,
       unmodified,
-      nextFetchAt,
-      lastFetchedAt: new Date(),
-      updatedAt: new Date(),
+      nextFetchAt, // Kept as Date for query compatibility
+      lastFetchedAt: new Date().toISOString(),
+      updatedAt: new Date().toISOString(),
     };
   } else {
     updateData = {
       ...extra,
-      lastFetchedAt: new Date(),
-      updatedAt: new Date(),
+      lastFetchedAt: new Date().toISOString(),
+      updatedAt: new Date().toISOString(),
     };
   }
 
@@ -280,7 +280,7 @@ export async function updateFeedWebsub(application, id, websub) {
     {
       $set: {
         websub: websubData,
-        updatedAt: new Date(),
+        updatedAt: new Date().toISOString(),
       },
     },
     { returnDocument: "after" },
@@ -314,12 +314,12 @@ export async function updateFeedStatus(application, id, status) {
   const objectId = typeof id === "string" ? new ObjectId(id) : id;
 
   const updateFields = {
-    updatedAt: new Date(),
+    updatedAt: new Date().toISOString(),
   };
 
   if (status.success) {
     updateFields.status = "active";
-    updateFields.lastSuccessAt = new Date();
+    updateFields.lastSuccessAt = new Date().toISOString();
     updateFields.consecutiveErrors = 0;
     updateFields.lastError = undefined;
     updateFields.lastErrorAt = undefined;
@@ -330,7 +330,7 @@ export async function updateFeedStatus(application, id, status) {
   } else {
     updateFields.status = "error";
     updateFields.lastError = status.error;
-    updateFields.lastErrorAt = new Date();
+    updateFields.lastErrorAt = new Date().toISOString();
   }
 
   // Use $set for most fields, $inc for consecutiveErrors on failure

package/lib/storage/items.js

@@ -49,8 +49,8 @@ export async function addItem(application, { channelId, feedId, uid, item }) {
     name: item.name || undefined,
     content: item.content || undefined,
     summary: item.summary || undefined,
-    published: item.published ? new Date(item.published) : new Date(),
-    updated: item.updated ? new Date(item.updated) : undefined,
+    published: item.published ? new Date(item.published) : new Date(), // Keep as Date for query compatibility
+    updated: item.updated ? new Date(item.updated) : undefined, // Keep as Date for query compatibility
     author: item.author || undefined,
     category: item.category || [],
     photo: item.photo || [],
@@ -62,7 +62,7 @@ export async function addItem(application, { channelId, feedId, uid, item }) {
     inReplyTo: item["in-reply-to"] || item.inReplyTo || [],
     source: item._source || undefined,
     readBy: [], // Array of user IDs who have read this item
-    createdAt: new Date(),
+    createdAt: new Date().toISOString(),
   };
 
   await collection.insertOne(document);
@@ -182,7 +182,7 @@ function transformToJf2(item, userId) {
     type: item.type,
     uid: item.uid,
     url: item.url,
-    published: item.published?.toISOString(),
+    published: item.published?.toISOString(), // Convert Date to ISO string
     _id: item._id.toString(),
     _is_read: userId ? item.readBy?.includes(userId) : false,
   };
@@ -191,7 +191,7 @@ function transformToJf2(item, userId) {
   if (item.name) jf2.name = item.name;
   if (item.content) jf2.content = item.content;
   if (item.summary) jf2.summary = item.summary;
-  if (item.updated) jf2.updated = item.updated.toISOString();
+  if (item.updated) jf2.updated = item.updated.toISOString(); // Convert Date to ISO string
   if (item.author) jf2.author = normalizeAuthor(item.author);
   if (item.category?.length > 0) jf2.category = item.category;
 

package/lib/webmention/processor.js

@@ -61,10 +61,10 @@ export async function processWebmention(application, source, target, userId) {
     url: verification.url,
     published: verification.published
       ? new Date(verification.published)
-      : new Date(),
+      : new Date(), // Keep as Date for query compatibility
     verified: true,
     readBy: [],
-    updatedAt: new Date(),
+    updatedAt: new Date().toISOString(),
   };
 
   if (existing) {
@@ -73,7 +73,7 @@ export async function processWebmention(application, source, target, userId) {
     notification._id = existing._id;
   } else {
     // Insert new notification
-    notification.createdAt = new Date();
+    notification.createdAt = new Date().toISOString();
     await collection.insertOne(notification);
   }
 
@@ -190,7 +190,7 @@ function transformNotification(notification, userId) {
     type: "entry",
     uid: notification._id?.toString(),
     url: notification.url || notification.source,
-    published: notification.published?.toISOString(),
+    published: notification.published?.toISOString(), // Convert Date to ISO string
     author: notification.author,
     content: notification.content,
     _source: notification.source,

package/locales/de.json

@@ -0,0 +1,104 @@
+{
+  "microsub": {
+    "reader": {
+      "title": "Leser",
+      "empty": "Keine Elemente zum Anzeigen",
+      "markAllRead": "Alle als gelesen markieren",
+      "showRead": "Gelesene anzeigen ({{count}})",
+      "hideRead": "Gelesene Elemente ausblenden",
+      "allRead": "Alles aufgeholt!",
+      "newer": "Neuer",
+      "older": "Älter"
+    },
+    "channels": {
+      "title": "Kanäle",
+      "name": "Kanalname",
+      "new": "Neuer Kanal",
+      "create": "Kanal erstellen",
+      "delete": "Kanal löschen",
+      "settings": "Kanaleinstellungen",
+      "empty": "Noch keine Kanäle. Erstellen Sie einen, um zu beginnen.",
+      "notifications": "Benachrichtigungen"
+    },
+    "timeline": {
+      "title": "Zeitleiste",
+      "empty": "Keine Elemente in diesem Kanal",
+      "markRead": "Als gelesen markieren",
+      "markUnread": "Als ungelesen markieren",
+      "remove": "Entfernen"
+    },
+    "feeds": {
+      "title": "Feeds",
+      "follow": "Folgen",
+      "subscribe": "Einen Feed abonnieren",
+      "unfollow": "Entfolgen",
+      "empty": "Keine Feeds in diesem Kanal abonniert",
+      "url": "Feed-URL",
+      "urlPlaceholder": "https://beispiel.de/feed.xml",
+      "edit": "Feed bearbeiten",
+      "rediscover": "Feed neu entdecken",
+      "refresh": "Jetzt aktualisieren",
+      "status": {
+        "active": "Aktiv",
+        "error": "Fehler",
+        "stale": "Veraltet"
+      }
+    },
+    "item": {
+      "reply": "Antworten",
+      "like": "Gefällt mir",
+      "repost": "Teilen",
+      "bookmark": "Lesezeichen",
+      "viewOriginal": "Original anzeigen"
+    },
+    "compose": {
+      "title": "Verfassen",
+      "content": "Was beschäftigt Sie?",
+      "comment": "Kommentar hinzufügen (optional)",
+      "commentHint": "Ihr Kommentar wird bei der Syndizierung mit einbezogen",
+      "syndicateTo": "Syndizieren an",
+      "syndicateHint": "Wählen Sie aus, wo dies quergepostet werden soll",
+      "submit": "Veröffentlichen",
+      "cancel": "Abbrechen",
+      "replyTo": "Antworten an",
+      "likeOf": "Gefällt mir",
+      "repostOf": "Teilen",
+      "bookmarkOf": "Lesezeichen setzen"
+    },
+    "settings": {
+      "title": "{{channel}}-Einstellungen",
+      "excludeTypes": "Interaktionstypen ausschließen",
+      "excludeTypesHelp": "Wählen Sie Beitragstypen aus, die in diesem Kanal ausgeblendet werden sollen",
+      "excludeRegex": "Ausschlussmuster",
+      "excludeRegexHelp": "Regulärer Ausdruck zum Herausfiltern übereinstimmender Inhalte",
+      "save": "Einstellungen speichern",
+      "dangerZone": "Gefahrenzone",
+      "deleteWarning": "Das Löschen dieses Kanals entfernt dauerhaft alle Feeds und Elemente. Diese Aktion kann nicht rückgängig gemacht werden.",
+      "deleteConfirm": "Sind Sie sicher, dass Sie diesen Kanal und alle seine Inhalte löschen möchten?",
+      "delete": "Kanal löschen",
+      "types": {
+        "like": "Gefällt mir",
+        "repost": "Geteilte Beiträge",
+        "bookmark": "Lesezeichen",
+        "reply": "Antworten",
+        "checkin": "Check-ins"
+      }
+    },
+    "search": {
+      "title": "Suchen",
+      "placeholder": "URL oder Suchbegriff eingeben",
+      "submit": "Suchen",
+      "noResults": "Keine Ergebnisse gefunden"
+    },
+    "preview": {
+      "title": "Vorschau",
+      "subscribe": "Diesen Feed abonnieren"
+    },
+    "error": {
+      "channelNotFound": "Kanal nicht gefunden",
+      "feedNotFound": "Feed nicht gefunden",
+      "invalidUrl": "Ungültige URL",
+      "invalidAction": "Ungültige Aktion"
+    }
+  }
+}

package/locales/es-419.json

@@ -0,0 +1,104 @@
+{
+  "microsub": {
+    "reader": {
+      "title": "Lector",
+      "empty": "No hay elementos para mostrar",
+      "markAllRead": "Marcar todo como leído",
+      "showRead": "Mostrar leídos ({{count}})",
+      "hideRead": "Ocultar elementos leídos",
+      "allRead": "¡Todo al día!",
+      "newer": "Más reciente",
+      "older": "Más antiguo"
+    },
+    "channels": {
+      "title": "Canales",
+      "name": "Nombre del canal",
+      "new": "Nuevo canal",
+      "create": "Crear canal",
+      "delete": "Eliminar canal",
+      "settings": "Configuración del canal",
+      "empty": "Todavía no hay canales. Creá uno para empezar.",
+      "notifications": "Notificaciones"
+    },
+    "timeline": {
+      "title": "Línea de tiempo",
+      "empty": "No hay elementos en este canal",
+      "markRead": "Marcar como leído",
+      "markUnread": "Marcar como no leído",
+      "remove": "Eliminar"
+    },
+    "feeds": {
+      "title": "Feeds",
+      "follow": "Seguir",
+      "subscribe": "Suscribirse a un feed",
+      "unfollow": "Dejar de seguir",
+      "empty": "No se sigue ningún feed en este canal",
+      "url": "URL del feed",
+      "urlPlaceholder": "https://ejemplo.com/feed.xml",
+      "edit": "Editar feed",
+      "rediscover": "Redescubrir feed",
+      "refresh": "Actualizar ahora",
+      "status": {
+        "active": "Activo",
+        "error": "Error",
+        "stale": "Obsoleto"
+      }
+    },
+    "item": {
+      "reply": "Responder",
+      "like": "Me gusta",
+      "repost": "Repostear",
+      "bookmark": "Marcador",
+      "viewOriginal": "Ver original"
+    },
+    "compose": {
+      "title": "Redactar",
+      "content": "¿Qué estás pensando?",
+      "comment": "Agregar un comentario (opcional)",
+      "commentHint": "Tu comentario se incluirá cuando esto se sindique",
+      "syndicateTo": "Sindicar a",
+      "syndicateHint": "Seleccioná dónde publicar esto",
+      "submit": "Publicar",
+      "cancel": "Cancelar",
+      "replyTo": "Respondiendo a",
+      "likeOf": "Me gusta",
+      "repostOf": "Reposteando",
+      "bookmarkOf": "Marcando"
+    },
+    "settings": {
+      "title": "Configuración de {{channel}}",
+      "excludeTypes": "Excluir tipos de interacción",
+      "excludeTypesHelp": "Seleccioná los tipos de publicaciones que querés ocultar de este canal",
+      "excludeRegex": "Patrón de exclusión",
+      "excludeRegexHelp": "Expresión regular para filtrar contenido coincidente",
+      "save": "Guardar configuración",
+      "dangerZone": "Zona de peligro",
+      "deleteWarning": "Eliminar este canal eliminará permanentemente todos los feeds y elementos. Esta acción no se puede deshacer.",
+      "deleteConfirm": "¿Estás seguro de que querés eliminar este canal y todo su contenido?",
+      "delete": "Eliminar canal",
+      "types": {
+        "like": "Me gusta",
+        "repost": "Reposteos",
+        "bookmark": "Marcadores",
+        "reply": "Respuestas",
+        "checkin": "Registros"
+      }
+    },
+    "search": {
+      "title": "Buscar",
+      "placeholder": "Ingresá URL o término de búsqueda",
+      "submit": "Buscar",
+      "noResults": "No se encontraron resultados"
+    },
+    "preview": {
+      "title": "Vista previa",
+      "subscribe": "Suscribirse a este feed"
+    },
+    "error": {
+      "channelNotFound": "Canal no encontrado",
+      "feedNotFound": "Feed no encontrado",
+      "invalidUrl": "URL no válida",
+      "invalidAction": "Acción no válida"
+    }
+  }
+}