@rmdes/indiekit-endpoint-activitypub 2.0.26 → 2.0.28

package/README.md

@@ -268,11 +268,66 @@ The JF2-to-ActivityStreams converter handles these Indiekit post types:
 
 Categories are converted to `Hashtag` tags. Bookmarks include a bookmark emoji and link.
 
+## Fedify Workarounds and Implementation Notes
+
+This plugin uses [Fedify](https://fedify.dev) 2.0 but carries several workarounds for issues in Fedify or its Express integration. These are documented here so they can be revisited when Fedify upgrades.
+
+### Custom Express Bridge (instead of `@fedify/express`)
+
+**File:** `lib/federation-bridge.js`
+**Upstream issue:** `@fedify/express` uses `req.url` ([source](https://github.com/fedify-dev/fedify/blob/main/packages/express/src/index.ts), line 73), not `req.originalUrl`.
+
+Indiekit plugins mount at a sub-path (e.g. `/activitypub`). Express strips the mount prefix from `req.url`, so Fedify's URI template matching breaks — WebFinger, actor endpoints, and inbox all return 404. The custom bridge uses `req.originalUrl` to preserve the full path.
+
+The bridge also reconstructs POST bodies that Express's body parser has already consumed (`req.readable === false`). Without this, Fedify handlers like the `@fedify/debugger` login form receive empty bodies.
+
+**Revisit when:** `@fedify/express` switches to `req.originalUrl`, or provides an option to pass a custom URL builder.
+
+### JSON-LD Attachment Array Compaction
+
+**File:** `lib/federation-bridge.js` (in `sendFedifyResponse()`)
+**Upstream issue:** JSON-LD compaction collapses single-element arrays to plain objects.
+
+Mastodon's `update_account_fields` checks `attachment.is_a?(Array)` and silently skips profile links (PropertyValues) when `attachment` is a plain object instead of an array. The bridge intercepts actor JSON-LD responses and forces `attachment` to always be an array.
+
+**Revisit when:** Fedify adds an option to preserve arrays during JSON-LD serialization, or Mastodon fixes their array check.
+
+### `.authorize()` Not Chained on Actor Dispatcher
+
+**File:** `lib/federation-setup.js` (line ~254)
+**Upstream issue:** No authenticated document loading for outgoing key fetches during signature verification.
+
+Fedify's `.authorize()` predicate triggers HTTP Signature verification on every GET to the actor endpoint. When a remote server that requires Authorized Fetch (e.g. kobolds.online) requests our actor, Fedify tries to fetch *their* public key to verify the signature. Those servers return 401 on unsigned GETs, causing uncaught `FetchError` and 500 responses.
+
+This means we do **not** enforce Authorized Fetch on our actor endpoint. Any server can read our actor document without signing the request.
+
+**Revisit when:** Fedify supports using the instance actor's keys for outgoing document fetches during signature verification (i.e. authenticated document loading in the verification path, not just in inbox handlers).
+
+### `importSpkiPem()` / `importPkcs8Pem()` — Local PEM Import
+
+**File:** `lib/federation-setup.js` (lines ~784–816)
+**Upstream change:** Fedify 1.x exported `importSpki()` for loading PEM public keys. This was removed in Fedify 2.0.
+
+The plugin carries local `importSpkiPem()` and `importPkcs8Pem()` functions that use the Web Crypto API directly (`crypto.subtle.importKey`) to load legacy RSA key pairs stored in MongoDB from the Fedify 1.x era. New key pairs are generated using Fedify 2.0's `generateCryptoKeyPair()` and stored as JWK, so these functions only matter for existing installations that migrated from Fedify 1.x.
+
+**Revisit when:** All existing installations have been migrated to JWK-stored keys, or Fedify re-exports a PEM import utility.
+
+### Authenticated Document Loader for Inbox Handlers
+
+**File:** `lib/inbox-listeners.js`
+**Upstream behavior:** Fedify's personal inbox handlers do not automatically use authenticated (signed) HTTP fetches.
+
+All `.getActor()`, `.getObject()`, and `.getTarget()` calls in inbox handlers must explicitly pass an authenticated `DocumentLoader` obtained via `ctx.getDocumentLoader({ identifier: handle })`. Without this, fetches to Authorized Fetch (Secure Mode) servers like hachyderm.io fail with 401, causing timeline items to show "Unknown" authors and missing content.
+
+This is not a bug — Fedify requires explicit opt-in for signed fetches. But it's a pattern that every inbox handler must follow, and forgetting it silently degrades functionality.
+
+**Revisit when:** Fedify provides an option to default to authenticated fetches in inbox handler context, or adds a middleware layer that handles this automatically.
+
 ## Known Limitations
 
 - **No automated tests** — Manual testing against real fediverse servers
 - **Single actor** — One fediverse identity per Indiekit instance
-- **No Authorized Fetch enforcement** — Disabled due to Fedify's current limitation with authenticated outgoing fetches (causes infinite loops with servers that require it)
+- **No Authorized Fetch enforcement** — `.authorize()` disabled on actor dispatcher (see workarounds above)
 - **No image upload in reader** — Compose form is text-only
 - **In-process queue without Redis** — Activities may be lost on restart
 

package/assets/reader-infinite-scroll.js

@@ -0,0 +1,183 @@
+/**
+ * Infinite scroll — AlpineJS component for AJAX load-more on the timeline
+ * Registers the `apInfiniteScroll` Alpine data component.
+ */
+
+document.addEventListener("alpine:init", () => {
+  // eslint-disable-next-line no-undef
+  Alpine.data("apExploreScroll", () => ({
+    loading: false,
+    done: false,
+    maxId: null,
+    instance: "",
+    scope: "local",
+    observer: null,
+
+    init() {
+      const el = this.$el;
+      this.maxId = el.dataset.maxId || null;
+      this.instance = el.dataset.instance || "";
+      this.scope = el.dataset.scope || "local";
+
+      if (!this.maxId) {
+        this.done = true;
+        return;
+      }
+
+      this.observer = new IntersectionObserver(
+        (entries) => {
+          for (const entry of entries) {
+            if (entry.isIntersecting && !this.loading && !this.done) {
+              this.loadMore();
+            }
+          }
+        },
+        { rootMargin: "200px" }
+      );
+
+      if (this.$refs.sentinel) {
+        this.observer.observe(this.$refs.sentinel);
+      }
+    },
+
+    async loadMore() {
+      if (this.loading || this.done || !this.maxId) return;
+
+      this.loading = true;
+
+      const timeline = document.getElementById("ap-explore-timeline");
+      const mountPath = timeline ? timeline.dataset.mountPath : "";
+
+      const params = new URLSearchParams({
+        instance: this.instance,
+        scope: this.scope,
+        max_id: this.maxId,
+      });
+
+      try {
+        const res = await fetch(
+          `${mountPath}/admin/reader/api/explore?${params.toString()}`,
+          { headers: { Accept: "application/json" } }
+        );
+
+        if (!res.ok) throw new Error(`HTTP ${res.status}`);
+
+        const data = await res.json();
+
+        if (data.html && timeline) {
+          timeline.insertAdjacentHTML("beforeend", data.html);
+        }
+
+        if (data.maxId) {
+          this.maxId = data.maxId;
+        } else {
+          this.done = true;
+          if (this.observer) this.observer.disconnect();
+        }
+      } catch (err) {
+        console.error("[ap-explore-scroll] load failed:", err.message);
+      } finally {
+        this.loading = false;
+      }
+    },
+
+    destroy() {
+      if (this.observer) this.observer.disconnect();
+    },
+  }));
+
+  // eslint-disable-next-line no-undef
+  Alpine.data("apInfiniteScroll", () => ({
+    loading: false,
+    done: false,
+    before: null,
+    tab: "",
+    tag: "",
+    observer: null,
+
+    init() {
+      const el = this.$el;
+      this.before = el.dataset.before || null;
+      this.tab = el.dataset.tab || "";
+      this.tag = el.dataset.tag || "";
+
+      // Hide the no-JS pagination fallback now that JS is active
+      const paginationEl =
+        document.getElementById("ap-reader-pagination") ||
+        document.getElementById("ap-tag-pagination");
+      if (paginationEl) {
+        paginationEl.style.display = "none";
+      }
+
+      if (!this.before) {
+        this.done = true;
+        return;
+      }
+
+      // Set up IntersectionObserver to auto-load when sentinel comes into view
+      this.observer = new IntersectionObserver(
+        (entries) => {
+          for (const entry of entries) {
+            if (entry.isIntersecting && !this.loading && !this.done) {
+              this.loadMore();
+            }
+          }
+        },
+        { rootMargin: "200px" }
+      );
+
+      if (this.$refs.sentinel) {
+        this.observer.observe(this.$refs.sentinel);
+      }
+    },
+
+    async loadMore() {
+      if (this.loading || this.done || !this.before) return;
+
+      this.loading = true;
+
+      const timeline = document.getElementById("ap-timeline");
+      const mountPath = timeline ? timeline.dataset.mountPath : "";
+
+      const params = new URLSearchParams({ before: this.before });
+      if (this.tab) params.set("tab", this.tab);
+      if (this.tag) params.set("tag", this.tag);
+
+      try {
+        const res = await fetch(
+          `${mountPath}/admin/reader/api/timeline?${params.toString()}`,
+          { headers: { Accept: "application/json" } }
+        );
+
+        if (!res.ok) throw new Error(`HTTP ${res.status}`);
+
+        const data = await res.json();
+
+        if (data.html && timeline) {
+          // Append the returned pre-rendered HTML
+          timeline.insertAdjacentHTML("beforeend", data.html);
+        }
+
+        if (data.before) {
+          this.before = data.before;
+        } else {
+          // No more items
+          this.done = true;
+          if (this.observer) this.observer.disconnect();
+        }
+      } catch (err) {
+        console.error("[ap-infinite-scroll] load failed:", err.message);
+      } finally {
+        this.loading = false;
+      }
+    },
+
+    appendItems(/* detail */) {
+      // Custom event hook — not used in this implementation but kept for extensibility
+    },
+
+    destroy() {
+      if (this.observer) this.observer.disconnect();
+    },
+  }));
+});

package/assets/reader.css

@@ -655,6 +655,25 @@
   color: var(--color-on-background);
 }
 
+.ap-card__mention {
+  background: color-mix(in srgb, var(--color-accent) 12%, transparent);
+  border-radius: var(--border-radius-large);
+  color: var(--color-accent);
+  font-size: var(--font-size-s);
+  padding: 2px var(--space-xs);
+  text-decoration: none;
+}
+
+.ap-card__mention:hover {
+  background: color-mix(in srgb, var(--color-accent) 22%, transparent);
+  color: var(--color-accent);
+}
+
+.ap-card__mention--legacy {
+  cursor: default;
+  opacity: 0.7;
+}
+
 /* ==========================================================================
    Interaction Buttons
    ========================================================================== */
@@ -735,6 +754,55 @@
   text-decoration: underline;
 }
 
+/* Hidden once Alpine is active (JS replaces with infinite scroll) */
+.ap-pagination--js-hidden {
+  /* Shown by default for no-JS fallback — Alpine hides via display:none */
+}
+
+/* ==========================================================================
+   Infinite Scroll / Load More
+   ========================================================================== */
+
+.ap-load-more {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  gap: var(--space-s);
+  padding: var(--space-m) 0;
+}
+
+.ap-load-more__sentinel {
+  height: 1px;
+  width: 100%;
+}
+
+.ap-load-more__btn {
+  background: var(--color-offset);
+  border: var(--border-width-thin) solid var(--color-outline);
+  border-radius: var(--border-radius-small);
+  color: var(--color-on-background);
+  cursor: pointer;
+  font-size: var(--font-size-s);
+  padding: var(--space-xs) var(--space-m);
+  transition: background 0.15s;
+}
+
+.ap-load-more__btn:hover:not(:disabled) {
+  background: var(--color-offset-variant);
+}
+
+.ap-load-more__btn:disabled {
+  cursor: wait;
+  opacity: 0.6;
+}
+
+.ap-load-more__done {
+  color: var(--color-on-offset);
+  font-size: var(--font-size-s);
+  margin: 0;
+  text-align: center;
+}
+
 /* ==========================================================================
    Compose Form
    ========================================================================== */
@@ -1572,6 +1640,204 @@
   box-shadow: 0 0 0 1px var(--color-primary);
 }
 
+/* ==========================================================================
+   Tag Timeline Header
+   ========================================================================== */
+
+.ap-tag-header {
+  align-items: flex-start;
+  background: var(--color-offset);
+  border-bottom: var(--border-width-thin) solid var(--color-outline);
+  border-radius: var(--border-radius-small);
+  display: flex;
+  gap: var(--space-m);
+  justify-content: space-between;
+  margin-bottom: var(--space-m);
+  padding: var(--space-m);
+}
+
+.ap-tag-header__title {
+  font-size: var(--font-size-xl);
+  font-weight: var(--font-weight-bold);
+  margin: 0 0 var(--space-xs);
+}
+
+.ap-tag-header__count {
+  color: var(--color-on-offset);
+  font-size: var(--font-size-s);
+  margin: 0;
+}
+
+.ap-tag-header__actions {
+  align-items: center;
+  display: flex;
+  flex-shrink: 0;
+  gap: var(--space-s);
+}
+
+.ap-tag-header__follow-btn {
+  background: var(--color-accent);
+  border: none;
+  border-radius: var(--border-radius-small);
+  color: var(--color-on-accent);
+  cursor: pointer;
+  font-size: var(--font-size-s);
+  padding: var(--space-xs) var(--space-s);
+}
+
+.ap-tag-header__follow-btn:hover {
+  opacity: 0.85;
+}
+
+.ap-tag-header__unfollow-btn {
+  background: transparent;
+  border: var(--border-width-thin) solid var(--color-outline);
+  border-radius: var(--border-radius-small);
+  color: var(--color-on-background);
+  cursor: pointer;
+  font-size: var(--font-size-s);
+  padding: var(--space-xs) var(--space-s);
+}
+
+.ap-tag-header__unfollow-btn:hover {
+  border-color: var(--color-on-background);
+}
+
+.ap-tag-header__back {
+  color: var(--color-on-offset);
+  font-size: var(--font-size-s);
+  text-decoration: none;
+}
+
+.ap-tag-header__back:hover {
+  color: var(--color-on-background);
+  text-decoration: underline;
+}
+
+@media (max-width: 640px) {
+  .ap-tag-header {
+    flex-direction: column;
+    gap: var(--space-s);
+  }
+
+  .ap-tag-header__actions {
+    flex-wrap: wrap;
+  }
+}
+
+/* ==========================================================================
+   Reader Tools Bar (Explore link, etc.)
+   ========================================================================== */
+
+.ap-reader-tools {
+  display: flex;
+  gap: var(--space-s);
+  justify-content: flex-end;
+  margin-bottom: var(--space-s);
+}
+
+.ap-reader-tools__explore {
+  color: var(--color-on-offset);
+  font-size: var(--font-size-s);
+  text-decoration: none;
+}
+
+.ap-reader-tools__explore:hover {
+  color: var(--color-on-background);
+  text-decoration: underline;
+}
+
+/* ==========================================================================
+   Explore Page
+   ========================================================================== */
+
+.ap-explore-header {
+  margin-bottom: var(--space-m);
+}
+
+.ap-explore-header__title {
+  font-size: var(--font-size-xl);
+  margin: 0 0 var(--space-xs);
+}
+
+.ap-explore-header__desc {
+  color: var(--color-on-offset);
+  font-size: var(--font-size-s);
+  margin: 0;
+}
+
+.ap-explore-form {
+  background: var(--color-offset);
+  border: var(--border-width-thin) solid var(--color-outline);
+  border-radius: var(--border-radius-small);
+  margin-bottom: var(--space-m);
+  padding: var(--space-m);
+}
+
+.ap-explore-form__row {
+  align-items: center;
+  display: flex;
+  gap: var(--space-s);
+  flex-wrap: wrap;
+}
+
+.ap-explore-form__input {
+  border: var(--border-width-thin) solid var(--color-outline);
+  border-radius: var(--border-radius-small);
+  flex: 1;
+  font-size: var(--font-size-base);
+  min-width: 0;
+  padding: var(--space-xs) var(--space-s);
+}
+
+.ap-explore-form__scope {
+  display: flex;
+  gap: var(--space-s);
+}
+
+.ap-explore-form__scope-label {
+  align-items: center;
+  cursor: pointer;
+  display: flex;
+  font-size: var(--font-size-s);
+  gap: var(--space-xs);
+}
+
+.ap-explore-form__btn {
+  background: var(--color-primary);
+  border: none;
+  border-radius: var(--border-radius-small);
+  color: var(--color-on-primary);
+  cursor: pointer;
+  font-size: var(--font-size-s);
+  padding: var(--space-xs) var(--space-m);
+  white-space: nowrap;
+}
+
+.ap-explore-form__btn:hover {
+  opacity: 0.85;
+}
+
+.ap-explore-error {
+  background: color-mix(in srgb, var(--color-red45) 10%, transparent);
+  border: var(--border-width-thin) solid var(--color-red45);
+  border-radius: var(--border-radius-small);
+  color: var(--color-red45);
+  margin-bottom: var(--space-m);
+  padding: var(--space-s) var(--space-m);
+}
+
+@media (max-width: 640px) {
+  .ap-explore-form__row {
+    flex-direction: column;
+    align-items: stretch;
+  }
+
+  .ap-explore-form__btn {
+    width: 100%;
+  }
+}
+
 /* Replies — indented from the other side */
 .ap-post-detail__replies {
   margin-left: var(--space-l);
@@ -1582,3 +1848,19 @@
   padding-left: var(--space-m);
   margin-bottom: var(--space-xs);
 }
+
+/* Followed tags bar */
+.ap-followed-tags {
+  display: flex;
+  flex-wrap: wrap;
+  align-items: center;
+  gap: var(--space-xs);
+  padding: var(--space-xs) 0;
+  margin-bottom: var(--space-s);
+  font-size: var(--font-size-s);
+}
+
+.ap-followed-tags__label {
+  color: var(--color-on-offset);
+  font-weight: 600;
+}

package/index.js

@@ -59,6 +59,10 @@ import {
   featuredTagsRemoveController,
 } from "./lib/controllers/featured-tags.js";
 import { resolveController } from "./lib/controllers/resolve.js";
+import { tagTimelineController } from "./lib/controllers/tag-timeline.js";
+import { apiTimelineController } from "./lib/controllers/api-timeline.js";
+import { exploreController, exploreApiController } from "./lib/controllers/explore.js";
+import { followTagController, unfollowTagController } from "./lib/controllers/follow-tag.js";
 import { publicProfileController } from "./lib/controllers/public-profile.js";
 import { authorizeInteractionController } from "./lib/controllers/authorize-interaction.js";
 import { myProfileController } from "./lib/controllers/my-profile.js";
@@ -71,6 +75,7 @@ import {
 import { startBatchRefollow } from "./lib/batch-refollow.js";
 import { logActivity } from "./lib/activity-log.js";
 import { scheduleCleanup } from "./lib/timeline-cleanup.js";
+import { runSeparateMentionsMigration } from "./lib/migrations/separate-mentions.js";
 
 const defaults = {
   mountPath: "/activitypub",
@@ -218,6 +223,12 @@ export default class ActivityPubEndpoint {
 
     router.get("/", dashboardController(mp));
     router.get("/admin/reader", readerController(mp));
+    router.get("/admin/reader/tag", tagTimelineController(mp));
+    router.get("/admin/reader/api/timeline", apiTimelineController(mp));
+    router.get("/admin/reader/explore", exploreController(mp));
+    router.get("/admin/reader/api/explore", exploreApiController(mp));
+    router.post("/admin/reader/follow-tag", followTagController(mp));
+    router.post("/admin/reader/unfollow-tag", unfollowTagController(mp));
     router.get("/admin/reader/notifications", notificationsController(mp));
     router.post("/admin/reader/notifications/mark-read", markAllNotificationsReadController(mp));
     router.post("/admin/reader/notifications/clear", clearAllNotificationsController(mp));
@@ -855,6 +866,7 @@ export default class ActivityPubEndpoint {
     Indiekit.addCollection("ap_blocked");
     Indiekit.addCollection("ap_interactions");
     Indiekit.addCollection("ap_notes");
+    Indiekit.addCollection("ap_followed_tags");
 
     // Store collection references (posts resolved lazily)
     const indiekitCollections = Indiekit.collections;
@@ -874,6 +886,7 @@ export default class ActivityPubEndpoint {
       ap_blocked: indiekitCollections.get("ap_blocked"),
       ap_interactions: indiekitCollections.get("ap_interactions"),
       ap_notes: indiekitCollections.get("ap_notes"),
+      ap_followed_tags: indiekitCollections.get("ap_followed_tags"),
       get posts() {
         return indiekitCollections.get("posts");
       },
@@ -985,6 +998,18 @@ export default class ActivityPubEndpoint {
         { type: 1 },
         { background: true },
       );
+
+      // Followed hashtags — unique on tag (case-insensitive via normalization at write time)
+      this._collections.ap_followed_tags.createIndex(
+        { tag: 1 },
+        { unique: true, background: true },
+      );
+
+      // Tag filtering index on timeline
+      this._collections.ap_timeline.createIndex(
+        { category: 1, published: -1 },
+        { background: true },
+      );
     } catch {
       // Index creation failed — collections not yet available.
       // Indexes already exist from previous startups; non-fatal.
@@ -1039,6 +1064,15 @@ export default class ActivityPubEndpoint {
       });
     }, 10_000);
 
+    // Run one-time migrations (idempotent — safe to run on every startup)
+    runSeparateMentionsMigration(this._collections).then(({ skipped, updated }) => {
+      if (!skipped) {
+        console.log(`[ActivityPub] Migration separate-mentions: updated ${updated} timeline items`);
+      }
+    }).catch((error) => {
+      console.error("[ActivityPub] Migration separate-mentions failed:", error.message);
+    });
+
     // Schedule timeline retention cleanup (runs on startup + every 24h)
     if (this.options.timelineRetention > 0) {
       scheduleCleanup(this._collections, this.options.timelineRetention);