fedipod 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -7
- package/bin/fedipod.mjs +4 -4
- package/groups.md +3 -3
- package/gui.md +28 -2
- package/lib/acctfeed.mjs +257 -0
- package/lib/admin.mjs +145 -3
- package/lib/bskygroup.mjs +1 -1
- package/lib/fediacct.mjs +257 -0
- package/lib/mastoapi.mjs +101 -2
- package/lib/store.mjs +26 -1
- package/lib/wire.mjs +2 -2
- package/package.json +1 -1
- package/phanpy/dist/assets/main-BdqNbG-a.js +1 -1
- package/phanpy/dist/compose/index.html +1 -1
- package/phanpy/dist/index.html +1 -1
- package/phanpy/dist/sw.js +2 -25
- package/phanpy/dist/sw.js.map +1 -0
- package/run-agent.mjs +31 -0
- package/scripts/verify-ui.mjs +7 -4
- package/web/admin/admin.js +81 -22
- package/web/admin/index.html +37 -8
package/README.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
# FediPod
|
|
2
2
|
|
|
3
|
-
- access the
|
|
3
|
+
- access the Fediverse from a Solid pod
|
|
4
4
|
|
|
5
5
|
`FediPod` is a merger of three federating protocols - feeds from Mastodon and other Fediverse servers and from Bluesky and other ATProto servers, with all user data stored on a Solid pod. There are four different kinds of FediPod accounts :
|
|
6
6
|
|
|
7
7
|
| Kind of Account | Description | User requirements | Host requirements |
|
|
8
8
|
|---|---|---|---|
|
|
9
|
-
| FediPod Solo | Browse & interact with the
|
|
10
|
-
| FediPod Group | Host a discussion group of
|
|
9
|
+
| FediPod Solo | Browse & interact with the Fediverse and ATProto from a Solid pod | an always-on pod, a usually on local agent | — |
|
|
10
|
+
| FediPod Group | Host a discussion group of Fediverse/ATProto users from a Solid pod | an always-on pod, a usually on local agent | — |
|
|
11
11
|
| FediPod Gateway | Solo or Group with spam filtering & optional community identity | same + gateway pass-through account | any always-on host, Netlify will do |
|
|
12
12
|
| FediPod Server | Full ActivityPub server as a CSS component | a pod on a CSS server implementing the FediPod component | a CSS server implementing the FediPod component |
|
|
13
13
|
|
|
@@ -51,7 +51,7 @@ does the same from the terminal. `AP_UPDATE_CHECK=0` turns the check off.
|
|
|
51
51
|
|
|
52
52
|
## A FediPod.net Gateway account (reccommended)
|
|
53
53
|
|
|
54
|
-
Most of what a
|
|
54
|
+
Most of what a Fediverse inbox receives is broadcast noise. A gateway is a
|
|
55
55
|
shared, always-on door that verifies each delivery, drops the junk, and
|
|
56
56
|
passes the rest to your pod — while your key and data stay on your own
|
|
57
57
|
pod. There is a free gateway at [fedipod.net](https://fedipod.net/). Attaching to it or detaching from it is a few wizard-guided clicks.
|
|
@@ -61,7 +61,7 @@ Users are recommended to associate with a gateway as it reduces the mail load on
|
|
|
61
61
|
## Fediverse Clients
|
|
62
62
|
|
|
63
63
|
The UI is the bundled [Phanpy](https://github.com/cheeaun/phanpy)
|
|
64
|
-
client (MIT, by Chee Aun). It is served same-origin over the agent's Mastodon client-API facade
|
|
64
|
+
client (MIT, by Chee Aun). It is served same-origin over the agent's Mastodon client-API facade, and logging in is one click — the instance is filled in for you. If you ever enter it by hand, use the address on the record's **local host** row.
|
|
65
65
|
|
|
66
66
|
The agent federates for real: follow/unfollow, post, reply, favourite,
|
|
67
67
|
boost, media, delete; incoming boosts from people you follow and a
|
|
@@ -85,7 +85,12 @@ notifications that reach you while the client is closed.
|
|
|
85
85
|
|
|
86
86
|
## Bluesky and ATProto
|
|
87
87
|
|
|
88
|
-
`FediPod` includes experimental support for ATProto. A Bluesky connection lets your `FediPod` agent drive an existing Bluesky (or other ATProto) account alongside your
|
|
88
|
+
`FediPod` includes experimental support for ATProto. A Bluesky connection lets your `FediPod` agent drive an existing Bluesky (or other ATProto) account alongside your Fediverse identity. When you post a public post from `FediPod`, it will be cross-posted to Bluesky as a mirror (a toggle you can turn off). Private DMs from `FediPod` to Bluesky are not currently supported. Bluesky replies and activity flow into your timeline so you see, for example, a combined Bluesky and Mastodon feed. You can like, boost, and reply to Bluesky posts from within `FediPod`. None of these features require `Bridgy Fed`, however, fully joining a `FediPod` group from a Bluesky client other than `FediPod` does require a bridge.
|
|
89
|
+
|
|
90
|
+
## Your other Fediverse accounts
|
|
91
|
+
|
|
92
|
+
If you also hold an account on Mastodon — or on any other server speaking the Mastodon API, such as GoToSocial, Pleroma, Akkoma, Pixelfed or Friendica — you can connect it and read it here. Its home timeline and notifications join the feed you already read, interleaved by time, and a post both accounts see appears once rather than twice. Favouriting, boosting and replying act as the account the post came through, on its own server. Connect one from the **Other identities** row of the admin page: you sign in at that server, and the token it hands back stays on this machine.
|
|
93
|
+
|
|
89
94
|
|
|
90
95
|
## Architecture
|
|
91
96
|
|
|
@@ -112,7 +117,7 @@ as owner-only documents. See [Groups](https://github.com/jeff-zucker/FediPod/blo
|
|
|
112
117
|
|
|
113
118
|
## Acknowledgements
|
|
114
119
|
|
|
115
|
-
This project is inspired by the fantastic [ActivityPods project](https://github.com/activitypods) and is meant to be a lightweight alternative rather than a replacement. Thanks to [Sébastien](https://github.com/srosset81) and collaborators for all your work. Thanks to [Damon](https://github.com/outlaw-dame), [Mikhal](https://github.com/mrkvon), [Alain](https://github.com/bourgeoa), [Sharon](https://github.com/SharonStrats),
|
|
120
|
+
This project is inspired by the fantastic [ActivityPods project](https://github.com/activitypods) and is meant to be a lightweight alternative rather than a replacement. Thanks to [Sébastien](https://github.com/srosset81) and collaborators for all your work. Thanks to [Damon](https://github.com/outlaw-dame), [Mikhal](https://github.com/mrkvon), [Alain](https://github.com/bourgeoa), and [Sharon](https://github.com/SharonStrats), for testing and encouragement. Special thanks are due to [Joseph](https://github.com/jg10-mastodon-social) whose client-to-server authentication approach and the netlify/fronted-identity ideas I borrowed from [solid-activitypub-netlify](https://github.com/jg10-mastodon-social/solid-activitypub-netlify) and to [Vincent](https://github.com/Vinnl) and [Emilia](https://github.com/ThisIsMissEm) whose ideas on the multiple Fediverse accounts sparked FediPod's support for it.
|
|
116
121
|
|
|
117
122
|
## Transparency
|
|
118
123
|
|
package/bin/fedipod.mjs
CHANGED
|
@@ -505,7 +505,7 @@ if (cmd === 'up') {
|
|
|
505
505
|
if (/^y/i.test(have)) {
|
|
506
506
|
// Either tuck the fediverse account into the pod they already have,
|
|
507
507
|
// or make a fresh pod on the same Solid account for it.
|
|
508
|
-
const where = await ask('store your
|
|
508
|
+
const where = await ask('store your Fediverse account in that pod, or in a new pod? (existing/new)', 'existing');
|
|
509
509
|
if (/^n/i.test(where)) newAccount = true;
|
|
510
510
|
else pod = await ask('your pod address (e.g. https://you.solidcommunity.net/)');
|
|
511
511
|
} else {
|
|
@@ -575,8 +575,8 @@ if (cmd === 'up') {
|
|
|
575
575
|
}
|
|
576
576
|
console.log('The display name can be changed later; the handle and pod cannot.');
|
|
577
577
|
const go = await ask(newAccount
|
|
578
|
-
? (kind === 'group' ? 'create pod and group? (y/n)' : 'create pod and
|
|
579
|
-
: (kind === 'group' ? 'create group on this pod? (y/n)' : 'create
|
|
578
|
+
? (kind === 'group' ? 'create pod and group? (y/n)' : 'create pod and Fediverse account? (y/n)')
|
|
579
|
+
: (kind === 'group' ? 'create group on this pod? (y/n)' : 'create Fediverse account on this pod? (y/n)'), 'y');
|
|
580
580
|
if (!/^y/i.test(go)) { console.log('nothing was created'); process.exit(0); }
|
|
581
581
|
endAsking(); // hand the tty to the password prompt
|
|
582
582
|
|
|
@@ -589,7 +589,7 @@ if (cmd === 'up') {
|
|
|
589
589
|
console.log(`account + pod created: ${pod}`);
|
|
590
590
|
if (!webfingerHost(pod)) {
|
|
591
591
|
console.log(`\n${issuerHost} created the pod at a path rather than on its own subdomain,`);
|
|
592
|
-
console.log(`so @${handle}@\u2026 cannot be discovered by other
|
|
592
|
+
console.log(`so @${handle}@\u2026 cannot be discovered by other Fediverse servers.`);
|
|
593
593
|
const cont = kind === 'group' ? 'n' : (interactive ? await ask('continue anyway? (y/n)', 'n') : 'y');
|
|
594
594
|
endAsking();
|
|
595
595
|
if (!/^y/i.test(cont)) {
|
package/groups.md
CHANGED
|
@@ -68,15 +68,15 @@ A group can be handed on rather than abandoned using the `Transfer this account
|
|
|
68
68
|
|
|
69
69
|
A group with a connected Bluesky account is joinable from Bluesky: following
|
|
70
70
|
the group's Bluesky account is a join, and mentioning its handle submits a
|
|
71
|
-
post. Both pass through the same moderation settings as
|
|
71
|
+
post. Both pass through the same moderation settings as Fediverse members —
|
|
72
72
|
join review, post review, and muting all apply, and an approved post is
|
|
73
73
|
carried to the group's Bluesky followers as a repost.
|
|
74
74
|
A follow from an already-bridged Bluesky account is ignored on the Bluesky
|
|
75
75
|
side — its join arrives over ActivityPub from the bridge instead. Carrying
|
|
76
|
-
|
|
76
|
+
Fediverse posts to the group's Bluesky followers happens only while
|
|
77
77
|
crossposting is on.
|
|
78
78
|
|
|
79
|
-
A Bluesky member's posts reach the
|
|
79
|
+
A Bluesky member's posts reach the Fediverse side only if their account is
|
|
80
80
|
bridged (it follows @ap.brid.gy on Bluesky). Unbridged members are welcomed
|
|
81
81
|
with a single post mentioning them explaining that, and everything else about their
|
|
82
82
|
membership still works, and bridging later upgrades them in place: the
|
package/gui.md
CHANGED
|
@@ -45,7 +45,7 @@ Each identity has a page anyone can open, at `ap/profile.html` under its pod
|
|
|
45
45
|
shows the name, bio and address, and offers a Follow box: a visitor types
|
|
46
46
|
their own server and lands on that server's follow screen. Hand out that
|
|
47
47
|
link, or the `@name@host` address itself, which works in the search box of
|
|
48
|
-
any
|
|
48
|
+
any Fediverse app.
|
|
49
49
|
|
|
50
50
|
## Bluesky
|
|
51
51
|
|
|
@@ -55,13 +55,39 @@ ones), deleting a post deletes its Bluesky copy, and the account's timeline
|
|
|
55
55
|
and notifications appear in your home feed. Replying to, favouriting and
|
|
56
56
|
boosting a Bluesky post act as the connected account. A reply lives on
|
|
57
57
|
Bluesky only — it is always public and fits Bluesky's 300-character limit —
|
|
58
|
-
because your
|
|
58
|
+
because your Fediverse followers cannot see the post it answers. The
|
|
59
59
|
**crosspost** control stops the
|
|
60
60
|
mirroring without disconnecting. Disconnecting forgets the login but removes
|
|
61
61
|
nothing already posted. For a group, the connected account is the group's
|
|
62
62
|
presence on Bluesky — joining and posting through it follow the group's own
|
|
63
63
|
moderation settings; see [Groups](groups.md).
|
|
64
64
|
|
|
65
|
+
## Your other Fediverse accounts
|
|
66
|
+
|
|
67
|
+
Connecting an account you hold on another server brings its home timeline and
|
|
68
|
+
its notifications into the feed you already read here, interleaved by time.
|
|
69
|
+
Mastodon, GoToSocial, Pleroma, Akkoma, Pixelfed and Friendica all work. You
|
|
70
|
+
sign in at that server rather than here: this agent never sees a password, and
|
|
71
|
+
the token it is given stays on this machine and is never written to your pod.
|
|
72
|
+
|
|
73
|
+
Favouriting, boosting or replying to one of those posts acts **as that
|
|
74
|
+
account**, on its own server, because that is where the conversation is. A
|
|
75
|
+
reply from a connected account is public or unlisted. A post you wrote on that
|
|
76
|
+
account can be deleted from here, and it is deleted where it lives.
|
|
77
|
+
|
|
78
|
+
A post that two of your accounts both see appears once, not twice. The
|
|
79
|
+
favourite and boost controls are lit when any of your accounts has done it, so
|
|
80
|
+
using one again undoes it everywhere rather than adding a second from a second
|
|
81
|
+
account.
|
|
82
|
+
|
|
83
|
+
What arrives is held on this machine for reading. None of it is written to your
|
|
84
|
+
pod and none of it is republished — your own posts still go out from your pod
|
|
85
|
+
account alone.
|
|
86
|
+
|
|
87
|
+
Disconnecting stops the reading and removes the stored token; nothing already
|
|
88
|
+
posted is touched. If the other server stops accepting the token, the account
|
|
89
|
+
is marked **sign in again** rather than quietly dropped.
|
|
90
|
+
|
|
65
91
|
## Handling follow requests
|
|
66
92
|
|
|
67
93
|
New followers appear under **Follow requests** with **Accept** and **Refuse**
|
package/lib/acctfeed.mjs
ADDED
|
@@ -0,0 +1,257 @@
|
|
|
1
|
+
// acctfeed.mjs — polls each connected fediverse account's home timeline and
|
|
2
|
+
// notifications and merges them into the statuses index as kind 'acct'. View
|
|
3
|
+
// cache only: none of it is written to the pod, which holds followed and own
|
|
4
|
+
// content. Modeled on bskyfeed.mjs, which is modeled on tagfeed.mjs.
|
|
5
|
+
//
|
|
6
|
+
// A note is NOT re-fetched at its origin before being stored. The dereference
|
|
7
|
+
// rule guards material an unauthenticated stranger chose for us; this is the
|
|
8
|
+
// owner's own authenticated account answering with the result of the owner's
|
|
9
|
+
// own follow decisions there — the same trust anchor that lets bskyfeed skip
|
|
10
|
+
// it. A local block still applies, because that is the owner's decision.
|
|
11
|
+
//
|
|
12
|
+
// Config in fediacctfeed.json: { intervalMin, accounts: { <id>: marks } }.
|
|
13
|
+
|
|
14
|
+
import { sanitizeHtml } from './wire.mjs';
|
|
15
|
+
|
|
16
|
+
const DEFAULTS = { intervalMin: 5 };
|
|
17
|
+
const PER_SWEEP = 40;
|
|
18
|
+
const MAX_PER_ACCT = 150;
|
|
19
|
+
const MAX_ACCT_ENTRIES = 400;
|
|
20
|
+
const BACKOFF_MIN_MS = 5 * 60_000;
|
|
21
|
+
const BACKOFF_MAX_MS = 2 * 60 * 60_000;
|
|
22
|
+
// Mastodon allows 300 requests per five minutes per token. Four go per sweep,
|
|
23
|
+
// so the floor is only ever reached by something else using the same token.
|
|
24
|
+
const RATE_FLOOR = 20;
|
|
25
|
+
|
|
26
|
+
const MEDIA_TYPES = {
|
|
27
|
+
image: 'image/jpeg', gifv: 'image/gif', video: 'video/mp4', audio: 'audio/mpeg',
|
|
28
|
+
};
|
|
29
|
+
|
|
30
|
+
export class AcctFeed {
|
|
31
|
+
constructor({ store, accounts, log = console.log, onNotification = null }) {
|
|
32
|
+
Object.assign(this, { store, accounts, log, onNotification });
|
|
33
|
+
this.state = new Map(); // account id → { failures, quietUntil }
|
|
34
|
+
this.lastSweep = null;
|
|
35
|
+
this.lastAdded = 0;
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
config() { return { ...DEFAULTS, ...this.store.read('fediacctfeed.json', {}) }; }
|
|
39
|
+
|
|
40
|
+
setConfig(patch) {
|
|
41
|
+
const clean = {};
|
|
42
|
+
if (patch.intervalMin) clean.intervalMin = Math.max(1, Number(patch.intervalMin) || DEFAULTS.intervalMin);
|
|
43
|
+
this.store.write('fediacctfeed.json', { ...this.config(), ...clean });
|
|
44
|
+
this.stop();
|
|
45
|
+
this.start();
|
|
46
|
+
return this.config();
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
marks(id) { return this.config().accounts?.[id] || {}; }
|
|
50
|
+
|
|
51
|
+
setMarks(id, patch) {
|
|
52
|
+
const cfg = this.config();
|
|
53
|
+
this.store.write('fediacctfeed.json', {
|
|
54
|
+
...cfg, accounts: { ...(cfg.accounts || {}), [id]: { ...(cfg.accounts?.[id] || {}), ...patch } },
|
|
55
|
+
});
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
start() {
|
|
59
|
+
this.stopped = false;
|
|
60
|
+
this.sweep().catch(e => this.log(`acctfeed: ${e.message}`));
|
|
61
|
+
const tick = () => {
|
|
62
|
+
this.timer = setTimeout(() => {
|
|
63
|
+
this.sweep()
|
|
64
|
+
.catch(e => this.log(`acctfeed: ${e.message}`))
|
|
65
|
+
.finally(() => { if (!this.stopped) tick(); });
|
|
66
|
+
}, Math.round(this.config().intervalMin * 60_000 * (0.85 + Math.random() * 0.3)));
|
|
67
|
+
this.timer.unref?.();
|
|
68
|
+
};
|
|
69
|
+
tick();
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
stop() { this.stopped = true; clearTimeout(this.timer); }
|
|
73
|
+
|
|
74
|
+
// Per account, not per feed: one server being down must not silence the rest.
|
|
75
|
+
_stateOf(id) {
|
|
76
|
+
if (!this.state.has(id)) this.state.set(id, { failures: 0, quietUntil: 0 });
|
|
77
|
+
return this.state.get(id);
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
_backOff(id, handle, status, retryAfter) {
|
|
81
|
+
const st = this._stateOf(id);
|
|
82
|
+
st.failures += 1;
|
|
83
|
+
const ladder = Math.min(BACKOFF_MIN_MS * 2 ** (st.failures - 1), BACKOFF_MAX_MS);
|
|
84
|
+
const wait = retryAfter || Math.round(ladder * (0.85 + Math.random() * 0.3));
|
|
85
|
+
st.quietUntil = Date.now() + wait;
|
|
86
|
+
this.log(`acctfeed: ${handle} ${status ? `answered ${status}` : 'did not answer'} — not asking again for ${Math.round(wait / 60_000)} min`);
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
// One author into the shared actor cache. `uri` is the ActivityPub id, so a
|
|
90
|
+
// person the pod already knows from its own inbox stays one actor here.
|
|
91
|
+
_rememberAuthor(a) {
|
|
92
|
+
const url = a?.uri || a?.url;
|
|
93
|
+
if (!url) return null;
|
|
94
|
+
if (!this.store.getActors()[url]) {
|
|
95
|
+
this.store.cacheActor(url, {
|
|
96
|
+
name: a.display_name || a.username, preferredUsername: a.username,
|
|
97
|
+
icon: a.avatar || null, type: a.bot ? 'Service' : 'Person',
|
|
98
|
+
});
|
|
99
|
+
}
|
|
100
|
+
return url;
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
// One Mastodon status into the statuses index. Never pre-checks for a
|
|
104
|
+
// duplicate: a repeat sighting is what carries the second account's
|
|
105
|
+
// provenance, and addStatus is the only thing that knows how to merge it.
|
|
106
|
+
_mirror(st, { acct, via = null, parents = null }) {
|
|
107
|
+
const noteId = st.uri;
|
|
108
|
+
if (!noteId) return false;
|
|
109
|
+
const actor = this._rememberAuthor(st.account);
|
|
110
|
+
if (!actor || this.store.isBlocked(actor)) return false;
|
|
111
|
+
const inReplyTo = st.in_reply_to_id ? parents?.get(String(st.in_reply_to_id)) : null;
|
|
112
|
+
const out = this.store.addStatus({
|
|
113
|
+
noteId, actor,
|
|
114
|
+
content: sanitizeHtml(st.content || ''),
|
|
115
|
+
published: st.created_at,
|
|
116
|
+
kind: 'acct',
|
|
117
|
+
sourceAccts: [{ acct, remoteId: String(st.id) }],
|
|
118
|
+
...(st.url && st.url !== noteId ? { link: st.url } : {}),
|
|
119
|
+
...(inReplyTo ? { inReplyTo } : {}),
|
|
120
|
+
...(via ? { via } : {}),
|
|
121
|
+
...(st.spoiler_text ? { spoiler: st.spoiler_text } : {}),
|
|
122
|
+
...(st.media_attachments?.length ? {
|
|
123
|
+
attachments: st.media_attachments.map(m => ({
|
|
124
|
+
url: m.url, mediaType: MEDIA_TYPES[m.type] || 'image/jpeg', description: m.description || '',
|
|
125
|
+
})),
|
|
126
|
+
} : {}),
|
|
127
|
+
});
|
|
128
|
+
return !!out?.added;
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
// Mastodon threads by its own local ids, which mean nothing anywhere else.
|
|
132
|
+
// A parent we already hold from this same account can be resolved locally,
|
|
133
|
+
// and one we do not hold simply has no parent here rather than a fetch.
|
|
134
|
+
_parentMap(acct) {
|
|
135
|
+
const map = new Map();
|
|
136
|
+
for (const s of this.store.getStatuses()) {
|
|
137
|
+
for (const v of s.sourceAccts || []) {
|
|
138
|
+
if (v.acct === acct && v.remoteId) map.set(String(v.remoteId), s.noteId);
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
return map;
|
|
142
|
+
}
|
|
143
|
+
|
|
144
|
+
_rateGuard(id, handle, res) {
|
|
145
|
+
const left = Number(res.headers.get('x-ratelimit-remaining'));
|
|
146
|
+
if (!Number.isFinite(left) || left > RATE_FLOOR) return false;
|
|
147
|
+
const reset = Date.parse(res.headers.get('x-ratelimit-reset') || '');
|
|
148
|
+
const wait = Number.isFinite(reset) ? Math.max(0, reset - Date.now()) : BACKOFF_MIN_MS;
|
|
149
|
+
this._stateOf(id).quietUntil = Date.now() + wait;
|
|
150
|
+
this.log(`acctfeed: ${handle} has ${left} requests left — waiting ${Math.round(wait / 1000)}s`);
|
|
151
|
+
return true;
|
|
152
|
+
}
|
|
153
|
+
|
|
154
|
+
async sweep() {
|
|
155
|
+
const rows = (this.accounts?.list() || []).filter(r => r.token && r.enabled !== false);
|
|
156
|
+
if (!rows.length) return;
|
|
157
|
+
this.lastSweep = new Date().toISOString();
|
|
158
|
+
let added = 0;
|
|
159
|
+
// The 300ms debounce cannot coalesce a sweep that awaits between items —
|
|
160
|
+
// every fetch outlives it — so the whole sweep is one commit boundary.
|
|
161
|
+
// Without this, statuses.json is serialized whole once per post.
|
|
162
|
+
this.store.hold();
|
|
163
|
+
try {
|
|
164
|
+
for (const rec of rows) {
|
|
165
|
+
if (this.stopped) break;
|
|
166
|
+
added += await this._sweepOne(rec).catch((e) => {
|
|
167
|
+
this._backOff(rec.id, rec.handle, e.status || 0, null);
|
|
168
|
+
return 0;
|
|
169
|
+
});
|
|
170
|
+
}
|
|
171
|
+
this._prune();
|
|
172
|
+
} finally {
|
|
173
|
+
this.store.release();
|
|
174
|
+
}
|
|
175
|
+
this.lastAdded = added;
|
|
176
|
+
if (added) this.log(`acctfeed: +${added} from ${rows.length} connected account(s)`);
|
|
177
|
+
}
|
|
178
|
+
|
|
179
|
+
async _sweepOne(rec) {
|
|
180
|
+
const st = this._stateOf(rec.id);
|
|
181
|
+
if (st.quietUntil && Date.now() < st.quietUntil) return 0;
|
|
182
|
+
const marks = this.marks(rec.id);
|
|
183
|
+
const parents = this._parentMap(rec.id);
|
|
184
|
+
let added = 0;
|
|
185
|
+
|
|
186
|
+
const homeQ = marks.homeSinceId ? `&since_id=${encodeURIComponent(marks.homeSinceId)}` : '';
|
|
187
|
+
const homeRes = await this.accounts.api(rec.id, `/api/v1/timelines/home?limit=${PER_SWEEP}${homeQ}`);
|
|
188
|
+
if (this._rateGuard(rec.id, rec.handle, homeRes)) return 0;
|
|
189
|
+
const home = await homeRes.json().catch(() => []);
|
|
190
|
+
if (!Array.isArray(home)) throw new Error(`${rec.handle} answered with no timeline`);
|
|
191
|
+
for (const item of home) {
|
|
192
|
+
if (item?.reblog) {
|
|
193
|
+
// A boost: the inner post is the content, the booster is the carrier —
|
|
194
|
+
// the same envelope statusOrBoost already renders for our own timeline.
|
|
195
|
+
const via = this._rememberAuthor(item.account);
|
|
196
|
+
if (this._mirror(item.reblog, { acct: rec.id, via, parents })) added++;
|
|
197
|
+
} else if (item?.uri) {
|
|
198
|
+
if (this._mirror(item, { acct: rec.id, parents })) added++;
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
// Mastodon returns newest first, so the first row is the new mark.
|
|
202
|
+
// Ids are 19-digit snowflakes: opaque strings, never numbers.
|
|
203
|
+
if (home.length && home[0]?.id) this.setMarks(rec.id, { homeSinceId: String(home[0].id) });
|
|
204
|
+
|
|
205
|
+
const notifQ = marks.notifSinceId ? `&since_id=${encodeURIComponent(marks.notifSinceId)}` : '';
|
|
206
|
+
const notifRes = await this.accounts.api(rec.id, `/api/v1/notifications?limit=${PER_SWEEP}${notifQ}`);
|
|
207
|
+
if (this._rateGuard(rec.id, rec.handle, notifRes)) return added;
|
|
208
|
+
const notes = await notifRes.json().catch(() => []);
|
|
209
|
+
if (Array.isArray(notes)) {
|
|
210
|
+
for (const n of notes) {
|
|
211
|
+
if (!n?.account || String(n.account.id) === String(rec.accountId)) continue;
|
|
212
|
+
const actor = this._rememberAuthor(n.account);
|
|
213
|
+
if (!actor || this.store.isBlocked(actor)) continue;
|
|
214
|
+
// The post a favourite or boost is about is one of this account's own,
|
|
215
|
+
// so it is mirrored first — a notification pointing at nothing is a
|
|
216
|
+
// row the client drops.
|
|
217
|
+
if (n.status?.uri) this._mirror(n.status, { acct: rec.id, parents });
|
|
218
|
+
if (n.type === 'favourite' || n.type === 'reblog') {
|
|
219
|
+
this.store.addNotification({
|
|
220
|
+
type: n.type === 'favourite' ? 'favourite' : 'reblog',
|
|
221
|
+
actor, noteId: n.status?.uri, via: rec.id,
|
|
222
|
+
});
|
|
223
|
+
} else if (n.type === 'follow' || n.type === 'follow_request') {
|
|
224
|
+
this.store.addNotification({ type: 'follow', actor, via: rec.id });
|
|
225
|
+
await this.onNotification?.(n, { actor, acct: rec.id });
|
|
226
|
+
} else if (n.type === 'mention') {
|
|
227
|
+
this.store.addNotification({ type: 'mention', actor, noteId: n.status?.uri, via: rec.id });
|
|
228
|
+
await this.onNotification?.(n, { actor, acct: rec.id });
|
|
229
|
+
}
|
|
230
|
+
}
|
|
231
|
+
if (notes.length && notes[0]?.id) this.setMarks(rec.id, { notifSinceId: String(notes[0].id) });
|
|
232
|
+
}
|
|
233
|
+
|
|
234
|
+
st.failures = 0;
|
|
235
|
+
return added;
|
|
236
|
+
}
|
|
237
|
+
|
|
238
|
+
// Two caps. The per-account one keeps a busy account from crowding out a
|
|
239
|
+
// quiet one; the total keeps every connected account together from evicting
|
|
240
|
+
// the pod's own posts and the timeline it already had.
|
|
241
|
+
_prune() {
|
|
242
|
+
const all = this.store.getStatuses();
|
|
243
|
+
const mine = all.filter(s => s.kind === 'acct');
|
|
244
|
+
if (!mine.length) return;
|
|
245
|
+
const drop = new Set();
|
|
246
|
+
const seen = new Map();
|
|
247
|
+
for (const s of mine) {
|
|
248
|
+
const acct = s.sourceAccts?.[0]?.acct || '?';
|
|
249
|
+
const n = (seen.get(acct) || 0) + 1;
|
|
250
|
+
seen.set(acct, n);
|
|
251
|
+
if (n > MAX_PER_ACCT) drop.add(s.noteId);
|
|
252
|
+
}
|
|
253
|
+
for (const s of mine.slice(MAX_ACCT_ENTRIES)) drop.add(s.noteId);
|
|
254
|
+
if (!drop.size) return;
|
|
255
|
+
this.store.write('statuses.json', all.filter(s => !drop.has(s.noteId)));
|
|
256
|
+
}
|
|
257
|
+
}
|
package/lib/admin.mjs
CHANGED
|
@@ -39,6 +39,38 @@ const projectRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '
|
|
|
39
39
|
const { makeGate } = require(path.join(projectRoot, 'vendor/gate.cjs'));
|
|
40
40
|
|
|
41
41
|
const PHANPY_DIR = path.join(projectRoot, 'phanpy/dist');
|
|
42
|
+
// The vendored client is upstream's build, byte for byte, so the integrity
|
|
43
|
+
// check can say so. The two things it needs changed happen on the way out
|
|
44
|
+
// instead of in the files.
|
|
45
|
+
//
|
|
46
|
+
// vite-plugin-pwa injects this registration into every page it builds. The
|
|
47
|
+
// worker it installs answers navigations from a precache and replays the
|
|
48
|
+
// headers stored at install time, so a CSP change can never reach a browser
|
|
49
|
+
// that already has one.
|
|
50
|
+
const SW_REGISTER = /<script id="vite-plugin-pwa:inline-sw">[\s\S]*?<\/script>/i;
|
|
51
|
+
// Removing it stops new installs. This replaces the worker itself, so the ones
|
|
52
|
+
// already out there clean up: browsers re-fetch sw.js on navigation and
|
|
53
|
+
// install what they find. No fetch handler on purpose — a worker without one
|
|
54
|
+
// never intercepts a request, so pages go straight to the network while it
|
|
55
|
+
// runs.
|
|
56
|
+
const SW_KILL = `self.addEventListener('install', () => self.skipWaiting());
|
|
57
|
+
|
|
58
|
+
self.addEventListener('activate', (event) => {
|
|
59
|
+
event.waitUntil((async () => {
|
|
60
|
+
for (const key of await caches.keys()) await caches.delete(key);
|
|
61
|
+
await self.registration.unregister();
|
|
62
|
+
for (const client of await self.clients.matchAll({ type: 'window' })) {
|
|
63
|
+
try { await client.navigate(client.url); } catch { /* tab will refresh on its own */ }
|
|
64
|
+
}
|
|
65
|
+
})());
|
|
66
|
+
});
|
|
67
|
+
`;
|
|
68
|
+
// Null rather than the original when the tag is gone: an upstream change that
|
|
69
|
+
// silently no-opped here would put the worker back, which is the whole thing
|
|
70
|
+
// this exists to prevent.
|
|
71
|
+
function stripSwRegistration(html) {
|
|
72
|
+
return SW_REGISTER.test(html) ? html.replace(SW_REGISTER, '') : null;
|
|
73
|
+
}
|
|
42
74
|
const UI_DIR = path.join(projectRoot, 'ui'); // extra client dists: ui/<name>/ → /<name>/
|
|
43
75
|
// Our own pages, kept out of ui/ for two reasons: a client dist dropped in
|
|
44
76
|
// there under the same name would shadow them, and a group serves these and
|
|
@@ -81,6 +113,35 @@ const AGENT_VERSION = JSON.parse(fs.readFileSync(path.join(projectRoot, 'package
|
|
|
81
113
|
// the case it exists for; it is in LOCAL_ONLY_POSTS below, so it still answers
|
|
82
114
|
// only to this machine.
|
|
83
115
|
const OPEN_POSTS = new Set(['/block', '/unblock', '/setup', '/setup/check', '/shutdown']);
|
|
116
|
+
|
|
117
|
+
// The page the other server's redirect lands on. Self-contained on purpose:
|
|
118
|
+
// the browser arrives here from somewhere else, and nothing may load from
|
|
119
|
+
// that somewhere.
|
|
120
|
+
function callbackPage(ok, msg) {
|
|
121
|
+
const esc = (s) => String(s).replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c]));
|
|
122
|
+
const head = ok ? 'Account connected' : 'Not connected';
|
|
123
|
+
return `<!doctype html>
|
|
124
|
+
<html lang="en"><head><meta charset="utf-8">
|
|
125
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
126
|
+
<title>${head}</title>
|
|
127
|
+
<style>
|
|
128
|
+
:root { color-scheme: light dark; }
|
|
129
|
+
body { font: 16px/1.7 system-ui, -apple-system, sans-serif; margin: 0;
|
|
130
|
+
padding: 2.5rem 2rem; background: #ffffff; color: #1a1a1a; }
|
|
131
|
+
main { max-width: 32rem; margin: 0 auto; }
|
|
132
|
+
h1 { font-size: 1.375rem; font-weight: 500; margin: 0 0 1rem; }
|
|
133
|
+
p { margin: 0 0 1rem; }
|
|
134
|
+
@media (prefers-color-scheme: dark) {
|
|
135
|
+
body { background: #16161a; color: #ececf0; }
|
|
136
|
+
}
|
|
137
|
+
</style></head>
|
|
138
|
+
<body><main>
|
|
139
|
+
<h1>${head}</h1>
|
|
140
|
+
<p>${esc(msg)}</p>
|
|
141
|
+
<p>You can close this tab and go back to FediPod.</p>
|
|
142
|
+
</main></body></html>
|
|
143
|
+
`;
|
|
144
|
+
}
|
|
84
145
|
// Routes that manage a local agent process — spawning siblings, killing this
|
|
85
146
|
// one, moving files on the machine, running setup in a browser. Inside a pod
|
|
86
147
|
// server there is no such process and no such machine: identities come from
|
|
@@ -90,7 +151,7 @@ const EMBEDDED_CUT = new Set(['/profiles', '/shutdown', '/new-actor', '/start-ac
|
|
|
90
151
|
// AP_ALLOWED_HOSTS may name a tailnet host or a reverse-proxy domain. The
|
|
91
152
|
// fediverse is welcome there; creating accounts and editing the record is for
|
|
92
153
|
// whoever is sitting at the machine.
|
|
93
|
-
const LOCAL_ONLY_POSTS = new Set(['/setup', '/setup/check', '/config', '/new-actor', '/start-actor', '/shutdown', '/state-move', '/atproto/connect', '/gateway', '/alias', '/import', '/update']);
|
|
154
|
+
const LOCAL_ONLY_POSTS = new Set(['/setup', '/setup/check', '/config', '/new-actor', '/start-actor', '/shutdown', '/state-move', '/atproto/connect', '/fediacct/connect', '/fediacct/disconnect', '/fediacct', '/gateway', '/alias', '/import', '/update']);
|
|
94
155
|
// The identity itself. Changing any of these means a different actor at a
|
|
95
156
|
// different address, which is a new setup, not an edit.
|
|
96
157
|
const PERMANENT_CONFIG = ['handle', 'remotePod', 'issuer', 'root', 'kind'];
|
|
@@ -118,7 +179,10 @@ function inlineScriptHashes() {
|
|
|
118
179
|
if (inlineHashes) return inlineHashes;
|
|
119
180
|
inlineHashes = [];
|
|
120
181
|
try {
|
|
121
|
-
|
|
182
|
+
// The page as it is SERVED, not as it sits on disk: hashing the
|
|
183
|
+
// registration script we strip would allow a script nobody gets.
|
|
184
|
+
const raw = fs.readFileSync(path.join(PHANPY_DIR, 'index.html'), 'utf8');
|
|
185
|
+
const html = stripSwRegistration(raw) ?? raw;
|
|
122
186
|
for (const m of html.matchAll(/<script(?![^>]*\ssrc=)[^>]*>([\s\S]*?)<\/script>/gi)) {
|
|
123
187
|
const digest = crypto.createHash('sha256').update(m[1], 'utf8').digest('base64');
|
|
124
188
|
inlineHashes.push(`'sha256-${digest}'`);
|
|
@@ -296,7 +360,23 @@ function sendFile(res, baseDir, rel, auth) {
|
|
|
296
360
|
res.writeHead(403); res.end(); return true;
|
|
297
361
|
}
|
|
298
362
|
const ext = path.extname(real);
|
|
299
|
-
|
|
363
|
+
let body = fs.readFileSync(real);
|
|
364
|
+
// The vendored client, adjusted on the way out rather than in the files.
|
|
365
|
+
if (real.startsWith(fs.realpathSync(PHANPY_DIR))) {
|
|
366
|
+
if (path.basename(real) === 'sw.js') {
|
|
367
|
+
body = Buffer.from(SW_KILL);
|
|
368
|
+
} else if (ext === '.html') {
|
|
369
|
+
const stripped = stripSwRegistration(body.toString('utf8'));
|
|
370
|
+
if (stripped === null) {
|
|
371
|
+
console.error(`refusing to serve ${path.basename(real)}: no service-worker `
|
|
372
|
+
+ 'registration to remove — the vendored client changed shape');
|
|
373
|
+
res.writeHead(500, { 'content-type': 'text/plain', ...securityHeaders(auth, false) });
|
|
374
|
+
res.end('the bundled client changed shape; refusing to serve it\n');
|
|
375
|
+
return true;
|
|
376
|
+
}
|
|
377
|
+
body = Buffer.from(stripped);
|
|
378
|
+
}
|
|
379
|
+
}
|
|
300
380
|
// Our own pages are read straight off disk and change whenever the project
|
|
301
381
|
// does. With no cache headers a browser is free to reuse them without
|
|
302
382
|
// asking, so an edited page keeps rendering the old one and looks like the
|
|
@@ -511,6 +591,32 @@ export function buildAdminSurface({ agent, gate, allowed, log = console.log,
|
|
|
511
591
|
if (req.method === 'GET' && p === '/blocks') return json(res, 200, agent.store.getBlocklist());
|
|
512
592
|
if (req.method === 'GET' && p === '/log') return json(res, 200, { lines: agent.logLines(200) });
|
|
513
593
|
if (req.method === 'GET' && p === '/deadletter') return json(res, 200, { items: agent.store.getDeadLetters() });
|
|
594
|
+
if (req.method === 'GET' && p === '/fediacct') {
|
|
595
|
+
return json(res, 200, { accounts: agent.fediaccts?.status() || [] });
|
|
596
|
+
}
|
|
597
|
+
// Where the other server sends the browser back. A GET, so it gets none
|
|
598
|
+
// of the POST branch's local-only protection for free and asks for its
|
|
599
|
+
// own — a code arriving from anywhere else is not the owner's.
|
|
600
|
+
if (req.method === 'GET' && p === '/fediacct/callback') {
|
|
601
|
+
if (!embedded && !allowed.isLocalRequest(req)) {
|
|
602
|
+
return json(res, 403, { error: 'connecting an account is available on this machine only' });
|
|
603
|
+
}
|
|
604
|
+
const q = new URL(req.url, 'https://x.invalid').searchParams;
|
|
605
|
+
const done = (ok, msg) => {
|
|
606
|
+
res.writeHead(ok ? 200 : 400, { 'content-type': 'text/html; charset=utf-8' });
|
|
607
|
+
res.end(callbackPage(ok, msg));
|
|
608
|
+
};
|
|
609
|
+
if (q.get('error')) return done(false, q.get('error_description') || q.get('error'));
|
|
610
|
+
if (!q.get('code') || !q.get('state')) return done(false, 'that sign-in came back incomplete');
|
|
611
|
+
try {
|
|
612
|
+
const row = await agent.fediaccts.complete({ state: q.get('state'), code: q.get('code') });
|
|
613
|
+
const cfg = agent.store.getConfig();
|
|
614
|
+
agent.store.setConfig({ ...cfg, fediAccounts: agent.fediaccts.roster() });
|
|
615
|
+
await agent.store.flush();
|
|
616
|
+
agent.restartAccts?.();
|
|
617
|
+
return done(true, `${row.handle} is connected.`);
|
|
618
|
+
} catch (e) { return done(false, e.message); }
|
|
619
|
+
}
|
|
514
620
|
if (req.method === 'GET' && p === '/tagfeed') {
|
|
515
621
|
return json(res, 200, agent.tagfeed
|
|
516
622
|
? { ...agent.tagfeed.config(), lastSweep: agent.tagfeed.lastSweep, lastAdded: agent.tagfeed.lastAdded }
|
|
@@ -613,6 +719,10 @@ export function buildAdminSurface({ agent, gate, allowed, log = console.log,
|
|
|
613
719
|
atproto: cfg.atproto
|
|
614
720
|
? { ...cfg.atproto, connected: !!agent.atproto?.connected() }
|
|
615
721
|
: null,
|
|
722
|
+
// Connected fediverse accounts, from the credentials themselves
|
|
723
|
+
// rather than from config — a roster entry whose file was deleted
|
|
724
|
+
// should stop being listed, not linger.
|
|
725
|
+
fediAccounts: agent.fediaccts?.status() || [],
|
|
616
726
|
origins: {
|
|
617
727
|
loopback: publicOrigin
|
|
618
728
|
|| `https://localhost:${port}/`,
|
|
@@ -1304,6 +1414,38 @@ export function buildAdminSurface({ agent, gate, allowed, log = console.log,
|
|
|
1304
1414
|
await agent.store.flush();
|
|
1305
1415
|
return json(res, 200, { ok: true, atproto: { ...cfg.atproto, crossPost: !!body.crossPost } });
|
|
1306
1416
|
}
|
|
1417
|
+
case '/fediacct/connect': {
|
|
1418
|
+
// Starts the sign-in at the other server. Loopback-only, and the
|
|
1419
|
+
// redirect returns to the origin the owner is actually using, not a
|
|
1420
|
+
// guess — a client reaches this agent by several names.
|
|
1421
|
+
if (!body.host) return json(res, 400, { error: 'the address of the server is required' });
|
|
1422
|
+
try {
|
|
1423
|
+
const { url } = await agent.fediaccts.begin({
|
|
1424
|
+
host: body.host, redirectUri: `https://${req.headers.host}/fediacct/callback`,
|
|
1425
|
+
});
|
|
1426
|
+
return json(res, 200, { ok: true, authorize: url });
|
|
1427
|
+
} catch (e) { return json(res, 400, { error: e.message }); }
|
|
1428
|
+
}
|
|
1429
|
+
case '/fediacct/disconnect': {
|
|
1430
|
+
if (!body.id) return json(res, 400, { error: 'id required' });
|
|
1431
|
+
if (!agent.fediaccts.remove(body.id)) return json(res, 404, { error: 'no such account' });
|
|
1432
|
+
const cfg = agent.store.getConfig();
|
|
1433
|
+
agent.store.setConfig({ ...cfg, fediAccounts: agent.fediaccts.roster() });
|
|
1434
|
+
await agent.store.flush();
|
|
1435
|
+
agent.restartAccts?.();
|
|
1436
|
+
return json(res, 200, { ok: true });
|
|
1437
|
+
}
|
|
1438
|
+
case '/fediacct': {
|
|
1439
|
+
// Non-secret settings only — today that is whether it is polled.
|
|
1440
|
+
if (!body.id) return json(res, 400, { error: 'id required' });
|
|
1441
|
+
const row = agent.fediaccts.setEnabled(body.id, !!body.enabled);
|
|
1442
|
+
if (!row) return json(res, 404, { error: 'no such account' });
|
|
1443
|
+
const cfg = agent.store.getConfig();
|
|
1444
|
+
agent.store.setConfig({ ...cfg, fediAccounts: agent.fediaccts.roster() });
|
|
1445
|
+
await agent.store.flush();
|
|
1446
|
+
agent.restartAccts?.();
|
|
1447
|
+
return json(res, 200, { ok: true, account: row });
|
|
1448
|
+
}
|
|
1307
1449
|
case '/archive': {
|
|
1308
1450
|
// Whether drained mail's original bytes are kept in the private
|
|
1309
1451
|
// half's inbox-archive/. Absent means on.
|
package/lib/bskygroup.mjs
CHANGED
|
@@ -30,7 +30,7 @@ export class BskyGroup {
|
|
|
30
30
|
// ONE reply ever: tell an unbridged joiner how to reach the fediverse side.
|
|
31
31
|
async nudge({ did, handle }) {
|
|
32
32
|
const text = `@${handle} welcome! Follow @ap.brid.gy and your posts will reach `
|
|
33
|
-
+ 'the
|
|
33
|
+
+ 'the Fediverse side of this group too.';
|
|
34
34
|
const start = 0;
|
|
35
35
|
const rec = this.atproto.read();
|
|
36
36
|
await this.atproto.xrpc('com.atproto.repo.createRecord', {
|