@audiofool/moondog 0.3.0-beta.10 → 0.3.0-beta.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  <h1 align="center">Moondog</h1>
6
6
 
7
- <p align="center">Source candidate 0.3.0-beta.10</p>
7
+ <p align="center">Source candidate 0.3.0-beta.11</p>
8
8
 
9
9
  <p align="center"><strong>A listening room that shows its work.</strong></p>
10
10
 
@@ -44,7 +44,8 @@ See [updating and release channels](docs/UPDATES.md), including the one-time upg
44
44
 
45
45
  [Moondog for Mac](app/README.md) is a native window on the same core as the terminal room.
46
46
  It shares your profile, settings, sign-ins, and conversations with `moondog`, and draws its own conversation, now playing, and up next.
47
- Run it from this checkout with `cd app && npm ci && npm start`.
47
+ Download the notarized build for Apple silicon from the [project page](https://moondog.audiofool.ai), or run it from this checkout with `cd app && npm ci && npm start`.
48
+ It updates itself, on a Stable or Beta channel you choose in Settings.
48
49
 
49
50
  The [project page](https://moondog.audiofool.ai) plays a short scene in this window, in a browser, with a made-up listener ([how](site/README.md)).
50
51
 
@@ -147,11 +148,17 @@ OpenAI takes your ChatGPT subscription or an API key.
147
148
  In the Mac app, any service that speaks OpenAI's or Anthropic's API can be added as an endpoint by its address, and `/endpoint` does it in the terminal.
148
149
  You can switch models in the middle of a conversation.
149
150
  The [model setup guide](docs/TERMINAL_GUIDE.md#enable-agent-conversation) lists each provider and the environment variable it reads.
151
+ `/model roles` shows the models and services behind side tasks: a vision model for images, and the web search service.
152
+ Moondog takes images too: an image path in your message in the terminal, or a pasted or dropped image in the Mac app.
153
+ You can also change settings in your own words, such as which model answers or the language it replies in.
150
154
 
151
155
  While an answer is on the way, each step appears in the conversation as it starts.
152
156
  The steps stay above the reply, with a count of what finished, what failed, and what was not confirmed.
153
157
  The agent can use musical knowledge and web research to find candidates, then verify the recordings on Spotify before playing or queueing them.
154
158
  Without Spotify, it plays 30-second Apple Music previews from the terminal instead, each next to a link to the full song.
159
+ With NetEase Music, it plays whole songs through NetEase's own `ncm-cli`.
160
+ In the Mac app, Apple Music plays whole songs with a subscription.
161
+ `/service` chooses where Moondog plays.
155
162
  It also sees what you already play: the song and playlist in the Music app on a Mac, or on Spotify's active device, so “more like this playlist” needs no names.
156
163
  macOS asks once before Moondog may read the Music app.
157
164
  For example, “Pon doce de salsa clásica a la cola” asks it to add up to twelve verified songs directly; your history guides the choice without limiting it to your library.
@@ -164,13 +171,28 @@ Songs Moondog chose for you do not count as your own taste.
164
171
  | A model through Pi | Conversation, discovery, and playlists built together. |
165
172
  | A web search service | `/web` lookups with links to their sources, through DeepSeek (`/auth deepseek`, which also reads pages), Brave Search (`/auth brave`), or your own signed-in Codex CLI. |
166
173
  | Spotify sign-in | Playback, and private playlists made only when you ask. |
174
+ | NetEase Music | Whole songs through its own `ncm-cli`, for listeners in mainland China. |
167
175
 
168
176
  Your profile never touches the network.
169
177
  The agent talks only to the services you set up.
170
178
  `/web` sends the question or the link you give it, and nothing from your profile, history, or files.
179
+ An image you attach goes to the model that reads it.
171
180
  A Spotify ZIP does not need a Spotify sign-in.
172
181
  Read the [connection and data details](docs/TERMINAL_GUIDE.md) before you connect anything.
173
182
 
183
+ ## Your library
184
+
185
+ `/library` keeps songs and the tracks Moondog made side by side: Saved, then your playlists.
186
+ `/library save` keeps what is playing, and `/library play Night drive` plays a playlist on the service you chose.
187
+ `/library import <link>` copies a Spotify, Apple Music, NetEase Music or QQ Music playlist into it.
188
+ A playlist you made on Spotify also shapes your taste; for any other one, you decide.
189
+
190
+ ## Music it makes
191
+
192
+ On an Apple silicon Mac, Moondog can make instrumental tracks offline with Stable Audio 3, powered by Stability AI.
193
+ `/generation download medium` sets it up after you agree to its licenses, and `/generation on` turns it on.
194
+ Ask for a track, or type `/onair` and Moondog keeps making music from your taste until you stop it.
195
+
174
196
  ## Getting around
175
197
 
176
198
  | Command or key | What it does |
@@ -186,7 +208,10 @@ Read the [connection and data details](docs/TERMINAL_GUIDE.md) before you connec
186
208
  | Enter while Moondog is working | Queue the next message. It sends when this one finishes. |
187
209
  | `/resume`, `/new` | Pick up a saved conversation, or start a fresh one. |
188
210
  | `/home` | Back to the sleeve, without clearing the conversation. |
189
- | `/skip`, `/stop` | Skip to the next song preview, or stop the previews. |
211
+ | `/skip`, `/stop` | Skip to the next song preview or stream track, or stop them. |
212
+ | `/library` | Your saved songs and playlists. |
213
+ | `/onair` | Go on air, or come off it. |
214
+ | `/rewind`, `/branch` | Change an earlier message, or go on in a new conversation. |
190
215
  | `/lyrics` | See which line is on the sleeve, and which song it came from. |
191
216
  | `/theme paper` | Black ink on white. |
192
217
  | `/theme charcoal` | Moonlight on a black sky. |
@@ -213,4 +238,4 @@ Your profile, and anything Moondog remembers, carry over.
213
238
 
214
239
  Moondog is open source under the [Apache License 2.0](LICENSE).
215
240
  Copyright 2026 Audiofool.
216
- Music making runs Stable Audio 3, which you install yourself; [NOTICE](NOTICE) holds its attribution, and [licenses/](licenses/stability-ai-community-license.md) holds the Stability AI Community License.
241
+ Music making runs Stable Audio 3, which Moondog downloads from Stability AI when you ask, or your own install; [NOTICE](NOTICE) holds its attribution, and [licenses/](licenses/stability-ai-community-license.md) holds the Stability AI Community License.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@audiofool/moondog",
3
- "version": "0.3.0-beta.10",
3
+ "version": "0.3.0-beta.11",
4
4
  "type": "module",
5
5
  "description": "Local-first personal AI music agent.",
6
6
  "license": "Apache-2.0",
@@ -33,6 +33,7 @@ import { runCatalogCommand } from "../src/surfaces/cli/catalog-command.mjs";
33
33
  import { runProfileCommand } from "../src/surfaces/cli/profile-command.mjs";
34
34
  import { runListenerProfileCommand, savedProfileActions } from "../src/surfaces/cli/listener-profile-command.mjs";
35
35
  import { createWebResearch } from "../src/integrations/web/web-research.mjs";
36
+ import { runModelRolesCommand } from "../src/surfaces/cli/model-roles-command.mjs";
36
37
  import { runWebCommand } from "../src/surfaces/cli/web-command.mjs";
37
38
  import { openSpotifyConnection } from "../src/integrations/spotify/connection.mjs";
38
39
  import { preferredIdleDevice, thisComputerName } from "../src/integrations/spotify/service.mjs";
@@ -1235,10 +1236,14 @@ async function main() {
1235
1236
  // The desktop window lists models and signs in here, with no terminal.
1236
1237
  modelSettings,
1237
1238
  // A model chosen or a sign-in made on the other surface. Model variables fix the model at
1238
- // launch, so a saved choice has nothing to change then.
1239
- watchModelSettings: (onChange) => (process.env.MOONDOG_PROVIDER !== undefined || process.env.MOONDOG_MODEL !== undefined
1240
- ? () => {}
1241
- : watchRuntimeSettings({ environment: process.env, onChange })),
1239
+ // launch, so a saved choice has nothing to change then; a role chosen elsewhere still does.
1240
+ watchModelSettings: (onChange) => {
1241
+ const fixed = process.env.MOONDOG_PROVIDER !== undefined || process.env.MOONDOG_MODEL !== undefined;
1242
+ return watchRuntimeSettings({ environment: process.env, onChange: (selection, changed) => {
1243
+ if (changed === "roles") return void webResearch.refresh().then(() => onChange(selection, changed), () => {});
1244
+ if (!fixed) onChange(selection, changed);
1245
+ } });
1246
+ },
1242
1247
  // The one-time music service and model setup, shared by the terminal and the window.
1243
1248
  firstRun: createFirstRun({ environment: process.env }),
1244
1249
  // A host with its own terminal streams takes the sign-in prompts. The
@@ -1264,6 +1269,7 @@ async function main() {
1264
1269
  }),
1265
1270
  }),
1266
1271
  runWeb: (args, options) => runWebCommand({ args, webResearch, ...options }),
1272
+ runModelRoles: (args) => runModelRolesCommand({ args: args.map(String), modelSettings, webResearch }),
1267
1273
  runProfileAction: (args) => runProfileCommand({
1268
1274
  args,
1269
1275
  commandPrefix: "/profile",
@@ -1296,6 +1296,10 @@ export class AppleProjectionDomainServices {
1296
1296
  return this.#listeningHistoryStore?.recentAttention?.({ subjectId: this.#subjectId }) ?? null;
1297
1297
  }
1298
1298
 
1299
+ listeningDays() {
1300
+ return this.#listeningHistoryStore?.listeningDays?.({ subjectId: this.#subjectId }) ?? null;
1301
+ }
1302
+
1299
1303
  // Moondog's suggestions ledger and the declared mix (src/profile/moondog-suggestions.mjs).
1300
1304
  recordMoondogSuggestions(entries) {
1301
1305
  return this.#listeningHistoryStore?.recordMoondogSuggestions?.({ entries }) ?? null;
@@ -506,6 +506,10 @@ export class ListeningProfileDomainServices {
506
506
  return this.#store.recentAttention?.({ subjectId: this.#subjectId }) ?? null;
507
507
  }
508
508
 
509
+ listeningDays() {
510
+ return this.#store.listeningDays?.({ subjectId: this.#subjectId }) ?? null;
511
+ }
512
+
509
513
  // Moondog's suggestions ledger and the declared mix (src/profile/moondog-suggestions.mjs).
510
514
  recordMoondogSuggestions(entries) {
511
515
  return this.#store.recordMoondogSuggestions?.({ entries }) ?? null;
@@ -740,7 +740,7 @@ export class MoondogApplication {
740
740
  found = { provider: "spotify", id: parsed.id, url: parsed.url, name: playlist.name || "Spotify playlist", total: playlist.tracks_total, owned: playlist.owned,
741
741
  tracks: playlist.tracks.map((track) => { const credit = track.artists.join(", "); return { id: track.uri, title: track.name, artist: credit,
742
742
  ...(track.album ? { album: track.album } : {}), ...(track.isrc ? { isrc: track.isrc } : {}),
743
- refs: { spotify_uri: track.uri, spotify_title: track.name, spotify_credit: credit } }; }) };
743
+ refs: { spotify_uri: track.uri, spotify_title: track.name, spotify_credit: credit, ...(track.image ? { artwork_url: track.image } : {}) } }; }) };
744
744
  } else if (parsed.provider === "apple_music") {
745
745
  const playlist = await readApplePlaylist(parsed, { signal });
746
746
  // A playlist a listener shared has a pl.u- id; the others are Apple's own.
@@ -748,7 +748,8 @@ export class MoondogApplication {
748
748
  tracks: playlist.tracks.map((track) => ({ id: track.id, title: track.title, artist: track.artist, refs: { apple_music_id: track.id, ...(track.artwork ? { artwork_url: track.artwork } : {}) } })) };
749
749
  } else {
750
750
  const playlist = await readPublicPlaylistLink(parsed);
751
- found = { ...playlist, owned: null, tracks: playlist.tracks.map((track) => ({ ...track, ...(playlist.provider === "netease" ? { refs: { netease_id: String(track.id) } } : {}) })) };
751
+ found = { ...playlist, owned: null, tracks: playlist.tracks.map(({ artwork, ...track }) => ({ ...track,
752
+ refs: { ...(playlist.provider === "netease" ? { netease_id: String(track.id) } : {}), ...(artwork ? { artwork_url: artwork } : {}) } })) };
752
753
  }
753
754
  signal?.throwIfAborted();
754
755
  // A library playlist holds up to 2,000 songs; a longer one comes in from its start, and total says how long it is.
@@ -1404,6 +1405,13 @@ export class MoondogApplication {
1404
1405
  return this.domainServices?.listeningCoverage?.() ?? null;
1405
1406
  }
1406
1407
 
1408
+ // The listener's own plays a day at a time, with the most played artists' and Moondog's own kept
1409
+ // apart, for the window's taste record. Host-only; never a model tool.
1410
+ listeningDays() {
1411
+ if (!this.profileServicesReady()) return null;
1412
+ return this.domainServices?.listeningDays?.() ?? null;
1413
+ }
1414
+
1407
1415
  // What the listener has played lately, apart from the saved reading: stored plays from
1408
1416
  // the last four weeks, and Spotify's own four-week top artists when the room has read them.
1409
1417
  recentAttention() {
@@ -713,6 +713,8 @@ export default {
713
713
  "Use /taste, or /taste report.": "Usa /taste o /taste report.",
714
714
  "Full report above. /taste opens the profile.": "El informe completo está arriba. /taste abre el perfil.",
715
715
  "Web lookups aren't available when Moondog is started this way.": "Las consultas web no están disponibles cuando Moondog se abre de esta forma.",
716
+ "Model roles aren't available when Moondog is started this way.": "Los usos de los modelos no están disponibles cuando Moondog se abre de esta forma.",
717
+ "Looking at the attached images... Ctrl+C cancels.": "Mirando las imágenes adjuntas... Ctrl+C cancela.",
716
718
  "/home doesn't take anything after it.": "/home no admite nada después.",
717
719
  "Say anything, or press Tab to look around.": "Di lo que quieras, o pulsa Tab para echar un vistazo.",
718
720
  "Use /theme, or /theme paper, charcoal, terminal, or auto.": "Usa /theme, o /theme paper, charcoal, terminal o auto.",
@@ -713,6 +713,8 @@ export default {
713
713
  "Use /taste, or /taste report.": "/taste、または /taste report を使ってください。",
714
714
  "Full report above. /taste opens the profile.": "完全なレポートは上にあります。/taste でプロフィールを開けます。",
715
715
  "Web lookups aren't available when Moondog is started this way.": "この起動方法では Web 検索を使えません。",
716
+ "Model roles aren't available when Moondog is started this way.": "この起動方法ではモデルの用途を設定できません。",
717
+ "Looking at the attached images... Ctrl+C cancels.": "添付の画像を見ています... Ctrl+C で取り消します。",
716
718
  "/home doesn't take anything after it.": "/home のあとには何も付けません。",
717
719
  "Say anything, or press Tab to look around.": "なんでも話しかけてください。Tab で見て回ることもできます。",
718
720
  "Use /theme, or /theme paper, charcoal, terminal, or auto.": "/theme、または /theme paper、charcoal、terminal、auto を使ってください。",
@@ -713,6 +713,8 @@ export default {
713
713
  "Use /taste, or /taste report.": "Use /taste ou /taste report.",
714
714
  "Full report above. /taste opens the profile.": "O relatório completo está acima. /taste abre o perfil.",
715
715
  "Web lookups aren't available when Moondog is started this way.": "As consultas na web não estão disponíveis quando o Moondog é aberto deste jeito.",
716
+ "Model roles aren't available when Moondog is started this way.": "Os usos dos modelos não estão disponíveis quando o Moondog é aberto deste jeito.",
717
+ "Looking at the attached images... Ctrl+C cancels.": "Olhando as imagens anexadas... Ctrl+C cancela.",
716
718
  "/home doesn't take anything after it.": "/home não aceita nada depois.",
717
719
  "Say anything, or press Tab to look around.": "Diga qualquer coisa, ou aperte Tab para dar uma olhada.",
718
720
  "Use /theme, or /theme paper, charcoal, terminal, or auto.": "Use /theme, ou /theme paper, charcoal, terminal ou auto.",
@@ -713,6 +713,8 @@ export default {
713
713
  "Use /taste, or /taste report.": "使用 /taste 或 /taste report。",
714
714
  "Full report above. /taste opens the profile.": "完整报告在上面。/taste 打开资料。",
715
715
  "Web lookups aren't available when Moondog is started this way.": "这种启动方式不支持网页查询。",
716
+ "Model roles aren't available when Moondog is started this way.": "这种启动方式不支持设置模型用途。",
717
+ "Looking at the attached images... Ctrl+C cancels.": "正在看附上的图片…… Ctrl+C 取消。",
716
718
  "/home doesn't take anything after it.": "/home 后面不需要加内容。",
717
719
  "Say anything, or press Tab to look around.": "随便说点什么,或按 Tab 四处看看。",
718
720
  "Use /theme, or /theme paper, charcoal, terminal, or auto.": "使用 /theme,或 /theme paper、charcoal、terminal 或 auto。",
@@ -90,7 +90,13 @@ export async function readApplePlaylist({ id, url }, { fetchImpl = fetch, signal
90
90
 
91
91
  /** A NetEase Music or QQ Music playlist through its public share link, as library songs. */
92
92
  export async function readPublicPlaylistLink({ url }, { fetchImpl = fetch } = {}) {
93
- const playlist = await fetchPublicPlaylist({ input: url, fetchImpl });
93
+ // The public reader explains itself in words; the library's surfaces also need its kind, to say it in their language.
94
+ const playlist = await fetchPublicPlaylist({ input: url, fetchImpl }).catch((error) => {
95
+ // Only the link checks' own refusals are about the link; any answer the service gave is about the playlist.
96
+ const aboutLink = /^(Paste a QQ Music|Use a playlist link|Open this share link|This link belongs to another service)/u;
97
+ if (!error.code) error.code = /^Could not reach/u.test(error.message) ? "playlist_unreachable" : aboutLink.test(error.message) ? "playlist_link_unsupported" : "playlist_unavailable";
98
+ throw error;
99
+ });
94
100
  return { provider: playlist.provider, id: playlist.id, url: playlist.url, name: playlist.name, total: playlist.total,
95
- tracks: playlist.rows.map((row) => ({ id: row.id, title: row.title, artist: row.artist, ...(row.album ? { album: row.album } : {}) })).filter((track) => track.artist) };
101
+ tracks: playlist.rows.map((row) => ({ id: row.id, title: row.title, artist: row.artist, ...(row.album ? { album: row.album } : {}), ...(row.artwork ? { artwork: row.artwork } : {}) })).filter((track) => track.artist) };
96
102
  }
@@ -54,6 +54,30 @@ async function boundedJson(response) {
54
54
  catch { throw new Error("The service changed its playlist response or requires sign-in. No music was imported."); }
55
55
  }
56
56
 
57
+ // A NetEase playlist, as { code, result: { id, name, privacy, trackCount, tracks } }. NetEase's web player reads
58
+ // playlists with v6, which answers every public one, a listener's Liked songs among them, with all its song ids but
59
+ // details for only the first few without a sign-in; the rest come from song/detail, 200 at a time, in the
60
+ // playlist's order. The older endpoint, which some playlists answer with code 20001, is the fallback.
61
+ const NETEASE_SONGS_MAX = 3000;
62
+ async function neteasePlaylist(id, fetchImpl) {
63
+ const referer = "https://music.163.com/";
64
+ const current = await boundedJson(await responseFor(`https://music.163.com/api/v6/playlist/detail?id=${id}&n=100000`, fetchImpl, referer)).catch(() => null);
65
+ const playlist = current?.code === 200 ? current.playlist : null;
66
+ if (!playlist || !Array.isArray(playlist.trackIds)) {
67
+ return boundedJson(await responseFor(`https://music.163.com/api/playlist/detail?id=${id}`, fetchImpl, referer));
68
+ }
69
+ const ids = playlist.trackIds.map((entry) => String(entry?.id ?? "")).filter((value) => numericId.test(value)).slice(0, NETEASE_SONGS_MAX);
70
+ const songs = new Map((Array.isArray(playlist.tracks) ? playlist.tracks : []).map((song) => [String(song?.id ?? ""), song]));
71
+ const missing = ids.filter((songId) => !songs.has(songId));
72
+ for (let start = 0; start < missing.length; start += 200) {
73
+ const batch = missing.slice(start, start + 200);
74
+ const detail = await boundedJson(await responseFor(`https://music.163.com/api/song/detail/?ids=${encodeURIComponent(`[${batch.join(",")}]`)}`, fetchImpl, referer));
75
+ for (const song of Array.isArray(detail?.songs) ? detail.songs : []) songs.set(String(song?.id ?? ""), song);
76
+ }
77
+ return { code: 200, result: { id: playlist.id, name: playlist.name, privacy: playlist.privacy, trackCount: playlist.trackCount ?? ids.length,
78
+ tracks: ids.map((songId) => songs.get(songId)).filter(Boolean) } };
79
+ }
80
+
57
81
  /** A public QQ Music or NetEase playlist by its share link: { provider, id, url, name, total, rows, skipped }. */
58
82
  export async function fetchPublicPlaylist({ input, provider, fetchImpl = fetch }) {
59
83
  let url = playlistLink(input);
@@ -70,10 +94,9 @@ export async function fetchPublicPlaylist({ input, provider, fetchImpl = fetch }
70
94
  if (!identity) throw new Error("Open this share link in your browser, then copy the full playlist page URL. Song, album and private playlist links are not supported.");
71
95
  if (provider && ["qq_music", "netease"].includes(provider) && provider !== identity.provider) throw new Error("This link belongs to another service. Go back and choose that service, or paste the matching playlist link.");
72
96
  const qq = identity.provider === "qq_music";
73
- const endpoint = qq
74
- ? `https://c.y.qq.com/qzone/fcg-bin/fcg_ucc_getcdinfo_byids_cp.fcg?type=1&json=1&utf8=1&onlysong=0&disstid=${identity.id}&format=json`
75
- : `https://music.163.com/api/playlist/detail?id=${identity.id}`;
76
- const data = await boundedJson(await responseFor(endpoint, fetchImpl, qq ? "https://y.qq.com/" : "https://music.163.com/"));
97
+ const data = qq
98
+ ? await boundedJson(await responseFor(`https://c.y.qq.com/qzone/fcg-bin/fcg_ucc_getcdinfo_byids_cp.fcg?type=1&json=1&utf8=1&onlysong=0&disstid=${identity.id}&format=json`, fetchImpl, "https://y.qq.com/"))
99
+ : await neteasePlaylist(identity.id, fetchImpl);
77
100
  const playlist = qq ? data?.cdlist?.[0] : data?.result;
78
101
  if ((qq ? data?.code !== 0 || data?.subcode !== 0 : data?.code !== 200) || !playlist
79
102
  || (qq ? Number(playlist.dir_show) === 0 : Number(playlist.privacy) === 10)) throw new Error("This playlist is unavailable publicly. Choose a playlist you can open without signing in; no account password or cookies are needed.");
@@ -91,6 +114,8 @@ export async function fetchPublicPlaylist({ input, provider, fetchImpl = fetch }
91
114
  title: musicText(qq ? song.songname : song.name),
92
115
  artist: musicText(Array.isArray(artists) ? artists.map((artist) => musicText(artist?.name)).filter(Boolean).join(", ") : ""),
93
116
  album: musicText(qq ? song.albumname : (song.album ?? song.al)?.name),
117
+ // NetEase names each album cover on its own image host; it is shown over https.
118
+ ...(!qq && /^https?:\/\/p[0-9]+\.music\.126\.net\//u.test((song.album ?? song.al)?.picUrl ?? "") ? { artwork: (song.album ?? song.al).picUrl.replace(/^http:/u, "https:") } : {}),
94
119
  playlistId: identity.id, playlistName: name, position: index + 1, member: "public-playlist.json",
95
120
  };
96
121
  }).filter((row) => row.id && row.title);
@@ -69,7 +69,7 @@ export function createDeepSeekWebResearch({ apiKey, fetch: fetchImpl = globalThi
69
69
  if (status !== 200) throw serviceError(status, body);
70
70
  const title = typeof body.title === "string" && body.title.trim() ? clip(body.title.trim(), 300) : url;
71
71
  return webResult({ provider: "deepseek", kind: "read", status: "found", retrievedAt,
72
- summary: clip(text, webLimits.textChars), truncated: text.length > webLimits.textChars,
72
+ summary: clip(text, webLimits.fullTextChars), truncated: text.length > webLimits.fullTextChars,
73
73
  sources: [{ title, url, snippet: "", published_at: null }] });
74
74
  }
75
75
 
@@ -1,3 +1,4 @@
1
+ import { readModelRoles } from "../../runtime/pi/model-roles.mjs";
1
2
  import { createPersistentCredentialStore } from "../../runtime/pi/persistent-credential-store.mjs";
2
3
  import { createBraveWebResearch } from "./brave-web.mjs";
3
4
  import { createCodexWebResearch } from "./codex-web.mjs";
@@ -5,8 +6,13 @@ import { createDeepSeekWebResearch } from "./deepseek-web.mjs";
5
6
  import { failure, webProviderName } from "./web-result.mjs";
6
7
 
7
8
  // Search goes to the first backend that is set up; reading needs a backend that can open pages.
8
- // MOONDOG_WEB_SEARCH_PROVIDER and MOONDOG_WEB_READ_PROVIDER pick one instead.
9
+ // The web search role in Settings (model-roles.json) picks one instead, and MOONDOG_WEB_SEARCH_PROVIDER
10
+ // and MOONDOG_WEB_READ_PROVIDER pick over that. As in Hermes Agent, a call that fails on the chosen
11
+ // backend is tried once more on each other backend that is set up, so one service being down or out
12
+ // of credit doesn't leave the agent without the web.
9
13
  const order = Object.freeze({ search: ["deepseek", "brave", "codex"], read: ["deepseek", "codex"] });
14
+ // Failures that another backend would only repeat.
15
+ const final = new Set(["web_cancelled", "web_input_invalid", "web_url_invalid"]);
10
16
  const providerIds = Object.freeze({ deepseek: "deepseek", brave: "brave", codex: "codex_cli" });
11
17
  const keys = Object.freeze({
12
18
  deepseek: { credential: "deepseek", variable: "DEEPSEEK_API_KEY" },
@@ -33,43 +39,103 @@ export async function createWebResearch({
33
39
  credentials = createPersistentCredentialStore(environment),
34
40
  fetch = globalThis.fetch,
35
41
  createCodex = createCodexWebResearch,
42
+ readRoles = readModelRoles,
36
43
  now,
37
44
  } = {}) {
38
- let current = { backends: {}, choices: {} };
45
+ let current = { backends: {}, choices: {}, found: {} };
46
+
47
+ // A backend whose key did not change keeps its cache. A working Codex stays; one that was not ready
48
+ // is checked again, since the listener may have signed in since.
49
+ async function backendFor(name, found, previous) {
50
+ if (name === "codex") {
51
+ const kept = previous?.backends.codex;
52
+ return kept?.publicStatus().state === "configured" ? kept : createCodex({ environment, now });
53
+ }
54
+ const kept = previous?.backends[name] && previous.found[name]?.key === found[name]?.key;
55
+ return kept ? previous.backends[name] : { deepseek: createDeepSeekWebResearch, brave: createBraveWebResearch }[name]({ apiKey: found[name]?.key, fetch, now });
56
+ }
39
57
 
40
58
  async function load(previous) {
41
59
  const found = {
42
60
  deepseek: await readKey(credentials, environment, keys.deepseek),
43
61
  brave: await readKey(credentials, environment, keys.brave),
44
62
  };
63
+ const role = (await readRoles(environment).catch(() => ({}))).web_search?.service ?? null;
45
64
  const choices = {};
46
65
  for (const kind of ["search", "read"]) {
47
66
  const wanted = environment[`MOONDOG_WEB_${kind.toUpperCase()}_PROVIDER`]?.trim().toLowerCase();
48
- if (wanted) choices[kind] = order[kind].includes(wanted) ? { name: wanted, override: true } : { name: null, reason: "web_provider_unknown", wanted };
67
+ if (wanted) choices[kind] = order[kind].includes(wanted) ? { name: wanted, selectedBy: `MOONDOG_WEB_${kind.toUpperCase()}_PROVIDER` } : { name: null, reason: "web_provider_unknown", wanted };
68
+ // Brave only searches, so with Brave chosen a read still goes to the first backend that can.
69
+ else if (role && order[kind].includes(role)) choices[kind] = { name: role, selectedBy: "settings" };
49
70
  else choices[kind] = { name: order[kind].find((name) => name !== "codex" && found[name]) ?? "codex" };
50
71
  }
51
72
  const backends = {};
52
73
  for (const name of new Set(Object.values(choices).map((choice) => choice.name).filter(Boolean))) {
53
- // A backend whose key did not change keeps its cache.
54
- const create = { deepseek: createDeepSeekWebResearch, brave: createBraveWebResearch }[name];
55
- if (create) {
56
- const kept = previous?.backends[name] && previous.found[name]?.key === found[name]?.key;
57
- backends[name] = kept ? previous.backends[name] : create({ apiKey: found[name]?.key, fetch, now });
58
- }
59
- // A working Codex stays; one that was not ready is checked again, since the listener may have signed in since.
60
- if (name === "codex") {
61
- const kept = previous?.backends.codex;
62
- backends.codex = kept?.publicStatus().state === "configured" ? kept : await createCodex({ environment, now });
63
- }
74
+ backends[name] = await backendFor(name, found, previous);
64
75
  }
65
76
  return { backends, choices, found };
66
77
  }
67
78
 
79
+ // The other backends a failed call can go to: ones with a key, and Codex once it is signed in.
80
+ async function alternatives(kind, tried) {
81
+ const names = [];
82
+ for (const name of order[kind]) {
83
+ if (name === tried) continue;
84
+ if (name !== "codex" && !current.found[name]) continue;
85
+ current.backends[name] ??= await backendFor(name, current.found, null);
86
+ if (name === "codex" && current.backends.codex.publicStatus().state !== "configured") continue;
87
+ names.push(name);
88
+ }
89
+ return names;
90
+ }
91
+
92
+ async function research(kind, input, options) {
93
+ const first = backend(kind);
94
+ const name = current.choices[kind].name;
95
+ let failed;
96
+ try {
97
+ const result = await first[kind](input, options);
98
+ // A page the chosen backend could not read may open on another.
99
+ if (kind === "search" || result.status !== "unavailable") return result;
100
+ failed = result;
101
+ } catch (error) {
102
+ if (final.has(error?.code) || options?.signal?.aborted) throw error;
103
+ failed = error;
104
+ }
105
+ for (const other of await alternatives(kind, name)) {
106
+ try {
107
+ const result = await current.backends[other][kind](input, options);
108
+ if (kind === "search" || result.status !== "unavailable") return { ...result, fallback_from: providerIds[name] };
109
+ } catch (error) {
110
+ if (final.has(error?.code) || options?.signal?.aborted) throw error;
111
+ }
112
+ }
113
+ if (failed instanceof Error) throw failed;
114
+ return failed;
115
+ }
116
+
117
+ /** Each service, whether it is set up for each operation, and which one the role chose; for Settings. */
118
+ async function services() {
119
+ const role = (await readRoles(environment).catch(() => ({}))).web_search?.service ?? null;
120
+ current.backends.codex ??= await backendFor("codex", current.found, null);
121
+ const codex = current.backends.codex.publicStatus().state === "configured";
122
+ return {
123
+ chosen: role,
124
+ search: operationStatus("search"),
125
+ read: operationStatus("read"),
126
+ services: order.search.map((name) => {
127
+ const ready = name === "codex" ? codex : Boolean(current.found[name]);
128
+ return { id: name, provider: providerIds[name], name: webProviderName(providerIds[name]), ready,
129
+ search: ready, read: ready && order.read.includes(name) };
130
+ }),
131
+ };
132
+ }
133
+
68
134
  function operationStatus(kind) {
69
135
  const choice = current.choices[kind];
70
136
  if (!choice?.name) return { provider: null, state: "unavailable", reason: choice?.reason ?? "web_provider_unknown", ...(choice?.wanted ? { requested: choice.wanted } : {}) };
71
137
  const provider = providerIds[choice.name];
72
- const override = choice.override ? { selected_by: `MOONDOG_WEB_${kind.toUpperCase()}_PROVIDER` } : {};
138
+ const override = choice.selectedBy ? { selected_by: choice.selectedBy } : {};
73
139
  if (choice.name === "codex") {
74
140
  const codex = current.backends.codex.publicStatus();
75
141
  return { provider, state: codex.state, ...(codex.reason ? { reason: codex.reason } : {}), ...override };
@@ -103,8 +169,9 @@ export async function createWebResearch({
103
169
  current = await load();
104
170
  return {
105
171
  publicStatus,
106
- search: async (input, options) => backend("search").search(input, options),
107
- read: async (input, options) => backend("read").read(input, options),
172
+ services,
173
+ search: (input, options) => research("search", input, options),
174
+ read: (input, options) => research("read", input, options),
108
175
  /** Reads keys and settings again, after /auth or /reload. */
109
176
  async refresh() {
110
177
  const previous = current;
@@ -4,8 +4,12 @@ import Ajv from "ajv";
4
4
 
5
5
  // Every web backend returns this one result shape, checked by the same rules,
6
6
  // so the agent and the screen never need to know which service answered.
7
+ // After Hermes Agent's web tools: a search gives five sources unless the agent asks for up to ten,
8
+ // and a read keeps the page's whole text, handing it out a window at a time; the agent asks for the
9
+ // next window with next_offset instead of losing everything past the first.
7
10
  export const webLimits = Object.freeze({
8
- sources: 5, snippetChars: 800, textChars: 12_000, cacheMs: 300_000, cacheEntries: 32, responseBytes: 2_000_000,
11
+ sources: 5, maxSources: 10, snippetChars: 1200, textChars: 15_000, fullTextChars: 200_000,
12
+ cacheMs: 300_000, cacheEntries: 32, responseBytes: 2_000_000,
9
13
  });
10
14
 
11
15
  // What each backend's text is, so the agent can tell a search engine's snippet from a model's summary.
@@ -19,13 +23,14 @@ export function webProviderName(provider) {
19
23
  return backends[provider]?.name ?? null;
20
24
  }
21
25
 
22
- const resultSchema = {
26
+ // A backend's result holds a page's whole text; the agent's holds one window of it.
27
+ const resultSchema = (textChars) => ({
23
28
  type: "object", additionalProperties: false,
24
29
  properties: {
25
30
  status: { type: "string", enum: ["found", "not_found", "unavailable"] },
26
- summary: { type: "string", maxLength: webLimits.textChars },
31
+ summary: { type: "string", maxLength: textChars },
27
32
  sources: {
28
- type: "array", maxItems: webLimits.sources,
33
+ type: "array", maxItems: webLimits.maxSources,
29
34
  items: {
30
35
  type: "object", additionalProperties: false,
31
36
  properties: {
@@ -39,8 +44,11 @@ const resultSchema = {
39
44
  },
40
45
  },
41
46
  required: ["status", "summary", "sources"],
42
- };
43
- const validate = new Ajv({ strict: false }).compile(resultSchema);
47
+ });
48
+ const ajv = new Ajv({ strict: false });
49
+ const validate = ajv.compile(resultSchema(webLimits.fullTextChars));
50
+ const validateWindow = ajv.compile(resultSchema(webLimits.textChars));
51
+ const offsetOf = (value) => (Number.isInteger(value) && value >= 0 ? value : null);
44
52
 
45
53
  export function failure(code, message) {
46
54
  return Object.assign(new Error(message), { code });
@@ -101,7 +109,7 @@ export function webResult({ provider, kind, status, summary = "", sources = [],
101
109
  export function projectWebResearchResult(value) {
102
110
  const payload = { status: value?.status, summary: value?.summary, sources: value?.sources };
103
111
  const backend = backends[value?.provider];
104
- if (!validate(payload) || (payload.status === "found" && !payload.sources.length) ||
112
+ if (!validateWindow(payload) || (payload.status === "found" && !payload.sources.length) ||
105
113
  !backend?.content[value.kind] || value.evidence !== backend.evidence || !Number.isFinite(Date.parse(value.retrieved_at))) {
106
114
  throw failure("web_output_invalid", `${backend?.name ?? "The web service"} returned an invalid or unsourced research result.`);
107
115
  }
@@ -110,6 +118,9 @@ export function projectWebResearchResult(value) {
110
118
  retrieved_at: new Date(value.retrieved_at).toISOString(), from_cache: value.from_cache === true,
111
119
  content_kind: backend.content[value.kind], trust: "untrusted_web_content", evidence: backend.evidence,
112
120
  ...(value.truncated === true ? { truncated: true } : {}),
121
+ ...(value.kind === "read" && offsetOf(value.total_chars) !== null ? {
122
+ text_offset: offsetOf(value.text_offset) ?? 0, total_chars: value.total_chars, next_offset: offsetOf(value.next_offset),
123
+ } : {}),
113
124
  summary: cleanText(payload.summary), sources: projectSources(payload.sources),
114
125
  };
115
126
  }
@@ -131,14 +142,46 @@ export function searchSources(items) {
131
142
  if (snippet) seen.add(snippet);
132
143
  const date = typeof item.published_at === "string" ? cleanText(item.published_at).slice(0, 100) : "";
133
144
  sources.push({ title, url, snippet, published_at: date || null });
134
- if (sources.length === webLimits.sources) break;
145
+ if (sources.length === webLimits.maxSources) break;
135
146
  }
136
147
  return sources;
137
148
  }
138
149
 
150
+ // One window of a page's text, ending at a line or word break near the limit when there is one.
151
+ function textWindow(text, offset) {
152
+ let end = Math.min(text.length, offset + webLimits.textChars);
153
+ if (end < text.length) {
154
+ const floor = offset + Math.floor(webLimits.textChars * 0.8);
155
+ const lineBreak = text.lastIndexOf("\n", end);
156
+ const space = text.lastIndexOf(" ", end);
157
+ end = lineBreak > floor ? lineBreak : space > floor ? space : end;
158
+ }
159
+ return { summary: text.slice(offset, end), end };
160
+ }
161
+
162
+ /** What one call asked for: up to `limit` sources of a search, or the window of a read from `offset`. */
163
+ function windowed(result, kind, { limit, offset }) {
164
+ if (kind === "search") return { ...result, sources: result.sources.slice(0, limit) };
165
+ const text = result.summary;
166
+ if (offset > 0 && offset >= text.length) {
167
+ throw failure("web_input_invalid", `The page has ${text.length} characters, so offset ${offset} is past its end.`);
168
+ }
169
+ const { summary, end } = textWindow(text, offset);
170
+ const more = end < text.length;
171
+ return { ...result, summary, text_offset: offset, total_chars: text.length, next_offset: more ? end : null,
172
+ ...(more || result.truncated ? { truncated: true } : {}) };
173
+ }
174
+
175
+ function integerIn(value, minimum, maximum, fallback, message) {
176
+ if (value === undefined) return fallback;
177
+ if (!Number.isInteger(value) || value < minimum || value > maximum) throw failure("web_input_invalid", message);
178
+ return value;
179
+ }
180
+
139
181
  /**
140
182
  * The parts every backend shares: one query or URL in, a cancellable request,
141
- * and a five-minute process-local cache of found results.
183
+ * and a five-minute process-local cache of found results. The cache keeps the whole result, so a
184
+ * smaller limit or a later window of the same page needs no second request.
142
185
  */
143
186
  export function createWebOperations({ unavailable = () => null, queryChars = 500, run, now = () => new Date() }) {
144
187
  const cache = new Map();
@@ -146,9 +189,14 @@ export function createWebOperations({ unavailable = () => null, queryChars = 500
146
189
  async function research(kind, input, { signal } = {}) {
147
190
  const blocked = unavailable(kind);
148
191
  if (blocked) throw blocked;
149
- if (!input || typeof input !== "object" || Object.keys(input).some((key) => key !== (kind === "search" ? "query" : "url"))) {
192
+ const allowed = kind === "search" ? ["query", "limit"] : ["url", "offset"];
193
+ if (!input || typeof input !== "object" || Object.keys(input).some((key) => !allowed.includes(key))) {
150
194
  throw failure("web_input_invalid", "Supply one search query or public URL.");
151
195
  }
196
+ const view = {
197
+ limit: integerIn(input.limit, 1, webLimits.maxSources, webLimits.sources, `Ask for 1 to ${webLimits.maxSources} search results.`),
198
+ offset: integerIn(input.offset, 0, webLimits.fullTextChars, 0, "The offset must be a whole number of characters, from 0."),
199
+ };
152
200
  const value = kind === "read" ? publicWebUrl(input.url) : typeof input.query === "string" ? input.query.trim() : input.query;
153
201
  if (typeof value !== "string" || !value) throw failure("web_input_invalid", "The web query or URL is empty.");
154
202
  if (value.length > (kind === "search" ? queryChars : 2048)) {
@@ -158,7 +206,7 @@ export function createWebOperations({ unavailable = () => null, queryChars = 500
158
206
  const key = `${kind}:${value}`;
159
207
  const cached = cache.get(key);
160
208
  if (cached && now().getTime() - Date.parse(cached.retrieved_at) < webLimits.cacheMs) {
161
- return { ...structuredClone(cached), from_cache: true };
209
+ return windowed({ ...structuredClone(cached), from_cache: true }, kind, view);
162
210
  }
163
211
  const controller = new AbortController();
164
212
  active.add(controller);
@@ -168,7 +216,7 @@ export function createWebOperations({ unavailable = () => null, queryChars = 500
168
216
  cache.set(key, result);
169
217
  if (cache.size > webLimits.cacheEntries) cache.delete(cache.keys().next().value);
170
218
  }
171
- return structuredClone(result);
219
+ return windowed(structuredClone(result), kind, view);
172
220
  } finally {
173
221
  active.delete(controller);
174
222
  }