backend-manager 5.9.24 → 5.9.26
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +6 -2
- package/docs/ghostii.md +27 -7
- package/package.json +9 -9
- package/src/manager/events/cron/daily/blog-auto-publisher.js +75 -15
- package/src/manager/libraries/content/source-resolver.js +30 -1
- package/src/test/fixtures/firebase-project/.temp/test-mode.json +1 -1
- package/src/test/fixtures/firebase-project/database-debug.log +8 -8
- package/src/test/fixtures/firebase-project/firestore-debug.log +56 -58
- package/src/test/fixtures/firebase-project/pubsub-debug.log +3 -3
package/CLAUDE.md
CHANGED
|
@@ -41,10 +41,13 @@ All `npx mgr <cmd>` aliases work: `npx bm <cmd>`, `npx bem <cmd>`, `npx backend-
|
|
|
41
41
|
|
|
42
42
|
### For Framework Development (This Repository)
|
|
43
43
|
|
|
44
|
+
> **🚫 NEVER use `npx mgr ...` from the framework repo.** `npx mgr` is for CONSUMER projects (where the bin is linked in `functions/node_modules/.bin/`). From the framework repo, use `npm test`, `npm run prepare`, etc. — the `scripts` in `package.json` call the local `bin/` directly.
|
|
45
|
+
|
|
44
46
|
1. `npm install` — install BEM's own deps
|
|
45
47
|
2. `npm run prepare` — build once: copies `src/` → `dist/` via prepare-package
|
|
46
48
|
3. `npm run prepare:watch` — watch mode
|
|
47
|
-
4.
|
|
49
|
+
4. `npm test` — run framework tests via the bundled fixture project (equivalent to `node bin/backend-manager test`). Accepts the same target syntax: `npm test -- mgr:helpers/content/blog-auto-publisher`
|
|
50
|
+
5. Test in the **designated test consumer** — `../ultimate-jekyll-backend` is BEM's consumer for validating framework changes end-to-end (exercise any consumer-level flow there freely: emulator, tests, deploy paths). From inside it, run `npx mgr install dev` to swap BEM to this local repo — required whenever you edit the framework source and want the consumer to pick up the changes (the consumer otherwise keeps its installed `node_modules/backend-manager`). Reverse with `npx mgr install live`. If `npx mgr` then errors with "could not determine executable to run", the local install skipped bin-linking — re-run `npm install` to relink, or call `node node_modules/backend-manager/bin/backend-manager <cmd>` directly.
|
|
48
51
|
|
|
49
52
|
## Architecture
|
|
50
53
|
|
|
@@ -91,7 +94,8 @@ See [docs/cli-firestore-auth.md](docs/cli-firestore-auth.md) and [docs/cli-logs.
|
|
|
91
94
|
|
|
92
95
|
## Development Workflow
|
|
93
96
|
|
|
94
|
-
- **🚫 NEVER
|
|
97
|
+
- **🚫 NEVER use `npx mgr ...` from the framework repo** — `npx mgr` is for CONSUMER projects only (where the bin lives in `functions/node_modules/.bin/`). From the framework repo, use `npm test`, `npm run prepare`, etc. — the `scripts` in `package.json` call `node bin/backend-manager` directly. This applies to ALL four OMEGA frameworks (BEM/UJM/BXM/EM).
|
|
98
|
+
- **🚫 NEVER run `npx mgr serve` / `npx mgr emulator`** (consumer projects) — they're the user's long-running dev processes. Assume they're already running; if they aren't, **instruct the user to run them** rather than running them yourself (running them again kills theirs). To see output, **read the `functions/*.log` files** (`dev.log`, `emulator.log`, `test.log`) — never tail/attach to the process. Running `npx mgr test` is fine (it auto-starts its own emulator if needed).
|
|
95
99
|
- **Where the output logs live:** BEM CLI commands tee output to `<projectDir>/functions/` (not `logs/` — BEM's deliberate exception, co-located with firebase-tools' `*-debug.log`): `dev.log` (`npx mgr serve`), `deploy.log` (`npx mgr deploy`), `emulator.log` (`npx mgr emulator` / test with own emulator), `test.log` (`npx mgr test`), `production.log` (`npx mgr logs`). The `dev`/`test` names match EM/BXM/UJM; see [docs/logging.md](docs/logging.md).
|
|
96
100
|
- **If the user reports an error**, check the emulator/test output for the root cause before guessing.
|
|
97
101
|
- **Live-test UI changes via CDP.** When working on admin dashboards or browser-facing endpoints, use the `chrome-devtools` MCP tools (screenshots, click, evaluate JS, console logs) to verify the change works in the running browser. See `~/.claude/mcp-server/servers/chrome-devtools/CLAUDE.md`.
|
package/docs/ghostii.md
CHANGED
|
@@ -49,18 +49,19 @@ Each `blog.content[]` entry has a `sources` array. The cron picks one at random
|
|
|
49
49
|
|
|
50
50
|
1. Fetch and parse the RSS/Atom/JSON feed via `feed-parser.parseFeed()`
|
|
51
51
|
2. Query `content-sources` in Firestore for already-processed items
|
|
52
|
-
3. Filter to unprocessed items
|
|
53
|
-
4.
|
|
54
|
-
5.
|
|
55
|
-
6.
|
|
56
|
-
7.
|
|
52
|
+
3. Filter to unprocessed items; also filter items whose title is too similar to recently published articles (cross-feed dedup via `isTitleTooSimilar`)
|
|
53
|
+
4. Pick the first remaining item
|
|
54
|
+
5. Extract full article content from the item URL via `feed-parser.extractArticleContent()`
|
|
55
|
+
6. Pass extracted text as `sourceContent` to the provider API (separate from the `description` prompt)
|
|
56
|
+
7. After publish, write a tracking doc to `content-sources/{hash}`
|
|
57
|
+
8. On failure (feed unreachable, unparseable, exhausted): return null so `harvest()` tries the next source. Only falls back to `$brand` if `$brand` is explicitly in the entry's `sources` array.
|
|
57
58
|
|
|
58
59
|
### Parent source flow
|
|
59
60
|
|
|
60
61
|
1. Fetch available sources from parent server via `GET {parentUrl}/newsletter-sources` (no `claimFor` -- read-only)
|
|
61
62
|
2. Query `content-sources` in Firestore for already-used sources with `origin: '$parent'`
|
|
62
63
|
3. Filter out already-used sources, pick the first available
|
|
63
|
-
4. On failure (no parent URL, no sources, all exhausted):
|
|
64
|
+
4. On failure (no parent URL, no sources, all exhausted): return null so `harvest()` tries the next source. Only falls back to `$brand` if `$brand` is explicitly in the entry's `sources` array.
|
|
64
65
|
|
|
65
66
|
### Content source tracking
|
|
66
67
|
|
|
@@ -74,7 +75,8 @@ content-sources/{sha256(origin + '::' + url).slice(0,20)}: {
|
|
|
74
75
|
origin: '$feed:https://...', // or '$parent', '$brand'
|
|
75
76
|
feedUrl: 'https://...', // for feed sources
|
|
76
77
|
itemId: 'guid-or-url',
|
|
77
|
-
itemTitle: '...',
|
|
78
|
+
itemTitle: '...', // source article title
|
|
79
|
+
postTitle: '...', // generated article title (for topic dedup)
|
|
78
80
|
usedBy: 'blog', // or 'newsletter'
|
|
79
81
|
brandId: '...',
|
|
80
82
|
postUrl: '...',
|
|
@@ -86,6 +88,24 @@ content-sources/{sha256(origin + '::' + url).slice(0,20)}: {
|
|
|
86
88
|
}
|
|
87
89
|
```
|
|
88
90
|
|
|
91
|
+
### Topic deduplication
|
|
92
|
+
|
|
93
|
+
The blog auto-publisher prevents near-duplicate articles at three levels:
|
|
94
|
+
|
|
95
|
+
1. **Per-feed dedup**: `processFeedSource()` queries `content-sources` by `feedUrl` to skip items already used from that specific feed.
|
|
96
|
+
|
|
97
|
+
2. **Cross-feed dedup** (structural): Before picking a feed item, `processFeedSource()` compares each candidate's title against ALL recent titles (from any source) using `isTitleTooSimilar()` — a word-overlap check with basic stemming. This catches the same news event reported by different publications (e.g. Guardian and NYT both covering Apple's price hike). Items with >= 50% significant-word overlap are skipped.
|
|
98
|
+
|
|
99
|
+
3. **Topic dedup** (prompt-based): Before generating each article, `harvest()` appends all recent titles (Firestore history + same-run) to the Ghostii prompt as a strict avoidance list. The prompt forbids writing about the same topic, theme, or keyword combination — not just the same story.
|
|
100
|
+
|
|
101
|
+
Within a single cron run, both generated post titles AND source item titles are tracked in `runTitles` and fed into levels 2 and 3 for subsequent articles.
|
|
102
|
+
|
|
103
|
+
All source types (`$feed`, `$parent`, `$brand`, URL, text) are tracked in `content-sources` after publishing.
|
|
104
|
+
|
|
105
|
+
### Source fallback behavior
|
|
106
|
+
|
|
107
|
+
When a source fails or is exhausted, `resolveSource()` only falls back to `$brand` if `$brand` is explicitly listed in the entry's `sources` array. Otherwise it returns null, and `harvest()` tries the next source in a shuffled copy of the pool. If all sources are exhausted, that article slot is skipped.
|
|
108
|
+
|
|
89
109
|
## Configuration
|
|
90
110
|
|
|
91
111
|
```js
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "backend-manager",
|
|
3
|
-
"version": "5.9.
|
|
3
|
+
"version": "5.9.26",
|
|
4
4
|
"description": "Quick tools for developing Firebase functions",
|
|
5
5
|
"main": "src/manager/index.js",
|
|
6
6
|
"files": [
|
|
@@ -70,22 +70,22 @@
|
|
|
70
70
|
"@anthropic-ai/sdk": "^0.99.0",
|
|
71
71
|
"@firebase/rules-unit-testing": "^5.0.1",
|
|
72
72
|
"@google-cloud/firestore": "^7.11.6",
|
|
73
|
-
"@google-cloud/pubsub": "^5.3.
|
|
74
|
-
"@google-cloud/storage": "^7.
|
|
75
|
-
"@inquirer/prompts": "^8.5.
|
|
73
|
+
"@google-cloud/pubsub": "^5.3.1",
|
|
74
|
+
"@google-cloud/storage": "^7.21.0",
|
|
75
|
+
"@inquirer/prompts": "^8.5.2",
|
|
76
76
|
"@modelcontextprotocol/sdk": "^1.29.0",
|
|
77
77
|
"@octokit/rest": "^22.0.1",
|
|
78
78
|
"@resvg/resvg-js": "^2.6.2",
|
|
79
79
|
"@sendgrid/mail": "^8.1.6",
|
|
80
80
|
"@sentry/node": "^10.54.0",
|
|
81
|
-
"body-parser": "^2.
|
|
81
|
+
"body-parser": "^2.3.0",
|
|
82
82
|
"busboy": "^1.6.0",
|
|
83
83
|
"chalk": "^5.6.2",
|
|
84
84
|
"cheerio": "^1.2.0",
|
|
85
85
|
"cors": "^2.8.6",
|
|
86
86
|
"dotenv": "^17.4.2",
|
|
87
87
|
"express": "^5.2.1",
|
|
88
|
-
"fast-xml-parser": "^5.9.
|
|
88
|
+
"fast-xml-parser": "^5.9.3",
|
|
89
89
|
"fs-jetpack": "^5.1.0",
|
|
90
90
|
"glob": "^13.0.6",
|
|
91
91
|
"hcaptcha": "^0.2.0",
|
|
@@ -103,11 +103,11 @@
|
|
|
103
103
|
"node-powertools": "^3.0.0",
|
|
104
104
|
"npm-api": "^1.0.1",
|
|
105
105
|
"pushid": "^1.0.0",
|
|
106
|
-
"sanitize-html": "^2.17.
|
|
107
|
-
"sharp": "^0.
|
|
106
|
+
"sanitize-html": "^2.17.5",
|
|
107
|
+
"sharp": "^0.35.2",
|
|
108
108
|
"stripe": "^22.2.0",
|
|
109
109
|
"uid-generator": "^2.0.0",
|
|
110
|
-
"uuid": "^14.0.
|
|
110
|
+
"uuid": "^14.0.1",
|
|
111
111
|
"wonderful-fetch": "^2.0.5",
|
|
112
112
|
"wonderful-log": "^1.0.7",
|
|
113
113
|
"wonderful-version": "^1.3.2",
|
|
@@ -15,7 +15,8 @@
|
|
|
15
15
|
*
|
|
16
16
|
* All feed/parent sources are tracked in Firestore (`content-sources`) so the
|
|
17
17
|
* same article is never processed twice. When a feed is unreachable or
|
|
18
|
-
* exhausted, the
|
|
18
|
+
* exhausted, harvest() tries the next source in the pool. Only falls
|
|
19
|
+
* back to $brand if '$brand' is explicitly listed in the entry's sources.
|
|
19
20
|
*
|
|
20
21
|
* Source resolution, tracking, and prompt constants live in the shared
|
|
21
22
|
* source-resolver library (src/manager/libraries/content/source-resolver.js).
|
|
@@ -32,6 +33,7 @@ const {
|
|
|
32
33
|
trackContentSource,
|
|
33
34
|
contentSourceHash,
|
|
34
35
|
getProcessedItemIds,
|
|
36
|
+
getRecentTitles,
|
|
35
37
|
getURLContent,
|
|
36
38
|
isURL,
|
|
37
39
|
} = require('../../../libraries/content/source-resolver.js');
|
|
@@ -118,19 +120,49 @@ function fetchRemoteBrand(brandUrl) {
|
|
|
118
120
|
}
|
|
119
121
|
|
|
120
122
|
async function harvest(assistant, entry, admin, provider, Manager) {
|
|
121
|
-
const date = moment().format('MMMM YYYY');
|
|
122
|
-
|
|
123
123
|
assistant.log(`harvest(): Starting ${entry.brand.brand.id}...`);
|
|
124
124
|
|
|
125
|
+
const recentTitles = admin
|
|
126
|
+
? await getRecentTitles(admin, entry.brand.brand.id).catch((e) => {
|
|
127
|
+
assistant.error('harvest(): Error fetching recent titles (continuing)', e);
|
|
128
|
+
return [];
|
|
129
|
+
})
|
|
130
|
+
: [];
|
|
131
|
+
const runTitles = [];
|
|
132
|
+
|
|
133
|
+
if (recentTitles.length) {
|
|
134
|
+
assistant.log(`harvest(): ${recentTitles.length} recent titles loaded for topic dedup`);
|
|
135
|
+
}
|
|
136
|
+
|
|
125
137
|
for (let index = 0; index < entry.quantity; index++) {
|
|
126
|
-
const
|
|
138
|
+
const allKnownTitles = [...recentTitles, ...runTitles];
|
|
127
139
|
|
|
128
|
-
|
|
140
|
+
let resolved = null;
|
|
141
|
+
const shuffled = randomize([...entry.sources]);
|
|
142
|
+
for (const source of shuffled) {
|
|
143
|
+
assistant.log(`harvest(): Processing ${index + 1}/${entry.quantity}`, source);
|
|
129
144
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
145
|
+
resolved = await resolveSource(assistant, source, entry, admin, Manager).catch((e) => {
|
|
146
|
+
assistant.error('harvest(): Error resolving source', e);
|
|
147
|
+
return null;
|
|
148
|
+
});
|
|
149
|
+
if (resolved) { break; }
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
if (!resolved) {
|
|
153
|
+
assistant.log('harvest(): All sources exhausted for this article, skipping');
|
|
154
|
+
continue;
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
if (allKnownTitles.length) {
|
|
158
|
+
resolved.description += '\n\nTOPIC DEDUPLICATION (STRICT):\n'
|
|
159
|
+
+ 'These articles were RECENTLY published on our blog. You MUST NOT write about the same topic, theme, or subject area as ANY of them.\n'
|
|
160
|
+
+ 'Do NOT cover the same story from a different angle, do NOT write about the same theme with different examples, '
|
|
161
|
+
+ 'and do NOT reuse the same primary keyword/category combination. '
|
|
162
|
+
+ 'If a recent article covers "lifestyle tech + habits", do NOT write about "lifestyle tech + policy" — that is the SAME theme. '
|
|
163
|
+
+ 'Pick an entirely different domain.\n'
|
|
164
|
+
+ 'Recent articles:\n'
|
|
165
|
+
+ allKnownTitles.slice(0, 25).map((t) => `- ${t}`).join('\n');
|
|
134
166
|
}
|
|
135
167
|
|
|
136
168
|
assistant.log('harvest(): Resolved source', resolved);
|
|
@@ -154,6 +186,9 @@ async function harvest(assistant, entry, admin, provider, Manager) {
|
|
|
154
186
|
|
|
155
187
|
assistant.log('harvest(): Article', article);
|
|
156
188
|
|
|
189
|
+
const post = provider.blocksToPost?.(article.json);
|
|
190
|
+
const generatedTitle = post?.title || article.title || '';
|
|
191
|
+
|
|
157
192
|
const publishArgs = {
|
|
158
193
|
brand: entry.brand,
|
|
159
194
|
article,
|
|
@@ -172,12 +207,28 @@ async function harvest(assistant, entry, admin, provider, Manager) {
|
|
|
172
207
|
|
|
173
208
|
assistant.log('harvest(): Uploaded post', uploadedPost);
|
|
174
209
|
|
|
175
|
-
if (
|
|
210
|
+
if (generatedTitle) {
|
|
211
|
+
runTitles.push(generatedTitle);
|
|
212
|
+
}
|
|
213
|
+
if (resolved.trackingData?.itemTitle) {
|
|
214
|
+
runTitles.push(resolved.trackingData.itemTitle);
|
|
215
|
+
}
|
|
216
|
+
|
|
217
|
+
if (!resolved.trackingData) {
|
|
218
|
+
resolved.trackingData = {
|
|
219
|
+
url: uploadedPost.slug || `brand-${postId}`,
|
|
220
|
+
origin: '$brand',
|
|
221
|
+
usedBy: 'blog',
|
|
222
|
+
};
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
if (admin) {
|
|
176
226
|
await trackContentSource(admin, {
|
|
177
227
|
...resolved.trackingData,
|
|
178
228
|
brandId: entry.brand.brand.id,
|
|
179
229
|
postUrl: uploadedPost.url,
|
|
180
230
|
postSlug: uploadedPost.slug,
|
|
231
|
+
postTitle: generatedTitle || null,
|
|
181
232
|
}).catch((e) => {
|
|
182
233
|
assistant.error('harvest(): Error tracking content source (non-fatal)', e);
|
|
183
234
|
});
|
|
@@ -205,8 +256,11 @@ async function resolveSource(assistant, source, entry, admin, Manager) {
|
|
|
205
256
|
const feedResult = await processFeedSource(assistant, feedUrl, entry.brand.brand.id, admin).catch((e) => e);
|
|
206
257
|
|
|
207
258
|
if (feedResult instanceof Error || !feedResult) {
|
|
208
|
-
assistant.log('resolveSource(): Feed failed or exhausted
|
|
209
|
-
|
|
259
|
+
assistant.log('resolveSource(): Feed failed or exhausted');
|
|
260
|
+
if (entry.sources.includes('$brand')) {
|
|
261
|
+
return resolveSource(assistant, '$brand', entry, admin, Manager);
|
|
262
|
+
}
|
|
263
|
+
return null;
|
|
210
264
|
}
|
|
211
265
|
|
|
212
266
|
const description = powertools.template(PROMPT_SOURCE, {
|
|
@@ -235,8 +289,11 @@ async function resolveSource(assistant, source, entry, admin, Manager) {
|
|
|
235
289
|
const parentResults = await processParentSource(assistant, entry, admin, Manager).catch((e) => e);
|
|
236
290
|
|
|
237
291
|
if (parentResults instanceof Error || !parentResults?.length) {
|
|
238
|
-
assistant.log('resolveSource(): Parent source failed or exhausted
|
|
239
|
-
|
|
292
|
+
assistant.log('resolveSource(): Parent source failed or exhausted');
|
|
293
|
+
if (entry.sources.includes('$brand')) {
|
|
294
|
+
return resolveSource(assistant, '$brand', entry, admin, Manager);
|
|
295
|
+
}
|
|
296
|
+
return null;
|
|
240
297
|
}
|
|
241
298
|
|
|
242
299
|
const parentResult = parentResults[0];
|
|
@@ -275,7 +332,10 @@ async function resolveSource(assistant, source, entry, admin, Manager) {
|
|
|
275
332
|
|
|
276
333
|
if (suggestion instanceof Error) {
|
|
277
334
|
assistant.error(`resolveSource(): Error fetching URL ${source}`, suggestion);
|
|
278
|
-
|
|
335
|
+
if (entry.sources.includes('$brand')) {
|
|
336
|
+
return resolveSource(assistant, '$brand', entry, admin, Manager);
|
|
337
|
+
}
|
|
338
|
+
return null;
|
|
279
339
|
}
|
|
280
340
|
|
|
281
341
|
const description = powertools.template(PROMPT, {
|
|
@@ -269,7 +269,7 @@ async function resolveNewsletterSources({ sources, categories, admin, Manager, a
|
|
|
269
269
|
|
|
270
270
|
// ── Firestore tracking ──────────────────────────────────────────────
|
|
271
271
|
|
|
272
|
-
async function trackContentSource(admin, { url, origin, feedUrl, itemId, itemTitle, usedBy, brandId, postUrl, postSlug }) {
|
|
272
|
+
async function trackContentSource(admin, { url, origin, feedUrl, itemId, itemTitle, usedBy, brandId, postUrl, postSlug, postTitle }) {
|
|
273
273
|
const docId = contentSourceHash(origin || '', url || '');
|
|
274
274
|
const nowISO = new Date().toISOString();
|
|
275
275
|
const nowUNIX = Math.round(Date.now() / 1000);
|
|
@@ -280,6 +280,7 @@ async function trackContentSource(admin, { url, origin, feedUrl, itemId, itemTit
|
|
|
280
280
|
feedUrl: feedUrl || null,
|
|
281
281
|
itemId: itemId || null,
|
|
282
282
|
itemTitle: itemTitle || null,
|
|
283
|
+
postTitle: postTitle || null,
|
|
283
284
|
usedBy: usedBy || null,
|
|
284
285
|
brandId: brandId || null,
|
|
285
286
|
postUrl: postUrl || null,
|
|
@@ -374,6 +375,33 @@ function tryParse(json) {
|
|
|
374
375
|
}
|
|
375
376
|
}
|
|
376
377
|
|
|
378
|
+
// ── Recent title loading (topic dedup) ───────────────────────────────
|
|
379
|
+
|
|
380
|
+
async function getRecentTitles(admin, brandId, days) {
|
|
381
|
+
if (!admin) { return []; }
|
|
382
|
+
|
|
383
|
+
days = days || 14;
|
|
384
|
+
const cutoff = Math.round(Date.now() / 1000) - (days * 86400);
|
|
385
|
+
|
|
386
|
+
const snapshot = await admin.firestore()
|
|
387
|
+
.collection(CONTENT_SOURCES_COLLECTION)
|
|
388
|
+
.where('brandId', '==', brandId)
|
|
389
|
+
.select('postTitle', 'itemTitle', 'metadata')
|
|
390
|
+
.get();
|
|
391
|
+
|
|
392
|
+
const titles = new Set();
|
|
393
|
+
snapshot.docs.forEach((doc) => {
|
|
394
|
+
const data = doc.data();
|
|
395
|
+
const created = data.metadata?.created?.timestampUNIX || 0;
|
|
396
|
+
if (created < cutoff) { return; }
|
|
397
|
+
|
|
398
|
+
if (data.postTitle) { titles.add(data.postTitle); }
|
|
399
|
+
if (data.itemTitle) { titles.add(data.itemTitle); }
|
|
400
|
+
});
|
|
401
|
+
|
|
402
|
+
return [...titles];
|
|
403
|
+
}
|
|
404
|
+
|
|
377
405
|
// ── Exports ─────────────────────────────────────────────────────────
|
|
378
406
|
|
|
379
407
|
module.exports = {
|
|
@@ -391,6 +419,7 @@ module.exports = {
|
|
|
391
419
|
trackContentSource,
|
|
392
420
|
contentSourceHash,
|
|
393
421
|
getProcessedItemIds,
|
|
422
|
+
getRecentTitles,
|
|
394
423
|
getURLContent,
|
|
395
424
|
isURL,
|
|
396
425
|
extractBodyContent,
|
|
@@ -2,11 +2,11 @@ WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
|
|
|
2
2
|
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by akka.util.Unsafe (file:/Users/ian/.cache/firebase/emulators/firebase-database-emulator-v4.11.2.jar)
|
|
3
3
|
WARNING: Please consider reporting this to the maintainers of class akka.util.Unsafe
|
|
4
4
|
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
5
|
+
02:47:10.430 [NamespaceSystem-akka.actor.default-dispatcher-4] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
|
|
6
|
+
02:47:10.526 [main] INFO com.firebase.server.forge.App$ - Listening at 127.0.0.1:9000
|
|
7
|
+
02:47:20.571 [NamespaceSystem-akka.actor.default-dispatcher-4] INFO com.firebase.core.namespace.NamespaceActor - demo-backend-manager-default-rtdb successfully activated FBKV (SurveyIdle(0)) wait: 68ms, init: 0ms
|
|
8
|
+
02:47:20.599 [NamespaceSystem-blocking-namespace-operation-dispatcher-6] INFO com.firebase.core.namespace.StateManager - Namespace demo-backend-manager-default-rtdb status Active to Active
|
|
9
|
+
02:47:32.673 [Thread-0] INFO com.firebase.server.forge.App$ - Attempting graceful shutdown.
|
|
10
|
+
02:47:32.678 [NamespaceSystem-akka.actor.default-dispatcher-4] INFO com.firebase.core.namespace.Terminator$Terminator - 1 actors left to terminate: demo-backend-manager-default-rtdb
|
|
11
|
+
02:47:32.681 [NamespaceSystem-akka.actor.default-dispatcher-5] INFO com.firebase.core.namespace.NamespaceActor - stopped namespace actor for demo-backend-manager-default-rtdb
|
|
12
|
+
02:47:32.684 [Thread-0] INFO com.firebase.server.forge.App$ - Graceful shutdown complete.
|
|
@@ -2,7 +2,7 @@ WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
|
|
|
2
2
|
WARNING: sun.misc.Unsafe::allocateMemory has been called by io.netty.util.internal.PlatformDependent0$2 (file:/Users/ian/.cache/firebase/emulators/cloud-firestore-emulator-v1.21.0.jar)
|
|
3
3
|
WARNING: Please consider reporting this to the maintainers of class io.netty.util.internal.PlatformDependent0$2
|
|
4
4
|
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
|
|
5
|
-
Jun
|
|
5
|
+
Jun 30, 2026 2:47:09 AM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start
|
|
6
6
|
INFO: Started WebSocket server on ws://127.0.0.1:9150
|
|
7
7
|
|
|
8
8
|
API endpoint: http://127.0.0.1:8080
|
|
@@ -20,117 +20,115 @@ If you are running a Firestore in Datastore Mode project, run:
|
|
|
20
20
|
Note: Support for Datastore Mode is in preview. If you encounter any bugs please file at https://github.com/firebase/firebase-tools/issues.
|
|
21
21
|
Dev App Server is now running.
|
|
22
22
|
|
|
23
|
-
Jun
|
|
23
|
+
Jun 30, 2026 2:47:19 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
24
24
|
INFO: Detected non-HTTP/2 connection.
|
|
25
|
-
Jun
|
|
25
|
+
Jun 30, 2026 2:47:20 AM io.grpc.netty.TcpMetrics loadEpollInfo
|
|
26
26
|
INFO: Epoll available during static init of TcpMetrics:false
|
|
27
|
-
Jun
|
|
27
|
+
Jun 30, 2026 2:47:20 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
28
28
|
INFO: Detected HTTP/2 connection.
|
|
29
|
-
Jun
|
|
29
|
+
Jun 30, 2026 2:47:21 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
30
30
|
INFO: Detected HTTP/2 connection.
|
|
31
|
-
Jun
|
|
31
|
+
Jun 30, 2026 2:47:21 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
32
32
|
INFO: Detected HTTP/2 connection.
|
|
33
|
-
Jun
|
|
33
|
+
Jun 30, 2026 2:47:30 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
34
34
|
INFO: Detected HTTP/2 connection.
|
|
35
|
-
Jun
|
|
35
|
+
Jun 30, 2026 2:47:30 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
36
36
|
INFO: Detected HTTP/2 connection.
|
|
37
|
-
Jun
|
|
37
|
+
Jun 30, 2026 2:47:30 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
38
38
|
INFO: Detected HTTP/2 connection.
|
|
39
|
-
Jun
|
|
39
|
+
Jun 30, 2026 2:47:30 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
40
40
|
INFO: Detected HTTP/2 connection.
|
|
41
|
-
Jun
|
|
41
|
+
Jun 30, 2026 2:47:30 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
42
42
|
INFO: Detected HTTP/2 connection.
|
|
43
|
-
Jun
|
|
43
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
44
44
|
INFO: Detected HTTP/2 connection.
|
|
45
|
-
Jun
|
|
45
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
46
46
|
INFO: Detected HTTP/2 connection.
|
|
47
|
-
Jun
|
|
47
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
48
48
|
INFO: Detected HTTP/2 connection.
|
|
49
|
-
Jun
|
|
49
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
50
50
|
INFO: Detected HTTP/2 connection.
|
|
51
|
-
Jun
|
|
51
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
52
52
|
INFO: Detected HTTP/2 connection.
|
|
53
|
-
Jun
|
|
53
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
54
54
|
INFO: Detected HTTP/2 connection.
|
|
55
|
-
Jun
|
|
55
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
56
56
|
INFO: Detected HTTP/2 connection.
|
|
57
|
-
Jun
|
|
57
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
58
58
|
INFO: Detected HTTP/2 connection.
|
|
59
|
-
Jun
|
|
59
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
60
60
|
INFO: Detected HTTP/2 connection.
|
|
61
|
-
Jun
|
|
61
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
62
62
|
INFO: Detected HTTP/2 connection.
|
|
63
|
-
Jun
|
|
63
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
64
64
|
INFO: Detected HTTP/2 connection.
|
|
65
|
-
Jun
|
|
65
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
66
66
|
INFO: Detected HTTP/2 connection.
|
|
67
|
-
Jun
|
|
67
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
68
68
|
INFO: Detected HTTP/2 connection.
|
|
69
|
-
Jun
|
|
69
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
70
70
|
INFO: Detected HTTP/2 connection.
|
|
71
|
-
Jun
|
|
71
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
72
72
|
INFO: Detected HTTP/2 connection.
|
|
73
|
-
Jun
|
|
73
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
74
74
|
INFO: Detected HTTP/2 connection.
|
|
75
|
-
Jun
|
|
75
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
76
76
|
INFO: Detected HTTP/2 connection.
|
|
77
|
-
Jun
|
|
77
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
78
78
|
INFO: Detected HTTP/2 connection.
|
|
79
|
-
Jun
|
|
79
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
80
80
|
INFO: Detected HTTP/2 connection.
|
|
81
|
-
Jun
|
|
81
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
82
82
|
INFO: Detected HTTP/2 connection.
|
|
83
|
-
Jun
|
|
83
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
84
84
|
INFO: Detected HTTP/2 connection.
|
|
85
|
-
Jun
|
|
85
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
86
86
|
INFO: Detected HTTP/2 connection.
|
|
87
|
-
Jun
|
|
87
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
88
88
|
INFO: Detected HTTP/2 connection.
|
|
89
|
-
Jun
|
|
89
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
90
90
|
INFO: Detected HTTP/2 connection.
|
|
91
|
-
Jun
|
|
91
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
92
92
|
INFO: Detected HTTP/2 connection.
|
|
93
|
-
Jun
|
|
93
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
94
94
|
INFO: Detected HTTP/2 connection.
|
|
95
|
-
Jun
|
|
95
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
96
96
|
INFO: Detected HTTP/2 connection.
|
|
97
|
-
Jun
|
|
97
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
98
98
|
INFO: Detected HTTP/2 connection.
|
|
99
|
-
Jun
|
|
99
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
100
100
|
INFO: Detected HTTP/2 connection.
|
|
101
|
-
Jun
|
|
101
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
102
102
|
INFO: Detected HTTP/2 connection.
|
|
103
|
-
Jun
|
|
103
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
104
104
|
INFO: Detected HTTP/2 connection.
|
|
105
|
-
Jun
|
|
105
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
106
106
|
INFO: Detected HTTP/2 connection.
|
|
107
|
-
Jun
|
|
107
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
108
108
|
INFO: Detected HTTP/2 connection.
|
|
109
|
-
Jun
|
|
109
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
110
110
|
INFO: Detected HTTP/2 connection.
|
|
111
|
-
Jun
|
|
111
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
112
112
|
INFO: Detected HTTP/2 connection.
|
|
113
|
-
Jun
|
|
113
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
114
114
|
INFO: Detected HTTP/2 connection.
|
|
115
|
-
Jun
|
|
115
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
116
116
|
INFO: Detected HTTP/2 connection.
|
|
117
|
-
Jun
|
|
117
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
118
118
|
INFO: Detected HTTP/2 connection.
|
|
119
|
-
Jun
|
|
119
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
120
120
|
INFO: Detected HTTP/2 connection.
|
|
121
|
-
Jun
|
|
121
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
122
122
|
INFO: Detected HTTP/2 connection.
|
|
123
|
-
Jun
|
|
123
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
124
124
|
INFO: Detected HTTP/2 connection.
|
|
125
|
-
Jun
|
|
125
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
126
126
|
INFO: Detected HTTP/2 connection.
|
|
127
|
-
Jun
|
|
127
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
128
128
|
INFO: Detected HTTP/2 connection.
|
|
129
|
-
Jun
|
|
129
|
+
Jun 30, 2026 2:47:31 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
130
130
|
INFO: Detected HTTP/2 connection.
|
|
131
|
-
Jun
|
|
132
|
-
INFO: Detected HTTP/2 connection.
|
|
133
|
-
Jun 29, 2026 11:29:32 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
131
|
+
Jun 30, 2026 2:47:32 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
134
132
|
INFO: Detected HTTP/2 connection.
|
|
135
133
|
*** shutting down gRPC server since JVM is shutting down
|
|
136
134
|
*** server shut down
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
This is the Google Pub/Sub fake.
|
|
2
2
|
Implementation may be incomplete or differ from the real system.
|
|
3
|
-
Jun
|
|
3
|
+
Jun 30, 2026 2:47:13 AM com.google.cloud.pubsub.testing.v1.Main main
|
|
4
4
|
INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
|
|
5
5
|
SLF4J(W): No SLF4J providers were found.
|
|
6
6
|
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
|
|
@@ -9,9 +9,9 @@ WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
|
|
|
9
9
|
WARNING: sun.misc.Unsafe::allocateMemory has been called by io.netty.util.internal.PlatformDependent0$2 (file:/Users/ian/.cache/firebase/emulators/pubsub-emulator-0.8.33/pubsub-emulator/lib/cloud-pubsub-emulator-0.8.33-all.jar)
|
|
10
10
|
WARNING: Please consider reporting this to the maintainers of class io.netty.util.internal.PlatformDependent0$2
|
|
11
11
|
WARNING: sun.misc.Unsafe::allocateMemory will be removed in a future release
|
|
12
|
-
Jun
|
|
12
|
+
Jun 30, 2026 2:47:14 AM com.google.cloud.pubsub.testing.v1.Main main
|
|
13
13
|
INFO: Server started, listening on 8085
|
|
14
|
-
Jun
|
|
14
|
+
Jun 30, 2026 2:47:20 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
|
|
15
15
|
INFO: Detected HTTP/2 connection.
|
|
16
16
|
*** shutting down gRPC server since JVM is shutting down
|
|
17
17
|
*** server shut down
|