birdclaw 0.6.0 → 0.8.0

package/CHANGELOG.md

@@ -1,5 +1,69 @@
 # CHANGELOG
 
+## 0.8.0 - 2026-06-10
+
+### Added
+
+- Add localized Today/digest reports through `--language <locale-id>`, `BIRDCLAW_DIGEST_LANGUAGE`, and the period-digest API, with canonical locale validation and separate caches. (#47 - thanks @yujiawei)
+
+### Changed
+
+- Update runtime and development dependencies, align TanStack Start packages, and move pnpm's native build allowlist into workspace configuration.
+
+### Fixed
+
+- Add accurate archive-first Sign in and archive-request onboarding, including account-binding requirements, current xurl/bird setup, scoped transport selection, and clean autolink rendering. (#46 - thanks @peetzweg)
+- Respect `OPENAI_BASE_URL` when sending Today/digest requests to OpenAI-compatible API endpoints.
+- Anchor Link Insights' Today range to UTC midnight so it matches stored `created_at` timestamps across local time zones.
+
+## 0.7.0 - 2026-06-01
+
+### Added
+
+- Stream live `birdclaw import archive` progress to stderr: per-slice parsing ticks (tweets, DMs, likes, bookmarks, follows, media) and chunked write-phase progress every 1,000 rows for profiles, tweets, likes+bookmarks, and DM messages. `--json` still keeps stdout clean for scripting.
+- Add `birdclaw discuss <query>` and a Discuss web view for live keyword search via `bird`/`xurl`, persisted search-result tweets, and streaming OpenAI summaries with optional private DM context.
+- Add `birdclaw profile-analyze <handle>` plus a Profile Analyse web view that backfills profile timelines and conversation context through `xurl`, caches the fetched context and AI result in SQLite, and exposes Analyse actions on tweet cards.
+- Add canonical `/profiles/:handle` pages with profile headers and cached Profile Analyse output.
+- Add a Rate Limits web view for observed `xurl` profile-analysis calls, 429s, local throttle settings, and documented X API recent-search windows.
+- Add a Network Map web view for current followers/following, with SQLite geocode caching, OpenCage refreshes, Mapbox rendering, and a local fallback map.
+- Render Network Map clusters as avatar stacks with relationship-weighted rings and avatar-rich profile/cluster overlays.
+- Make the Network Map people list follow the current viewport with an in-view search panel.
+- Add a Data Sources web view showing Birdclaw, bird, and xurl health, authenticated accounts, and automatic fallback order.
+- Prefetch cached avatars for Discuss hover citations so source previews avoid fallback initials once profile metadata includes an avatar URL.
+- Refresh Today digests from live `xurl` home timelines, mentions, and mention conversations before AI analysis so reports see more current context and reply parents.
+
+### Changed
+
+- Let Today and Discuss fetch much deeper live `xurl` data for the selected time window while keeping the AI prompt constrained to a large model-context budget.
+
+### Fixed
+
+- Implement `birdclaw auth use <auto|bird|xurl>` so the documented command persists the preferred moderation action transport. (#45 - thanks @peetzweg)
+- Keep `birdclaw init` alive when the macOS Downloads scan is blocked, falling back to the other archive discovery paths. (#44 - thanks @peetzweg)
+- Show live Today fetch progress while Birdclaw pulls X home timeline, mentions, and reply context before the first AI tokens arrive.
+- Include live fetch counts and page/thread progress in Today status messages before AI summary streaming begins.
+- Recover live `xurl` sync when the valid OAuth token is stored under a different local xurl username label than the Birdclaw account handle.
+- Keep Profile Analyse citation hover cards linked to real tweet/avatar sources, throttle `xurl` conversation searches, and retry 429s before continuing AI summaries with partial context.
+- Open `/profiles/:handle` analysis streams immediately, use same-origin profile fetches, and let `BIRDCLAW_PROFILE_ANALYSIS_ACCOUNT` select the xurl account used for profile backfills.
+- Keep Profile Analyse headers from slicing through loaded avatars/names and turn unresolved numeric tweet citations into safe X source links without leaking raw IDs.
+- Group adjacent Profile Analyse tweet citations so cached AI reports show numbered source links instead of repeated generic `source` labels.
+- Highlight hydrated Profile Analyse `@handle` mentions with profile previews and link multi-source citations to readable clauses when possible.
+- Hydrate Profile Analyse header bio `@handle` mentions as soon as the profile context loads, so affiliation-style bios show profile hover previews.
+- Flip tweet and profile hover previews above their trigger when there is not enough room below.
+- Show expanded URLs instead of `t.co` shortlinks in tweet citation hover previews whenever tweet URL entities are available.
+- Show expanded URLs instead of `t.co` shortlinks in Profile Analyse account bios when X description URL entities are available.
+- Keep emoji-bearing profile bios and media tweets aligned with X entity ranges, and route `@handle` profile-preview links to internal `/profiles/:handle` analysis pages.
+- Make Discuss search source/mode controls look like dropdowns in one row, raise live tweet search depth to 20,000 results / 200 pages, combine bird plus xurl in auto mode, and include matching local timeline/saved tweets in Live search discussions.
+- Default Discuss live mode to xurl now that OAuth2 search is authorized.
+- Use the default authorized xurl OAuth2 user for Discuss/Profile Analyse recent-search reads instead of the selected Birdclaw account handle.
+- Keep Discuss/Profile Analyse recent-search reads from inheriting `BIRDCLAW_XURL_OAUTH2_*` overrides, so account-scoped xurl settings do not force stale app/user auth into global search calls.
+- Let normal Discuss web searches reuse cached AI discussions while keeping the Refresh button as the explicit forced-refresh path.
+- Keep Discuss Live search scoped to live/search-result tweets instead of sweeping every local timeline bucket before AI streaming starts.
+- Link unresolved model-emitted `tweet_<id>` citations in AI reports to X source URLs instead of showing raw citation tokens.
+- Tighten AI report line height and first-block spacing in Today and Discuss.
+- Keep Network Map profile positions anchored to exact geocoded locations and render dense areas through smarter avatar clusters instead of random scatter.
+- Ignore stale configured OAuth2 xurl account overrides for Profile Analyse user lookup and profile timeline reads.
+
 ## 0.6.0 - 2026-05-22
 
 ### Added

package/README.md

@@ -219,6 +219,8 @@ birdclaw auth status --json
 birdclaw db stats --json
 ```
 
+`auth status` reports Birdclaw's coarse xurl status. Verify xurl with `xurl whoami` and bird with `bird whoami`. For setup and transport selection, see [Sign in](docs/auth.md).
+
 Find and import an archive:
 
 ```bash
@@ -227,6 +229,8 @@ birdclaw import archive --json
 birdclaw import archive ~/Downloads/twitter-archive-2025.zip --json
 ```
 
+Don't have an archive yet? Request it from <https://x.com/settings/download_your_data>; X emails a download link when it is ready, which may take a few days. A fresh Birdclaw database needs the archive import to establish account identity before live sync. See [Archive Import → Get an archive](docs/archive.md#get-an-archive).
+
 Optional profile hydration can improve bios, follower counts, and avatars, but it performs live X profile reads and can spend API credits on large archives:
 
 ```bash
@@ -387,19 +391,45 @@ birdclaw research "codex" --limit 20 --thread-depth 10 --json
 birdclaw research --account acct_primary --out ~/research/codex.md
 ```
 
+### Discuss keyword searches
+
+`birdclaw discuss` fetches live keyword matches through `bird` or `xurl`, stores them as local `search` tweets, then streams an OpenAI Markdown summary and discussion. DMs are excluded unless explicitly included.
+
+```bash
+birdclaw discuss "local-first" --mode bird
+birdclaw discuss "sync engine" --question "what changed over time?"
+birdclaw discuss "prototype" --include-dms --limit 500 --max-pages 5 --json
+```
+
+### Profile analysis
+
+`birdclaw profile-analyze` resolves a profile through `xurl`, walks as much of the retrievable timeline as the API allows, backfills high-signal conversations, caches both the fetched context and AI result in SQLite, and writes a Markdown profile brief.
+
+Conversation backfill uses X recent search, so Birdclaw paces those calls by default (`BIRDCLAW_PROFILE_ANALYSIS_CONVERSATION_DELAY_MS`, default `3100`) and retries a 429 once after `BIRDCLAW_PROFILE_ANALYSIS_RATE_LIMIT_RETRY_MS` (default `60000`) before continuing with partial context. Set `BIRDCLAW_PROFILE_ANALYSIS_RATE_LIMIT_MAX_RETRIES` or the matching CLI flags when you want different behavior.
+
+When `xurl` has multiple OAuth2 labels, set `BIRDCLAW_XURL_OAUTH2_APP` and `BIRDCLAW_XURL_OAUTH2_USERNAME` to force the known-good token. Set `BIRDCLAW_PROFILE_ANALYSIS_ACCOUNT` to an account id or handle when profile backfills should use a non-default Birdclaw account.
+
+The web UI uses `/profiles/<handle>` for the canonical profile page, `/profile-analyze` for the analysis/search utility page, and `/rate-limits` for observed `xurl` pressure, recent 429s, and the active Profile Analyse throttle settings.
+
+```bash
+birdclaw profile-analyze steipete
+birdclaw profile-analyse openai --max-pages 20 --max-conversations 40 --conversation-delay-ms 3100 --rate-limit-retries 2 --json
+```
+
 ### What happened today
 
-`birdclaw today` streams a local "what happened" digest from the SQLite store. It uses the OpenAI Responses API with `gpt-5.5`, medium reasoning, and priority service tier by default. Set `OPENAI_API_KEY`; override with `BIRDCLAW_AI_MODEL`, `BIRDCLAW_OPENAI_REASONING_EFFORT`, or `BIRDCLAW_OPENAI_SERVICE_TIER` when needed.
+`birdclaw today` streams a local "what happened" digest from the SQLite store. It uses the OpenAI Responses API with `gpt-5.5`, medium reasoning, and priority service tier by default. Set `OPENAI_API_KEY`; override with `BIRDCLAW_AI_MODEL`, `BIRDCLAW_OPENAI_REASONING_EFFORT`, or `BIRDCLAW_OPENAI_SERVICE_TIER` when needed. Use `--language <locale-id>` or `BIRDCLAW_DIGEST_LANGUAGE` for localized reports.
 
 ```bash
 birdclaw today
+birdclaw today --language zh-CN
 birdclaw digest 24h --refresh
 birdclaw digest week --json
 birdclaw digest --since 2026-05-16T00:00:00Z --until 2026-05-17T00:00:00Z
 birdclaw digest today --include-dms
 ```
 
-The web UI exposes the same stream under `What happened`. DMs are excluded unless explicitly enabled. Final structured results are cached by the exact local context hash, model, reasoning effort, and service tier.
+The web UI exposes the same stream under `What happened`. DMs are excluded unless explicitly enabled. Final structured results are cached by the exact local context hash, model, reasoning effort, service tier, and report language.
 
 ### Search and triage DMs
 

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "birdclaw",
-	"version": "0.6.0",
+	"version": "0.8.0",
 	"description": "Local Twitter memory in SQLite for archives, DMs, likes, bookmarks, and moderation",
 	"homepage": "https://github.com/steipete/birdclaw#readme",
 	"license": "MIT",
@@ -56,48 +56,47 @@
 		"@tailwindcss/vite": "^4.3.0",
 		"@tanstack/devtools-vite": "0.7.0",
 		"@tanstack/react-devtools": "0.10.5",
-		"@tanstack/react-router": "1.169.2",
-		"@tanstack/react-router-devtools": "1.166.13",
-		"@tanstack/react-router-ssr-query": "1.166.12",
-		"@tanstack/react-start": "1.167.65",
-		"@tanstack/router-plugin": "^1.167.35",
-		"@vitejs/plugin-react": "^6.0.1",
-		"commander": "^14.0.3",
-		"effect": "^3.21.2",
-		"kysely": "^0.29.0",
-		"lucide-react": "^1.16.0",
-		"react": "^19.2.6",
-		"react-dom": "^19.2.6",
+		"@tanstack/react-router": "1.170.15",
+		"@tanstack/react-router-devtools": "1.167.0",
+		"@tanstack/react-router-ssr-query": "1.167.1",
+		"@tanstack/react-start": "1.168.25",
+		"@tanstack/router-plugin": "^1.168.18",
+		"@vitejs/plugin-react": "^6.0.2",
+		"commander": "^15.0.0",
+		"effect": "^3.21.3",
+		"kysely": "^0.29.2",
+		"lucide-react": "^1.17.0",
+		"mapbox-gl": "^3.24.0",
+		"react": "^19.2.7",
+		"react-dom": "^19.2.7",
+		"react-map-gl": "^8.1.1",
+		"supercluster": "^8.0.1",
 		"tailwindcss": "^4.3.0",
-		"tsx": "^4.22.0",
-		"vite": "^8.0.13",
+		"tsx": "^4.22.4",
+		"vite": "^8.0.16",
 		"zod": "^4.4.3"
 	},
 	"devDependencies": {
 		"@playwright/test": "^1.60.0",
-		"@tailwindcss/typography": "^0.5.19",
+		"@tailwindcss/typography": "^0.5.20",
 		"@testing-library/dom": "^10.4.1",
 		"@testing-library/jest-dom": "^6.9.1",
 		"@testing-library/react": "^16.3.2",
-		"@types/node": "^25.8.0",
-		"@types/react": "^19.2.14",
+		"@types/geojson": "^7946.0.16",
+		"@types/node": "^25.9.2",
+		"@types/react": "^19.2.17",
 		"@types/react-dom": "^19.2.3",
-		"@typescript/native-preview": "7.0.0-dev.20260514.1",
-		"@vitest/coverage-v8": "^4.1.6",
+		"@types/supercluster": "^7.1.3",
+		"@typescript/native-preview": "7.0.0-dev.20260609.1",
+		"@vitest/coverage-v8": "^4.1.8",
 		"jsdom": "^29.1.1",
-		"oxfmt": "^0.49.0",
-		"oxlint": "^1.64.0",
+		"oxfmt": "^0.54.0",
+		"oxlint": "^1.69.0",
 		"typescript": "^6.0.3",
-		"vitest": "^4.1.6"
+		"vitest": "^4.1.8"
 	},
 	"engines": {
 		"node": ">=25.8.1 <27"
 	},
-	"packageManager": "pnpm@10.31.0",
-	"pnpm": {
-		"onlyBuiltDependencies": [
-			"esbuild",
-			"lightningcss"
-		]
-	}
+	"packageManager": "pnpm@10.31.0"
 }

package/scripts/build-docs-site.mjs

@@ -24,7 +24,10 @@ const productDescription =
 const brewInstall = "brew install steipete/tap/birdclaw";
 
 const sections = [
-	["Start", ["index.md", "install.md", "quickstart.md", "configuration.md"]],
+	[
+		"Start",
+		["index.md", "install.md", "auth.md", "quickstart.md", "configuration.md"],
+	],
 	[
 		"Archive & Sync",
 		["archive.md", "sync.md", "media.md", "backup.md", "jobs.md"],
@@ -449,19 +452,41 @@ function inline(text, currentRel) {
 		stash.push(`<code>${escapeHtml(code)}</code>`);
 		return `\u0000${stash.length - 1}\u0000`;
 	});
-	out = escapeHtml(out)
-		.replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>")
-		.replace(/(^|[^*])\*([^*\s][^*]*?)\*(?!\*)/g, "$1<em>$2</em>")
-		.replace(/(^|[^_])_([^_\s][^_]*?)_(?!_)/g, "$1<em>$2</em>")
-		.replace(
-			/\[([^\]]+)\]\(([^)]+)\)/g,
-			(_, label, href) =>
-				`<a href="${escapeAttr(rewriteHref(href, currentRel))}">${label}</a>`,
-		)
-		.replace(/&lt;(https?:\/\/[^\s<>]+)&gt;/g, '<a href="$1">$1</a>');
+	const stashLink = (label, href) => {
+		stash.push(
+			`<a href="${escapeAttr(rewriteHref(href, currentRel))}">${renderInlineText(label, stash)}</a>`,
+		);
+		return `\u0000${stash.length - 1}\u0000`;
+	};
+	out = out.replace(/\[([^\]]+)\]\(<([^<>]+)>\)/g, (_, label, href) =>
+		stashLink(label, href),
+	);
+	out = out.replace(/\[([^\]]+)\]\(([^)]+)\)/g, (_, label, href) =>
+		stashLink(label, href),
+	);
+	out = out.replace(/<(https?:\/\/[^\s<>]+)>/g, (_, url) => {
+		stash.push(`<a href="${escapeAttr(url)}">${escapeHtml(url)}</a>`);
+		return `\u0000${stash.length - 1}\u0000`;
+	});
+	return renderInlineText(out, stash);
+}
+
+function renderInlineText(text, stash) {
+	let out = formatInlineText(text);
 	out = out.replace(/\\\|/g, "|");
 	out = out.replace(/&lt;br&gt;/g, "<br>");
-	return out.replace(/\u0000(\d+)\u0000/g, (_, i) => stash[Number(i)]);
+	return restoreInlineStash(out, stash);
+}
+
+function formatInlineText(text) {
+	return escapeHtml(text)
+		.replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>")
+		.replace(/(^|[^*])\*([^*\s][^*]*?)\*(?!\*)/g, "$1<em>$2</em>")
+		.replace(/(^|[^_])_([^_\s][^_]*?)_(?!_)/g, "$1<em>$2</em>");
+}
+
+function restoreInlineStash(text, stash) {
+	return text.replace(/\u0000(\d+)\u0000/g, (_, i) => stash[Number(i)]);
 }
 
 function rewriteHref(href, currentRel) {
@@ -886,6 +911,8 @@ function highlightYamlValue(rest) {
 	return escapeHtml(rest);
 }
 
+export const __test__ = { inline };
+
 function validateLinks(outputDir) {
 	const failures = [];
 	const placeholderHrefs =