birdclaw 0.1.1 → 0.2.1

package/CHANGELOG.md

@@ -2,6 +2,34 @@
 
 All notable changes to this project will be documented in this file.
 
+## 0.2.1 - 2026-04-27
+
+### Changed
+
+- Use Twitter wording in public descriptions, docs, CLI help, and release notes.
+
+## 0.2.0 - 2026-04-27
+
+### Added
+
+- Add live likes and bookmarks sync through `xurl`/`bird`, local search filters, archive import support, and dedicated Likes/Bookmarks web views.
+- Add Git-friendly JSONL backup sync, export, import, validation, and stale-aware auto-sync for rebuilding or merging the local SQLite store from text shards across machines.
+- Add a scheduled bookmark sync job with launchd installation, JSONL audit logging, overlap locking, and automatic Git backup sync after each refresh.
+- Add launchd env-file support so scheduled bookmark sync can source `bird` credentials without storing secrets in the plist.
+
+### Changed
+
+- Update the README tagline and package description for local Twitter memory across archives, DMs, likes, bookmarks, and moderation.
+- Refresh dependencies, including `jsdom` 29.1.0.
+- Hide reply state and reply actions in saved likes/bookmarks web lanes.
+- Shard backup DMs by year and route unknown tweet dates to `data/tweets/unknown.jsonl` so Git backups stay compact and avoid bogus 1970 files.
+- Speed up archive imports plus JSONL backup export, import, and validation for large local datasets.
+
+### Fixed
+
+- Fix live bookmark sync to use stored Twitter user ids, force OAuth2 for `xurl` collection reads, and tolerate large/current `bird` bookmark payloads.
+- Fix fresh-machine backup sync so demo data is never exported into Git backups, and keep no-op syncs from creating metadata-only commits.
+
 ## 0.1.1 - 2026-04-27
 
 ### Added
@@ -16,7 +44,7 @@ All notable changes to this project will be documented in this file.
 
 ### Added
 
-- Add X web cookie fallback for block and unblock actions when the X API rejects OAuth2 block writes.
+- Add Twitter web cookie fallback for block and unblock actions when the Twitter API rejects OAuth2 block writes.
 - Add `profiles replies` so moderation triage can inspect a user's recent reply pattern before blocking.
 - Add `blocks import <path>` for one-shot blocklist application from a file.
 - Add paged `mentions export --mode xurl --all --max-pages <n>` so moderation loops can scan the full retrievable mentions window.

package/README.md

@@ -1,12 +1,12 @@
-# birdclaw
+# birdclaw 🪶 — Local Twitter memory in SQLite: archives, DMs, likes, bookmarks
 
-`birdclaw` is a local-first X workspace: archive import, cached live reads, focused triage, and reply flows in one local web app + CLI.
+`birdclaw` is a local-first Twitter workspace: archive import, cached live reads, focused triage, and reply flows in one local web app + CLI. Built by [@steipete](https://github.com/steipete/).
 
 Status: WIP. Real and usable. Not done. Expect schema churn, transport gaps, and rough edges while the core settles.
 
 ## What It Does
 
-- keeps your X data in local SQLite
+- keeps your Twitter data in local SQLite
 - stores media and avatar cache under `~/.birdclaw`
 - imports archives when you have them
 - still works when you do not
@@ -21,7 +21,10 @@ Status: WIP. Real and usable. Not done. Expect schema churn, transport gaps, and
 - FTS5 search over tweets and DMs
 - archive autodiscovery on macOS
 - archive import for tweets, likes, profiles, and full DMs
-- profile hydration from live X metadata
+- archive import for bookmark exports when present
+- live likes and bookmarks sync through `xurl` or `bird`
+- Git-friendly text backups with yearly tweet shards and per-conversation DM shards
+- profile hydration from live Twitter metadata
 - local avatar cache
 - local media cache root under `~/.birdclaw`
 
@@ -29,6 +32,7 @@ Status: WIP. Real and usable. Not done. Expect schema churn, transport gaps, and
 
 - `Home` timeline
 - `Mentions` queue
+- `Likes` and `Bookmarks` review lanes
 - `DMs` workspace with two-column layout
 - `Inbox` for mixed mention + DM triage
 - `Blocks` for local blocklist maintenance
@@ -44,6 +48,7 @@ Status: WIP. Real and usable. Not done. Expect schema churn, transport gaps, and
 - AI-ranked inbox for mentions + DMs
 - OpenAI scoring hook for low-signal filtering
 - cached live mentions export in `xurl`-compatible JSON
+- liked/bookmarked tweet filters for archive and live-synced collections
 - live profile-reply inspection for borderline AI/slop triage
 - one-shot blocklist import from a file for batch moderation passes
 
@@ -56,7 +61,7 @@ Status: WIP. Real and usable. Not done. Expect schema churn, transport gaps, and
 - import batch blocklists in one call
 - add / remove local mutes
 - sync remote blocks through `xurl` when available
-- fall back to the X web cookie session when OAuth2 block writes are rejected
+- fall back to the Twitter web cookie session when OAuth2 block writes are rejected
 
 ### Safety
 
@@ -77,8 +82,9 @@ If you need polished product-grade sync parity today, this is not there yet.
 
 ## Screens
 
-- `Home`: read and reply without fighting the main X timeline
+- `Home`: read and reply without fighting the main Twitter timeline
 - `Mentions`: work the reply queue with clean filters
+- `Likes` / `Bookmarks`: revisit saved posts from archive or live sync
 - `DMs`: triage by sender context, follower count, and influence
 - `Inbox`: let heuristics / OpenAI float likely-important items
 - `Blocks`: maintain a local-first account-scoped blocklist
@@ -110,6 +116,7 @@ export BIRDCLAW_HOME=/path/to/custom/root
 - `pnpm`
 - macOS recommended for Spotlight archive discovery
 - `xurl` optional for live reads / writes
+- `bird` optional for cookie-backed likes, bookmarks, mentions, DMs, and write fallback
 - OpenAI API key optional for inbox scoring
 
 ## Install
@@ -158,6 +165,18 @@ birdclaw import archive ~/Downloads/twitter-archive-2025.zip --json
 birdclaw import hydrate-profiles --json
 ```
 
+Back up the local SQLite store as canonical JSONL text:
+
+```bash
+birdclaw backup sync --repo ~/Projects/backup-birdclaw --remote https://github.com/steipete/backup-birdclaw.git --json
+```
+
+Merge the backup into the current `BIRDCLAW_HOME`:
+
+```bash
+birdclaw backup import ~/Projects/backup-birdclaw --json
+```
+
 Start the app:
 
 ```bash
@@ -180,6 +199,18 @@ pnpm cli blocks import ~/triage/blocklist.txt --account acct_primary --json
 pnpm cli search tweets "local-first" --json
 pnpm cli search tweets "sync engine" --limit 20 --json
 pnpm cli search tweets --since 2020-01-01 --until 2021-01-01 --originals-only --hide-low-quality --limit 500 --json
+pnpm cli search tweets --liked --limit 20 --json
+pnpm cli search tweets --bookmarked --limit 20 --json
+```
+
+### Sync likes and bookmarks
+
+`auto` tries `xurl` first, then falls back to `bird`. Use `bird` directly when the API path is unavailable for the account/token you have locally.
+
+```bash
+pnpm cli sync likes --mode auto --limit 100 --refresh --json
+pnpm cli sync bookmarks --mode auto --limit 100 --refresh --json
+pnpm cli sync bookmarks --mode bird --all --max-pages 5 --limit 100 --refresh --json
 ```
 
 ### Export mentions for agents
@@ -205,13 +236,13 @@ Home config lives in `~/.birdclaw/config.json`. Example:
 
 ```json
 {
-  "actions": {
-    "transport": "auto"
-  },
-  "mentions": {
-    "dataSource": "bird",
-    "birdCommand": "/Users/steipete/Projects/bird/bird"
-  }
+	"actions": {
+		"transport": "auto"
+	},
+	"mentions": {
+		"dataSource": "bird",
+		"birdCommand": "/Users/steipete/Projects/bird/bird"
+	}
 }
 ```
 
@@ -226,6 +257,7 @@ Notes:
 - `actions.transport` accepts `auto`, `bird`, or `xurl`
 - `bird` mode uses your local `bird` CLI and caches its mentions output into birdclaw's canonical store
 - filters still work in `xurl` mode; filtered payloads are rebuilt from the local canonical store after sync
+- `sync likes` and `sync bookmarks` store live results in the same local timeline table, so `search tweets --liked` and `search tweets --bookmarked` work across archive and live data
 
 ### Search and triage DMs
 
@@ -263,7 +295,7 @@ Notes:
 - `ban` / `unban` accept `--transport auto|bird|xurl`
 - `auto` tries `bird` first, then falls back to `xurl` when bird fails
 - forced `xurl` writes still verify through `bird status` before sqlite changes
-- X still rejects pure OAuth2 block writes, so `auto` is the safe default for block/unblock
+- Twitter still rejects pure OAuth2 block writes, so `auto` is the safe default for block/unblock
 - `blocks import` accepts newline-delimited blocklists with comments and markdown bullets
 - `blocks sync` is for slow/manual remote reconciliation; not for a hot cron loop
 - `blocks record` stores a known-good remote block locally without issuing another live write
@@ -324,10 +356,83 @@ pnpm cli compose reply tweet_004 "On it."
 pnpm cli compose dm dm_003 "Send it over."
 ```
 
+### Text Backup
+
+`birdclaw backup export` writes deterministic JSONL shards that can rebuild the local SQLite index without committing SQLite WAL/SHM files, FTS shadow tables, or transient live caches.
+
+Layout:
+
+```text
+manifest.json
+data/accounts.jsonl
+data/profiles.jsonl
+data/tweets/YYYY.jsonl
+data/tweets/unknown.jsonl
+data/collections/likes.jsonl
+data/collections/bookmarks.jsonl
+data/dms/conversations.jsonl
+data/dms/YYYY.jsonl
+data/moderation/blocks.jsonl
+data/moderation/mutes.jsonl
+```
+
+Tweets are sharded by year for human browsing and yearly analysis. Collection-only tweets whose real creation date is unknown go into `data/tweets/unknown.jsonl` instead of pretending they belong to 1970. DMs are sharded by year with `conversation_id` in each row; this keeps Git fast while preserving conversation membership. Likes and bookmarks are stored as collection edges in `data/collections` and mirrored into the timeline rows for current query compatibility.
+
+Use `backup sync` when the target is a private Git repo. It pulls first, merge-imports the remote backup into local SQLite, exports the local union back into text shards, commits, and pushes.
+
+```bash
+pnpm cli backup sync --repo ~/Projects/backup-birdclaw --remote https://github.com/steipete/backup-birdclaw.git --json
+pnpm cli backup validate ~/Projects/backup-birdclaw --json
+```
+
+Configure stale-aware backup reads in `~/.birdclaw/config.json`:
+
+```json
+{
+	"backup": {
+		"repoPath": "/Users/steipete/Projects/backup-birdclaw",
+		"remote": "https://github.com/steipete/backup-birdclaw.git",
+		"autoSync": true,
+		"staleAfterSeconds": 900
+	}
+}
+```
+
+Read paths such as CLI search, inbox, API status/query, and web startup pull + merge from Git only when the last backup check is stale. Data-changing commands run a full backup sync afterward when this config is enabled. Set `BIRDCLAW_BACKUP_AUTO_SYNC=0` to disable that behavior for one process.
+
+### Scheduled Bookmark Sync
+
+`birdclaw jobs sync-bookmarks` refreshes live bookmarks and appends one JSONL audit entry per run. Each entry includes host, timestamps, duration, before/after bookmark counts, source transport, fetched count, backup sync result, and any error.
+
+```bash
+birdclaw --json jobs sync-bookmarks --mode auto --limit 100 --max-pages 5 --refresh
+tail -n 5 ~/.birdclaw/audit/bookmarks-sync.jsonl | jq .
+```
+
+After a successful bookmark refresh, the job runs the normal backup auto-sync path. If `~/.birdclaw/config.json` has `backup.autoSync` enabled, the changed local data is merged into the configured Git backup repo, committed, and pushed. The audit entry records that backup result so scheduled runs are inspectable later.
+
+On macOS, install the 3-hour LaunchAgent after choosing the Birdclaw executable path for that machine:
+
+```bash
+birdclaw --json jobs install-bookmarks-launchd --program /opt/homebrew/bin/birdclaw
+```
+
+If the machine uses `bird` with browser cookies that are not available to launchd, write an export-only env file with mode `0600` and install with `--env-file ~/.config/bird/env.sh`. Birdclaw sources that file inside the scheduled process without storing the secrets in the plist.
+
+The LaunchAgent writes `~/Library/LaunchAgents/com.steipete.birdclaw.bookmarks-sync.plist`, runs at load, then every 10,800 seconds. It writes the audit log to `~/.birdclaw/audit/bookmarks-sync.jsonl` and stdout/stderr to `~/.birdclaw/logs/bookmarks-sync.*.log`. A lock file prevents overlapping runs and records an `already-running` skip when needed. The default job fetches up to 5 pages every 3 hours; pass `--all` if you want every retrievable page each run.
+
+Useful checks:
+
+```bash
+launchctl print gui/$(id -u)/com.steipete.birdclaw.bookmarks-sync
+launchctl kickstart -k gui/$(id -u)/com.steipete.birdclaw.bookmarks-sync
+tail -n 1 ~/.birdclaw/audit/bookmarks-sync.jsonl | jq .
+```
+
 ## Typical Workflow
 
 1. import your archive if you have one
-2. hydrate imported profiles from live X metadata
+2. hydrate imported profiles from live Twitter metadata
 3. use `Home` for reading
 4. use `Mentions` for reply triage
 5. when one account feels borderline, inspect `profiles replies`
@@ -341,8 +446,9 @@ pnpm cli compose dm dm_003 "Send it over."
 Current preference:
 
 - `xurl` first
+- `bird` fallback for surfaces where cookie-backed reads work better
 
-Without `xurl`, `birdclaw` still works in local/archive mode.
+Without `xurl` or `bird`, `birdclaw` still works in local/archive mode.
 
 Check transport:
 

package/package.json

@@ -1,7 +1,7 @@
 {
 	"name": "birdclaw",
-	"version": "0.1.1",
-	"description": "Local-first X workspace for archives, DMs, mentions, and moderation",
+	"version": "0.2.1",
+	"description": "Local Twitter memory in SQLite for archives, DMs, likes, bookmarks, and moderation",
 	"homepage": "https://github.com/steipete/birdclaw#readme",
 	"license": "MIT",
 	"repository": {
@@ -41,6 +41,7 @@
 		"test": "env -u NODE_OPTIONS node ./scripts/run-vitest.mjs run",
 		"coverage": "env -u NODE_OPTIONS node ./scripts/run-vitest.mjs run --coverage",
 		"e2e": "playwright test",
+		"typecheck": "tsgo --noEmit",
 		"format": "oxfmt --write src scripts playwright vite.config.ts vitest.config.ts playwright.config.ts",
 		"format:check": "oxfmt --check src scripts playwright vite.config.ts vitest.config.ts playwright.config.ts",
 		"lint": "oxlint --import-plugin --node-plugin --vitest-plugin --deny-warnings -A require-mock-type-parameters -A no-control-regex src scripts playwright vite.config.ts vitest.config.ts playwright.config.ts",
@@ -80,7 +81,7 @@
 		"@types/react": "^19.2.14",
 		"@types/react-dom": "^19.2.3",
 		"@vitest/coverage-v8": "^4.1.5",
-		"jsdom": "^29.0.2",
+		"jsdom": "^29.1.0",
 		"oxfmt": "^0.46.0",
 		"oxlint": "^1.61.0",
 		"typescript": "^6.0.3",