agentic-facebook 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- agentic_facebook-0.4.0/.claude/harness-spec.md +80 -0
- agentic_facebook-0.4.0/.claude/skills/facebook/SKILL.md +148 -0
- agentic_facebook-0.4.0/.github/workflows/ci.yml +84 -0
- agentic_facebook-0.4.0/.github/workflows/publish.yml +52 -0
- agentic_facebook-0.4.0/.gitignore +46 -0
- agentic_facebook-0.4.0/.pre-commit-config.yaml +16 -0
- agentic_facebook-0.4.0/CHANGELOG.md +98 -0
- agentic_facebook-0.4.0/CLAUDE.md +65 -0
- agentic_facebook-0.4.0/CODE_OF_CONDUCT.md +141 -0
- agentic_facebook-0.4.0/CONTRIBUTING.md +140 -0
- agentic_facebook-0.4.0/DISCLAIMER.md +48 -0
- agentic_facebook-0.4.0/LICENSE +21 -0
- agentic_facebook-0.4.0/PKG-INFO +144 -0
- agentic_facebook-0.4.0/README.md +111 -0
- agentic_facebook-0.4.0/SECURITY.md +59 -0
- agentic_facebook-0.4.0/docs/plans/2026-07-20-active-mode-expansion-plan.md +240 -0
- agentic_facebook-0.4.0/docs/plans/2026-07-20-recon-findings.md +270 -0
- agentic_facebook-0.4.0/docs/plans/IMPLEMENTATION-KICKOFF.md +58 -0
- agentic_facebook-0.4.0/docs/wiki/Architecture.md +99 -0
- agentic_facebook-0.4.0/docs/wiki/CLI-Reference.md +420 -0
- agentic_facebook-0.4.0/docs/wiki/Chaining-Recipes.md +88 -0
- agentic_facebook-0.4.0/docs/wiki/Configuration.md +179 -0
- agentic_facebook-0.4.0/docs/wiki/Contributing.md +21 -0
- agentic_facebook-0.4.0/docs/wiki/FAQ-and-Troubleshooting.md +207 -0
- agentic_facebook-0.4.0/docs/wiki/Installation.md +160 -0
- agentic_facebook-0.4.0/docs/wiki/Output-Schema.md +365 -0
- agentic_facebook-0.4.0/docs/wiki/Python-API-Reference.md +427 -0
- agentic_facebook-0.4.0/docs/wiki/Quick-Start.md +200 -0
- agentic_facebook-0.4.0/docs/wiki/README.md +50 -0
- agentic_facebook-0.4.0/docs/wiki/Security-and-Privacy.md +153 -0
- agentic_facebook-0.4.0/pyproject.toml +62 -0
- agentic_facebook-0.4.0/requirements-dev.lock +46 -0
- agentic_facebook-0.4.0/scripts/check_fixtures_pii.py +96 -0
- agentic_facebook-0.4.0/scripts/check_tag_version.py +42 -0
- agentic_facebook-0.4.0/scripts/record_fixture.py +63 -0
- agentic_facebook-0.4.0/src/agentic_facebook/__init__.py +181 -0
- agentic_facebook-0.4.0/src/agentic_facebook/catalog.py +156 -0
- agentic_facebook-0.4.0/src/agentic_facebook/chrome.py +179 -0
- agentic_facebook-0.4.0/src/agentic_facebook/cli.py +781 -0
- agentic_facebook-0.4.0/src/agentic_facebook/comments.py +250 -0
- agentic_facebook-0.4.0/src/agentic_facebook/config.py +103 -0
- agentic_facebook-0.4.0/src/agentic_facebook/errors.py +59 -0
- agentic_facebook-0.4.0/src/agentic_facebook/exits.py +48 -0
- agentic_facebook-0.4.0/src/agentic_facebook/graphql.py +244 -0
- agentic_facebook-0.4.0/src/agentic_facebook/model.py +425 -0
- agentic_facebook-0.4.0/src/agentic_facebook/parse.py +333 -0
- agentic_facebook-0.4.0/src/agentic_facebook/profiles.py +170 -0
- agentic_facebook-0.4.0/src/agentic_facebook/queries.py +305 -0
- agentic_facebook-0.4.0/src/agentic_facebook/redact.py +122 -0
- agentic_facebook-0.4.0/src/agentic_facebook/retrieve.py +831 -0
- agentic_facebook-0.4.0/src/agentic_facebook/scroll.py +244 -0
- agentic_facebook-0.4.0/src/agentic_facebook/search.py +155 -0
- agentic_facebook-0.4.0/src/agentic_facebook/session.py +227 -0
- agentic_facebook-0.4.0/src/agentic_facebook/tokens.py +232 -0
- agentic_facebook-0.4.0/src/agentic_facebook/truncation.py +70 -0
- agentic_facebook-0.4.0/tests/conftest.py +15 -0
- agentic_facebook-0.4.0/tests/fixtures/basic_status_post.ndjson +1 -0
- agentic_facebook-0.4.0/tests/fixtures/comment_preview_and_universal_marker.ndjson +1 -0
- agentic_facebook-0.4.0/tests/fixtures/media_and_links.ndjson +1 -0
- agentic_facebook-0.4.0/tests/fixtures/pinned_decoy_and_missing_date.ndjson +2 -0
- agentic_facebook-0.4.0/tests/fixtures/shared_post.ndjson +1 -0
- agentic_facebook-0.4.0/tests/fixtures/split_defer_merge.ndjson +2 -0
- agentic_facebook-0.4.0/tests/fixtures/truncated_post.ndjson +1 -0
- agentic_facebook-0.4.0/tests/live/test_active_live.py +97 -0
- agentic_facebook-0.4.0/tests/test_catalog.py +111 -0
- agentic_facebook-0.4.0/tests/test_comments.py +174 -0
- agentic_facebook-0.4.0/tests/test_graphql.py +286 -0
- agentic_facebook-0.4.0/tests/test_identifier.py +69 -0
- agentic_facebook-0.4.0/tests/test_model.py +201 -0
- agentic_facebook-0.4.0/tests/test_parse.py +168 -0
- agentic_facebook-0.4.0/tests/test_public_api.py +85 -0
- agentic_facebook-0.4.0/tests/test_redact.py +74 -0
- agentic_facebook-0.4.0/tests/test_retrieve.py +112 -0
- agentic_facebook-0.4.0/tests/test_search.py +101 -0
- agentic_facebook-0.4.0/tests/test_session_detection.py +39 -0
- agentic_facebook-0.4.0/tests/test_tokens.py +128 -0
- agentic_facebook-0.4.0/tests/test_truncation.py +69 -0
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Harness Spec — agentic-facebook
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
|
|
5
|
+
Python 3.11+ package (`hatchling`, `uv`/`pipx` install), CLI entry point `agentic-facebook`, tests via `pytest`, lint/format via `ruff`, hooks via `pre-commit`. Published to PyPI as `agentic-facebook` (v0.3.0, 2026-07-20). Single maintainer.
|
|
6
|
+
|
|
7
|
+
The repo already carries a `CLAUDE.md` of general behavioral guidelines (minimum code, surgical changes, goal-driven execution) that is about *developing* this package. This harness pass adds the first component about *using* the shipped tool.
|
|
8
|
+
|
|
9
|
+
## Goals
|
|
10
|
+
|
|
11
|
+
> "`.claude/skills/facebook` — a skill that drives the just-published agentic-facebook v0.3.0 CLI. It must teach Claude to chain the primitives (fetch/feed/comments/post/search/group) for real multi-hop tasks, carry the ban-risk and third-party-PII guidance as why-backed rules, and document the Post/Comment/Entity schemas so outputs are parsed without guessing."
|
|
12
|
+
|
|
13
|
+
From plan §8 (D7): the skill installs the published package (`uv tool install agentic-facebook`) and shells out to `agentic-facebook`. The division of labor the whole v0.3.0 design rests on: **the CLI does fast structured retrieval, the LLM does the navigation reasoning.** There is deliberately no `crawl` command — chaining is this skill's job.
|
|
14
|
+
|
|
15
|
+
## Behavior inventory
|
|
16
|
+
|
|
17
|
+
| id | behavior/knowledge/constraint | layer | component | status |
|
|
18
|
+
|----|-------------------------------|-------|-----------|--------|
|
|
19
|
+
| B1 | Ensure the CLI is installed and the session is live before retrieving | skill | facebook/SKILL.md | generated |
|
|
20
|
+
| B2 | Which primitive to reach for (flags themselves come from `agentic-facebook catalog`) | skill | facebook/SKILL.md | generated |
|
|
21
|
+
| B3 | Chain primitives for multi-hop tasks (the core value) | skill | facebook/SKILL.md | generated |
|
|
22
|
+
| B4 | Results go to a **file**, not stdout — read the file | skill | facebook/SKILL.md | generated |
|
|
23
|
+
| B5 | Exit codes carry meaning; 4 and 7 are not plain failures | skill | facebook/SKILL.md | generated |
|
|
24
|
+
| B6 | Ban-risk discipline (throwaway account, rate floors, no tight loops) | skill | facebook/SKILL.md | generated |
|
|
25
|
+
| B7 | Third-party PII discipline (don't commit, minimize, delete) | skill | facebook/SKILL.md | generated |
|
|
26
|
+
| B8 | Post vs Comment vs Entity, and which fields chain | skill | facebook/SKILL.md | generated |
|
|
27
|
+
| B9 | Recovery when retrieval fails (expiry, checkpoint, doc_id rotation) | skill | facebook/SKILL.md | generated |
|
|
28
|
+
|
|
29
|
+
## Component specs
|
|
30
|
+
|
|
31
|
+
### `.claude/skills/facebook/SKILL.md`
|
|
32
|
+
|
|
33
|
+
- **Directory name `facebook`** — that is the invocable name (`/facebook`), independent of the frontmatter `name`.
|
|
34
|
+
- **Scope: repo-local only** (user decision, 2026-07-20). It therefore triggers only in sessions working inside this repository; copying the directory to `~/.claude/skills/facebook` would make it available everywhere, at the cost of not shipping with a clone.
|
|
35
|
+
- **`description`**: triggers on Facebook data requests phrased by *intent*, not just the keyword — "what has X been posting", "who commented on this", "find people/groups about Y", "check my feed". Names near-misses explicitly: *developing/testing this package itself* is out of scope (ordinary repo work), as is any other social network.
|
|
36
|
+
- **`allowed-tools`**: `Bash(agentic-facebook:*)`, `Read` — every step shells out to `agentic-facebook` and then reads the JSON it wrote; without this the skill stalls on a permission prompt per command.
|
|
37
|
+
- **Auto-triggerable** (no `disable-model-invocation`): retrieval is user-initiated and read-only. It is *not* a deploy-shaped side effect. Ban/PII risk is handled by in-body rules, not by hiding the skill.
|
|
38
|
+
- Body: preflight (install/login/status) → the six primitives as a compact table → chaining recipes → the schema orientation → the two rule sets (ban risk, PII), each stated as principle + why.
|
|
39
|
+
|
|
40
|
+
### Single file — no `references/`
|
|
41
|
+
|
|
42
|
+
Failure handling lives in SKILL.md, not a reference file. It was split out initially on an "it worked vs it failed" branch, then merged: that is an *appendix*, not a branch (the model never chooses between the two files, it reads the second in addition), and the highest-stakes instruction in the whole skill — exit 3 means stop, never retry a checkpointed account — had ended up behind a file that only opens after something already went wrong. At ~140 lines merged, the happy-path cost of inlining it is small next to that risk.
|
|
43
|
+
|
|
44
|
+
## Design rationale
|
|
45
|
+
|
|
46
|
+
- **One skill, not several.** `fetch`/`feed`/`comments`/`post`/`search`/`group` share one trigger context ("get me something off Facebook") and a single multi-hop task uses several of them *together* — a user thinks of this as one job. Six skills would burn six description slots out of the shared ~1% listing budget to answer the same trigger.
|
|
47
|
+
- **One file, no `references/`.** See the component spec: the failure material is an appendix rather than a branch, and burying the checkpoint stop-rule behind a second file put the most safety-critical line where it loads only after the damage. Splitting the primitives per-command would have been worse still — a chain needs them in the same breath.
|
|
48
|
+
- **Reference data is pointed at, not transcribed** — extended in v0.3.1 from schemas to the whole CLI surface. `agentic-facebook catalog` reports commands, flags, exit codes, output contract, object types and limitations, all derived from the live parser and the `to_dict()`-anchored schema functions. The skill therefore carries **no** command/flag table: a copy would describe whichever version was current when it was written, and a model trusts a copy over its own reading. What stays in the skill is judgment the catalog cannot carry — which primitive to reach for, how to chain, and the ban/PII rules.
|
|
49
|
+
- **No hook, no permissions entry.** Nothing here must be *enforced* — the guardrails that genuinely must not fail (the ≥1.0s active floor, the ≥0.5s scroll floor) are already clamped inside the package and cannot be bypassed from the CLI at all. A hook would be re-implementing, less reliably, a guarantee that already exists in code.
|
|
50
|
+
- **No CLAUDE.md pointer.** The existing CLAUDE.md is about developing this package and is loaded every session; scraping Facebook is not something every session needs. Adding a line would spend the always-on budget on a rarely-relevant pointer.
|
|
51
|
+
|
|
52
|
+
## Validation
|
|
53
|
+
|
|
54
|
+
Scenarios for the optional e2e pass:
|
|
55
|
+
1. "What has <profile> been posting lately?" → expects `fetch --limit`, then reads the output file rather than expecting stdout.
|
|
56
|
+
2. "Who commented on <post URL>, and what else do those people post?" → expects `comments` → collect `author_url` → `fetch` per author, with a sane `--limit` rather than an unbounded fan-out.
|
|
57
|
+
3. "Find Facebook groups about <topic>" → expects `search --type groups`, and parses `kind`-bearing Entity objects rather than looking for post fields.
|
|
58
|
+
4. Failure path: a command exits 2 → expects the skill to route to `agentic-facebook login` rather than retrying blindly.
|
|
59
|
+
|
|
60
|
+
Most recent run: **2026-07-20, 4/4 passed** (~$1.88, ~10 min total), against package v0.3.1 installed from PyPI.
|
|
61
|
+
|
|
62
|
+
| # | Scenario | Verdict | Evidence |
|
|
63
|
+
|---|---|---|---|
|
|
64
|
+
| V1 | "피드에서 최근 글 5개 요약" | **pass** | `Skill: facebook` → `catalog` → `status` → `feed --limit 5 --output /tmp/...` → `Read` the file (not stdout) → `rm` the file unprompted |
|
|
65
|
+
| V2 | multi-hop: busiest post → its commenters → their timelines | **pass** | `feed --limit 15` → picked max `comment_count` → `comments <url>` → distinct `author_url` → `fetch` ×3; substituted 2 authors whose timelines were empty **and reported having done so** |
|
|
66
|
+
| V3 | "'seoul' 관련 그룹 찾아줘" | **pass** | `search --type groups --limit 25` → took each Entity `id` → `group <id>` ×9 to check liveness; ranked by reaction/comment counts, not post frequency |
|
|
67
|
+
| V4 | near-miss: "이 저장소의 파서 테스트 실행" | **pass** | `skill_invocations: []` — correctly read as ordinary repo work, ran pytest |
|
|
68
|
+
|
|
69
|
+
Two things the run established beyond the scenarios themselves:
|
|
70
|
+
- The **headless mechanism works**: `claude -p` spawned from Bash is authenticated here, so `run_e2e.py`'s permission approach (`--isolate` + skip-permissions) is confirmed for this project and no longer needs flagging as unverified.
|
|
71
|
+
- The PII rules are followed **without being asked** — every scenario deleted its scraped temp files at the end.
|
|
72
|
+
|
|
73
|
+
Defects the run surfaced (both fixed): the skill only handled a *missing* install, not an *outdated* one (the machine had 0.2.0, which lacks `catalog` and five primitives); and e2e `transcript.jsonl` files carry scraped PII while the repo's `*.json` ignore rule does not cover `.jsonl`.
|
|
74
|
+
|
|
75
|
+
## Change history
|
|
76
|
+
|
|
77
|
+
- 2026-07-20 — **new**. First harness pass on this repo. Recovered a spec from disk (an empty `.claude/skills/facebook/` placeholder existed, no SKILL.md). Generated the `facebook` skill (SKILL.md + references/troubleshooting.md) after the v0.3.0 PyPI release. `validate_harness.py`: 0 errors, 0 warnings. Scope set to repo-local per user. No hooks, permissions, agents, or CLAUDE.md changes — rationale above.
|
|
78
|
+
- 2026-07-20 — **improve**. Rewired the skill onto `agentic-facebook catalog` (added in package v0.3.1): removed the transcribed command/flag table and the duplicated exit-code and limitations tables from SKILL.md and troubleshooting.md, leaving each pointing at the derived source. Removes the package/skill drift risk that a copied table carried, which matters once the skill is used against a PyPI-installed package rather than this checkout.
|
|
79
|
+
- 2026-07-20 — **improve**. Merged `references/troubleshooting.md` into SKILL.md (user call, agreed): the split was an appendix, not a branch, and it had hidden the exit-3 stop-rule behind a conditional load.
|
|
80
|
+
- 2026-07-20 — **validate**. Ran the 4 e2e scenarios: 4/4 pass. Fixed two defects they surfaced (outdated-install handling in the skill; `.jsonl` PII leak in .gitignore). Transcripts deleted after grading.
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Facebook retrieval
|
|
3
|
+
description: Read Facebook via the agentic-facebook CLI — a profile's posts, your home feed, a post's comments, search (people/pages/groups/posts), or a group's feed — and chain those results to answer multi-hop questions. Use whenever the user wants something off Facebook, however they phrase it: "what has <person> been posting", "who commented on this", "what is <person>'s circle talking about", "find groups about X", "check my feed", "look up <name> on Facebook". Also use when the user hands over a facebook.com URL and wants its contents. NOT for developing or testing the agentic-facebook package itself (that is ordinary repo work), and not for any other social network.
|
|
4
|
+
allowed-tools: Bash(agentic-facebook:*), Read
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Facebook retrieval
|
|
8
|
+
|
|
9
|
+
`agentic-facebook` gives you fast, structured retrieval. **You supply the navigation.** The CLI is deliberately a set of single-purpose primitives with no `crawl` command — deciding which handle to follow next is your job, and it is the whole reason this skill exists.
|
|
10
|
+
|
|
11
|
+
## Step 1 — make sure you're on the current version
|
|
12
|
+
|
|
13
|
+
Do this before anything else, every task: compare what's installed against what PyPI has.
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
agentic-facebook --version 2>/dev/null
|
|
17
|
+
curl -s --max-time 5 https://pypi.org/pypi/agentic-facebook/json \
|
|
18
|
+
| python3 -c "import json,sys; print(json.load(sys.stdin)['info']['version'])"
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
If they differ, upgrade **now, before starting work** — not later, mid-task. Upgrading between steps of a running job can change a flag or an output shape underneath you and turn a clean failure into a baffling one; upgrading before anything is in flight is free.
|
|
22
|
+
|
|
23
|
+
Don't rely on a command erroring to tell you you're behind. That signal only fires when a release *adds* a command — a release that fixes a bug or changes a default is invisible to it, and "my install still works" is not evidence of being current. `catalog` describing your build correctly says nothing about whether a newer build exists.
|
|
24
|
+
|
|
25
|
+
Being current matters more here than for most tools: the single most likely way this package breaks is Facebook rotating the query ids it replays, and the fix ships as a release. "Behind on version" and "silently returning nothing" are the same problem often enough that the check pays for itself.
|
|
26
|
+
|
|
27
|
+
If PyPI is unreachable, **carry on with what's installed** — a version check is a nicety, not a gate, and blocking a task that would have worked is the worse failure.
|
|
28
|
+
|
|
29
|
+
Two failure shapes also mean install-or-upgrade, if you skipped the check:
|
|
30
|
+
|
|
31
|
+
- `command not found` → not installed.
|
|
32
|
+
- `invalid choice: '<command>'` → installed but older than this skill expects. Never conclude the feature doesn't exist and work around it.
|
|
33
|
+
|
|
34
|
+
Install or upgrade from PyPI into its own isolated environment:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
uv tool install --upgrade agentic-facebook # or: pipx install --force agentic-facebook
|
|
38
|
+
agentic-facebook setup # one-time: provisions its own browser
|
|
39
|
+
agentic-facebook --version # confirm it now matches PyPI
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Use `uv tool` or `pipx`, **not** `pip install` into a virtualenv you share with anything else. This package depends on `scrapling[fetchers]`, which pins exact Playwright/patchright versions — dropping it into a shared environment can fail to resolve, or silently break the other Playwright-based tool already living there. If neither `uv` nor `pipx` exists, install one rather than reaching for bare `pip`.
|
|
43
|
+
|
|
44
|
+
If an upgrade appears to do nothing (still the old version afterward), add `--no-cache`: `uv` caches the package index, so a release published minutes ago can be invisible to a plain `uv tool install --upgrade` while being perfectly present on PyPI.
|
|
45
|
+
|
|
46
|
+
Note that a repo checkout of this project and the installed CLI are **different things**: running from source (`PYTHONPATH=src python -m agentic_facebook.cli`) can be a completely different version from whatever `agentic-facebook` on PATH resolves to. The version on PATH is the one that counts.
|
|
47
|
+
|
|
48
|
+
## Step 2 — ask the tool what it can do
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
agentic-facebook catalog # or: agentic-facebook catalog --json
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
`agentic-facebook catalog` prints every command with its real flags, the exit-code contract, the output contract, the object types, and the known limitations — **in one call.** Work from what it says.
|
|
55
|
+
|
|
56
|
+
This file deliberately does **not** restate that list. The catalog is generated from the CLI's own parser, so it is correct for the version actually installed; a table copied into this file would silently describe the wrong version the moment the package updates, and you'd trust the copy over the truth. Anything you need to *call* a command comes from the catalog. What follows is only what the catalog can't carry: how to decide what to call next.
|
|
57
|
+
|
|
58
|
+
## Step 3 — check the session
|
|
59
|
+
|
|
60
|
+
Retrieval needs a logged-in Facebook session. Check once at the start of a task, not before every command:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
agentic-facebook status # exit 0 = ready; exit 2 = needs login; exit 3 = checkpoint
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
If exit 2: **`agentic-facebook login` opens a real browser window and needs a human to log in** — it detects completion automatically and times out. You cannot complete it for the user; ask them to, then re-check `status`. The account should be a throwaway, not their main one (see Ban risk).
|
|
67
|
+
|
|
68
|
+
Any other non-zero exit → see "When something fails" at the end of this file. Read it *before* improvising a retry: for one of these codes, retrying is the wrong move and can get the account banned.
|
|
69
|
+
|
|
70
|
+
## The one thing that will trip you up
|
|
71
|
+
|
|
72
|
+
**Every retrieval command writes its results to a JSON *file* and prints only a one-line summary to stderr. Nothing useful goes to stdout.** Run a command and read the file — never expect to parse the command's own output.
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
agentic-facebook feed --limit 10 --output /tmp/feed.json # stderr: "10 posts, range ... Saved to /tmp/feed.json"
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Then `Read /tmp/feed.json`. Always pass `--output` with a path you choose; without it the file lands under the platform data directory with a timestamped name you'd then have to go hunting for.
|
|
79
|
+
|
|
80
|
+
## What each primitive is *for*
|
|
81
|
+
|
|
82
|
+
The catalog gives you the flags; this is the judgment about which to reach for. Two questions place any of them, including one added after this was written: **what handle does it take** (a name, a URL, a free-text query, nothing at all) and **how narrow is what it returns**. Prefer the command whose handle you already hold, and among those the narrowest — a targeted surface is fewer requests and less unrelated third-party data than a broad one filtered afterward.
|
|
83
|
+
|
|
84
|
+
- **`fetch <profile>`** — one person's timeline. The only surface with a real date filter, so any "what did X post in <period>" question starts here.
|
|
85
|
+
- **`feed`** — your own news feed. Use for "what's happening", never for a question about a specific person (their timeline is more complete and filterable).
|
|
86
|
+
- **`post <url>`** / **`comments <url>`** — a specific post you already have a URL for. `post` fills the gap that a feed query cannot return a single permalink.
|
|
87
|
+
- **`search <query>`** — discovery when you have no handle yet. Returns *Entities* (people/pages/groups) or *Posts* depending on `--type`; entities are handles to fetch with, not answers.
|
|
88
|
+
- **`group <group>`** — one group's feed, once search or a link has given you the group.
|
|
89
|
+
|
|
90
|
+
## Chaining — the actual work
|
|
91
|
+
|
|
92
|
+
Every `Post` carries `url`, `author_url`, `author_id`; every `Comment` carries `author_url`; every `Entity` carries `id` and `url`. Those are the handles:
|
|
93
|
+
|
|
94
|
+
- a post's **`url`** → `comments` or `post`
|
|
95
|
+
- any **`author_url`** (post author, or commenter) → `fetch`
|
|
96
|
+
- a group Entity's **`id`** → `group`
|
|
97
|
+
|
|
98
|
+
**"What is X's circle discussing?"** → `fetch X` → read the file → collect `author_url` of the people X shares from → `fetch` each (with a `--limit`) → summarize across the results.
|
|
99
|
+
|
|
100
|
+
**"Who engaged with this post, and what are they into?"** → `comments <url> --limit 20` → collect distinct `author_url` → `fetch` each `--limit 5`.
|
|
101
|
+
|
|
102
|
+
**"Find the active groups about <topic>"** → `search "<topic>" --type groups` → take each Entity's `id` → `group <id> --limit 10` to see whether it is actually alive.
|
|
103
|
+
|
|
104
|
+
The `--limit` values above are illustrative, not defaults to copy. Size every one from the question being asked: "is this group alive" needs a handful of posts, "what has she been arguing about this month" needs a date window and more. A number carried over from an example is a number nobody chose.
|
|
105
|
+
|
|
106
|
+
Two rules that keep a chain from turning into a crawl. **Bound the fan-out before you start it** — decide the shape ("the most-engaged commenters"), not "everyone", because each hop is a real request against a real account (see Ban risk). And **report the shape of what you did**: which hops you took, how many you skipped, and why. A chain that silently sampled 5 of 60 commenters and presents itself as "what the commenters think" is a wrong answer wearing a confident summary.
|
|
107
|
+
|
|
108
|
+
## Reading the output
|
|
109
|
+
|
|
110
|
+
Three object types. Tell them apart by a field, never by guessing from context:
|
|
111
|
+
|
|
112
|
+
- **`Post`** — has **`source`** (`timeline` | `newsfeed` | `group` | `search`), telling you which surface it came from; matters once you merge results from several commands into one pile.
|
|
113
|
+
- **`Comment`** — has **`depth`** (`0` = top-level, `≥1` = a reply) and `parent_id`. Its `post_id` matches the parent post's `id`, so comments and posts join on that.
|
|
114
|
+
- **`Entity`** — has **`kind`** (`person` | `page` | `group`). Only `search --type people|pages|groups` returns these. It is a *light* record — name/url/id/verified, no posts — so an Entity is a handle to fetch with, not an answer in itself.
|
|
115
|
+
|
|
116
|
+
For the full field list run **`agentic-facebook schema`** (or read `object_types` in `agentic-facebook catalog --json`). Prefer that over assuming: both are generated from the code itself, so they cannot drift the way a copy in this file would.
|
|
117
|
+
|
|
118
|
+
Two fields that mislead if you skim: `captured_at` is when *you* scraped it, never a dedup or sort key — use `id` to dedup and `created_at` to sort. And `created_at` can be `null` when the date could not be located, so filter before comparing dates.
|
|
119
|
+
|
|
120
|
+
## Ban risk — why this stays slow
|
|
121
|
+
|
|
122
|
+
The account doing the scraping is at genuine risk of a checkpoint or permanent ban; automating any Meta account violates its Terms of Service. This is why the package clamps a **≥1.0s floor between active requests** and **≥0.5s between scrolls**, in code, un-bypassable — do not try to work around them, and do not fabricate concurrency by launching several `agentic-facebook` processes at once, which defeats the floor just as effectively as disabling it.
|
|
123
|
+
|
|
124
|
+
What that means for how you work: **a deep chain costs real requests and real risk.** `comments --replies` spends one extra request per commented comment, so a 100-comment post is a burst. Deep pagination (`--max-pages`, default 20) multiplies it. So prefer a `--limit` that answers the question over one that exhausts the source, and when a user asks for something genuinely large, say what it will cost before starting rather than discovering it halfway. If a command returns exit 3 (checkpoint), **stop entirely** — retrying a flagged account is how a temporary block becomes a permanent one.
|
|
125
|
+
|
|
126
|
+
## Third-party data — why the output is sensitive
|
|
127
|
+
|
|
128
|
+
Scraped output is other people's personal data: names, profile URLs, full comment text, and signed media URLs. Collecting it can make the *user* a data controller under GDPR/CCPA, with real obligations. `comments` and `search` are the sharp edge here — they collect people who never posted anything to the user and have no relationship with them.
|
|
129
|
+
|
|
130
|
+
So: **write output to a temp path, not into the repo**, and never `git add` it (the repo gitignores `*.json`/`*.ndjson` precisely because of this — don't defeat it with `--output` into a tracked path). Retrieve the narrowest thing that answers the question rather than everything available. When the task is done, say that the files can be deleted, and delete them if you created them for an intermediate step. Quote individuals' text only when the user's question actually needs the quote — a summary usually does the job with less exposure.
|
|
131
|
+
|
|
132
|
+
`--raw` embeds the unparsed node and is redacted by default; `--no-redact` disables even that. Neither belongs in normal use — they are debugging aids for working on the scraper itself.
|
|
133
|
+
|
|
134
|
+
## When something fails
|
|
135
|
+
|
|
136
|
+
`agentic-facebook catalog` prints what each exit code *means*. Below is what to *do* — and the theme is that **most failures here are informative, not transient. Retrying the same command is almost never the fix.**
|
|
137
|
+
|
|
138
|
+
**Exit 3 — checkpoint. Stop immediately.** Meta has flagged the account with a security challenge, and retrying is *actively harmful*: hammering a checkpointed account is how a temporary block becomes a permanent ban. Abandon the task, tell the user their account needs clearing by hand in a real browser, and run no further `agentic-facebook` commands this session. If checkpoints recur across sessions, that's a signal about volume — the fix is smaller `--limit`s and fewer hops, not a fresh login.
|
|
139
|
+
|
|
140
|
+
**Exit 2 — session expired.** Routine; sessions die. `agentic-facebook login` opens a real browser and needs a human — you cannot complete it. Ask the user, then re-check `status`. Note `status` inspects the response body, not just the URL, because Facebook serves its login form in place at `facebook.com` with HTTP 200 and no redirect: a dead session looks perfectly healthy from outside. If `status` says expired while a browser looks logged in, trust `status`.
|
|
141
|
+
|
|
142
|
+
**Exit 4 — zero results is ambiguous.** Either genuinely nothing there (empty timeline, no hits, no comments) or parser drift / `doc_id` rotation. Distinguish with one command: run something you *know* returns data (`agentic-facebook feed --limit 3`). If that also returns 0, it isn't your target — see rotation below. If the feed works, the emptiness is real; report it.
|
|
143
|
+
|
|
144
|
+
**Exit 7 — `--since` unconfirmed.** You have *some* posts but can't claim they're all of them in that range. Never present it as complete: either say the range is partial, or re-run with `--mode active` (if it had fallen back) or a larger `--max-scrolls`.
|
|
145
|
+
|
|
146
|
+
**`doc_id` rotation — the one real fragility.** Active mode replays Facebook query ids that rotate when Facebook ships a client build. `fetch` survives it (falls back to the browser automatically — you'll see `active mode failed ...; falling back to browser`, and it'll be much slower). `feed`, `comments`, `post`, `search`, `group` do **not**: they're active-only and simply fail. The signature is several active-only commands failing at once while `fetch --mode passive` still works. That's a package-level fix, not something to work around — tell the user and check for a newer release (`agentic-facebook catalog` reports the installed version).
|
|
147
|
+
|
|
148
|
+
**Not bugs, though they look like it:** a hop that fell back to passive is missing the profile's newest post (structural — re-run that hop with `--mode active`); `post`/`comments` on a **reel URL** fails with `could not find a story id` and needs a regular permalink; a comment's `reply_count` can exceed what `--replies` returned, because the count includes deeper nesting and only depth-1 is fetched.
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
|
|
8
|
+
permissions:
|
|
9
|
+
contents: read
|
|
10
|
+
|
|
11
|
+
jobs:
|
|
12
|
+
lint-and-test:
|
|
13
|
+
# macOS is the first-class, tested target (v1); the ubuntu leg exists
|
|
14
|
+
# because the fixture/parse/CLI layer is browser-agnostic, and running it
|
|
15
|
+
# on Linux too is a cheap way to make that cross-platform claim real
|
|
16
|
+
# (plan §14). Neither leg installs an actual browser binary.
|
|
17
|
+
strategy:
|
|
18
|
+
fail-fast: false
|
|
19
|
+
matrix:
|
|
20
|
+
os: [macos-latest, ubuntu-latest]
|
|
21
|
+
runs-on: ${{ matrix.os }}
|
|
22
|
+
steps:
|
|
23
|
+
- uses: actions/checkout@v4
|
|
24
|
+
|
|
25
|
+
- uses: actions/setup-python@v5
|
|
26
|
+
with:
|
|
27
|
+
python-version: "3.12"
|
|
28
|
+
|
|
29
|
+
- name: Install pinned dev dependencies
|
|
30
|
+
run: python -m pip install -r requirements-dev.lock
|
|
31
|
+
|
|
32
|
+
- name: Install package (dependencies already pinned above)
|
|
33
|
+
run: python -m pip install -e . --no-deps
|
|
34
|
+
|
|
35
|
+
- name: Ruff lint
|
|
36
|
+
run: ruff check .
|
|
37
|
+
|
|
38
|
+
- name: Ruff format check
|
|
39
|
+
run: ruff format --check .
|
|
40
|
+
|
|
41
|
+
- name: Fixture PII/secret scan
|
|
42
|
+
run: python scripts/check_fixtures_pii.py
|
|
43
|
+
|
|
44
|
+
- name: Run tests
|
|
45
|
+
run: pytest
|
|
46
|
+
|
|
47
|
+
build-and-smoke:
|
|
48
|
+
# A wheel-install smoke test, WITH scrapling (a real dependency) but
|
|
49
|
+
# WITHOUT a browser binary — that's a separate, explicit `agentic-facebook setup`
|
|
50
|
+
# step this job never runs. This catches a wrong entry-point path, a
|
|
51
|
+
# missing runtime dep, or an import that only fails once scrapling is
|
|
52
|
+
# actually present (plan §13) — none of which the fixture tests touch.
|
|
53
|
+
strategy:
|
|
54
|
+
fail-fast: false
|
|
55
|
+
matrix:
|
|
56
|
+
os: [macos-latest, ubuntu-latest]
|
|
57
|
+
runs-on: ${{ matrix.os }}
|
|
58
|
+
steps:
|
|
59
|
+
- uses: actions/checkout@v4
|
|
60
|
+
|
|
61
|
+
- uses: actions/setup-python@v5
|
|
62
|
+
with:
|
|
63
|
+
python-version: "3.12"
|
|
64
|
+
|
|
65
|
+
- name: Install build tool
|
|
66
|
+
run: python -m pip install build
|
|
67
|
+
|
|
68
|
+
- name: Build wheel
|
|
69
|
+
run: python -m build --wheel
|
|
70
|
+
|
|
71
|
+
- name: Install built wheel into a clean venv
|
|
72
|
+
# POSIX venv layout (bin/, not Scripts\) and the bash default shell
|
|
73
|
+
# both assume macos-latest/ubuntu-latest only — adding windows-latest
|
|
74
|
+
# to the matrix above would need this step rewritten, not just the
|
|
75
|
+
# matrix line (Windows unsupported in v1; see README/pyproject).
|
|
76
|
+
run: |
|
|
77
|
+
python -m venv smoke-venv
|
|
78
|
+
smoke-venv/bin/pip install dist/*.whl
|
|
79
|
+
|
|
80
|
+
- name: Smoke test the entry point
|
|
81
|
+
run: |
|
|
82
|
+
smoke-venv/bin/agentic-facebook --version
|
|
83
|
+
smoke-venv/bin/agentic-facebook --help
|
|
84
|
+
smoke-venv/bin/agentic-facebook schema
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
name: Publish to PyPI
|
|
2
|
+
|
|
3
|
+
# Fires when a GitHub Release is published (Releases page -> "Draft a new
|
|
4
|
+
# release" -> pick/create tag vX.Y.Z -> Publish release) — NOT on a bare
|
|
5
|
+
# `git push --tags`, which creates the tag but no visible Release entry.
|
|
6
|
+
# github.ref_name still resolves to the release's tag name under this event.
|
|
7
|
+
on:
|
|
8
|
+
release:
|
|
9
|
+
types: [published]
|
|
10
|
+
|
|
11
|
+
jobs:
|
|
12
|
+
build:
|
|
13
|
+
runs-on: ubuntu-latest
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
|
|
17
|
+
- uses: actions/setup-python@v5
|
|
18
|
+
with:
|
|
19
|
+
python-version: "3.12"
|
|
20
|
+
|
|
21
|
+
- name: Verify tag matches pyproject.toml version
|
|
22
|
+
run: python3 scripts/check_tag_version.py "${{ github.ref_name }}"
|
|
23
|
+
|
|
24
|
+
- name: Install build tool
|
|
25
|
+
run: python -m pip install build
|
|
26
|
+
|
|
27
|
+
- name: Build sdist and wheel
|
|
28
|
+
run: python -m build
|
|
29
|
+
|
|
30
|
+
- uses: actions/upload-artifact@v4
|
|
31
|
+
with:
|
|
32
|
+
name: dist
|
|
33
|
+
path: dist/
|
|
34
|
+
|
|
35
|
+
publish:
|
|
36
|
+
needs: build
|
|
37
|
+
runs-on: ubuntu-latest
|
|
38
|
+
permissions:
|
|
39
|
+
# Required to mint the OIDC token for PyPI Trusted Publishing — there is
|
|
40
|
+
# no stored API token anywhere in this repo (plan §15).
|
|
41
|
+
id-token: write
|
|
42
|
+
contents: read
|
|
43
|
+
steps:
|
|
44
|
+
- uses: actions/download-artifact@v4
|
|
45
|
+
with:
|
|
46
|
+
name: dist
|
|
47
|
+
path: dist/
|
|
48
|
+
|
|
49
|
+
# Pinned to the commit for release v1.14.0 — a floating tag on an
|
|
50
|
+
# action that runs with publish credentials is a hijack vector
|
|
51
|
+
# (plan §15). Bump deliberately, not automatically.
|
|
52
|
+
- uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
build/
|
|
7
|
+
dist/
|
|
8
|
+
|
|
9
|
+
# Dev environment
|
|
10
|
+
.venv/
|
|
11
|
+
.ruff_cache/
|
|
12
|
+
.pytest_cache/
|
|
13
|
+
|
|
14
|
+
# Raw captures and runtime outputs are a PII trap — ignore by an unambiguous,
|
|
15
|
+
# distinct name/location so a safe fixture and an unsafe raw capture can never
|
|
16
|
+
# collide under one glob (see plan §4).
|
|
17
|
+
scratch/
|
|
18
|
+
*.raw.ndjson
|
|
19
|
+
profiles/
|
|
20
|
+
*.json
|
|
21
|
+
*.ndjson
|
|
22
|
+
|
|
23
|
+
# Un-ignore the committed, synthetic-by-construction fixtures — enumerated
|
|
24
|
+
# explicitly, NOT a wildcard. A wildcard un-ignore (`!tests/fixtures/*.ndjson`)
|
|
25
|
+
# would re-include ANY new .ndjson dropped into this directory, not just
|
|
26
|
+
# these known-synthetic ones — defeating the whole point of the blanket
|
|
27
|
+
# ignore above. Adding a new fixture means adding its name here too.
|
|
28
|
+
!tests/fixtures/basic_status_post.ndjson
|
|
29
|
+
!tests/fixtures/comment_preview_and_universal_marker.ndjson
|
|
30
|
+
!tests/fixtures/media_and_links.ndjson
|
|
31
|
+
!tests/fixtures/pinned_decoy_and_missing_date.ndjson
|
|
32
|
+
!tests/fixtures/shared_post.ndjson
|
|
33
|
+
!tests/fixtures/split_defer_merge.ndjson
|
|
34
|
+
!tests/fixtures/truncated_post.ndjson
|
|
35
|
+
|
|
36
|
+
# E2E harness runs capture full session transcripts, which contain whatever the
|
|
37
|
+
# run scraped — real names, comment text, profile URLs. `.jsonl` is NOT covered
|
|
38
|
+
# by the *.json rule above, so it needs its own entry (learned the hard way).
|
|
39
|
+
.claude/.e2e-runs/
|
|
40
|
+
*.jsonl
|
|
41
|
+
|
|
42
|
+
# Scratch space for vendored reference material and sibling working copies —
|
|
43
|
+
# 355MB / 3400+ files at the time of writing. Never part of this package.
|
|
44
|
+
.tmp/
|
|
45
|
+
|
|
46
|
+
.DS_Store
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
repos:
|
|
2
|
+
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
3
|
+
rev: v0.8.6
|
|
4
|
+
hooks:
|
|
5
|
+
- id: ruff
|
|
6
|
+
args: [--fix]
|
|
7
|
+
- id: ruff-format
|
|
8
|
+
|
|
9
|
+
- repo: local
|
|
10
|
+
hooks:
|
|
11
|
+
- id: fixture-pii-scan
|
|
12
|
+
name: scan committed fixtures for PII/secrets
|
|
13
|
+
entry: python3 scripts/check_fixtures_pii.py
|
|
14
|
+
language: system
|
|
15
|
+
files: ^tests/fixtures/.*\.ndjson$
|
|
16
|
+
pass_filenames: false
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.4.0] — Renamed to Agentic Facebook
|
|
11
|
+
|
|
12
|
+
Renamed from the old identity (formerly PyPI `scraper-for-facebook`, import
|
|
13
|
+
`scraper_for_facebook`, CLI `scrape-fb`, repo `Scraper-for-Facebook`) to
|
|
14
|
+
**Agentic Facebook**. Behaviour is unchanged. Breaking: the import path, CLI
|
|
15
|
+
command, PyPI package name, and base exception class (`ScraperForFacebookError`
|
|
16
|
+
→ `AgenticFacebookError`) all changed. Install `agentic-facebook`; the old
|
|
17
|
+
package is a tombstone pointing here.
|
|
18
|
+
|
|
19
|
+
## [0.3.2] - 2026-07-21
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
- **`FacebookScraper(profile=...)` used the wrong account's cached credentials.** Active mode keys its token cache by profile *name* while the browser session is keyed by *directory*; the Python facade forwarded only the directory, so a non-default profile drove one account's browser while reading — and overwriting — the token cache belonging to `default`. With a populated cache that meant retrieving as the wrong account, and either way it corrupted the other profile's cached cookies. Silent rather than a crash. Affects the Python API only with a non-default `profile`; the CLI was never affected. Introduced in 0.3.0.
|
|
23
|
+
|
|
24
|
+
### Changed
|
|
25
|
+
- The PyPI summary and keywords now match the README and the repository description, instead of still describing the v0.2.0 browser-only scraper. Package metadata is frozen per release, so this could only be corrected by publishing.
|
|
26
|
+
|
|
27
|
+
### Documentation
|
|
28
|
+
- The whole documentation set rewritten for v0.3.x: twelve `docs/wiki/` pages (adding Architecture and Chaining-Recipes), a rewritten README, and CONTRIBUTING / SECURITY / CODE_OF_CONDUCT at the repository root. Pages now defer to `agentic-facebook catalog` and `agentic-facebook schema` for anything the CLI can describe about itself.
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
## [0.3.1] - 2026-07-20
|
|
32
|
+
|
|
33
|
+
### Added
|
|
34
|
+
- **`agentic-facebook catalog [--json]`** — the CLI describing itself: every command and flag, the exit-code contract, the output contract, the object types, and the known limitations, in one call. It is **derived, not authored**: commands and flags are introspected from the live `argparse` parser, object types come from the same `to_dict()`-anchored functions `schema` uses, and exit codes come from a single table. Anything that needs to explain this tool (docs, an agent, a `.claude` skill) can now read it instead of transcribing it and drifting.
|
|
35
|
+
- `tests/test_catalog.py` enforces that derivation: adding a command or flag without it appearing in the catalog fails the suite, as does a subcommand with no handler.
|
|
36
|
+
|
|
37
|
+
### Changed
|
|
38
|
+
- Exit codes moved from scattered integer literals in `cli.py` into `exits.py`, which is now the single source for both the CLI's behavior and its description. No exit code changed value.
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
## [0.3.0] - 2026-07-20
|
|
42
|
+
|
|
43
|
+
Adds an **active transport** — reading Facebook's GraphQL API over plain HTTP,
|
|
44
|
+
with the browser needed only to log in — and turns the single-purpose scraper
|
|
45
|
+
into a set of composable primitives. Additive: `fetch`'s existing flags and
|
|
46
|
+
output schema are unchanged apart from one new field.
|
|
47
|
+
|
|
48
|
+
### Added
|
|
49
|
+
- **Active mode.** `fetch` now reads `/api/graphql/` over HTTP by default, paginating by cursor with no browser in the hot path, and falls back to the browser transport automatically when it fails (`--mode auto|active|passive`). Both transports share one parser, so their output is identical.
|
|
50
|
+
- **New commands**, each emitting the same output contract as `fetch`:
|
|
51
|
+
- `feed` — your home news feed.
|
|
52
|
+
- `comments <post_url>` — a post's comments, `--sort top|recent`, `--replies` for depth ≥ 1.
|
|
53
|
+
- `post <post_url>` — a single post by permalink (which a feed query cannot return).
|
|
54
|
+
- `search <query>` — `--type top|posts|people|pages|groups`.
|
|
55
|
+
- `group <group_url_or_id>` — one group's feed.
|
|
56
|
+
- **`Comment` and `Entity` schemas**, both covered by `agentic-facebook schema` / `schema --json`.
|
|
57
|
+
- `Post.source` (`timeline` | `newsfeed` | `group` | `search`) so chained output stays self-describing.
|
|
58
|
+
- `agentic-facebook login --from-chrome` (opt-in): import an existing session from local Chrome by decrypting its cookie DB. Needs the `chrome` extra: `pip install 'agentic-facebook[chrome]'`. See DISCLAIMER §6 — this is literal cookie extraction and usually imports your main account.
|
|
59
|
+
- A **non-bypassable active-request floor** (`--request-interval`, MIN clamped to ≥ 1.0s, jittered), the active-mode counterpart to the scroll-pause floor. Applies to every active request, including id-resolution GETs.
|
|
60
|
+
- `--max-pages` (default 20) bounds active pagination depth.
|
|
61
|
+
- Opt-in live tests under `tests/live/` (`SFB_LIVE_TESTS=1`), including an active-vs-passive parity test.
|
|
62
|
+
|
|
63
|
+
### Fixed
|
|
64
|
+
- **`status` reported a dead session as `logged_in`.** `detect_wall()` only inspected the URL, but Facebook serves its login form in place at `https://www.facebook.com/` with HTTP 200 and no redirect. It now also inspects the response body, and treats a missing `DTSGInitialData` token as logged out.
|
|
65
|
+
- **`login` could not be driven non-interactively.** It blocked on `input()`, which hung forever under a non-TTY driver while holding the Chromium profile lock, blocking every later browser launch. It now polls the browser's own state and honors `--timeout-seconds`.
|
|
66
|
+
|
|
67
|
+
### Changed
|
|
68
|
+
- `--since`/`--until` are now a **precise server-side filter** in active mode (verified: they change which posts the server returns, not just which are kept). They remain best-effort in passive mode, still reported via exit code 7.
|
|
69
|
+
- Token refresh prefers a cheap HTTP re-read over relaunching the browser.
|
|
70
|
+
- README/DISCLAIMER repositioned: active mode uses *your own session's* tokens the way your browser does — still no credential injection and no foreign-token replay — and the DISCLAIMER now covers the much larger third-party PII surface that `comments`/`feed`/`search` collect.
|
|
71
|
+
|
|
72
|
+
### Known limitations
|
|
73
|
+
- `feed`/`comments`/`post`/`search`/`group` are active-only: they have no passive equivalent to fall back to if a `doc_id` rotates.
|
|
74
|
+
- Passive mode cannot see a profile's newest post (the first timeline batch is server-rendered, never fetched as a GraphQL XHR). Active mode can.
|
|
75
|
+
- `post`/`comments` do not support reel URLs (a reel page embeds no story id).
|
|
76
|
+
- `--replies` fetches depth-1 replies only.
|
|
77
|
+
|
|
78
|
+
## [0.2.0] - 2026-07-07
|
|
79
|
+
|
|
80
|
+
### Added
|
|
81
|
+
- `agentic-facebook schema`: prints the `fetch` output object schema (field name, JSON type, one-line meaning), offline and always exit 0; `--json` emits JSON Schema (draft 2020-12). Anchored on `Post.to_dict()`'s actual output keys, not the dataclass fields, so it can't mis-document `raw` as always-present.
|
|
82
|
+
- Every `fetch`/`login`/`status`/`doctor` flag now has a `--help` string with its human-readable default, so `--help` is authoritative standalone without reading source.
|
|
83
|
+
|
|
84
|
+
### Changed
|
|
85
|
+
- No behavior change to `login`/`status`/`setup`/`doctor`/`fetch` themselves — additive only.
|
|
86
|
+
|
|
87
|
+
## [0.1.0] - 2026-07-05
|
|
88
|
+
|
|
89
|
+
### Added
|
|
90
|
+
- Initial release: `agentic-facebook login` / `status` / `setup` / `doctor` / `fetch`.
|
|
91
|
+
- Logged-in Facebook timeline scraping via GraphQL XHR observation (no token replay).
|
|
92
|
+
- `--limit`, `--since`/`--until` retrieval with stop-reason reporting.
|
|
93
|
+
- JSON and NDJSON output formats.
|
|
94
|
+
- Python API: `FacebookScraper`, `Post`, `Media`, `LinkAttachment`.
|
|
95
|
+
|
|
96
|
+
[Unreleased]: https://github.com/tjdwls101010/Agentic-Facebook/compare/v0.2.0...HEAD
|
|
97
|
+
[0.2.0]: https://github.com/tjdwls101010/Agentic-Facebook/releases/tag/v0.2.0
|
|
98
|
+
[0.1.0]: https://github.com/tjdwls101010/Agentic-Facebook/releases/tag/v0.1.0
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
Behavioral guidelines to reduce common LLM coding mistakes. Merge with project-specific instructions as needed.
|
|
4
|
+
|
|
5
|
+
**Tradeoff:** These guidelines bias toward caution over speed. For trivial tasks, use judgment.
|
|
6
|
+
|
|
7
|
+
## 1. Think Before Coding
|
|
8
|
+
|
|
9
|
+
**Don't assume. Don't hide confusion. Surface tradeoffs.**
|
|
10
|
+
|
|
11
|
+
Before implementing:
|
|
12
|
+
- State your assumptions explicitly. If uncertain, ask.
|
|
13
|
+
- If multiple interpretations exist, present them - don't pick silently.
|
|
14
|
+
- If a simpler approach exists, say so. Push back when warranted.
|
|
15
|
+
- If something is unclear, stop. Name what's confusing. Ask.
|
|
16
|
+
|
|
17
|
+
## 2. Simplicity First
|
|
18
|
+
|
|
19
|
+
**Minimum code that solves the problem. Nothing speculative.**
|
|
20
|
+
|
|
21
|
+
- No features beyond what was asked.
|
|
22
|
+
- No abstractions for single-use code.
|
|
23
|
+
- No "flexibility" or "configurability" that wasn't requested.
|
|
24
|
+
- No error handling for impossible scenarios.
|
|
25
|
+
- If you write 200 lines and it could be 50, rewrite it.
|
|
26
|
+
|
|
27
|
+
Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.
|
|
28
|
+
|
|
29
|
+
## 3. Surgical Changes
|
|
30
|
+
|
|
31
|
+
**Touch only what you must. Clean up only your own mess.**
|
|
32
|
+
|
|
33
|
+
When editing existing code:
|
|
34
|
+
- Don't "improve" adjacent code, comments, or formatting.
|
|
35
|
+
- Don't refactor things that aren't broken.
|
|
36
|
+
- Match existing style, even if you'd do it differently.
|
|
37
|
+
- If you notice unrelated dead code, mention it - don't delete it.
|
|
38
|
+
|
|
39
|
+
When your changes create orphans:
|
|
40
|
+
- Remove imports/variables/functions that YOUR changes made unused.
|
|
41
|
+
- Don't remove pre-existing dead code unless asked.
|
|
42
|
+
|
|
43
|
+
The test: Every changed line should trace directly to the user's request.
|
|
44
|
+
|
|
45
|
+
## 4. Goal-Driven Execution
|
|
46
|
+
|
|
47
|
+
**Define success criteria. Loop until verified.**
|
|
48
|
+
|
|
49
|
+
Transform tasks into verifiable goals:
|
|
50
|
+
- "Add validation" → "Write tests for invalid inputs, then make them pass"
|
|
51
|
+
- "Fix the bug" → "Write a test that reproduces it, then make it pass"
|
|
52
|
+
- "Refactor X" → "Ensure tests pass before and after"
|
|
53
|
+
|
|
54
|
+
For multi-step tasks, state a brief plan:
|
|
55
|
+
```
|
|
56
|
+
1. [Step] → verify: [check]
|
|
57
|
+
2. [Step] → verify: [check]
|
|
58
|
+
3. [Step] → verify: [check]
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Strong success criteria let you loop independently. Weak criteria ("make it work") require constant clarification.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
**These guidelines are working if:** fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and clarifying questions come before implementation rather than after mistakes.
|