@crievo/omni 0.0.0-stage → 0.1.1

package/reference/omnichat.md

@@ -0,0 +1,305 @@
+# Omnichat on Instagram comments
+
+Like ManyChat: whoever comments the keyword on a post gets a DM with the link
+(the comment's *private reply*) and a public reply on the comment ("te mandei
+no direct"). Tool: `tools/omnichat.mjs`. Phase 1, no webhook: the launchd
+agent polls the API every minute.
+
+It works for any post in the account's feed: Reel, carousel (`CAROUSEL_ALBUM`) and
+photo (`IMAGE`). Each rule lives in a target folder, the one with the
+`publication.json` holding the media id:
+
+| target | folder | who publishes |
+|---|---|---|
+| a video's Reel | `videos/<slug>/` | `publish.mjs` |
+| the carousel of the same content | `videos/<slug>/carousel/` | `publish.mjs` (carousel); `published` enables the rule |
+| a post from outside omnivideo (Reel, carousel or photo made by hand) | `.omnichat/posts/<code>/` | you; `post <link>` creates the folder |
+
+The carousel finds the term in the slides' text, in the caption and in
+`carousel/script.md`, and the link in the caption, in its script, in what the
+content's video says (`publication.json`, `brief.md`, `source/request.md`) and,
+last, in the video's omnichat rule: the carousel of a video that already delivers a
+link delivers the same one.
+
+## First: the token with the new permissions
+
+The `publish.mjs` token only publishes. Omnichat uses the same token, with two
+more permissions:
+
+1. At [developers.facebook.com/apps](https://developers.facebook.com/apps) → your app →
+   Instagram → **API setup with Instagram login**, under "Generate access tokens",
+   generate the token again checking `instagram_business_basic`,
+   `instagram_business_content_publish`, `instagram_business_manage_comments` and
+   `instagram_business_manage_messages`.
+2. `node tools/publish.mjs auth instagram <token>` (or paste it in the studio under
+   Configurações (Settings) → Plataformas (Platforms) → Instagram).
+3. In the Instagram app, if the DM fails with a permission error: Configurações →
+   Mensagens e respostas ao story → Controles de mensagens → **Ferramentas
+   conectadas** → allow access to messages (Settings → Messages and story replies →
+   Message controls → Connected tools).
+4. `node tools/omnichat.mjs status` checks it: "comentários: lidos pela API"
+   (comments: read through the API).
+
+5. **The app must be published (Live).** In development mode the API hides the
+   comments of anyone without a role in the app: the Reel shows `comments_count` 23
+   and the list comes back empty, with no error. Publishing only asks for the
+   privacy policy URL (App settings → Basic; any public page of yours works, like a
+   gist with the policy) and the dashboard's Publish button. No App Review:
+   Standard Access is enough for your own account. The webhook (replies within
+   seconds, phase 3) is what requires Advanced Access and Business Verification.
+
+## Usage
+
+```bash
+# an omnivideo video (the Reel) and its carousel
+node tools/omnichat.mjs rule videos/<slug> --keyword QUERO --link https://…
+node tools/omnichat.mjs rule videos/<slug>/carousel --keyword QUERO --link https://…
+# a post that is already live (from outside omnivideo, or just the link): Reel, carousel or photo
+node tools/omnichat.mjs post https://www.instagram.com/p/<code>/ --keyword QUERO --link https://…
+node tools/omnichat.mjs post https://www.instagram.com/reel/<code>/ --keyword QUERO --link https://…
+
+node tools/omnichat.mjs run <dir> --dry-run       # reads the comments and shows what it would send
+node tools/omnichat.mjs enable <dir> --confirm    # starts replying
+node tools/omnichat.mjs agent install             # the agent that checks every minute
+node tools/omnichat.mjs log <dir>
+node tools/omnichat.mjs comments <dir>                          # reads the post now: who has no DM
+node tools/omnichat.mjs delete <dir> <id> [<id>…] --confirm     # deletes comments on Instagram (no way back)
+node tools/omnichat.mjs send <dir> <id> [<id>…] --confirm       # the rule's DM to these, by hand
+node tools/omnichat.mjs send <dir> <id> --public-reply "Valeu, @{username}!" --confirm   # with a written reply
+node tools/omnichat.mjs send <dir> <id> --public-reply-only --public-reply "…" --confirm # comment only, no DM
+```
+
+**By hand: who has no DM.** In the rule panel (Omnichat page), "Buscar comentários
+sem DM" (Find comments without a DM) reads the Reel right away, without waiting
+for the agent, and lists who didn't get the DM: what the rule skipped (you already
+replied, Claude read it as something else), what it didn't catch (the comment
+didn't have the term, or was read before the rule changed), what is queued and
+what failed. Mark who gets it ("Marcar os que pedem" (Mark the ones asking) picks
+the ones that trigger) and send: the saved rule's DM goes out, with the comment
+reply if checked, even to people who didn't comment the term. It goes outward, in
+the account's name: it asks for confirmation, and Claude only sends when the user
+asks (`--confirm`). It runs at the agent's pace (`pause_between_s`, up to
+`max_per_pass` at a time), with its lock and respecting the account pause. The
+record stores `manual: true`, and nobody gets it twice. A comment older than 7
+days gets no DM.
+
+The comment reply can be written right there (`--public-reply`; `{username}`
+becomes the @; the old `{usuario}` placeholder is still accepted): if empty, one
+of the rule's replies goes, picked at random. The selector chooses what goes: DM
+and reply, only the DM, or only the comment reply (`--public-reply-only`). Comment
+only, without a DM, the reply must be written (the rule's ones say "te mandei no
+direct"), needs no rule and also works for people past the 7 days. It goes once,
+without going back to the queue; a comment that wasn't in the record yet enters
+as skipped ("respondido à mão no comentário" (replied by hand on the comment)),
+like the ones you answer through the app.
+
+**The comments as a chat.** The panel ("Comentários") shows the post like Instagram's
+thread: each comment is a ghost card (username, text, "Responder", status badges,
+the DM icon and "Apagar") with its replies nested inside, the account's own
+included (`comments --json` returns them as `thread`, next to `items`, the ones
+without a DM). The account's loose comment that starts with `@person` nests under
+that person's comment. The DM icon opens what was sent to that person; `dm.text`
+is stored when the DM goes out, and older records show the rule's text as it
+reads today (`exact: false`). "Sem DM" filters who didn't get it, and only there
+do the checkboxes and the batch appear.
+
+"Responder" opens the field right below the card (Esc closes, ⌘↩ replies). Writing
+and clicking Responder is the request, without the batch confirmation; "Mandar
+também a DM da regra" (Also send the rule's DM) adds the DM (within the 7 days,
+with a rule). A reply can be answered too: Instagram threads two levels, so it goes
+to the top-level comment with the person's `@` in front (and never with a DM).
+
+"Apagar" (Delete) removes the comment on Instagram (`DELETE /{comment-id}`: a
+person's on the post, or the account's own reply) and has no way back, so the card
+asks a second click first; `delete --confirm` is that click. A comment the agent
+still had queued stops waiting (`skipped`, "apagado por você"); with the cloud
+answering, the cloud's record isn't touched from here.
+Liking comments isn't available: the Instagram API with Instagram login (the one
+the token uses, `graph.instagram.com`) doesn't like comments.
+
+**On publish.** With `omnichat.instagram.enable_on_publish: true` (Configurações →
+Publicação → Omnichat → Ao publicar (Settings → Publishing → Omnichat → On
+publish); it is the permanent approval to reply without the enable click),
+`publish.mjs` calls `omnichat.mjs published` right after the Reel (or the carousel:
+`published videos/<slug>/carousel`) goes up, now or through the schedule, and its
+rule turns on by itself:
+
+- with no rule, it is born from the narration's ask: the word after "comenta" in
+  a sentence that promises to send something ("Comenta REA que eu te mando o
+  link", "Comente MUSE que te envio") becomes the term, with the generic phrases,
+  the default DM and replies, and the link found in the video (description,
+  `brief.md` or `source/request.md`; social networks don't count);
+- a draft, saved and never enabled, is enabled as it is.
+
+It doesn't enable a rule you disabled (the Publicar (Publish) stage switch turned
+off before uploading records `disabled_at`, even without a rule), an any-comment
+rule, or one that isn't complete: without the link the DM uses, it stays saved and
+disabled, and the publish output says what is missing. With no ask in the
+narration, nothing is enabled.
+
+**The DM quotes the Reel.** The private reply is a single message and the Reel is
+not a message in the conversation, so it can't be replied to the way a message
+is. With `quote_post` in the rule, the Reel's link goes on the DM's first line:
+Instagram shows its card on top and the message reads as a reply to it. `{post}`
+in the DM puts the link somewhere else; a DM that already has the Reel's link
+doesn't repeat it. A new rule follows `omnichat.instagram.quote_post` (on by
+default); each rule changes it in the Omnichat panel or with `--quote-post` and
+`--no-quote-post`. Meta says the private reply already carries a link to the
+commented post: the card is so the reader gets it at once.
+
+`post` finds the media by link among the account's posts. If it belongs to an
+omnivideo video, it uses the video's folder; otherwise it creates
+`.omnichat/posts/<code>/`, which works in every command that takes `videos/<slug>`.
+
+**Retroactive.** Enabled on a post that is already running, the rule reads the
+comments from the last 7 days and replies to the ones that trigger, 15 per pass.
+Older than 7 days gets no DM: Meta only accepts the private reply up to 7 days
+after the comment. A comment the account already answered by hand is skipped:
+both a reply nested under the comment and a separate comment from the account
+with `@someone` count (that's how the app usually replies).
+`--answered send --answered-if "não consegui"` sends only to people you replied to
+saying they couldn't. A DM sent by hand, with nothing on the comment, is invisible
+to the API: that person gets it again.
+
+## Button and "only if you follow me" (phase 2)
+
+Like ManyChat: the comment's DM comes with a button, and the link only goes out when the person taps it.
+With `require_follow`, the tap also checks that the person follows the account; if not, they get
+"follow and tap again" with a new button, and the link goes out on the next tap once they follow.
+
+```bash
+node tools/omnichat.mjs rule videos/<slug> --button                       # link only after the tap
+node tools/omnichat.mjs rule videos/<slug> --require-follow               # and only for followers
+node tools/omnichat.mjs rule videos/<slug> --button-label "Quero o link" --opening-dm "Oi, @{username}! Toca aqui 👇" \
+  --not-following-dm "Segue o perfil e volta no botão 👀" --retry-label "Pronto, já sigo!"
+node tools/omnichat.mjs rule videos/<slug> --skip-dm "@{username}, aqui está! {link}"   # message of "Receber sem seguir"
+node tools/omnichat.mjs test videos/<slug> "QUERO"                        # shows opener and delivery
+```
+
+Also in the studio, in the rule panel ("Entregar só quando a pessoa tocar num botão").
+
+**What happens**
+
+1. The agent sends the private reply as a *button template* (text up to 640 characters, button title up to 20),
+   with the payload `omc1|<media>|<comment>`. If Instagram refuses the template for a private reply, the same
+   ask goes as quick replies (`gate.via` in the record says which). The public reply goes as always.
+2. The tap arrives at the webhook (`messaging_postbacks`, or `messages` for a quick reply) and opens the 24 h window.
+3. With `require_follow`, `GET /<igsid>?fields=is_user_follow_business` decides. A follower gets the rule's DM
+   (`dm`, with `{link}`); anyone else gets `not_following_dm` with the `retry_label` button.
+   Whoever taps again still without following gets a second button, `skip_label` ("Receber sem seguir"), and that
+   one delivers the link with `skip_dm` instead of `dm` (default: thanks and a soft ask to follow next time; same
+   `{username}`, `{link}`, `link_button` and `quote_post` rules as `dm`). `follow_skip: false` removes the button.
+4. The record keeps `gate`: `awaiting_tap` → `awaiting_follow` → `delivered`, the taps and the handled event ids
+   (`mids`, so an event Meta repeats never sends twice). A tap handled with an error answers 500 and Meta delivers it again.
+
+**Likes cannot be a criterion.** The API does not say who liked a post (ManyChat can't either). What can be checked:
+commented (the trigger), follows the account, tapped the button.
+
+**One-time setup**
+
+```bash
+node tools/omnichat.mjs webhook setup       # creates INSTAGRAM_WEBHOOK_VERIFY_TOKEN and prints the steps
+# put the app secret in .env: INSTAGRAM_APP_SECRET (Instagram → API setup with Instagram login → App secret)
+node tools/omnichat.mjs webhook serve       # listens on 127.0.0.1:5434/webhook
+cloudflared tunnel --url http://127.0.0.1:5434   # the public HTTPS URL Meta needs
+# Meta panel → Instagram → API setup → Configure webhooks: Callback URL = <tunnel>/webhook, Verify token = the one setup printed,
+# fields messages and messaging_postbacks
+node tools/omnichat.mjs webhook subscribe   # signs the account up to those fields
+node tools/omnichat.mjs webhook status
+```
+
+- The app must be Live. To receive events from people with no role in the app, `instagram_business_manage_messages`
+  needs Advanced Access (and Business Verification); in development mode only the app's testers can tap.
+- Meta checks the callback URL (handshake with the verify token) and signs every event with the app secret
+  (`X-Hub-Signature-256`); the server refuses anything unsigned.
+- The tap is answered by the cloud Worker: the same `handleWebhook` (`tools/lib/omnichat/webhook.mjs`)
+  runs in a function with its own storage; `tools/lib/omnichat/gate.mjs` has the decision and the sends.
+- The record is written under the pass lock, so the agent and the tap don't overwrite each other.
+- Start `webhook serve` and the tunnel only when the user asks; enabling the rule is what authorizes the replies.
+
+## In the cloud (Cloudflare Worker + Postgres)
+
+The same Worker also collects the Reel metrics every hour (`metrics.mjs cloud push`, `cloud.metrics`) and
+publishes the schedule (`publish.mjs cloud push`, `cloud.schedule`, reference/publish.md). Which jobs it
+does is the config's `cloud`: `{ url, omnichat, metrics, schedule }` (the old `omnichat.cloud` still reads).
+
+`cloud/omnichat` is a Worker that answers instead of this machine, with the same core
+(`tools/lib/omnichat`): Meta's webhook (taps, and new comments, which run that post's pass in seconds),
+a fallback pass every 30 min, the token renewal once a day and an API for the CLI and the studio. The
+rule and the record keep the shape of the local files, in Postgres (Neon, over HTTP). No LLM there: an
+uncertain comment waits for a pass with Claude (the CLI).
+
+```bash
+# once: accounts and the Worker
+cd cloud/omnichat && bun install
+npx wrangler login
+psql "$DATABASE_URL" -f schema.sql                       # the Neon database (neon.com, free plan)
+npx wrangler secret put DATABASE_URL                     # and TOKEN_KEY, SYNC_SECRET,
+                                                         # INSTAGRAM_APP_SECRET, INSTAGRAM_WEBHOOK_VERIFY_TOKEN
+npx wrangler deploy                                      # prints https://omnichat.<you>.workers.dev
+# here: cloud.url in the config, OMNICHAT_CLOUD_SECRET (= SYNC_SECRET) in .env
+node tools/omnichat.mjs cloud push --records             # config, account, rules and the local records
+node tools/omnichat.mjs cloud status
+# the switch: Meta's Callback URL → https://omnichat.<you>.workers.dev/webhook (fields messages,
+# messaging_postbacks and comments), then cloud.omnichat: true, agent uninstall, stop webhook serve
+```
+
+- With `owns: false` the cloud only receives; this machine keeps answering. With `owns: true` every saved
+  rule goes up on its own, `log`, `status` and `comments` read the cloud's record (mirrored into
+  omnichat-record.json for the studio), `send` goes through the cloud, `retry` is refused and the local
+  agent stops.
+- Never both answering: two records would each think the other's DM was never sent.
+- The free plans shape it: Neon stays asleep without traffic (a query every minute would burn its
+  100 compute-hours), so the webhook drives and the cron is every 30 min. The Worker gets 10 ms of CPU
+  and 50 subrequests per run, so a cloud pass sends at most 5 DMs and reads 3 comment pages (the rest
+  on the next one). Workers Paid ($5/month) lifts both if a post gets more than that.
+- `node cloud/omnichat/dev-local.mjs` runs the Worker here over an in-memory Postgres, and
+  `OMNICHAT_CLOUD_URL=http://127.0.0.1:8787 OMNICHAT_CLOUD_SECRET=dev` points the CLI at it.
+  `cd cloud/omnichat && bun run test` tests it against PGlite.
+
+## Generic phrases and context
+
+Besides the keyword, generic phrases (`omnichat.instagram.generic_phrases` in the
+config: "quero", "eu quero", "me manda", "manda", "me envia", "envia", "me passa",
+"link") also trigger, by default. With context:
+
+- a comment that is only the ask, with at most the filler words of `filler_words`
+  ("sim", "pfv", "o link", "no direct", emoji…), sends right away: "eu quero!!",
+  "me manda o link pfv", "quero no direct 🙏";
+- an ask in the middle of a sentence is uncertain, and the agent (Claude with `context_model`,
+  on the agent accounts in rotation, `tools/lib/agents.mjs`) reads the comment with the post caption, the link and the
+  rule's `offer` before deciding: "eu quero que faça sol hoje" and "manda mais
+  vídeo assim" don't send; "quero muito testar isso" sends. When in doubt, it
+  doesn't send. What doesn't send stays in the log as `skipped`, with the reason.
+
+`--no-generic` leaves the rule with only the keyword. `test <dir> "comment"`
+shows the decision, including Claude's.
+
+## Meta rules that shape the flow
+
+- One private reply per comment, up to 7 days after it.
+- For people who don't follow the account, the DM lands in **Message requests**.
+  People who closed their messages may not get it: the comment becomes `failed`
+  and the public reply doesn't go (without a DM, "te mandei no direct" would be a
+  lie).
+- The conversation only continues if the person replies or taps a button: that opens the 24 h
+  window. The button and "only if you follow me" are in the next section (they need the
+  messaging webhook); email collection is not built yet.
+- DM text up to 1,000 bytes.
+
+## Files
+
+- `omnichat.json` (in the video's, the carousel's or the post's folder): the rule. Born disabled.
+- `omnichat-record.json`: what the tool did with each comment (`queued`,
+  `retry`, `done`, `failed`, `skipped`), the last one seen and the next check. With
+  more than 1,000 new comments at once (20 pages of 50), the pass stores where it
+  stopped (`resume`, without the token) and the next one continues from there,
+  without skipping the older ones.
+- `.omnichat/`: the pass lock, `pause.json` (a Meta limit, a block or a token
+  without permission pause the account for `rate_limit_pause_min`) and the agent log.
+
+Pace and limits in `omnichat.instagram` in the config: `interval_min` (post
+younger than 48 h, first week, after), `max_per_pass`, `pause_between_s`,
+`attempts`, `default_dm`, `default_public_replies`, `enable_on_publish`,
+`quote_post`. `retry <dir>` puts back in the queue whatever failed without a DM
+(after swapping the token, for example).

package/reference/premiere-mcp.md

@@ -0,0 +1,91 @@
+# Premiere through MCP
+
+## The pieces
+
+| piece | where |
+|---|---|
+| MCP server | npm package `adobe-premiere-pro-mcp` v1.2.8, global on node v24.7.0 |
+| Premiere extension | CEP `MCP Bridge (CEP)`, in `~/Library/Application Support/Adobe/CEP/extensions/MCPBridgeCEP` |
+| channel between the two | files in `/tmp/premiere-mcp-bridge` (command, response, heartbeat) — no network port |
+| registration in Claude Code | `premiere-pro` (user scope), stdio |
+| omnivideo client | `tools/premiere.mjs` |
+
+Premiere 2026 (26.5.2). The extension opens in Window → Extensions (Legacy) →
+MCP Bridge (CEP). CEP debug mode (`PlayerDebugMode`) is on for CSXS versions
+10 to 12.
+
+## How to call it
+
+The server announces only 5 tools: `verify_premiere_connection`,
+`list_sequences`, `search_tools`, `get_tool_schema`, `invoke_tool`. The other
+278 live in the catalog: search with `search_tools`, read the schema with
+`get_tool_schema` and call with `invoke_tool { name, arguments }`.
+`tools/premiere.mjs` takes this detour by itself.
+
+Tools from the expanded catalog respond `{ success, tool, data: {…} }`; the
+domain tools return the fields directly. The client flattens both shapes.
+
+## Tools omnivideo uses
+
+| tool | arguments | returns |
+|---|---|---|
+| `verify_premiere_connection` | — | `status`, `premiere.version`, `project`, `activeSequence` |
+| `create_bin` | `name`, `parentBinName?` | `binId` |
+| `import_media` | `filePath` (absolute) | the item `id` — **ignores `binName`** |
+| `move_item_to_bin` | `projectItemId`, `targetBinId` | |
+| `create_sequence_from_clips` | `name`, `projectItemIds[]` | `sequenceId` |
+| `add_to_timeline` | `sequenceId`, `projectItemId`, `trackIndex`, `time`, `insertMode` | audio goes to the audio track at that index |
+| `create_caption_track` | `sequenceId`, `projectItemId` (imported SRT), `startTime` | |
+| `add_marker` | `sequenceId`, `time`, `name`, `color?`, `duration?` | |
+| `list_sequence_tracks` | `sequenceId` | clips per track with `startTime`/`endTime` |
+| `export_frame` | `sequenceId`, `time`, `outputPath`, `format` | PNG of the frame, captions included |
+| `export_sequence` | `sequenceId`, `outputPath`, `presetPath` | **depends on Media Encoder** |
+| `execute_extendscript` | `script` | runs the text as is; a top-level `return` works |
+| `delete_sequence`, `delete_project_item` | id | to clean up what omnivideo itself created |
+
+## Confirmed limitations
+
+- **No caption styling.** No action changes font, color, size or position.
+- **No caption reading.** `read_sequence_captions` always returns zero; it is a
+  limitation of Premiere's scripting DOM. To check, use `export_frame`.
+- **`create_sequence` ignores fps** and requires a `.sqpreset`. Use
+  `create_sequence_from_clips` (inherits everything from the first clip) or
+  `duplicate_sequence` with `clearContents: true`.
+- **Buggy Media Encoder detection (v1.2.8).** On macOS the server only accepts
+  `/Applications/Adobe Media Encoder <year>.app` directly in `/Applications`
+  (`findInstalledMediaEncoder` in `dist/bridge/index.js`), but Adobe installs
+  it at `/Applications/Adobe Media Encoder 2026/Adobe Media Encoder 2026.app`.
+  Result: `MEDIA_ENCODER_NOT_INSTALLED` with AME installed, and nothing reaches
+  Premiere. `export_sequence` and `add_to_render_queue` go through the same check.
+  **Workaround in use** (approved by the user on 2026-10-12): a symbolic link
+  `/Applications/Adobe Media Encoder 2026.app` → the app inside the folder. To
+  undo: `rm "/Applications/Adobe Media Encoder 2026.app"`. When switching AME
+  versions, redo the link to the new folder.
+- **Export:** `export_sequence` and `add_to_render_queue` queue in Adobe Media
+  Encoder (installed since 2026-10-12, but see the bug above). The alternative is
+  `sequence.exportAsMediaDirect(out, preset, app.encoder.ENCODE_ENTIRE)` via
+  `execute_extendscript`, which the server only allows with explicit approval.
+- **Timeout:** the server waits 60 s for a response and the panel 45 s. A long
+  operation (export) keeps running in Premiere after that; check by the result.
+- **`readOnly: true`** in the `verify_premiere_connection` response only says
+  that this check changes nothing. The other actions write normally.
+
+## Export preset
+
+`/Applications/Adobe Premiere Pro 2026/Adobe Premiere Pro 2026.app/Contents/MediaIO/systempresets/4E49434B_48323634/01 - Match Source - High bitrate.epr`
+
+The `4E49434B_48323634` folder is the H.264 one. The `01 - …` file is "Match
+Source – Adaptive High Bitrate", Premiere's default.
+
+## Bridge down
+
+`bridge_unavailable` with Premiere open: the panel stopped writing the
+heartbeat (`/tmp/premiere-mcp-bridge/bridge-heartbeat.json`). Call
+`verify_premiere_connection` **once** and, if it doesn't come back, ask the
+user to close and reopen the panel.
+
+`verify_premiere_connection` (and `premiere.mjs status`) **opens Premiere** if
+it is closed: `launchIfNeeded` defaults to on. To only check,
+`premiere.mjs call verify_premiere_connection '{"launchIfNeeded": false}'`, or
+read the heartbeat: `t` in milliseconds; stale for more than 2.5 s means the
+bridge is down. That is how the studio sees the bridge.