friend-assistant 0.1.32 → 0.1.34

package/README.md

@@ -85,13 +85,13 @@ Every command starts with `friend`. If you run Friend from the source code, writ
 
 | Command | What it does | Use it when |
 |---|---|---|
-| `friend password` | Sets a new sign-in password (asks twice, nothing is shown while you type). The old one is not needed, because you are at this PC, so it is also the way back in if you forgot it. Every browser and phone is signed out. | You forgot the password, or want a new one. From the screen: Esc → Settings → Security. |
-| `friend pin` | Sets a new PIN, the number Friend asks for before risky actions. | You forgot the PIN, or want a new one. |
-| `friend daemon start` | Runs Friend in the background, with no terminal window needed. Prints the address, keeps a log in the data folder (`friend.log`). | You want Friend always available, even after you close the terminal. |
-| `friend daemon stop` | Stops the background Friend and the speech engine with it. | Shutting it down. |
-| `friend daemon restart` | Stops it and starts it again. | After changing a setting such as `friend lan on` or a different model. |
-| `friend daemon status` | Says whether the background Friend is running, for how long, and at which address. Exit code 0 when running, 1 when not. | Checking, or in a script. |
-| `friend upgrade` | Updates Friend in one go: checks npm for the newest version, stops the background Friend, installs it, and starts it again. Add `--force` to reinstall the same version. If Friend runs in a terminal instead, it asks you to press Ctrl+C there first. | A new version is out. |
+| `friend password` | Sets a new sign-in password (asks twice, nothing is shown while you type). The old one is not needed, because you are at this PC, so it is also the way back in if you forgot it. Every browser and phone is signed out. | You forgot the password, or want a new one. From the screen: Esc → Settings → Security. |
+| `friend pin` | Sets a new PIN, the number Friend asks for before risky actions. | You forgot the PIN, or want a new one. |
+| `friend daemon start` | Runs Friend in the background, with no terminal window needed. Prints the address, keeps a log in the data folder (`friend.log`). | You want Friend always available, even after you close the terminal. |
+| `friend daemon stop` | Stops the background Friend and the speech engine with it. | Shutting it down. |
+| `friend daemon restart` | Stops it and starts it again. | After changing a setting such as `friend lan on` or a different model. |
+| `friend daemon status` | Says whether the background Friend is running, for how long, and at which address. Exit code 0 when running, 1 when not. | Checking, or in a script. |
+| `friend upgrade` | Updates Friend in one go: checks npm for the newest version, stops the background Friend, installs it, and starts it again. Add `--force` to reinstall the same version. If Friend runs in a terminal instead, it asks you to press Ctrl+C there first. | A new version is out. |
 | `friend lan` (or `friend lan status`) | Shows what Friend listens on and the addresses to open. | Checking, or finding the address for your phone. |
 | `friend lan on` | Lets other devices on your network open Friend. Prints the addresses and the things to know (plain http, no microphone on other devices, firewall prompt). | Using Friend from a phone or another PC. |
 | `friend lan on --https` | The same, with a secure `https://` address (Friend makes its own certificate), so a phone may use the microphone and camera. | Talking to Friend from a phone. |
@@ -131,13 +131,13 @@ Connectors (Gmail, Chrome control, any MCP service) are managed inside the app,
 | Key | What it is |
 |---|---|
 | `provider` | Which model answers: `claude` (API key), `claude-code` (your subscription), `ollama` (this PC) or `openai` (any OpenAI-compatible server, such as OmniRoute). |
-| `approvals` | How much Friend asks before acting: `manual` (asks every time, the default), `edits` (file work and the sandboxed code runner go ahead) or `auto` (commands and tools go ahead too). Risky commands (they need your PIN), the camera, adding connectors or plugins, and Friend's own data folder always ask, in every mode. The AI cannot change this. Easier: Esc → Settings → Approvals. |
-| `ownerName` | Your name, shown on the profile badge (a shield in the corner) while you are signed in; it shows whoever Friend just recognized when they speak. Empty shows "Owner". |
+| `approvals` | How much Friend asks before acting: `manual` (asks every time, the default), `edits` (file work and the sandboxed code runner go ahead) or `auto` (commands and tools go ahead too). Risky commands (they need your PIN), the camera, adding connectors or plugins, and Friend's own data folder always ask, in every mode. The AI cannot change this. Easier: Esc → Settings → Approvals. |
+| `ownerName` | Your name, shown on the profile badge (a shield in the corner) while you are signed in; it shows whoever Friend just recognized when they speak. Empty shows "Owner". |
 | `apiKey`, `model` | The Anthropic API key and the Claude model used with it. |
 | `claudeCode.model`, `claudeCode.command`, `claudeCode.fallback` | The subscription route: model (`sonnet`, `haiku`, `opus`), the command that starts Claude Code, and `ollama` or `none` for what to use when Claude is unavailable. |
 | `ollama.url`, `ollama.model`, `ollama.vision`, `ollama.contextTokens` | The local model: server address, model name, whether it can see pictures, and how much it remembers at once. |
-| `openai.url`, `openai.model`, `openai.apiKey`, `openai.vision` | An OpenAI-compatible server: its address (ending in `/v1`), the model name it knows, its API key (or empty), and whether the model can see pictures. For OmniRoute (https://github.com/diegosouzapw/OmniRoute) run `npm install -g omniroute` and `omniroute`, then `friend config set provider openai` (the defaults are OmniRoute's: `http://localhost:20128/v1` and model `auto`). Also works with LM Studio, LiteLLM, vLLM and OpenRouter. |
-| `voice.hindiVoice`, `voice.marathiVoice` | The voices that read Hindi and Marathi (Devanagari) text: `hi_IN-priyamvada-medium`, `hi_IN-pratham-medium`, `hi_IN-rohan-medium`, `mr_IN-google-medium`. Empty makes the main voice read that language. See "Hindi, Marathi and English together". |
+| `openai.url`, `openai.model`, `openai.apiKey`, `openai.vision` | An OpenAI-compatible server: its address (ending in `/v1`), the model name it knows, its API key (or empty), and whether the model can see pictures. For OmniRoute (https://github.com/diegosouzapw/OmniRoute) run `npm install -g omniroute` and `omniroute`, then `friend config set provider openai` (the defaults are OmniRoute's: `http://localhost:20128/v1` and model `auto`). Also works with LM Studio, LiteLLM, vLLM and OpenRouter. |
+| `voice.hindiVoice`, `voice.marathiVoice` | The voices that read Hindi and Marathi (Devanagari) text: `hi_IN-priyamvada-medium`, `hi_IN-pratham-medium`, `hi_IN-rohan-medium`, `mr_IN-google-medium`. Empty makes the main voice read that language. See "Hindi, Marathi and English together". |
 | `voice.enabled`, `voice.sttModel`, `voice.ttsVoice`, `voice.language`, `voice.speakTyped`, `voice.accept`, `voice.challenge` | Speech on or off, the listening model, the speaking voice, a forced language, speaking typed replies, and the two thresholds for recognising your voice. |
 | `bind`, `port` | Where Friend listens (easier: `friend lan`). |
 | `https` | `true` serves a secure `https://` address with a certificate Friend makes for itself, so a phone may use the microphone (easier: `friend lan https on`). |
@@ -196,8 +196,8 @@ pnpm friend setup          # password, PIN, assistant name, workspace folder, AP
 pnpm friend start          # gateway + local device node, http://127.0.0.1:8787
 ```
 
-Sign-in is guarded against guessing: after **three wrong passwords** (or PINs) from one address, that address is locked for a minute, and every wrong try after that doubles the wait (up to an hour). The sign-in screen shows the tries left and a countdown, and even the right password waits out a lock. Many addresses together are limited too. A refresh never shows any part of the home screen before sign-in: the page asks the server first and draws only the sign-in screen until you are in.
-
+Sign-in is guarded against guessing: after **three wrong passwords** (or PINs) from one address, that address is locked for a minute, and every wrong try after that doubles the wait (up to an hour). The sign-in screen shows the tries left and a countdown, and even the right password waits out a lock. Many addresses together are limited too. A refresh never shows any part of the home screen before sign-in: the page asks the server first and draws only the sign-in screen until you are in.
+
 Open the address in a browser, sign in, and press any key to talk. Press `Esc` (or long-press the top-left corner) for the owner menu: mute stage, log out, emergency lock.
 
 Preview the whole look with no API key and no network: `pnpm build && pnpm demo`, then open the printed address. Each message you send shows the next scene (gauges and charts, a timeline, a graph, the orb and ring voice styles, a warm mood).
@@ -285,73 +285,76 @@ Press **Esc** (or long-press the top-left corner), choose **Enroll my voice**, r
 | Another voice (below 0.58) | A guest: polite answer, no tools, none of your conversation or screen, nothing saved. You get a warning with what was said. |
 
 - `friend voice status` shows whether a voice is enrolled; `friend voice forget` deletes it after the PIN (so does the enrollment panel). `friend config set voice.accept 0.75` and `voice.challenge 0.6` tune the two thresholds.
-- **Family voices.** Once your own voice is saved, press **Esc**, then **Family voices**, to add your wife, son, daughter or anyone at home (up to eight people). Hand the device to them: they read a few sentences, you enter your PIN. From then on Friend knows who is talking, calls them by name, and keeps a short conversation with each of them. A recognized person can chat, get help, ask for the weather and have pictures shown. They cannot use your PC, files, camera, messages, settings, notes or schedule, never see your private things, and nothing they say is stored or logged. If two voices sound too alike to tell apart, Friend asks for your PIN instead of guessing. You can remove a person any time (PIN needed).
+- **Family voices.** Once your own voice is saved, press **Esc**, then **Family voices**, to add your wife, son, daughter or anyone at home (up to eight people). Hand the device to them: they read a few sentences, you enter your PIN. From then on Friend knows who is talking, calls them by name, and keeps a short conversation with each of them. A recognized person can chat, get help, ask for the weather and have pictures shown. They cannot use your PC, files, camera, messages, settings, notes or schedule, never see your private things, and nothing they say is stored or logged. If two voices sound too alike to tell apart, Friend asks for your PIN instead of guessing. You can remove a person any time (PIN needed).
 - Emergency lock drops any pending challenge and any enrollment in progress.
 - A voice match is a convenience, not a lock: a recording or a cloned voice can fool any speaker model. That is why a borderline match needs the PIN and risky actions always ask for it.
 - Python tests for the speech engine: `cd packages/voice && python -m venv .venv && .venv/Scripts/pip install -e .[dev] && .venv/Scripts/python -m pytest`.
 
-## Hindi, Marathi and English together
-
-Speak or type in English, Hindi, Marathi, or a mix of them in one sentence ("आज का weather बताओ"). Friend answers in the language you used.
-
-- **Understanding you.** Leave `voice.language` empty so each request is understood in the language you spoke. When it hears Hindi or Marathi it listens a second time with a hint that writes those in Devanagari and keeps English words in Latin letters. The default listening model (`base`) mishears Hindi and Marathi a lot: use `friend config set voice.sttModel small` (about 460 MB, a bit slower), run `friend voice setup` once, and restart.
-- **Answering you.** The assistant is told to write Hindi and Marathi in Devanagari and English in Latin letters. Friend then says each piece with the right voice: Hindi with `voice.hindiVoice` (Priyamvada by default, or Pratham or Rohan), Marathi with `voice.marathiVoice`, English words with your main voice. Hindi or Marathi is told apart by the words used. The first time a voice is used it downloads about 64 MB (Settings → Voice picks them and starts the download). If one cannot be loaded, the main voice reads that language instead of staying silent.
-- **Typed text** works the same: Devanagari in, Devanagari out, spoken with the Hindi or Marathi voice. Roman-letter Hindi ("kya haal hai") is understood by the model but is read by the English voice, so ask for Devanagari when you want it spoken well.
-- Notes: the Hindi voices Priyamvada and Pratham are free for personal use but not for commercial use. A small local model (Ollama) writes Hindi and especially Marathi much worse than Claude does.
-- Other languages are not set up with voices yet: the assistant may answer in them, but they are spoken by the English voice.
-
-## Your screen: it knows what it draws on, and you can keep widgets
-
-- **Friend knows the screen.** Your browser tells Friend the size and kind of its screen (phone, tablet or desktop, portrait or landscape) when it connects and whenever you resize or turn it. The assistant lays things out for that screen: on a phone it shows one or two short pictures in a column, on a wide desktop it uses the whole stage. It also knows which view you chose and which pictures and widgets are showing.
-- **Temporary pictures and widgets.** What Friend shows while it works is temporary: it takes it off the screen when the task is done, and the screen goes back to how you keep it. To keep something, press the pin on its corner, or ask "keep this on screen". A pinned widget is a widget: it stays across refreshes, signing out and in, and restarts, in the place you gave it. "Clear the screen" leaves your widgets alone; the **×** on a widget removes it for good (the assistant never removes or unpins your widgets unless you ask).
-- **Move and resize.** On a computer or tablet, drag an object by the bar at its top and resize it by its bottom-right corner (or focus the bar and use the arrow keys; hold Shift to resize). Objects stay inside the part of the screen that is free of the menu, captions, status line and clock.
-- **The profile badge.** A shield in the corner shows your name (`friend config set ownerName "Your name"`, or Settings → Voice presence) while you are signed in. When Friend recognizes a family member's voice the badge shows their name; an unrecognized voice shows Guest. Drag it where you like; it is hidden in the Minimal view.
-
-## Showing files, and looking at its own screen
-
-- **Widgets.** Besides card, stat, gauge, list, table, code, chart, timeline, graph and sandbox, Friend can show **markdown** (formatted text with tables), **progress** bars, **checklists**, **key-value** lists, a **countdown**, **image**, **video** (sound too), **pdf** and a **gallery** of pictures and clips. A **group** puts several of them into one object (a dashboard) and **tabs** put several views into one, so it does not need a card per fact.
-- **Your files.** Ask "show me C:\\Users\\me\\Pictures\\trip.jpg" or "open my notes.md". Friend shows images and GIFs, video and sound, PDFs, Markdown, text, CSV (as a table) and code. **Every file needs your PIN**, and the approval card shows the full path. The file is served only to your signed-in browser, under a random address; nothing inside Friend's own data folder can be shown. A web address can never be used as a picture or clip, so a page or file cannot make Friend send data out through an image link.
-- **Looking at its own screen.** The assistant can take a picture of the Friend screen to check its layout, **after you enter the PIN**, with no plugin and no screen-sharing prompt. It needs a model that can see pictures. Sandbox pages, video and PDFs appear empty in that picture. Pictures are never kept in the conversation.
-- **The assistant has a manual for the screen.** It can read it any time (the `stage.guide` tool): every widget with its props, how to arrange them, the theme, files and ready recipes. It also knows what to do when the screen already has things on it: a short explanation goes **on top** as an overlay (it floats over a dimmed screen, moves nothing underneath, and leaves by itself after 30 seconds, or when you click away); something **important** clears the screen first (your pinned widgets stay) and is shown alone; a new topic replaces the old one; more of the same topic updates what is there.
-- **The theme.** Friend is a HUD: a near-black blue background, glass panels with a thin glowing edge, one accent colour, small letter-spaced labels and monospace numbers. The assistant is told the exact colours and your current accent, and the pages it draws in a sandbox use your colours automatically (`var(--accent)`, `var(--ink)` and so on).
-
-## Telegram, WhatsApp and SMS (plugins)
-
-These are **plugins**: Esc → **Plugins**. **Telegram** and **WhatsApp and SMS (Twilio)** are listed under *Available*; press **Install** (PIN). Once installed they move to *Installed* with **Configure** (the steps left, and the form for the token, numbers and linking), **Turn on / Turn off** and **Remove** (which forgets the tokens). You can also just ask Friend: "set up Telegram for me". It can install the plugin (you approve with your PIN), tell you exactly which step is next, set what is not secret (a phone number, the on/off switch) and open the Configure page on your screen. A token or password is only ever typed by you, on that page. Two things come out of it: you can **talk to Friend from your phone**, and Friend can **message people for you and read their replies**.
-
-- **Telegram** needs no public address. Create a bot with **@BotFather** (`/newbot`), paste the token, switch it on and save. Then press **Get a link code** (needs your PIN) and send `/link CODE` to your bot from your own Telegram. That chat is now yours: what you write there is answered like on your PC, in the same chat.
-- **SMS and WhatsApp** use one **Twilio** account (account SID, auth token, your Twilio SMS number and WhatsApp sender, and your own phone number). Twilio delivers incoming messages to `https://your-public-address/api/channels/twilio` (set it in Twilio as the "a message comes in" webhook, HTTP POST), so Friend needs a public address (`friend config set publicUrl https://…`). Every request is checked against Twilio's signature, and a message Twilio sends twice is run once.
-- **Who counts as you.** Only your linked Telegram chat and your own phone number. Everyone else who writes to your bot or number goes to an **inbox** as untrusted mail: nothing runs, nothing is answered, you get a notice on the screen, and you can ask Friend "did anyone reply?" (it reads them as data, never as instructions).
-- **Sending.** "Message Anu on WhatsApp that I am late" uses the same approval rule as everything else (Settings → Approvals): in **manual** you read the recipient and the text and approve; in **auto** it goes without asking. Risky actions asked for from a phone are still decided on the PC.
-- **Safe by design.** Tokens are write-only (never shown again, never sent to the screen), the AI cannot change channel settings, one sender cannot flood you (limits per sender and per minute), and a link code works once, for ten minutes, and dies after five wrong guesses. Calls are not built yet.
-
-## Portfolio and brokers (Zerodha)
-
-A plugin: Esc → **Plugins** → **Zerodha** → **Install** (PIN) → **Configure**. Friend reads your Zerodha account through Zerodha's Kite Connect API and answers "how is my portfolio doing?", on screen or by voice, or sends a **report** to Telegram or WhatsApp at the times you choose (for example 9:30 and 3:45 on weekdays). The full walk-through, including the first live login, is in [docs/guides/zerodha-setup.md](docs/guides/zerodha-setup.md).
-
-- **Read only.** Friend can look at holdings, positions, funds and orders. It cannot place, change or cancel orders, never tells you what to buy or sell and never predicts prices. Settings → **Trading** shows *Read only*; the order levels are shown as "later version" and are locked. The assistant cannot change the broker settings or this level, even with your PIN.
-- **Cost.** Nothing extra: no third-party service, Friend talks to Zerodha from your PC. Zerodha's free *Personal* API plan covers everything used here (live streaming and history are on their paid plan, which Friend does not need).
-- **Login.** You sign in on Zerodha's own page, once a day (Zerodha ends the login about 6 am). Your password and 2FA never reach Friend. The API key and secret are write-only; the day's token is kept in a file only you can read. If the login has ended when a report is due, you get the login link on your phone instead, and the report follows once you sign in.
-- **Reports.** Add them on the Configure page (PIN) or ask in chat. Currency sign and word, number format (en-IN, en-US, de-DE...), warning levels, cache time and limits are settings under Report options. Every figure is worked out by Friend from your account, never written by the AI model; a restart never sends one twice. Time zone, market-holiday dates and the login reminder (minutes before the first report) are on the same page. They are also settings: `brokers.timezone`, `brokers.skipDates`, `brokers.loginReminderMinutes`, and `trading.level` (only `read`), all changed from the Settings screen.
-- **Privacy.** The figures you ask about pass through the AI model you chose (Claude from Anthropic, or Ollama on this PC); a report goes to the messaging app you pick (Telegram or WhatsApp). The audit log records that a report was made, never the amounts.
-- **More brokers.** A broker is one file implementing the `BrokerProvider` contract plus the shared contract tests (`runBrokerContract`); reports, tools and screens do not change.
-
-## Alexa (plugin)
-
-A plugin: Esc → **Plugins** → **Alexa** → **Install** (PIN) → **Configure**. You make a small private skill in your own Amazon developer account (free, left in development mode, never published) and Friend is its server, so "Alexa, ask jarvis assistant how my portfolio is" is answered by Friend, aloud and on the Friend screen if one is open. No AWS account and no paid plan. The full walk-through is in [docs/guides/alexa-setup.md](docs/guides/alexa-setup.md).
-
-- **What it does.** Fixed questions (portfolio, weather, tasks, status) are answered straight from Friend's tools. Open questions go to the assistant: start with one of about 30 words ("how", "what", "tell me", "remind me", "can you", "find", "check", "explain", "calculate", "translate"...) and then say anything. By voice the assistant can look things up, answer questions and add reminders; anything beyond that (messages, files, commands, settings, email) is refused or asked on your phone first. English (India) and Hindi (India); Marathi is not available on Alexa.
-- **What it cannot do.** Alexa cannot make an Echo speak on its own (use an Alexa Routine for a scheduled spoken report), and a slow answer is held ("Still working. Ask me if it is ready.") because Alexa waits only about 8 seconds.
-- **Safe by design.** Every request is checked as really from Amazon (certificate, signature, 150-second timestamp), for your skill, from an Echo you allowed with your PIN. The voice can only read: anything that would ask you is sent to your Telegram or WhatsApp for a YES (or skipped), and anything needing your PIN is refused. The assistant cannot change Alexa settings.
-- **Configurable.** Skill name, languages, answer and permission times, request and Echo limits, and extra tools a voice may use are settings on the Configure page (Advanced), with safe ranges; nothing is tied to one person's setup.
-- **Setup.** Friend needs a public HTTPS address with a real certificate (`friend config set publicUrl https://your-address`). The Configure page shows the endpoint, lets you download the skill files (manifest and English and Hindi models) to paste into the console, and lists your Echo devices. If your address changes later, paste the new endpoint into the console again; nothing else changes.
-
+## Hindi, Marathi and English together
+
+Speak or type in English, Hindi, Marathi, or a mix of them in one sentence ("आज का weather बताओ"). Friend answers in the language you used.
+
+- **Understanding you.** Leave `voice.language` empty so each request is understood in the language you spoke. When it hears Hindi or Marathi it listens a second time with a hint that writes those in Devanagari and keeps English words in Latin letters. The default listening model (`base`) mishears Hindi and Marathi a lot: use `friend config set voice.sttModel small` (about 460 MB, a bit slower), run `friend voice setup` once, and restart.
+- **Answering you.** The assistant is told to write Hindi and Marathi in Devanagari and English in Latin letters. Friend then says each piece with the right voice: Hindi with `voice.hindiVoice` (Priyamvada by default, or Pratham or Rohan), Marathi with `voice.marathiVoice`, English words with your main voice. Hindi or Marathi is told apart by the words used. The first time a voice is used it downloads about 64 MB (Settings → Voice picks them and starts the download). If one cannot be loaded, the main voice reads that language instead of staying silent.
+- **Typed text** works the same: Devanagari in, Devanagari out, spoken with the Hindi or Marathi voice. Roman-letter Hindi ("kya haal hai") is understood by the model but is read by the English voice, so ask for Devanagari when you want it spoken well.
+- Notes: the Hindi voices Priyamvada and Pratham are free for personal use but not for commercial use. A small local model (Ollama) writes Hindi and especially Marathi much worse than Claude does.
+- Other languages are not set up with voices yet: the assistant may answer in them, but they are spoken by the English voice.
+
+## Your screen: it knows what it draws on, and you can keep widgets
+
+- **Friend knows the screen.** Your browser tells Friend the size and kind of its screen (phone, tablet or desktop, portrait or landscape) when it connects and whenever you resize or turn it. The assistant lays things out for that screen: on a phone it shows one or two short pictures in a column, on a wide desktop it uses the whole stage. It also knows which view you chose and which pictures and widgets are showing.
+- **Temporary pictures and widgets.** What Friend shows while it works is temporary: it takes it off the screen when the task is done, and the screen goes back to how you keep it. To keep something, press the pin on its corner, or ask "keep this on screen". A pinned widget is a widget: it stays across refreshes, signing out and in, and restarts, in the place you gave it. "Clear the screen" leaves your widgets alone; the **×** on a widget removes it for good (the assistant never removes or unpins your widgets unless you ask).
+- **Move and resize.** On a computer or tablet, drag an object by the bar at its top and resize it by its bottom-right corner (or focus the bar and use the arrow keys; hold Shift to resize). Objects stay inside the part of the screen that is free of the menu, captions, status line and clock.
+- **The profile badge.** A shield in the corner shows your name (`friend config set ownerName "Your name"`, or Settings → Voice presence) while you are signed in. When Friend recognizes a family member's voice the badge shows their name; an unrecognized voice shows Guest. Drag it where you like; it is hidden in the Minimal view.
+
+## Showing files, and looking at its own screen
+
+- **Widgets.** Besides card, stat, gauge, list, table, code, chart, timeline, graph and sandbox, Friend can show **markdown** (formatted text with tables), **progress** bars, **checklists**, **key-value** lists, a **countdown**, **image**, **video** (sound too), **pdf** and a **gallery** of pictures and clips. A **group** puts several of them into one object (a dashboard) and **tabs** put several views into one, so it does not need a card per fact.
+- **Your files.** Ask "show me C:\\Users\\me\\Pictures\\trip.jpg" or "open my notes.md". Friend shows images and GIFs, video and sound, PDFs, Markdown, text, CSV (as a table) and code. **Every file needs your PIN**, and the approval card shows the full path. The file is served only to your signed-in browser, under a random address; nothing inside Friend's own data folder can be shown. A web address can never be used as a picture or clip, so a page or file cannot make Friend send data out through an image link.
+- **Looking at its own screen.** The assistant can take a picture of the Friend screen to check its layout, **after you enter the PIN**, with no plugin and no screen-sharing prompt. It needs a model that can see pictures. Sandbox pages, video and PDFs appear empty in that picture. Pictures are never kept in the conversation.
+- **The assistant has a manual for the screen.** It can read it any time (the `stage.guide` tool): every widget with its props, how to arrange them, the theme, files and ready recipes. It also knows what to do when the screen already has things on it: a short explanation goes **on top** as an overlay (it floats over a dimmed screen, moves nothing underneath, and leaves by itself after 30 seconds, or when you click away); something **important** clears the screen first (your pinned widgets stay) and is shown alone; a new topic replaces the old one; more of the same topic updates what is there.
+- **The theme.** Friend is a HUD: a near-black blue background, glass panels with a thin glowing edge, one accent colour, small letter-spaced labels and monospace numbers. The assistant is told the exact colours and your current accent, and the pages it draws in a sandbox use your colours automatically (`var(--accent)`, `var(--ink)` and so on).
+
+## Telegram, WhatsApp and SMS (plugins)
+
+These are **plugins**: Esc → **Plugins**. **Telegram** and **WhatsApp and SMS (Twilio)** are listed under *Available*; press **Install** (PIN). Once installed they move to *Installed* with **Configure** (the steps left, and the form for the token, numbers and linking), **Turn on / Turn off** and **Remove** (which forgets the tokens). You can also just ask Friend: "set up Telegram for me". It can install the plugin (you approve with your PIN), tell you exactly which step is next, set what is not secret (a phone number, the on/off switch) and open the Configure page on your screen. A token or password is only ever typed by you, on that page. Two things come out of it: you can **talk to Friend from your phone**, and Friend can **message people for you and read their replies**.
+
+- **Telegram** needs no public address. Create a bot with **@BotFather** (`/newbot`), paste the token, switch it on and save. Then press **Get a link code** (needs your PIN) and send `/link CODE` to your bot from your own Telegram. That chat is now yours: what you write there is answered like on your PC, in the same chat.
+- **SMS and WhatsApp** use one **Twilio** account (account SID, auth token, your Twilio SMS number and WhatsApp sender, and your own phone number). Twilio delivers incoming messages to `https://your-public-address/api/channels/twilio` (set it in Twilio as the "a message comes in" webhook, HTTP POST), so Friend needs a public address (`friend config set publicUrl https://…`). Every request is checked against Twilio's signature, and a message Twilio sends twice is run once.
+- **Who counts as you.** Only your linked Telegram chat and your own phone number. Everyone else who writes to your bot or number goes to an **inbox** as untrusted mail: nothing runs, nothing is answered, you get a notice on the screen, and you can ask Friend "did anyone reply?" (it reads them as data, never as instructions).
+- **Sending.** "Message Anu on WhatsApp that I am late" uses the same approval rule as everything else (Settings → Approvals): in **manual** you read the recipient and the text and approve; in **auto** it goes without asking. Risky actions asked for from a phone are still decided on the PC.
+- **Safe by design.** Tokens are write-only (never shown again, never sent to the screen), the AI cannot change channel settings, one sender cannot flood you (limits per sender and per minute), and a link code works once, for ten minutes, and dies after five wrong guesses. Calls are not built yet.
+
+## Portfolio and brokers (Zerodha)
+
+A plugin: Esc → **Plugins** → **Zerodha** → **Install** (PIN) → **Configure**. Friend reads your Zerodha account through Zerodha's Kite Connect API and answers "how is my portfolio doing?", on screen or by voice, or sends a **report** to Telegram or WhatsApp at the times you choose (for example 9:30 and 3:45 on weekdays). The full walk-through, including the first live login, is in [docs/guides/zerodha-setup.md](docs/guides/zerodha-setup.md).
+
+- **Read only.** Friend can look at holdings, positions, funds and orders. It cannot place, change or cancel orders, never tells you what to buy or sell and never predicts prices. Settings → **Trading** shows *Read only*; the order levels are shown as "later version" and are locked. The assistant cannot change the broker settings or this level, even with your PIN.
+- **Cost.** Nothing extra: no third-party service, Friend talks to Zerodha from your PC. Zerodha's free *Personal* API plan covers everything used here (live streaming and history are on their paid plan, which Friend does not need).
+- **Login.** You sign in on Zerodha's own page, once a day (Zerodha ends the login about 6 am). Your password and 2FA never reach Friend. The API key and secret are write-only; the day's token is kept in a file only you can read. If the login has ended when a report is due, you get the login link on your phone instead, and the report follows once you sign in.
+- **Reports.** Add them on the Configure page (PIN) or ask in chat. Currency sign and word, number format (en-IN, en-US, de-DE...), warning levels, cache time and limits are settings under Report options. Every figure is worked out by Friend from your account, never written by the AI model; a restart never sends one twice. Time zone, market-holiday dates and the login reminder (minutes before the first report) are on the same page. They are also settings: `brokers.timezone`, `brokers.skipDates`, `brokers.loginReminderMinutes`, and `trading.level` (only `read`), all changed from the Settings screen.
+- **Privacy.** The figures you ask about pass through the AI model you chose (Claude from Anthropic, or Ollama on this PC); a report goes to the messaging app you pick (Telegram or WhatsApp). The audit log records that a report was made, never the amounts.
+- **More brokers.** A broker is one file implementing the `BrokerProvider` contract plus the shared contract tests (`runBrokerContract`); reports, tools and screens do not change.
+
+## Alexa (plugin)
+
+A plugin: Esc → **Plugins** → **Alexa** → **Install** (PIN) → **Configure**. You make a small private skill in your own Amazon developer account (free, left in development mode, never published) and Friend is its server, so "Alexa, ask jarvis assistant how my portfolio is" is answered by Friend, aloud and on the Friend screen if one is open. No AWS account and no paid plan. The full walk-through is in [docs/guides/alexa-setup.md](docs/guides/alexa-setup.md).
+
+- **What it does.** Fixed questions (portfolio, weather, tasks, status) are answered straight from Friend's tools. Open questions go to the assistant: start with one of about 30 words ("how", "what", "tell me", "remind me", "can you", "find", "check", "explain", "calculate", "translate"...) and then say anything. By voice the assistant can look things up, answer questions and add reminders; anything beyond that (messages, files, commands, settings, email) is refused or asked on your phone first. English (India) and Hindi (India); Marathi is not available on Alexa.
+- **What it cannot do.** Alexa cannot make an Echo speak on its own (use an Alexa Routine for a scheduled spoken report), and a slow answer is held ("Still working. Ask me if it is ready.") because Alexa waits only about 8 seconds.
+- **Safe by design.** Every request is checked as really from Amazon (certificate, signature, 150-second timestamp), for your skill, from an Echo you allowed with your PIN. The voice can only read: anything that would ask you is sent to your Telegram or WhatsApp for a YES (or skipped), and anything needing your PIN is refused. The assistant cannot change Alexa settings.
+- **Configurable.** Skill name, languages, answer and permission times, request and Echo limits, and extra tools a voice may use are settings on the Configure page (Advanced), with safe ranges; nothing is tied to one person's setup.
+- **Setup.** Friend needs a public HTTPS address with a real certificate (`friend config set publicUrl https://your-address`). The Configure page shows the endpoint, lets you download the skill files (manifest and English and Hindi models) to paste into the console, and lists your Echo devices. If your address changes later, paste the new endpoint into the console again; nothing else changes.
+
 ## Music player and screen widgets
 
 Friend's screen has a media player and four widgets drawn in a HUD style (angled corners, thin lines, your accent colour): a **profile** tag, a **date and time**, **quick-launch buttons** and the **music player**. Esc → **Settings** → **Widgets** (PIN) sets them all.
 
 - **Music.** Pick a folder on the PC that runs Friend (sub-folders are searched; the folder is checked when you save). Friend reads the tags and cover art from the files (free, no account) and plays them in the browser, with a playlist, shuffle, repeat, volume and search. The file types, the most tracks, how deep to search, the start volume, repeat, shuffle and the moving bars are all settings. Files are only read from inside the folder; links that point out of it are not followed.
 - **Spotify (plugin).** Connect your own Spotify account with OAuth (Plugins → Spotify): the assistant and the player can then search and play from it next to your folder, and the player's Library tab gets a Spotify choice. You sign in on Spotify's own page; Friend never sees your password. Playing needs Spotify Premium on your account and a free app of your own at developer.spotify.com (the guide walks through it: [docs/guides/spotify-setup.md](docs/guides/spotify-setup.md)). Any plan or approval a service asks for is on your own account; Friend never pays for or shares one. JioSaavn is not supported: it has no public API and its policy forbids unlicensed access.
+- **SoundCloud (plugin).** The same for your own SoundCloud account (Plugins → SoundCloud): you register a free app of your own (SoundCloud decides who gets one), paste its client id and secret (the secret is write-only), and sign in on SoundCloud's own page. Tracks stream through Friend to the screen, or play from the PC through mpv. See `docs/guides/soundcloud-setup.md`.
+- **Voice messages in Telegram and WhatsApp.** Send Friend a voice message from your own linked Telegram chat or your own WhatsApp number and it is understood and answered like a typed one (the reply starts with what it heard). It uses the same voice engine as the PC microphone, so that must be running; if not, you are told. Up to 5 minutes or 8 MB; from anyone else voice messages are ignored.
+- **Audius (plugin).** Free music from Audius, an open music service: no account, no key, no payment. Install it from Plugins and switch it on; the assistant and the player can then search and play it, on the screen or from the PC. See `docs/guides/audius-setup.md`.
 - **Play from the PC's speakers.** Settings → Widgets → *Where the sound comes out*: the screen's browser (default) or the PC that runs Friend, with no browser open. The folder's music is played by mpv (free; "Check mpv" finds it or shows the install command for your system) and Spotify on a Spotify device you name. The screen then shows what the PC plays and controls it. See [docs/guides/host-output.md](docs/guides/host-output.md).
 - **Control by the assistant.** Say or type "play something by ...", "next", "turn it down", "shuffle", "what's playing". The assistant can play, pause, skip, seek, change volume, shuffle, repeat and edit the queue, and search the library; it cannot change the folder or any widget setting. Over Alexa the music tools are off unless you add them to the voice tools yourself.
 - **Quick launch.** Up to 12 buttons, each opening an address (http or https only), asking the assistant something, opening a panel (chat, tasks, plugins, settings, music) or controlling the music.

package/dist/BUILD.json

@@ -1,5 +1,5 @@
 {
   "name": "friend-assistant",
-  "version": "0.1.32",
-  "builtAt": "2026-10-11T08:58:51.365Z"
+  "version": "0.1.34",
+  "builtAt": "2026-10-11T14:55:42.233Z"
 }