brainiac 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +16 -3
- data/bin/brainiac +48 -0
- data/bin/brainiac-completion.bash +1 -1
- data/docs/multi-machine.md +380 -0
- data/lib/brainiac/cron.rb +4 -0
- data/lib/brainiac/handlers/shared/inline_tags.rb +10 -1
- data/lib/brainiac/helpers.rb +72 -3
- data/lib/brainiac/hooks.rb +2 -0
- data/lib/brainiac/hub/claims.rb +36 -0
- data/lib/brainiac/hub/cli.rb +217 -0
- data/lib/brainiac/hub/client.rb +49 -0
- data/lib/brainiac/hub/inbox.rb +84 -0
- data/lib/brainiac/hub/ledger.rb +41 -0
- data/lib/brainiac/hub/node.rb +207 -0
- data/lib/brainiac/hub/settings.rb +46 -0
- data/lib/brainiac/hub/tunnel.rb +83 -0
- data/lib/brainiac/hub.rb +86 -0
- data/lib/brainiac/version.rb +1 -1
- data/receiver.rb +33 -13
- data/templates/cli-providers/hooks/kiro-agent-setup.sh.example +4 -2
- metadata +11 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: dc86815004b7331d0a347ab374b33d38101f00797d72c800c78e57f55bdceba5
|
|
4
|
+
data.tar.gz: e9e59676e2b9882290586f4b026d306b28c78a4dceb52461ce3c6dcb927ab8e0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 19e53ec087a79079b449ed54ae3fba0d591ca07865edc202bd94a2bb0ed34bcdd129e01a61408513f27f5801ff4b8f330b283f7d65b7b6b351b0d2fb9967e69a
|
|
7
|
+
data.tar.gz: e2c3e3a7cf5c7f4dee801a7362debbecc33a273c5133af7faaa3a41bd849ab903f4296b03a0db49c7a0be7e0db812c32944f4b85767185c4cb181c3475bfa894
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
- Multi-machine support through a `brainiac-hub`. With a `hub` block in `brainiac.json`, the server checks in with the hub, runs a tunnel (or reports a fixed public URL), acknowledges hub deliveries before running them, and only the leader machine runs cron and the Discord gateways. New `brainiac hub` and `brainiac machines` commands, new `:hub_leader_acquired` / `:hub_leader_lost` hooks. Off unless configured. See `docs/multi-machine.md`.
|
|
6
|
+
|
|
3
7
|
## 0.9.2 — 2026-11-02
|
|
4
8
|
|
|
5
9
|
- Add filesystem-search guardrail to PROMPT_CORE — forbids `find /`, `find ~`, `grep -r /`, and `locate`, which hang indefinitely on macOS (autofs, iCloud, sealed sim volumes). Teaches agents to use `gem which` / `Gem.loaded_specs` and scope `find` to known gem/repo roots with a depth limit.
|
data/README.md
CHANGED
|
@@ -238,7 +238,7 @@ Writing a plugin that should take part in this? See "Agent Onboarding (`on_agent
|
|
|
238
238
|
|------------|----------|---------|
|
|
239
239
|
| Ruby 3.4+ | Yes | [mise](https://mise.jdx.dev), rbenv, or system |
|
|
240
240
|
| An AI agent CLI | Yes | [Kiro CLI](https://kiro.dev), [Grok CLI](https://x.ai), or any CLI that accepts prompts on stdin |
|
|
241
|
-
| [ngrok](https://ngrok.com) | Yes | Webhook tunneling |
|
|
241
|
+
| [ngrok](https://ngrok.com) or [cloudflared](https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/) | Yes | Webhook tunneling (cloudflared for [multi-machine](docs/multi-machine.md)) |
|
|
242
242
|
| [qmd](https://github.com/tobi/qmd) | For brain | `npm install -g @tobilu/qmd` (Node.js >= 22) |
|
|
243
243
|
| [Fizzy CLI](https://github.com/robzolkos/fizzy-cli) | For `brainiac-fizzy` | Card management |
|
|
244
244
|
| [GitHub CLI](https://cli.github.com) (`gh`) | For `brainiac-github` | PR/issue operations |
|
|
@@ -452,6 +452,15 @@ Andy's Linux box Adam's macOS
|
|
|
452
452
|
|
|
453
453
|
Both machines receive the same webhooks. The agent registry (`~/.brainiac/agents.json`) controls which agents are local — only agents marked `"local": true` will be dispatched on that machine.
|
|
454
454
|
|
|
455
|
+
### One fleet, several machines
|
|
456
|
+
|
|
457
|
+
The setup above gives each machine its own agents. To run the *same* agents on several machines (a desktop, a mini-PC, a GPU box) without two of them answering the same event, put a hub in front: webhooks go to the hub, and the hub pushes each one to a single machine. See [docs/multi-machine.md](docs/multi-machine.md).
|
|
458
|
+
|
|
459
|
+
```bash
|
|
460
|
+
brainiac hub setup --url <hub url> --token <node token> --machine mini --tunnel cloudflared
|
|
461
|
+
brainiac machines # who is online, who holds Discord and cron
|
|
462
|
+
```
|
|
463
|
+
|
|
455
464
|
### Step 1: Agent Registry
|
|
456
465
|
|
|
457
466
|
`brainiac agent create` writes the entry, persona, role and plugin tokens for you (see [Adding an Agent](#adding-an-agent)). The rest of this section explains what it produces, in case you want to edit it by hand.
|
|
@@ -770,7 +779,11 @@ Channel mappings and authorization are stored in `~/.brainiac/discord.json`:
|
|
|
770
779
|
|
|
771
780
|
`owner_discord_id` identifies the server owner for admin-level notifications.
|
|
772
781
|
|
|
773
|
-
`dashboard_token` protects the web dashboard at `/dashboard
|
|
782
|
+
`dashboard_token` protects the web dashboard at `/dashboard` and the admin API under `/api/`. Requests made on the machine itself (the CLI, waybar) never need it. Any other request, including one that arrives through an ngrok or Cloudflare tunnel, must send it as `?token=` or `Authorization: Bearer`. With no `dashboard_token` set, remote requests to those paths are refused.
|
|
783
|
+
|
|
784
|
+
Run `brainiac token` to generate one and save it to `brainiac.json` (`brainiac token --rotate` replaces it), then `brainiac restart`.
|
|
785
|
+
|
|
786
|
+
Refused attempts are logged as `[Security]`. To also be told about them, set the `NOTIFICATION_COMMAND` env var (for example `notify-send`) or add a `"security"` entry under `notifications`. Alerts are limited to one per ten minutes per kind.
|
|
774
787
|
|
|
775
788
|
`user_mappings` maps display names to Discord user IDs. This serves two purposes: agents use it to format proper `<@ID>` mentions (plain text `@Name` doesn't work in Discord), and the bot recognition system uses it to identify remote agent bots running on other machines. Add humans, remote agents, and anyone else the agents might need to mention. Get IDs by enabling Developer Mode in Discord and right-clicking a user.
|
|
776
789
|
|
|
@@ -975,7 +988,7 @@ If the target channel is a forum channel, a new forum post is created with a cus
|
|
|
975
988
|
|
|
976
989
|
## Web Dashboard
|
|
977
990
|
|
|
978
|
-
Brainiac includes a web dashboard at `http://localhost:4567/dashboard` that shows active agent sessions, project status, and system health.
|
|
991
|
+
Brainiac includes a web dashboard at `http://localhost:4567/dashboard` that shows active agent sessions, project status, and system health. From any other machine it needs the `dashboard_token` from `~/.brainiac/brainiac.json`; without one, remote access is refused.
|
|
979
992
|
|
|
980
993
|
## Monitoring
|
|
981
994
|
|
data/bin/brainiac
CHANGED
|
@@ -8,6 +8,7 @@ require "open3"
|
|
|
8
8
|
require "net/http"
|
|
9
9
|
require "uri"
|
|
10
10
|
require "time"
|
|
11
|
+
require "securerandom"
|
|
11
12
|
|
|
12
13
|
# Brainiac CLI - Register and manage projects with Brainiac server
|
|
13
14
|
|
|
@@ -692,6 +693,36 @@ def update_handler_config(name, enabled)
|
|
|
692
693
|
puts " Restart the server for changes to take effect: brainiac restart"
|
|
693
694
|
end
|
|
694
695
|
|
|
696
|
+
# Generate the dashboard/admin API token and store it in brainiac.json.
|
|
697
|
+
# An existing token is kept unless rotate is set, so re-running the command
|
|
698
|
+
# never locks out whoever already holds it.
|
|
699
|
+
def configure_dashboard_token(rotate: false)
|
|
700
|
+
brainiac_config_path = File.join(BRAINIAC_DIR, "brainiac.json")
|
|
701
|
+
config = File.exist?(brainiac_config_path) ? JSON.parse(File.read(brainiac_config_path)) : {}
|
|
702
|
+
abort "Error: #{brainiac_config_path} must contain a JSON object." unless config.is_a?(Hash)
|
|
703
|
+
|
|
704
|
+
existing = config["dashboard_token"].to_s
|
|
705
|
+
if !existing.empty? && !rotate
|
|
706
|
+
File.chmod(0o600, brainiac_config_path)
|
|
707
|
+
puts "A dashboard token is already set in #{brainiac_config_path}."
|
|
708
|
+
puts " Replace it with: brainiac token --rotate"
|
|
709
|
+
return
|
|
710
|
+
end
|
|
711
|
+
|
|
712
|
+
token = SecureRandom.hex(32)
|
|
713
|
+
config["dashboard_token"] = token
|
|
714
|
+
ensure_brainiac_dir
|
|
715
|
+
File.write(brainiac_config_path, JSON.pretty_generate(config))
|
|
716
|
+
File.chmod(0o600, brainiac_config_path)
|
|
717
|
+
|
|
718
|
+
puts "✓ Dashboard token #{existing.empty? ? "generated" : "rotated"} and saved to #{brainiac_config_path}"
|
|
719
|
+
puts " Token: #{token}"
|
|
720
|
+
puts " Remote requests to /dashboard and /api/ must send it as ?token= or Authorization: Bearer."
|
|
721
|
+
puts " Restart the server for changes to take effect: brainiac restart"
|
|
722
|
+
rescue JSON::ParserError => e
|
|
723
|
+
abort "Error: #{brainiac_config_path} is not valid JSON, nothing written (#{e.message})"
|
|
724
|
+
end
|
|
725
|
+
|
|
695
726
|
# After installing a plugin, load it and run its setup if it provides one.
|
|
696
727
|
# Only runs if the plugin defines .cli and responds to a "setup" subcommand via .configured?
|
|
697
728
|
def run_plugin_setup(name, local_path: nil)
|
|
@@ -1865,6 +1896,14 @@ when "config"
|
|
|
1865
1896
|
|
|
1866
1897
|
configure_server(options)
|
|
1867
1898
|
|
|
1899
|
+
when "token"
|
|
1900
|
+
OptionParser.new do |opts|
|
|
1901
|
+
opts.banner = "Usage: brainiac token [options]"
|
|
1902
|
+
opts.on("--rotate", "Replace the existing token")
|
|
1903
|
+
end.parse!(into: options)
|
|
1904
|
+
|
|
1905
|
+
configure_dashboard_token(rotate: options[:rotate])
|
|
1906
|
+
|
|
1868
1907
|
when "server", "s", "start"
|
|
1869
1908
|
OptionParser.new do |opts|
|
|
1870
1909
|
opts.banner = "Usage: brainiac server [options]"
|
|
@@ -3844,6 +3883,12 @@ when "uninstall"
|
|
|
3844
3883
|
puts " To fully remove: gem uninstall #{gem_name}"
|
|
3845
3884
|
puts " Restart the server to apply: brainiac restart"
|
|
3846
3885
|
|
|
3886
|
+
when "hub", "machines"
|
|
3887
|
+
# Multi-machine: this machine as a node of a brainiac-hub (docs/multi-machine.md)
|
|
3888
|
+
require_relative "../lib/brainiac/hub/cli"
|
|
3889
|
+
ARGV.unshift("status") if subcommand == "machines"
|
|
3890
|
+
exit Brainiac::Hub::CLI.new(config_path: File.join(BRAINIAC_DIR, "brainiac.json")).run(ARGV)
|
|
3891
|
+
|
|
3847
3892
|
when "plugins"
|
|
3848
3893
|
plugins_cmd = ARGV.shift
|
|
3849
3894
|
|
|
@@ -4791,11 +4836,14 @@ when "help", "--help", "-h", nil
|
|
|
4791
4836
|
brainiac plugins List installed plugins
|
|
4792
4837
|
brainiac brain <command> Manage agent long-term memory (brain)
|
|
4793
4838
|
brainiac cron <command> Manage scheduled agent tasks
|
|
4839
|
+
brainiac hub <command> Multi-machine: connect to a hub, see machines, move work
|
|
4840
|
+
brainiac machines Short for 'brainiac hub status'
|
|
4794
4841
|
brainiac provider <command> Manage CLI providers
|
|
4795
4842
|
brainiac role <command> Manage agent roles
|
|
4796
4843
|
brainiac agent <command> Manage agent registry (env, list, show)
|
|
4797
4844
|
brainiac screenshot <app> <path> Take screenshot with 1Password credentials
|
|
4798
4845
|
brainiac config Configure Brainiac CLI
|
|
4846
|
+
brainiac token [--rotate] Generate the dashboard/admin API token and save it to brainiac.json
|
|
4799
4847
|
brainiac path Show Brainiac config directory
|
|
4800
4848
|
brainiac version Show version
|
|
4801
4849
|
brainiac help Show this help message
|
|
@@ -9,7 +9,7 @@ _brainiac() {
|
|
|
9
9
|
local brainiac_dir="${BRAINIAC_DIR:-$HOME/.brainiac}"
|
|
10
10
|
|
|
11
11
|
# Top-level commands (built-in + installed plugins)
|
|
12
|
-
local commands="server start stop restart logs status register unregister list show brain cron provider role agent config path version help setup projects project card-map handler plugin install uninstall plugins"
|
|
12
|
+
local commands="server start stop restart logs status register unregister list show brain cron provider role agent config token path version help setup projects project card-map handler plugin install uninstall plugins hub machines"
|
|
13
13
|
|
|
14
14
|
# Add installed plugin names as top-level commands
|
|
15
15
|
if [[ -f "$brainiac_dir/plugins.json" ]]; then
|
|
@@ -0,0 +1,380 @@
|
|
|
1
|
+
# Running Brainiac on more than one machine
|
|
2
|
+
|
|
3
|
+
Run the same agents on several machines without two of them answering the same event.
|
|
4
|
+
|
|
5
|
+
- [How it works](#how-it-works)
|
|
6
|
+
- [What you need](#what-you-need)
|
|
7
|
+
- [Part 1: Deploy the hub (once)](#part-1-deploy-the-hub-once)
|
|
8
|
+
- [Part 2: Connect the machine you already run](#part-2-connect-the-machine-you-already-run)
|
|
9
|
+
- [Part 3: Add another machine](#part-3-add-another-machine)
|
|
10
|
+
- [Choosing which machine does the work](#choosing-which-machine-does-the-work)
|
|
11
|
+
- [Discord: choosing the machine](#discord-choosing-the-machine)
|
|
12
|
+
- [Discord and cron: the leader](#discord-and-cron-the-leader)
|
|
13
|
+
- [Tunnels: how the hub reaches a machine](#tunnels-how-the-hub-reaches-a-machine)
|
|
14
|
+
- [When a machine is offline](#when-a-machine-is-offline)
|
|
15
|
+
- [Commands](#commands)
|
|
16
|
+
- [Settings reference](#settings-reference)
|
|
17
|
+
- [Limits](#limits)
|
|
18
|
+
- [Troubleshooting](#troubleshooting)
|
|
19
|
+
|
|
20
|
+
## How it works
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
Fizzy / GitHub / other webhooks
|
|
24
|
+
|
|
|
25
|
+
brainiac-hub a small cloud app (AWS Lambda + DynamoDB)
|
|
26
|
+
picks ONE machine per event
|
|
27
|
+
/ | \
|
|
28
|
+
main box mini-PC Windows (WSL2)
|
|
29
|
+
each runs `brainiac server` and a tunnel the hub can push to
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
- **The hub** is the only address your webhooks point at. For each event it picks one machine and pushes the event to it. It never runs agents and holds no webhook secrets or bot tokens.
|
|
33
|
+
- **A machine** (a "node") runs `brainiac server` as usual. On start it checks in with the hub: "I'm `mini`, reach me at this URL". It checks in again every two minutes.
|
|
34
|
+
- **Work is sticky.** The first event for a card picks a machine. Every later event for that card, its branch and its pull request goes to the same machine, because the worktree and the CLI session live there.
|
|
35
|
+
- **One machine is the leader.** Discord delivers messages over a connection that has to stay open, which a serverless hub cannot hold. So one machine holds the Discord connections and runs cron. If it goes away, another machine takes over.
|
|
36
|
+
|
|
37
|
+
Nothing changes on a machine with no `hub` settings. Single-machine setups keep working as they are.
|
|
38
|
+
|
|
39
|
+
## What you need
|
|
40
|
+
|
|
41
|
+
- An AWS account for the hub. Expect roughly $0 to $1 a month: three machines checking in every two minutes is about 65,000 requests a month.
|
|
42
|
+
- On every machine: Brainiac, your plugins, your agent CLIs logged in, your repos cloned, and a tunnel (see [Tunnels](#tunnels-how-the-hub-reaches-a-machine)). `cloudflared` is free and needs no account.
|
|
43
|
+
- `brainiac-discord` at a version with leader support, if you use Discord. An older plugin opens the Discord connections on every machine and every bot answers once per machine.
|
|
44
|
+
- Windows machines run Brainiac inside WSL2. Follow the Linux steps there.
|
|
45
|
+
|
|
46
|
+
## Part 1: Deploy the hub (once)
|
|
47
|
+
|
|
48
|
+
The hub lives in its own repository, `brainiac-hub`. Its README has the full deploy steps. The short version:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
git clone <your brainiac-hub repo> && cd brainiac-hub
|
|
52
|
+
bundle install
|
|
53
|
+
belt setup state # once per AWS account: S3 bucket for Terraform state
|
|
54
|
+
belt deploy prod
|
|
55
|
+
belt output prod # prints api_url
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
You need two values from the deploy:
|
|
59
|
+
|
|
60
|
+
1. **Hub URL**: the `api_url` output, for example `https://abc123.execute-api.us-east-1.amazonaws.com/prod`.
|
|
61
|
+
2. **Node token**: the shared secret every machine uses.
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
aws ssm get-parameter --name /brainiac-hub/prod/node_token \
|
|
65
|
+
--with-decryption --query Parameter.Value --output text
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Part 2: Connect the machine you already run
|
|
69
|
+
|
|
70
|
+
Do this on the machine that runs Brainiac today.
|
|
71
|
+
|
|
72
|
+
**1. Update Brainiac and the Discord plugin**, then install a tunnel:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
brainiac update
|
|
76
|
+
# Arch: sudo pacman -S cloudflared
|
|
77
|
+
# macOS: brew install cloudflared
|
|
78
|
+
# Debian/Ubuntu: see "Tunnels" below
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
**2. Point this machine at the hub.**
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
brainiac hub setup \
|
|
85
|
+
--url https://abc123.execute-api.us-east-1.amazonaws.com/prod \
|
|
86
|
+
--token <node token> \
|
|
87
|
+
--machine main \
|
|
88
|
+
--priority 20 \
|
|
89
|
+
--tunnel cloudflared
|
|
90
|
+
brainiac restart
|
|
91
|
+
brainiac machines # should list "main" as online and as the leader
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
If this machine already has a public address (an ngrok static domain, a named Cloudflare tunnel), use `--public-url https://...` instead of `--tunnel cloudflared` and keep running that tunnel yourself.
|
|
95
|
+
|
|
96
|
+
**3. Move your webhooks to the hub.** The path after `/hooks` is the path the plugin serves on a machine.
|
|
97
|
+
|
|
98
|
+
| Plugin | Old URL (your tunnel) | New URL (the hub) |
|
|
99
|
+
|---|---|---|
|
|
100
|
+
| Fizzy | `https://<tunnel>/fizzy/<board>` | `https://<hub>/hooks/fizzy/<board>` |
|
|
101
|
+
| GitHub | `https://<tunnel>/github` | `https://<hub>/hooks/github` |
|
|
102
|
+
| Zoho | `https://<tunnel>/zoho` | `https://<hub>/hooks/zoho` |
|
|
103
|
+
| Basecamp | `https://<tunnel>/basecamp` | `https://<hub>/hooks/basecamp` |
|
|
104
|
+
|
|
105
|
+
For GitHub, change the Payload URL in the app or repository webhook settings and keep the content type as `application/json`. For Fizzy, change the webhook's URL in the board's settings if it lets you edit it in place. The hub forwards the body and signature untouched, so as long as the signing secret is unchanged the machine checks them as before.
|
|
106
|
+
|
|
107
|
+
The plugin can also replace the Fizzy webhook for you (`brainiac fizzy board webhook <board> https://<hub>/hooks/fizzy/<board>`), but that creates a **new** signing secret and saves it to this machine's `fizzy.json`. If you do that after adding other machines, copy `fizzy.json` to them again.
|
|
108
|
+
|
|
109
|
+
**4. Test it.** Comment on a card. `brainiac logs` on the machine shows a line like `[Hub] Job <id> /fizzy/<board> -> 200`.
|
|
110
|
+
|
|
111
|
+
At this point you have one machine behind the hub. Everything behaves as before, and you can add machines.
|
|
112
|
+
|
|
113
|
+
## Part 3: Add another machine
|
|
114
|
+
|
|
115
|
+
Each machine is a complete Brainiac install that can run any of your agents. The hub decides which one gets each job.
|
|
116
|
+
|
|
117
|
+
**1. Install the basics** (see the main README's prerequisites): Ruby 3.4+, your agent CLIs (`claude`, `kiro-cli`, ...) **logged in**, `git`, `gh` (logged in), `qmd`, and any plugin CLIs you use (`fizzy`, ...).
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
gem install brainiac
|
|
121
|
+
brainiac setup
|
|
122
|
+
brainiac install discord # the same plugins as your first machine
|
|
123
|
+
brainiac install fizzy
|
|
124
|
+
brainiac install github
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
**2. Copy your configuration from the first machine.** These files in `~/.brainiac/` describe your agents and integrations and must match across machines:
|
|
128
|
+
|
|
129
|
+
| File | Why |
|
|
130
|
+
|---|---|
|
|
131
|
+
| `agents.json` | Agents, their tokens, and `"local": true`. An agent only runs on a machine where it is local. |
|
|
132
|
+
| `fizzy.json`, `github.json`, `discord.json`, other plugin files | Webhook secrets and plugin settings. Any machine may be the one that verifies a webhook. |
|
|
133
|
+
| `users.json`, `roles/`, `cli-providers/`, `profiles.json` | Identity, roles and CLI settings |
|
|
134
|
+
| `cron.json` | Only if this machine may become the leader |
|
|
135
|
+
|
|
136
|
+
Do **not** copy `brainiac.json` wholesale (it holds this machine's hub settings), `projects.json` (paths differ per machine), `work_items.json`, or `tmp/`. They are per machine.
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
# from the first machine
|
|
140
|
+
cd ~/.brainiac
|
|
141
|
+
scp -r agents.json users.json fizzy.json github.json discord.json roles cli-providers newbox:.brainiac/
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
These files contain tokens. Copy them over SSH, not through chat or email.
|
|
145
|
+
|
|
146
|
+
**3. Clone the brain.** The brain is a git repository (see "Git Sync" in the README). Every machine pulls it before a session and pushes after, which is how memory follows the work.
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
git clone <your brain repo> ~/.brainiac/brain
|
|
150
|
+
brainiac brain init
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
**4. Clone and register your projects.** Paths can differ between machines.
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
git clone git@github.com:yourorg/my-app.git ~/Code/my-app
|
|
157
|
+
cd ~/Code/my-app && brainiac register
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
A machine only takes work for projects it has registered. If a project is missing, events for it are ignored on that machine.
|
|
161
|
+
|
|
162
|
+
**5. Install the tunnel and connect to the hub.**
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
sudo pacman -S cloudflared # or brew / apt, see "Tunnels"
|
|
166
|
+
brainiac hub setup \
|
|
167
|
+
--url https://abc123.execute-api.us-east-1.amazonaws.com/prod \
|
|
168
|
+
--token <node token> \
|
|
169
|
+
--machine mini \
|
|
170
|
+
--priority 10 \
|
|
171
|
+
--tunnel cloudflared
|
|
172
|
+
brainiac server --daemon
|
|
173
|
+
brainiac machines
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
`brainiac machines` on any machine should now show both, one of them marked leader.
|
|
177
|
+
|
|
178
|
+
**6. Send it some work.** Add the tag `machine:mini` to a Fizzy card and assign it to an agent. `brainiac hub assignments fizzy:` shows where each card lives.
|
|
179
|
+
|
|
180
|
+
## Choosing which machine does the work
|
|
181
|
+
|
|
182
|
+
For **new work** the hub picks, in this order:
|
|
183
|
+
|
|
184
|
+
1. **The machine you asked for**, if it is online:
|
|
185
|
+
- Fizzy: a card tag `machine:<name>` (also `machine-<name>`). Add the tag before the card is first assigned or commented on. After that the card has an owner and the tag is not looked at again.
|
|
186
|
+
- Any webhook: `?machine=<name>` on the webhook URL. A board whose webhook is `https://<hub>/hooks/fizzy/<board>?machine=gpu` sends all its new cards to `gpu`.
|
|
187
|
+
2. **The online machine with the lowest `priority` number.** If the machine you asked for is offline, this is the fallback: priorities are your fallback order.
|
|
188
|
+
3. If no machine is online, the event waits in the hub for the first machine that checks in.
|
|
189
|
+
|
|
190
|
+
For **work that already has an owner** (a card someone started, its branch, its pull request) the hub always uses the owner. To move it:
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
brainiac hub move fizzy:1834 gpu # push the branch on the old machine first
|
|
194
|
+
brainiac hub release fizzy:1834 # or: forget the owner, let the next event choose
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
The new machine starts from the pushed branch and the brain's memory. The CLI's own conversation history stays on the old machine.
|
|
198
|
+
|
|
199
|
+
**GitHub events** carry a branch, not a card. The hub maps them two ways: a branch named `fizzy-<number>-...` follows that card, and each machine reports the branches it has worktrees for when it checks in.
|
|
200
|
+
|
|
201
|
+
## Discord: choosing the machine
|
|
202
|
+
|
|
203
|
+
Discord has no webhook for the hub to receive, so the leader does the routing. For each message an agent should answer, the leader picks:
|
|
204
|
+
|
|
205
|
+
1. **The machine named in the message**: `[machine:<name>]`, written like the other inline tags.
|
|
206
|
+
```
|
|
207
|
+
@Galen [machine:mini] fix the flaky login test
|
|
208
|
+
```
|
|
209
|
+
2. **The thread's machine.** Once a machine has made a worktree for a thread it owns that thread (`discord:<agent>:<thread id>` in `brainiac hub assignments discord:`). Follow-ups go back to it without a tag, whichever machine is the leader.
|
|
210
|
+
3. **The leader itself**, for everything else.
|
|
211
|
+
|
|
212
|
+
When the work belongs to another machine, the leader passes the message through the hub. That machine runs the agent in its own worktree and posts the reply as the same bot. Each reply ends with a small `ran on <machine>` line (`"machine_footer": false` in `discord.json` turns it off).
|
|
213
|
+
|
|
214
|
+
A tag inside an existing thread **moves the thread**. A machine new to the thread makes a worktree from the thread's branch on the remote, so push first. A machine that had the thread before reuses its old worktree as it was, so pull there if another machine pushed since. `[machine:<name of the leader>]` brings a thread back to the leader.
|
|
215
|
+
|
|
216
|
+
A thread keeps its machine after Discord archives it and its worktree is cleaned up: the next message goes back there and gets a fresh worktree. `brainiac hub release discord:<agent>:<thread id>` forgets the owner.
|
|
217
|
+
|
|
218
|
+
The leader answers in the channel when it cannot do what you asked:
|
|
219
|
+
|
|
220
|
+
| Situation | What happens |
|
|
221
|
+
|-----------|--------------|
|
|
222
|
+
| No machine has that name | Nothing runs. The leader lists the machines it knows |
|
|
223
|
+
| Machine offline, new conversation | The next machine by `priority` runs it, and the leader says which |
|
|
224
|
+
| Machine offline, existing thread | The thread stays where it is, and the leader says so |
|
|
225
|
+
| The thread's own machine is offline | The message waits in the hub for it. Send it again with `[machine:<name>]` to move the thread |
|
|
226
|
+
| The hub cannot be reached while handing work over | Nothing runs. Send it again |
|
|
227
|
+
|
|
228
|
+
Messages sent while the agent is working, and the ❌ ❔ ❓ 🧠 reactions, reach the machine doing the work.
|
|
229
|
+
|
|
230
|
+
Every machine you send Discord work to needs the same agents in `agents.json` (with their `DISCORD_BOT_TOKEN`), the same `discord.json` (it maps channels to projects), the project registered, and the agent CLI logged in. A machine that lacks the agent's token ignores the work, and nothing is posted.
|
|
231
|
+
|
|
232
|
+
## Discord and cron: the leader
|
|
233
|
+
|
|
234
|
+
One machine at a time is the leader. The leader:
|
|
235
|
+
|
|
236
|
+
- holds the Discord connection for every bot. It hears every Discord message and either runs it or hands it to another machine (see [Discord: choosing the machine](#discord-choosing-the-machine))
|
|
237
|
+
- runs `cron.json`
|
|
238
|
+
- runs the daily skill curation
|
|
239
|
+
|
|
240
|
+
Any machine can still post to Discord (a Fizzy job on `mini` posts its own notifications): posting does not need the open connection.
|
|
241
|
+
|
|
242
|
+
The leader is whichever machine asked first. It keeps the role until it shuts down or stops checking in. A clean shutdown hands over within one check-in (two minutes by default). After a crash or network loss it takes up to about nine minutes: the hub waits three missed check-ins before it gives the role to someone else. A leader that cannot reach the hub drops the role by itself after two missed check-ins, before another machine can pick it up, so two machines never answer at once.
|
|
243
|
+
|
|
244
|
+
To keep the role off a machine, set `--no-leader` on it. To pick the leader, set `--no-leader` on every machine except the one you want. Cron jobs must exist on every machine that may lead.
|
|
245
|
+
|
|
246
|
+
## Tunnels: how the hub reaches a machine
|
|
247
|
+
|
|
248
|
+
The hub pushes over HTTPS, so each machine needs a public URL that forwards to `localhost:4567`.
|
|
249
|
+
|
|
250
|
+
| | Cloudflare quick tunnel | Cloudflare named tunnel | ngrok |
|
|
251
|
+
|---|---|---|---|
|
|
252
|
+
| Cost | Free | Needs a domain you own with its DNS on Cloudflare. Check Cloudflare's current plans for Tunnel pricing | Free plan: one domain per account. Several machines need a paid plan or several accounts |
|
|
253
|
+
| Account | None | Cloudflare account, domain's DNS on Cloudflare | ngrok account |
|
|
254
|
+
| URL | New random URL on every start | Fixed hostname you choose | Fixed (one free dev domain) |
|
|
255
|
+
| Setup | `--tunnel cloudflared` | About ten minutes per machine | Run ngrok yourself, `--public-url` |
|
|
256
|
+
| Limits | "No uptime guarantee", 200 requests in flight, meant for testing and development | Cloudflare's production offering | See ngrok's plan limits |
|
|
257
|
+
|
|
258
|
+
**Recommended: the quick tunnel.** A changing URL is normally a problem with webhooks, and here it is not: Brainiac starts `cloudflared`, reads the new URL and reports it to the hub on every start, and restarts the tunnel if it dies. Your webhooks only ever know the hub's URL. The cost is Cloudflare's "no uptime guarantee" wording. If a quick tunnel drops, events for that machine wait in the hub until it is back (see [When a machine is offline](#when-a-machine-is-offline)).
|
|
259
|
+
|
|
260
|
+
Install `cloudflared`:
|
|
261
|
+
|
|
262
|
+
```bash
|
|
263
|
+
# Arch
|
|
264
|
+
sudo pacman -S cloudflared
|
|
265
|
+
|
|
266
|
+
# macOS
|
|
267
|
+
brew install cloudflared
|
|
268
|
+
|
|
269
|
+
# Debian / Ubuntu (including WSL2)
|
|
270
|
+
sudo mkdir -p --mode=0755 /usr/share/keyrings
|
|
271
|
+
curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null
|
|
272
|
+
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared any main' | sudo tee /etc/apt/sources.list.d/cloudflared.list
|
|
273
|
+
sudo apt-get update && sudo apt-get install cloudflared
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
Check it by hand once: `cloudflared tunnel --url http://localhost:4567` prints a `https://<words>.trycloudflare.com` URL. Stop it again; Brainiac runs its own.
|
|
277
|
+
|
|
278
|
+
**A fixed hostname with a named tunnel** (needs a domain on Cloudflare):
|
|
279
|
+
|
|
280
|
+
```bash
|
|
281
|
+
cloudflared tunnel login
|
|
282
|
+
cloudflared tunnel create brainiac-mini
|
|
283
|
+
cloudflared tunnel route dns brainiac-mini mini.example.com
|
|
284
|
+
cat > ~/.cloudflared/config.yml <<'EOF'
|
|
285
|
+
url: http://localhost:4567
|
|
286
|
+
tunnel: <Tunnel-UUID>
|
|
287
|
+
credentials-file: /home/<you>/.cloudflared/<Tunnel-UUID>.json
|
|
288
|
+
EOF
|
|
289
|
+
cloudflared tunnel run brainiac-mini # or install it as a service
|
|
290
|
+
brainiac hub setup --public-url https://mini.example.com
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
**ngrok or anything else:** run it yourself and give Brainiac the URL: `brainiac hub setup --public-url https://your-name.ngrok-free.app`.
|
|
294
|
+
|
|
295
|
+
A tunnel exposes the whole Brainiac server, as your ngrok tunnel does today. Set `dashboard_token` in `brainiac.json` so the dashboard and `/api/*` are not open to anyone who learns the URL.
|
|
296
|
+
|
|
297
|
+
## When a machine is offline
|
|
298
|
+
|
|
299
|
+
| Situation | What happens |
|
|
300
|
+
|---|---|
|
|
301
|
+
| New work, requested machine offline | Goes to the next online machine by priority |
|
|
302
|
+
| New work, nothing online | Waits in the hub (up to 24 hours) for the first machine to check in |
|
|
303
|
+
| Follow-up for a card owned by an offline machine | Waits in the hub for that machine. It does not move by itself. Use `brainiac hub move` to move it |
|
|
304
|
+
| A machine misses one push | The hub marks it unreachable and stops pushing until its next check-in (within two minutes), then hands over what queued |
|
|
305
|
+
| Leader goes offline | Another machine takes over Discord and cron |
|
|
306
|
+
| The hub itself is unreachable | Machines keep running what they have. No new webhooks arrive until it is back. The leader steps down after two missed check-ins, so Discord goes quiet too |
|
|
307
|
+
|
|
308
|
+
`brainiac machines` shows what is waiting and for whom.
|
|
309
|
+
|
|
310
|
+
## Commands
|
|
311
|
+
|
|
312
|
+
```bash
|
|
313
|
+
brainiac machines # who is online, who leads, what is queued
|
|
314
|
+
brainiac hub setup [options] # connect this machine (see Settings reference)
|
|
315
|
+
brainiac hub show # this machine's hub settings (token hidden)
|
|
316
|
+
brainiac hub assignments [prefix] # owners, e.g. `fizzy:` or `branch:yourorg/my-app:`
|
|
317
|
+
brainiac hub move <key> <machine> # move a card or branch
|
|
318
|
+
brainiac hub release <key> # forget the owner
|
|
319
|
+
brainiac hub disable # back to a standalone machine
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
Keys look like `fizzy:1834`, `branch:yourorg/my-app:fizzy-1834-fix-login`, `pr:yourorg/my-app:212` and `discord:galen:1572301950530363575` (agent, then thread id).
|
|
323
|
+
|
|
324
|
+
## Settings reference
|
|
325
|
+
|
|
326
|
+
`brainiac hub setup` writes this block to `~/.brainiac/brainiac.json`:
|
|
327
|
+
|
|
328
|
+
```json
|
|
329
|
+
{
|
|
330
|
+
"hub": {
|
|
331
|
+
"url": "https://abc123.execute-api.us-east-1.amazonaws.com/prod",
|
|
332
|
+
"token": "<node token>",
|
|
333
|
+
"machine": "mini",
|
|
334
|
+
"priority": 10,
|
|
335
|
+
"leader": true,
|
|
336
|
+
"interval": 120,
|
|
337
|
+
"tunnel": "cloudflared"
|
|
338
|
+
}
|
|
339
|
+
}
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
| Key | Default | Meaning |
|
|
343
|
+
|---|---|---|
|
|
344
|
+
| `url` | none | Hub base URL. With `token`, this switches multi-machine on |
|
|
345
|
+
| `token` | none | Node token. The `BRAINIAC_HUB_TOKEN` environment variable wins over the file |
|
|
346
|
+
| `machine` | hostname | This machine's name: lowercase letters, digits, `.`, `-`, `_` |
|
|
347
|
+
| `priority` | `100` | Lower number is preferred for new work |
|
|
348
|
+
| `leader` | `true` | Whether this machine may hold Discord and cron |
|
|
349
|
+
| `interval` | `120` | Seconds between check-ins. Lower means faster failover and more requests |
|
|
350
|
+
| `tunnel` | none | `"cloudflared"`, or `{ "command": [...], "url_pattern": "regex" }` for another tool. `%<port>d` in the command becomes the server port |
|
|
351
|
+
| `public_url` | none | A fixed public URL for this machine. Used instead of `tunnel` |
|
|
352
|
+
|
|
353
|
+
Restart the server after changing any of these.
|
|
354
|
+
|
|
355
|
+
## Limits
|
|
356
|
+
|
|
357
|
+
- **Untagged Discord work starts on the leader.** A new conversation runs on the leader unless the message says `[machine:<name>]`. Direct messages have no thread to own, so each one needs the tag. Fizzy tags and `priority` do not apply to Discord.
|
|
358
|
+
- **Thread owner lookup needs the hub.** If the hub does not answer when the leader asks who owns a thread, the leader runs an untagged message itself.
|
|
359
|
+
- **Discord reactions on older messages.** The leader keeps the list of messages it handed to other machines in memory. After the leader restarts or changes, ❌ ❔ ❓ 🧠 on those earlier messages are not forwarded.
|
|
360
|
+
- **In-flight work does not move by itself.** See [When a machine is offline](#when-a-machine-is-offline).
|
|
361
|
+
- **A Discord thread without a worktree does not follow a leader change.** Threads with a worktree are owned and keep going to their machine. A thread in a channel with no project has nothing on disk, so its next message runs on whichever machine leads, from the brain's memory of the thread.
|
|
362
|
+
- **The sender sees the hub's answer, not the plugin's.** The hub always answers `202`. A sender that reads the response body (a webhook "test" button, a setup handshake) gets the hub's. Zoho's first-time handshake has not been tried through the hub: do that setup against a machine directly.
|
|
363
|
+
- **A queued event can go stale.** Senders that sign a timestamp (the Posh Nosh plugin allows five minutes) reject an event that waited longer than that.
|
|
364
|
+
- **OAuth setup pages** (`/zoho/auth`, `/zoho/callback`) are not served through the hub. Run those against a machine directly.
|
|
365
|
+
- **The monitor and dashboard show one machine.** Each shows the sessions on its own machine.
|
|
366
|
+
- **Agents that only one machine has.** If an agent is `local` on one machine only, send its work there with a tag or `?machine=`. Otherwise another machine may receive the event and ignore it.
|
|
367
|
+
|
|
368
|
+
## Troubleshooting
|
|
369
|
+
|
|
370
|
+
**`brainiac machines` says it could not reach the hub.** Check `brainiac hub show` for the URL, and that the token matches the one in SSM.
|
|
371
|
+
|
|
372
|
+
**A machine shows OFFLINE while its server is running.** Look in `brainiac logs` for `[Hub]` lines. `No public URL yet` means the tunnel has not produced a URL: check that `cloudflared` is installed and runs by hand. `POST /nodes -> 401` means the token is wrong.
|
|
373
|
+
|
|
374
|
+
**Webhooks arrive at the hub but nothing happens.** `brainiac machines` shows whether they are queued. If a machine is online and still gets nothing, its tunnel URL may be dead: restart the server to get a new one.
|
|
375
|
+
|
|
376
|
+
**The machine logs `Job ... -> 401`.** The hub delivered the event and the plugin rejected the signature. That machine's webhook secret (`fizzy.json`, `github.json`) does not match the first machine's.
|
|
377
|
+
|
|
378
|
+
**Two bots answer every Discord message.** One machine runs a Discord plugin without leader support. Update it there.
|
|
379
|
+
|
|
380
|
+
**Discord is quiet.** `brainiac machines` shows the leader. If it says `nobody`, every machine has `leader: false` or none is online.
|
data/lib/brainiac/cron.rb
CHANGED
|
@@ -617,6 +617,10 @@ def cron_loop
|
|
|
617
617
|
|
|
618
618
|
now = Time.now
|
|
619
619
|
|
|
620
|
+
# Multi-machine: a job copied to several machines must fire once, so only
|
|
621
|
+
# the hub leader runs cron. Without a hub this is always true.
|
|
622
|
+
next if defined?(Brainiac::Hub) && !Brainiac::Hub.leader?
|
|
623
|
+
|
|
620
624
|
CRON_JOBS_MUTEX.synchronize do
|
|
621
625
|
CRON_JOBS.each_value do |job|
|
|
622
626
|
next unless job[:enabled]
|
|
@@ -4,7 +4,8 @@
|
|
|
4
4
|
#
|
|
5
5
|
# Messages from any channel can contain inline tags like:
|
|
6
6
|
# [project:my-project], [opus], [effort:high], [cli:grok], [chat], [plan],
|
|
7
|
-
# [fresh], [branch:feature-xyz], [workitem:wi-abc123], [profile:k+] ([p:k+])
|
|
7
|
+
# [fresh], [branch:feature-xyz], [workitem:wi-abc123], [profile:k+] ([p:k+]),
|
|
8
|
+
# [machine:mini]
|
|
8
9
|
#
|
|
9
10
|
# This module provides a single parser that extracts all tags and returns
|
|
10
11
|
# a structured result with the cleaned text.
|
|
@@ -17,6 +18,7 @@
|
|
|
17
18
|
# effort: "high" or nil,
|
|
18
19
|
# cli_provider: "grok" or nil,
|
|
19
20
|
# profile: "k+" or nil,
|
|
21
|
+
# machine: "mini" or nil (multi-machine: the machine asked to do the work),
|
|
20
22
|
# chat_mode: true/false,
|
|
21
23
|
# planning: true/false,
|
|
22
24
|
# fresh: true/false,
|
|
@@ -32,6 +34,7 @@ def parse_inline_tags(text)
|
|
|
32
34
|
effort: nil,
|
|
33
35
|
cli_provider: nil,
|
|
34
36
|
profile: nil,
|
|
37
|
+
machine: nil,
|
|
35
38
|
chat_mode: false,
|
|
36
39
|
planning: false,
|
|
37
40
|
fresh: false,
|
|
@@ -78,6 +81,12 @@ def parse_value_tags(result)
|
|
|
78
81
|
result[:clean_text].sub!(match[0], "")
|
|
79
82
|
end
|
|
80
83
|
|
|
84
|
+
# [machine:mini] — multi-machine: run this on a named machine (see docs/multi-machine.md)
|
|
85
|
+
if (match = result[:clean_text].match(/\[machine:([\w.-]+)\]/i))
|
|
86
|
+
result[:machine] = match[1].downcase
|
|
87
|
+
result[:clean_text].sub!(match[0], "")
|
|
88
|
+
end
|
|
89
|
+
|
|
81
90
|
# [deploy] or [deploy:dev01]
|
|
82
91
|
if (match = result[:clean_text].match(/\[deploy(?::([^\]]+))?\]/i))
|
|
83
92
|
result[:deploy_intent] = match[1]&.strip&.downcase || :auto
|