relay-flow 0.2.4-alpha → 0.2.6-alpha

package/README.md

@@ -177,7 +177,13 @@ bd init \
 # Example external workspace: /var/lib/beads/payments/.beads
 ```
 
-Register the code repository and its Beads workspace separately. `beadsDir` is required for every Beads repo, must name an existing directory, and is the task scope used to reject duplicate workspace registration:
+For the relay-flow installation used by this repository, the configured Beads workspace is:
+
+```text
+/home/raj/.beads/relay-flow/.beads
+```
+
+Register the code repository and its Beads workspace separately. `beadsDir` is required for every Beads repo and must name an existing directory. It selects the Beads workspace/database; it is not the code repository path or a workflow filter:
 
 ```sh
 # Local/embedded workspace
@@ -193,7 +199,7 @@ relay-flow repo register \
   --set beadsDir=/var/lib/beads/payments/.beads
 ```
 
-The registered `--path` remains the code/runner repository. Every `bd` command runs with that path as its working directory and the configured `beadsDir` as `BEADS_DIR`, even when unrelated Beads selector variables exist in the relay-flow environment. Two repos may register different canonical `beadsDir` values; a second repo pointing at the same workspace is rejected. A Beads prefix such as `payments-...` is optional and only makes issue IDs recognizable—it is not a component, workspace selector, poller selector, or database isolation mechanism.
+The registered `--path` remains the code/runner repository. Every `bd` command runs with that path as its working directory and the configured `beadsDir` as `BEADS_DIR`, even when unrelated Beads selector variables exist in the relay-flow environment. Multiple code repositories may share one Beads/Dolt workspace when their derived repository labels differ. Relay-flow derives one reserved label, `repo:<lowercase-registered-name>`, for each repository; names must use letters, numbers, `.`, `_`, or `-`, and be at most 250 characters so the derived label stays within Beads' 255-character limit. The label is not entered in workflow configuration. A parent must carry exactly one such repository label to be eligible for that repo's poller: missing or ambiguous `repo:` labels are ignored before workflow filters and routing. Workflow ownership labels remain independent `wf:<workflow>` labels. A collision between derived labels in the same workspace is rejected before the second repository is registered. A Beads prefix such as `payments-...` is optional and only makes issue IDs recognizable—it is not a component, workspace selector, poller selector, or database isolation mechanism.
 
 Beads workflow filters are structured and evaluated in relay-flow. For example, `examples/beads-workflow.yaml` uses the Beads status and issue-type fields:
 
@@ -211,7 +217,7 @@ Filter values are exact and are not translated between providers.
 
 The supported Beads status names are `open`, `in_progress`, `blocked`, `deferred`, `hooked`, and `closed`. The claimed-parent poll uses the canonical active set `open,in_progress,blocked,deferred`; it intentionally does not substitute `hooked` for `deferred`. Omitted lifecycle settings move the parent to `in_progress` at `start`, a work-node mailbox to `in_progress`, and the parent to `closed` at `end` — the same shape as Jira, with Beads-native values. Relay-flow creates one Repo Poller per registered repo, not one poller per workflow. Each poll reads ready top-level parents and relay-owned active parents, deduplicates them, and never routes mailbox children. Claims are permanent `wf:<workflow>` labels.
 
-Beads does not need relay-flow credentials or a Beads-specific poller. In server mode, leave Dolt and Beads server setup running outside relay-flow and point each repo at its own `beadsDir`.
+Beads does not need relay-flow credentials or a Beads-specific poller. In server mode, leave Dolt and Beads server setup running outside relay-flow; each registered code repository still supplies its `beadsDir`, and repositories that share that workspace are isolated by their derived `repo:` labels.
 
 ### Submit a workflow
 
@@ -354,37 +360,42 @@ lower-scope input.
 
 Pi has one built-in coding agent. Relay-flow keeps Pi's configured model,
 provider, tools, extensions, and settings, while allowing a workflow node to
-select a repository-owned role prompt. `agent: default` uses Pi's built-in
-coding agent without an additional role prompt. A non-default value such as
-`coder` or `reviewer` is resolved to a readable, non-empty
-`.pi/roles/<agent>.md` file in the registered repository and passed to Pi with
-`--append-system-prompt`.
+select a project-owned prompt template. `agent: default` uses Pi's built-in
+coding agent without an additional prompt template. A non-default value such as
+`coder` or `reviewer` selects Pi's native project template
+`.pi/prompts/<agent>.md` and submits the rendered task text as the template's
+arguments with `/<agent> ...`.
 
-Pi has no native `pi agent list` command. Role existence is therefore verified
-by the filesystem check above; missing, empty, non-regular, or unsafe role
-paths fail workflow preflight. The workflow agent value is never treated as a
-model ID and relay-flow never passes an OpenCode-style `--agent` option.
+Pi owns prompt-template discovery and expansion. Relay-flow does not maintain a
+named-agent registry or inspect template files; it only validates the safe
+template command name and passes Pi the corresponding
+`--prompt-template .pi/prompts/<agent>.md` resource. The workflow agent value is
+never treated as a model ID and relay-flow never passes an OpenCode-style
+`--agent` option.
 
-For example, a Pi workflow with repository roles uses:
+For example, a Pi workflow with project prompt templates uses:
 
 ```yaml
 type: agent
 agent: coder
 ```
 
-with:
+with project templates:
 
 ```text
-.pi/roles/coder.md
-.pi/roles/reviewer.md
+.pi/prompts/coder.md
+.pi/prompts/reviewer.md
 ```
 
-Pi node launches use the installed Pi 0.84.1 interactive command contract:
+Pi node launches use the installed Pi interactive command contract:
 
 ```text
-pi --name <ticket>:<node> [--append-system-prompt <role-file>] [--session-id <persisted-session-id>] <prompt>
+pi --name <ticket>:<node> [--prompt-template .pi/prompts/<agent>.md] [--session-id <persisted-session-id>] <prompt>
 ```
 
+For a named agent, `<prompt>` is `/agent <rendered relay-flow task or
+feedback prompt>`; `default` keeps the rendered prompt unchanged.
+
 The prompt is one positional argument. Pi 0.84.1 rejects a bare `--`, so the
 launch command does not include one. A persisted session uses
 `--session-id`; print mode, JSON/RPC mode, and extension-install flags are not

package/cmd/relay-flow/backend_selection_test.go

@@ -0,0 +1,149 @@
+package main
+
+import (
+	"context"
+	"database/sql"
+	"fmt"
+	"os"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/rajpopat27/relay-flow/internal/config"
+	"github.com/rajpopat27/relay-flow/internal/execution/projection"
+	"github.com/rajpopat27/relay-flow/internal/paths"
+	"github.com/rajpopat27/relay-flow/internal/server"
+	_ "modernc.org/sqlite"
+)
+
+// These integration tests are deliberately live-gated because they exercise
+// the selected durable backend and the local Temporal service. They use the
+// same fake task/runner/harness factories as the composition scenario, but no
+// task-system state or workflow is needed to prove worker selection.
+func TestServeSelectsConfiguredTemporalExecutor(t *testing.T) {
+	if os.Getenv("RELAY_FLOW_TEMPORAL_LIVE") != "1" {
+		t.Skip("set RELAY_FLOW_TEMPORAL_LIVE=1 to run backend selection against Temporal")
+	}
+	assertServeUsesTemporal(t, false)
+}
+
+func TestServeRecoverSelectsConfiguredTemporalExecutor(t *testing.T) {
+	if os.Getenv("RELAY_FLOW_TEMPORAL_LIVE") != "1" {
+		t.Skip("set RELAY_FLOW_TEMPORAL_LIVE=1 to run backend recovery against Temporal")
+	}
+	assertServeUsesTemporal(t, true)
+}
+
+func assertServeUsesTemporal(t *testing.T, recover bool) {
+	t.Helper()
+	log := newScenarioLog()
+	setScenarioFactoryAdapters(newScenarioTaskSystem(log), newScenarioRunner(log), newScenarioHarness(log))
+
+	root := filepath.Join(t.TempDir(), ".relay-flow")
+	t.Setenv("RELAY_FLOW_HOME", root)
+	p := pathsForRoot(root)
+	if err := paths.Ensure(p); err != nil {
+		t.Fatal(err)
+	}
+	namespace := fmt.Sprintf("relay-flow-serve-%d", time.Now().UnixNano())
+	if err := ensureTemporalNamespace(context.Background(), "localhost:7233", namespace, 30); err != nil {
+		t.Fatal(err)
+	}
+	// Namespace registration is acknowledged before every frontend worker
+	// necessarily observes it.
+	time.Sleep(5 * time.Second)
+	if err := config.SaveMachine(p.Config, &config.Machine{
+		TaskPlugin:                scenarioTaskPlugin,
+		RunnerPlugin:              scenarioRunnerPlugin,
+		HarnessPlugin:             scenarioHarnessPlugin,
+		ExecutorPlugin:            "temporal",
+		TemporalAddress:           "localhost:7233",
+		TemporalNamespace:         namespace,
+		CompletedRunRetentionDays: 30,
+	}); err != nil {
+		t.Fatal(err)
+	}
+	if err := projection.InitDatabaseWithIdentity(p.Database, projection.ExecutorIdentity{
+		ExecutorPlugin: "temporal", TemporalAddress: "localhost:7233", TemporalNamespace: namespace,
+	}); err != nil {
+		t.Fatal(err)
+	}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	done := make(chan error, 1)
+	go func() { done <- serveRoot(ctx, p, recover) }()
+	client := server.NewClient(p.Socket)
+	waitForServerOrError(t, client, done)
+
+	assertNoEmbeddedExecutionTables(t, p.Database)
+	if err := client.Stop(context.Background()); err != nil {
+		t.Fatal(err)
+	}
+	select {
+	case err := <-done:
+		if err != nil {
+			t.Fatalf("serveRoot(%v): %v", recover, err)
+		}
+	case <-time.After(15 * time.Second):
+		t.Fatalf("serveRoot(%v) did not stop", recover)
+	}
+}
+
+func waitForServerOrError(t *testing.T, client *server.Client, done <-chan error) {
+	t.Helper()
+	deadline := time.Now().Add(20 * time.Second)
+	for time.Now().Before(deadline) {
+		select {
+		case err := <-done:
+			t.Fatalf("serveRoot exited before socket became ready: %v", err)
+		default:
+		}
+		ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
+		_, err := client.ListRepos(ctx)
+		cancel()
+		if err == nil {
+			return
+		}
+		time.Sleep(20 * time.Millisecond)
+	}
+	select {
+	case err := <-done:
+		t.Fatalf("serveRoot exited before socket became ready: %v", err)
+	default:
+		t.Fatal("server did not become ready")
+	}
+}
+
+func assertNoEmbeddedExecutionTables(t *testing.T, databasePath string) {
+	t.Helper()
+	db, err := sql.Open("sqlite", databasePath)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	var count int
+	if err := db.QueryRow(`SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name IN ('instances', 'history_events', 'activity_tasks')`).Scan(&count); err != nil {
+		t.Fatal(err)
+	}
+	if count != 0 {
+		t.Fatalf("Temporal projection contains embedded go-workflows tables: %d", count)
+	}
+	var plugin, address, namespace string
+	if err := db.QueryRow(`SELECT executor_plugin, temporal_address, temporal_namespace FROM relay_executor_identity WHERE singleton = 1`).Scan(&plugin, &address, &namespace); err != nil {
+		t.Fatal(err)
+	}
+	if plugin != "temporal" || address != "localhost:7233" || namespace == "" {
+		t.Fatalf("executor identity = %q/%q/%q", plugin, address, namespace)
+	}
+}
+
+func pathsForRoot(root string) paths.Paths {
+	return paths.Paths{
+		Root: root, Config: filepath.Join(root, "config.yaml"),
+		Credentials: filepath.Join(root, "credentials.yaml"), Workflows: filepath.Join(root, "workflows"),
+		Database: filepath.Join(root, "state.db"), Socket: filepath.Join(root, "server.sock"),
+		Lock: filepath.Join(root, "server.lock"), ServerLog: filepath.Join(root, "server.log"),
+		PluginLog: filepath.Join(root, "plugin.log"),
+	}
+}

package/cmd/relay-flow/beads_composition_test.go

@@ -361,9 +361,9 @@ issue_json() {
   case "$issue" in
     demo-parent)
       if [ -f "$BD_STATE/claimed" ]; then
-        labels='["wf:beadsComposition"]'
+        labels='["repo:payments","wf:beadsComposition"]'
       else
-        labels='[]'
+        labels='["repo:payments"]'
       fi
       printf '[{"id":"demo-parent","title":"Composition parent","status":"%s","issue_type":"epic","priority":1,"labels":%s}]\n' "$status" "$labels"
       ;;
@@ -385,7 +385,7 @@ if [ "$#" -eq 6 ] && [ "$1" = list ] && [ "$2" = --ready ] && [ "$3" = --no-pare
   if [ -f "$BD_STATE/claimed" ]; then
     printf '[]\n'
   else
-    printf '[{"id":"demo-parent","title":"Composition parent","status":"open","issue_type":"epic","priority":1,"labels":[]}]\n'
+    printf '[{"id":"demo-parent","title":"Composition parent","status":"open","issue_type":"epic","priority":1,"labels":["repo:payments"]}]\n'
   fi
   exit 0
 fi

package/cmd/relay-flow/main.go

@@ -24,7 +24,7 @@ import (
 	"github.com/charmbracelet/huh"
 	"github.com/mattn/go-isatty"
 	"github.com/rajpopat27/relay-flow/internal/config"
-	"github.com/rajpopat27/relay-flow/internal/execution/goworkflows"
+	"github.com/rajpopat27/relay-flow/internal/execution/projection"
 	"github.com/rajpopat27/relay-flow/internal/harness"
 	"github.com/rajpopat27/relay-flow/internal/logging"
 	"github.com/rajpopat27/relay-flow/internal/paths"
@@ -134,6 +134,8 @@ func usage(w io.Writer) {
 
 Usage:
   relay-flow init [--force] [--task-plugin <name> --runner-plugin <name> --harness-plugin <name>]
+                   [--executor-plugin <goworkflows|temporal>]
+                   [--temporal-address <host:port>] [--temporal-namespace <name>]
   relay-flow task auth [task-plugin options]
   relay-flow serve [--recover] [--debug] [--background]
   relay-flow stop
@@ -170,6 +172,9 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 	taskName := fs.String("task-plugin", "", "task plugin name (non-interactive)")
 	runnerName := fs.String("runner-plugin", "", "runner plugin name (non-interactive)")
 	harnessName := fs.String("harness-plugin", "", "harness plugin name (non-interactive)")
+	executorName := fs.String("executor-plugin", "", "durable executor name (goworkflows or temporal)")
+	temporalAddress := fs.String("temporal-address", "", "Temporal server address")
+	temporalNamespace := fs.String("temporal-namespace", "", "Temporal namespace/team name")
 	force := fs.Bool("force", false, "update plugin selections while preserving existing state")
 	if err := fs.Parse(args); err != nil {
 		return exitUsage
@@ -215,7 +220,7 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 		}
 		defer unlock()
 		if databaseExists {
-			active, err := goworkflows.HasNonterminalRuns(p.Database)
+			active, err := projection.HasNonterminalRuns(p.Database)
 			if err != nil {
 				fmt.Fprintln(os.Stderr, "init: "+err.Error())
 				return exitFail
@@ -227,9 +232,11 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 		}
 	}
 
-	// Selection precedence: flags → TTY form → stdin lines. The stdin path
-	// is the documented test seam and script path.
+	// Selection precedence: flags → TTY form → legacy stdin lines. The
+	// three-line stdin path remains embedded-mode input and never consumes
+	// Temporal answers.
 	var names []string
+	executor := strings.TrimSpace(*executorName)
 	switch {
 	case flagged:
 		names = []string{*taskName, *runnerName, *harnessName}
@@ -240,6 +247,10 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 			fmt.Fprintln(os.Stderr, "init: "+err.Error())
 			return exitFail
 		}
+		if len(names) == 4 {
+			executor = names[3]
+			names = names[:3]
+		}
 	default:
 		names = readInitLines(stdin)
 		if names == nil {
@@ -247,6 +258,22 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 			return exitFail
 		}
 	}
+	if executor == executorTemporal && isTTY(stdin) {
+		address, namespace, err := promptTemporalSettings(*temporalAddress, *temporalNamespace)
+		if err != nil {
+			fmt.Fprintln(os.Stderr, "init: "+err.Error())
+			return exitFail
+		}
+		*temporalAddress = address
+		*temporalNamespace = namespace
+	}
+	if executor == "" {
+		executor = executorGoworkflows
+	}
+	if executor != executorGoworkflows && executor != executorTemporal {
+		fmt.Fprintln(os.Stderr, "init: unknown executor plugin "+executor)
+		return exitUsage
+	}
 	// Validate against the registered factories; unknown names list the
 	// registered set per design (no silent acceptance).
 	if err := task.ValidateName(names[0]); err != nil {
@@ -262,13 +289,42 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 		return exitFail
 	}
 
-	cfg := &config.Machine{KeepTerminalsAlive: true, KeepSessionsAlive: true}
+	cfg := &config.Machine{
+		PollIntervalSeconds: 15, CompletedRunRetentionDays: 30,
+		KeepTerminalsAlive: true, KeepSessionsAlive: true,
+		ExecutorPlugin: executor,
+	}
 	if *force && configExists {
 		cfg, err = config.LoadMachine(p.Config)
 		if err != nil {
 			fmt.Fprintln(os.Stderr, err)
 			return exitFail
 		}
+		if strings.TrimSpace(*executorName) == "" {
+			executor = cfg.ExecutorPlugin
+		}
+		cfg.ExecutorPlugin = executor
+	}
+	if executor == executorTemporal {
+		if strings.TrimSpace(*temporalAddress) != "" {
+			cfg.TemporalAddress = strings.TrimSpace(*temporalAddress)
+		} else if cfg.TemporalAddress == "" {
+			cfg.TemporalAddress = defaultTemporalAddr
+		}
+		if strings.TrimSpace(*temporalNamespace) != "" {
+			cfg.TemporalNamespace = strings.TrimSpace(*temporalNamespace)
+		}
+		if cfg.TemporalNamespace == "" {
+			fmt.Fprintln(os.Stderr, "init: --temporal-namespace is required for executor-plugin temporal")
+			return exitUsage
+		}
+	} else {
+		if strings.TrimSpace(*temporalAddress) != "" || strings.TrimSpace(*temporalNamespace) != "" {
+			fmt.Fprintln(os.Stderr, "init: Temporal address/namespace require executor-plugin temporal")
+			return exitUsage
+		}
+		cfg.TemporalAddress = ""
+		cfg.TemporalNamespace = ""
 	}
 	cfg.TaskPlugin = names[0]
 	cfg.RunnerPlugin = names[1]
@@ -297,17 +353,37 @@ func cmdInit(p paths.Paths, args []string, stdin io.Reader) int {
 	} else {
 		cfg.HarnessConfig = config.Merge(harnessDefaults, cfg.HarnessConfig)
 	}
-	if err := config.SaveMachine(p.Config, cfg); err != nil {
-		fmt.Fprintln(os.Stderr, err)
-		return exitFail
+	identity := projection.ExecutorIdentity{
+		ExecutorPlugin:    executor,
+		TemporalAddress:   cfg.TemporalAddress,
+		TemporalNamespace: cfg.TemporalNamespace,
+	}
+	// Force mode verifies the immutable durable identity before contacting
+	// Temporal or replacing configuration. Legacy marker-less homes may only
+	// be adopted by goworkflows.
+	if databaseExists {
+		if err := verifyExecutorIdentity(p.Database, identity); err != nil {
+			fmt.Fprintln(os.Stderr, "init: "+err.Error())
+			return exitFail
+		}
+	}
+	if executor == executorTemporal {
+		if err := ensureTemporalNamespace(context.Background(), cfg.TemporalAddress, cfg.TemporalNamespace, cfg.CompletedRunRetentionDays); err != nil {
+			fmt.Fprintln(os.Stderr, "init: "+err.Error())
+			return exitFail
+		}
 	}
 	if !databaseExists {
-		if err := goworkflows.InitDatabase(p.Database); err != nil {
-			fmt.Fprintln(os.Stderr, err)
+		if err := projection.InitDatabaseWithIdentity(p.Database, identity); err != nil {
+			fmt.Fprintln(os.Stderr, "init: "+err.Error())
 			return exitFail
 		}
 	}
-	fmt.Printf("Task system: %s\nRunner: %s\nHarness: %s\nRelay-flow initialized\n", names[0], names[1], names[2])
+	if err := config.SaveMachine(p.Config, cfg); err != nil {
+		fmt.Fprintln(os.Stderr, err)
+		return exitFail
+	}
+	fmt.Printf("Task system: %s\nRunner: %s\nHarness: %s\nExecutor: %s\nRelay-flow initialized\n", names[0], names[1], names[2], executor)
 	return exitOK
 }
 
@@ -345,8 +421,8 @@ func isTTY(stdin io.Reader) bool {
 
 // pickPluginsInteractive runs one searchable select per plugin type.
 func pickPluginsInteractive() ([]string, error) {
-	names := make([]string, 3)
-	groups := make([]*huh.Group, 0, 3)
+	names := make([]string, 4)
+	groups := make([]*huh.Group, 0, 4)
 	for _, selection := range []struct {
 		title   string
 		options []string
@@ -364,6 +440,11 @@ func pickPluginsInteractive() ([]string, error) {
 			groups = append(groups, huh.NewGroup(field))
 		}
 	}
+	executorField, err := executorSelectField(&names[3])
+	if err != nil {
+		return nil, err
+	}
+	groups = append(groups, huh.NewGroup(executorField))
 	if len(groups) == 0 {
 		return names, nil
 	}

package/cmd/relay-flow/scenario_test.go

@@ -801,6 +801,8 @@ type scenarioTaskSystem struct {
 	creates          map[string]int
 	completions      map[string]int
 	completeFailures int
+	recoveryTickets  []task.Ticket
+	recoveryPolls    int
 }
 
 var _ task.System = (*scenarioTaskSystem)(nil)
@@ -816,11 +818,19 @@ func newScenarioTaskSystem(log *scenarioLog) *scenarioTaskSystem {
 func (s *scenarioTaskSystem) Poll(context.Context) ([]task.Ticket, error) {
 	s.log.add("poll")
 	s.mu.Lock()
-	defer s.mu.Unlock()
 	if s.claimed {
+		if s.recoveryPolls > 0 {
+			s.recoveryPolls--
+			tickets := append([]task.Ticket(nil), s.recoveryTickets...)
+			s.mu.Unlock()
+			return tickets, nil
+		}
+		s.mu.Unlock()
 		return nil, nil
 	}
-	return []task.Ticket{{ID: "ticket-1", Key: scenarioTicket, Title: "Scenario ticket"}}, nil
+	tickets := []task.Ticket{{ID: "ticket-1", Key: scenarioTicket, Title: "Scenario ticket"}}
+	s.mu.Unlock()
+	return tickets, nil
 }
 
 func (s *scenarioTaskSystem) CompileFilter(config.RawValues) (func(task.Ticket) bool, error) {
@@ -958,6 +968,13 @@ func (s *scenarioTaskSystem) setCompleteFailures(n int) {
 	s.mu.Unlock()
 }
 
+func (s *scenarioTaskSystem) setRecoveryTickets(tickets []task.Ticket, polls int) {
+	s.mu.Lock()
+	s.recoveryTickets = append([]task.Ticket(nil), tickets...)
+	s.recoveryPolls = polls
+	s.mu.Unlock()
+}
+
 func (s *scenarioTaskSystem) mailboxCreateCount(node string) int {
 	s.mu.Lock()
 	defer s.mu.Unlock()