flexiq-executor 2.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 0b8ca3b245c729d6976dd32d3ade3a3dee78e3fb7b0fc3420bae5d5dd019bd4c
+  data.tar.gz: ec20c0917a4be4f285099103c0438816ce00541c111d2890d264af00e629ffcb
+SHA512:
+  metadata.gz: 90aaacc72f5576f6ca1616881cd14e7c2b326504a1fdc7c044c2f848bf3b93f99ce428bb695e438f19d3385beb86135aa6038c11fce5da40e78df71a8b9eaa2c
+  data.tar.gz: bf43349dbc5d6b306be85e483e8ff001ccf55458e8a2f6d93013c58bfc780a20f910dcd534774383ea722c8c96578e1039ea8db11787b35bbdeb2f6081ad0a91

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pratyush Sharma
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,124 @@
+# FlexiQ Ruby executor
+
+A Ruby client for the **executor door** of a running
+[`flexiq-server`](../../crates/flexiq-server): attach over gRPC, run the jobs the scheduler
+dispatches, and settle each one with its result or its error.
+
+It is the other half of the [`flexiq`](../ruby) producer gem. The two are separate gems because
+they are separate doors behind separate scopes: an `execute` token cannot enqueue, a `produce`
+token cannot attach, and this door has no HTTP binding, so it needs the `grpc` gem's native
+extension where the producer gem needs nothing at all.
+
+## Install
+
+Ruby 3.3 or newer. Neither gem is on RubyGems yet; take both from this repository with Bundler:
+
+```ruby
+# Gemfile
+git "https://github.com/ByteVeda/flexiq", glob: "sdks/{ruby,ruby-executor}/*.gemspec" do
+  gem "flexiq"
+  gem "flexiq-executor"
+end
+```
+
+`grpc` and `google-protobuf` ship precompiled for Linux and macOS, so nothing is compiled on
+install.
+
+## Quickstart
+
+```ruby
+require "flexiq/executor"
+
+worker = FlexiQ::Executor::Worker.new("flexiq.internal:50051",
+                                      token: ENV.fetch("FLEXIQ_EXECUTE_TOKEN"),
+                                      id: "billing-1",
+                                      slots: 8)
+
+worker.handle("billing.charge") do |job|
+  order = job.args.first            # positional arguments, decoded from the payload
+  job.progress(50)
+  job.log("info", "charging", extra: { "order" => order["id"] })
+  { "receipt" => Gateway.charge(order) }  # the job's result
+end
+
+trap("TERM") { worker.stop }
+worker.run
+```
+
+`run` blocks. It attaches, dispatches each job to its handler on a thread of its own, and
+reconnects when the scheduler rotates the stream, which it does every half hour by default
+(`FLEXIQ_GRPC_EXECUTOR_STREAM_MAX_AGE`): a gRPC stream cannot be load-balanced once it has
+started. A rotation is not a failure and costs no job in flight. `run` returns when the
+scheduler sends `shutdown` or `stop` is called; it raises `FlexiQ::Executor::AttachError` for a
+refusal that reconnecting cannot fix, such as a bad token or a protocol version mismatch.
+
+TLS is on by default and uses the system roots. Pass `credentials:` (a
+`GRPC::Core::ChannelCredentials`) for a private CA, or `insecure: true` for loopback only.
+
+## What a handler returns
+
+- **A value** is a success. It is encoded with the producer gem's CBOR codec, so a producer reads
+  it back with `FlexiQ::Payload.decode_result`. `nil` means the task returned nothing.
+- **An exception** is a failure, written to `Job#error` as
+  `{"errtype","message","traceback"}` JSON. It is retried while retries remain.
+- **`FlexiQ::Executor::Fatal`** fails the job without a retry. Whether to retry is the
+  executor's decision: only it can see the exception. Raised inside a `rescue`, the error names
+  the original exception.
+
+## Cancellation and timeouts
+
+Ruby cannot stop a thread safely, so both are cooperative. `job.check!` raises
+`FlexiQ::Executor::Cancelled` once the scheduler asked for the job to stop, and
+`FlexiQ::Executor::TimeoutError` once its timeout has passed; a long handler calls it between
+units of work. A handler that lets `Cancelled` propagate settles the job as cancelled. One that
+raises after its timeout settles it as a timed-out failure, with the same message the server
+writes for a timeout it reaps itself.
+
+`job.cancelled?`, `job.timed_out?` and `job.deadline` (monotonic seconds) are there for handlers
+that check on their own terms.
+
+## Settings
+
+| Keyword | Default | |
+|---|---|---|
+| `token:` | required | carries the `execute` scope |
+| `id:` | `ruby-<host>-<pid>` | unique among attached executors; a second attach under one id is refused, and retried with backoff |
+| `slots:` | processor count | jobs run at once |
+| `credentials:` / `insecure:` | system roots / `false` | transport security |
+| `max_message_bytes:` | 68 MiB | the door's own ceiling; gRPC's 4 MiB default fails on the first large job |
+| `handshake_timeout:` | 10 | seconds to wait for `hello_ack` |
+| `heartbeat_interval:` | 5 | seconds between heartbeats |
+| `shutdown_drain:` | 30 | seconds running jobs get to finish when draining |
+| `backoff_min:` / `backoff_max:` | 0.25 / 30 | reconnect delay bounds, in seconds |
+| `logger:` | `$stderr` at INFO | any `Logger` |
+
+## Forking
+
+Run the worker as a process of its own. The `grpc` gem is not fork-safe: `Worker.new` opens a
+gRPC channel, and once a process has done that, a child it forks cannot open one at all. The
+child's `Worker.new` raises `RuntimeError` ("grpc cannot be used before and after forking...").
+`GRPC_ENABLE_FORK_SUPPORT=1` does not help here, because `GRPC.prefork` refuses while a
+bidirectional stream is open, and the attach stream is one.
+
+Requiring the gem in a parent is fine; constructing the worker is not. Under a preforking host
+(Puma in cluster mode, Unicorn, a forking job runner), build and `run` the worker in each child
+after the fork, never in an initializer that runs before it.
+
+## What this gem does not do
+
+- **Durable steps.** The `steps` capability is not advertised, so the scheduler sends no step
+  frames. It fails rather than degrades, so it will land as its own feature.
+- **Enqueue.** The executor door has no enqueue-shaped RPC. A task that fans out uses the
+  `flexiq` gem with a `produce` token of its own.
+
+The wire contract this implements is
+[`contracts/REMOTE_SDK_CONTRACT.md`](../../contracts/REMOTE_SDK_CONTRACT.md).
+
+## Development
+
+```sh
+bundle install
+bundle exec rake            # RuboCop and the unit suite, against an in-process scheduler
+bundle exec rake e2e        # against a real flexiq-server; see test/e2e/support/server.rb
+buf generate                # regenerate lib/flexiq/executor/v1 after a contracts/proto edit
+```

data/lib/flexiq/executor/backoff.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module FlexiQ
+  module Executor
+    # Reconnect delay: `min` doubling to `max`, with full jitter so a fleet the scheduler dropped
+    # together does not return together. `reset` after an attach succeeds.
+    class Backoff
+      def initialize(min, max, random: Random.new)
+        @min = min.to_f
+        @max = max.to_f
+        @random = random
+        reset
+      end
+
+      def reset
+        @ceiling = @min
+      end
+
+      def next_delay
+        delay = @min + (@random.rand * (@ceiling - @min))
+        @ceiling = [@ceiling * 2, @max].min
+        delay
+      end
+    end
+  end
+end

data/lib/flexiq/executor/clock.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module FlexiQ
+  module Executor
+    # Monotonic seconds: deadlines and wall times must not move with the system clock.
+    module Clock
+      def self.now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+  end
+end

data/lib/flexiq/executor/config.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "etc"
+require "logger"
+require "socket"
+
+module FlexiQ
+  module Executor
+    # The executor door's message ceiling: the 64 MiB a payload may be, plus 4 MiB of envelope.
+    # Leaving gRPC's 4 MiB default attaches cleanly and fails on the first large job.
+    MAX_MESSAGE_BYTES = 68 * 1024 * 1024
+
+    # The attach protocol this client speaks; `hello_ack` must answer the same number.
+    PROTOCOL_VERSION = 1
+
+    # `hello.slots` is a uint32.
+    MAX_SLOTS = 0xFFFF_FFFF
+
+    CAP_SIDE_CHANNEL = "side_channel"
+    CAP_LEASE = "lease"
+
+    # Capabilities this client implements. `steps` is absent: durable steps fail rather than
+    # degrade, so advertising them without an implementation would be a lie the scheduler acts on.
+    CAPABILITIES = [CAP_SIDE_CHANNEL, CAP_LEASE].freeze
+
+    # Settings for a Worker; see Worker.new for what each one means.
+    Config = Data.define(
+      :token, :id, :slots, :insecure, :credentials, :max_message_bytes, :user_agent, :sdk, :version,
+      :handshake_timeout, :heartbeat_interval, :shutdown_drain, :backoff_min, :backoff_max, :logger
+    ) do
+      def self.default_id = "ruby-#{Socket.gethostname}-#{Process.pid}"
+
+      def self.default_logger = Logger.new($stderr, progname: "flexiq-executor", level: Logger::INFO)
+
+      def initialize(token:, id: Config.default_id, slots: Etc.nprocessors, insecure: false, credentials: nil,
+                     max_message_bytes: MAX_MESSAGE_BYTES, user_agent: nil, sdk: "ruby", version: VERSION,
+                     handshake_timeout: 10, heartbeat_interval: 5, shutdown_drain: 30,
+                     backoff_min: 0.25, backoff_max: 30, logger: Config.default_logger)
+        super
+        validate!
+      end
+
+      def bearer = "Bearer #{token}"
+
+      # What the channel is opened with: TLS unless the caller opted out, and the raised limit
+      # in both directions.
+      def channel_credentials
+        return :this_channel_is_insecure if insecure
+
+        credentials || GRPC::Core::ChannelCredentials.new
+      end
+
+      def channel_args
+        args = { "grpc.max_send_message_length" => max_message_bytes,
+                 "grpc.max_receive_message_length" => max_message_bytes }
+        agent = [user_agent, "flexiq-ruby-executor/#{VERSION}"].compact.join(" ")
+        args.merge("grpc.primary_user_agent" => agent)
+      end
+
+      private
+
+      def validate!
+        validate_token!
+        raise ConfigurationError, "executor id must not be empty" if id.to_s.empty?
+        unless slots.is_a?(Integer) && slots.between?(1, MAX_SLOTS)
+          raise ConfigurationError, "slots must be between 1 and #{MAX_SLOTS}"
+        end
+        raise ConfigurationError, "max_message_bytes must be positive" unless max_message_bytes.to_i.positive?
+
+        validate_durations!
+      end
+
+      def validate_token!
+        raise ConfigurationError, "no token: every call to the executor door carries one" if token.to_s.empty?
+        raise ConfigurationError, "token must be a String" unless token.is_a?(String)
+        raise ConfigurationError, "token must not contain line breaks" if token.match?(/[\r\n]/)
+      end
+
+      def validate_durations!
+        unless [handshake_timeout, heartbeat_interval, shutdown_drain].all? { |value| value.to_f.positive? }
+          raise ConfigurationError, "handshake, heartbeat and drain durations must be positive"
+        end
+        return if backoff_min.to_f.positive? && backoff_max.to_f >= backoff_min.to_f
+
+        raise ConfigurationError, "reconnect backoff must be positive and non-decreasing"
+      end
+    end
+  end
+end

data/lib/flexiq/executor/errors.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+module FlexiQ
+  module Executor
+    # Raise from a handler to fail the job without a retry. Only the executor can see the
+    # exception, so whether to retry is its decision, not the scheduler's.
+    #
+    # Raised with a cause (`raise Fatal, "…"` inside a `rescue`), the job's error names the cause.
+    class Fatal < StandardError; end
+
+    # Raised by Job#check! once the scheduler asked for the job to stop. A handler that lets it
+    # propagate settles the job as cancelled rather than failed.
+    class Cancelled < StandardError; end
+
+    # Raised by Job#check! once the job's timeout has passed. The scheduler never tells an
+    # attached executor about a timeout, so this is enforced here, cooperatively.
+    class TimeoutError < StandardError; end
+
+    # The scheduler refused the attach, or the stream failed.
+    #
+    # `permanent?` is true where reconnecting would only be refused again: a bad credential, a
+    # missing scope, a protocol version mismatch. ALREADY_EXISTS is not among them: a reconnect
+    # can race the scheduler noticing this executor's previous stream is gone.
+    class AttachError < FlexiQ::Error
+      PERMANENT_CODES = [
+        GRPC::Core::StatusCodes::FAILED_PRECONDITION,
+        GRPC::Core::StatusCodes::UNAUTHENTICATED,
+        GRPC::Core::StatusCodes::PERMISSION_DENIED
+      ].freeze
+
+      attr_reader :code, :details
+
+      def initialize(code, details, permanent: PERMANENT_CODES.include?(code))
+        @code = code
+        @details = details
+        @permanent = permanent
+        super("flexiq: attach refused (code #{code}): #{details}")
+      end
+
+      def permanent? = @permanent
+
+      def self.from(status) = new(status.code, status.details)
+    end
+  end
+end

data/lib/flexiq/executor/job.rb

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+
+require "json"
+
+module FlexiQ
+  module Executor
+    # One dispatched job, as a handler sees it.
+    #
+    # Cancellation and timeouts are cooperative: Ruby cannot stop a thread safely, so a long
+    # handler calls `check!` between units of work, or reads `cancelled?` and `deadline` itself.
+    class Job
+      LOG_LEVELS = %w[info warn error result].freeze
+
+      attr_reader :id, :task_name, :queue, :namespace, :retry_count, :max_retries, :timeout,
+                  :disabled_middleware, :metadata, :payload, :lease, :deadline
+
+      # `side` (an Outbox) receives progress and log frames; nil when the scheduler did not
+      # acknowledge `side_channel`, which makes both calls no-ops.
+      def initialize(frame, side: nil, now: Clock.now)
+        @id = frame.id
+        @task_name = frame.task_name
+        @queue = frame.queue
+        @namespace = frame.has_namespace? ? frame.namespace : nil
+        @retry_count = frame.retry_count
+        @max_retries = frame.max_retries
+        @timeout = seconds(frame.timeout)
+        @disabled_middleware = frame.disabled_middleware.to_a.freeze
+        @metadata = frame.has_metadata? ? frame.metadata : nil
+        @payload = frame.payload.b.freeze
+        # Echoed on every frame about this attempt, never inspected or built here.
+        @lease = frame.has_lease? && !frame.lease.empty? ? frame.lease.b.freeze : nil
+        @deadline = @timeout ? now + @timeout : nil
+        @side = side
+        @cancel_requested = false
+      end
+
+      # Positional arguments, decoded from the payload on first use.
+      def args = call[0]
+
+      # Keyword arguments as a Hash with String keys, decoded from the payload on first use.
+      def kwargs = call[1]
+
+      def cancelled? = @cancel_requested
+
+      def timed_out? = !@deadline.nil? && Clock.now >= @deadline
+
+      # Raises Cancelled once the scheduler asked for this job to stop, TimeoutError once its
+      # timeout has passed.
+      def check!
+        raise Cancelled, "job #{id} was cancelled" if cancelled?
+        raise TimeoutError, "job #{id} timed out after #{(timeout * 1000).round}ms" if timed_out?
+      end
+
+      # Reports progress, clamped to 0..100. A no-op without the side channel.
+      def progress(percent)
+        return unless @side
+
+        frame = V1::ProgressFrame.new(job_id: id, progress: percent.to_i.clamp(0, 100), lease: lease)
+        @side.push_side(id, V1::AttachRequest.new(progress: frame), progress: true)
+      end
+
+      # Writes a task log line. `extra` is any JSON-encodable value.
+      def log(level, message, extra: nil)
+        level = level.to_s
+        raise ArgumentError, "log level must be one of #{LOG_LEVELS.join(", ")}" unless LOG_LEVELS.include?(level)
+
+        # Encoded before the side-channel check, so a bad value fails the same way either way.
+        encoded = extra.nil? ? nil : JSON.generate(extra)
+        return unless @side
+
+        frame = V1::TaskLogFrame.new(job_id: id, task_name: task_name, level: level, message: message.to_s,
+                                     extra: encoded&.b, lease: lease)
+        @side.push_side(id, V1::AttachRequest.new(task_log: frame), progress: false)
+      end
+
+      # Publishes a partial result: a log line at level `result` whose value lives in `extra`.
+      def publish(value) = log("result", "", extra: value)
+
+      # Called by the session when a `cancel` frame names this job.
+      def request_cancel! = (@cancel_requested = true)
+
+      private
+
+      def call = (@call ||= FlexiQ::Payload.decode_call(payload))
+
+      def seconds(duration)
+        return nil if duration.nil?
+
+        value = duration.seconds + Rational(duration.nanos, 1_000_000_000)
+        value.positive? ? value : nil
+      end
+    end
+  end
+end

data/lib/flexiq/executor/outbox.rb

@@ -0,0 +1,124 @@
+# frozen_string_literal: true
+
+module FlexiQ
+  module Executor
+    # Every frame bound for the attach stream, handed to gRPC through one Enumerator.
+    #
+    # grpc-ruby drains a bidi request Enumerator on a single thread of its own, so the stream has
+    # exactly one writer. Two queues feed it:
+    #
+    # * settling frames (and `hello`) are never dropped;
+    # * side-channel frames are bounded: a job's progress coalesces to its latest value, and past
+    #   the limit the oldest frame is dropped. Telemetry must not hold back a result.
+    #
+    # A job's pending side-channel frames are promoted ahead of its settling frame: the scheduler
+    # drops a log line that lands after the job it names has settled.
+    class Outbox
+      SIDE_LIMIT = 256
+      # How often an idle writer checks whether the stream has ended under it.
+      IDLE_POLL = 0.05
+
+      Entry = Struct.new(:job_id, :progress, :request)
+      private_constant :Entry
+
+      attr_reader :dropped
+
+      def initialize
+        @mutex = Mutex.new
+        @changed = ConditionVariable.new
+        @settles = []
+        @side = []
+        @dropped = 0
+        @closed = false
+        @drained = false
+      end
+
+      def push(request)
+        @mutex.synchronize do
+          @settles << request
+          @changed.broadcast
+        end
+      end
+
+      def push_side(job_id, request, progress:)
+        @mutex.synchronize do
+          entry = progress && @side.find { |queued| queued.progress && queued.job_id == job_id }
+          if entry
+            entry.request = request
+          else
+            @side << Entry.new(job_id, progress, request)
+            @dropped += 1 if @side.length > SIDE_LIMIT && @side.shift
+          end
+          @changed.broadcast
+        end
+      end
+
+      # Moves a job's pending side-channel frames onto the settling queue, in order.
+      def promote(job_id)
+        @mutex.synchronize do
+          taken, @side = @side.partition { |entry| entry.job_id == job_id }
+          @settles.concat(taken.map(&:request))
+        end
+      end
+
+      # No more frames will be pushed: the writer sends what is queued, then ends the request
+      # stream, which half-closes the call.
+      def close
+        @mutex.synchronize do
+          @closed = true
+          @changed.broadcast
+        end
+      end
+
+      # Waits until the writer has sent everything and ended, up to `budget` seconds.
+      def await_drained(budget)
+        deadline = Clock.now + budget
+        @mutex.synchronize do
+          until @drained
+            remaining = deadline - Clock.now
+            return false unless remaining.positive?
+
+            @changed.wait(@mutex, remaining)
+          end
+          true
+        end
+      end
+
+      # The request stream. `ended` is polled while idle: grpc-ruby does not finish reading a
+      # call until its request Enumerator finishes, so a stream the scheduler ended (a rotation)
+      # would otherwise wait on a writer with nothing to write.
+      def frames(ended)
+        Enumerator.new do |out|
+          while (request = next_frame(ended))
+            out << request
+          end
+        ensure
+          mark_drained
+        end
+      end
+
+      private
+
+      def next_frame(ended)
+        @mutex.synchronize do
+          loop do
+            return nil if ended.call
+
+            request = @settles.shift || @side.shift&.request
+            return request if request
+            return nil if @closed
+
+            @changed.wait(@mutex, IDLE_POLL)
+          end
+        end
+      end
+
+      def mark_drained
+        @mutex.synchronize do
+          @drained = true
+          @changed.broadcast
+        end
+      end
+    end
+  end
+end

data/lib/flexiq/executor/outcome.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+module FlexiQ
+  module Executor
+    # How a handler ended, and the one settling frame that reports it.
+    #
+    # `value` is what the handler returned and `error` what it raised; exactly one is meaningful.
+    # `timed_out` and `cancelled` are read off the job when the handler ends.
+    Outcome = Data.define(:value, :error, :wall, :timed_out, :cancelled) do
+      # The settling frame, with the job's lease echoed when the dispatch carried one.
+      def frame(job)
+        return success(job) if error.nil?
+        return Outcome.timeout_failure(job, wall) if timed_out
+        return cancelled_frame(job) if cancelled && error.is_a?(Cancelled)
+
+        errtype, message, traceback = Outcome.describe(error)
+        Outcome.failure(job, TaskError.encode(errtype, message, traceback), should_retry: !error.is_a?(Fatal),
+                                                                            wall: wall)
+      end
+
+      # The error text core writes for a timeout it reaps itself, so both read alike.
+      def self.timeout_failure(job, wall)
+        message = "job timed out after #{(job.timeout * 1000).round}ms"
+        failure(job, TaskError.encode("TimeoutError", message), should_retry: true, wall: wall, timed_out: true)
+      end
+
+      def self.failure(job, error_json, should_retry:, wall: nil, timed_out: false)
+        V1::AttachRequest.new(failure: V1::FailureFrame.new(
+          job_id: job.id, task_name: job.task_name, error: error_json,
+          retry_count: job.retry_count, max_retries: job.max_retries,
+          wall_time: wall && duration(wall), should_retry: should_retry, timed_out: timed_out, lease: job.lease
+        ))
+      end
+
+      # A Fatal raised with a cause reports the cause: Fatal is the retry decision, not the error.
+      def self.describe(error)
+        error = error.cause if error.is_a?(Fatal) && error.cause
+        [error.class.name || "Error", error.message.to_s, error.backtrace || []]
+      end
+
+      def self.duration(seconds)
+        whole = seconds.floor
+        Google::Protobuf::Duration.new(seconds: whole, nanos: ((seconds - whole) * 1_000_000_000).round)
+      end
+
+      private
+
+      def success(job)
+        result = value.nil? ? nil : FlexiQ::Payload.encode_result(value)
+        V1::AttachRequest.new(success: V1::SuccessFrame.new(
+          job_id: job.id, task_name: job.task_name, result: result,
+          wall_time: Outcome.duration(wall), lease: job.lease
+        ))
+      rescue CodecError => e
+        # The handler's side effects already happened; a retry would repeat them and fail the
+        # same way, so this is fatal.
+        message = "task #{job.task_name.inspect} returned a value that does not encode: #{e.message}"
+        Outcome.failure(job, TaskError.encode("ResultEncodeError", message), should_retry: false, wall: wall)
+      end
+
+      def cancelled_frame(job)
+        V1::AttachRequest.new(cancelled: V1::CancelledFrame.new(
+          job_id: job.id, task_name: job.task_name, wall_time: Outcome.duration(wall), lease: job.lease
+        ))
+      end
+    end
+  end
+end