librats 2.1.4 → 2.2.0

package/native-src/src/librats/subsystems/relay.h

@@ -0,0 +1,211 @@
+#pragma once
+
+/**
+ * @file relay.h
+ * @brief Reaching a peer through a node both ends can already reach.
+ *
+ * The last rung of the connectivity ladder. A direct dial handles the easy cases;
+ * PortMappingService and HolePunch handle most of the rest. What is left is the
+ * pair of nodes for which no endpoint either one can advertise will ever work —
+ * a symmetric NAT on one side, a network that drops UDP and blocks inbound TCP,
+ * a punch that simply never lands. For them the only way through is to borrow a
+ * path: a node connected to both carries the bytes.
+ *
+ * ── What is relayed, and what is not ────────────────────────────────────────
+ * The relayed thing is a *byte stream*, not a message. It becomes a `Circuit` and
+ * a `RelayLink` (transport/relay_link.h), the reactor adopts it as an ordinary
+ * Connection, and everything above the Link runs unchanged — which is why:
+ *
+ *   - the Noise_XX handshake is END TO END. The relay moves ciphertext it cannot
+ *     read, cannot forge and cannot replay. It is a pipe, not a party, and it
+ *     learns nothing beyond who is talking to whom and how much;
+ *   - the peers authenticate each other's real keys, so a relay cannot substitute
+ *     itself for either end;
+ *   - every subsystem — pub/sub, file transfer, PEX — works over a relayed peer
+ *     with no code of its own. To the application it is simply a peer, and
+ *     PeerInfo::transport is the only thing that says otherwise.
+ *
+ * ── Finding a relay ─────────────────────────────────────────────────────────
+ * This stage covers the case where the two ends share a peer — the same topology
+ * HolePunch already relies on for its rendezvous, and the one a bootstrap or DHT
+ * mesh produces on its own. The initiator asks a few of its peers "do you hold
+ * this id?" (Probe) and opens a circuit with the first that says yes. Probing
+ * first, rather than opening with everybody at once, is what keeps a successful
+ * search from producing three redundant connections to the same peer — each with
+ * its own handshake — for the peer table to then tear two of down.
+ *
+ * Not covered here, deliberately: reaching a peer with whom we share NO peer. That
+ * needs reservations (a node asking a relay to hold a slot for it) and a way to
+ * advertise "reachable via R" as an address, which touches identify and PEX. The
+ * version byte and the free op-codes in the wire format leave room for it.
+ *
+ * ── Getting off the relay again ─────────────────────────────────────────────
+ * A circuit is a fallback, not a destination: it costs a third node bandwidth and
+ * a round trip. So when one comes up, this module asks HolePunch to try the target
+ * again — now that the two ends are peers, they can exchange what a punch needs
+ * over the very circuit that is carrying them. If the punch lands, PeerTable::add
+ * prefers the direct link at BOTH ends (any direct transport outranks Relay) and
+ * swaps the route with no disconnect event: the application never sees the seam.
+ *
+ * ── Carrying other peers' traffic ───────────────────────────────────────────
+ * Serving as a relay is off by default. Forwarding a rendezvous, as HolePunch
+ * does, is a few dozen bytes; forwarding a connection is somebody else's file
+ * transfer on your uplink, and that is a decision to be made rather than assumed.
+ * A node that turns it on is protected by, in order of how much they matter:
+ *
+ *   - the end-to-end credit window (transport/relay_link.h), which bounds what one
+ *     circuit can make the relay hold no matter what either end does. Without it a
+ *     slow receiver would grow the relay's send queue until the relay dropped that
+ *     peer entirely — losing all of its traffic, not just the circuit's;
+ *   - forwarding only between peers it already holds. A relay never dials, never
+ *     resolves an address, and can only ever deliver to somebody that chose to
+ *     connect to it, so it cannot be turned into an open reflector;
+ *   - no chaining. A circuit is refused if either end is itself only reachable
+ *     through a relay, which is what stops loops and multi-hop amplification;
+ *   - per-circuit byte and duration caps, per-peer and total circuit counts, and a
+ *     rate limit on the requests themselves.
+ *
+ * ── Threading ───────────────────────────────────────────────────────────────
+ * Message handlers run on reactor threads; one worker thread drives attempt
+ * timeouts and the relay's own expiry. The tables are behind one mutex, which is
+ * never held across a call into a circuit or the reactor — a circuit is touched
+ * only by the reactor thread that owns its connection, which is by construction
+ * the thread that owns its carrier's (see node/circuit_service.h).
+ */
+
+#include "librats/util/rats_export.h"
+#include "librats/core/service_registry.h"
+#include "librats/node/peer_network.h"
+#include "librats/peer/peer_id.h"
+#include "librats/subsystems/relay_service.h"
+#include "librats/transport/relay_link.h"   // Circuit::kDefaultWindow
+
+#include <atomic>
+#include <chrono>
+#include <condition_variable>
+#include <cstdint>
+#include <memory>
+#include <mutex>
+#include <thread>
+
+namespace librats {
+
+class RATS_API Relay final : public Subsystem, public RelayService {
+public:
+    struct Config {
+        // ── Using relays to reach peers we cannot dial ──────────────────────
+
+        /// Open circuits through other peers. Off makes this a relay-only node.
+        bool enable_client = true;
+
+        /// Peers asked whether they hold a target, per attempt. Small: each probe
+        /// is a question to somebody who probably cannot help, and one useful
+        /// answer is all an attempt needs.
+        size_t max_probes = 3;
+
+        /// Circuits this node may hold open as a client at once. Bounds both memory
+        /// and how many relayed peers a discovery module can talk us into.
+        size_t max_outbound_circuits = 8;
+
+        /// How long an attempt may take from the first probe to a live connection.
+        /// Covers the probe round trip, the circuit setup, and the handshake across
+        /// two hops.
+        std::chrono::milliseconds open_timeout{8000};
+
+        /// How long after a failed attempt before the same target may be tried
+        /// again. Only an attempt WE started ever ends in one.
+        std::chrono::milliseconds cooldown{60000};
+
+        // ── Accepting circuits opened to us ─────────────────────────────────
+
+        /// Accept circuits other peers open toward us. Off makes this node
+        /// unreachable by relay, which is a choice a well-connected node can make.
+        bool accept_inbound = true;
+
+        /// Circuits opened TO us that may be live at once.
+        size_t max_inbound_circuits = 16;
+
+        // ── Carrying other peers' traffic ───────────────────────────────────
+
+        /// Serve as a relay. OFF by default: unlike a hole-punch rendezvous, this
+        /// spends real bandwidth on somebody else's connection, so it is opted into
+        /// rather than assumed. A mesh in which nobody serves cannot relay at all.
+        bool serve = false;
+
+        /// Circuits this node will carry for others at once.
+        size_t max_circuits = 32;
+
+        /// Circuits one peer may have us carrying, so a single peer cannot take the
+        /// whole budget.
+        size_t max_circuits_per_peer = 2;
+
+        /// Bytes one circuit may move before it is closed. 0 removes the cap. The
+        /// default is generous for a fallback path and still bounds what one pair
+        /// of peers can spend of ours.
+        uint64_t max_bytes_per_circuit = 64ull * 1024 * 1024;
+
+        /// How long one circuit may live. 0 removes the cap.
+        std::chrono::seconds max_circuit_duration{600};
+
+        /// Requests (probes and circuit openings) one peer may make per window.
+        size_t                    request_budget = 8;
+        std::chrono::milliseconds request_window{1000};
+
+        // ── Both ends ───────────────────────────────────────────────────────
+
+        /// Bytes this node is willing to have in flight per circuit — the credit
+        /// window it advertises (see transport/relay_link.h). Raising it makes a
+        /// relayed peer faster on a long path and asks the relay to hold more.
+        uint32_t window = Circuit::kDefaultWindow;
+
+        /// When a relayed peer connects, ask HolePunch to try it directly, so the
+        /// circuit is replaced by a direct link as soon as one is possible. Costs
+        /// nothing when no HolePunch is attached.
+        bool upgrade_with_hole_punch = true;
+
+        /// Bookkeeping cadence — attempt timeouts, cooldowns, circuit expiry.
+        std::chrono::milliseconds tick{250};
+    };
+
+    Relay();
+    explicit Relay(Config config);
+    ~Relay() override;
+
+    void attach(NodeContext& ctx) override;
+    void start() override;
+    void stop() override;
+
+    /// @copydoc RelayService::connect_via_relay
+    bool connect_via_relay(const PeerId& target) override;
+
+    // — diagnostics and tests —
+    /// Circuits this node terminates (relayed peers, live or still being set up).
+    size_t circuits() const;
+    /// Circuits this node is carrying on behalf of other peers.
+    size_t carried_circuits() const;
+    /// Bytes forwarded on behalf of other peers.
+    uint64_t carried_bytes() const;
+    /// Attempts to reach a peer through a relay that are currently running.
+    size_t attempts() const;
+
+private:
+    /// Everything mutable, held by shared_ptr because a Connection — and therefore
+    /// the Link that reports its circuit released — can outlive this subsystem: the
+    /// node stops subsystems before it stops the reactors that own the connections.
+    struct State;
+
+    Config                 config_;
+    std::shared_ptr<State> state_;
+    /// Kept from attach() so start() can resolve HolePunchService — which may be
+    /// attached after us, and is only guaranteed to have registered by then.
+    ServiceRegistry*       services_ = nullptr;
+
+    std::thread             worker_;
+    std::atomic<bool>       running_{false};
+    std::mutex              worker_mutex_;
+    std::condition_variable worker_cv_;
+
+    void loop();
+};
+
+} // namespace librats

package/native-src/src/librats/subsystems/relay_service.h

@@ -0,0 +1,46 @@
+#pragma once
+
+/**
+ * @file relay_service.h
+ * @brief Capability that lets a sibling module ask for a peer to be reached
+ *        through a third node, by PeerId.
+ *
+ * Published by Relay via ServiceRegistry (see service_registry.h). It is the last
+ * rung of the connectivity ladder — a direct dial, then a hole punch, then this —
+ * and the module that gives up on the rung above should not have to know anything
+ * about how the rung below works. HolePunch, for instance, learns that a target is
+ * unreachable (a symmetric NAT it declined to punch, or a rendezvous that ran out
+ * of attempts) and this is the whole contract it needs to hand the id on: no relay
+ * selection, no circuits, no policy, all of which are Relay's own business.
+ *
+ * Resolving it returns nullptr when relaying is not enabled, which is exactly what
+ * makes the fallback optional at the consumer's side:
+ *
+ *     if (auto* relay = ctx.services.get<RelayService>()) relay->connect_via_relay(id);
+ *
+ * The provider registers during attach(), so resolve it in start() (every attach()
+ * runs before any start()) rather than in your own attach(), where the provider may
+ * not have been attached yet. The pointer is NON-owning and valid while the node is.
+ */
+
+#include "librats/util/rats_export.h"
+#include "librats/peer/peer_id.h"
+
+namespace librats {
+
+struct RATS_API RelayService {
+    virtual ~RelayService() = default;
+
+    /// Try to reach `target` through a node both ends are connected to.
+    /// Non-blocking; success surfaces as an ordinary peer-connected event, and the
+    /// resulting peer is ordinary in every way except that its bytes take a detour
+    /// (PeerInfo::transport is TransportKind::Relay).
+    ///
+    /// @return whether an attempt actually started — false is routine (already
+    ///         connected, an attempt is already running, the target is in cooldown,
+    ///         or this node has no peer that could carry the circuit) and needs no
+    ///         handling by the caller.
+    virtual bool connect_via_relay(const PeerId& target) = 0;
+};
+
+} // namespace librats

package/native-src/src/librats/transport/reactor.cpp

@@ -150,6 +150,27 @@ void Reactor::start_dial(ConnId id, const std::string& host, int port, Transport
     conn->set_dial_address(host, static_cast<uint16_t>(port));
 }
 
+void Reactor::adopt_link(ConnId id, std::unique_ptr<Link> link, ConnRole role, bool connected) {
+    // A unique_ptr cannot be captured by a std::function, which is what the task
+    // queue holds — so the link travels in a shared holder it is moved out of on
+    // arrival. If the task is never run (a reactor stopped under us), the holder
+    // dies with it and takes the link with it, which is the right outcome.
+    auto holder = std::make_shared<std::unique_ptr<Link>>(std::move(link));
+    execute([this, id, holder, role, connected] {
+        Connection* conn = adopt(std::move(*holder), role, id);
+        if (!conn) return;
+        // Already-connected links skip the Connecting state entirely, exactly as an
+        // accepted socket does; the rest wait for the PollOut that wake() will bring
+        // once the far end answers.
+        if (connected) conn->start_handshake();
+    });
+}
+
+void Reactor::wake(ConnId id, uint32_t events) {
+    if (events == 0) return;
+    execute([this, id, events] { dispatch_events(id, events); });
+}
+
 void Reactor::abort_dial(ConnId id, const std::string& host, int port) {
     resolve_dial(id);  // the id is spent either way; release any cancellation slot
     delegate_.on_dial_aborted(index_, id, host, static_cast<uint16_t>(port));
@@ -243,6 +264,21 @@ void Reactor::run() {
         if (mux_) timeout = mux_->next_timeout_ms(timeout);
         const int n = poller_->wait(events, kMaxEvents, timeout);
 
+        // Empty the wakeup pipe BEFORE taking the task snapshot, never after.
+        //
+        // post() pushes a task and then writes a byte, and those two arrive here as
+        // two separate facts. Draining last would let a byte written *while this
+        // turn was running its tasks* be thrown away along with the ones that
+        // brought us here — and the task it belonged to was pushed after the
+        // snapshot, so it does not run either. It would then sit in the queue with
+        // nothing left to announce it, until the next poll timeout woke the loop
+        // for some other reason: up to kMaxPollMs of latency, at random.
+        //
+        // Draining first inverts that. A byte written any time after this line is
+        // still in the pipe when the loop comes back round, so wait() returns at
+        // once — whether or not its task made this turn's snapshot. The cost is one
+        // recv per turn, which is nothing against the work a turn does.
+        drain_wakeup();
         drain_tasks(task_batch);                         // connect/close/send-arm
         for (int i = 0; i < n; ++i) handle_event(events[i]);
         timers_.run_due();
@@ -325,7 +361,9 @@ void Reactor::drain_wakeup() {
 void Reactor::handle_event(const PollResult& ev) {
     const socket_t fd = ev.fd;
 
-    if (fd == wakeup_.fd()) { drain_wakeup(); return; }
+    // Already emptied at the top of the turn — see run(). Nothing to do but
+    // recognise it, so it is not mistaken for a connection's socket.
+    if (fd == wakeup_.fd()) return;
     if (fd == server_socket_) { if (ev.events & PollIn) do_accept(); return; }
     if (mux_ && fd == mux_->socket()) {
         // Any event, not just PollIn. An error on a datagram socket belongs to one
@@ -479,16 +517,26 @@ void Reactor::mark_for_close(ConnId id, CloseReason reason) {
 }
 
 void Reactor::flush_dirty() {
-    if (dirty_.empty()) return;
     // Swapped out first: a flush can close a connection, and a close can queue
     // further work — none of which should extend the batch being walked.
+    //
+    // Looped, because a flush can also *book another one*: writing a relayed
+    // circuit hands its bytes to the connection carrying it (see
+    // transport/relay_link.h), which books its own flush from inside this one. A
+    // single pass would leave those bytes queued until the next turn of the loop —
+    // i.e. behind a poll wait — adding tens of milliseconds to every hop of a
+    // relayed path. The bound is a backstop against a pathological chain, not an
+    // expected limit: honest work needs two passes at most.
     std::vector<ConnId> batch;
-    batch.swap(dirty_);
-    for (const ConnId id : batch) {
-        Connection* conn = find(id);
-        if (!conn) continue;                                   // gone since it booked
-        if (conn->state() != ConnState::Established) continue;  // closing; nothing to owe
-        if (!conn->flush_pending()) mark_for_close(id, conn->close_reason());
+    for (int pass = 0; pass < kMaxFlushPasses && !dirty_.empty(); ++pass) {
+        batch.clear();
+        batch.swap(dirty_);
+        for (const ConnId id : batch) {
+            Connection* conn = find(id);
+            if (!conn) continue;                                   // gone since it booked
+            if (conn->state() != ConnState::Established) continue;  // closing; nothing to owe
+            if (!conn->flush_pending()) mark_for_close(id, conn->close_reason());
+        }
     }
 }
 

package/native-src/src/librats/transport/reactor.h

@@ -104,6 +104,41 @@ public:
     /// Valid for an id whose dial has not started yet — it is cancelled instead.
     void close(ConnId id, CloseReason reason);
 
+    // — links this reactor did not open itself ──────────────────────────────
+    //
+    // A relayed circuit is a byte stream that comes out of another peer's
+    // connection rather than out of a socket (see transport/relay_link.h), so the
+    // module that speaks the relay protocol builds the Link and hands it over.
+    // These three are that hand-over, and they are deliberately the whole of it:
+    // the reactor learns nothing about relaying, and the module learns nothing
+    // about reactors beyond "connections need to be woken".
+    //
+    // The circuit MUST be given to the reactor that owns its carrier's connection.
+    // That is what keeps every byte of it on one thread, exactly like the streams
+    // the mux hands over above, and it is the caller's job to arrange (see
+    // node/circuit_service.h).
+
+    /// Reserve a ConnId without starting anything. Thread-safe, and synchronous for
+    /// the same reason connect() is: the caller has to be able to name the
+    /// connection — to wake it, to close it — from the moment it asks for one,
+    /// rather than after a task it posted has run.
+    ConnId reserve_conn_id() noexcept {
+        return next_conn_id_.fetch_add(1, std::memory_order_relaxed);
+    }
+
+    /// Adopt an already-built Link under the id `reserve_conn_id()` handed out.
+    /// Thread-safe; the adoption itself runs on the reactor thread (inline when the
+    /// caller is already on it).
+    /// @param connected whether the far end is already there. Such a link starts its
+    ///        handshake at once, like an accepted socket; one that is not waits for
+    ///        a PollOut from wake(), exactly as an outbound connect does.
+    void adopt_link(ConnId id, std::unique_ptr<Link> link, ConnRole role, bool connected);
+
+    /// Deliver poll-equivalent events (PollIn/PollOut/PollErr) to a connection whose
+    /// Link has no socket for the poller to watch. Thread-safe. A no-op for an id
+    /// that is gone, so a late wake for a torn-down circuit costs nothing.
+    void wake(ConnId id, uint32_t events);
+
     /// Drop a dial that lost its race — but only while it is still a dial. Once an
     /// attempt has established it is not an attempt any more, it is one of possibly
     /// several links to a peer, and which of those survives is the peer table's
@@ -175,6 +210,9 @@ private:
     // Negligible idle cost; on epoll/kqueue the loop still wakes on real events,
     // so this only caps idle latency.
     static constexpr int kMaxPollMs = 50;
+    /// Passes flush_dirty() makes over the connections booked for a write. More than
+    /// one because a write can book another connection's write — see flush_dirty().
+    static constexpr int kMaxFlushPasses = 4;
     /// Deadline from adopt() to reaching Established (covers connect + handshake).
     static constexpr std::chrono::milliseconds kEstablishTimeout{15000};
     /// Cadence of the housekeeping sweep over this reactor's connections (currently

package/native-src/src/librats/transport/relay_link.cpp

@@ -0,0 +1,208 @@
+#include "librats/transport/relay_link.h"
+
+#include "librats/core/io_poller.h"   // PollIn / PollOut / PollErr
+#include "librats/util/logger.h"
+
+#include <algorithm>
+#include <cstring>
+#include <utility>
+
+namespace librats {
+
+namespace {
+
+/// Slices one data message gathers from the caller's list. The send buffer hands
+/// out a length prefix and a body per queued frame, so this covers a run of frames
+/// rather than a single one; a longer backlog than this simply comes back on the
+/// next turn of Connection::flush.
+constexpr size_t kMaxParts = 16;
+
+} // namespace
+
+Circuit::Circuit(uint32_t id, std::shared_ptr<CircuitCarrier> carrier, bool open,
+                 uint32_t recv_window, uint32_t peer_window)
+    : id_(id),
+      carrier_(std::move(carrier)),
+      state_(open ? State::Open : State::Pending),
+      recv_window_(recv_window == 0 ? kDefaultWindow : recv_window),
+      send_credit_(open ? peer_window : 0) {}
+
+std::shared_ptr<Circuit> Circuit::opening(uint32_t id, std::shared_ptr<CircuitCarrier> carrier,
+                                          uint32_t recv_window) {
+    return std::shared_ptr<Circuit>(
+        new Circuit(id, std::move(carrier), /*open=*/false, recv_window, /*peer_window=*/0));
+}
+
+std::shared_ptr<Circuit> Circuit::accepted(uint32_t id, std::shared_ptr<CircuitCarrier> carrier,
+                                           uint32_t peer_window, uint32_t recv_window) {
+    return std::shared_ptr<Circuit>(
+        new Circuit(id, std::move(carrier), /*open=*/true, recv_window, peer_window));
+}
+
+// ── Fed by the relay module ─────────────────────────────────────────────────
+
+uint32_t Circuit::on_data(ByteView bytes) {
+    if (state_ == State::Closed || bytes.empty()) return 0;
+
+    // The window is the ONE thing bounding what this circuit can cost us and the
+    // relay carrying it, so a peer that sends past what it was granted is not
+    // merely impolite — it is doing the thing the window exists to prevent. Fail
+    // the circuit rather than accept the bytes.
+    if (bytes.size() > recv_window_ - recv_in_flight_) {
+        LOG_WARN("relay", "Circuit " << id_ << " overran its receive window ("
+                 << bytes.size() << " B with " << (recv_window_ - recv_in_flight_)
+                 << " B granted); failing it");
+        return on_closed(CloseReason::ProtocolError, /*orderly=*/false);
+    }
+    recv_in_flight_ += static_cast<uint32_t>(bytes.size());
+
+    // Hand back the consumed prefix before growing: the connection normally drains
+    // the inbox to empty on every readable event, so this is nearly always a reset
+    // of an already-empty buffer rather than a move.
+    if (inbox_read_ == inbox_.size()) {
+        inbox_.clear();
+        inbox_read_ = 0;
+    } else if (inbox_read_ >= kCompactThreshold) {
+        inbox_.erase(inbox_.begin(), inbox_.begin() + static_cast<std::ptrdiff_t>(inbox_read_));
+        inbox_read_ = 0;
+    }
+
+    inbox_.insert(inbox_.end(), bytes.begin(), bytes.end());
+    return PollIn;
+}
+
+uint32_t Circuit::on_accept(uint32_t peer_window) {
+    if (state_ != State::Pending) return 0;
+    state_       = State::Open;
+    send_credit_ = peer_window;
+    // Unconditional, unlike the other openings below: the Connection is still in
+    // Connecting and has nothing queued, so want_write_ is false — yet this is
+    // precisely the event it is waiting for to finish connecting and start its
+    // handshake (see Connection::on_writable).
+    return PollOut;
+}
+
+uint32_t Circuit::on_credit(uint32_t bytes) {
+    if (state_ == State::Closed || bytes == 0) return 0;
+    // Saturate rather than wrap: a far end that grants nonsense should cost us
+    // nothing worse than an over-generous window, and the receiver's own window
+    // check is what actually bounds the traffic.
+    if (bytes > UINT32_MAX - send_credit_) send_credit_ = UINT32_MAX;
+    else                                   send_credit_ += bytes;
+
+    if (!credit_blocked_) return 0;
+    credit_blocked_ = false;
+    return writable_event();
+}
+
+uint32_t Circuit::on_closed(CloseReason reason, bool orderly) {
+    if (state_ == State::Closed) return 0;
+    state_   = State::Closed;
+    reason_  = reason;
+    orderly_ = orderly;
+    // An orderly close still owes the application whatever the far end sent before
+    // it: read() drains the inbox first and only then reports the end of stream, so
+    // what this needs is a readable event, not an error. That is the same contract
+    // the other links keep (see Link::read).
+    return orderly ? PollIn : PollErr;
+}
+
+uint32_t Circuit::on_carrier_writable() {
+    if (!carrier_blocked_) return 0;
+    carrier_blocked_ = false;
+    return writable_event();
+}
+
+uint32_t Circuit::writable_event() const noexcept {
+    if (blocked() || !want_write_ || state_ != State::Open) return 0;
+    return PollOut;
+}
+
+// ── Driven by the Connection ────────────────────────────────────────────────
+
+Link::IoResult Circuit::read(ByteSpan into) {
+    const size_t available = inbox_.size() - inbox_read_;
+    const size_t n = (std::min)(available, into.size());
+    if (n > 0) {
+        std::memcpy(into.data(), inbox_.data() + inbox_read_, n);
+        inbox_read_ += n;
+        if (inbox_read_ == inbox_.size()) {
+            inbox_.clear();
+            inbox_read_ = 0;
+        }
+
+        // These bytes are out of the pipe, so the far end may put that much back
+        // in. Granted in batches: one small message per half-window rather than one
+        // per chunk, which is what keeps the credit scheme's overhead invisible.
+        recv_in_flight_ -= static_cast<uint32_t>(n);
+        recv_ungranted_ += static_cast<uint32_t>(n);
+        if (state_ != State::Closed && recv_ungranted_ >= recv_window_ / kCreditGrantFraction) {
+            carrier_->circuit_send_credit(id_, recv_ungranted_);
+            recv_ungranted_ = 0;
+        }
+        return {n, Link::Status::Ok};
+    }
+
+    // Drained. Only now may the end of the stream be reported — data and
+    // end-of-stream are never delivered together.
+    if (state_ == State::Closed)
+        return {0, orderly_ ? Link::Status::Closed : Link::Status::Error};
+    return {0, Link::Status::WouldBlock};
+}
+
+Link::IoResult Circuit::write(const ByteView* slices, size_t count) {
+    if (state_ == State::Closed) return {0, Link::Status::Error};
+    // Pending: opened but not yet accepted. Nothing may go out — and nothing tries
+    // to, since the Connection is still Connecting.
+    if (state_ != State::Open) return {0, Link::Status::WouldBlock};
+    if (blocked())             return {0, Link::Status::WouldBlock};
+
+    const size_t budget = (std::min)(static_cast<size_t>(send_credit_), kMaxDataChunk);
+    if (budget == 0) {
+        credit_blocked_ = true;
+        return {0, Link::Status::WouldBlock};
+    }
+
+    // One data message per call, gathered across as many of the caller's slices as
+    // fit. Connection::flush loops while a link keeps reporting Ok, so a backlog
+    // larger than one chunk simply comes back around — no loop is needed here, and
+    // each turn of that loop re-checks the credit and the carrier.
+    ByteView parts[kMaxParts];
+    size_t   nparts = 0;
+    size_t   taken  = 0;
+    for (size_t i = 0; i < count && taken < budget && nparts < kMaxParts; ++i) {
+        if (slices[i].empty()) continue;
+        const size_t take = (std::min)(slices[i].size(), budget - taken);
+        parts[nparts++] = ByteView(slices[i].data(), take);
+        taken += take;
+    }
+    if (taken == 0) return {0, Link::Status::WouldBlock};
+
+    send_credit_ -= static_cast<uint32_t>(taken);
+    // The carrier queues what it is given either way; false is "stop and wait", so
+    // the bytes are ours to report as written and the block applies to the NEXT
+    // call (see CircuitCarrier::circuit_send_data).
+    if (!carrier_->circuit_send_data(id_, parts, nparts)) carrier_blocked_ = true;
+    return {taken, Link::Status::Ok};
+}
+
+void Circuit::shutdown(CloseReason reason) {
+    if (!close_sent_) {
+        close_sent_ = true;
+        carrier_->circuit_send_close(id_, reason);
+    }
+    if (state_ != State::Closed) {
+        state_   = State::Closed;
+        reason_  = reason;
+        orderly_ = false;
+    }
+}
+
+void Circuit::release() {
+    if (released_) return;
+    released_ = true;
+    state_    = State::Closed;
+    carrier_->circuit_released(id_);
+}
+
+} // namespace librats