winloop 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 52ea4b37e9fbf3f00710317ae3ce66560b014a39103fcd731c825c77ec8367b5
4
- data.tar.gz: c137e9507075259aaec8a604b5937bb305c8ed6134f33b54cecc8c19b756c21f
3
+ metadata.gz: 6920000bbfa8ee7b4ddb3a773e560ec9fd17ebf7912926fd47ea838b9ccf1ceb
4
+ data.tar.gz: 9c935d93743bf1858e5ab64326dbd3a2b32230a98672f4fbd76417095ceb2e1b
5
5
  SHA512:
6
- metadata.gz: 5406ad357675a336f7fece41a9a4f986449a71d29324b40eb1b852851517d25310417c0e88026bd353de16dd9cf4c9bcc735c7c799a298da8e08937176ad86f2
7
- data.tar.gz: fc833ec7d8d95317b010f9c25b5acae10db5366ce30ffb881c922ca05fb38fa480b928b14830758f1e4b4b02432b40d45dfb0cb7b06a31aa32db4693fcbd6059
6
+ metadata.gz: 3313cb9155d745caa65538b904b8615b6fd3f700c8d9d1d14028e0042fd3de2581acabfd126d783d22989c01d1cee28560f7231209ae6680f94561087e5e30dc
7
+ data.tar.gz: 6a7dda120d347e68e1dcd433eea90db639a7b9dcbfc15d47220b0503ee0ae8f29fc15b1e4a29cc3b8b79dda096e4ebc3607819e798b8339a9aefb15137f54ec3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,58 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.2.0
4
+
5
+ **Bring your own OVERLAPPED** — a generic OVERLAPPED/IOCP completion API, so any
6
+ gem can associate a Windows handle with the loop's completion port, submit its
7
+ own overlapped operation, and park the calling fiber until the completion packet
8
+ arrives. Strictly additive: nothing in the 0.1 API changes signature or behavior,
9
+ and all 34 pre-existing tests pass byte-unmodified.
10
+
11
+ ### Added
12
+
13
+ * **`Winloop::Backend` op API** (the power layer; also works standalone, with no
14
+ scheduler): `#associate(handle)`, `#op_prepare(handle, tag:, capacity:)` →
15
+ `[op_id, ov_addr, buf_addr]` (backend-owned OVERLAPPED + 8-byte-aligned
16
+ embedded buffer), `#op_submitted`, `#op_abandon` (synchronous-failure path —
17
+ the only no-packet case), `#op_cancel` (`CancelIoEx`; **never frees** — the
18
+ cancelled op still posts a packet, which is reaped normally; CancelIoEx
19
+ failures warn and return `false`, never raise), `#op_result` →
20
+ `[bytes, error, data]`, `#op_free`, `#op_state`, and `#port_handle`
21
+ (tests/diagnostics only).
22
+ * **`Winloop::Scheduler` op protocol** (the misuse-resistant cross-gem surface,
23
+ feature-gated by `Fiber.scheduler.respond_to?(:await_op)`): `op_associate`,
24
+ `op_prepare`, `op_submitted`, `op_abandon`, `op_cancel`, `op_state`, and
25
+ `await_op(op_id, timeout: nil)`, which parks the calling fiber until the
26
+ completion is reaped (returns the `[bytes, error, data]` triple, or `nil` on
27
+ timeout after auto-cancelling and orphaning the op). Loop-thread-only,
28
+ enforced.
29
+ * Constants `Winloop::EXTERNAL_KEY` (the completion key carried by all generic
30
+ ops) and `Winloop::OP_CAPACITY_MAX` (16 MiB embedded-buffer ceiling).
31
+ * **Completion errors are data, not exceptions**: the `error` element carries
32
+ the Win32 code mapped from the final NTSTATUS via `RtlNtStatusToDosError`
33
+ (`0` success, `995` cancelled, `1022` RDCW rescan, `38` EOF, …).
34
+ * `op_prepare` rejects handles never passed to `#associate` — an op on a handle
35
+ that is not on the port never completes (a silent hang, not an error).
36
+ * Defensive reap: unknown EXT packets and duplicate posts for already-completed
37
+ ops are skipped with a warning, never cast, never re-completed.
38
+ * `ObjectSpace.memsize_of(backend)` now reports live op-buffer bytes.
39
+ * Gemspec metadata tightened to suite grade: `bug_tracker_uri`,
40
+ `rubygems_mfa_required`.
41
+
42
+ ### Changed
43
+
44
+ * `Backend#shutdown` (and the GC free hook) now also cancels every in-flight
45
+ generic op per op (`CancelIoEx(handle, ov)`), drains their packets — including
46
+ one bounded 100 ms wait taken **only** when generic ops are still in flight —
47
+ and frees retired records. Ops whose packet never surfaced inside the bounded
48
+ drain are deliberately leaked (the kernel may still write their
49
+ OVERLAPPED/IOSB; a use-after-free beats a bounded leak). Existing AFD-only
50
+ workloads see byte-identical shutdown.
51
+ * `Backend#wait` may now return 4-element arrays `[op_id, bytes, error, tag]`
52
+ alongside the existing AFD 2-tuples — when, and only when, the new op API is
53
+ used in the process. Callers dispatch on tuple size; the AFD contract
54
+ (2-tuples, `[]` on timeout, wakeups skipped) is untouched.
55
+
3
56
  ## 0.1.0
4
57
 
5
58
  Initial release.
data/LICENSE.txt CHANGED
@@ -1,6 +1,6 @@
1
1
  MIT License
2
2
 
3
- Copyright (c) 2026 Leiz
3
+ Copyright (c) 2026 ned
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -86,6 +86,7 @@ Inside `Winloop.run { ... }` (and any `Fiber.schedule` started within it):
86
86
  | `Mutex`, `ConditionVariable`, `Queue`, `Thread#join` | `block`/`unblock` | waiter lists + `PostQueuedCompletionStatus` |
87
87
  | `Addrinfo.getaddrinfo`, DNS in `TCPSocket.new` | `address_resolve` | resolved on a worker thread |
88
88
  | `Process.wait` | `process_wait` | reaped on a worker thread |
89
+ | your own OVERLAPPED ops (any gem) | `await_op` | generic completion packets on the same IOCP |
89
90
 
90
91
  ## API
91
92
 
@@ -107,7 +108,144 @@ Fiber.schedule { ... }
107
108
  scheduler.close # runs the event loop to completion
108
109
  ```
109
110
 
110
- ## Scope and limitations (v0.1)
111
+ The generic-op surface (winloop 0.2 — see the next section):
112
+
113
+ ```ruby
114
+ # Scheduler (the cross-gem protocol; loop-thread-only):
115
+ sched.op_associate(handle) # => true (permanent; handle stays yours)
116
+ sched.op_prepare(handle, tag: 0, capacity: 0) # => [op_id, ov_addr, buf_addr]
117
+ sched.op_submitted(op_id) # => true (native call returned TRUE or ERROR_IO_PENDING)
118
+ sched.op_abandon(op_id) # => true (native call failed synchronously)
119
+ sched.op_cancel(op_id) # => true | false (false: already completed, or CancelIoEx failed — warned, never raised)
120
+ sched.op_state(op_id) # => :prepared | :submitted | :completed
121
+ sched.await_op(op_id, timeout: nil) # => [bytes, error, data] | nil (timeout; op auto-cancelled)
122
+
123
+ # Backend (the power layer; also the standalone-embedding layer):
124
+ backend.associate(handle) # => true
125
+ backend.op_prepare(handle, tag: 0, capacity: 0) # => [op_id, ov_addr, buf_addr]
126
+ backend.op_submitted(op_id) # => true
127
+ backend.op_abandon(op_id) # => true
128
+ backend.op_cancel(op_id) # => true | false
129
+ backend.op_result(op_id) # => [bytes, error, data] (frees the record)
130
+ backend.op_free(op_id) # => true (retire a completion nobody wants)
131
+ backend.op_state(op_id) # => :prepared | :submitted | :completed
132
+ backend.port_handle # => Integer (read-only; tests/diagnostics)
133
+ backend.wait(timeout_ms) # => [[id, events], ..., [op_id, bytes, error, tag], ...]
134
+ Winloop::EXTERNAL_KEY # => 0x45585431 — completion key of all generic ops
135
+ Winloop::OP_CAPACITY_MAX # => 16 MiB — op_prepare capacity ceiling
136
+ ```
137
+
138
+ ## Bring your own OVERLAPPED (generic completions)
139
+
140
+ **winloop 0.2 lets any gem associate a Windows handle with the loop's completion
141
+ port, submit its own overlapped operation, and park the calling fiber until the
142
+ completion packet arrives — cancel and synchronous-completion semantics done
143
+ right, with zero change to everything winloop already does.** This is a seam for
144
+ **gem authors** (file watchers, named pipes, mailslots…), not app code.
145
+
146
+ The contract loop: `op_associate` → `op_prepare` → your native submit →
147
+ `op_submitted` (success **or** `ERROR_IO_PENDING` — both queue a packet) /
148
+ `op_abandon` (synchronous failure — the only no-packet case) → `await_op` →
149
+ `[bytes, error, data]`.
150
+
151
+ A real overlapped file read driven from plain Ruby via Fiddle (stdlib):
152
+
153
+ ```ruby
154
+ require "winloop"
155
+ require "fiddle/import"
156
+
157
+ module K32
158
+ extend Fiddle::Importer
159
+ dlload "kernel32.dll"
160
+ extern "void* CreateFileW(void*, unsigned long, unsigned long, void*, unsigned long, unsigned long, void*)"
161
+ extern "int ReadFile(void*, void*, unsigned long, void*, void*)"
162
+ extern "int CloseHandle(void*)"
163
+ end
164
+
165
+ GENERIC_READ = 0x8000_0000
166
+ FILE_SHARE_READ = 0x1
167
+ OPEN_EXISTING = 3
168
+ FILE_FLAG_OVERLAPPED = 0x4000_0000
169
+ ERROR_IO_PENDING = 997
170
+
171
+ Winloop.run do
172
+ sched = Fiber.scheduler # a Winloop::Scheduler
173
+ wpath = (File.expand_path(__FILE__) + "\0").encode("UTF-16LE")
174
+ h = K32.CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, nil,
175
+ OPEN_EXISTING, FILE_FLAG_OVERLAPPED, nil).to_i
176
+ sched.op_associate(h)
177
+ op_id, ov, buf = sched.op_prepare(h, capacity: 4096)
178
+ ok = K32.ReadFile(h, buf, 4096, nil, ov)
179
+ if ok != 0 || Fiddle.win32_last_error == ERROR_IO_PENDING
180
+ sched.op_submitted(op_id) # success AND pending both queue a packet
181
+ bytes, error, data = sched.await_op(op_id, timeout: 5) # fiber parks HERE
182
+ raise Winloop::Error, "read failed (#{error})" unless error.zero?
183
+ puts "read #{bytes} bytes: #{data[0, 40].inspect}"
184
+ else
185
+ sched.op_abandon(op_id) # synchronous failure: no packet will come
186
+ raise Winloop::Error, "ReadFile failed (#{Fiddle.win32_last_error})"
187
+ end
188
+ K32.CloseHandle(h)
189
+ end
190
+ ```
191
+
192
+ ### Rules (read this twice)
193
+
194
+ * **Handles stay yours.** winloop owns the op memory (OVERLAPPED + buffer, one
195
+ backend-owned heap allocation); you open and close your handles — winloop
196
+ never calls `CloseHandle` on them.
197
+ * **Association is permanent** — one port per handle, no disassociation (Win32
198
+ rule). After the loop shuts down an associated handle can never join another
199
+ winloop loop; re-open per `Winloop.run` session.
200
+ * **Associate before prepare.** `op_prepare` rejects handles it has never seen,
201
+ because an op on a handle that is not on the port never completes: not an
202
+ error, a **silent hang**. The one case the check cannot catch — closing a
203
+ handle mid-flight so the OS recycles its value — is on you: cancel → await →
204
+ only then `CloseHandle`.
205
+ * **Use `await_op` timeouts while first integrating a new native API**; switch
206
+ to `nil` once the submit path is proven.
207
+ * Never set `FILE_SKIP_COMPLETION_PORT_ON_SUCCESS` on an associated handle (it
208
+ is irreversible and per-handle): winloop's invariant is exactly one packet per
209
+ submitted op, always. Don't use `ReadFileEx`/`WriteFileEx` (APC I/O) on
210
+ associated handles, and don't duplicate/inherit them.
211
+ * **Cancel never frees** — the cancelled op still posts a packet (usually error
212
+ 995), which is reaped normally.
213
+ * **After `await_op` returns — value or `nil` — the op id is dead.** Don't use
214
+ it again.
215
+ * Loop thread only: ops are submitted on the loop thread (fibers run there
216
+ anyway); the scheduler wrappers enforce it.
217
+ * **Completion errors are values, not exceptions**: `995` cancelled, `1022`
218
+ RDCW rescan, `38` EOF, … A zero-byte successful completion is a real signal
219
+ (RDCW overflow ⇒ rescan), delivered verbatim.
220
+ * No PQCS injection — out of scope in v1. `Backend#port_handle` exists for
221
+ tests and diagnostics only.
222
+
223
+ ### Standalone (no scheduler)
224
+
225
+ Client gems never hard-depend on winloop. The protocol is duck-typed on the
226
+ scheduler:
227
+
228
+ ```ruby
229
+ sched = Fiber.scheduler
230
+ if sched&.respond_to?(:await_op)
231
+ # winloop (or compatible) IOCP path: op_associate / op_prepare / native
232
+ # submit / op_submitted / await_op — as above.
233
+ else
234
+ # the client gem's own standalone mechanism, e.g. the winipc pattern:
235
+ # stack OVERLAPPED + manual-reset hEvent, WaitForSingleObject without the
236
+ # GVL with ubf = CancelIoEx, GetOverlappedResult; or a gem-owned wait thread
237
+ # for continuous streams.
238
+ end
239
+ ```
240
+
241
+ winloop ships **no** shared standalone wait thread or service (dependency
242
+ direction, irreversible association, ownership — clients own their threads).
243
+ `Winloop::Backend` itself does work standalone — create, `associate`,
244
+ prepare/submit, `wait`-poll (dispatch on tuple size: AFD 2-tuples vs op
245
+ 4-tuples in the same array), `op_result`, `shutdown` — which is how an embedder
246
+ or a test drives it deterministically.
247
+
248
+ ## Scope and limitations (v0.2)
111
249
 
112
250
  * **Sockets are the focus.** TCP/UDP sockets get the full IOCP/AFD path. Pipes and
113
251
  regular files fall back to a blocking read inside `io_read` (async file I/O on
@@ -117,6 +255,15 @@ scheduler.close # runs the event loop to completion
117
255
  itself is not a multi-threaded worker pool.
118
256
  * `IOCTL_AFD_POLL` is undocumented; winloop resolves its `ntdll` entry points at
119
257
  runtime and degrades by raising a clear error if `\Device\Afd` is unavailable.
258
+ * **The ops API is for gem authors.** Buffers are backend-owned only (no
259
+ IO::Buffer/String pinning — copy-out removes the whole lifetime/compaction bug
260
+ class). Ops still in flight at shutdown are cancelled and drained for a
261
+ bounded interval; stragglers may be deliberately leaked rather than freed
262
+ under possible kernel ownership.
263
+ * **x64 only.** arm64-mswin is expected to work (all code is `_WIN64`/
264
+ `uintptr_t`-clean, no arch-specific anything — AFD + IOCP are
265
+ production-proven on arm64 via libuv/Node and mio/Rust) but is untested and
266
+ unsupported until an arm64-mswin Ruby distribution exists.
120
267
 
121
268
  ## How it compares
122
269
 
@@ -58,8 +58,14 @@
58
58
 
59
59
  #define WAKE_KEY ((ULONG_PTR)0x57414B45ull) /* "WAKE" — PostQueuedCompletionStatus marker */
60
60
  #define AFD_KEY ((ULONG_PTR)0x41464430ull) /* "AFD0" — the AFD handle's completion key */
61
+ #define EXT_KEY ((ULONG_PTR)0x45585431ull) /* "EXT1" — ALL generic client ops (winloop 0.2) */
61
62
  #define REAP_CAP 64
62
63
 
64
+ /* Generic op state machine: prepare -> submit -> complete (reaped) -> retire (freed).
65
+ * Only a reaped completion ends kernel ownership of the OVERLAPPED + buffer; the
66
+ * sole exception is op_abandon on a PREPARED op the kernel never saw. */
67
+ enum { OP_PREPARED = 0, OP_SUBMITTED = 1, OP_COMPLETED = 2 };
68
+
63
69
  typedef struct _AFD_POLL_HANDLE_INFO {
64
70
  HANDLE Handle;
65
71
  ULONG Events;
@@ -77,10 +83,12 @@ typedef NTSTATUS (NTAPI *PFN_NtCreateFile)(PHANDLE, ACCESS_MASK, POBJECT_ATTRIBU
77
83
  typedef NTSTATUS (NTAPI *PFN_NtDeviceIoControlFile)(HANDLE, HANDLE, PIO_APC_ROUTINE,
78
84
  PVOID, PIO_STATUS_BLOCK, ULONG, PVOID, ULONG, PVOID, ULONG);
79
85
  typedef VOID (NTAPI *PFN_RtlInitUnicodeString)(PUNICODE_STRING, PCWSTR);
86
+ typedef ULONG (NTAPI *PFN_RtlNtStatusToDosError)(NTSTATUS);
80
87
 
81
- static PFN_NtCreateFile pNtCreateFile;
82
- static PFN_NtDeviceIoControlFile pNtDeviceIoControlFile;
83
- static PFN_RtlInitUnicodeString pRtlInitUnicodeString;
88
+ static PFN_NtCreateFile pNtCreateFile;
89
+ static PFN_NtDeviceIoControlFile pNtDeviceIoControlFile;
90
+ static PFN_RtlInitUnicodeString pRtlInitUnicodeString;
91
+ static PFN_RtlNtStatusToDosError pRtlNtStatusToDosError;
84
92
 
85
93
  #ifndef InitializeObjectAttributes
86
94
  #define InitializeObjectAttributes(p, n, a, r, s) do { \
@@ -97,11 +105,33 @@ typedef struct {
97
105
  AFD_POLL_INFO out;
98
106
  } winloop_req;
99
107
 
108
+ /* One generic op (winloop 0.2). OVERLAPPED MUST be first: a completion's
109
+ * lpOverlapped casts back to the record — the same proven pattern as winloop_req. */
110
+ typedef struct {
111
+ OVERLAPPED ov; /* zeroed at prepare; kernel writes Internal/InternalHigh */
112
+ uint64_t id; /* Ruby-visible op id (shares next_id with AFD reqs) */
113
+ uint64_t tag; /* user value, echoed in the completion tuple */
114
+ HANDLE handle; /* associated client handle for CancelIoEx; never NULL */
115
+ int state; /* OP_PREPARED -> OP_SUBMITTED -> OP_COMPLETED */
116
+ DWORD bytes; /* filled at reap */
117
+ DWORD error; /* Win32 code at reap (RtlNtStatusToDosError(ov.Internal)) */
118
+ size_t cap; /* embedded buffer capacity (0 = none) */
119
+ /* unsigned char buf[cap] co-allocated at OP_BUF_OFFSET (8-byte aligned) */
120
+ } winloop_op;
121
+
122
+ #define OP_BUF_OFFSET ((sizeof(winloop_op) + 7) & ~(size_t)7)
123
+
100
124
  typedef struct {
101
125
  HANDLE iocp;
102
126
  HANDLE afd;
103
127
  uint64_t next_id;
104
- st_table *reqs; /* id -> winloop_req* */
128
+ st_table *reqs; /* id -> winloop_req* */
129
+ st_table *ops_by_id; /* id -> winloop_op* (Ruby-facing op methods) */
130
+ st_table *ops_by_addr; /* OVERLAPPED addr -> winloop_op* (defensive reap) */
131
+ st_table *assoc_handles; /* set of HANDLE values passed to #associate (op_prepare
132
+ validation ONLY — never used for blanket cancels, so
133
+ a recycled handle value can't trigger a wrong cancel) */
134
+ size_t op_bytes; /* live op allocations, reported by backend_memsize */
105
135
  int closed;
106
136
  } winloop_backend;
107
137
 
@@ -114,7 +144,9 @@ static int resolve_ntdll(void) {
114
144
  pNtCreateFile = (PFN_NtCreateFile) GetProcAddress(nt, "NtCreateFile");
115
145
  pNtDeviceIoControlFile = (PFN_NtDeviceIoControlFile) GetProcAddress(nt, "NtDeviceIoControlFile");
116
146
  pRtlInitUnicodeString = (PFN_RtlInitUnicodeString) GetProcAddress(nt, "RtlInitUnicodeString");
117
- return pNtCreateFile && pNtDeviceIoControlFile && pRtlInitUnicodeString;
147
+ pRtlNtStatusToDosError = (PFN_RtlNtStatusToDosError) GetProcAddress(nt, "RtlNtStatusToDosError");
148
+ return pNtCreateFile && pNtDeviceIoControlFile && pRtlInitUnicodeString &&
149
+ pRtlNtStatusToDosError;
118
150
  }
119
151
 
120
152
  static HANDLE afd_open(void) {
@@ -159,6 +191,45 @@ static int free_req_i(st_data_t key, st_data_t val, st_data_t arg) {
159
191
  xfree((void *)val);
160
192
  return ST_CONTINUE;
161
193
  }
194
+ /* Shutdown sweep step 1: cancel every still-SUBMITTED generic op, per op
195
+ * (CancelIoEx(handle, ov) — precise; a blanket per-handle cancel would be exposed
196
+ * to the recycled-handle-value hazard if a client closed a handle early). */
197
+ static int cancel_submitted_op_i(st_data_t key, st_data_t val, st_data_t arg) {
198
+ winloop_op *op = (winloop_op *)val;
199
+ (void)key; (void)arg;
200
+ if (op->state == OP_SUBMITTED) CancelIoEx(op->handle, &op->ov);
201
+ /* failures ignored: ERROR_NOT_FOUND = already completed; a (contract-
202
+ * violating) closed handle simply fails the lookup */
203
+ return ST_CONTINUE;
204
+ }
205
+ static int count_submitted_op_i(st_data_t key, st_data_t val, st_data_t arg) {
206
+ winloop_op *op = (winloop_op *)val;
207
+ (void)key;
208
+ if (op->state == OP_SUBMITTED) (*(size_t *)arg)++;
209
+ return ST_CONTINUE;
210
+ }
211
+ /* Shutdown sweep step 4: free every op the kernel can no longer touch. Ops still
212
+ * OP_SUBMITTED (their packet never surfaced inside the bounded drain) are
213
+ * DELIBERATELY LEAKED — the kernel may still write their OVERLAPPED/IOSB, and a
214
+ * use-after-free beats a bounded leak (the libuv tradeoff). In practice the
215
+ * per-op CancelIoEx + the 100 ms drain retires them. */
216
+ static int free_op_unless_submitted_i(st_data_t key, st_data_t val, st_data_t arg) {
217
+ winloop_op *op = (winloop_op *)val;
218
+ (void)key; (void)arg;
219
+ if (op->state != OP_SUBMITTED) xfree(op);
220
+ return ST_CONTINUE;
221
+ }
222
+ /* During the shutdown drain, mark reaped EXT packets' ops OP_COMPLETED so the
223
+ * sweep may free them. Pure C — also runs from the GC free hook (no Ruby calls). */
224
+ static void drain_mark_ext(winloop_backend *b, OVERLAPPED_ENTRY *ents, ULONG n) {
225
+ ULONG i;
226
+ for (i = 0; i < n; i++) {
227
+ st_data_t val;
228
+ if (ents[i].lpCompletionKey != EXT_KEY || ents[i].lpOverlapped == NULL) continue;
229
+ if (!st_lookup(b->ops_by_addr, (st_data_t)(uintptr_t)ents[i].lpOverlapped, &val)) continue;
230
+ ((winloop_op *)val)->state = OP_COMPLETED;
231
+ }
232
+ }
162
233
  static void backend_drain_and_close(winloop_backend *b) {
163
234
  if (b->closed) return;
164
235
  b->closed = 1;
@@ -166,16 +237,36 @@ static void backend_drain_and_close(winloop_backend *b) {
166
237
  /* Cancel everything still pending so the kernel stops owning our OVERLAPPEDs. */
167
238
  if (b->afd != INVALID_HANDLE_VALUE) CancelIoEx(b->afd, NULL);
168
239
  }
240
+ if (b->ops_by_id) st_foreach(b->ops_by_id, cancel_submitted_op_i, 0);
169
241
  /* Drain queued completions (best-effort) so no packet references freed memory. */
170
242
  if (b->iocp) {
171
243
  OVERLAPPED_ENTRY ents[REAP_CAP]; ULONG n = 0; int spins = 0;
172
244
  while (GetQueuedCompletionStatusEx(b->iocp, ents, REAP_CAP, &n, 0, FALSE) && n && spins++ < 4096) {
173
- /* drop them; the requests are freed below from the table */
245
+ if (b->ops_by_addr) drain_mark_ext(b, ents, n);
246
+ }
247
+ /* One bounded 100 ms drain — ONLY when generic ops are still in flight
248
+ * (their cancel completions are usually instants away). Existing AFD-only
249
+ * workloads never take this branch; the GVL stays held (this also runs
250
+ * from the GC free hook, where releasing the GVL is not an option). */
251
+ if (b->ops_by_id) {
252
+ size_t still = 0;
253
+ st_foreach(b->ops_by_id, count_submitted_op_i, (st_data_t)&still);
254
+ if (still &&
255
+ GetQueuedCompletionStatusEx(b->iocp, ents, REAP_CAP, &n, 100, FALSE) && n) {
256
+ drain_mark_ext(b, ents, n);
257
+ spins = 0;
258
+ while (GetQueuedCompletionStatusEx(b->iocp, ents, REAP_CAP, &n, 0, FALSE) && n && spins++ < 4096)
259
+ drain_mark_ext(b, ents, n);
260
+ }
174
261
  }
175
262
  }
176
263
  if (b->afd != INVALID_HANDLE_VALUE) { CloseHandle(b->afd); b->afd = INVALID_HANDLE_VALUE; }
177
264
  if (b->iocp) { CloseHandle(b->iocp); b->iocp = NULL; }
178
265
  if (b->reqs) { st_foreach(b->reqs, free_req_i, 0); st_free_table(b->reqs); b->reqs = NULL; }
266
+ if (b->ops_by_id) { st_foreach(b->ops_by_id, free_op_unless_submitted_i, 0); st_free_table(b->ops_by_id); b->ops_by_id = NULL; }
267
+ if (b->ops_by_addr) { st_free_table(b->ops_by_addr); b->ops_by_addr = NULL; }
268
+ if (b->assoc_handles) { st_free_table(b->assoc_handles); b->assoc_handles = NULL; }
269
+ b->op_bytes = 0;
179
270
  }
180
271
  static void backend_free(void *p) {
181
272
  winloop_backend *b = (winloop_backend *)p;
@@ -183,7 +274,10 @@ static void backend_free(void *p) {
183
274
  backend_drain_and_close(b);
184
275
  xfree(b);
185
276
  }
186
- static size_t backend_memsize(const void *p) { (void)p; return sizeof(winloop_backend); }
277
+ static size_t backend_memsize(const void *p) {
278
+ const winloop_backend *b = (const winloop_backend *)p;
279
+ return sizeof(winloop_backend) + (b ? b->op_bytes : 0);
280
+ }
187
281
  static const rb_data_type_t backend_type = {
188
282
  "Winloop::Backend", { 0, backend_free, backend_memsize }, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY
189
283
  };
@@ -195,6 +289,14 @@ static winloop_backend *get_backend(VALUE self) {
195
289
  return b;
196
290
  }
197
291
 
292
+ /* The op methods additionally need the op tables, which only #initialize creates
293
+ * (a Backend.allocate'd-but-never-initialized object must raise, not crash). */
294
+ static winloop_backend *get_backend_ops(VALUE self) {
295
+ winloop_backend *b = get_backend(self);
296
+ if (!b->ops_by_id) rb_raise(eError, "winloop: backend is closed");
297
+ return b;
298
+ }
299
+
198
300
  static VALUE backend_alloc(VALUE klass) {
199
301
  winloop_backend *b;
200
302
  VALUE obj = TypedData_Make_Struct(klass, winloop_backend, &backend_type, b);
@@ -219,6 +321,11 @@ static VALUE backend_initialize(VALUE self) {
219
321
  }
220
322
  b->next_id = 0;
221
323
  b->reqs = st_init_numtable();
324
+ /* st_data_t is pointer-sized; numtables hold ids/addresses/handles arm64-clean. */
325
+ b->ops_by_id = st_init_numtable();
326
+ b->ops_by_addr = st_init_numtable();
327
+ b->assoc_handles = st_init_numtable();
328
+ b->op_bytes = 0;
222
329
  b->closed = 0;
223
330
  return self;
224
331
  }
@@ -268,6 +375,197 @@ static VALUE backend_cancel(VALUE self, VALUE vid) {
268
375
  return Qnil;
269
376
  }
270
377
 
378
+ /* ===== Generic OVERLAPPED ops — "bring your own OVERLAPPED" (winloop 0.2) =====
379
+ *
380
+ * The whole feature is Integer-in/Integer-out: handles, addresses and ids cross
381
+ * the API as Ruby Integers (the cross-gem ABI — other gems never link winloop).
382
+ * winloop owns op memory (OVERLAPPED + embedded buffer, one heap allocation);
383
+ * clients own their handles — winloop NEVER calls CloseHandle on them. */
384
+
385
+ static const char *op_state_name(int state) {
386
+ switch (state) {
387
+ case OP_PREPARED: return "prepared";
388
+ case OP_SUBMITTED: return "submitted";
389
+ default: return "completed";
390
+ }
391
+ }
392
+
393
+ static winloop_op *op_lookup(winloop_backend *b, uint64_t id) {
394
+ st_data_t val;
395
+ if (!st_lookup(b->ops_by_id, (st_data_t)id, &val))
396
+ rb_raise(eError, "winloop: unknown op id %llu", (unsigned long long)id);
397
+ return (winloop_op *)val;
398
+ }
399
+
400
+ /* Remove an op from both tables and free it (the ONLY free path besides the
401
+ * shutdown sweep). Callers must have established the kernel no longer owns it. */
402
+ static void op_retire(winloop_backend *b, winloop_op *op) {
403
+ st_data_t key = (st_data_t)op->id;
404
+ st_delete(b->ops_by_id, &key, NULL);
405
+ key = (st_data_t)(uintptr_t)&op->ov;
406
+ st_delete(b->ops_by_addr, &key, NULL);
407
+ b->op_bytes -= OP_BUF_OFFSET + op->cap;
408
+ xfree(op);
409
+ }
410
+
411
+ /* Associate `handle` (opened FILE_FLAG_OVERLAPPED) with the backend's IOCP under
412
+ * EXT_KEY. Permanent for the handle's lifetime (Win32: one port per handle, no
413
+ * disassociation); the caller keeps ownership. */
414
+ static VALUE backend_associate(VALUE self, VALUE vhandle) {
415
+ winloop_backend *b = get_backend_ops(self);
416
+ uint64_t hv = NUM2ULL(vhandle); /* coercion first: a raise here owns nothing */
417
+ DWORD gle;
418
+ if (hv == 0 || hv == UINT64_MAX)
419
+ rb_raise(rb_eArgError, "winloop: %llu is not a usable HANDLE value",
420
+ (unsigned long long)hv);
421
+ if (!CreateIoCompletionPort((HANDLE)(uintptr_t)hv, b->iocp, EXT_KEY, 0)) {
422
+ gle = GetLastError(); /* captured before any Ruby allocation */
423
+ rb_raise(eError, "winloop: could not associate handle %llu with the completion "
424
+ "port (error %lu: already associated with a completion port, or not "
425
+ "opened with FILE_FLAG_OVERLAPPED)", (unsigned long long)hv, gle);
426
+ }
427
+ st_insert(b->assoc_handles, (st_data_t)hv, (st_data_t)1);
428
+ return Qtrue;
429
+ }
430
+
431
+ /* Private primitive behind Winloop::Backend#op_prepare (lib/winloop/ops.rb owns
432
+ * the kwargs + range validation). Allocates one op record: zeroed OVERLAPPED +
433
+ * tag + optional embedded buffer; returns [op_id, ov_addr, buf_addr]. */
434
+ static VALUE backend__op_prepare(VALUE self, VALUE vhandle, VALUE vtag, VALUE vcap) {
435
+ winloop_backend *b = get_backend_ops(self);
436
+ /* All coercion before any allocation (raise hygiene). */
437
+ uint64_t hv = NUM2ULL(vhandle);
438
+ uint64_t tag = NUM2ULL(vtag);
439
+ uint64_t cap = NUM2ULL(vcap);
440
+ size_t total;
441
+ winloop_op *op;
442
+ if (cap > 16ull * 1024 * 1024) /* defensive twin of OP_CAPACITY_MAX (send bypass) */
443
+ rb_raise(rb_eArgError, "winloop: capacity %llu exceeds OP_CAPACITY_MAX",
444
+ (unsigned long long)cap);
445
+ /* The anti-hang check (§3.4 rule 8): an op on a handle that is not on the
446
+ * port never completes — not an error, a silent loop freeze. Honestly: a
447
+ * handle that was associated, closed and recycled to the same value passes. */
448
+ if (!st_lookup(b->assoc_handles, (st_data_t)hv, NULL))
449
+ rb_raise(eError, "winloop: handle %llu was never associated with this backend "
450
+ "— call #associate first", (unsigned long long)hv);
451
+
452
+ total = OP_BUF_OFFSET + (size_t)cap;
453
+ op = (winloop_op *)xmalloc(total); /* a raise above this line leaks nothing */
454
+ memset(op, 0, total); /* zeroed OVERLAPPED (offset 0 reads); buffer never leaks heap */
455
+ op->id = ++b->next_id;
456
+ op->tag = tag;
457
+ op->handle = (HANDLE)(uintptr_t)hv;
458
+ op->state = OP_PREPARED;
459
+ op->cap = (size_t)cap;
460
+ st_insert(b->ops_by_id, (st_data_t)op->id, (st_data_t)op);
461
+ /* An OOM longjmp from this second insert leaks one record until the shutdown
462
+ * sweep — the identical, accepted exposure as backend_poll's st_insert. */
463
+ st_insert(b->ops_by_addr, (st_data_t)(uintptr_t)&op->ov, (st_data_t)op);
464
+ b->op_bytes += total;
465
+ return rb_ary_new_from_args(3, ULL2NUM(op->id), ULL2NUM((uintptr_t)&op->ov),
466
+ cap ? ULL2NUM((uintptr_t)((char *)op + OP_BUF_OFFSET))
467
+ : ULL2NUM(0));
468
+ }
469
+
470
+ /* Call after the client's native call returned success OR ERROR_IO_PENDING —
471
+ * both queue exactly one completion packet (no skip-on-success, ever). */
472
+ static VALUE backend_op_submitted(VALUE self, VALUE vid) {
473
+ winloop_backend *b = get_backend_ops(self);
474
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
475
+ if (op->state != OP_PREPARED)
476
+ rb_raise(eError, "winloop: op %llu is %s; op_submitted is only legal on a "
477
+ "prepared op", (unsigned long long)op->id, op_state_name(op->state));
478
+ op->state = OP_SUBMITTED;
479
+ return Qtrue;
480
+ }
481
+
482
+ /* For a native call that failed SYNCHRONOUSLY (GetLastError != ERROR_IO_PENDING):
483
+ * the kernel never saw the op, no packet will ever arrive — free immediately. */
484
+ static VALUE backend_op_abandon(VALUE self, VALUE vid) {
485
+ winloop_backend *b = get_backend_ops(self);
486
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
487
+ if (op->state == OP_SUBMITTED)
488
+ rb_raise(eError, "winloop: op %llu is submitted; a submitted op is retired by "
489
+ "its completion, not op_abandon", (unsigned long long)op->id);
490
+ if (op->state == OP_COMPLETED)
491
+ rb_raise(eError, "winloop: op %llu is completed; retire it with op_result or "
492
+ "op_free, not op_abandon", (unsigned long long)op->id);
493
+ op_retire(b, op);
494
+ return Qtrue;
495
+ }
496
+
497
+ /* CancelIoEx(handle, ov) — targeted, cross-thread-safe. NEVER frees: a cancelled
498
+ * op still posts a packet (usually error 995) which #wait reaps. Returns true if
499
+ * a cancel was issued; false on ERROR_NOT_FOUND (already completed — benign) or
500
+ * ANY other CancelIoEx failure (rb_warn'ed, never raised: await_op's ensure path
501
+ * must never mask an in-flight Timeout/Fiber#raise unwind with a Winloop::Error). */
502
+ static VALUE backend_op_cancel(VALUE self, VALUE vid) {
503
+ winloop_backend *b = get_backend_ops(self);
504
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
505
+ DWORD gle;
506
+ if (op->state == OP_PREPARED)
507
+ rb_raise(eError, "winloop: op %llu is prepared; only a submitted op can be "
508
+ "cancelled (a prepared op is retired with op_abandon)",
509
+ (unsigned long long)op->id);
510
+ if (CancelIoEx(op->handle, &op->ov)) return Qtrue;
511
+ gle = GetLastError(); /* captured immediately, before any Ruby call */
512
+ if (gle != ERROR_NOT_FOUND)
513
+ rb_warn("winloop: CancelIoEx for op %llu failed (error %lu) — treated as "
514
+ "no-cancel, never an exception", (unsigned long long)op->id, gle);
515
+ return Qfalse;
516
+ }
517
+
518
+ /* [bytes, error, data] for a COMPLETED op; frees the record. The String is built
519
+ * BEFORE the free so an OOM raise leaves the op intact and retryable. `data` is
520
+ * clamped to min(bytes, capacity): a stray packet's bogus byte count cannot make
521
+ * winloop read past its own buffer. */
522
+ static VALUE backend_op_result(VALUE self, VALUE vid) {
523
+ winloop_backend *b = get_backend_ops(self);
524
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
525
+ VALUE data = Qnil, result;
526
+ if (op->state != OP_COMPLETED)
527
+ rb_raise(eError, "winloop: op %llu is %s; op_result is only legal once its "
528
+ "completion has been reaped by #wait",
529
+ (unsigned long long)op->id, op_state_name(op->state));
530
+ if (op->cap) {
531
+ size_t n = (size_t)op->bytes < op->cap ? (size_t)op->bytes : op->cap;
532
+ data = rb_str_new((const char *)op + OP_BUF_OFFSET, (long)n); /* ASCII-8BIT */
533
+ }
534
+ result = rb_ary_new_from_args(3, ULONG2NUM(op->bytes), ULONG2NUM(op->error), data);
535
+ op_retire(b, op);
536
+ return result;
537
+ }
538
+
539
+ /* Retire a COMPLETED op without materializing its data (orphaned completions). */
540
+ static VALUE backend_op_free(VALUE self, VALUE vid) {
541
+ winloop_backend *b = get_backend_ops(self);
542
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
543
+ if (op->state != OP_COMPLETED)
544
+ rb_raise(eError, "winloop: op %llu is %s; op_free is only legal once its "
545
+ "completion has been reaped by #wait",
546
+ (unsigned long long)op->id, op_state_name(op->state));
547
+ op_retire(b, op);
548
+ return Qtrue;
549
+ }
550
+
551
+ static VALUE backend_op_state(VALUE self, VALUE vid) {
552
+ winloop_backend *b = get_backend_ops(self);
553
+ winloop_op *op = op_lookup(b, NUM2ULL(vid));
554
+ switch (op->state) {
555
+ case OP_PREPARED: return ID2SYM(rb_intern("prepared"));
556
+ case OP_SUBMITTED: return ID2SYM(rb_intern("submitted"));
557
+ default: return ID2SYM(rb_intern("completed"));
558
+ }
559
+ }
560
+
561
+ /* The raw IOCP HANDLE value — read-only, for tests and diagnostics. Hard rules:
562
+ * never wait on it, never close it, never associate handles behind winloop's
563
+ * back, never post after shutdown, at most ONE post per op. */
564
+ static VALUE backend_port_handle(VALUE self) {
565
+ winloop_backend *b = get_backend_ops(self);
566
+ return ULL2NUM((uintptr_t)b->iocp);
567
+ }
568
+
271
569
  /* Reap completions in one GetQueuedCompletionStatusEx. Returns an Array of
272
570
  [id, ready_events]; wakeup packets are skipped. Empty array on timeout. */
273
571
  /* The blocking GQCSEx call, run WITHOUT the GVL so other threads (and the very
@@ -311,6 +609,45 @@ static VALUE backend_wait(VALUE self, VALUE vtimeout_ms) {
311
609
  }
312
610
  for (ULONG i = 0; i < n; i++) {
313
611
  if (ents[i].lpCompletionKey == WAKE_KEY || ents[i].lpOverlapped == NULL) continue;
612
+ if (ents[i].lpCompletionKey == EXT_KEY) {
613
+ /* Generic-op path (winloop 0.2). Defensive dispatch: verify membership
614
+ * in ops_by_addr AND op state before acting — an unknown lpOverlapped
615
+ * is never cast, an already-COMPLETED op (double post) is never
616
+ * re-completed and never yields a second tuple. The only accepted
617
+ * anomaly is a packet for a still-PREPARED op (a standalone embedder
618
+ * called #wait between the native submit and op_submitted): the packet
619
+ * is real and the memory is ours, so accept it with a warning. */
620
+ st_data_t val;
621
+ winloop_op *op;
622
+ if (!st_lookup(b->ops_by_addr, (st_data_t)(uintptr_t)ents[i].lpOverlapped, &val)) {
623
+ rb_warn("winloop: dropped a completion packet for an unknown OVERLAPPED "
624
+ "0x%llx (stray or duplicate post)",
625
+ (unsigned long long)(uintptr_t)ents[i].lpOverlapped);
626
+ continue;
627
+ }
628
+ op = (winloop_op *)val;
629
+ if (op->state == OP_COMPLETED) {
630
+ rb_warn("winloop: dropped a duplicate completion packet for op %llu "
631
+ "(already completed; bytes/error preserved)",
632
+ (unsigned long long)op->id);
633
+ continue;
634
+ }
635
+ if (op->state == OP_PREPARED)
636
+ rb_warn("winloop: completion packet reaped for op %llu before "
637
+ "op_submitted (protocol violation; accepting the packet)",
638
+ (unsigned long long)op->id);
639
+ op->state = OP_COMPLETED;
640
+ op->bytes = ents[i].dwNumberOfBytesTransferred;
641
+ /* Status lives in the OVERLAPPED's Internal (an NTSTATUS), readable
642
+ * only after the packet left the port; map it to a familiar Win32
643
+ * code (0 ok, 995 cancelled, 1022 rescan, 38 EOF, ...). */
644
+ op->error = pRtlNtStatusToDosError((NTSTATUS)op->ov.Internal);
645
+ rb_ary_push(result, rb_ary_new_from_args(4, ULL2NUM(op->id),
646
+ ULONG2NUM(op->bytes), ULONG2NUM(op->error), ULL2NUM(op->tag)));
647
+ continue;
648
+ }
649
+ /* AFD path — byte-for-byte the pre-0.2 code (only the AFD handle carries
650
+ * AFD_KEY on this port). */
314
651
  winloop_req *req = (winloop_req *)ents[i].lpOverlapped;
315
652
  st_data_t key = (st_data_t)req->id;
316
653
  st_delete(b->reqs, &key, NULL);
@@ -348,6 +685,11 @@ void Init_winloop(void) {
348
685
  rb_define_const(mWinloop, "WRITABLE", INT2NUM(RB_WRITABLE));
349
686
  rb_define_const(mWinloop, "PRIORITY", INT2NUM(RB_PRIORITY));
350
687
 
688
+ /* The completion key winloop assigns to ALL generic ops on the port
689
+ * (internal dispatch; exposed for tests and a possible future injection
690
+ * protocol — PQCS injection is NOT a v1 protocol). */
691
+ rb_define_const(mWinloop, "EXTERNAL_KEY", ULL2NUM(0x45585431ull));
692
+
351
693
  rb_define_alloc_func(cBackend, backend_alloc);
352
694
  rb_define_method(cBackend, "initialize", RUBY_METHOD_FUNC(backend_initialize), 0);
353
695
  rb_define_method(cBackend, "poll", RUBY_METHOD_FUNC(backend_poll), 2);
@@ -355,4 +697,16 @@ void Init_winloop(void) {
355
697
  rb_define_method(cBackend, "wait", RUBY_METHOD_FUNC(backend_wait), 1);
356
698
  rb_define_method(cBackend, "wakeup", RUBY_METHOD_FUNC(backend_wakeup), 0);
357
699
  rb_define_method(cBackend, "shutdown", RUBY_METHOD_FUNC(backend_shutdown), 0);
700
+
701
+ /* Generic OVERLAPPED ops (winloop 0.2). `_op_prepare` is the private bridge
702
+ * behind the validated Winloop::Backend#op_prepare in lib/winloop/ops.rb. */
703
+ rb_define_method(cBackend, "associate", RUBY_METHOD_FUNC(backend_associate), 1);
704
+ rb_define_method(cBackend, "_op_prepare", RUBY_METHOD_FUNC(backend__op_prepare), 3);
705
+ rb_define_method(cBackend, "op_submitted", RUBY_METHOD_FUNC(backend_op_submitted), 1);
706
+ rb_define_method(cBackend, "op_abandon", RUBY_METHOD_FUNC(backend_op_abandon), 1);
707
+ rb_define_method(cBackend, "op_cancel", RUBY_METHOD_FUNC(backend_op_cancel), 1);
708
+ rb_define_method(cBackend, "op_result", RUBY_METHOD_FUNC(backend_op_result), 1);
709
+ rb_define_method(cBackend, "op_free", RUBY_METHOD_FUNC(backend_op_free), 1);
710
+ rb_define_method(cBackend, "op_state", RUBY_METHOD_FUNC(backend_op_state), 1);
711
+ rb_define_method(cBackend, "port_handle", RUBY_METHOD_FUNC(backend_port_handle), 0);
358
712
  }
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Winloop
4
+ # Upper bound for Backend#op_prepare's `capacity:` — 16 MiB. Fits a DWORD with
5
+ # huge margin (and is far past ReadDirectoryChangesW's 64 KiB network ceiling).
6
+ OP_CAPACITY_MAX = 16 * 1024 * 1024
7
+
8
+ # The validated, keyword-argument face of the generic-op preparation primitive.
9
+ # Suite convention: validated kwargs live in Ruby; the C bridge (_op_prepare)
10
+ # is underscore-private so the checks can't be skipped.
11
+ class Backend
12
+ # Allocate one backend-owned op record (state :prepared): a zeroed OVERLAPPED,
13
+ # a `tag` echoed back in the completion tuple, and an optional embedded data
14
+ # buffer. Returns [op_id, ov_addr, buf_addr] (all Integers; buf_addr is 0 when
15
+ # capacity is 0). `handle` must have been passed to #associate on this backend
16
+ # first — an op on a handle that is not on the port never completes (a silent
17
+ # hang, not an error), so unknown handles raise Winloop::Error up front.
18
+ def op_prepare(handle, tag: 0, capacity: 0)
19
+ raise TypeError, "handle must be an Integer (got #{handle.class})" unless handle.is_a?(Integer)
20
+ raise TypeError, "tag must be an Integer (got #{tag.class})" unless tag.is_a?(Integer)
21
+ raise TypeError, "capacity must be an Integer (got #{capacity.class})" unless capacity.is_a?(Integer)
22
+ if handle.zero? || handle == 0xFFFF_FFFF_FFFF_FFFF || !handle.between?(1, 2**64 - 1)
23
+ raise ArgumentError, "#{handle} is not a usable HANDLE value"
24
+ end
25
+ unless tag.between?(0, 2**64 - 1)
26
+ raise ArgumentError, "tag must be in 0..2**64-1 (got #{tag})"
27
+ end
28
+ unless capacity.between?(0, OP_CAPACITY_MAX)
29
+ raise ArgumentError, "capacity must be in 0..#{OP_CAPACITY_MAX} (got #{capacity})"
30
+ end
31
+ _op_prepare(handle, tag, capacity)
32
+ end
33
+
34
+ private :_op_prepare
35
+ end
36
+ end
@@ -49,6 +49,16 @@ module Winloop
49
49
  @lock = Thread::Mutex.new # guards @ready + @parked (the cross-thread path)
50
50
  @loop = Fiber.current # the thread's root fiber == the loop
51
51
  @closed = false
52
+
53
+ # ---- generic OVERLAPPED op state (winloop 0.2) ----
54
+ @thread = Thread.current # the op API is loop-thread-only (enforced)
55
+ @op_waiters = {} # op_id => Waiter (a fiber parked in await_op)
56
+ @op_orphans = {} # op_id => true (timed-out/unwound awaits; the late
57
+ # packet is retired without waking anyone)
58
+ @op_done = {} # op_id => [bytes, error] (completed before anyone awaited)
59
+ @op_shut = false # true once the loop has fully finished (backend shut down).
60
+ # NOT @closed: under Winloop.run, #close IS the event loop,
61
+ # so @closed is true while op-protocol fibers still run.
52
62
  end
53
63
 
54
64
  def monotonic
@@ -242,6 +252,99 @@ module Winloop
242
252
  await_on_thread { Process::Status.wait(pid, flags) }
243
253
  end
244
254
 
255
+ # ---- generic OVERLAPPED completions (the cross-gem op protocol) ---------
256
+ #
257
+ # Client gems feature-detect with `Fiber.scheduler.respond_to?(:await_op)`
258
+ # and talk ONLY to these wrappers (never the Backend — they cannot reach
259
+ # #wait/#shutdown and corrupt the loop). The contract loop:
260
+ #
261
+ # op_associate(h) # once, at handle-open time
262
+ # op_id, ov, buf = op_prepare(h, capacity: n)
263
+ # ok = <the client's own native submit using ov/buf>
264
+ # if ok # native call returned TRUE or GetLastError == ERROR_IO_PENDING
265
+ # op_submitted(op_id)
266
+ # bytes, error, data = await_op(op_id, timeout: t) # fiber parks here
267
+ # else # synchronous failure: no packet will ever arrive
268
+ # op_abandon(op_id)
269
+ # end
270
+ #
271
+ # All of these are loop-thread-only and raise once the scheduler has shut down.
272
+
273
+ def op_associate(handle)
274
+ op_guard!
275
+ @backend.associate(handle)
276
+ end
277
+
278
+ def op_prepare(handle, tag: 0, capacity: 0)
279
+ op_guard!
280
+ @backend.op_prepare(handle, tag: tag, capacity: capacity)
281
+ end
282
+
283
+ def op_submitted(op_id)
284
+ op_guard!
285
+ @backend.op_submitted(op_id)
286
+ end
287
+
288
+ def op_abandon(op_id)
289
+ op_guard!
290
+ @backend.op_abandon(op_id)
291
+ end
292
+
293
+ def op_cancel(op_id)
294
+ op_guard!
295
+ @backend.op_cancel(op_id)
296
+ end
297
+
298
+ def op_state(op_id)
299
+ op_guard!
300
+ @backend.op_state(op_id)
301
+ end
302
+
303
+ # Park the calling fiber until op_id's completion packet is reaped by the
304
+ # loop, then return [bytes, error, data] (op_result is pulled HERE, in the
305
+ # woken fiber, so orphaned completions never allocate Strings). On timeout:
306
+ # returns nil, auto-cancels and orphans the op. Either way the op is retired:
307
+ # after await_op returns — value or nil — the op_id must not be used again.
308
+ #
309
+ # timeout: nil (wait forever — safe provided the op was submitted on a handle
310
+ # that is still the associated handle) or a positive Numeric in seconds.
311
+ # Recommendation: pass a real timeout while first integrating a new native
312
+ # API; switch to nil once the submit path is proven.
313
+ def await_op(op_id, timeout: nil)
314
+ op_guard!
315
+ validate_op_timeout!(timeout)
316
+ if @op_done.delete(op_id) # early completion: no yield
317
+ return @backend.op_result(op_id)
318
+ end
319
+ raise Error, "winloop: op #{op_id} already has a waiter" if @op_waiters.key?(op_id)
320
+ raise Error, "winloop: op #{op_id} has not been submitted" if @backend.op_state(op_id) == :prepared
321
+ waiter = Waiter.new(Fiber.current, false)
322
+ @op_waiters[op_id] = waiter
323
+ @timers.push(Timer.new(monotonic + timeout, waiter, nil, nil)) if timeout
324
+ completed = Fiber.yield # [bytes, error] or nil (timer fired)
325
+ completed ? @backend.op_result(op_id) : nil
326
+ ensure
327
+ # NOTHING in this ensure may raise — it runs during Timeout::Error /
328
+ # Fiber#raise / Fiber#kill unwinds and must never mask the in-flight
329
+ # exception (op_cancel's no-raise-on-CancelIoEx contract exists for this).
330
+ if waiter
331
+ waiter.settled = true
332
+ if @op_waiters.delete(op_id) # loop did NOT deliver: timeout or unwind
333
+ @op_orphans[op_id] = true
334
+ begin
335
+ @backend.op_cancel(op_id) # late packet -> orphan path retires it
336
+ rescue Error
337
+ # backend already shut down mid-unwind: nothing left to cancel
338
+ end
339
+ elsif !completed # loop DID deliver, but we were unwound
340
+ # (Fiber#kill, or a sibling raise) before resuming with the value: the
341
+ # record sits OP_COMPLETED with nobody left to op_result it — free it
342
+ # now. Guarded: the loop's settled/timer branch may have retired the id.
343
+ @backend.op_free(op_id) if op_completed_unconsumed?(op_id)
344
+ end
345
+ end
346
+ end
347
+
245
348
  # ---- lifecycle ---------------------------------------------------------
246
349
 
247
350
  # MRI calls this automatically at thread/program exit. It IS the event loop.
@@ -250,6 +353,7 @@ module Winloop
250
353
  @closed = true
251
354
  run
252
355
  ensure
356
+ @op_shut = true # the op protocol is over; op_* / await_op raise from here on
253
357
  @backend.shutdown
254
358
  # NEVER call Fiber.set_scheduler(nil) here — it raises SystemStackError
255
359
  # during MRI's implicit close at thread exit.
@@ -270,7 +374,10 @@ module Winloop
270
374
  # #unblock that is moving a waiter into @ready right now.
271
375
  @lock.synchronize do
272
376
  drop_settled_timers
273
- @ready.empty? && @timers.empty? && @ops.empty? && @parked.empty?
377
+ @ready.empty? && @timers.empty? && @ops.empty? && @parked.empty? &&
378
+ @op_waiters.empty?
379
+ # @op_orphans/@op_done deliberately do NOT keep the loop alive —
380
+ # shutdown's cancel+drain+sweep retires whatever is left.
274
381
  end
275
382
  end
276
383
 
@@ -321,40 +428,66 @@ module Winloop
321
428
  elsif (t = @timers.peek)
322
429
  ms = ((t.deadline - monotonic) * 1000).ceil
323
430
  timeout_ms = ms < 0 ? 0 : ms
324
- elsif @ops.empty? && @parked.empty?
431
+ elsif @ops.empty? && @parked.empty? && @op_waiters.empty?
325
432
  idle = true # nothing to wait for
326
433
  else
327
- timeout_ms = nil # INFINITE — pending I/O or parked fibers wake us
434
+ timeout_ms = nil # INFINITE — pending I/O, parked fibers or awaited ops
435
+ # wake us. An awaited op pending => INFINITE is correct for contract-
436
+ # abiding ops: its completion (or its cancel's completion) wakes the
437
+ # port — guaranteed because op_prepare verified the handle is on it.
328
438
  end
329
439
  end
330
440
  return if idle
331
441
 
332
- # (d) one GetQueuedCompletionStatusEx; fan each completion out to the
333
- # io waiters whose interest the ready mask satisfies.
334
- @backend.wait(timeout_ms).each do |id, ready_events|
335
- entry = @ops.delete(id)
336
- next unless entry # stale: a superseded/cancelled poll
337
- entry.id = nil
338
- entry.armed = 0
339
- to_wake = []
340
- entry.waiters.reject! do |w|
341
- if w.settled
342
- true
343
- elsif (got = w.events & ready_events) != 0
344
- w.settled = true
345
- to_wake << [w.fiber, got]
346
- true
347
- else
348
- false # this waiter's direction didn't fire; keep it
442
+ # (d) one GetQueuedCompletionStatusEx; dispatch per entry on tuple size:
443
+ # AFD readiness completions are 2-tuples [id, ready_events]; generic
444
+ # op completions (winloop 0.2) are 4-tuples [op_id, bytes, error, tag].
445
+ @backend.wait(timeout_ms).each do |entry|
446
+ if entry.size == 4
447
+ op_id, bytes, error, _tag = entry
448
+ if (w = @op_waiters.delete(op_id))
449
+ if w.settled # its timer fired this same turn: orphan
450
+ @backend.op_free(op_id)
451
+ else
452
+ w.settled = true
453
+ @lock.synchronize { @ready << [w.fiber, [bytes, error]] }
454
+ end
455
+ elsif @op_orphans.delete(op_id) # await timed out / unwound earlier
456
+ @backend.op_free(op_id)
457
+ else # completed before anyone awaited
458
+ @op_done[op_id] = [bytes, error]
349
459
  end
460
+ else
461
+ fan_out_afd(entry[0], entry[1])
350
462
  end
351
- @lock.synchronize { @ready.concat(to_wake) } unless to_wake.empty?
352
- if entry.waiters.empty?
353
- @polls.delete(entry.key)
463
+ end
464
+ end
465
+
466
+ # Fan one AFD readiness completion out to the io waiters whose interest the
467
+ # ready mask satisfies (the pre-0.2 body of run_once step (d), unchanged).
468
+ def fan_out_afd(id, ready_events)
469
+ entry = @ops.delete(id)
470
+ return unless entry # stale: a superseded/cancelled poll
471
+ entry.id = nil
472
+ entry.armed = 0
473
+ to_wake = []
474
+ entry.waiters.reject! do |w|
475
+ if w.settled
476
+ true
477
+ elsif (got = w.events & ready_events) != 0
478
+ w.settled = true
479
+ to_wake << [w.fiber, got]
480
+ true
354
481
  else
355
- arm_poll(entry) # re-arm for the still-waiting fibers
482
+ false # this waiter's direction didn't fire; keep it
356
483
  end
357
484
  end
485
+ @lock.synchronize { @ready.concat(to_wake) } unless to_wake.empty?
486
+ if entry.waiters.empty?
487
+ @polls.delete(entry.key)
488
+ else
489
+ arm_poll(entry) # re-arm for the still-waiting fibers
490
+ end
358
491
  end
359
492
 
360
493
  # Ensure at most one AFD poll is outstanding for `entry`'s socket, covering
@@ -403,6 +536,33 @@ module Winloop
403
536
  @timers.pop while (t = @timers.peek) && t.waiter.settled
404
537
  end
405
538
 
539
+ # ---- generic-op helpers (winloop 0.2) ----------------------------------
540
+
541
+ # The op protocol is loop-thread-only (ops are submitted on the loop thread —
542
+ # fibers run there anyway; Ruby worker threads must not submit) and over once
543
+ # the loop has shut down.
544
+ def op_guard!
545
+ unless Thread.current == @thread
546
+ raise Error, "winloop: op API must be called on the scheduler's thread"
547
+ end
548
+ raise Error, "winloop: scheduler is closed" if @op_shut
549
+ end
550
+
551
+ def validate_op_timeout!(timeout)
552
+ return if timeout.nil?
553
+ return if timeout.is_a?(Numeric) && timeout.positive?
554
+ raise ArgumentError, "timeout must be nil or a positive Numeric (got #{timeout.inspect})"
555
+ end
556
+
557
+ # true iff the id is still live and in :completed state; false for retired ids
558
+ # (op_state raises "unknown op id" — swallowed here, never re-raised: this
559
+ # probe runs inside await_op's must-not-raise ensure).
560
+ def op_completed_unconsumed?(op_id)
561
+ @backend.op_state(op_id) == :completed
562
+ rescue Error
563
+ false
564
+ end
565
+
406
566
  # Enqueue a fiber to resume, honouring the one-shot guard. Caller holds @lock.
407
567
  def wake_locked(waiter, value)
408
568
  return if waiter.settled
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Winloop
4
- VERSION = "0.1.0"
4
+ VERSION = "0.2.0"
5
5
  end
data/lib/winloop.rb CHANGED
@@ -35,6 +35,7 @@ module Winloop
35
35
  end
36
36
 
37
37
  require_relative "winloop/scheduler"
38
+ require_relative "winloop/ops" # generic-op validation layer (OP_CAPACITY_MAX, op_prepare)
38
39
 
39
40
  module_function
40
41
 
metadata CHANGED
@@ -1,10 +1,10 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: winloop
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
- - Leiz
7
+ - ned
8
8
  bindir: bin
9
9
  cert_chain: []
10
10
  date: 1980-01-02 00:00:00.000000000 Z
@@ -69,16 +69,19 @@ files:
69
69
  - ext/winloop/extconf.rb
70
70
  - ext/winloop/winloop.c
71
71
  - lib/winloop.rb
72
+ - lib/winloop/ops.rb
72
73
  - lib/winloop/scheduler.rb
73
74
  - lib/winloop/version.rb
74
- homepage: https://github.com/leiz/winloop
75
+ homepage: https://github.com/main-path/winloop
75
76
  licenses:
76
77
  - MIT
77
78
  metadata:
78
- homepage_uri: https://github.com/leiz/winloop
79
- source_code_uri: https://github.com/leiz/winloop
80
- changelog_uri: https://github.com/leiz/winloop/blob/main/CHANGELOG.md
79
+ homepage_uri: https://github.com/main-path/winloop
80
+ source_code_uri: https://github.com/main-path/winloop
81
+ changelog_uri: https://github.com/main-path/winloop/blob/main/CHANGELOG.md
82
+ bug_tracker_uri: https://github.com/main-path/winloop/issues
81
83
  allowed_push_host: https://rubygems.org
84
+ rubygems_mfa_required: 'true'
82
85
  rdoc_options: []
83
86
  require_paths:
84
87
  - lib