duckling 0.4.7-aarch64-linux

data/Rakefile

@@ -0,0 +1,243 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "dotenv"
+require "minitest/test_task"
+require "rb_sys/extensiontask"
+
+require_relative "cross_targets"
+
+# Loads RB_SYS_CARGO_PROFILE=dev from .env.local when present (seeded by
+# bin/setup from .env.local.example), so local compiles default to the dev
+# profile without needing the :dev task below. .env.local is gitignored and
+# never checked out in CI, so `bundle exec rake` there still builds release.
+Dotenv.load(".env.local")
+
+GEMSPEC = Gem::Specification.load("duckling.gemspec")
+
+RbSys::ExtensionTask.new("duckling", GEMSPEC) do |ext|
+  ext.lib_dir = "lib/duckling"
+
+  # rake-compiler derives a native gem's required_ruby_version from the ABIs
+  # it cross-compiled against, and writes its own floor over the gemspec's.
+  # The gemspec's floor can be stricter, so state both bounds here: the
+  # gemspec's own requirement, plus the ceiling from cross_targets.rb.
+  ext.cross_compiling do |spec|
+    spec.required_ruby_version =
+      GEMSPEC.required_ruby_version.as_list + ["< #{CrossTargets::ABI_CEILING}"]
+  end
+end
+
+# rake-compiler always builds two things for this extension:
+# - The real cross build for RUBY_TARGET.
+# - An extra "local" build for the host Ruby. rake-compiler runs this
+#   local build even when cross_compile is off.
+#
+# Gem::PackageTask lists the local build's plain output path as a
+# prerequisite of the final .gem file.
+#
+# Each rbsys/<platform> Docker image sets RUST_TARGET and
+# CARGO_BUILD_TARGET in its environment. These variables make a plain
+# `cargo build` target that image's platform. Every process in the
+# container inherits them, including the local build.
+#
+# The Ruby that drives rake in each container is an x86_64 Linux Ruby.
+# So the local build always makes a file named duckling.so.
+#
+# On x86_64-linux, this causes no problem. The local build's target is
+# already correct.
+#
+# On x86_64-darwin and arm64-darwin, this causes no problem either.
+# Those gems need duckling.bundle. They do not list duckling.so, so the
+# local build is not a prerequisite and never runs.
+#
+# On aarch64-linux, the two names collide. The gem needs duckling.so,
+# which is the name the local build makes. So the packaging step waits
+# for the local build. That build compiles code for aarch64, but links
+# the code with the host's plain gcc. rake-compiler picked this gcc for
+# the host platform. The link step then fails, and stops the build.
+#
+# extconf.rb runs in its own subprocess. ENV changes made there do not
+# reach the parent process, and the parent process runs `make`.
+#
+# So this fix changes ENV here, in the Rakefile.
+# The fix adds a prerequisite task to the local build's Makefile task.
+# This prerequisite task sets the correct host target. It runs before
+# the Makefile task, and before `make` runs later.
+if (ruby_target = ENV["RUBY_TARGET"]) && ruby_target != RUBY_PLATFORM
+  local_makefile = "tmp/#{RUBY_PLATFORM}/duckling/#{RUBY_VERSION}/Makefile"
+
+  # CARGO_BUILD_TARGET must hold a real triple before RUST_TARGET goes
+  # away. Clearing RUST_TARGET alone lets rb_sys fall back to the target
+  # baked into the container's $CARGO_HOME/config.toml, which is the
+  # cross-compile target again. So a host triple that cannot be read is a
+  # hard error.
+  #
+  # Both variables belong to the whole rake process, and the cross build
+  # reads them too. This is safe only because rake generates the cross
+  # Makefile, which bakes its own --target in, before packaging reaches
+  # the local pass. Nothing in rake states that order. If it ever
+  # inverted, the cross build would compile for the host and produce a
+  # correctly *named* binary for the wrong architecture — which is what
+  # `file(1)` on every binary in test/gem/packaged_gem_test.rb catches.
+  task :fix_local_pass_cargo_target do
+    rustc_version_info = begin
+      `rustc -vV`
+    rescue Errno::ENOENT
+      raise "Cannot run `rustc -vV` to find the host target triple. rustc must be on PATH."
+    end
+
+    host_target = rustc_version_info[/^host: (\S+)$/, 1]
+    raise "`rustc -vV` printed no `host:` line:\n#{rustc_version_info}" unless host_target
+
+    ENV["CARGO_BUILD_TARGET"] = host_target
+    ENV.delete("RUST_TARGET")
+  end
+
+  # local_makefile reconstructs a path rake-compiler builds from its own
+  # internals (tmp_dir, extension name, the local pass's Ruby version).
+  # A gem upgrade can change any of them. Say so here, because the
+  # alternative is a silent no-op and a link failure deep in a container.
+  unless Rake::Task.task_defined?(local_makefile)
+    raise "Expected rake-compiler to define a Makefile task at #{local_makefile}. " \
+      "The local-pass Cargo target override needs that exact task name — check " \
+      "define_compile_tasks in rake-compiler's extensiontask.rb for the current path."
+  end
+
+  Rake::Task[local_makefile].enhance([:fix_local_pass_cargo_target])
+end
+
+task :dev do
+  ENV["RB_SYS_CARGO_PROFILE"] = "dev"
+end
+
+# rb-sys-dock mounts the working directory into the build container with
+# `-v $(pwd):$(pwd)` and nothing else. In a git worktree — which is how
+# bin/worktree sets up every branch here — .git is a *file* naming a path under
+# the main checkout's .git/worktrees/, which that mount does not cover. Every
+# git command inside the container then fails, including the `git ls-files` in
+# duckling.gemspec, whose file list comes back empty. The gem that comes out
+# holds the compiled binaries and no Ruby at all.
+#
+# So build from a throwaway plain clone instead, whose .git is a real
+# directory under the mount. Two consequences: the gem carries *committed*
+# state (uncommitted changes are excluded), and the clone gets its own Cargo
+# target directory separate from this checkout's.
+CLONE_DIR = "tmp/native_gem_clone"
+
+def build_native_gem(platform)
+  # bundler exports BUNDLE_GEMFILE pointing at this checkout, and rb-sys-dock
+  # mounts $(pwd) — the two have to move together or the container gets one
+  # directory's Gemfile and another's source.
+  Bundler.with_unbundled_env do
+    sh "bundle", "install"
+    sh "bundle", "exec", "rb-sys-dock", "--platform", platform,
+      "--ruby-versions", CrossTargets::RUBY_ABIS.join(","), "--build"
+  end
+end
+
+desc "Cross-compile the native extension for a given platform via rb-sys-dock (e.g. `rake 'native_gem[x86_64-linux]'`)"
+task :native_gem, [:platform] do |_t, platform:|
+  next build_native_gem(platform) unless File.file?(".git")
+
+  head = `git rev-parse HEAD`.strip
+  raise "Could not read HEAD to pin the build clone." if head.empty?
+
+  unless `git status --porcelain`.empty?
+    warn "native_gem: building #{head[0, 7]} from a clone — uncommitted changes are not in this gem."
+  end
+
+  rm_rf CLONE_DIR
+  begin
+    sh "git", "clone", "--local", "--no-checkout", Dir.pwd, CLONE_DIR
+    sh "git", "-C", CLONE_DIR, "checkout", "--detach", head
+
+    Dir.chdir(CLONE_DIR) { build_native_gem(platform) }
+
+    mkdir_p "pkg"
+    cp FileList["#{CLONE_DIR}/pkg/*.gem"], "pkg"
+  ensure
+    rm_rf CLONE_DIR
+  end
+end
+
+task :benchmark_env do
+  # Force a realistic release-profile build regardless of .env.local's
+  # RB_SYS_CARGO_PROFILE=dev (local dev checkouts only; CI never has it).
+  # Must reenable :compile in case it already ran earlier in this same
+  # rake process, so it recompiles under the forced profile. A stale
+  # dev-profile build would be reused otherwise.
+  ENV.delete("RB_SYS_CARGO_PROFILE")
+  Rake::Task[:compile].reenable
+end
+
+desc "Run the benchmark-ips suite (console output only, no file writes)"
+task benchmark: [:benchmark_env, :compile] do
+  ruby "-Ilib", "benchmark/parse_benchmark.rb"
+end
+
+namespace :benchmark do
+  desc "Run benchmarks, write docs/benchmarks/<environment>/<version>.json, regenerate docs/benchmarks/README.md"
+  task record: [:benchmark_env, :compile] do
+    ruby "-Ilib", "benchmark/report.rb"
+  end
+
+  desc "Run :record on a fresh branch off origin/main, then commit/push and open+auto-merge a PR via gh"
+  task record_pr: ["release:guard_clean"] do
+    # Explicit bash: Rake's default `sh -c` is dash on Debian/Ubuntu
+    # runners, and dash's `set` doesn't support the `-o pipefail` flag below.
+    sh("bash", "-c", <<~SH)
+      set -euo pipefail
+      original_ref="$(git symbolic-ref -q --short HEAD || git rev-parse HEAD)"
+      git fetch origin main
+      git checkout -b "benchmark/pending-$(date +%s)" origin/main
+
+      bundle exec rake benchmark:record
+
+      version="$(ruby -Ilib -e 'require "duckling"; puts Duckling::VERSION')"
+      environment="$(ruby -Ilib -e 'require_relative "benchmark/report"; puts DucklingBenchmark::Report::ENVIRONMENT')"
+      branch="benchmark/${environment}/${version}-$(date +%s)"
+      git branch -m "$branch"
+
+      git add docs/benchmarks
+      git commit -m "Record ${environment} benchmark results for ${version}"
+      git push origin "$branch"
+      gh pr create --base main --head "$branch" \\
+        --title "Benchmark results (${environment}, ${version})" \\
+        --body "Automated benchmark recording from ${environment}."
+      gh pr merge "$branch" --auto --squash
+
+      git checkout "$original_ref"
+      git branch -D "$branch"
+    SH
+  end
+end
+
+# The default suite exercises the extension compiled in this checkout.
+# Three test subtrees run outside it:
+# - test/gem/ exercises a *built* or *installed* gem instead — it needs one
+#   handed to it, and the installed suite must not see this checkout's lib/.
+# - test/capabilities/ is loaded by test_helper itself, gated on the tz probes.
+# - test/environments/ holds contracts invoked directly by their CI step.
+# See docs/tz-database-axis.md.
+Minitest::TestTask.create do |t|
+  t.test_globs = FileList["test/**/*_test.rb"].exclude("test/gem/**/*", "test/capabilities/**/*", "test/environments/**/*")
+end
+
+# Minitest::TestTask has no built-in way to declare a task dependency, and
+# `task default: %i[standard compile test]`'s array ordering only protects
+# `bundle exec rake` itself — `bundle exec rake test` run directly has no
+# guarantee `compile` ran first, which would surface as a confusing
+# LoadError/stale-behavior failure unrelated to the code under test.
+task test: %i[compile]
+
+require "standard/rake"
+
+task default: %i[standard compile test]
+
+# bundler/gem_tasks's default `release` task builds and pushes the .gem
+# itself, which would race the tag-triggered CI pipeline in
+# .github/workflows/release.yml that already does the actual build and
+# publish once a vX.Y.Z tag lands. Narrow `release` to just tagging.
+Rake::Task["release"].clear
+task release: ["release:guard_clean", "release:source_control_push"]

data/docs/2026-07-01-roadmap.md

@@ -0,0 +1,106 @@
+# Roadmap — Issue #1: Ship duckling gem (time extraction via Magnus + wafer-inc-duckling)
+
+**0.2.0 has shipped.** The original three-plan implementation sequence (native
+extension → Ruby API → test suite/CI) fully executed — see
+[the wiki's stale plans](https://github.com/cpb/duckling/wiki/plans-stale) for
+that historical record. All research and design-exploration docs that grounded these
+decisions have moved to [the project wiki](https://github.com/cpb/duckling/wiki); this
+file is the only planning document that stays in the repo.
+
+This document is the live plan: **what's left, tracked as GitHub issues**, not prose
+duplicated here. Nothing below needs to happen for the gem to work today — these are
+0.2.x-and-beyond follow-ups raised during PR #3 review, while re-verifying the stale
+plans against what actually shipped, and during issue #57's research into the
+async-reactor-blocking fix.
+
+If you're new here, read this file top-to-bottom once; after that, treat it as a
+living index and jump straight to a section via the table of contents below.
+
+## Table of contents
+
+| Section | Covers |
+|---|---|
+| [Environment & tooling alignment](#environment--tooling-alignment) | Keeping local dev, Claude Code Web, and CI on the same Ruby/Rust versions |
+| [Developer workflow (Rakefile)](#developer-workflow-rakefile) | Rake task dependency ordering |
+| [API design exploration (post-0.2.0 direction)](#api-design-exploration-post-020-direction) | Whether 0.2.0's manual Magnus/Hash API is the final shape |
+| [Test coverage](#test-coverage) | Extending and auditing the test corpus |
+| [Performance & concurrency](#performance--concurrency) | Benchmarking, and unblocking `Duckling.parse` under Fiber-scheduled reactors |
+| [Settled Decisions (0.2.0, verified shipped)](#settled-decisions-020-verified-shipped) | Choices already made and shipped — no action needed |
+| [Further reading](#further-reading) | Where the research that grounded all of the above lives |
+
+## Environment & tooling alignment
+
+Keep local dev, Claude Code Web, and CI targeting the same Ruby/Rust versions so nothing
+needs manual installation in any of them.
+
+- ~~[#26](https://github.com/cpb/duckling/issues/26)~~ `bin/setup` Brewfile step for Rust — **shipped**
+- ~~[#27](https://github.com/cpb/duckling/issues/27)~~ `bin/claude-code-web-setup` just-in-time deps — **shipped**
+- ~~[#28](https://github.com/cpb/duckling/issues/28)~~ Pin CI's Rust toolchain to Claude Code Web's version — **shipped**
+
+| Issue | What |
+|-------|------|
+| [#29](https://github.com/cpb/duckling/issues/29) | Expand the CI matrix to Ruby 3.4 / latest Ruby / latest Rust (forward-compat signal) |
+| [#43](https://github.com/cpb/duckling/issues/43) | Publish precompiled binary gems for `x86_64-darwin-24` and `x86_64-linux` |
+
+## Developer workflow (Rakefile)
+
+- ~~[#30](https://github.com/cpb/duckling/issues/30)~~ `:dev` Rake task (`RB_SYS_CARGO_PROFILE=dev`) — **shipped**
+
+| Issue | What |
+|-------|------|
+| [#31](https://github.com/cpb/duckling/issues/31) | Explore making `test` explicitly depend on `:compile`, instead of relying on `default` task array ordering |
+
+## API design exploration (post-0.2.0 direction)
+
+The 0.2.0 API (manual Magnus hash mapping, matching pyduckling's Hash-based shape) is
+not necessarily the final shape — see "Option D" in
+[research-type-mapping-strategy-serialization-options](https://github.com/cpb/duckling/wiki/research-type-mapping-strategy-serialization-options)
+on the wiki.
+
+| Issue | What |
+|-------|------|
+| [#32](https://github.com/cpb/duckling/issues/32) | Explore serde_magnus (symbol keys) + Ruby pattern-matching `Data` factories as a Hash-free API |
+| [#33](https://github.com/cpb/duckling/issues/33) | v0.3.0: handle Naive time values the Rails ActiveSupport way (resolve against reference zone) |
+| [#45](https://github.com/cpb/duckling/issues/45) | `reference_time:` — accept a Ruby `Time` object to preserve UTC offset |
+| [#46](https://github.com/cpb/duckling/issues/46) | Implement the remaining 13 `DimensionValue` variants beyond `Time` |
+| [#47](https://github.com/cpb/duckling/issues/47) | Explore upstreaming serde container attributes to wafer-inc/duckling |
+
+## Test coverage
+
+| Issue | What |
+|-------|------|
+| [#34](https://github.com/cpb/duckling/issues/34) | Implement the extended test corpus designed in [research-test-coverage-ruby-test-design](https://github.com/cpb/duckling/wiki/research-test-coverage-ruby-test-design) on the wiki (`test-first`) |
+| [#35](https://github.com/cpb/duckling/issues/35) | Audit wafer-inc-duckling's test coverage against pyduckling and upstream Haskell duckling |
+
+## Performance & concurrency
+
+- ~~[#36](https://github.com/cpb/duckling/issues/36)~~ `benchmark-ips` suite with automated README/CHANGELOG reporting — **shipped** (`benchmark/parse_benchmark.rb`, `bin/benchmark record-pr`; see `docs/benchmarks/README.md`)
+
+| Issue | What |
+|-------|------|
+| [#38](https://github.com/cpb/duckling/issues/38) | Test-drive the Falcon Fiber-blocking claim in [research-ffi-risks](https://github.com/cpb/duckling/wiki/research-ffi-risks) on the wiki (`test-first`) — hill test drafted in [PR #50](https://github.com/cpb/duckling/pull/50), pending human review |
+| [#64](https://github.com/cpb/duckling/issues/64) | Implement thread-per-call GVL release (`rb_thread_call_without_gvl` + a spawned background `Thread`) to make `test/falcon_fiber_blocking_test.rb` pass once #38's hill lands — see [research-async-reactor-blocking](https://github.com/cpb/duckling/wiki/research-async-reactor-blocking) on the wiki for why a bare GVL release isn't enough. Includes splitting `Duckling::Native.parse` out from `Duckling.parse` as a benchmarking seam, and a with-thread-vs-without-thread benchmark comparison. |
+
+**Deferred follow-ups from #57's research** (explicitly out of scope for #64 itself — revisit only if their trigger condition below actually occurs):
+
+- **Persistent worker-pool dispatch** instead of thread-per-call — only worth it if a production workload dominated by many back-to-back `empty`/`no_match`-shaped calls makes the ~70µs/call thread-spawn floor a measurable fraction of total latency. See [research-concurrency-alternatives-comparison-worker-pool](https://github.com/cpb/duckling/wiki/research-concurrency-alternatives-comparison-worker-pool) — a single-worker pool would serialize concurrent callers despite the wrapped crate supporting true concurrency, so this is a real regression risk, not a free win; a *bounded* pool (N > 1) was flagged as uninvestigated if this is ever picked up.
+- **Conditional thread-spawn based on `Fiber.current_scheduler`** (skip the `Thread.new` when no reactor is present, e.g. a plain script or non-Fiber-scheduled Puma worker) — the detection signal was analyzed as sound (correctly distinguishes "no reactor" from "reactor present but current Fiber is blocking," both cases where spawning a thread buys nothing) but was never exercised in a test. Deferred because #64 (and #57 more broadly) has no throughput-optimization goal — adding a conditional branch here is itself a latency optimization for the non-reactor case, the same category of complexity rejected for the worker-pool alternative. Revisit if the ~70µs/call floor is ever shown to matter for a non-reactor caller in practice.
+- **Ruby version floor** (`>= 3.2.0` today) — [#77](https://github.com/cpb/duckling/issues/77) spiked whether calling `rb_nogvl` directly with `RB_NOGVL_OFFLOAD_SAFE` (instead of `rb_thread_call_without_gvl`, which #64 uses and which always passes `flags: 0`) lets `Fiber::Scheduler#blocking_operation_wait` auto-offload `duckling::parse`, obviating #64's Thread wrapper on that floor. **Verdict: CONFIRMED** on Ruby 3.4+ — see the wiki's `research-rb-nogvl-offload-safe-spike` for the full methodology and data. Important caveat found along the way: `io-event`'s automatic `WorkerPool` support (what lets the real `async`/Falcon gem exercise this hook without a hand-rolled scheduler) requires `rb_fiber_scheduler_blocking_operation_extract`, which only ships in Ruby 4.0+ — so `async`-based callers won't see any benefit until they're on Ruby 4.0, even though the underlying mechanism works on 3.4. Not proposing a floor bump for #64's sake either way; a follow-up implementation issue to actually add the 3.4+-gated dispatch path is recommended but was deferred to #77's PR review rather than opened by the spike itself.
+
+## Settled Decisions (0.2.0, verified shipped)
+
+- **[duckling](https://github.com/wafer-inc/duckling) on crates.io** — Published as `duckling = "0.4"`.
+- **Symbol keys and Symbol values throughout** — `:body`, `:dim`, `:value`, `:type`, `:grain`, etc. Settled by the hill tests in PR #2, confirmed shipped in `test/duckling_test.rb` on `main`.
+- **NaiveDateTime → bare ISO8601 (no offset)** — Option N1. Shipped as-is; Option N2 (ActiveSupport-style zone resolution) is tracked for 0.3.0 as [#33](https://github.com/cpb/duckling/issues/33).
+- **Manual Magnus mapping, not serde_magnus** — shipped as Option B; `magnus = "0.8"` (not `"0.9"`, which was never published to crates.io).
+- **Source gem, not pre-compiled binaries** — shipped this way for 0.2.0; [#43](https://github.com/cpb/duckling/issues/43) tracks adding pre-compiled binary gems.
+
+## Further reading
+
+The full research that grounded these decisions — wafer-inc-duckling's API surface,
+Magnus/rb-sys build wiring, type-mapping strategy options, test corpus design,
+empirical FFI risk analysis (GVL blocking, panic safety, GC pressure, day-of-week
+validation gap), and issue #57's async-reactor-blocking investigation (GVL-release
+mechanics, the wrapped crate's thread/panic-safety, dispatch-strategy comparison, and
+the empirical spike proving GVL-release-alone doesn't work) — lives on
+[the project wiki](https://github.com/cpb/duckling/wiki), not in this repo.