scint 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d1c0c2f09127da28473aad491848c5b3e7b976fa41405a94ce542859fa1aabf8
-  data.tar.gz: 2e7b1801b9a3004bc0d9709de9502bfff5a8b475178aa102ebb730a6cb1b6344
+  metadata.gz: 54c3f7baaea3992bcba60136b76f3b08caa580717d82906ebed3a0a69fb23314
+  data.tar.gz: 246966d0ba167f43b8c0aee1205e95377afec245dca33060fe3fcce202d0c3ab
 SHA512:
-  metadata.gz: 3d95e5006966b2025c5e21951bc918849081c330a2872e7e665cafa6d73ab522871f7e41aa2ed4c61bbed37e73363f19650dce16bbc1e472b3e4fe54c0177d97
-  data.tar.gz: 40fba5802be2359b85d201c2c732f8edac15f1e1be7ee17b2d59a39de9ac2f1ca6befd14d527fcacb4a5e763408c1296abe35301f8224a4b1138a5a4a1706096
+  metadata.gz: 84c58296b1c391e68503090ea3889a3a469c8f63ca6c289710dc6a5f986cd33059aec6a61038ab14c2f653aefaa1675815d2e13a5ba3fb49dad6fe825cf8d3a5
+  data.tar.gz: 1cde1ae7ff37856bf44a4f909e024cb094ba1436fe062536aa4c73860ce0aa7c2494c2f5dd09ca14af5be8b226d8ada36ec212278b08a55902da708d734bea3d

data/README.md

@@ -37,6 +37,29 @@ scint cache clear
 scint cache dir
 ```
 
+Benchmark helpers:
+
+```bash
+bin/scint-vs-bundler [--force] [--test-root /tmp/scint-tests] /path/to/project
+bin/scint-bench-matrix [--force] --root /path/to/projects
+```
+
+`bin/scint-bench-matrix` is a generic runner for a root directory where each
+immediate subdirectory is a Ruby project under git with both `Gemfile` and
+`Gemfile.lock`. It runs bundler cold/warm and scint cold/warm via
+`bin/scint-vs-bundler` and writes:
+
+1. `logs/bench-<timestamp>/summary.tsv`
+2. `logs/bench-<timestamp>/table.md`
+
+Optional project smoke test convention:
+
+1. If `<root>/<project>-test.sh` exists, matrix runs it after the benchmark.
+2. Execution is:
+   `cd <root>/<project> && scint exec ../<project>-test.sh`
+3. The script runs against the warm scint install and is included in
+   `summary.tsv`/`table.md` status.
+
 Performance and IO diagnostics:
 
 ```bash
@@ -64,45 +87,45 @@ Defaults:
 1. Local install/runtime directory: `.bundle/`
 2. Global cache root: `~/.cache/scint` (or `XDG_CACHE_HOME`)
 
-## Install Architecture
+## Install Architecture (Target)
+
+Scint should have one clear cache lifecycle:
 
-Scint install is phase-oriented. Each phase has explicit responsibilities and feeds the next phase.
+1. `inbound`
+2. `assembling`
+3. `cached`
+4. `materialize`
 
-1. Parse inputs (`Gemfile`, optional `Gemfile.lock`)
-2. Fetch source metadata (indexes, git clones)
-3. Resolve dependency graph
-4. Plan actions (`skip`, `link`, `download`, `build_ext`)
-5. Download/extract/cache artifacts
-6. Link into local `.bundle` runtime
-7. Build native extensions after link phase is ready
-8. Write outputs (`Gemfile.lock`, runtime lock, warnings/summary)
+Resolution/planning still decides *what* to install; this pipeline defines *how* each artifact becomes globally reusable.
+
+### Phase Contract
+
+1. Fetch into `inbound`
+   - Gem payloads go to `inbound/gems/`.
+   - Git repositories go to `inbound/gits/` using deterministic names (for example `https_github_com__tobi__try`).
+2. Assemble into `assembling`
+   - For `.gem` sources: unpack into `assembling/<abi>/<full_name>/`.
+   - For git sources: fetch/checkout/submodules in `inbound/gits`, then export/copy the selected tree into `assembling/<abi>/<full_name>/`.
+3. Compile in `assembling`
+   - Native extension build happens inside the assembling directory so successful outputs are part of the final cached tree.
+4. Promote atomically to `cached`
+   - On success, move `assembling/<abi>/<full_name>/` to `cached/<abi>/<full_name>/`.
+   - Write `cached/<abi>/<full_name>.spec.marshal`.
+   - Write optional manifest metadata for fast materialization.
+5. Materialize to project path (`.bundle` or `BUNDLE_PATH`)
+   - Use clonefile/reflink/hardlink/copy fallback from `cached/<abi>/...`.
+   - Do not rebuild if cached artifact is already complete.
+
+This gives one primary truth source for warm installs: `cached/<abi>`.
 
 ```mermaid
 flowchart LR
-    A[Gemfile + Gemfile.lock] --> B[Parse + Source Discovery]
-    B --> C[Fetch Indexes / Clone Git]
-    C --> D[Resolve Graph]
-    D --> E[Planner]
-    E -->|skip| F[Already Installed]
-    E -->|link| G[Link from Extracted Cache]
-    E -->|download| H[Download .gem]
-    H --> I[Extract + Cache Metadata]
-    I --> G
-    G --> J[All Links Complete]
-    J --> K[Native Extension Build]
-    K --> L[Runtime + Lockfile Write]
-    L --> M[Done]
-
-    subgraph GlobalCache["Global Cache (~/.cache/scint)"]
-      H
-      I
-    end
-
-    subgraph ProjectRuntime["Project Runtime (.bundle)"]
-      G
-      K
-      L
-    end
+    A[Resolve + Plan] --> B[Fetch to inbound]
+    B --> C[Assemble in assembling]
+    C --> D[Compile in assembling]
+    D --> E[Promote to cached]
+    E --> F[Materialize to .bundle]
+    F --> G[Write Runtime + Lockfile]
 ```
 
 ## Scheduler as Session Object
@@ -154,24 +177,50 @@ stateDiagram-v2
     failed --> [*]
 ```
 
-## Data Layout
+## Data Layout (Target)
 
 Global cache (`~/.cache/scint`):
 
-1. `inbound/` downloaded gem files
-2. `extracted/` unpacked gem trees
-3. `ext/` compiled extension cache keyed by ABI
-4. `index/` source metadata/index cache
-5. `git/` cached git repositories
+```text
+~/.cache/scint/
+  inbound/
+    gems/
+      <full_name>.gem
+    gits/
+      <deterministic_repo_slug>/
+  assembling/
+    <ruby-abi>/
+      <full_name>/
+  cached/
+    <ruby-abi>/
+      <full_name>/
+      <full_name>.spec.marshal
+      <full_name>.manifest
+  index/
+```
+
+Example ABI key and gem directory:
+
+1. `cached/ruby-3.4.5-arm64-darwin24/zlib-3.2.1/`
+2. `cached/ruby-3.4.5-arm64-darwin24/zlib-3.2.1.spec.marshal`
 
 Project-local runtime (`.bundle/`):
 
-1. `ruby/<major.minor.0>/gems/` linked gem trees
+1. `ruby/<major.minor.0>/gems/` materialized gem trees
 2. `ruby/<major.minor.0>/specifications/` gemspecs
 3. `ruby/<major.minor.0>/bin/` gem binstubs
 4. `bin/` project-level wrappers
 5. `scint.lock.marshal` runtime lock for `scint exec`
 
+## Warm Path Guarantees
+
+Required behavior:
+
+1. If `cached/<abi>/<full_name>/` exists and is valid, no fetch/extract/compile occurs for that gem.
+2. Deleting only `.bundle/` should trigger only materialization work.
+3. Materialization should be IO-bound and close to instantaneous on warm cache.
+4. Incomplete assemblies must never be promoted; promotion is atomic.
+
 ## Concurrency Model
 
 Scint parallelizes all non-conflicting work aggressively:

data/VERSION

@@ -1 +1 @@
-0.6.0
+0.7.0

data/bin/scint

@@ -1,6 +1,15 @@
 #!/usr/bin/env ruby
 # frozen_string_literal: true
 
+# Enable YJIT as early as possible when available.
+if defined?(RubyVM::YJIT) && RubyVM::YJIT.respond_to?(:enable)
+  begin
+    RubyVM::YJIT.enable unless RubyVM::YJIT.enabled?
+  rescue StandardError
+    # Keep startup resilient when YJIT is unavailable or disabled at runtime.
+  end
+end
+
 # Ensure lib is on the load path
 $LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
 

data/lib/bundler.rb

@@ -100,10 +100,14 @@ module Bundler
           Kernel.require(candidate)
           return true
         rescue LoadError => e
+          raise e unless retryable_load_error?(e, candidate)
+
           last_error = e
         end
       end
 
+      return true if require_matching_basename(name)
+
       raise last_error if last_error
     end
 
@@ -164,5 +168,107 @@ module Bundler
       end
       filtered.join(" ")
     end
+
+    def require_matching_basename(name)
+      target = normalize_basename(name)
+      return false if target.empty?
+
+      exact_candidates = []
+      fuzzy_candidates = []
+
+      $LOAD_PATH.each do |load_dir|
+        next unless File.directory?(load_dir)
+
+        Dir.children(load_dir).sort.each do |entry|
+          next unless entry.end_with?(".rb")
+
+          basename = entry.delete_suffix(".rb")
+          normalized = normalize_basename(basename)
+          if normalized == target
+            exact_candidates << basename
+          elsif compatible_basename?(target, normalized)
+            fuzzy_candidates << [basename, normalized]
+          end
+        end
+      rescue StandardError
+        next
+      end
+
+      exact_candidates.uniq.each do |candidate|
+        begin
+          Kernel.require(candidate)
+          return true
+        rescue LoadError => e
+          raise e unless retryable_load_error?(e, candidate)
+
+          next
+        end
+      end
+
+      fuzzy_candidates
+        .uniq
+        .sort_by { |_basename, normalized| [((target.length - normalized.length).abs), -normalized.length] }
+        .map(&:first)
+        .each do |candidate|
+          begin
+            Kernel.require(candidate)
+            return true
+          rescue LoadError => e
+            raise e unless retryable_load_error?(e, candidate)
+
+            next
+          end
+        end
+
+      false
+    end
+
+    def normalize_basename(name)
+      name.to_s.downcase.gsub(/[^a-z0-9]/, "")
+    end
+
+    def retryable_load_error?(error, candidate)
+      requested = if error.respond_to?(:path)
+                    error.path
+                  else
+                    nil
+                  end
+      return true if requested.nil? || requested.empty?
+
+      variants = [
+        candidate.to_s,
+        "#{candidate}.rb",
+        "#{candidate}.so",
+      ]
+      variants.include?(requested)
+    end
+
+    def compatible_basename?(target, normalized)
+      return false if target.length < 3 || normalized.length < 3
+
+      target_variants = basename_variants(target)
+      normalized_variants = basename_variants(normalized)
+      return true if (target_variants & normalized_variants).any?
+
+      target_variants.any? do |t|
+        normalized_variants.any? { |n| t.start_with?(n) || n.start_with?(t) }
+      end
+    end
+
+    def basename_variants(name)
+      value = name.to_s.downcase
+      variants = [value]
+
+      plural_to_s = value.sub(/ties\z/, "s")
+      plural_to_y = value.sub(/ies\z/, "y")
+      singular = value.sub(/s\z/, "")
+      strip_ruby_prefix = value.sub(/\Aruby/, "")
+
+      variants << plural_to_s unless plural_to_s == value
+      variants << plural_to_y unless plural_to_y == value
+      variants << singular unless singular == value
+      variants << strip_ruby_prefix unless strip_ruby_prefix == value
+      variants.uniq.select { |v| v.length >= 3 }
+    end
   end
 end

data/lib/scint/cache/layout.rb

@@ -2,6 +2,7 @@
 
 require_relative "../fs"
 require_relative "../platform"
+require_relative "../spec_utils"
 require "digest"
 require "uri"
 
@@ -35,7 +36,7 @@ module Scint
       end
 
       def git_dir
-        File.join(@root, "git")
+        File.join(inbound_dir, "git")
       end
 
       # Isolated gem home used while compiling native extensions during install.
@@ -45,7 +46,7 @@ module Scint
       end
 
       def install_ruby_dir
-        File.join(install_env_dir, "ruby", RUBY_VERSION.split(".")[0, 2].join(".") + ".0")
+        Platform.ruby_install_dir(install_env_dir)
       end
 
       # -- Per-spec paths ------------------------------------------------------
@@ -78,23 +79,20 @@ module Scint
       end
 
       def git_path(uri)
-        slug = Digest::SHA256.hexdigest(uri.to_s)[0, 16]
-        File.join(git_dir, slug)
+        slug = git_slug(uri)
+        File.join(git_dir, "repos", slug)
+      end
+
+      def git_checkout_path(uri, revision)
+        slug = git_slug(uri)
+        rev = revision.to_s.gsub(/[^0-9A-Za-z._-]/, "_")
+        File.join(git_dir, "checkouts", slug, rev)
       end
 
       # -- Helpers -------------------------------------------------------------
 
       def full_name(spec)
-        name = spec.respond_to?(:name) ? spec.name : spec[:name]
-        version = spec.respond_to?(:version) ? spec.version : spec[:version]
-        platform = spec.respond_to?(:platform) ? spec.platform : spec[:platform]
-
-        base = "#{name}-#{version}"
-        if platform && platform.to_s != "ruby" && platform.to_s != ""
-          "#{base}-#{platform}"
-        else
-          base
-        end
+        SpecUtils.full_name(spec)
       end
 
       # Ensure a directory exists (thread-safe, cached).
@@ -126,6 +124,10 @@ module Scint
           Digest::SHA256.hexdigest(str)[0, 16]
         end
       end
+
+      def git_slug(uri)
+        Digest::SHA256.hexdigest(uri.to_s)[0, 16]
+      end
     end
   end
 end

data/lib/scint/cache/metadata_store.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require_relative "../fs"
+require_relative "../spec_utils"
 
 module Scint
   module Cache
@@ -30,7 +31,7 @@ module Scint
       # Check if a gem is installed. specs_hash keys are "name-version" or "name-version-platform".
       def installed?(name, version, platform = "ruby")
         data = load
-        key = cache_key(name, version, platform)
+        key = SpecUtils.full_name_for(name, version, platform)
         data.key?(key)
       end
 
@@ -38,7 +39,7 @@ module Scint
       def add(name, version, platform = "ruby")
         @mutex.synchronize do
           @data ||= load_from_disk
-          key = cache_key(name, version, platform)
+          key = SpecUtils.full_name_for(name, version, platform)
           @data[key] = true
           FS.atomic_write(@path, Marshal.dump(@data))
         end
@@ -48,7 +49,7 @@ module Scint
       def remove(name, version, platform = "ruby")
         @mutex.synchronize do
           @data ||= load_from_disk
-          key = cache_key(name, version, platform)
+          key = SpecUtils.full_name_for(name, version, platform)
           @data.delete(key)
           FS.atomic_write(@path, Marshal.dump(@data))
         end
@@ -56,14 +57,6 @@ module Scint
 
       private
 
-      def cache_key(name, version, platform)
-        if platform && platform.to_s != "ruby" && platform.to_s != ""
-          "#{name}-#{version}-#{platform}"
-        else
-          "#{name}-#{version}"
-        end
-      end
-
       def load_from_disk
         return {} unless File.exist?(@path)
         Marshal.load(File.binread(@path))

data/lib/scint/cli/cache.rb

@@ -9,6 +9,7 @@ require_relative "../gemfile/parser"
 require_relative "../lockfile/parser"
 require_relative "install"
 require_relative "../fs"
+require_relative "../spec_utils"
 
 module Scint
   module CLI
@@ -306,7 +307,7 @@ module Scint
       def dedupe_specs(specs)
         seen = {}
         specs.each do |spec|
-          key = "#{spec.name}-#{spec.version}-#{spec.platform}"
+          key = SpecUtils.full_key(spec)
           seen[key] ||= spec
         end
         seen.values

data/lib/scint/cli/exec.rb

@@ -3,7 +3,9 @@
 require_relative "../runtime/exec"
 require_relative "../fs"
 require_relative "../platform"
+require_relative "../spec_utils"
 require_relative "../lockfile/parser"
+require "pathname"
 
 module Scint
   module CLI
@@ -68,19 +70,18 @@ module Scint
         data = {}
 
         lockfile.specs.each do |spec|
-          full = spec_full_name(spec)
+          full = SpecUtils.full_name(spec)
           gem_dir = File.join(ruby_dir, "gems", full)
           next unless Dir.exist?(gem_dir)
 
           spec_file = File.join(ruby_dir, "specifications", "#{full}.gemspec")
           require_paths = read_require_paths(spec_file)
           load_paths = require_paths
-            .map { |rp| File.join(gem_dir, rp) }
+            .map { |rp| expand_require_path(gem_dir, rp) }
             .select { |path| Dir.exist?(path) }
 
           lib_path = File.join(gem_dir, "lib")
           load_paths << lib_path if load_paths.empty? && Dir.exist?(lib_path)
-          load_paths.concat(detect_nested_lib_paths(gem_dir))
           load_paths.uniq!
 
           ext_path = File.join(ruby_dir, "extensions",
@@ -102,7 +103,7 @@ module Scint
       end
 
       def detect_ruby_dir(bundle_dir)
-        target = RUBY_VERSION.split(".")[0, 2].join(".") + ".0"
+        target = Platform.ruby_minor_version_dir
         preferred = File.join(bundle_dir, "ruby", target)
         return preferred if Dir.exist?(preferred)
 
@@ -111,13 +112,7 @@ module Scint
       end
 
       def spec_full_name(spec)
-        name = spec[:name]
-        version = spec[:version]
-        platform = spec[:platform]
-        base = "#{name}-#{version}"
-        return base if platform.nil? || platform.to_s == "ruby" || platform.to_s.empty?
-
-        "#{base}-#{platform}"
+        SpecUtils.full_name(spec)
       end
 
       def read_require_paths(spec_file)
@@ -130,20 +125,13 @@ module Scint
         ["lib"]
       end
 
-      def detect_nested_lib_paths(gem_dir)
-        lib_dir = File.join(gem_dir, "lib")
-        return [] unless Dir.exist?(lib_dir)
-
-        children = Dir.children(lib_dir)
-        top_level_rb = children.any? do |entry|
-          path = File.join(lib_dir, entry)
-          File.file?(path) && entry.end_with?(".rb")
-        end
-        return [] if top_level_rb
+      def expand_require_path(gem_dir, require_path)
+        value = require_path.to_s
+        return value if Pathname.new(value).absolute?
 
-        children
-          .map { |entry| File.join(lib_dir, entry) }
-          .select { |path| File.directory?(path) }
+        File.join(gem_dir, value)
+      rescue StandardError
+        File.join(gem_dir, require_path.to_s)
       end
     end
   end