ruby-bindgen 1.0.0

data/docs/type_spelling.md

@@ -0,0 +1,200 @@
+# Type Spelling in `ruby-bindgen`
+
+This document explains why generating correct, fully-qualified C++ type spellings from libclang is inherently complex, and how `ruby-bindgen` addresses that complexity.
+
+## TL;DR
+
+- There is no single libclang API that always returns the right C++ spelling for code generation.
+- Canonical types are often semantically correct but syntactically wrong for generated bindings.
+- `ruby-bindgen` reconstructs type spellings based on cursor kind and context, with canonicalization only as a fallback.
+
+The objective is **not** to compute a canonical or normalized type. The objective is to emit a C++ spelling that:
+
+1. Compiles correctly at the binding site
+2. Preserves user-facing API intent (typedefs, aliases, dependent names)
+3. Is stable across compilers and standard library implementations
+
+## The Problem
+
+When generating Rice bindings, `ruby-bindgen` must emit fully-qualified C++ type names that are valid in generated code.
+
+For example:
+
+```cpp
+// Required
+Constructor<cv::Mat, const cv::Range&, const cv::Range&>
+
+// Incorrect (missing namespace qualification)
+Constructor<Mat, const Range&, const Range&>
+```
+
+libclang exposes multiple APIs for retrieving type information, but **none provide a complete, correct spelling for all C++ constructs**.
+
+This is a fundamental consequence of C++’s type system and libclang’s design goals.
+
+## Why a Canonical-Based Approach Fails
+
+An initial strategy was to rely on `type.canonical.spelling` and then filter out implementation details. This approach fails in several unavoidable cases.
+
+### Failure Modes
+
+| Feature | Result from `canonical.spelling` | Why this is incorrect |
+|-------|----------------------------------|------------------------|
+| **Typedefs / aliases** | `SizeArray` → `int[3]` | Destroys alias intent and public API spelling |
+| **Templates** | `iterator` → `std::iterator` | Loses specialization and template context |
+| **Dependent types** | Missing `typename`, incorrect qualification | Canonical types erase dependency information |
+| **Namespaces** | Over- or under-qualified names | Ignores lexical context |
+
+In addition, canonical spellings frequently expose **implementation details** that must never appear in generated bindings:
+
+- `__gnu_cxx::__normal_iterator<...>` (libstdc++)
+- `_Ty`, `_Alloc`, `_Vector_iterator` (MSVC STL)
+
+Filtering these reliably is not possible without reconstructing the original spelling logic, which defeats the purpose of canonicalization.
+
+### Key Insight
+
+`canonical.spelling` answers:
+
+> “What is this type semantically?”
+
+`ruby-bindgen` must answer:
+
+> “How must this type be written so that user code compiles correctly and reflects the original API?”
+
+These questions are fundamentally different.
+
+## What libclang Provides
+
+libclang exposes several partial representations of a type, each optimized for a different purpose:
+
+| API | Returns | Limitation |
+|----|---------|------------|
+| `type.spelling` | Source-level spelling | Often unqualified |
+| `type.canonical.spelling` | Fully desugared type | Erases typedefs and context |
+| `declaration.qualified_name` | Namespace-qualified name | Drops template arguments |
+| `declaration.qualified_display_name` | Name + template parameters | May omit enclosing namespaces |
+
+No single API preserves both **spelling fidelity** and **correct qualification**.
+
+## `ruby-bindgen`’s Strategy
+
+`ruby-bindgen` does not attempt to normalize all types through a single representation.
+
+Instead, `type_spelling` reconstructs the correct spelling **based on cursor kind and context**, using canonical information only as a constrained fallback.
+
+### Design Principle
+
+> **Spelling fidelity is primary; canonicalization is secondary and opportunistic.**
+
+### Cursor-Specific Handling
+
+#### `cursor_class_template`
+
+Template definitions (e.g., `template<typename T> class Vec`).
+
+- Reconstruct template arguments explicitly
+- Use `qualify_dependent_types_in_template_args`
+- Do not consult `@type_name_map` (template parameters must remain dependent)
+
+#### `cursor_typedef_decl` inside a class template
+
+Dependent typedefs (e.g., `DataType<_Tp>::value_type`).
+
+- Emit `typename` (required by the C++ standard)
+- Combine `qualified_name` with `qualified_display_name`
+- Preserve dependency instead of resolving it
+
+#### `cursor_typedef_decl` (non-dependent)
+
+Public typedefs (e.g., `typedef Point_<int> Point2i`).
+
+- Preserve the typedef name
+- Do not desugar to the underlying type
+- Qualify template arguments via `@type_name_map`
+
+#### `cursor_type_alias_decl`
+
+C++11 `using` declarations.
+
+- Treated identically to `cursor_typedef_decl`
+- Required for cross-compiler support (MSVC favors `using`)
+
+#### `cursor_class_decl` and related types
+
+Concrete types and template instantiations.
+
+- Start from `fully_qualified_name`
+- Optionally consult `canonical.spelling`
+    - Only when it does not introduce implementation types
+- Qualify template arguments using `qualify_template_args`
+
+## The `@type_name_map`
+
+During translation unit processing, `ruby-bindgen` builds a map from simple identifiers to fully-qualified names:
+
+```ruby
+{
+  "Range" => "cv::Range",
+  "Mat"   => "cv::Mat",
+  "Pixel" => "iter::Pixel"
+}
+```
+
+This map is used to qualify **unqualified template arguments**, not to rewrite dependent names or template parameters.
+
+## Where Canonicalization Works
+
+`canonical.spelling` is useful only in limited cases:
+
+- Non-dependent `cursor_class_decl` types
+- Situations where alias preservation is irrelevant
+- As a fallback sanity check for namespace qualification
+
+It is not the primary source of truth.
+
+## Why Not Use Clang’s C++ API?
+
+Clang’s C++ API (`libTooling`) provides:
+
+- `PrintingPolicy`
+- Direct AST printers for fully-qualified names
+- Precise control over dependent type emission
+
+However:
+
+- `ruby-bindgen` uses `ffi-clang`, which exposes only libclang’s C API
+- libclang is designed for IDEs and static analysis tools, not code generation
+- Reimplementing spelling logic is unavoidable in this environment
+
+`ruby-bindgen`’s type spelling logic exists specifically to bridge this gap.
+
+## Summary
+
+There is no single libclang call that can produce correct C++ type spellings in all cases.
+
+The complexity in `ruby-bindgen` is inherent:
+
+- C++ has typedefs, aliases, templates, dependent types, and contextual name lookup
+- libclang exposes these differently depending on cursor kind
+- Correct code generation requires spelling reconstruction, not canonicalization
+
+The current design reflects these constraints deliberately.
+
+## Code Locations
+
+All type-spelling logic lives in the `RubyBindgen::Generators::Rice::TypeSpeller` class:
+
+- `lib/ruby-bindgen/generators/rice/type_speller.rb`
+  - Top-level entry points: `type_spelling`, `type_spellings`,
+    `qualified_class_name`, `qualified_display_name`
+  - Declared / unexposed type handling: `type_spelling_declared`,
+    `type_spelling_unexposed`, `type_spelling_pointer`
+  - Template handling: `qualify_template_args`,
+    `qualify_dependent_types_in_template_args`,
+    `qualify_template_parameter_packs`,
+    `qualify_class_template_typedefs`
+  - Static member qualification: `qualify_class_static_members`
+
+The `fully_qualified_name` helper used by `TypeSpeller` is provided
+directly by `FFI::Clang::Type` (ffi-clang ≥ 0.16).

data/docs/updating_bindings.md

@@ -0,0 +1,55 @@
+# Updating Bindings
+
+After generating bindings with `ruby-bindgen`, you'll need to regenerate them periodically as the upstream library evolves, `ruby-bindgen` improves, or you upgrade dependencies. This document covers strategies for managing that process.
+
+## Simple Case
+
+If you have no [manual edits or refinements](cpp/customizing.md) to preserve, you can simply regenerate the bindings. Configuration changes such as adding new skip patterns or version guards are fine — they are read from the YAML file each time.
+
+```bash
+ruby-bindgen bindings.yaml
+```
+
+In most cases, the updated bindings should compile and work.
+
+The rest of this page discusses strategies for more complex cases where you want to preserve [customizations](cpp/customizing.md) when regenerating bindings.
+
+## Preserving Refinements
+
+[Refinements](cpp/customizing.md#refinements) live in a separate directory that `ruby-bindgen` never touches, so the source files survive regeneration automatically. However, the top-level `-rb.cpp` file is regenerated, so you will need to re-add the `#include` directives and `Init_*_Refinements` calls.
+
+## Preserving Manual Edits
+
+If you have [manual edits](cpp/customizing.md#manual-edits) to generated files, you need a way to reapply them after regeneration. Two possible approaches include:
+
+### Option A: Diff File
+
+Maintain a diff file that can be reapplied after regeneration:
+
+```bash
+# After making all manual changes to generated files
+git diff -- ext/ ':(exclude)ext/manual_updates.md' > ext/manual_updates.diff
+```
+
+After regenerating:
+
+```bash
+# Regenerate
+ruby-bindgen bindings.yaml
+
+# Reapply manual changes
+cd ext/
+git apply manual_updates.diff
+```
+
+**Pros**: Simple, standard tooling, machine-readable.
+
+**Cons**: Diffs are fragile. If `ruby-bindgen`'s output changes line numbers or formatting, the diff won't apply cleanly. You'll need to manually resolve failures and regenerate the diff.
+
+### Option B: Agent File
+
+Maintain a structured document (`manual_updates.md`) that describes each change declaratively. An AI assistant or a human can follow the instructions after each regeneration. For an example, see opencv-ruby's [manual_updates.md](https://github.com/cfis/opencv-ruby/blob/main/ext/manual_updates.md).
+
+**Pros**: Survives large formatting changes, human-readable, an AI assistant can apply the instructions even when line numbers shift.
+
+**Cons**: Must be kept in sync with actual changes.

data/docs/version_guards.md

@@ -0,0 +1,69 @@
+# Version Guards
+
+Some libraries ship symbols that only exist in certain versions. For example, OpenCV 4.5 added CUDA codec APIs that don't exist in 4.1. Version guards let you generate bindings that work across multiple library versions by wrapping version-specific symbols in conditional checks.
+
+The mechanism differs by format:
+
+- **Rice (C++)** — `#if` / `#endif` preprocessor directives checked at compile time
+- **FFI (C)** — Ruby `if` conditionals checked at runtime via a user-implemented version method
+
+## Rice (C++)
+
+### Configuration
+
+Two config options work together:
+
+1. **`version_check`** — the C preprocessor macro to test (e.g., `CV_VERSION`)
+2. **`symbols.versions`** — which symbols to guard and at what version
+
+```yaml
+format: Rice
+version_check: CV_VERSION
+symbols:
+  versions:
+    40100:
+      - cv::Foo::newMethod
+    40500:
+      - /cv::cuda::.*/
+```
+
+Symbol names support the same syntax as skip symbols: simple names, fully qualified names, signatures, and regex patterns. See [Symbols](configuration.md#symbols) for details.
+
+### Generated Output
+
+#### Class methods
+
+Version-guarded methods within a class produce inline `#if` / `#endif` around the chained method definition:
+
+```cpp
+Rice::Data_Type<cv::Foo> rb_cFoo = define_class<cv::Foo>("Foo")
+  .define_method("bar", &cv::Foo::bar)
+#if CV_VERSION >= 40100
+  .define_method("new_method", &cv::Foo::newMethod)
+#endif
+  ;
+```
+
+#### Top-level functions
+
+Version-guarded free functions are wrapped at the statement level:
+
+```cpp
+#if CV_VERSION >= 40100
+define_global_function("new_func", &cv::newFunc);
+#endif
+```
+
+### How It Works
+
+When `version_check` is set, the generator looks up each symbol's version via the `symbols` config. Symbols with a version are grouped by their version value. The code generator emits `#if VERSION_MACRO >= version` before the group and `#endif` after it. Unversioned symbols are emitted normally with no guards.
+
+## FFI (C)
+
+For C libraries, version detection happens at runtime. When `symbols.versions` has entries, the generator:
+
+1. Wraps version-specific symbols in `if {version_check} >= version` conditionals
+2. Adds `require_relative '{project}_version'` to the project loader file
+3. Generates a `{project}_version.rb` skeleton (once — won't overwrite if it already exists)
+
+The user implements the `version_check` method by calling the library's own version API. See [Version Detection](c/version_guards.md) for a complete example using PROJ.

data/lib/ruby-bindgen/config.rb

@@ -0,0 +1,63 @@
+require 'yaml'
+require 'pathname'
+
+module RubyBindgen
+  class Config
+    def initialize(config_path)
+      @config_dir = File.dirname(File.expand_path(config_path))
+      raw = YAML.safe_load(File.read(config_path), permitted_classes: [], permitted_symbols: [], aliases: true)
+      # YAML.safe_load returns nil for an empty file. Treat that as an empty
+      # config so the CLI's validate_config can produce its useful "Config
+      # must specify 'output'" message instead of a NoMethodError.
+      @data = raw.is_a?(Hash) ? symbolize_keys(raw) : {}
+      resolve_toolchain
+      resolve_paths
+    end
+
+    def [](key)
+      @data[key]
+    end
+
+    # @api private
+    # Used by tests for ad-hoc overrides and by the CLI to default :input to
+    # :output. Not intended for downstream code; the public contract is read-only.
+    def []=(key, value)
+      @data[key] = value
+    end
+
+    private
+
+    def resolve_toolchain
+      toolchain = RUBY_PLATFORM =~ /mswin/ ? :'clang-cl' : :clang
+
+      toolchain_config = @data[toolchain]
+      if toolchain_config.is_a?(Hash)
+        @data[:libclang] = toolchain_config[:libclang]
+        @data[:clang_args] = toolchain_config[:args]&.map do |arg|
+          # Resolve relative -I paths relative to config directory
+          arg.start_with?('-I') ? "-I#{File.expand_path(arg[2..], @config_dir)}" : arg
+        end
+      end
+    end
+
+    def resolve_paths
+      @data[:input] = resolve_path(@data[:input]) if @data[:input]
+      @data[:output] = resolve_path(@data[:output]) if @data[:output]
+    end
+
+    def resolve_path(path)
+      return path if Pathname.new(path).absolute?
+      File.expand_path(path, @config_dir)
+    end
+
+    def symbolize_keys(hash)
+      hash.transform_keys { |k| k.is_a?(String) ? k.to_sym : k }.transform_values do |value|
+        case value
+        when Hash then symbolize_keys(value)
+        when Array then value.map { |v| v.is_a?(Hash) ? symbolize_keys(v) : v }
+        else value
+        end
+      end
+    end
+  end
+end

data/lib/ruby-bindgen/generators/cmake/cmake.rb

@@ -0,0 +1,171 @@
+module RubyBindgen
+  module Generators
+    class CMake < Generator
+      GuardedEntry = Data.define(:condition, :directories, :files)
+
+      def self.template_dir
+        __dir__
+      end
+
+      def include_dirs
+        config[:include_dirs] || []
+      end
+
+      def guards
+        @guards ||= begin
+          config_guards = config[:guards] || {}
+          config_guards.map do |condition, patterns|
+            Guard.new(condition: condition, patterns: patterns, base_path: @inputter.base_path)
+          end
+        end
+      end
+
+      def generate
+        base = Pathname.new(@inputter.base_path)
+
+        # Collect files from inputter, grouped by relative directory
+        files_by_dir = Hash.new do |hash, key|
+          hash[key] = []
+        end
+
+        @inputter.each do |path, relative_path|
+          dir = File.dirname(relative_path)
+          files_by_dir[dir] << Pathname.new(path)
+        end
+
+        # Collect all directories that need CMakeLists.txt files
+        # (directories containing files, plus all ancestor directories)
+        all_dirs = Set.new
+        files_by_dir.each_key do |dir|
+          next if dir == "."
+          parts = Pathname.new(dir).each_filename.to_a
+          parts.length.times do |i|
+            all_dirs << parts[0..i].join("/")
+          end
+        end
+
+        # Build parent -> immediate child directories mapping
+        child_dirs_of = Hash.new do |hash, key|
+          hash[key] = []
+        end
+        all_dirs.each do |dir|
+          parent = File.dirname(dir)
+          parent = "." if parent == dir
+          child_dirs_of[parent] << base.join(dir)
+        end
+        child_dirs_of.each_value(&:sort!)
+
+        file_guards, directory_guards = build_guard_maps(files_by_dir, all_dirs.to_a)
+
+        if @project
+          # Root CMakeLists.txt
+          content = render_template("project",
+                                    :project => self.project,
+                                    :directories => unguarded_paths(child_dirs_of["."], directory_guards, base),
+                                    :files => unguarded_paths(files_by_dir["."].sort, file_guards, base),
+                                    :guarded_entries => guarded_entries(child_dirs_of["."],
+                                                                        directory_guards,
+                                                                        files_by_dir["."].sort,
+                                                                        file_guards,
+                                                                        base),
+                                    :include_dirs => self.include_dirs)
+          self.outputter.write("CMakeLists.txt", content)
+
+          # Presets
+          content = render_template("presets")
+          self.outputter.write("CMakePresets.json", content)
+        end
+
+        # Subdirectory CMakeLists.txt files
+        all_dirs.sort.each do |dir|
+          content = render_template("directory",
+                                    :project => self.project,
+                                    :directories => unguarded_paths(child_dirs_of[dir], directory_guards, base),
+                                    :files => unguarded_paths((files_by_dir[dir] || []).sort, file_guards, base),
+                                    :guarded_entries => guarded_entries(child_dirs_of[dir],
+                                                                        directory_guards,
+                                                                        (files_by_dir[dir] || []).sort,
+                                                                        file_guards,
+                                                                        base))
+          self.outputter.write(File.join(dir, "CMakeLists.txt"), content)
+        end
+      end
+
+      private
+
+      def build_guard_maps(files_by_dir, all_dirs)
+        file_paths = files_by_dir.values.flatten.map do |path|
+          expand_path(path)
+        end.sort
+        directory_paths = all_dirs.map do |dir|
+          expand_path(dir, @inputter.base_path)
+        end.sort
+        file_guards = {}
+        directory_guards = {}
+
+        guards.each do |guard|
+          match = guard.match(file_paths: file_paths, directory_paths: directory_paths)
+
+          match.files.each do |path|
+            assign_guard!(file_guards, path, match.condition)
+          end
+
+          match.directories.each do |path|
+            assign_guard!(directory_guards, path, match.condition)
+          end
+        end
+
+        [file_guards, directory_guards]
+      end
+
+      def assign_guard!(assignments, path, condition)
+        previous = assignments[path]
+        if previous && previous != condition
+          raise ArgumentError, "#{path} matched multiple guard conditions: #{previous.inspect}, #{condition.inspect}"
+        end
+
+        assignments[path] = condition
+      end
+
+      def guarded_entries(directories, directory_guards, files, file_guards, base)
+        grouped = Hash.new do |hash, key|
+          hash[key] = GuardedEntry.new(condition: key, directories: [], files: [])
+        end
+
+        directories.each do |directory|
+          condition = directory_guards[expand_path(directory, base)]
+          next unless condition
+
+          grouped[condition].directories << directory
+        end
+
+        files.each do |file|
+          condition = file_guards[expand_path(file, base)]
+          next unless condition
+
+          grouped[condition].files << file
+        end
+
+        guards.map(&:condition).filter_map do |condition|
+          entry = grouped[condition]
+          next if entry.nil? || (entry.directories.empty? && entry.files.empty?)
+
+          entry
+        end
+      end
+
+      def expand_path(path, base = @inputter.base_path)
+        File.expand_path(path.to_s, base.to_s)
+      end
+
+      def unguarded_paths(paths, guard_map, base)
+        paths.reject do |path|
+          guard_map.key?(expand_path(path, base))
+        end
+      end
+
+    end
+  end
+end
+
+require_relative 'guard'

data/lib/ruby-bindgen/generators/cmake/directory.erb

@@ -0,0 +1,29 @@
+# Generated by ruby-bindgen (<%= RubyBindgen::VERSION %>)
+
+# Subdirectories
+<% directories.each do |directory| -%>
+add_subdirectory ("<%= directory.relative_path_from(directory.parent) %>")
+<% end -%>
+
+# Sources
+<% if !files.empty? -%>
+target_sources(${CMAKE_PROJECT_NAME} PRIVATE
+<% files.each do |file| -%>
+  "<%= file.relative_path_from(file.parent) %>"
+<% end -%>
+)
+<% end -%>
+<% guarded_entries.each do |entry| -%>
+if(<%= entry.condition %>)
+<% entry.directories.each do |directory| -%>
+  add_subdirectory ("<%= directory.relative_path_from(directory.parent) %>")
+<% end -%>
+<% if !entry.files.empty? -%>
+  target_sources(${CMAKE_PROJECT_NAME} PRIVATE
+<% entry.files.each do |file| -%>
+    "<%= file.relative_path_from(file.parent) %>"
+<% end -%>
+  )
+<% end -%>
+endif()
+<% end -%>

data/lib/ruby-bindgen/generators/cmake/guard.rb

@@ -0,0 +1,55 @@
+module RubyBindgen
+  module Generators
+    class CMake
+      class Guard
+        GuardMatch = Data.define(:condition, :directories, :files)
+
+        attr_reader :condition, :patterns
+
+        def initialize(condition:, patterns:, base_path:)
+          @condition = condition.to_s
+          @patterns = Array(patterns).map do |pattern|
+            expand_path(pattern.to_s, base_path)
+          end
+        end
+
+        def match(file_paths:, directory_paths:)
+          matched_files = []
+          matched_directories = []
+
+          patterns.each do |pattern|
+            files = file_paths.select do |path|
+              path_match?(pattern, path)
+            end
+            directories = directory_paths.select do |path|
+              path_match?(pattern, path)
+            end
+
+            warn_unmatched(pattern) if files.empty? && directories.empty?
+
+            matched_files.concat(files)
+            matched_directories.concat(directories)
+          end
+
+          GuardMatch.new(condition: condition,
+                         directories: matched_directories.uniq.sort,
+                         files: matched_files.uniq.sort)
+        end
+
+        private
+
+        def path_match?(pattern, path)
+          File.fnmatch?(pattern, path, File::FNM_PATHNAME)
+        end
+
+        def expand_path(path, base)
+          File.expand_path(path.to_s, base.to_s)
+        end
+
+        def warn_unmatched(pattern)
+          warn "CMake guard #{condition.inspect} did not match any generated paths for pattern #{pattern.inspect}"
+        end
+      end
+    end
+  end
+end