logica_compiler 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: a21c97a7ed5735e73959fa9a4e6e04d3a517c76f65c2d183d76f4b8aa3d02c57
+  data.tar.gz: eec80700ad9eac6739a4ee2a3789cb014d88492aa3c9bfefd2a887239ca87a4c
+SHA512:
+  metadata.gz: 4c3ea01fcf9492835cd2cea7fa91cf69a0844a38079e8b624f2ea97a932cca6f6587fd3dfe7f276f5562a83bb4dcdc4f748a40dd74ac350a8ec690f787731ba0
+  data.tar.gz: 725002578ad009f78b8b20b7eae0208efec117aa449972b1cba226c4baa9cfac94d9c03c0d8d47c197a2f2a578bd6a5e80c8a966bfedf1319a2dafad948d5752

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 jasl
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,138 @@
+# logica_compiler
+
+`logica_compiler` integrates [Logica](https://github.com/evgskv/logica) as an **offline SQL compiler**:
+
+- You author Logica programs (`.l`)
+- During build/dev, they are compiled into **digested** `.sql` files + a `manifest.json`
+- At runtime, your Rails app **only reads the manifest + SQL** and executes via ActiveRecord
+
+This keeps your web process fast (no runtime compilation) and avoids running a separate Python query service.
+
+## What this gem provides
+
+- **Compiler**: `.l` → digested `.sql` + `.meta.json` + `manifest.json`
+- **Safety checks**: only allow `SELECT/WITH`, disallow multi-statement `;`, disallow row locks (`FOR UPDATE`, …) and DDL/DML keywords
+- **CLI (Thor)**: `install / compile / clean / watch / version`
+- **Optional ActiveRecord runner**: execute compiled SQL with binds, dialect-aware placeholders
+  - Postgres: `$1, $2, ...` + `SET LOCAL statement_timeout`
+  - SQLite: `?` placeholders + timeout no-op
+- **Rails install task** (via Railtie): `rake logica_compiler:install` generates thin integration files into your app
+
+## Installation
+
+```ruby
+# Gemfile
+gem "logica_compiler", github: "jasl/logica_compiler.rb", require: "logica_compiler"
+```
+
+Then:
+
+```bash
+bundle install
+bundle exec rake logica_compiler:install
+```
+
+This generates (idempotently):
+
+- `logica/config.yml` (with a demo query)
+- `logica/requirements.txt` (pins `logica==1.3.1415926535897`)
+- `logica/programs/hello_world.l` (demo program)
+- `logica/programs/.keep`, `logica/compiled/.keep`
+- `.gitignore` rules to ignore `logica/compiled/*` (but keep `.keep`)
+- `bin/logica` (thin wrapper around the gem CLI)
+- `config/initializers/logica_compiler.rb` (injects registry + runner into `Rails.application.config.x.logica`)
+
+## CLI usage
+
+From the Rails app root:
+
+```bash
+# ensure Python Logica is available (uses .venv/bin/logica if present, otherwise `logica` on PATH, otherwise installs into tmp/logica_venv)
+bin/logica install
+
+# compile everything in logica/config.yml to digested SQL + manifest
+bin/logica compile
+
+# compile a single query
+bin/logica compile hello_world
+
+# watch logica/programs/**/*.l and recompile on change
+bin/logica watch
+```
+
+### Using a system-installed Python `logica`
+
+If you install Python Logica globally (and `logica` is on `PATH`), `bin/logica compile` will use it automatically.
+You can still force a specific executable path via `LOGICA_BIN`:
+
+```bash
+python -m pip install logica==1.3.1415926535897
+bin/logica compile
+
+# or override explicitly
+LOGICA_BIN=/usr/local/bin/logica bin/logica compile
+```
+
+## Configuration (`logica/config.yml`)
+
+Example:
+
+```yaml
+engine: postgres
+output_dir: logica/compiled
+
+queries:
+  hello_world:
+    program: logica/programs/hello_world.l
+    predicate: Greet
+```
+
+Notes:
+
+- If your program declares `@Engine("...");`, it must match `engine` in config.
+- If you omit `@Engine(...)`, the compiler will prepend `@Engine("<engine>");` automatically.
+
+## Running compiled queries in Rails
+
+After compilation, the initializer provides:
+
+- `Rails.application.config.x.logica.registry`
+- `Rails.application.config.x.logica.runner`
+
+Example:
+
+```ruby
+Rails.application.config.x.logica.runner.exec(:hello_world, statement_timeout: nil)
+```
+
+## Development (inside this repo)
+
+Run gem tests:
+
+```bash
+bundle install --all
+bundle exec rake
+```
+
+The SQLite e2e test (`test/sqlite_e2e_test.rb`) will **skip** if:
+
+- ActiveRecord/sqlite3 gems are unavailable, or
+- Python `logica` CLI cannot be found (via `LOGICA_BIN` or on PATH)
+
+CI installs Python Logica so the e2e test runs there.
+
+## Environment variables
+
+- `LOGICA_BIN`: path or command name of the Python `logica` executable (optional; defaults to `.venv/bin/logica` if present, otherwise `logica` on `PATH`, otherwise `tmp/logica_venv/.../logica`)
+- `LOGICA_COMPILE_TIMEOUT`: compile timeout seconds (default 30)
+- `LOGICA_PYTHON`, `LOGICA_REQUIREMENTS`, `LOGICA_VENV`: advanced install overrides for the CLI
+- `FORCE=1`: for `rake logica_compiler:install` only (overwrite generated files if they already exist)
+
+For compilation, use CLI flags instead:
+
+- `bin/logica compile --force`
+- `bin/logica compile --no-prune`
+
+## License
+
+MIT. See `LICENSE.txt`.

data/Rakefile

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "minitest/test_task"
+
+Minitest::TestTask.create
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[test rubocop]
+
+desc "Run tests with a lib/ coverage summary (local use; not wired into CI)"
+task :coverage do
+  ENV["MT_NO_PLUGINS"] = "1"
+
+  require "coverage"
+  Coverage.start
+
+  lib_root = File.expand_path("lib", __dir__)
+  test_root = File.expand_path("test", __dir__)
+
+  $LOAD_PATH.unshift(lib_root) unless $LOAD_PATH.include?(lib_root)
+  $LOAD_PATH.unshift(test_root) unless $LOAD_PATH.include?(test_root)
+
+  # Load all gem library files so untested files show up in the report too.
+  Dir[File.join(lib_root, "**/*.rb")].sort.each do |path|
+    rel = path.sub("#{lib_root}/", "").sub(/\.rb\z/, "")
+    next if rel == "logica_compiler/railtie" # Rails is optional in this gem.
+
+    require rel
+  end
+
+  require "test_helper"
+
+  Dir[File.join(test_root, "**/*_test.rb")].sort.each do |path|
+    next if path.end_with?("test_helper.rb")
+
+    require path
+  end
+
+  Minitest.after_run do
+    result = Coverage.result
+    lib_prefix = "#{lib_root}#{File::SEPARATOR}"
+
+    rows = []
+    result.each do |path, counts|
+      next unless path.start_with?(lib_prefix)
+      next unless counts.is_a?(Array)
+
+      total = 0
+      covered = 0
+      counts.each do |c|
+        next if c.nil?
+        total += 1
+        covered += 1 if c.positive?
+      end
+
+      pct = total.positive? ? (covered * 100.0 / total) : 0
+      rel = path.sub("#{File.expand_path(__dir__)}#{File::SEPARATOR}", "")
+      rows << [pct, covered, total, rel]
+    end
+
+    rows.sort_by!(&:first)
+    puts "\nlib/ coverage by file (low → high):"
+    rows.each do |pct, covered, total, rel|
+      puts format("%6.1f%%  %4d/%-4d  %s", pct, covered, total, rel)
+    end
+
+    total_covered = rows.sum { |r| r[1] }
+    total_lines = rows.sum { |r| r[2] }
+    total_pct = total_lines.positive? ? (total_covered * 100.0 / total_lines) : 0
+    puts format("\nTOTAL: %d/%d (%.1f%%)\n", total_covered, total_lines, total_pct)
+  end
+end

data/exe/logica_compiler

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "logica_compiler"
+require "logica_compiler/cli"
+
+LogicaCompiler::CLI.start(ARGV)

data/lib/logica_compiler/active_record/runner.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "active_record"
+
+require_relative "../sql_dialect/postgres"
+require_relative "../sql_dialect/sqlite"
+
+module LogicaCompiler
+  module ActiveRecord
+    class Runner
+      DEFAULT_STATEMENT_TIMEOUT_MS = 3000
+
+      def initialize(registry:, connection: nil, connection_pool: ::ActiveRecord::Base.connection_pool, dialect: nil)
+        @registry = registry
+        @connection = connection
+        @connection_pool = connection_pool
+        @dialect = dialect
+      end
+
+      def exec(name, filters: {}, limit: nil, statement_timeout: DEFAULT_STATEMENT_TIMEOUT_MS)
+        return exec_with_connection(@connection, name, filters:, limit:, statement_timeout:) if @connection
+
+        @connection_pool.with_connection do |conn|
+          exec_with_connection(conn, name, filters:, limit:, statement_timeout:)
+        end
+      end
+
+      private
+
+      def exec_with_connection(conn, name, filters:, limit:, statement_timeout:)
+        dialect = @dialect || dialect_for(conn)
+        sql, binds = wrap_sql(@registry.sql(name), filters:, limit:, connection: conn, dialect:)
+        label = "Logica/#{name}"
+
+        timeout = format_statement_timeout(statement_timeout)
+        timeout_sql = timeout && dialect.statement_timeout_sql(conn, timeout)
+
+        return conn.exec_query(sql, label, binds) unless timeout_sql
+
+        conn.transaction do
+          conn.execute(timeout_sql)
+          conn.exec_query(sql, label, binds)
+        end
+      end
+
+      def dialect_for(conn)
+        adapter = conn.adapter_name.to_s.downcase
+        return SqlDialect::Sqlite.new if adapter.include?("sqlite")
+
+        SqlDialect::Postgres.new
+      end
+
+      def wrap_sql(base_sql, filters:, limit:, connection:, dialect:)
+        where_clauses = []
+        binds = []
+        idx = 1
+
+        filters.each do |key, value|
+          column = key.to_s
+          raise ArgumentError, "Invalid filter column: #{column.inspect}" unless column.match?(/\A[a-z_][a-z0-9_]*\z/i)
+
+          where_clauses << "t.#{connection.quote_column_name(column)} = #{dialect.placeholder(idx)}"
+          binds << ::ActiveRecord::Relation::QueryAttribute.new(
+            column,
+            value,
+            ::ActiveRecord::Type::Value.new
+          )
+          idx += 1
+        end
+
+        base_sql = base_sql.to_s.strip.sub(/;\s*\z/, "")
+        sql = +"SELECT * FROM (#{base_sql}) AS t"
+        sql << " WHERE " << where_clauses.join(" AND ") unless where_clauses.empty?
+        sql << " LIMIT #{Integer(limit)}" if limit
+
+        [sql, binds]
+      end
+
+      def format_statement_timeout(value)
+        return nil if value.nil?
+        return "#{Integer(value)}ms" if value.is_a?(Integer) || value.to_s.match?(/\A\d+\z/)
+
+        value.to_s
+      rescue ArgumentError
+        nil
+      end
+    end
+  end
+end

data/lib/logica_compiler/cli.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+require "thor"
+
+require_relative "commands/clean"
+require_relative "commands/compile"
+require_relative "commands/install"
+require_relative "commands/version"
+require_relative "commands/watch"
+
+module LogicaCompiler
+  class CLI < Thor
+    default_command :help
+
+    class_option :config,
+                 type: :string,
+                 default: "logica/config.yml",
+                 desc: "Path to logica config.yml"
+
+    class_option :logica_bin,
+                 type: :string,
+                 desc: "Override LOGICA_BIN (path or command name)"
+
+    desc "install", "Install pinned Logica CLI into tmp/logica_venv (no-op if LOGICA_BIN points elsewhere)"
+    def install
+      with_cli_errors do
+        version = Commands::Install.new.call(config_path: options[:config], logica_bin_override: options[:logica_bin])
+        say version, :green
+      end
+    end
+
+    desc "compile [NAME]", "Compile Logica programs to digested SQL + manifest (optional single query NAME)"
+    method_option :force, type: :boolean, default: false, desc: "Recompile even if digest matches"
+    method_option :prune, type: :boolean, default: true, desc: "Remove old digested artifacts for the same query name"
+    def compile(name = nil)
+      with_cli_errors do
+        Commands::Compile.new.call(
+          config_path: options[:config],
+          logica_bin_override: options[:logica_bin],
+          name:,
+          force: options[:force],
+          prune: options[:prune]
+        )
+      end
+    end
+
+    desc "clean", "Remove compiled SQL artifacts (keeps logica/compiled/.keep)"
+    def clean
+      with_cli_errors do
+        Commands::Clean.new.call(config_path: options[:config], logica_bin_override: options[:logica_bin])
+      end
+    end
+
+    desc "watch", "Watch logica/programs and auto-compile on change (development only)"
+    def watch
+      with_cli_errors do
+        Commands::Watch.new.call(config_path: options[:config], logica_bin_override: options[:logica_bin])
+      end
+    end
+
+    desc "version", "Print pinned/installed Logica version"
+    def version
+      with_cli_errors do
+        say Commands::Version.new.call(config_path: options[:config], logica_bin_override: options[:logica_bin])
+      end
+    end
+
+    private
+
+    def with_cli_errors
+      yield
+    rescue LogicaCompiler::Error => e
+      raise Thor::Error, e.message
+    end
+  end
+end

data/lib/logica_compiler/commands/base.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+require_relative "../config"
+require_relative "../deps/clock"
+require_relative "../deps/env"
+require_relative "../deps/shell"
+require_relative "../util"
+
+module LogicaCompiler
+  module Commands
+    class Base
+      def initialize(
+        env: Deps::Env.new,
+        shell: Deps::Shell.new,
+        clock: Deps::Clock.new,
+        config_loader: Config.method(:load!),
+        which: Util.method(:which)
+      )
+        @env = env
+        @shell = shell
+        @clock = clock
+        @config_loader = config_loader
+        @which = which
+      end
+
+      private
+
+      attr_reader :env, :shell, :clock, :config_loader, :which
+
+      def with_logica_bin_override(override)
+        value = override.to_s
+        value = nil if value.strip.empty?
+        return yield if value.nil?
+
+        env.with_temp("LOGICA_BIN" => value) { yield }
+      end
+
+      def load_config(config_path:, logica_bin_override: nil)
+        with_logica_bin_override(logica_bin_override) do
+          config_loader.call(config_path)
+        end
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/clean.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require "fileutils"
+
+require_relative "base"
+
+module LogicaCompiler
+  module Commands
+    class Clean < Base
+      def call(config_path:, logica_bin_override: nil)
+        config = load_config(config_path:, logica_bin_override:)
+        dir = config.output_dir_path
+        FileUtils.mkdir_p(dir)
+
+        Dir.glob(dir.join("*").to_s).each do |path|
+          next if File.basename(path) == ".keep"
+
+          FileUtils.rm_rf(path)
+        end
+
+        true
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/compile.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+require_relative "../compiler"
+require_relative "../manifest"
+require_relative "base"
+require_relative "logica"
+
+module LogicaCompiler
+  module Commands
+    class Compile < Base
+      def initialize(compiler_factory: ->(config) { Compiler.new(config:) }, **kwargs)
+        super(**kwargs)
+        @logica = Logica.new(**kwargs)
+        @compiler_factory = compiler_factory
+      end
+
+      def call(config_path:, logica_bin_override: nil, name: nil, force: false, prune: true)
+        config = load_config(config_path:, logica_bin_override:)
+        @logica.ensure_available!(config, allow_install: true)
+
+        compiler = @compiler_factory.call(config)
+
+        if name.to_s.strip.empty?
+          compiler.compile_all!(force:, prune:)
+        else
+          compile_one_and_update_manifest!(compiler, config, name.to_s, force:, prune:)
+        end
+      end
+
+      private
+
+      def compile_one_and_update_manifest!(compiler, config, name, force:, prune:)
+        sym = name.to_sym
+        unless config.queries.key?(sym)
+          raise UnknownQueryError, "Unknown Logica query: #{name}. Known: #{config.queries.keys.map(&:to_s).sort.join(", ")}"
+        end
+
+        entry = compiler.compile_one!(name:, query: config.queries.fetch(sym), force:, prune:)
+
+        manifest =
+          begin
+            Manifest.load(config.manifest_path)
+          rescue StandardError
+            Manifest.build(engine: config.engine, logica_version: compiler.logica_version, queries: {})
+          end
+
+        manifest["engine"] = config.engine
+        manifest["logica"] = { "pypi" => "logica", "version" => compiler.logica_version.to_s }
+        manifest["compiled_at"] = clock.now_utc.iso8601
+        manifest["queries"] ||= {}
+        manifest["queries"][name.to_s] = entry
+        Manifest.write!(config.manifest_path, manifest)
+        manifest
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/install.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+require_relative "base"
+require_relative "logica"
+
+module LogicaCompiler
+  module Commands
+    class Install < Base
+      def initialize(**kwargs)
+        super
+        @logica = Logica.new(**kwargs)
+      end
+
+      def call(config_path:, logica_bin_override: nil)
+        config = load_config(config_path:, logica_bin_override:)
+        @logica.ensure_available!(config, allow_install: true)
+        config.pinned_logica_version || "unknown"
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/logica.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require "digest"
+require "fileutils"
+require "pathname"
+
+require_relative "base"
+
+module LogicaCompiler
+  module Commands
+    class Logica < Base
+      DEFAULT_PYTHON = "python3"
+
+      def ensure_available!(config, allow_install:)
+        bin = config.logica_bin.to_s
+
+        if Util.path_like?(bin)
+          return if File.executable?(bin)
+
+          if allow_install && bin == default_venv_logica_path(config).to_s
+            install_venv_logica!(config)
+            return if File.executable?(bin)
+          end
+
+          raise LogicaError, "Logica CLI not found: #{bin.inspect}"
+        end
+
+        return if which.call(bin)
+
+        raise LogicaError, "Logica CLI #{bin.inspect} not found on PATH. Run `bin/logica install` or set LOGICA_BIN."
+      end
+
+      def default_venv_logica_path(config)
+        config.root.join("tmp/logica_venv", Util.venv_bin_dir, "logica")
+      end
+
+      def install_venv_logica!(config)
+        python = env.fetch("LOGICA_PYTHON", DEFAULT_PYTHON)
+        requirements = Pathname(env.fetch("LOGICA_REQUIREMENTS", config.requirements_path.to_s))
+        venv_dir = Pathname(env.fetch("LOGICA_VENV", config.root.join("tmp/logica_venv").to_s))
+
+        raise LogicaError, "#{python} not found. Install Python 3 and try again." unless which.call(python)
+        raise LogicaError, "requirements not found: #{requirements}" unless requirements.exist?
+
+        FileUtils.mkdir_p(venv_dir.parent)
+        shell.system!(python, "-m", "venv", venv_dir.to_s) unless venv_dir.exist?
+
+        stamp = venv_dir.join(".requirements.sha256")
+        req_sha = Digest::SHA256.hexdigest(requirements.read)
+
+        return if stamp.exist? && stamp.read.strip == req_sha
+
+        venv_python = venv_dir.join(Util.venv_bin_dir, "python").to_s
+        shell.system!(venv_python, "-m", "pip", "install", "--upgrade", "pip")
+        shell.system!(venv_python, "-m", "pip", "install", "-r", requirements.to_s)
+
+        stamp.write(req_sha)
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/version.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+require_relative "base"
+
+module LogicaCompiler
+  module Commands
+    class Version < Base
+      def call(config_path:, logica_bin_override: nil)
+        config = load_config(config_path:, logica_bin_override:)
+        config.pinned_logica_version || "unknown"
+      end
+    end
+  end
+end

data/lib/logica_compiler/commands/watch.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+require_relative "../watcher"
+require_relative "base"
+require_relative "logica"
+
+module LogicaCompiler
+  module Commands
+    class Watch < Base
+      def initialize(watcher: Watcher, **kwargs)
+        super(**kwargs)
+        @watcher = watcher
+        @logica = Logica.new(**kwargs)
+      end
+
+      def call(config_path:, logica_bin_override: nil)
+        config = load_config(config_path:, logica_bin_override:)
+        @logica.ensure_available!(config, allow_install: true)
+        @watcher.start!(config_path:)
+      end
+    end
+  end
+end