collie 0.1.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c8b86b09838d7091834f3243ae717931e2085c16013aa17e0dff5e8569e8975b
-  data.tar.gz: 4d09ba7a44de27ff966cf4edc0da4b9824749d719f1051d7fce6d8c39e36c97d
+  metadata.gz: 94fb085cbc3a181a76d745558276b76e4922ad401f5335a42d95e8ca4fcfd1b1
+  data.tar.gz: 4648e0cf533ddfb58a5d0d4b640951a42cb8f0c80e00ebd207757c8c8f450312
 SHA512:
-  metadata.gz: fbf34be5a91e5a0fb0d5bf334e0bffe92ad289720b059022e9e6c050facbba9e0956ea49d4820eec03fc06e81138a4beb06a1e872031472eaffb2f55ec770f18
-  data.tar.gz: 7d323a0aaca55514da222f2ba48f8bd436fb1139a43d83d7bac90845f06844bd29f9ab280d1859f42e54dc260d9e142af410260042546848f6f7693bf4a9bd11
+  metadata.gz: 80384473f720db64d99532419949bcf3b26227d03a0288b5e4dd5c27cd73ea4fdde583518ec659d80ff8a439bcd1a0058cc3a936f95487097b0413c00112d4b9
+  data.tar.gz: 13883b3575a3efc000fcb867a526a8288de0000512e910608e5b628541ab185355146bb37c6540c4d2bb25fca95c76786cdb5644161d38bef92fa1e23399a6d8

data/CHANGELOG.md

@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+## 1.0.0 - 2026-05-20
+
+### Added
+
+- Added stable `collie lint`, `collie fmt`, `collie rules`, and `collie version` CLI workflows for Lrama-style grammar files.
+- Added `collie explain` for rule metadata, `collie init` for generating profile-based configuration, and `collie config-schema` for editor/tooling integration.
+- Added parser debug commands, `collie tokens` and `collie ast`, with JSON output.
+- Added stdin support for linting and formatting with `--stdin` and `--stdin-filename`.
+- Added SARIF output for code scanning, alongside text, JSON, and GitHub Actions annotation reporters.
+- Added lint coverage for symbol conflicts, duplicate precedence declarations, non-productive grammar cycles, Lrama declarations, inline rule dependencies, and precedence declarations used as tokens.
+
+### Changed
+
+- Expanded file target handling so `lint` and `fmt` can accept directories and glob patterns.
+- Improved formatter preservation for comments, unknown directive blocks, declaration order, parameterless rule declarations, and grammar structure.
+- Made formatter layout options configurable and verified formatted output by reparsing it before writing.
+- Improved CLI failure behavior for CI, including parse errors as diagnostics and configurable `--fail-level` thresholds.
+- Made rule filtering stricter by validating unknown `--only` and `--except` rule names before running lint checks.
+- Reflected configured rule enablement and severity in `collie rules --format json`.
+
+### Fixed
+
+- Fixed repeated `--only` and `--except` handling so rule filters do not consume positional file arguments.
+- Fixed config loading errors for missing, invalid, inherited, and non-mapping YAML files.
+- Fixed autocorrection so one correction does not clobber changes made by another correction.
+- Reduced false positives in right-recursion and unused-rule diagnostics for LR grammar workflows.
+
 ## 0.1.0 - 2025-12-17
 
-- Initial commit
+- Initial commit

data/README.md

@@ -1,18 +1,11 @@
 # Collie
 
-A linter and formatter for Lrama Style BNF grammar files (.y files). Collie helps you write clean, maintainable, and error-free grammar files for parser generators like Lrama, Yacc, and Bison.
+A linter and formatter for Lrama Style BNF grammar files (`.y` files).
 
 [![CI](https://github.com/ydah/collie/workflows/CI/badge.svg)](https://github.com/ydah/collie/actions)
 [![Gem Version](https://badge.fury.io/rb/collie.svg)](https://badge.fury.io/rb/collie)
 
-## Features
-
-- 18 Built-in Lint Rules - Catch common errors and suggest improvements
-- Lrama Extension Support - Full support for parameterized rules, named references, and inline rules
-- Smart Formatting - Consistent indentation, alignment, and spacing
-- Configurable - Customize rules and formatting options via `.collie.yml`
-- Multiple Output Formats - Text, JSON, and GitHub Actions annotations
-- Auto-correction - Automatically fix certain issues
+Collie checks grammar files for common mistakes, formats them consistently, and supports Lrama-specific syntax such as parameterized rules, named references, and inline rules.
 
 ## Installation
 
@@ -20,199 +13,86 @@ A linter and formatter for Lrama Style BNF grammar files (.y files). Collie help
 gem install collie
 ```
 
-Or add to your Gemfile:
+Or add it to your Gemfile:
 
 ```ruby
-gem 'collie', require: false
+gem "collie", require: false
 ```
 
-## Quick Start
-
-### Try Online
-
-Try Collie in your browser without installing anything:
-
-[Open Playground](https://ydah.github.io/collie/playground/) (Coming soon)
+Collie requires Ruby 3.2 or newer.
 
-### Lint a grammar file
+## Quick Start
 
 ```bash
-# Check for issues
+# Lint files, globs, or directories
 collie lint parse.y
+collie lint "src/**/*.y"
+collie lint grammars/
 
-# Auto-fix issues where possible
-collie lint -a parse.y
-```
-
-### Format a grammar file
-
-```bash
-# Check formatting
+# Format
+collie fmt parse.y
 collie fmt --check parse.y
+collie fmt --diff parse.y
 
-# Format in-place
-collie fmt parse.y
+# Auto-correct supported lint offenses
+collie lint -a parse.y
 
-# Show diff
-collie fmt --diff parse.y
+# Inspect rules
+collie rules
+collie explain DuplicateToken
 ```
 
-### List all available rules
+## Configuration
+
+Generate a config file:
 
 ```bash
-collie rules
+collie init
+collie init --profile lrama
+collie init --profile strict --path .collie.yml
 ```
 
-## Configuration
+Profiles: `default`, `lrama`, `bison`, `strict`, `minimal`.
 
-Create a `.collie.yml` file in your project root:
+Minimal `.collie.yml`:
 
 ```yaml
-# Inherit from another config (optional)
-inherit_from: .collie_base.yml
+include:
+  - "src/**/*.y"
+exclude:
+  - "vendor/**/*"
 
-# Rule configuration
-rules:
-  DuplicateToken:
-    enabled: true
-    severity: error
+formatter:
+  indent_size: 2
+  max_line_length: 120
 
+rules:
   TokenNaming:
-    enabled: true
     severity: convention
-    pattern: '^[A-Z][A-Z0-9_]*$'
-
-  LongRule:
-    enabled: true
-    max_alternatives: 10
-
-  # Disable specific rules
   LeftRecursion:
     enabled: false
-
-# Formatter options
-formatter:
-  indent_size: 4
-  align_tokens: true
-  align_alternatives: true
-  blank_lines_around_sections: 2
-  max_line_length: 120
-
-# File patterns
-include:
-  - '**/*.y'
-exclude:
-  - 'vendor/**/*'
-  - 'tmp/**/*'
-```
-
-## Available Rules
-
-### Validation Rules (Error)
-
-| Rule | Description | Auto-fix |
-|------|-------------|----------|
-| `DuplicateToken` | Token defined multiple times | No |
-| `UndefinedSymbol` | Reference to undeclared token/nonterminal | No |
-| `UnreachableRule` | Rule not derivable from start symbol | No |
-| `CircularReference` | Infinite recursion in grammar | No |
-| `MissingStartSymbol` | No `%start` declaration with ambiguous default | No |
-
-### Warning Rules
-
-| Rule | Description | Auto-fix |
-|------|-------------|----------|
-| `UnusedNonterminal` | Nonterminal defined but never referenced | No |
-| `UnusedToken` | Token declared but never used | No |
-| `LeftRecursion` | Detects left recursion (informational) | No |
-| `RightRecursion` | Suggests left recursion conversion | No |
-| `AmbiguousPrecedence` | Operators without explicit precedence | No |
-
-### Style Rules (Convention)
-
-| Rule | Description | Auto-fix |
-|------|-------------|----------|
-| `TokenNaming` | Tokens should be UPPER_CASE | No |
-| `NonterminalNaming` | Nonterminals should be snake_case | No |
-| `ConsistentTagNaming` | Type tags should be consistent | No |
-| `TrailingWhitespace` | No trailing whitespace at end of lines | Yes |
-| `EmptyAction` | Warns on empty `{ }` actions | Yes |
-| `LongRule` | Rule with too many alternatives | No |
-
-### Optimization Rules (Info)
-
-| Rule | Description | Auto-fix |
-|------|-------------|----------|
-| `FactorizableRules` | Suggests factoring common prefixes | No |
-| `RedundantEpsilon` | Unnecessary epsilon productions | No |
-| `PrecImprovement` | Suggests `%prec` improvements | No |
-
-## Usage Examples
-
-### Example Grammar File
-
-```yacc
-%token <node> CLASS MODULE DEF
-%token <id> IDENTIFIER CONSTANT
-%token <num> INTEGER FLOAT
-
-%left '+' '-'
-%left '*' '/'
-%right '^'
-
-%%
-
-program
-    : class_definition
-    | module_definition
-    ;
-
-class_definition
-    : CLASS CONSTANT '{' class_body '}'
-        { $$ = make_class($2, $4); }
-    ;
-
-expr
-    : expr '+' expr    { $$ = add($1, $3); }
-    | expr '-' expr    { $$ = sub($1, $3); }
-    | expr '*' expr    { $$ = mul($1, $3); }
-    | '(' expr ')'     { $$ = $2; }
-    | IDENTIFIER       { $$ = var($1); }
-    | INTEGER          { $$ = num($1); }
-    ;
-
-%%
 ```
 
-### Lrama Extensions
-
-Collie fully supports Lrama-specific syntax:
-
-```yacc
-# Parameterized Rules
-%rule pair(X, Y): X COMMA Y ;
+## Commands
 
-number_pair
-    : pair(NUMBER, NUMBER)
-        { $$ = make_pair($1, $3); }
-    ;
+| Command | Purpose |
+| --- | --- |
+| `collie lint [OPTIONS] FILES` | Lint grammar files. Supports `--format text\|json\|github\|sarif`, `--fail-level`, `--only`, `--except`, `--stdin`, and `--autocorrect`. |
+| `collie fmt [OPTIONS] FILES` | Format grammar files. Supports `--check`, `--diff`, `--config`, and `--stdin`. |
+| `collie rules [--format text\|json]` | List available lint rules. |
+| `collie explain RULE [--format text\|json]` | Show rule metadata. |
+| `collie config-schema` | Print the JSON Schema for `.collie.yml`. |
+| `collie tokens FILE` | Print lexer tokens as JSON. |
+| `collie ast FILE` | Print the parsed AST as JSON. |
+| `collie version` | Print the installed version. |
 
-# Named References
-assignment
-    : IDENTIFIER[var] EQUALS NUMBER[value]
-        { assign($var, $value); }
-    ;
-
-# Inline Rules
-%inline opt(X): /* empty */ | X ;
-```
+Run `collie help COMMAND` for all options.
 
-### CI Integration (GitHub Actions)
+## CI
 
-Use the reusable workflow in your project:
+Use the reusable GitHub Actions workflow:
 
 ```yaml
-# .github/workflows/lint.yml
 name: Lint Grammar Files
 
 on: [push, pull_request]
@@ -221,113 +101,30 @@ jobs:
   lint:
     uses: ydah/collie/.github/workflows/lint.yml@main
     with:
-      files: 'src/**/*.y'
-      config: '.collie.yml'
+      files: "src/**/*.y"
+      config: ".collie.yml"
       fail-on-warnings: true
 ```
 
-### Programmatic Usage
-
-```ruby
-require 'collie'
-
-# Parse a grammar file
-parser = Collie::Parser::Parser.new(source_code)
-ast = parser.parse
-
-# Analyze the grammar
-symbol_table = Collie::Analyzer::SymbolTable.new(ast)
-symbol_table.build
-
-# Run linter
-config = Collie::Config.new
-linter = Collie::Linter.new(config)
-offenses = linter.lint(ast)
-
-# Format the grammar
-formatter = Collie::Formatter::Formatter.new(config.formatter_options)
-formatted_code = formatter.format(ast)
-```
-
-## Command Line Options
-
-### `collie lint`
-
-```bash
-collie lint [OPTIONS] FILES
-
-Options:
-  --config PATH         Path to config file (default: .collie.yml)
-  --format FORMAT       Output format: text, json, github (default: text)
-  -a, --autocorrect     Auto-fix offenses where possible
-  --only RULES          Run only specified rules (comma-separated)
-  --except RULES        Exclude specified rules (comma-separated)
-```
-
-### `collie fmt`
+For code scanning integrations:
 
 ```bash
-collie fmt [OPTIONS] FILES
-
-Options:
-  --config PATH         Path to config file
-  --check               Check only, don't modify files
-  --diff                Show diff of changes
-```
-
-### `collie rules`
-
-```bash
-collie rules [OPTIONS]
-
-Options:
-  --format FORMAT       Output format: text, json (default: text)
+collie lint --format sarif parse.y
 ```
 
 ## Development
 
 ```bash
-# Install dependencies
 bundle install
-
-# Run tests
 bundle exec rspec
-
-# Run linter
-bundle exec rubocop
-
-# Run all checks
 bundle exec rake
 ```
 
-## Contributing
+## Documentation
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+- [Tutorial](docs/TUTORIAL.md)
+- [Changelog](CHANGELOG.md)
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-
-## Credits
-
-Developed for improving the development experience with [Lrama](https://github.com/ruby/lrama), the next-generation parser generator for Ruby.
-
-## Related Projects
-
-### Editor Integration (Planned)
-
-- collie-lsp - LSP (Language Server Protocol) implementation for Collie
-- vscode-collie - VS Code extension for Collie
-
-### Parser Generators
-
-- [Lrama](https://github.com/ruby/lrama) - LALR (1) parser generator
-- [Bison](https://www.gnu.org/software/bison/) - GNU parser generator
-
-### Inspiration
-
-- [RuboCop](https://github.com/rubocop/rubocop) - Ruby static code analyzer (inspiration for architecture)
+MIT License. See [LICENSE.txt](LICENSE.txt).

data/lib/collie/analyzer/reachability.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "set"
+
 require_relative "../ast"
 
 module Collie
@@ -27,40 +29,35 @@ module Collie
       private
 
       def build_dependency_graph
-        # Process normal rules
-        @grammar.rules.each do |rule|
+        rule_like_nodes.each do |rule|
+          current_rule_name = rule_name(rule)
+
           rule.alternatives.each do |alt|
             alt.symbols.each do |symbol|
               if symbol.nonterminal?
-                @dependencies[rule.name] << symbol.name
+                @dependencies[current_rule_name] << symbol.name
                 # Also consider parameterized rule call arguments: list(expr)
                 if symbol.arguments
                   symbol.arguments.each do |arg|
-                    @dependencies[rule.name] << arg.name if arg.nonterminal?
+                    @dependencies[current_rule_name] << arg.name if arg.nonterminal?
                   end
                 end
               end
             end
           end
         end
+      end
 
-        # Process parameterized rules (%rule)
-        @grammar.declarations.each do |decl|
-          next unless decl.is_a?(AST::ParameterizedRule)
+      def rule_like_nodes
+        @grammar.rules + @grammar.declarations.select { |declaration| rule_like_declaration?(declaration) }
+      end
 
-          decl.alternatives.each do |alt|
-            alt.symbols.each do |symbol|
-              if symbol.nonterminal?
-                @dependencies[decl.name] << symbol.name
-                if symbol.arguments
-                  symbol.arguments.each do |arg|
-                    @dependencies[decl.name] << arg.name if arg.nonterminal?
-                  end
-                end
-              end
-            end
-          end
-        end
+      def rule_like_declaration?(declaration)
+        declaration.is_a?(AST::ParameterizedRule) || declaration.is_a?(AST::InlineRule)
+      end
+
+      def rule_name(rule)
+        rule.is_a?(AST::InlineRule) ? rule.rule : rule.name
       end
 
       def infer_start_symbol

data/lib/collie/analyzer/recursion.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require "set"
+
 require_relative "../ast"
 
 module Collie
@@ -13,7 +15,7 @@ module Collie
       end
 
       def analyze
-        @grammar.rules.each do |rule|
+        rule_like_nodes.each do |rule|
           check_left_recursion(rule)
           check_right_recursion(rule)
         end
@@ -34,13 +36,27 @@ module Collie
 
       private
 
+      def rule_like_nodes
+        @grammar.rules + @grammar.declarations.select { |declaration| rule_like_declaration?(declaration) }
+      end
+
+      def rule_like_declaration?(declaration)
+        declaration.is_a?(AST::ParameterizedRule) || declaration.is_a?(AST::InlineRule)
+      end
+
+      def rule_name(rule)
+        rule.is_a?(AST::InlineRule) ? rule.rule : rule.name
+      end
+
       def check_left_recursion(rule)
+        name = rule_name(rule)
+
         rule.alternatives.each do |alt|
           next if alt.symbols.empty?
 
           first_symbol = alt.symbols.first
-          if first_symbol.nonterminal? && first_symbol.name == rule.name && !@left_recursive.include?(rule.name)
-            @left_recursive << rule.name
+          if first_symbol.nonterminal? && first_symbol.name == name && !@left_recursive.include?(name)
+            @left_recursive << name
           end
         end
 
@@ -49,23 +65,26 @@ module Collie
       end
 
       def check_right_recursion(rule)
+        name = rule_name(rule)
+
         rule.alternatives.each do |alt|
           next if alt.symbols.empty?
 
           last_symbol = alt.symbols.last
-          if last_symbol.nonterminal? && last_symbol.name == rule.name && !@right_recursive.include?(rule.name)
-            @right_recursive << rule.name
+          if last_symbol.nonterminal? && last_symbol.name == name && !@right_recursive.include?(name)
+            @right_recursive << name
           end
         end
       end
 
       def check_indirect_left_recursion(rule, visited = Set.new)
-        return if visited.include?(rule.name)
+        name = rule_name(rule)
+        return if visited.include?(name)
 
-        visited << rule.name
+        visited << name
 
         rule.alternatives.each do |alt|
-          check_alternative_for_indirect_recursion(alt, rule.name)
+          check_alternative_for_indirect_recursion(alt, name)
         end
       end
 
@@ -75,7 +94,7 @@ module Collie
         first_symbol = alt.symbols.first
         return unless first_symbol.nonterminal?
 
-        dependent_rule = @grammar.rules.find { |r| r.name == first_symbol.name }
+        dependent_rule = rule_like_nodes.find { |candidate| rule_name(candidate) == first_symbol.name }
         return unless dependent_rule
 
         check_dependent_rule_for_recursion(dependent_rule, rule_name)

data/lib/collie/analyzer/symbol_resolver.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+require_relative "../ast"
+
+module Collie
+  module Analyzer
+    # Resolves symbol kinds using declared tokens and nonterminals.
+    class SymbolResolver
+      def self.resolve(grammar, symbol_table)
+        new(grammar, symbol_table).resolve
+      end
+
+      def initialize(grammar, symbol_table)
+        @grammar = grammar
+        @symbol_table = symbol_table
+      end
+
+      def resolve
+        each_alternative do |alternative|
+          alternative.symbols.each { |symbol| resolve_symbol(symbol) }
+        end
+
+        @grammar
+      end
+
+      private
+
+      def each_alternative
+        @grammar.rules.each do |rule|
+          rule.alternatives.each { |alternative| yield alternative }
+        end
+
+        @grammar.declarations.each do |declaration|
+          next unless declaration.is_a?(AST::ParameterizedRule) || declaration.is_a?(AST::InlineRule)
+
+          declaration.alternatives.each { |alternative| yield alternative }
+        end
+      end
+
+      def resolve_symbol(symbol)
+        if @symbol_table.token?(symbol.name)
+          symbol.kind = :terminal
+        elsif @symbol_table.nonterminal?(symbol.name)
+          symbol.kind = :nonterminal
+        end
+
+        symbol.arguments&.each { |argument| resolve_symbol(argument) }
+      end
+    end
+  end
+end

data/lib/collie/ast.rb

@@ -84,6 +84,16 @@ module Collie
       end
     end
 
+    # Declaration node for directives Collie does not understand yet.
+    class UnknownDeclaration
+      attr_accessor :source, :location
+
+      def initialize(source:, location: nil)
+        @source = source
+        @location = location
+      end
+    end
+
     # Grammar rule node
     class Rule
       attr_accessor :name, :alternatives, :location
@@ -97,12 +107,14 @@ module Collie
 
     # Alternative production for a rule
     class Alternative
-      attr_accessor :symbols, :action, :prec, :location
+      attr_accessor :symbols, :action, :prec, :explicit_empty, :empty_marker, :location
 
-      def initialize(symbols: [], action: nil, prec: nil, location: nil)
+      def initialize(symbols: [], action: nil, prec: nil, explicit_empty: false, empty_marker: nil, location: nil)
         @symbols = symbols
         @action = action
         @prec = prec
+        @explicit_empty = explicit_empty
+        @empty_marker = empty_marker
         @location = location
       end
     end
@@ -152,10 +164,12 @@ module Collie
 
     # Lrama extension: Inline rule
     class InlineRule
-      attr_accessor :rule, :location
+      attr_accessor :rule, :parameters, :alternatives, :location
 
-      def initialize(rule:, location: nil)
+      def initialize(rule:, parameters: [], alternatives: [], location: nil)
         @rule = rule
+        @parameters = parameters
+        @alternatives = alternatives
         @location = location
       end
     end