ffast 0.2.2 → 0.2.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ce52e9ab5a9d9da32683b21584af707cf5c87e460f3bd74297e9038ea38c5494
-  data.tar.gz: 2b68a433982c97f278829b95234aed4ea5e4aec19264f15f9bfbe8615da19c9a
+  metadata.gz: 4b5b5e27292bfe589236ccf40952b5da73f4dfc4b35df85220d9beebae7c725e
+  data.tar.gz: 6c4466b554e164c447cfb02760c5b6cd262c0f904f7ea8c35897288efa4668ec
 SHA512:
-  metadata.gz: 246e52812d6f332636363e9f8dd960fa5e5ad3ecc50666007ae62041a212efc0e17f79dc01a675e15cd862f04bcc6771ac2ede3c8fd98c656b0efe7f34b67ffc
-  data.tar.gz: 641af33dbc8858631ae458bc53a4c1b7ce08c44da6788d0cdf7c8c3cae571a9bcb20630f3f9527aeea9a6023a920ac8fe8b0df83672b8280ab8415977114ff4b
+  metadata.gz: 0f8ccd9bcd621b69847fd1092b0fa08cada32258a2ec1f6b04654744c8930718628955698673577e4431a6fe6146da744bcb6aafdada442aa08bfe8e5ba32780
+  data.tar.gz: 263b032df9ed1ea5b8d7cd71e1bd613dc781c53fc1dd10cb8bedbe526d904543e1331bb43f345679954e7ebc404b0bdfb5f965dffd35b74c454038ba7ae0fb31

data/.agents/fast-pattern-expert/SKILL.md

@@ -0,0 +1,71 @@
+# Skill: Fast Pattern Expert
+
+You are an expert in constructing and translating natural language queries into `Fast` AST patterns for Ruby.
+
+## Core Expertise
+- Translating structural descriptions of Ruby code into S-expression based AST patterns.
+- Understanding the Ruby AST (Prism/Parser based).
+- Navigating and searching codebases semantically using `Fast`.
+
+## Syntax Reference
+- `(node_type ...)` : Search for a node of a specific type.
+- `_` : Matches any non-nil node/value.
+- `nil` : Matches exactly `nil`.
+- `...` : When last in a list, matches zero or more remaining children. Elsewhere, matches a node with children.
+- `^` : Navigate to the parent node.
+- `$` : Capture the matched node or value.
+- `{type1 type2}` : Union (OR) - matches if any internal expression matches.
+- `[expr1 expr2]` : Intersection (AND) - matches only if all internal expressions match.
+- `!expr` : Negation (NOT) - matches if the expression does not match.
+- `?expr` : Maybe - matches if the node is nil or matches the expression.
+- `\1`, `\2` : Backreference to previous captures.
+- `#custom_method` : Call a custom Ruby method for validation.
+- `.instance_method?` : Call an instance method on the node for validation (e.g., `.odd?`).
+
+## Common Ruby AST Nodes
+- `(def name (args) body)` : Method definition.
+- `(defs receiver name (args) body)` : Singleton method definition.
+- `(send receiver method_name args...)` : Method call.
+- `(class name superclass body)` : Class definition.
+- `(module name body)` : Module definition.
+- `(const scope name)` : Constant reference (scope is nil for top-level).
+- `(casgn scope name value)` : Constant assignment.
+- `(lvar name)` : Local variable read.
+- `(lvasgn name value)` : Local variable assignment.
+- `(ivar name)` : Instance variable read.
+- `(ivasgn name value)` : Instance variable assignment.
+- `(hash (pair key value)...)` : Hash literal.
+- `(array elements...)` : Array literal.
+
+## Translation Examples
+
+### Methods
+- "Find all methods named 'process'" -> `(def process)`
+- "Find methods with at least 3 arguments" -> `(def _ (args _ _ _ ...))`
+- "Find singleton methods (self.method)" -> `(defs ...)`
+- "Find methods that call 'super'" -> `(def _ _ (send nil :super ...))`
+
+### Classes & Modules
+- "Find classes inheriting from ApplicationController" -> `(class _ (const nil ApplicationController))`
+- "Find classes defined inside the 'User' namespace" -> `(class (const (const nil User) _) ...)`
+- "Find modules that include 'Enumerable'" -> `(module _ (begin < (send nil include (const nil Enumerable)) ...))`
+
+### Method Calls
+- "Find all calls to 'User.find'" -> `(send (const nil User) find ...)`
+- "Find calls to 'where' with a hash argument" -> `(send _ where (hash ...))`
+- "Find calls to 'exit' or 'abort'" -> `(send nil {exit abort} ...)`
+
+### Variables & Constants
+- "Find where the 'DEBUG' constant is assigned" -> `(casgn nil DEBUG)`
+- "Find all uses of instance variable '@user'" -> `(ivar @user)`
+- "Find assignments to '@user'" -> `(ivasgn @user)`
+
+## Strategy for Complex Queries
+1. **Identify the anchor node**: What is the primary structure? (e.g., a method definition, a specific call).
+2. **Describe children**: What must be true about its arguments or body?
+3. **Use Union/Intersection**: Combine multiple constraints using `{}` or `[]`.
+4. **Capture if needed**: Use `$` if you only want a specific part of the match.
+5. **Validate**: Always use `validate_fast_pattern` if available to check syntax.
+
+## AST Triage
+If you are unsure of the AST structure for a piece of code, use `Fast.ast("your code snippet")` or `Fast.ast_from_file` to see the s-expression representation. This is the most reliable way to build a pattern.

data/.github/workflows/release.yml

@@ -0,0 +1,27 @@
+name: Release Gem
+
+on:
+  push:
+    tags:
+      - 'v*' # Triggers when a new tag like v0.2.0 is pushed
+
+jobs:
+  release:
+    name: Build and Release
+    runs-on: ubuntu-22.04
+    permissions:
+      contents: read     # Required to checkout the code
+      id-token: write    # Required for RubyGems Trusted Publishing
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: '3.3'
+          bundler-cache: true
+
+      - name: Publish to RubyGems
+        uses: rubygems/release-gem@v1

data/.github/workflows/ruby.yml

@@ -0,0 +1,34 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+# This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
+# For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
+
+name: Ruby
+
+on:
+  push:
+    branches: [ "master" ]
+  pull_request:
+    branches: [ "master" ]
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-22.04
+    strategy:
+      matrix:
+        ruby-version: ['3.0', '3.1', '3.2', '3.3']
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Ruby
+      uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: ${{ matrix.ruby-version }}
+        bundler-cache: true
+    - name: Run tests
+      run: bundle exec rake

data/.gitignore

@@ -5,8 +5,10 @@
 /coverage/
 /doc/
 /pkg/
+/site/
 /spec/reports/
 /tmp/
+.venv/
 
 # rspec failure tracking
 .rspec_status

data/Fastfile

@@ -1,4 +1,10 @@
 # frozen_string_literal: true
+begin
+  require 'fast/source'
+rescue LoadError
+  nil
+end
+
 # Fastfile is loaded when you start an expression with a dot.
 #
 # You can introduce shortcuts or methods that can be embedded during your
@@ -9,7 +15,7 @@ version_file = Dir['lib/*/version.rb'].first
 Fast.shortcut(:version, '(casgn nil VERSION (str _))', version_file)
 
 # Show all classes that inherits Fast::Find
-Fast.shortcut(:finders, '(class ... (const nil Find)', 'lib')
+Fast.shortcut(:finders, '(class ... (const nil Find))', 'lib')
 
 # You can run shortcuts appending a dot to the shortcut.
 #   $ fast .version
@@ -17,11 +23,13 @@ Fast.shortcut(:finders, '(class ... (const nil Find)', 'lib')
 #   VERSION = '0.1.2'
 
 # Simple shortcut that I used often to show how the expression parser works
-Fast.shortcut(:parser, '(class (const nil ExpressionParser)', 'lib/fast.rb')
+Fast.shortcut(:parser, '(class (const nil ExpressionParser))', 'lib/fast.rb')
+Fast.shortcut(:sql_parser, '(def parse ...)', 'lib/fast/sql.rb')
 
 # Use `fast .bump_version` to rewrite the version file
 Fast.shortcut :bump_version do
-  rewrite_file('(casgn nil VERSION (str _)', version_file) do |node|
+  new_version = nil
+  rewrite_file('(casgn nil VERSION (str _))', version_file) do |node|
     target = node.children.last.loc.expression
     pieces = target.source.split('.').map(&:to_i)
     pieces.reverse.each_with_index do |fragment, i|
@@ -32,7 +40,24 @@ Fast.shortcut :bump_version do
         pieces[-(i + 1)] = 0
       end
     end
-    replace(target, "'#{pieces.join('.')}'")
+    new_version = pieces.join('.')
+    replace(target, "'#{new_version}'")
+  end
+
+  print "Commit bumped version v#{new_version}? (y/n) "
+  if $stdin.gets.chomp.downcase == 'y'
+    system("git add #{version_file} && git commit -m 'Bump version to v#{new_version}'")
+
+    print "Tag version v#{new_version}? (y/n) "
+    if $stdin.gets.chomp.downcase == 'y'
+      system("git tag v#{new_version}")
+
+      print "Push commit and tag to origin? (y/n) "
+      if $stdin.gets.chomp.downcase == 'y'
+        system("git push origin HEAD")
+        system("git push origin v#{new_version}")
+      end
+    end
   end
 end
 
@@ -57,22 +82,9 @@ Fast.shortcut :intro do
   Fast.shortcuts[:walk].run
 end
 
-# Useful for `fast .walk file.md` but not required by the library.
-private
-def require_or_install_tty_md
-  require 'tty-markdown'
-rescue LoadError
-  puts 'Installing tty-markdown gem to better engage you :)'
-  Gem.install('tty-markdown')
-  puts 'Done! Now, back to our topic \o/'
-  system('clear')
-  retry
-end
-
 # Interactive command line walkthrough
 # fast .walk docs/walkthrough.md
 Fast.shortcut :walk do
-  require_or_install_tty_md
   file = ARGV.last
   execute = ->(line) { system(line) }
   walk = ->(line) { line.each_char { |c| sleep(0.02) and print(c) } }
@@ -87,7 +99,82 @@ Fast.shortcut :walk do
     when /^!{3}\s/
       # Skip warnings that are only for web tutorials
     else
-      walk[TTY::Markdown.parse(line)]
+      walk[Fast.render_markdown_for_terminal(line)]
     end
   end
 end
+
+# Format SQL
+Fast.shortcut :format_sql do
+  require 'fast/sql'
+  file = ARGV.last
+  method = File.exist?(file) ? :parse_sql_file : :parse_sql
+  ast = Fast.public_send(method, file)
+  ast = ast.first if ast.is_a? Array
+  eligible_kw = [:RESERVED_KEYWORD]
+  eligible_tokens = [:BY]
+
+  output = Fast::SQL.replace('_', ast) do |root|
+    sb = root.loc.expression.source_buffer
+    sb.tokens.each do |token|
+      if eligible_kw.include?(token.keyword_kind) || eligible_tokens.include?(token.token)
+        range = Fast::Source.range(sb, token.start, token.end)
+        replace(range, range.source.upcase)
+      end
+    end
+  end
+  require 'fast/cli'
+  puts Fast.highlight(output, sql: true)
+end
+
+# Anonymize SQL
+# fast .anonymize_sql file.sql
+Fast.shortcut :anonymize_sql do
+  require 'fast/sql'
+  file = ARGV.last
+  method = File.exist?(file) ? :parse_sql_file : :parse_sql
+  ast = Fast.public_send(method, file)
+  memo = {}
+
+  relnames = search("(relname $_)", ast).grep(String).uniq
+  pattern = "{relname (sval {#{relnames.map(&:inspect).join(' ')}})}"
+  puts "searching with #{pattern}"
+
+  content = Fast::SQL.replace(pattern, ast) do |node|
+    new_name = memo[node.source.tr(%|"'|, '')] ||= "x#{memo.size}"
+    new_name = "'#{new_name}'" if node.type == :sval
+    replace(node.loc.expression, new_name)
+  end
+  puts Fast.highlight(content, sql: true)
+end
+
+# Give all details in a shorter format
+# fast .summary file.rb
+Fast.shortcut :summary do
+  file = ARGV.reverse.find { |arg| !arg.start_with?('-') && File.exist?(arg) }
+  if file && File.exist?(file)
+    Fast.summary(IO.read(file), file: file, level: fast_option_value(ARGV, '-l', '--level')).summarize
+  else
+    puts "Please provide a valid file to summarize."
+  end
+end
+
+# Group and classify multiple Ruby files without printing full bodies
+# fast .scan lib app/models --no-color
+Fast.shortcut :scan do
+  locations = ARGV.select { |arg| !arg.start_with?('-') && File.exist?(arg) }
+  if locations.any?
+    Fast.scan(locations, level: fast_option_value(ARGV, '-l', '--level')).scan
+  else
+    puts "Please provide at least one valid file or directory to scan."
+  end
+end
+
+def fast_option_value(args, short_name, long_name)
+  args.each_with_index do |arg, index|
+    return args[index + 1] if arg == short_name || arg == long_name
+    return arg.split('=', 2).last if arg.start_with?("#{long_name}=")
+  end
+
+  nil
+end

data/README.md

@@ -57,18 +57,18 @@ For example, let's take an `Integer` in Ruby:
 1
 ```
 
-It's corresponding s-expression would be:
+Its corresponding s-expression would be:
 
 ```ruby
 s(:int, 1)
 ```
 
-`s` in `Fast` and `Parser` are a shorthand for creating an `Parser::AST::Node`.
+`s` in `Fast` is a shorthand for creating a `Fast::Node`.
 Each of these nodes has a `#type` and `#children` contained in it:
 
 ```ruby
 def s(type, *children)
-  Parser::AST::Node.new(type, children)
+  Fast::Node.new(type, children: children)
 end
 ```
 
@@ -198,7 +198,7 @@ def my_method
 end
 ```
 
-It's corresponding s-expression would be:
+Its corresponding s-expression would be:
 
 ```ruby
 ast =
@@ -372,7 +372,7 @@ ast = Fast.ast("def name; person.name end")
 Generally, we  use the `location.expression`:
 
 ```ruby
-ast.location.expression # => #<Parser::Source::Range (string) 0...25>
+ast.location.expression # => #<Fast::Source::Range (string) 0...25>
 ```
 
 But location also brings some metadata about specific fragments:
@@ -383,7 +383,7 @@ ast.location.instance_variables # => [:@keyword, :@operator, :@name, :@end, :@ex
 
 Range for the keyword that identifies the method definition:
 ```ruby
-ast.location.keyword # => #<Parser::Source::Range (string) 0...3>
+ast.location.keyword # => #<Fast::Source::Range (string) 0...3>
 ```
 
 You can always pick the source of a source range:
@@ -395,7 +395,7 @@ ast.location.keyword.source # => "def"
 Or only the method name:
 
 ```ruby
-ast.location.name # => #<Parser::Source::Range (string) 4...8>
+ast.location.name # => #<Fast::Source::Range (string) 4...8>
 ast.location.name.source # => "name"
 ```
 
@@ -680,6 +680,20 @@ code("a = 1") # => s(:lvasgn, s(:int, 1))
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
+## Documentation locally
+
+The documentation site is built with MkDocs Material and a few Markdown
+extensions. To run it locally:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+python3 -m pip install -r requirements-docs.txt
+mkdocs serve
+```
+
+Then open `http://127.0.0.1:8000`.
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/jonatas/fast. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.

data/bin/console

@@ -6,7 +6,7 @@ require 'fast'
 require 'fast/sql'
 
 def s(type, *children)
-  Parser::AST::Node.new(type, children)
+  Fast::Node.new(type, children: children)
 end
 
 def code(string)

data/bin/fast-experiment

@@ -8,6 +8,7 @@ require 'fast/experiment'
 require 'coderay'
 
 arguments = ARGV
+autoclean = arguments.delete('--autoclean')
 
 experiment_files = Fast.ruby_files_from(File.expand_path('../experiments', __dir__))
 experiment_files.each(&method(:require))
@@ -23,6 +24,7 @@ end
 if arguments.any?
   ruby_files = arguments.all? { |e| File.exist?(e) && e.end_with?('.rb') }
   experiments.each do |experiment|
+    experiment.autoclean = autoclean
     if ruby_files
       experiment.files = arguments
     else
@@ -30,4 +32,5 @@ if arguments.any?
     end
   end
 end
+experiments.each { |experiment| experiment.autoclean = autoclean }
 experiments.each(&:run)

data/bin/fast-mcp

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path('../../lib', __FILE__))
+require 'fast/mcp_server'
+
+Fast::McpServer.run!

data/fast.gemspec

@@ -17,7 +17,7 @@ Gem::Specification.new do |spec|
   spec.license       = 'MIT'
 
   spec.files         = `git ls-files -z`.split("\x0").reject do |f|
-    f.match(%r{^(test|spec|features|docs\/(assets|stylesheets))/})
+    f.match(%r{^(test|spec|experiments|examples|features|docs|assets|stylesheets|site)/})
   end
 
   spec.post_install_message = <<~THANKS
@@ -42,10 +42,8 @@ Gem::Specification.new do |spec|
   spec.executables   = %w[fast fast-experiment]
   spec.require_paths = %w[lib experiments]
 
-  spec.add_dependency 'astrolabe'
   spec.add_dependency 'coderay'
   spec.add_dependency 'parallel'
-  spec.add_dependency 'parser'
   spec.add_dependency 'pg_query'
 
   spec.add_development_dependency 'bundler'

data/ideia_blog_post.md

@@ -0,0 +1,36 @@
+# Empowering LLM Agents with Natural Language to Fast Patterns
+
+Today I'm excited to announce a significant set of improvements to `fast`, specifically designed to bridge the gap between human reasoning (and LLM agents) and the technical precision of Ruby AST searching.
+
+## The Problem: AST Patterns are Hard
+
+Searching through code using AST patterns is incredibly powerful, but constructing the right S-expression can be daunting. Whether you're a human or an AI agent, getting the syntax just right—especially for complex queries—often involves a lot of trial and error.
+
+## The Solution: NL-to-Fast Translation
+
+We've introduced three new tools to solve this:
+
+1.  **The `fast-pattern-expert` Skill**: A specialized Gemini CLI skill that provides deep guidance, syntax references, and few-shot examples for translating structural descriptions of Ruby code into valid `Fast` patterns.
+2.  **`validate_fast_pattern` MCP Tool**: A new tool for our Model Context Protocol (MCP) server that allows agents to verify their generated patterns before executing them.
+3.  **`--validate-pattern` CLI Flag**: A quick way for humans to check if their pattern syntax is correct, checking for balanced nesting and valid token types.
+
+### Example in Action:
+**Query:** "Find all classes that inherit from `BaseService` and have a `call` method."
+**Pattern:** `[(class _ (const nil BaseService)) (def call)]`
+
+## Robustness and Precision
+
+We've also beefed up the core of `fast`:
+- **Stricter Validation**: The `ExpressionParser` now tracks nesting levels and unconsumed tokens, providing helpful error messages instead of silently failing or returning partial matches.
+- **Improved `...` Matcher**: The `...` literal now acts as a "rest-of-children" matcher when used at the end of a pattern, making it much easier to match methods with any number of arguments.
+- **Graceful Degradation**: Multi-file scans (`.scan`) now degrade gracefully if a single file fails to parse, ensuring your reconnaissance isn't aborted by one unsupported syntax node.
+
+## A Safer Release Workflow
+
+On the operational side, I'm moving our gem release process to **GitHub Actions**. By releasing from a clean CI sandbox instead of my local machine, we ensure a much higher level of security and reproducibility. This move minimizes the risk of local environment contamination and provides a transparent, auditable path from source to RubyGems.
+
+## Release 0.2.4
+
+To celebrate these features, we are releasing version `0.2.4` today!
+
+Happy searching!