yarsh 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5273b4e67dcb97cfa9dd5af63b83cfe7aa1d47cf27803baf27a0e35544346c19
-  data.tar.gz: 195f587ad4ee4dda6890520fdf2f2be20c7bdb2a6ab92f6d709dc24aaac5a7ad
+  metadata.gz: 5c0badb7f9f245222eb3ff552fff43848240d14e3e8d8efddae83004a77d72ab
+  data.tar.gz: d2920f423aac65f6dec454ebf84104fb97c60923358790440463bee0ada9cd8b
 SHA512:
-  metadata.gz: d0b448b5c4fbdec98e05c3519989e5ff74f8b8a7a5372a7d8772196ced90b15a64a5be7b97af6a7b954d9c08d046172e1c60ebb348018552f56fd09ca85aa8ee
-  data.tar.gz: b177ff3a557c6acdf842e2e29af9dcba09a118926b7d43c7d74507e5877952bef9bdf5faeab6b5f8f210d45ad1e2ce13eb8f01bf5593e08dbbabb464d42aa767
+  metadata.gz: '097af7754f04c9f8ec210fcd844d745b01e5aa681f36619e47ec3ae979ac00f162e0579085acb8c7f5cbb7a6515a38fad5ee973456618bc32b8cd78f732f3f26'
+  data.tar.gz: 891cd97331a879529d3ad545fba07590c0d3ca1d09e295550d96444bda867bd69f9f10cf494ec0cf61944978813831422e9162826b0415c339434022c44db15e

data/AGENTS.md

@@ -15,13 +15,23 @@ A hybrid Ruby/shell interactive REPL. Input is evaluated as Ruby first; on
 
 ## Architecture
 
-- **Main logic** lives in `lib/yarsh.rb` (module `Yarsh` with `self.shell`,
-  `self.console`, `self.execute`, `self.shell_path?`, `self.execute_pipe`, etc.).
-  `bin/console` calls `Yarsh.console` (IRB session); `exe/yarsh` calls `Yarsh.shell`.
-- `exe/yarsh` is the gem executable (packaged via `spec.bindir = "exe"` in the
-  gemspec). `bin/console` is for development only (adds `bundler/setup`).
-- `Yarsh.shell_path?` recognizes absolute/relative/home paths for shell execution.
-- `Yarsh.execute_pipe` enables piping shell output into Ruby via the `-->` operator.
+- **Entry point** is `exe/yarsh`: it `extend`s `Yarsh::InstanceMethods` onto the
+  top-level object, creates `Yarsh::Shell.new(binding)`, and calls `.shell`
+  (the REPL loop). `bin/console` is for development only (adds `bundler/setup`
+  and starts an IRB session via `Yarsh.console`).
+- **`lib/yarsh.rb`** defines the module-level helpers: constants (`YARSH_DIR`,
+  `HISTORY_FILE`, `CONFIG_FILE`), `ExecOutput`, `self.current_shell`,
+  `self.expand_aliases`, `shell_path?` (absolute/relative/home path detection,
+  mixed into `Shell`), `self.execute_system_command` (Open3, dead code), and
+  `self.console`.
+- **`Yarsh::Shell`** (`lib/yarsh/shell.rb`) holds the REPL loop (`shell`),
+  the prompt/history/completion wiring (`setup_reline_*`), the evaluation
+  pipeline (`execute_multiline`, `yarsh_eval`, `execute_pipe`, `__sh_exec`),
+  `cd`, `shell_command?`, and `execute` (`system(cmd, exception: true)`).
+- Multiline input lives in `lib/yarsh/multiline.rb` (`Yarsh::Multiline`):
+  `incomplete?` decides when to keep reading, `transform` rewrites shell lines
+  into `__sh__`/`__yarsh__` calls, `classify_line` classifies each line. The
+  active shell is exposed as `Yarsh.current_shell` for `__sh__` dispatch.
 
 ## `-->` pipe operator
 
@@ -39,15 +49,72 @@ Examples:
 | `ls --> lines.count` | (file count) |
 | `cat data.txt --> _.split("\n").grep(/error/)` | matching lines |
 
-## REPL evaluation flow
+## Multiline input
+
+Input is read via `Reline.readmultiline`; the REPL keeps reading while the
+buffer is syntactically incomplete (`Yarsh::Multiline.incomplete?` checks the
+`SyntaxError` from `RubyVM::InstructionSequence.compile` for unterminated
+constructs: missing `end`, strings, heredocs, regexps, operators, `case`
+clauses). Known-bad syntax (`@@@`) is not incomplete. Shell-command-shaped
+lines (`cd /tmp`, `grep foo /etc/passwd`) never enter multiline mode even
+though they fail to compile as Ruby (unterminated regexp) — `shell_command?`
+is checked first. An empty line forces submission; continuation lines get a
+`  > ` prompt via `Reline.prompt_proc`.
+
+The finished buffer is rewritten by `Yarsh::Multiline.transform` (each line
+classified 1:1 by `classify_line`), then evaluated via `@bind.eval`:
+
+- `:structural` — lines whose `Ripper.lex` tokens include a structural
+  keyword (`do`, `if`, `def`, `class`, `when`, …) are kept verbatim, but an
+  opening structural line gets `bind = binding` appended (nested structures
+  get `bind = binding + bind`) so inner lines can reach outer locals
+- `:yarsh` — every other non-blank line (shell lines *and* Ruby expressions
+  inside a structural block) is rewritten to
+  `__yarsh__("...", bind|nil)`; local variables assigned in an outer
+  structural line are wrapped as `__yarsh__("name", bind)` references
+  (`IdentifierSurround`)
+- `:skip` — blank lines are dropped; heredoc bodies are never rewritten
+- `:shell` — reserved for `__sh__("...")` calls (currently unemitted, see
+  Known issues)
+
+`__yarsh__` (`Yarsh::InstanceMethods`, extended onto the top-level binding in
+`exe/yarsh`) temporarily swaps `Shell#bind` for the block binding and calls
+`Shell#yarsh_eval`, which handles `-->` pipes, shell paths, `cd`, and Ruby
+evaluation with the shell fallback (see REPL evaluation flow). Shell lines
+therefore work inside Ruby blocks, defs, and classes; the whole buffer is
+evaluated in the shared REPL binding so methods/variables persist. Examples:
+
+| Input | Result |
+|---|---|
+| `Dir['*.rb'].each do \|f|` … `ls -l` … `puts f` … `end` | runs `ls -l` for each file |
+| `def hi` … `puts 'yo'` … `end` then `hi` | `yo` (def persists) |
 
-All steps below are wrapped in a single `begin..rescue` so no error crashes the REPL:
+Known limitations:
+
+- Pure-bash multiline (e.g. bash `if ...; then`) is not supported; shell
+  lines only work inside Ruby structures.
+- Shell execution inside a block relies on the `NameError` fallback of
+  `yarsh_eval`, so a Ruby expression that raises a real `NameError` may be
+  shell-executed instead of reported.
+
+## REPL evaluation flow
 
-1. `input.include?("-->")` → `Yarsh.execute_pipe(input)`, print result
-2. `Yarsh.shell_path?(input)` → execute directly via `bash -c`
-3. `bind.eval(input)` — Ruby evaluation
-4. `NameError` → if first token is `cd`, call `Dir.chdir`; else shell-execute
-5. Other exceptions → print `Error: ExceptionClass: message`
+All steps below are wrapped in a single `begin..rescue` in `Shell#shell` so no
+error crashes the REPL:
+
+1. Input is read via `Reline.readmultiline` (see Multiline input above);
+   `read_input` keeps reading while `incomplete_line?` (incomplete and not a
+   shell command) or until a blank line ends the buffer
+2. The whole input goes through `execute_multiline`: `Yarsh::Multiline.transform`
+   rewrites non-structural lines to `__yarsh__(...)` calls (structural lines
+   kept verbatim), then `@bind.eval` runs the buffer; per-line execution
+   happens inside `Shell#yarsh_eval` / `Shell#__sh_exec`
+3. `yarsh_eval`: `-->` pipe → `execute_pipe`; shell path (`shell_path?`) →
+   `execute`; `cd`/`cd <path>` → `Dir.chdir`; otherwise `bind.eval` — on
+   `NameError`/`SyntaxError`/`ArgumentError` it falls back to aliases +
+   shell execution (unless the input looks like a Ruby error, then it is
+   reported)
+4. Other exceptions → print `Error: ExceptionClass: message`
 
 ## Error handling
 
@@ -60,12 +127,17 @@ All steps below are wrapped in a single `begin..rescue` so no error crashes the
 
 ## Known issues
 
-- **Bug:** `Yarsh.execute` (in `lib/yarsh.rb`) uses `system("bash -c 'source
-  ~/.bashrc; #{cmd}'")` — the command is passed correctly, but this depends on
-  `~/.bashrc` existing.
-- **Dead code:** `Yarsh.execute_system_command` (Open3 version) is defined but
-  never called. `Yarsh.shell` calls `Yarsh.execute`, not
-  `Yarsh.execute_system_command`.
+- **Dead code:** `Yarsh.execute_system_command` (Open3 version) in
+  `lib/yarsh.rb` is defined but never called; shell execution goes through
+  `Shell#execute` (`system(cmd, exception: true)`).
+- **Bug risk:** `setup_reline_history` replays history with
+  `@core.readmultiline`-independent heuristics (`incomplete_line?`),
+  which may mishandle historical shell-command lines (see Multiline input).
+- **Unemitted `:shell`:** `Multiline.transform` never emits `:shell`/`__sh__`
+  (the `classify_line` branches are commented out behind FIXMEs); shell lines
+  and Ruby expressions both go through `__yarsh__`, with the shell fallback of
+  `yarsh_eval` deciding. `__sh__`/`__sh_exec` still exist for potential
+  re-enabling.
 
 ## Style
 

data/CHANGES.md

@@ -26,19 +26,19 @@ a shell command, matching the existing `NameError` fallback behavior.
 
 ### REPL history persisted to `~/.yarsh/history`
 
-(53032be) Commands are saved to `~/.yarsh/history` across sessions. Max 1000 lines;
+Commands are saved to `~/.yarsh/history` across sessions. Max 1000 lines;
 concurrent-session safe via `File.flock`.
-(77af3b8) Add tests for append_history: creation, append, trim, concurrent safety.
+Add tests for append_history: creation, append, trim, concurrent safety.
 
 ### `bin/console` launches an IRB session
 
-(33f86c8) `bin/console` now starts a standard IRB session with the gem loaded
+`bin/console` now starts a standard IRB session with the gem loaded
 (`Yarsh.console`). The hybrid REPL remains available via `bundle exec ruby exe/yarsh`
 (`Yarsh.shell`).
 
 ### Test suite adapted to multi-file architecture
 
-(13655c4) Tests split per class: `test_yarsh.rb` (module-level: constants,
+Tests split per class: `test_yarsh.rb` (module-level: constants,
 `ExecOutput`, `console`), `test_shell.rb` (`Yarsh::Shell`: `shell_path?`, `cd`,
 completion, history, `execute_pipe` incl. bind-var feature), `test_config.rb`
 (`Yarsh::Config`: prompt, `log_level`), `test_prompt.rb` (`Prompt`,
@@ -47,14 +47,14 @@ validation in `Config#prompt=`.
 
 ### Console test works under minitest 6
 
-(e3d1f44) `test_console_launches_irb_session` no longer uses
+`test_console_launches_irb_session` no longer uses
 `IRB.stub` (`minitest/mock` was removed in minitest 6, breaking plain
 `rake test`); it now stubs `IRB.start` via `define_singleton_method`.
 README documents running tests with `bundle exec rake test`.
 
 ### `expand_aliases` no longer strips whitespace
 
-(ab3998e) Registering the first alias via `sh_alias`/`add_alias` ran the
+Registering the first alias via `sh_alias`/`add_alias` ran the
 command through `expand_aliases` with an empty alias hash; the empty
 alternation degraded the regex to `(^| +)() *` and every space was
 deleted (`exa -l --icons=always` was stored as `exa-l--icons=always`).
@@ -66,7 +66,7 @@ with a regression test for the empty-alias case.
 
 ### `Yarsh::InvalidPromptError` for invalid prompt configuration
 
-(4f0364e) `Config#prompt=` raises the new `Yarsh::InvalidPromptError`
+`Config#prompt=` raises the new `Yarsh::InvalidPromptError`
 (subclass of `Yarsh::Error`) instead of a bare `StandardError`. The
 message is a constructor argument with the documented default.
 `Yarsh::Error` is now defined before sub-files are required, fixing a
@@ -75,7 +75,7 @@ assert the error class, the default message, and custom messages.
 
 ### Logging methods in `InstanceMethods`
 
-(d867a11) Besides the existing `debug`, the REPL now exposes `info`,
+Besides the existing `debug`, the REPL now exposes `info`,
 `warn`, `error`, `fatal`, and `unknown` — each delegating to
 `Config.instance.logger`, covering all `Logger` severities. Methods are
 defined dynamically from the `SEVERITIES` constant
@@ -83,7 +83,7 @@ defined dynamically from the `SEVERITIES` constant
 
 ### 256-color and text styles in `AnsiColorFormatter`
 
-(3883da3) New `fg_color256`/`bg_color256` emit 8-bit indexed color
+New `fg_color256`/`bg_color256` emit 8-bit indexed color
 codes (`38;5;N` / `48;5;N`) and raise `Yarsh::Error` for indices outside
 0..255. Text styles are defined dynamically from `STYLE_CODES`: `bold`,
 `dim`, `italic`, `underline`, `blink`, `reverse`, `hidden`,
@@ -93,7 +93,7 @@ now tolerates calls with neither text nor a block, so bare style calls
 
 ### Real `ArgumentError`s are no longer shell-executed
 
-(13ea85f) Shell commands like `bundle exec rake test` raise
+Shell commands like `bundle exec rake test` raise
 `ArgumentError` (the innermost call resolves to `Kernel#test`, which
 requires 2 args). The fallback previously treated every `ArgumentError`
 as a shell command, so genuine Ruby errors (`"x".sub`) were
@@ -105,10 +105,99 @@ keep their unconditional shell fallback.
 
 ### Renamed from `rsh` to `yarsh`
 
-(9009bfb) The project is now called **yarsh** throughout: module `Rsh`
+The project is now called **yarsh** throughout: module `Rsh`
 → `Yarsh`, gem name `yarsh` (`spec.name`, require paths), data dir
 `~/.rsh` → `~/.yarsh`, executable `exe/ruby-shell` → `exe/yarsh`.
 Files renamed: `rsh.gemspec`, `lib/rsh.rb`, `lib/rsh/`,
 `sig/rsh.rbs`, `test/test_rsh.rb`. All references updated (tests,
 `bin/console`, README, AGENTS.md, CHANGES.md); `Gemfile.lock`
-regenerated. Version stays 0.1.0.
+regenerated.
+
+## 0.1.0
+
+First development release!
+
+* `eee1b3e` Initial commit for a clean git project
+* `da76411` and `b02a1b8` Updated README for better clarity and fix repository links
+
+## 0.1.1
+
+Small correction in the gem description
+
+* `5741a00` update the gem description
+
+## 0.1.2
+
+Bug fix and small improvement for the prompt type **powerline**
+
+See commits:
+
+* `d59834b`
+* `d81ffcb`
+* `3e93937`
+* `4b77901`
+
+## 0.2.0
+
+### Multi-line input with shell lines inside Ruby blocks
+
+Input is now read via `Reline.readmultiline` — the REPL keeps reading while the
+buffer is syntactically incomplete (`Yarsh::Multiline.incomplete?` checks the
+`SyntaxError` from `RubyVM::InstructionSequence.compile` for unterminated
+constructs: missing `end`, strings, heredocs, regexps, operators, `case`
+clauses). Known-bad syntax (`@@@`) is not treated as incomplete, and
+shell-command-shaped lines (`cd /tmp`, `grep foo /etc/passwd`) never enter
+multiline mode even though they fail to compile as Ruby (unterminated regexp).
+An empty line forces submission; continuation lines get a `  > ` prompt via
+`Reline.prompt_proc`.
+
+The finished buffer is rewritten by `Yarsh::Multiline.transform` (each line
+classified 1:1 by `classify_line`), then evaluated via `@bind.eval`:
+
+- Lines whose `Ripper.lex` tokens include a structural keyword (`do`, `if`,
+  `def`, `class`, `when`, …) are kept verbatim; an opening structural line
+  gets `bind = binding` appended (nested structures get `bind = binding +
+  bind`) so inner lines can reach outer locals
+- Every other non-blank line — shell lines *and* Ruby expressions inside a
+  structural block — is rewritten to `__yarsh__(...)` with the surrounding
+  binding; local variables assigned in an outer structural line are wrapped
+  as `__yarsh__("name", bind)` references (`IdentifierSurround`)
+- Blank lines are dropped; heredoc bodies are never rewritten
+
+`__yarsh__` (`Yarsh::InstanceMethods`, extended onto the top-level binding in
+`exe/yarsh`) temporarily swaps `Yarsh::Shell#bind` for the block binding and
+calls `Shell#yarsh_eval`, which handles `-->` pipes, shell paths, `cd`, and
+Ruby evaluation with the shell fallback. Shell lines therefore work inside
+Ruby blocks, defs, and classes; the whole buffer is evaluated in the shared
+REPL binding so methods and variables persist. Examples:
+
+```ruby
+Dir['*.rb'].each do |f|
+  ls -l
+  puts f
+end
+# runs `ls -l` for each file
+```
+
+```ruby
+def hi
+  puts 'yo'
+end
+hi
+#=> yo (def persists)
+```
+
+Known limitations:
+
+- Pure-bash multiline (e.g. bash `if ...; then`) is not supported; shell
+  lines only work inside Ruby structures.
+- Shell execution inside a block relies on the `NameError` fallback of
+  `yarsh_eval`, so a Ruby expression that raises a real `NameError` may be
+  shell-executed instead of reported.
+
+See commits: `bcc25b4`, `ea92742`, `51c9db8`, `2218f29`, `0fceeb8`. Tests in
+`test/test_multiline.rb` (completeness detection, line classification,
+transformation, heredoc handling, binding threading, integration with
+`Yarsh::Shell`).
+
+

data/README.md

@@ -4,7 +4,7 @@ Yet Another Ruby SHell is a mix of Ruby and standard Shell REPL. Priority is giv
 
 This project was made just for fun, to remember the basis of Ruby and to discover new features. To provide a good user experience, it uses `reline` under the hood which already implement all we want for a REPL: History, completion, etc.
 
-I wanted a simple version but already working version. If I have time, more features will come. For example, multi-line editing or non interactive with a file as argument. I dream of a shell syntax we can do:
+I wanted a simple version but already working version. If I have time, more features will come. For example, non interactive with a file as argument. I dream of a shell syntax we can do:
 
 ```ruby
 def install_rails
@@ -16,8 +16,15 @@ end
 install_rails
 ```
 
+And this is now a reality: multi-line editing is supported, and shell commands
+work inside Ruby blocks, defs, and classes (see [Multiline](#multiline)).
+
 I developed this project half of the time offline, another part with the help of AI. It was a way for me to try to develop with AI. I used `opencode` with DeepSeek V4 Flash Free.
 
+## install
+
+Run `gem install yarsh`
+
 ## Usage
 
 `yarsh` start the shell.
@@ -55,6 +62,49 @@ ls -al . -->(sorted) lines.sort {|a, b| a.length <=> b.length  }
 sorted.each { |file| puts "What ever you wanna do" }
 ```
 
+### Multiline
+
+Multi-line input is supported: the REPL keeps reading while the buffer is
+syntactically incomplete (missing `end`, unterminated string/heredoc/regexp,
+trailing operator, ...) and submits when the syntax is complete or on an empty
+line. Continuation lines get a `  > ` prompt.
+
+The best part: you can mix Ruby and shell lines in the same block, def, or
+class:
+
+```ruby
+def install_rails
+  sudo apt update
+  sudo apt install ruby
+  gem install rails
+end
+```
+
+```ruby
+Dir['*.rb'].each do |f|
+  ls -l
+  puts "Processed: #{f}"
+end
+```
+
+The whole buffer is evaluated in the shared REPL binding, so methods,
+variables, and shell side-effects persist between lines and across inputs.
+Shell lines are executed via the usual fallback (they are evaluated as Ruby
+first, then as a shell command). Heredoc bodies are kept verbatim:
+
+```ruby
+x = <<~EOF
+  not evaluated, just a string
+EOF
+```
+
+Known limitations:
+
+- Pure-bash multiline (e.g. bash `if ...; then`) is not supported; shell
+  lines only work inside Ruby structures.
+- A Ruby expression that raises a real `NameError` inside a block may be
+  executed as a shell command instead of being reported.
+
 ### Completion
 
 For the moment, there is autocompletion for exectuables in PATH and for file and dir path for shell commands

data/lib/yarsh/binding.rb

@@ -0,0 +1,10 @@
+class Binding
+  def +(other)
+    return self if other.nil?
+
+    other.local_variables.each do |var|
+      local_variable_set(var, other.local_variable_get(var))
+    end
+    self
+  end
+end

data/lib/yarsh/instance_methods.rb

@@ -8,6 +8,29 @@ module Yarsh
       Config.instance.add_alias(new_name, command)
     end
 
+    # Executes a shell line from inside a multiline Ruby buffer. Shell lines
+    # are rewritten to __sh__(...) by Yarsh::Multiline.transform.
+    def __sh__(command)
+      shell = Yarsh.current_shell
+      raise Error, 'No active Yarsh shell' if shell.nil?
+
+      shell.send(:__sh_exec, command)
+    end
+
+    def __yarsh__(exp, binding = nil)
+      shell = Yarsh.current_shell
+      raise Error, 'No active Yarsh shell' if shell.nil?
+
+      current_bind = nil
+      if binding
+        current_bind = shell.bind
+        shell.bind = binding
+      end
+      result = shell.send(:yarsh_eval, exp)
+      shell.bind = current_bind if binding
+      result
+    end
+
     def source(filename)
       load(filename, Yarsh::InstanceMethods)
     end

data/lib/yarsh/multiline.rb

@@ -0,0 +1,147 @@
+# frozen_string_literal: true
+
+module Yarsh
+  # Multiline input support: decides when a line is syntactically incomplete
+  # (keep reading) and transforms a finished buffer so shell lines can live
+  # inside Ruby blocks/defs/classes: shell lines are rewritten into __sh__ calls.
+  module Multiline
+    INCOMPLETE_MESSAGES = [
+      /unexpected end-of-input/,
+      /unterminated string/,
+      /unterminated heredoc/,
+      /unterminated regexp/,
+      /can't find string ".*" anywhere before EOF/,
+      /expected a `when` or `in` clause after `case`/,
+      /expected an `end` to close the/
+    ].freeze
+
+    # Keywords that must never be treated as shell, even though some of them
+    # (break, next, return, times, ...) are also bash builtins.
+    STRUCTURAL_KEYWORDS = %w[do end if else elsif when rescue ensure then begin
+                             case class def module while until for unless
+                             return break next yield].freeze
+    KEYWORD_REGEX = Regexp.new("\\A(?:#{STRUCTURAL_KEYWORDS.join('|')})\\b")
+
+    class IdentifierSurround
+      def initialize
+        @local_vars = {}
+      end
+
+      # detect if the line assign a new local variable
+      # if yes, save it's name
+      def save_local_vars(line)
+        tokens = Prism.lex(line).value.map { |token, _| token }
+        return unless tokens.any? { |token| token.type == :EQUAL }
+
+        tokens
+          .take_while { |token| token.type != :EQUAL }
+          .filter { |token| token.type == :IDENTIFIER }
+          .each do |token|
+          @local_vars[token.value] =
+            token
+        end
+      end
+
+      # surroud the local variable with 'bind.eval()'
+      # a local variable is an identifier who was registred
+      # in a precedent line
+      def surrond_with_bind(line, bind_name)
+        Prism.lex(line).value
+             .map { |token, _| token }
+             .filter { |token| token.type == :IDENTIFIER }
+             .each do |token|
+               next unless @local_vars[token.value]
+
+               location = token.location
+               line[location.start_column..(location.end_column - 1)] =
+                 "__yarsh__(\"#{token.value}\", #{bind_name})"
+             end
+        line
+      end
+    end
+
+    def self.incomplete?(src)
+      RubyVM::InstructionSequence.compile(src)
+      false
+    rescue SyntaxError => e
+      INCOMPLETE_MESSAGES.any? { |re| e.message =~ re }
+    end
+
+    # :ruby   - keep the line verbatim
+    # :shell  - rewrite the line into a __sh__ call
+    # :skip   - drop the line (blank)
+    def self.classify_line(line)
+      stripped = line.strip
+      return :skip if stripped.empty?
+      # return :ruby if structural?(stripped)
+      # return :shell if stripped.include?('-->') #FIXME: Should not be treated as a shell --> should allow multiline too
+      # return :shell if shell_command && shell_command.call(stripped) #FIXME: pwd = Dir.pwd is evaluated as a shell command instead of a ruby
+      # return :ruby if incomplete?(stripped)
+      # return :ruby if compiles?(stripped)
+      # #TODO: check what happend when it do not compile with another error message catched by incomplete? ?
+
+      # :shell
+      return :structural if structural?(stripped)
+
+      # return :shell if !compiles?(stripped) && shell_command&.call(stripped)
+
+      :yarsh
+    end
+
+    def self.transform(buffer, &shell_command)
+      heredoc_delim = nil
+      is_structural = false
+      print_binding = 'bind = binding'
+      ident_surround = IdentifierSurround.new
+      buffer.split("\n").map do |line|
+        if heredoc_delim
+          heredoc_delim = nil if line.strip == heredoc_delim
+          next line
+        end
+        if (delim = heredoc_delimiter(line))
+          heredoc_delim = delim
+          next line
+        end
+        case classify_line(line, &shell_command)
+        when :skip then nil
+        when :structural
+          next line if line.include? 'end'
+
+          if is_structural
+            line = ident_surround.surrond_with_bind(line, 'bind')
+            print_binding = 'bind = binding + bind'
+          end
+          is_structural = true
+          line + "\n" + print_binding
+        when :shell then "__sh__(#{line.strip.inspect})"
+        when :yarsh
+          ident_surround.save_local_vars(line)
+          "__yarsh__(#{line.strip.inspect}, #{is_structural ? 'bind' : 'nil'})"
+        end
+      end.join("\n")
+    end
+
+    def self.structural?(line)
+      # line.strip.match?(KEYWORD_REGEX)
+      Ripper.lex(line).any? { |token| STRUCTURAL_KEYWORDS.include?(token[2]) }
+    end
+
+    def self.compiles?(src)
+      RubyVM::InstructionSequence.compile(src)
+      true
+    rescue SyntaxError
+      false
+    end
+
+    def self.heredoc_delimiter(line)
+      return nil if line.include?('-->')
+      return nil unless line.include?('<<')
+
+      m = line.match(/<<(-|~)?(['"])([^'"]+)\2(?:\s*\.\w+)?\s*\z/)
+      return m[3] if m
+
+      m = line.match(/<<(-|~)?([A-Za-z_]\w*)(?:\s*\.\w+)?\s*\z/)
+      m && m[2]
+    end
+  end
+end

data/lib/yarsh/prompt.rb

@@ -8,8 +8,15 @@ module Yarsh
       seg = Dir.pwd.gsub(/^#{Dir.home}/, '~').split('/')
       seg = (seg.size > 3 ? [' …'] + seg[-3..-1] : seg)
       git_branch = ''
-      git_branch = `git status --porcelain -b`.split("\n")&.first&.gsub('## ', '') || '' if File.directory?('.git')
-      git_branch = '   ' + git_branch unless git_branch.empty?
+      if File.directory?('.git')
+        git_branch = `git status --porcelain -b`
+                     .split("\n")
+                     &.first
+                     &.gsub('## ', '')
+                     &.split('...', 2)
+                     &.first || ''
+      end
+      git_branch = '  ' + git_branch unless git_branch.empty?
       c.bg_color256(base_color)
        .fg_white("#{seg.join('  ')}#{git_branch}")
        .reset

data/lib/yarsh/shell.rb

@@ -1,10 +1,13 @@
 # frozen_string_literal: true
+
 module Yarsh
   class Shell
     include Yarsh
     include InstanceMethods
 
-    SHELL_BUILTINS = %w[alias bg bind break builtin cd command echo eval exec exit
+    attr_accessor :bind
+
+    SHELL_BUILTINS = %w[alias bg bind break builtin command echo eval exec exit
                         export false fc fg getopts hash help history jobs kill let
                         local logout mapfile popd printf pushd pwd read readonly
                         return set shift shopt source suspend test times trap true
@@ -16,8 +19,10 @@ module Yarsh
       @history_file = File.expand_path(HISTORY_FILE)
       @config_file = File.expand_path(CONFIG_FILE)
       @cf = Yarsh::AnsiColorFormatter.new
+      Yarsh.current_shell = self
       setup_reline_history
       setup_reline_completion
+      setup_reline_prompt
     end
 
     def shell
@@ -25,7 +30,7 @@ module Yarsh
       load @config_file, InstanceMethods if File.exist? @config_file
       loop do
         input = begin
-          @core.readline(Config.instance.prompt, true)
+          read_input
         rescue Interrupt
           next
         end
@@ -37,42 +42,7 @@ module Yarsh
         append_history(input, @history_file)
 
         begin
-          if input.include?('-->')
-            fp execute_pipe(input, @bind)
-            next
-          end
-
-          if shell_path?(input)
-            execute(input)
-            next
-          end
-
-          if input == 'cd' || input.start_with?('cd ')
-            cd(input.split(' ', 2)[1])
-            next
-          end
-
-          begin
-            result = @bind.eval(input)
-            fp result
-          rescue NameError, SyntaxError, ArgumentError => rb_error
-            unless shell_command?(input) || rb_error.is_a?(NameError) || rb_error.is_a?(SyntaxError)
-              $rber = rb_error
-              puts @cf.fg_red("Error: ArgumentError: #{rb_error.message}")
-              next
-            end
-            info("Assuming it is not a ruby expression, the error: #{rb_error.message}")
-            debug("The detail: #{rb_error.backtrace.join("\n")}")
-            begin
-              input = Yarsh.expand_aliases(input, Config.instance.aliases)
-              execute(input)
-            rescue Exception => e
-              $sher = e
-              info("#{e.class}: #{e.message}")
-              puts "#{@cf.bold.underline.fg_blue('Shell:')} '#{input}': #{e.message}"
-              puts "#{@cf.bold.underline.fg_red('Ruby:')} '#{input}': #{rb_error.message}"
-            end
-          end
+          fp execute_multiline(input)
         rescue Interrupt
           next
         rescue SystemExit
@@ -91,6 +61,74 @@ module Yarsh
     end
     alias fp format_output
 
+    def read_input
+      @core.readmultiline(Config.instance.prompt, true) do |buffer|
+        !incomplete_line?(buffer) || buffer.match?(/\n[ \t]*\n\z/)
+      end
+    end
+
+    # A line is "incomplete" (multiline continues) only when it is not a
+    # shell command: `cd /tmp`, `grep foo /etc/passwd` etc. fail to compile
+    # as Ruby (unterminated regexp) but must stay single-line.
+    def incomplete_line?(line)
+      Yarsh::Multiline.incomplete?(line) && !shell_command?(line)
+    end
+
+    def execute_multiline(input)
+      source = Yarsh::Multiline.transform(input) { |line| shell_command?(line) }
+      debug("What I will evaluate as multiline:\n#{@cf.bold.fg_blue(source)}")
+      @bind.eval(source)
+    end
+
+    # Evaluate a yarsh line. A yarsh line is a ruby expression or a shell syntax
+    def yarsh_eval(input)
+      debug("I want to evaluate the input: #{input}")
+      return execute_pipe(input, @bind) if input.include?('-->')
+
+      return execute(input) if shell_path?(input)
+
+      if input == 'cd' || input.start_with?('cd ')
+        cd(input.split(' ', 2)[1])
+        return nil
+      end
+
+      begin
+        @bind.eval(input)
+      rescue NameError, SyntaxError, ArgumentError => rb_error
+        unless shell_command?(input) || rb_error.is_a?(NameError) || rb_error.is_a?(SyntaxError)
+          $rber = rb_error
+          return @cf.fg_red("Error: ArgumentError: #{rb_error.message}")
+        end
+        info("Assuming it is not a ruby expression, the error: #{rb_error.message}")
+        debug("The detail: #{rb_error.backtrace.join("\n")}")
+        begin
+          input = Yarsh.expand_aliases(input, Config.instance.aliases)
+          execute(input)
+        rescue Exception => e
+          $sher = e
+          info("#{e.class}: #{e.message}")
+          puts "#{@cf.bold.underline.fg_blue('Shell:')} '#{input}': #{e.message}"
+          puts "#{@cf.bold.underline.fg_red('Ruby:')} '#{input}': #{rb_error.message}"
+        end
+      end
+    end
+
+    # Runs a shell line from inside a multiline Ruby buffer (via __sh__).
+    def __sh_exec(command)
+      command = Yarsh.expand_aliases(command, Config.instance.aliases)
+      if command.include?('-->')
+        puts execute_pipe(command, @bind)
+      else
+        execute(command)
+      end
+    end
+
+    def setup_reline_prompt
+      Reline.prompt_proc = proc do |lines|
+        [Config.instance.prompt] + Array.new([lines.size - 1, 0].max, '  > ')
+      end
+    end
+
     def cd(path = '~')
       Dir.chdir(File.expand_path(path || '~'))
     end
@@ -177,7 +215,30 @@ module Yarsh
 
     def setup_reline_history
       Dir.mkdir(File.dirname(@history_file)) unless Dir.exist?(File.dirname(@history_file))
-      File.readlines(@history_file, chomp: true).each { |line| Reline::HISTORY << line } if File.exist?(@history_file)
+      return unless File.exist?(@history_file)
+
+      pending = nil
+      File.readlines(@history_file, chomp: true).each do |line|
+        next if line.strip.empty?
+
+        unless pending
+          pending = line
+          next
+        end
+        if incomplete_line?(pending)
+          candidate = "#{pending}\n#{line}"
+          if Yarsh::Multiline.incomplete?(candidate)
+            pending = candidate
+          else
+            Reline::HISTORY << candidate
+            pending = nil
+          end
+        else
+          Reline::HISTORY << pending
+          pending = line
+        end
+      end
+      Reline::HISTORY << pending if pending
     end
   end
 end

data/lib/yarsh/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Yarsh
-  VERSION = '0.1.1'
+  VERSION = '0.2.0'
 end

data/lib/yarsh.rb

@@ -4,13 +4,15 @@ require 'reline'
 require 'open3'
 require 'singleton'
 require 'logger'
+require 'ripper'
+require 'prism'
 
 module Yarsh
   class Error < StandardError; end
 end
 
-Dir['lib/yarsh/**/*.rb'].each do |f|
-  require_relative File.join('..', f)
+Dir[File.expand_path('**/*.rb', __dir__)].each do |f|
+  require_relative f
 end
 
 module Yarsh
@@ -34,6 +36,14 @@ module Yarsh
 
   class Error < StandardError; end
 
+  def self.current_shell
+    @current_shell
+  end
+
+  def self.current_shell=(shell)
+    @current_shell = shell
+  end
+
   # in a Shell input, find all command matching the keys of aliases and replace them with
   # the values. Leading and trailing whitespace around the match is preserved.
   def self.expand_aliases(input, aliases)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: yarsh
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.0
 platform: ruby
 authors:
 - yarsh4all
@@ -44,8 +44,10 @@ files:
 - exe/yarsh
 - lib/yarsh.rb
 - lib/yarsh/ansi_color_formatter.rb
+- lib/yarsh/binding.rb
 - lib/yarsh/config.rb
 - lib/yarsh/instance_methods.rb
+- lib/yarsh/multiline.rb
 - lib/yarsh/prompt.rb
 - lib/yarsh/shell.rb
 - lib/yarsh/version.rb