rstfilter 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: acaf57f79e7bee11c42958205a4970e6c85a2de855f82513076e40eba42a5d4d
4
- data.tar.gz: 540a4a20d663769298947079626f5ec8d4ff1d10d72d724d126d5a7a2f9400bd
3
+ metadata.gz: 493aaf26c129bbac2f2e56a62a59575e0805be44d4c48e2707da454cd51d2972
4
+ data.tar.gz: 164cbf3204c18b09c8dca6cb78c9059f13e49cd255c77fa793721479b337237a
5
5
  SHA512:
6
- metadata.gz: a3371ad347e2797e051a9c9ac0621eca32022aae6b41ef859c961928e3b1159235e64e75e307901ca8dc17dba27e9e9e56d8646af02683b1eddc8eb2b68184a1
7
- data.tar.gz: 7abd528a4b660261a2be171609191c16ee848348ad0e6adf7d7c5421ba3aabf4757be69657f3fb6c6b4195083c4c9102d85964a10d086189a843924a923684d2
6
+ metadata.gz: 0ff94c098b3f781a767c0887b30b8b24ebb58e909930ee70865d666f920d1ff50e9de7ed47a0072bd667d39da285682ef83591d81078c6f1d5cca63cc2b07a5c
7
+ data.tar.gz: 62b140c8dfde2082e39a132403f34d81b89a34c4993e814a3e847fed949ff5bf78e64125e3e7770268c4ee1d47d326252dbe25a9469a46cbb5a667bc360362e9
@@ -0,0 +1,38 @@
1
+ # This workflow uses actions that are not certified by GitHub.
2
+ # They are provided by a third-party and are governed by
3
+ # separate terms of service, privacy policy, and support
4
+ # documentation.
5
+ # This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
6
+ # For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
7
+
8
+ name: Ruby
9
+
10
+ on:
11
+ push:
12
+ branches: [ main ]
13
+ pull_request:
14
+ branches: [ main ]
15
+
16
+ permissions:
17
+ contents: read
18
+
19
+ jobs:
20
+ test:
21
+
22
+ runs-on: ubuntu-latest
23
+ strategy:
24
+ matrix:
25
+ ruby-version: ['2.7', '3.0', '3.1', 'head', 'debug']
26
+
27
+ steps:
28
+ - uses: actions/checkout@v3
29
+ - name: Set up Ruby
30
+ # To automatically get bug fixes and new Ruby versions for ruby/setup-ruby,
31
+ # change this to (see https://github.com/ruby/setup-ruby#versioning):
32
+ # uses: ruby/setup-ruby@v1
33
+ uses: ruby/setup-ruby@2b019609e2b0f1ea1a2bc8ca11cb82ab46ada124
34
+ with:
35
+ ruby-version: ${{ matrix.ruby-version }}
36
+ bundler-cache: true # runs 'bundle install' and caches installed gems automatically
37
+ - name: Run tests
38
+ run: bundle exec rake
data/README.md CHANGED
@@ -1,3 +1,5 @@
1
+ [![GH Actions](https://github.com/ko1/rstfilter/actions/workflows/ruby.yml/badge.svg)](https://github.com/ko1/rstfilter/actions/workflows/ruby.yml)
2
+
1
3
  # Rstfilter
2
4
 
3
5
  This tool prints a Ruby script with execution results.
@@ -9,7 +11,7 @@ b = 2
9
11
  c = a + b
10
12
  puts "Hello" * c
11
13
 
12
- $ rstfilter sample.rb -a
14
+ $ rstfilter sample.rb -o
13
15
  a = 1 #=> 1
14
16
  b = 2 #=> 2
15
17
  c = a + b #=> 3
@@ -41,12 +43,51 @@ Usage: rstfilter [options] SCRIPT
41
43
  -o, --output Show output results
42
44
  -d, --decl Show results on declaration
43
45
  --no-exception Do not show exception
44
- -a, --all Show all results/output
45
46
  --pp Use pp to represent objects
47
+ -n, --nextline Put comments on next line
46
48
  --comment-indent=NUM Specify comment indent size (default: 50)
49
+ --comment-pattern=PAT Specify comment pattern of -c (default: '#=>')
50
+ --coment-label=LABEL Specify comment label (default: "")
51
+ -e, --command=COMMAND Execute Ruby script with given command
52
+ --no-filename Execute -e command without filename
53
+ -j, --json Print records in JSON format
54
+ --ignore-pragma Ignore pragma specifiers
47
55
  --verbose Verbose mode
48
56
  ```
49
57
 
58
+ Note that you can specify multiple `-e` options like that:
59
+
60
+ ```
61
+ $ rstfilter -o sample.rb -eruby27:/home/ko1/.rbenv/versions/2.7.6/bin/ruby -e ruby30:/home/ko1/.rbenv/versions/3.0.4/bin/ruby
62
+ a = 1
63
+ #=> ruby27: 1
64
+ #=> ruby30: 1
65
+ b = 2
66
+ #=> ruby27: 2
67
+ #=> ruby30: 2
68
+ c = a + b
69
+ #=> ruby27: 3
70
+ #=> ruby30: 3
71
+ puts "Hello" * c
72
+ #=> ruby27: nil
73
+ #=> ruby30: nil
74
+ #ruby27:out: HelloHelloHello
75
+ #ruby30:out: HelloHelloHello
76
+ ```
77
+
78
+ On this case, you can check results on multiple Ruby interpreters.
79
+
80
+ ## Advanced demo
81
+
82
+ https://user-images.githubusercontent.com/9558/170426066-e0c19185-10e9-4932-a1ce-3088a4189b34.mp4
83
+
84
+ This video shows advanced usage to show the results with modified script immediately.
85
+
86
+ * [kv](https://rubygems.org/gems/kv) is another pager.
87
+ * `kv -w SCRIPT` monitors SCRIPT file modification and reload it immediately.
88
+ * `kv --filter-process=cmd SCRIPT` shows the result of `cmd FILE` as a filter.
89
+ * Combination: `kv -w --filter-command='rstfilter -a' SCRIPT` shows modified script with execution results.
90
+
50
91
  ## Implementation
51
92
 
52
93
  With parser gem, rstfilter translates the given script and run it.
@@ -70,3 +111,8 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/ko1/rs
70
111
  ## License
71
112
 
72
113
  The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
114
+
115
+ ## Aside
116
+
117
+ * My motivation of this tool is to make it easy to annotate the script with execution results. For example, Ruby developer's meeting generates many code like: https://github.com/ruby/dev-meeting-log/blob/master/DevMeeting-2022-05-19.md
118
+ * The name "Rst" stands for "Result". This tool is inspired from [xmpfilter](https://github.com/rcodetools/rcodetools/blob/master/lib/rcodetools/xmpfilter.rb) and original author Gotoken-san told me that "xmp" is stand for "Example" (he had wanted to make a support tool for lectures). Respect to the "xmp" mysterious word, I choosed "Rst".
data/exe/rstfilter-lsp ADDED
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require_relative '../lib/rstfilter/lsp/handler'
4
+
5
+ RstFilter::LSP.new.start
@@ -0,0 +1,330 @@
1
+ require 'optparse'
2
+ require 'shellwords'
3
+
4
+ module RstFilter
5
+ class Exec
6
+ def update_opt opt = {}
7
+ opt = @opt.to_h.merge(opt)
8
+ @opt = ConfigOption.new(**opt)
9
+ end
10
+
11
+ attr_reader :output
12
+
13
+ def initialize opt = {}
14
+ @output = ''
15
+ @opt = ConfigOption.new(**DEFAULT_SETTING)
16
+ update_opt opt
17
+ end
18
+
19
+ DEFAULT_SETTING = {
20
+ # rewrite options
21
+ show_all_results: true,
22
+ show_exceptions: true,
23
+ show_output: false,
24
+ show_decl: false,
25
+ show_specific_line: nil,
26
+
27
+ use_pp: false,
28
+ comment_nextline: false,
29
+ comment_indent: 50,
30
+ comment_pattern: '#=>',
31
+ comment_label: nil,
32
+
33
+ # execute options
34
+ exec_command: false, # false: simply load file
35
+ # String value: launch given string as a command
36
+ exec_with_filename: true,
37
+
38
+ # dump
39
+ dump: nil, # :json
40
+
41
+ # general
42
+ verbose: false,
43
+ ignore_pragma: false,
44
+ }
45
+
46
+ ConfigOption = Struct.new(*DEFAULT_SETTING.keys, keyword_init: true)
47
+ Command = Struct.new(:label, :command)
48
+
49
+ def optparse! argv
50
+ opt = {}
51
+ o = OptionParser.new
52
+ o.on('-c', '--comment', 'Show result only on comment'){
53
+ opt[:show_all_results] = false
54
+ }
55
+ o.on('-o', '--output', 'Show output results'){
56
+ opt[:show_output] = true
57
+ }
58
+ o.on('-d', '--decl', 'Show results on declaration'){
59
+ opt[:show_decl] = true
60
+ }
61
+ o.on('--no-exception', 'Do not show exception'){
62
+ opt[:show_exception] = false
63
+ }
64
+ o.on('--pp', 'Use pp to represent objects'){
65
+ opt[:use_pp] = true
66
+ }
67
+ o.on('-n', '--nextline', 'Put comments on next line'){
68
+ opt[:comment_nextline] = true
69
+ }
70
+ o.on('--comment-indent=NUM', "Specify comment indent size (default: #{DEFAULT_SETTING[:comment_indent]})"){|n|
71
+ opt[:comment_indent] = n.to_i
72
+ }
73
+ o.on('--comment-pattern=PAT', "Specify comment pattern of -c (default: '#=>')"){|pat|
74
+ opt[:comment_pattern] = pat
75
+ }
76
+ o.on('--coment-label=LABEL', 'Specify comment label (default: "")'){|label|
77
+ opt[:comment_label] = label
78
+ }
79
+ o.on('--verbose', 'Verbose mode'){
80
+ opt[:verbose] = true
81
+ }
82
+ o.on('-e', '--command=COMMAND', 'Execute Ruby script with given command'){|cmdstr|
83
+ opt[:exec_command] ||= []
84
+
85
+ if /\A(.+):(.+)\z/ =~ cmdstr
86
+ cmd = Command.new($1, $2)
87
+ else
88
+ cmd = Command.new("e#{(opt[:exec_command]&.size || 0) + 1}", cmdstr)
89
+ end
90
+
91
+ opt[:exec_command] << cmd
92
+ }
93
+ o.on('--no-filename', 'Execute -e command without filename'){
94
+ opt[:exec_with_filename] = false
95
+ }
96
+ o.on('-j', '--json', 'Print records in JSON format'){
97
+ opt[:dump] = :json
98
+ }
99
+ o.on('--ignore-pragma', 'Ignore pragma specifiers'){
100
+ opt[:ignore_pragma] = true
101
+ }
102
+ o.on('--verbose', 'Verbose mode'){
103
+ opt[:verbose] = true
104
+ }
105
+ o.parse!(argv)
106
+ update_opt opt
107
+ end
108
+
109
+ def err msg
110
+ msg.each_line{|line|
111
+ STDERR.puts "[RstFilter] #{line}"
112
+ }
113
+ end
114
+
115
+ def comment_label
116
+ if l = @opt.comment_label
117
+ "#{l}: "
118
+ end
119
+ end
120
+
121
+ def puts_result prefix, results, line = nil
122
+ prefix = prefix.chomp
123
+
124
+ if results.size == 1
125
+ r = results.first
126
+ result_lines = r.lines
127
+ indent = ''
128
+
129
+ if @opt.comment_nextline
130
+ puts prefix
131
+ if prefix.match(/\A(\s+)/)
132
+ prefix = ' ' * $1.size
133
+ else
134
+ prefix = ''
135
+ end
136
+ puts "#{prefix}" + "#{@opt.comment_pattern}#{comment_label}#{result_lines.shift}"
137
+ else
138
+ if line
139
+ puts line.sub(/#{@opt.comment_pattern}.*$/, "#{@opt.comment_pattern} #{comment_label}#{result_lines.shift.chomp}")
140
+ else
141
+ indent = ' ' * [0, @opt.comment_indent - prefix.size].max
142
+ puts "#{prefix}#{indent} #=> #{comment_label}#{result_lines.shift}"
143
+ end
144
+ end
145
+
146
+ cont_comment = ' #' + ' ' * @opt.comment_pattern.size + ' '
147
+
148
+ result_lines.each{|result_line|
149
+ puts ' ' * prefix.size + indent + "#{cont_comment}#{result_line}"
150
+ }
151
+ else
152
+ puts prefix
153
+
154
+ if prefix.match(/\A(\s+)/)
155
+ prefix = ' ' * $1.size
156
+ else
157
+ prefix = ''
158
+ end
159
+
160
+ results.each.with_index{|r, i|
161
+ result_lines = r.lines
162
+ puts "#{prefix}#{@opt.comment_pattern} #{@opt.exec_command[i].label}: #{result_lines.first}"
163
+ }
164
+ end
165
+ end
166
+
167
+ def exec_mod_src mod_src
168
+ # execute modified src
169
+ ENV['RSTFILTER_SHOW_OUTPUT'] = @opt.show_output ? '1' : nil
170
+ ENV['RSTFILTER_SHOW_EXCEPTIONS'] = @opt.show_exceptions ? '1' : nil
171
+ ENV['RSTFILTER_FILENAME'] = @filename
172
+ ENV['RSTFILTER_PP'] = @opt.use_pp ? '1' : nil
173
+
174
+ case cs = @opt.exec_command
175
+ when Array
176
+ @output = String.new
177
+
178
+ cs.map do |c|
179
+ require 'tempfile'
180
+ recf = Tempfile.new('rstfilter-rec')
181
+ ENV['RSTFILTER_RECORD_PATH'] = recf.path
182
+ recf.close
183
+
184
+ modf = Tempfile.new('rstfilter-modsrc')
185
+ modf.write mod_src
186
+ modf.close
187
+ ENV['RSTFILTER_MOD_SRC_PATH'] = modf.path
188
+
189
+ ENV['RUBYOPT'] = "-r#{File.join(__dir__, 'exec_setup')} #{ENV['RUBYOPT']}"
190
+
191
+ cmd = c.command
192
+ cmd << ' ' + @filename if @opt.exec_with_filename
193
+ p exec:cmd if @opt.verbose
194
+
195
+ io = IO.popen(cmd, err: [:child, :out])
196
+ begin
197
+ Process.waitpid(io.pid)
198
+ ensure
199
+ begin
200
+ Process.kill(:KILL, io.pid)
201
+ rescue Errno::ESRCH
202
+ end
203
+ end
204
+
205
+ @output << io.read
206
+ open(recf.path){|f| Marshal.load f}
207
+ end
208
+ else
209
+ begin
210
+ begin
211
+ require_relative 'exec_setup'
212
+ ::RSTFILTER__.clear
213
+ ::TOPLEVEL_BINDING.eval(mod_src, @filename)
214
+ [::RSTFILTER__.records]
215
+ ensure
216
+ $stdout = $__rst_filter_prev_out if $__rst_filter_prev_out
217
+ $stderr = $__rst_filter_prev_err if $__rst_filter_prev_err
218
+ end
219
+ rescue Exception => e
220
+ if @opt.verbose
221
+ err e.inspect
222
+ err e.backtrace.join("\n")
223
+ else
224
+ err "exit with #{e.inspect}"
225
+ end
226
+ [::RSTFILTER__.records]
227
+ end
228
+ end
229
+ end
230
+
231
+ def modified_src src, filename = nil
232
+ rewriter = Rewriter.new @opt
233
+ rewriter.rewrite(src, filename)
234
+ end
235
+
236
+ def record_records filename
237
+ @filename = filename
238
+ src = File.read(filename)
239
+ src, mod_src, comments = modified_src(src, filename)
240
+
241
+ comments.each{|c|
242
+ case c.text
243
+ when /\A\#rstfilter\s(.+)/
244
+ optparse! Shellwords.split($1)
245
+ end
246
+ } unless @opt.ignore_pragma
247
+
248
+ return exec_mod_src(mod_src), src, comments
249
+ end
250
+
251
+ def make_line_records rs
252
+ lrs = {}
253
+ rs.each{|(_bl, _bc, el, _ec), result|
254
+ lrs[el] = result
255
+ }
256
+ lrs
257
+ end
258
+
259
+ def process filename
260
+ records, src, comments = record_records filename
261
+ pp records: records if @opt.verbose
262
+ line_records = records.map{|r|
263
+ make_line_records r
264
+ }
265
+
266
+ case @opt.dump
267
+ when :json
268
+ require 'json'
269
+ puts JSON.dump(records)
270
+ else
271
+ replace_comments = comments.filter_map{|c|
272
+ next unless c.text.start_with? @opt.comment_pattern
273
+ e = c.loc.expression
274
+ [e.begin.line, true]
275
+ }.to_h
276
+
277
+ src.each_line.with_index{|line, i|
278
+ lineno = i+1
279
+ line_results = line_records.map{|r| r[lineno]&.first}.compact
280
+
281
+ if line_results.empty?
282
+ puts line
283
+ else
284
+ if replace_comments[lineno]
285
+ line.match(/(.+)#{@opt.comment_pattern}.*$/) || raise("unreachable")
286
+ puts_result $1, line_results, line
287
+ elsif @opt.show_all_results
288
+ puts_result line, line_results
289
+ end
290
+ end
291
+
292
+ if @opt.show_output && !line_results.empty?
293
+ if m = line.match(/^\s+/)
294
+ indent = ' ' * m[0].size
295
+ else
296
+ indent = ''
297
+ end
298
+
299
+ line_outputs = line_records.map{|r| r[lineno]}.compact
300
+ line_outputs.each.with_index{|r, i|
301
+ out, err = *r[1..2]
302
+ label = @opt.exec_command && @opt.exec_command[i].label
303
+ label += ':' if label
304
+
305
+ {out: out, err: err}.each{|k, o|
306
+ o.strip!
307
+ o.each_line{|ol|
308
+ puts "#{indent}\##{label ? label : nil}#{k}: #{ol}"
309
+ } unless o.empty?
310
+ }
311
+ }
312
+ end
313
+ }
314
+
315
+ if !@opt.show_output && !@output.empty?
316
+ puts "# output"
317
+ puts output
318
+ end
319
+ end
320
+ end
321
+ end
322
+ end
323
+
324
+ if $0 == __FILE__
325
+ require_relative 'rewriter'
326
+ filter = RstFilter::Exec.new
327
+ filter.optparse! ['-v']
328
+ file = ARGV.shift || File.expand_path(__dir__ + '/../../sample.rb')
329
+ filter.process File.expand_path(file)
330
+ end
@@ -0,0 +1,95 @@
1
+ require 'stringio'
2
+
3
+ if defined?(RSTFILTER__)
4
+ Object.remove_const :RSTFILTER__
5
+ end
6
+
7
+ class RSTFILTER__
8
+ SHOW_EXCEPTION = ENV['RSTFILTER_SHOW_EXCEPTIONS']
9
+
10
+ if ::ENV['RSTFILTER_PP']
11
+ require 'pp'
12
+ def self.__rst_inspect_body__ val
13
+ ::PP.pp(val, '')
14
+ end
15
+ else
16
+ def self.__rst_inspect_body__ val
17
+ val.inspect
18
+ end
19
+ end
20
+
21
+ def self.__rst_inspect__ val
22
+ begin
23
+ __rst_inspect_body__ val
24
+ rescue Exception => e
25
+ "!! __rst_inspect__ failed: #{e}"
26
+ end
27
+ end
28
+
29
+ @@records = {}
30
+
31
+ def self.records
32
+ @@records
33
+ end
34
+
35
+ def self.clear
36
+ @@records.clear
37
+ end
38
+
39
+ def self.write begin_line, begin_col, end_line, end_col, val, prefix
40
+ # p [begin_line, begin_col, end_line, end_col]
41
+ out, err = *[$__rst_filter_captured_out, $__rst_filter_captured_err].map{|o|
42
+ str = o.string
43
+ o.string = ''
44
+ str
45
+ } if $__rst_filter_captured_out
46
+
47
+ @@records[[begin_line, begin_col, end_line, end_col]] = ["#{prefix}#{__rst_inspect__(val)}", out, err]
48
+ end
49
+
50
+ def self.record begin_line, begin_col, end_line, end_col
51
+ r = yield
52
+ write begin_line, begin_col, end_line, end_col, r, nil
53
+ r
54
+ rescue Exception => e
55
+ write begin_line, begin_col, end_line, end_col, e, 'raised '
56
+ raise
57
+ end
58
+ end
59
+
60
+ if ENV['RSTFILTER_SHOW_OUTPUT']
61
+ $__rst_filter_prev_out = $stdout
62
+ $__rst_filter_prev_err = $stderr
63
+
64
+ $__rst_filter_captured_out = $stdout = StringIO.new
65
+ if false # debug
66
+ $__rst_filter_captured_err = $captured_out
67
+ else
68
+ $__rst_filter_captured_err = $stderr = StringIO.new
69
+ end
70
+ else
71
+ $__rst_filter_prev_out = $__rst_filter_prev_err = nil
72
+ $__rst_filter_captured_out = $__rst_filter_captured_err = nil
73
+ end
74
+
75
+ if path = ENV['RSTFILTER_RECORD_PATH']
76
+ # inter-process communication
77
+
78
+ END{
79
+ open(path, 'wb'){|f|
80
+ f.write Marshal.dump(::RSTFILTER__.records)
81
+ }
82
+ }
83
+
84
+ class RubyVM::InstructionSequence
85
+ RST_FILENAME = ENV['RSTFILTER_FILENAME']
86
+ RST_MOD_SRC = File.read(ENV['RSTFILTER_MOD_SRC_PATH'])
87
+ @found = false
88
+ def self.translate iseq
89
+ if !@found && iseq.path == RST_FILENAME && iseq.label == "<main>"
90
+ @found = true
91
+ RubyVM::InstructionSequence.compile RST_MOD_SRC, RST_FILENAME
92
+ end
93
+ end
94
+ end
95
+ end
@@ -0,0 +1,291 @@
1
+ require 'json'
2
+ require_relative '../version'
3
+ require_relative '../rewriter'
4
+ require_relative '../exec'
5
+
6
+ module RstFilter
7
+ class LSP
8
+ def initialize input: $stdin, output: $stdout, err: $stderr, indent: 50
9
+ @input = input
10
+ @output = output
11
+ @err = err
12
+ @indent = indent
13
+ @records = {} # {filename => [record, line_record, src]}
14
+ @server_request_id = 0
15
+ @exit_status = 1
16
+ @running = {} # id for cancel
17
+ end
18
+
19
+ def self.reload
20
+ ::RstFilter.send(:remove_const, :LSP)
21
+ $".delete_if{|e| /lsp\/handler\.rb$/ =~ e}
22
+ require __FILE__
23
+ end
24
+
25
+ def start
26
+ # for reload
27
+ trap(:USR1){
28
+ Thread.main.raise "reload"
29
+ }
30
+
31
+ lsp = self
32
+ begin
33
+ lsp.event_loop
34
+ rescue Exception => e
35
+ log e
36
+ log e.backtrace
37
+
38
+ # reload
39
+ lsp.class.reload
40
+ lsp = RstFilter::LSP.new
41
+ retry
42
+ end
43
+ end
44
+
45
+ def event_loop
46
+ while req = recv_message
47
+ if req[:id]
48
+ handle_request req
49
+ else
50
+ handle_notification req
51
+ end
52
+ end
53
+ end
54
+
55
+ def log msg
56
+ @err.puts msg
57
+ end
58
+
59
+ def recv_message
60
+ log "wait from #{@input.inspect}"
61
+ line = @input.gets
62
+ line.match(/Content-Length: (\d+)/) || raise("irregular json-rpc: #{line}")
63
+ @input.gets
64
+ msg = JSON.parse(@input.read($1.to_i), symbolize_names: true)
65
+ log "[recv] #{msg.inspect}"
66
+ msg
67
+ end
68
+
69
+ def send_message type, msg_text
70
+ log "[#{type}] #{msg_text}"
71
+
72
+ text = "Content-Length: #{msg_text.size}\r\n" \
73
+ "\r\n" \
74
+ "#{msg_text}"
75
+ @output.write text
76
+ @output.flush
77
+ if true
78
+ log '----'
79
+ log text
80
+ log '----'
81
+ end
82
+ end
83
+
84
+ def send_response req, kw
85
+ res_text = JSON.dump({
86
+ jsonrpc: "2.0",
87
+ id: req[:id],
88
+ result: kw,
89
+ })
90
+
91
+ send_message 'response', res_text
92
+ end
93
+
94
+ def send_request method, kw = {}
95
+ msg_text = JSON.dump({
96
+ jsonrpc: "2.0",
97
+ method: method,
98
+ id: (@server_request_id+=1),
99
+ params: {
100
+ **kw
101
+ }
102
+ })
103
+
104
+ send_message 'request', msg_text
105
+ end
106
+
107
+ def send_notice method, kw = {}
108
+ msg_text = JSON.dump({
109
+ jsonrpc: "2.0",
110
+ method: method,
111
+ params: {
112
+ **kw
113
+ }
114
+ })
115
+ send_message "notice", msg_text
116
+ end
117
+
118
+ def handle_request req
119
+ if req[:error]
120
+ log "error: #{req.inspect}"
121
+ return
122
+ end
123
+
124
+ case req[:method]
125
+ when 'initialize'
126
+ send_response req, {
127
+ capabilities: {
128
+ textDocumentSync: {
129
+ openClose: true,
130
+ change: 2, # Incremental
131
+ save: true,
132
+ },
133
+
134
+ inlineValueProvider: true,
135
+
136
+ hoverProvider: true,
137
+
138
+ inlayHintProvider: {
139
+ resolveProvider: true,
140
+ },
141
+
142
+ },
143
+ serverInfo: {
144
+ name: "rstfilter-lsp-server",
145
+ version: '0.0.1',
146
+ }
147
+ }
148
+ when 'textDocument/hover'
149
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
150
+ line = req.dig(:params, :position, :line)
151
+ char = req.dig(:params, :position, :character)
152
+ if (record, _line_record, src = @records[filename])
153
+ line += 1
154
+ rs = record.find_all{|(bl, bc, el, ec), _v| cover?(line, char, bl, bc, el, ec)}
155
+ if rs.empty?
156
+ send_response req, nil
157
+ else
158
+ r = rs.min_by{|(_bl, _bc, _el, ec), _v| ec}
159
+ v = r[1][0].strip
160
+ pos = r[0]
161
+ send_response req, contents: {
162
+ kind: 'markdown',
163
+ value: "```\n#{v}```",
164
+ }, range: {
165
+ start: {line: pos[0]-1, character: pos[1],},
166
+ end: {line: pos[2]-1, character: pos[3],},
167
+ }
168
+ end
169
+ else
170
+ send_response req, nil
171
+ end
172
+ when 'textDocument/codeLens'
173
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
174
+ send_response req, codelens(filename)
175
+ when 'textDocument/inlayHint'
176
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
177
+ send_response req, inlayhints(filename)
178
+ when 'inlayHint/resolve'
179
+ hint = req.dig(:params)
180
+ send_response req, {
181
+ position: hint[:position],
182
+
183
+ label: [{
184
+ tooltip: {
185
+ kind: 'markdown',
186
+ value: hint[:label] + "\n 1. *foo* `bar` _baz_",
187
+ tooltip: hint[:label] + "\n 1. *foo* `bar` _baz_",
188
+ }
189
+ },
190
+ {
191
+ kind: 'markdown',
192
+ value: hint[:label] + "\n# 2. FOO\n*foo* `bar` _baz_",
193
+ }],
194
+ }
195
+ when 'shutdown'
196
+ @exit_status = 0
197
+ send_response req, nil
198
+ when nil
199
+ # reply
200
+ else
201
+ raise "unknown request: #{req.inspect}"
202
+ end
203
+ end
204
+
205
+ def cover? line, char, bl, bc, el, ec
206
+ return false if bl > line
207
+ return false if bl == line && char < bc
208
+ return false if el < line
209
+ return false if el == line && char >= ec
210
+ true
211
+ end
212
+
213
+ def inlayhints filename
214
+ if (_record, line_record, src = @records[filename])
215
+ src_lines = src.lines.to_a
216
+
217
+ line_record.sort_by{|k, v| k}.map do |lineno, r|
218
+ line = src_lines[lineno - 1]
219
+ next unless line
220
+
221
+ {
222
+ position: {
223
+ line: lineno - 1, # 0 origin
224
+ character: line.length,
225
+ },
226
+ label: ' ' * [@indent - line.size, 3].max + "#=> #{r.first.strip}",
227
+ # tooltip: "tooltip of #{lineno}",
228
+ paddingLeft: true,
229
+ kind: 1,
230
+ }
231
+ end.compact
232
+ else
233
+ nil
234
+ end
235
+ end
236
+
237
+ def take_record filename
238
+ send_notice 'rstfilter/start', {
239
+ uri: filename,
240
+ }
241
+ Thread.new do
242
+ filter = RstFilter::Exec.new
243
+ filter.optparse! ['--pp', '-eruby']
244
+ records, src, _comments = filter.record_records(filename)
245
+ records = records.first # only 1 process results
246
+ @records[filename] = [records, filter.make_line_records(records), src]
247
+ send_notice 'rstfilter/done'
248
+ unless filter.output.empty?
249
+ send_notice 'rstfilter/output', output: "# Output for #{filename}\n\n#{filter.output}"
250
+ end
251
+ send_request 'workspace/inlayHint/refresh'
252
+ end
253
+ end
254
+
255
+ def clear_record filename
256
+ @records[filename] = nil
257
+ end
258
+
259
+ def uri2filename uri
260
+ case uri
261
+ when /^file:\/\/(.+)/
262
+ $1
263
+ else
264
+ raise "unknown uri: #{uri}"
265
+ end
266
+ end
267
+
268
+ def handle_notification req
269
+ case req[:method]
270
+ when 'initialized'
271
+ send_notice 'rstfilter/version', {
272
+ version: "#{::RstFilter::VERSION} on #{RUBY_DESCRIPTION}",
273
+ }
274
+ when 'textDocument/didOpen',
275
+ 'textDocument/didSave'
276
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
277
+ take_record filename
278
+ when 'textDocument/didClose',
279
+ 'textDocument/didChange'
280
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
281
+ clear_record filename
282
+ when '$/cancelRequest'
283
+ # TODO: cancel
284
+ when 'exit'
285
+ exit(@exit_code)
286
+ else
287
+ raise
288
+ end
289
+ end
290
+ end
291
+ end
@@ -0,0 +1,182 @@
1
+
2
+ require 'parser/current'
3
+ require 'pp'
4
+
5
+ module RstFilter
6
+ class RecordAll < Parser::TreeRewriter
7
+ def initialize opt
8
+ @decl = opt.show_decl || (opt.show_all_results == false)
9
+
10
+ super()
11
+ end
12
+
13
+ def add_record node
14
+ if le = node&.location&.expression
15
+ pos = [le.begin.line, le.begin.column, le.end.line, le.end.column].join(',')
16
+ insert_before(le.begin, "::RSTFILTER__.record(#{pos}){")
17
+ insert_after(le.end, "}")
18
+ end
19
+ end
20
+
21
+ def add_paren node
22
+ if le = node&.location&.expression
23
+ insert_before(le.begin, '(')
24
+ insert_after(le.end, ")")
25
+ end
26
+ end
27
+
28
+ def process node
29
+ return unless node
30
+
31
+ super
32
+
33
+ case node.type
34
+ when :begin,
35
+ :resbody, :rescue,
36
+ :ensure,
37
+ :return,
38
+ :next,
39
+ :redo,
40
+ :retry,
41
+ :splat,
42
+ :block_pass,
43
+ :lvasgn,
44
+ :when
45
+ # skip
46
+ when :def, :class
47
+ add_record node if @decl
48
+ when :if
49
+ unless node.loc.expression.source.start_with? 'elsif'
50
+ add_record node
51
+ end
52
+ else
53
+ add_record node
54
+ end
55
+ end
56
+
57
+ def on_dstr node
58
+ end
59
+
60
+ def on_regexp node
61
+ end
62
+
63
+ def on_const node
64
+ end
65
+
66
+ def on_masgn node
67
+ _mlhs, rhs = node.children
68
+ if rhs.type == :array
69
+ rhs.children.each{|r| process r}
70
+ end
71
+ end
72
+
73
+ def on_class node
74
+ _name, sup, body = node.children
75
+ process sup
76
+ process body
77
+ end
78
+
79
+ def on_module node
80
+ _name, body = node.children
81
+ process body
82
+ end
83
+
84
+ def process_args args
85
+ args.children.each{|arg|
86
+ case arg.type
87
+ when :optarg
88
+ _name, opexpr = arg.children
89
+ process opexpr
90
+ when :kwoptarg
91
+ _name, kwexpr = arg.children
92
+ process kwexpr
93
+ end
94
+ }
95
+ end
96
+
97
+ def on_def node
98
+ _name, args, body = node.children
99
+ process_args args
100
+ process body
101
+ end
102
+
103
+ def on_defs node
104
+ recv, _name, args, body = node.children
105
+ process recv
106
+ add_paren recv
107
+ process_args args
108
+ process body
109
+ end
110
+
111
+ def process_pairs pairs
112
+ pairs.each{|pair|
113
+ key, val = pair.children
114
+ if key.type != :sym
115
+ process key
116
+ end
117
+ process val
118
+ }
119
+ end
120
+
121
+ def on_hash node
122
+ process_pairs node.children
123
+ end
124
+
125
+ def on_send node
126
+ recv, _name, *args = *node.children
127
+ process recv if recv
128
+
129
+ args.each{|arg|
130
+ if arg.type == :hash
131
+ process_pairs arg.children
132
+ else
133
+ process arg
134
+ end
135
+ }
136
+ end
137
+
138
+ def on_block node
139
+ _send, _args, block = *node.children
140
+ process block
141
+ end
142
+
143
+ def on_numblock node
144
+ on_block node
145
+ end
146
+ end
147
+
148
+ class Rewriter
149
+ def initialize opt
150
+ @opt = opt
151
+ end
152
+
153
+ def rewrite src, filename
154
+ begin
155
+ prev_v, $VERBOSE = $VERBOSE, false
156
+ ast = RubyVM::AbstractSyntaxTree.parse(src)
157
+ $VERBOSE = prev_v
158
+ last_lineno = ast.last_lineno
159
+ rescue SyntaxError => e
160
+ err e.inspect
161
+ exit 1
162
+ end
163
+
164
+ # rewrite
165
+ src = src.lines[0..last_lineno].join # remove __END__ and later
166
+ ast, comments = Parser::CurrentRuby.parse_with_comments(src)
167
+ buffer = Parser::Source::Buffer.new('(example)')
168
+ buffer.source = src
169
+ rewriter = RecordAll.new @opt
170
+ mod_src = rewriter.rewrite(buffer, ast)
171
+
172
+ if @opt.verbose
173
+ pp ast
174
+ puts " #{(0...80).map{|i| i%10}.join}"
175
+ puts mod_src.lines.map.with_index{|line, i| '%4d:%s' % [i+1, line] }
176
+ end
177
+
178
+ return src, mod_src, comments
179
+ end
180
+ end
181
+ end
182
+
@@ -1,3 +1,3 @@
1
- module Rstfilter
2
- VERSION = "0.1.0"
1
+ module RstFilter
2
+ VERSION = "0.2.0"
3
3
  end
data/lib/rstfilter.rb CHANGED
@@ -1,287 +1,11 @@
1
1
  # require "rstfilter/version"
2
2
 
3
- require 'parser/current'
4
- require 'stringio'
5
- require 'optparse'
6
- require 'ostruct'
7
- require 'pp'
8
-
9
- module RstFilter
10
- class RecordAll < Parser::TreeRewriter
11
- def initialize opt
12
- @decl = opt.show_decl || (opt.show_all_results == false)
13
-
14
- super()
15
- end
16
-
17
- def add_record node
18
- if le = node&.location&.expression
19
- insert_before(le.begin, '(')
20
- insert_after(le.end, ").__rst_record__(#{le.end.line}, #{le.end.column})")
21
- end
22
- end
23
-
24
- def process node
25
- return unless node
26
- case node.type
27
- when :resbody, :rescue, :begin
28
- # skip
29
- when :def, :class
30
- add_record node if @decl
31
- else
32
- add_record node
33
- end
34
-
35
- super
36
- end
37
-
38
- def on_class node
39
- _name, sup, body = node.children
40
- process sup
41
- process body
42
- end
43
-
44
- def on_def node
45
- _name, args, body = node.children
46
- args.children.each{|arg|
47
- case arg.type
48
- when :optarg
49
- _name, opexpr = arg.children
50
- process opexpr
51
- when :kwoptarg
52
- _name, kwexpr = arg.children
53
- process kwexpr
54
- end
55
- }
56
- process body
57
- end
58
-
59
- def on_send node
60
- end
61
- end
62
-
63
- class Exec
64
- DEFAULT_SETTING = {
65
- # default setting
66
- show_all_results: true,
67
- show_exceptions: true,
68
- show_output: false,
69
- show_decl: false,
70
- show_specific_line: nil,
71
-
72
- use_pp: false,
73
- comment_indent: 50,
74
- comment_pattern: '#=>',
75
- verbose: false,
76
- }
77
-
78
- ConfigOption = Struct.new(*DEFAULT_SETTING.keys, keyword_init: true)
79
-
80
- def update_opt opt = {}
81
- opt = @opt.to_h.merge(opt)
82
- @opt = ConfigOption.new(**opt)
83
- end
84
-
85
- def initialize
86
- @opt = ConfigOption.new(**DEFAULT_SETTING)
87
- end
88
-
89
- def optparse! argv
90
- opt = {}
91
- o = OptionParser.new
92
- o.on('-c', '--comment', 'Show result only on comment'){
93
- opt[:show_all_results] = false
94
- }
95
- o.on('-o', '--output', 'Show output results'){
96
- opt[:show_output] = true
97
- }
98
- o.on('-d', '--decl', 'Show results on declaration'){
99
- opt[:show_decl] = true
100
- }
101
- o.on('--no-exception', 'Do not show exception'){
102
- opt[:show_exception] = false
103
- }
104
- o.on('-a', '--all', 'Show all results/output'){
105
- opt[:show_output] = true
106
- opt[:show_all_results] = true
107
- opt[:show_exceptions] = true
108
- }
109
- o.on('--pp', 'Use pp to represent objects'){
110
- opt[:use_pp] = true
111
- }
112
- o.on('--comment-indent=NUM', "Specify comment indent size (default: #{DEFAULT_SETTING[:comment_indent]})"){|n|
113
- opt[:comment_indent] = n.to_i
114
- }
115
- o.on('--verbose', 'Verbose mode'){
116
- opt[:verbose] = true
117
- }
118
- o.parse!(argv)
119
- update_opt opt
120
- end
121
-
122
- def capture_out
123
- return yield unless @opt.show_output
124
-
125
- begin
126
- prev_out = $stdout
127
- prev_err = $stderr
128
-
129
- $captured_out = $stdout = StringIO.new
130
- if false # debug
131
- $captured_err = $captured_out
132
- else
133
- $captured_err = $stderr = StringIO.new
134
- end
135
-
136
- yield
137
- ensure
138
- $stdout = prev_out
139
- $stderr = prev_err
140
- end
141
- end
142
-
143
- def record_rescue
144
- if @opt.show_exceptions
145
- TracePoint.new(:raise) do |tp|
146
- caller_locations.each{|loc|
147
- if loc.path == @filename
148
- $__rst_record[loc.lineno][0] = [tp.raised_exception, '', '']
149
- break
150
- end
151
- }
152
- end.enable do
153
- yield
154
- end
155
- else
156
- yield
157
- end
158
- end
159
-
160
- class ::BasicObject
161
- def __rst_record__ line, col
162
- out, err = *[$captured_out, $captured_err].map{|o|
163
- str = o.string
164
- o.string = ''
165
- str
166
- } if $captured_out
167
-
168
- # ::STDERR.puts [line, col, self, out, err].inspect
169
- $__rst_record[line][col] = [self, out, err]
170
-
171
- self
172
- end
173
- end
174
-
175
- $__rst_record = Hash.new{|h, k| h[k] = []}
176
-
177
- def process filename
178
- @filename = filename
179
-
180
- code = File.read(filename)
181
-
182
- begin
183
- ast, comments = Parser::CurrentRuby.parse_with_comments(code)
184
- rescue Parser::SyntaxError => e
185
- puts e
186
- exit 1
187
- end
188
-
189
- end_line = ast.loc.expression.end.line
190
- code = code.lines[0..end_line].join # remove __END__ and later
191
-
192
- buffer = Parser::Source::Buffer.new('(example)')
193
- buffer.source = code
194
- rewriter = RecordAll.new @opt
195
-
196
- mod_src = rewriter.rewrite(buffer, ast)
197
- puts mod_src.lines.map.with_index{|line, i| '%4d: %s' % [i+1, line] } if @opt.verbose
198
-
199
- begin
200
- capture_out do
201
- record_rescue do
202
- ::TOPLEVEL_BINDING.eval(mod_src, filename)
203
- end
204
- end
205
- rescue Exception => e
206
- if @opt.verbose
207
- STDERR.puts e
208
- STDERR.puts e.backtrace
209
- else
210
- STDERR.puts "RstFilter: exit with #{e.inspect}"
211
- end
212
- end
213
-
214
- replace_comments = {}
215
-
216
- comments.each{|c|
217
- next unless c.text.start_with? @opt.comment_pattern
218
- e = c.loc.expression
219
- line, _col = e.begin.line, e.begin.column
220
- if $__rst_record.has_key? line
221
- result = $__rst_record[line].last
222
- replace_comments[line] = result
223
- end
224
- }
225
-
226
- pp $__rst_record if @opt.verbose
227
-
228
- code.each_line.with_index{|line, i|
229
- line_result = $__rst_record[i+1]&.last
230
-
231
- if line_result && line.match(/(.+)#{@opt.comment_pattern}.*$/)
232
- prefix = $1
233
- r = line_result.first
234
- if @opt.use_pp
235
- result_lines = PP.pp(r, '').lines
236
- else
237
- result_lines = r.inspect.lines
238
- end
239
- puts line.sub(/#{@opt.comment_pattern}.*$/, "#{@opt.comment_pattern} #{result_lines.shift.chomp}")
240
- cont_comment = '#' + ' ' * @opt.comment_pattern.size
241
- result_lines.each{|result_line|
242
- puts ' ' * prefix.size + "#{cont_comment}#{result_line}"
243
- }
244
- elsif @opt.show_all_results && line_result
245
- r = line_result.first
246
- indent = ' ' * [@opt.comment_indent - line.chomp.length, 0].max
247
- if @opt.use_pp
248
- result_lines = PP.pp(r, '').lines
249
- else
250
- result_lines = r.inspect.lines
251
- end
252
- prefix = line.chomp.concat "#{indent}"
253
- puts "#{prefix} #=> #{result_lines.shift}"
254
- cont_comment = '#' + ' ' * @opt.comment_pattern.size
255
- result_lines.each{|result_line|
256
- puts ' ' * prefix.size + " #{cont_comment}#{result_line}"
257
- }
258
- else
259
- puts line
260
- end
261
-
262
- if @opt.show_output && line_result
263
- out, err = *line_result[1..2]
264
- if m = line.match(/^\s+/)
265
- indent = ' ' * m[0].size
266
- else
267
- indent = ''
268
- end
269
-
270
- {out: out, err: err}.each{|k, o|
271
- o.strip!
272
- o.each_line{|ol|
273
- puts "#{indent}##{k}: #{ol}"
274
- } unless o.empty?
275
- }
276
- end
277
- }
278
- end
279
- end
280
- end
3
+ require_relative 'rstfilter/rewriter'
4
+ require_relative 'rstfilter/exec'
281
5
 
282
6
  if __FILE__ == $0
283
7
  filter = RstFilter::Exec.new
284
- filter.optparse ARGV
285
- file = ARGV.shift
8
+ filter.optparse! ['-o', '-v']
9
+ file = ARGV.shift || File.expand_path(__dir__ + '/../sample.rb')
286
10
  filter.process File.expand_path(file)
287
11
  end
data/rstfilter.gemspec CHANGED
@@ -2,7 +2,7 @@ require_relative 'lib/rstfilter/version'
2
2
 
3
3
  Gem::Specification.new do |spec|
4
4
  spec.name = "rstfilter"
5
- spec.version = Rstfilter::VERSION
5
+ spec.version = RstFilter::VERSION
6
6
  spec.authors = ["Koichi Sasada"]
7
7
  spec.email = ["ko1@atdot.net"]
8
8
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rstfilter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Koichi Sasada
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-05-26 00:00:00.000000000 Z
11
+ date: 2022-06-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: parser
@@ -29,9 +29,11 @@ email:
29
29
  - ko1@atdot.net
30
30
  executables:
31
31
  - rstfilter
32
+ - rstfilter-lsp
32
33
  extensions: []
33
34
  extra_rdoc_files: []
34
35
  files:
36
+ - ".github/workflows/ruby.yml"
35
37
  - ".gitignore"
36
38
  - ".travis.yml"
37
39
  - Gemfile
@@ -41,7 +43,12 @@ files:
41
43
  - bin/console
42
44
  - bin/setup
43
45
  - exe/rstfilter
46
+ - exe/rstfilter-lsp
44
47
  - lib/rstfilter.rb
48
+ - lib/rstfilter/exec.rb
49
+ - lib/rstfilter/exec_setup.rb
50
+ - lib/rstfilter/lsp/handler.rb
51
+ - lib/rstfilter/rewriter.rb
45
52
  - lib/rstfilter/version.rb
46
53
  - rstfilter.gemspec
47
54
  homepage: https://github.com/ko1/rstfilter
@@ -66,7 +73,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
66
73
  - !ruby/object:Gem::Version
67
74
  version: '0'
68
75
  requirements: []
69
- rubygems_version: 3.1.6
76
+ rubygems_version: 3.3.7
70
77
  signing_key:
71
78
  specification_version: 4
72
79
  summary: Show Ruby script with execution results.