rstfilter 0.1.0 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: acaf57f79e7bee11c42958205a4970e6c85a2de855f82513076e40eba42a5d4d
4
- data.tar.gz: 540a4a20d663769298947079626f5ec8d4ff1d10d72d724d126d5a7a2f9400bd
3
+ metadata.gz: daa8ebce99a8413104e64add5ee66e5f613c0e658b3f3beba239fc2f947e6213
4
+ data.tar.gz: 11a2afb17c240bb8d953b7111501d18f460da7442b6479679363a62ea33bba66
5
5
  SHA512:
6
- metadata.gz: a3371ad347e2797e051a9c9ac0621eca32022aae6b41ef859c961928e3b1159235e64e75e307901ca8dc17dba27e9e9e56d8646af02683b1eddc8eb2b68184a1
7
- data.tar.gz: 7abd528a4b660261a2be171609191c16ee848348ad0e6adf7d7c5421ba3aabf4757be69657f3fb6c6b4195083c4c9102d85964a10d086189a843924a923684d2
6
+ metadata.gz: 2c22b014632dbc34fdb7bca057294615af8713c56fafb8cc10121f8bf70dc6a802691f8b4aa79600430dc876fd22869980a444453e485665a4b1c12909478f93
7
+ data.tar.gz: 8af43aa24f7ef98ac513433704a8c6c22b3bea03d051adab744ba0342efed7c3af16c4a0e07b63bdf395d12fa4cfc290ccc7170b747a4be8757045d71e4fb1d2
@@ -0,0 +1,38 @@
1
+ # This workflow uses actions that are not certified by GitHub.
2
+ # They are provided by a third-party and are governed by
3
+ # separate terms of service, privacy policy, and support
4
+ # documentation.
5
+ # This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
6
+ # For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
7
+
8
+ name: Ruby
9
+
10
+ on:
11
+ push:
12
+ branches: [ main ]
13
+ pull_request:
14
+ branches: [ main ]
15
+
16
+ permissions:
17
+ contents: read
18
+
19
+ jobs:
20
+ test:
21
+
22
+ runs-on: ubuntu-latest
23
+ strategy:
24
+ matrix:
25
+ ruby-version: ['2.7', '3.0', '3.1', 'head', 'debug']
26
+
27
+ steps:
28
+ - uses: actions/checkout@v3
29
+ - name: Set up Ruby
30
+ # To automatically get bug fixes and new Ruby versions for ruby/setup-ruby,
31
+ # change this to (see https://github.com/ruby/setup-ruby#versioning):
32
+ # uses: ruby/setup-ruby@v1
33
+ uses: ruby/setup-ruby@2b019609e2b0f1ea1a2bc8ca11cb82ab46ada124
34
+ with:
35
+ ruby-version: ${{ matrix.ruby-version }}
36
+ bundler-cache: true # runs 'bundle install' and caches installed gems automatically
37
+ - name: Run tests
38
+ run: bundle exec rake
data/README.md CHANGED
@@ -1,3 +1,5 @@
1
+ [![GH Actions](https://github.com/ko1/rstfilter/actions/workflows/ruby.yml/badge.svg)](https://github.com/ko1/rstfilter/actions/workflows/ruby.yml)
2
+
1
3
  # Rstfilter
2
4
 
3
5
  This tool prints a Ruby script with execution results.
@@ -9,7 +11,7 @@ b = 2
9
11
  c = a + b
10
12
  puts "Hello" * c
11
13
 
12
- $ rstfilter sample.rb -a
14
+ $ rstfilter sample.rb -o
13
15
  a = 1 #=> 1
14
16
  b = 2 #=> 2
15
17
  c = a + b #=> 3
@@ -41,12 +43,51 @@ Usage: rstfilter [options] SCRIPT
41
43
  -o, --output Show output results
42
44
  -d, --decl Show results on declaration
43
45
  --no-exception Do not show exception
44
- -a, --all Show all results/output
45
46
  --pp Use pp to represent objects
47
+ -n, --nextline Put comments on next line
46
48
  --comment-indent=NUM Specify comment indent size (default: 50)
49
+ --comment-pattern=PAT Specify comment pattern of -c (default: '#=>')
50
+ --coment-label=LABEL Specify comment label (default: "")
51
+ -e, --command=COMMAND Execute Ruby script with given command
52
+ --no-filename Execute -e command without filename
53
+ -j, --json Print records in JSON format
54
+ --ignore-pragma Ignore pragma specifiers
47
55
  --verbose Verbose mode
48
56
  ```
49
57
 
58
+ Note that you can specify multiple `-e` options like that:
59
+
60
+ ```
61
+ $ rstfilter -o sample.rb -eruby27:/home/ko1/.rbenv/versions/2.7.6/bin/ruby -e ruby30:/home/ko1/.rbenv/versions/3.0.4/bin/ruby
62
+ a = 1
63
+ #=> ruby27: 1
64
+ #=> ruby30: 1
65
+ b = 2
66
+ #=> ruby27: 2
67
+ #=> ruby30: 2
68
+ c = a + b
69
+ #=> ruby27: 3
70
+ #=> ruby30: 3
71
+ puts "Hello" * c
72
+ #=> ruby27: nil
73
+ #=> ruby30: nil
74
+ #ruby27:out: HelloHelloHello
75
+ #ruby30:out: HelloHelloHello
76
+ ```
77
+
78
+ On this case, you can check results on multiple Ruby interpreters.
79
+
80
+ ## Advanced demo
81
+
82
+ https://user-images.githubusercontent.com/9558/170426066-e0c19185-10e9-4932-a1ce-3088a4189b34.mp4
83
+
84
+ This video shows advanced usage to show the results with modified script immediately.
85
+
86
+ * [kv](https://rubygems.org/gems/kv) is another pager.
87
+ * `kv -w SCRIPT` monitors SCRIPT file modification and reload it immediately.
88
+ * `kv --filter-process=cmd SCRIPT` shows the result of `cmd FILE` as a filter.
89
+ * Combination: `kv -w --filter-command='rstfilter -a' SCRIPT` shows modified script with execution results.
90
+
50
91
  ## Implementation
51
92
 
52
93
  With parser gem, rstfilter translates the given script and run it.
@@ -70,3 +111,8 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/ko1/rs
70
111
  ## License
71
112
 
72
113
  The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
114
+
115
+ ## Aside
116
+
117
+ * My motivation of this tool is to make it easy to annotate the script with execution results. For example, Ruby developer's meeting generates many code like: https://github.com/ruby/dev-meeting-log/blob/master/DevMeeting-2022-05-19.md
118
+ * The name "Rst" stands for "Result". This tool is inspired from [xmpfilter](https://github.com/rcodetools/rcodetools/blob/master/lib/rcodetools/xmpfilter.rb) and original author Gotoken-san told me that "xmp" is stand for "Example" (he had wanted to make a support tool for lectures). Respect to the "xmp" mysterious word, I choosed "Rst".
data/exe/rstfilter-lsp ADDED
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require_relative '../lib/rstfilter/lsp/handler'
4
+
5
+ RstFilter::LSP.new.start
@@ -0,0 +1,332 @@
1
+ require 'optparse'
2
+ require 'shellwords'
3
+
4
+ module RstFilter
5
+ class Exec
6
+ def update_opt opt = {}
7
+ opt = @opt.to_h.merge(opt)
8
+ @opt = ConfigOption.new(**opt)
9
+ end
10
+
11
+ attr_reader :output
12
+
13
+ def initialize opt = {}
14
+ @output = ''
15
+ @opt = ConfigOption.new(**DEFAULT_SETTING)
16
+ update_opt opt
17
+ end
18
+
19
+ DEFAULT_SETTING = {
20
+ # rewrite options
21
+ show_all_results: true,
22
+ show_exceptions: true,
23
+ show_output: false,
24
+ show_decl: false,
25
+ show_specific_line: nil,
26
+
27
+ use_pp: false,
28
+ comment_nextline: false,
29
+ comment_indent: 50,
30
+ comment_pattern: '#=>',
31
+ comment_label: nil,
32
+
33
+ # execute options
34
+ exec_command: false, # false: simply load file
35
+ # String value: launch given string as a command
36
+ exec_with_filename: true,
37
+
38
+ # dump
39
+ dump: nil, # :json
40
+
41
+ # general
42
+ verbose: false,
43
+ ignore_pragma: false,
44
+ }
45
+
46
+ ConfigOption = Struct.new(*DEFAULT_SETTING.keys, keyword_init: true)
47
+ Command = Struct.new(:label, :command)
48
+
49
+ def optparse! argv
50
+ opt = {}
51
+ o = OptionParser.new
52
+ o.on('-c', '--comment', 'Show result only on comment'){
53
+ opt[:show_all_results] = false
54
+ }
55
+ o.on('-o', '--output', 'Show output results'){
56
+ opt[:show_output] = true
57
+ }
58
+ o.on('-d', '--decl', 'Show results on declaration'){
59
+ opt[:show_decl] = true
60
+ }
61
+ o.on('--no-exception', 'Do not show exception'){
62
+ opt[:show_exception] = false
63
+ }
64
+ o.on('--pp', 'Use pp to represent objects'){
65
+ opt[:use_pp] = true
66
+ }
67
+ o.on('-n', '--nextline', 'Put comments on next line'){
68
+ opt[:comment_nextline] = true
69
+ }
70
+ o.on('--comment-indent=NUM', "Specify comment indent size (default: #{DEFAULT_SETTING[:comment_indent]})"){|n|
71
+ opt[:comment_indent] = n.to_i
72
+ }
73
+ o.on('--comment-pattern=PAT', "Specify comment pattern of -c (default: '#=>')"){|pat|
74
+ opt[:comment_pattern] = pat
75
+ }
76
+ o.on('--coment-label=LABEL', 'Specify comment label (default: "")'){|label|
77
+ opt[:comment_label] = label
78
+ }
79
+ o.on('--verbose', 'Verbose mode'){
80
+ opt[:verbose] = true
81
+ }
82
+ o.on('-e', '--command=COMMAND', 'Execute Ruby script with given command'){|cmdstr|
83
+ opt[:exec_command] ||= []
84
+
85
+ if /\A(.+):(.+)\z/ =~ cmdstr
86
+ cmd = Command.new($1, $2)
87
+ else
88
+ cmd = Command.new("e#{(opt[:exec_command]&.size || 0) + 1}", cmdstr)
89
+ end
90
+
91
+ opt[:exec_command] << cmd
92
+ }
93
+ o.on('--no-filename', 'Execute -e command without filename'){
94
+ opt[:exec_with_filename] = false
95
+ }
96
+ o.on('-j', '--json', 'Print records in JSON format'){
97
+ opt[:dump] = :json
98
+ }
99
+ o.on('--ignore-pragma', 'Ignore pragma specifiers'){
100
+ opt[:ignore_pragma] = true
101
+ }
102
+ o.on('--verbose', 'Verbose mode'){
103
+ opt[:verbose] = true
104
+ }
105
+ o.parse!(argv)
106
+ update_opt opt
107
+ end
108
+
109
+ def err msg
110
+ msg.each_line{|line|
111
+ STDERR.puts "[RstFilter] #{line}"
112
+ }
113
+ end
114
+
115
+ def comment_label
116
+ if l = @opt.comment_label
117
+ "#{l}: "
118
+ end
119
+ end
120
+
121
+ def puts_result prefix, results, line = nil
122
+ prefix = prefix.chomp
123
+
124
+ if results.size == 1
125
+ r = results.first
126
+ result_lines = r.lines
127
+ indent = ''
128
+
129
+ if @opt.comment_nextline
130
+ puts prefix
131
+ if prefix.match(/\A(\s+)/)
132
+ prefix = ' ' * $1.size
133
+ else
134
+ prefix = ''
135
+ end
136
+ puts "#{prefix}" + "#{@opt.comment_pattern}#{comment_label}#{result_lines.shift}"
137
+ else
138
+ if line
139
+ puts line.sub(/#{@opt.comment_pattern}.*$/, "#{@opt.comment_pattern} #{comment_label}#{result_lines.shift.chomp}")
140
+ else
141
+ indent = ' ' * [0, @opt.comment_indent - prefix.size].max
142
+ puts "#{prefix}#{indent} #=> #{comment_label}#{result_lines.shift}"
143
+ end
144
+ end
145
+
146
+ cont_comment = ' #' + ' ' * @opt.comment_pattern.size + ' '
147
+
148
+ result_lines.each{|result_line|
149
+ puts ' ' * prefix.size + indent + "#{cont_comment}#{result_line}"
150
+ }
151
+ else
152
+ puts prefix
153
+
154
+ if prefix.match(/\A(\s+)/)
155
+ prefix = ' ' * $1.size
156
+ else
157
+ prefix = ''
158
+ end
159
+
160
+ results.each.with_index{|r, i|
161
+ result_lines = r.lines
162
+ puts "#{prefix}#{@opt.comment_pattern} #{@opt.exec_command[i].label}: #{result_lines.first}"
163
+ }
164
+ end
165
+ end
166
+
167
+ def exec_mod_src mod_src
168
+ # execute modified src
169
+ ENV['RSTFILTER_SHOW_OUTPUT'] = @opt.show_output ? '1' : nil
170
+ ENV['RSTFILTER_SHOW_EXCEPTIONS'] = @opt.show_exceptions ? '1' : nil
171
+ ENV['RSTFILTER_FILENAME'] = @filename
172
+ ENV['RSTFILTER_PP'] = @opt.use_pp ? '1' : nil
173
+
174
+ case cs = @opt.exec_command
175
+ when Array
176
+ @output = String.new
177
+
178
+ cs.map do |c|
179
+ require 'tempfile'
180
+ recf = Tempfile.new('rstfilter-rec')
181
+ ENV['RSTFILTER_RECORD_PATH'] = recf.path
182
+ recf.close
183
+
184
+ modf = Tempfile.new('rstfilter-modsrc')
185
+ modf.write mod_src
186
+ modf.close
187
+ ENV['RSTFILTER_MOD_SRC_PATH'] = modf.path
188
+
189
+ ENV['RUBYOPT'] = "-r#{File.join(__dir__, 'exec_setup')} #{ENV['RUBYOPT']}"
190
+
191
+ cmd = c.command
192
+ cmd << ' ' + @filename if @opt.exec_with_filename
193
+ p exec:cmd if @opt.verbose
194
+
195
+ io = IO.popen(cmd, err: [:child, :out])
196
+ begin
197
+ Process.waitpid(io.pid)
198
+ ensure
199
+ begin
200
+ Process.kill(:KILL, io.pid)
201
+ rescue Errno::ESRCH, Errno::ECHILD
202
+ else
203
+ Process.waitpid(io.pid)
204
+ end
205
+ end
206
+
207
+ @output << io.read
208
+ open(recf.path){|f| Marshal.load f}
209
+ end
210
+ else
211
+ begin
212
+ begin
213
+ require_relative 'exec_setup'
214
+ ::RSTFILTER__.clear
215
+ ::TOPLEVEL_BINDING.eval(mod_src, @filename)
216
+ [::RSTFILTER__.records]
217
+ ensure
218
+ $stdout = $__rst_filter_prev_out if $__rst_filter_prev_out
219
+ $stderr = $__rst_filter_prev_err if $__rst_filter_prev_err
220
+ end
221
+ rescue Exception => e
222
+ if @opt.verbose
223
+ err e.inspect
224
+ err e.backtrace.join("\n")
225
+ else
226
+ err "exit with #{e.inspect}"
227
+ end
228
+ [::RSTFILTER__.records]
229
+ end
230
+ end
231
+ end
232
+
233
+ def modified_src src, filename = nil
234
+ rewriter = Rewriter.new @opt
235
+ rewriter.rewrite(src, filename)
236
+ end
237
+
238
+ def record_records filename
239
+ @filename = filename
240
+ src = File.read(filename)
241
+ src, mod_src, comments = modified_src(src, filename)
242
+
243
+ comments.each{|c|
244
+ case c.text
245
+ when /\A\#rstfilter\s(.+)/
246
+ optparse! Shellwords.split($1)
247
+ end
248
+ } unless @opt.ignore_pragma
249
+
250
+ return exec_mod_src(mod_src), src, comments
251
+ end
252
+
253
+ def make_line_records rs
254
+ lrs = {}
255
+ rs.each{|(_bl, _bc, el, _ec), result|
256
+ lrs[el] = result
257
+ }
258
+ lrs
259
+ end
260
+
261
+ def process filename
262
+ records, src, comments = record_records filename
263
+ pp records: records if @opt.verbose
264
+ line_records = records.map{|r|
265
+ make_line_records r
266
+ }
267
+
268
+ case @opt.dump
269
+ when :json
270
+ require 'json'
271
+ puts JSON.dump(records)
272
+ else
273
+ replace_comments = comments.filter_map{|c|
274
+ next unless c.text.start_with? @opt.comment_pattern
275
+ e = c.loc.expression
276
+ [e.begin.line, true]
277
+ }.to_h
278
+
279
+ src.each_line.with_index{|line, i|
280
+ lineno = i+1
281
+ line_results = line_records.map{|r| r[lineno]&.first}.compact
282
+
283
+ if line_results.empty?
284
+ puts line
285
+ else
286
+ if replace_comments[lineno]
287
+ line.match(/(.+)#{@opt.comment_pattern}.*$/) || raise("unreachable")
288
+ puts_result $1, line_results, line
289
+ elsif @opt.show_all_results
290
+ puts_result line, line_results
291
+ end
292
+ end
293
+
294
+ if @opt.show_output && !line_results.empty?
295
+ if m = line.match(/^\s+/)
296
+ indent = ' ' * m[0].size
297
+ else
298
+ indent = ''
299
+ end
300
+
301
+ line_outputs = line_records.map{|r| r[lineno]}.compact
302
+ line_outputs.each.with_index{|r, i|
303
+ out, err = *r[1..2]
304
+ label = @opt.exec_command && @opt.exec_command[i].label
305
+ label += ':' if label
306
+
307
+ {out: out, err: err}.each{|k, o|
308
+ o.strip!
309
+ o.each_line{|ol|
310
+ puts "#{indent}\##{label ? label : nil}#{k}: #{ol}"
311
+ } unless o.empty?
312
+ }
313
+ }
314
+ end
315
+ }
316
+
317
+ if !@opt.show_output && !@output.empty?
318
+ puts "# output"
319
+ puts output
320
+ end
321
+ end
322
+ end
323
+ end
324
+ end
325
+
326
+ if $0 == __FILE__
327
+ require_relative 'rewriter'
328
+ filter = RstFilter::Exec.new
329
+ filter.optparse! ['-v']
330
+ file = ARGV.shift || File.expand_path(__dir__ + '/../../sample.rb')
331
+ filter.process File.expand_path(file)
332
+ end
@@ -0,0 +1,95 @@
1
+ require 'stringio'
2
+
3
+ if defined?(RSTFILTER__)
4
+ Object.remove_const :RSTFILTER__
5
+ end
6
+
7
+ class RSTFILTER__
8
+ SHOW_EXCEPTION = ENV['RSTFILTER_SHOW_EXCEPTIONS']
9
+
10
+ if ::ENV['RSTFILTER_PP']
11
+ require 'pp'
12
+ def self.__rst_inspect_body__ val
13
+ ::PP.pp(val, '')
14
+ end
15
+ else
16
+ def self.__rst_inspect_body__ val
17
+ val.inspect
18
+ end
19
+ end
20
+
21
+ def self.__rst_inspect__ val
22
+ begin
23
+ __rst_inspect_body__ val
24
+ rescue Exception => e
25
+ "!! __rst_inspect__ failed: #{e}"
26
+ end
27
+ end
28
+
29
+ @@records = {}
30
+
31
+ def self.records
32
+ @@records
33
+ end
34
+
35
+ def self.clear
36
+ @@records.clear
37
+ end
38
+
39
+ def self.write begin_line, begin_col, end_line, end_col, val, prefix
40
+ # p [begin_line, begin_col, end_line, end_col]
41
+ out, err = *[$__rst_filter_captured_out, $__rst_filter_captured_err].map{|o|
42
+ str = o.string
43
+ o.string = ''
44
+ str
45
+ } if $__rst_filter_captured_out
46
+
47
+ @@records[[begin_line, begin_col, end_line, end_col]] = ["#{prefix}#{__rst_inspect__(val)}", out, err]
48
+ end
49
+
50
+ def self.record begin_line, begin_col, end_line, end_col
51
+ r = yield
52
+ write begin_line, begin_col, end_line, end_col, r, nil
53
+ r
54
+ rescue Exception => e
55
+ write begin_line, begin_col, end_line, end_col, e, 'raised '
56
+ raise
57
+ end
58
+ end
59
+
60
+ if ENV['RSTFILTER_SHOW_OUTPUT']
61
+ $__rst_filter_prev_out = $stdout
62
+ $__rst_filter_prev_err = $stderr
63
+
64
+ $__rst_filter_captured_out = $stdout = StringIO.new
65
+ if false # debug
66
+ $__rst_filter_captured_err = $captured_out
67
+ else
68
+ $__rst_filter_captured_err = $stderr = StringIO.new
69
+ end
70
+ else
71
+ $__rst_filter_prev_out = $__rst_filter_prev_err = nil
72
+ $__rst_filter_captured_out = $__rst_filter_captured_err = nil
73
+ end
74
+
75
+ if path = ENV['RSTFILTER_RECORD_PATH']
76
+ # inter-process communication
77
+
78
+ END{
79
+ open(path, 'wb'){|f|
80
+ f.write Marshal.dump(::RSTFILTER__.records)
81
+ }
82
+ }
83
+
84
+ class RubyVM::InstructionSequence
85
+ RST_FILENAME = ENV['RSTFILTER_FILENAME']
86
+ RST_MOD_SRC = File.read(ENV['RSTFILTER_MOD_SRC_PATH'])
87
+ @found = false
88
+ def self.translate iseq
89
+ if !@found && iseq.path == RST_FILENAME && iseq.label == "<main>"
90
+ @found = true
91
+ RubyVM::InstructionSequence.compile RST_MOD_SRC, RST_FILENAME
92
+ end
93
+ end
94
+ end
95
+ end
@@ -0,0 +1,312 @@
1
+ require 'json'
2
+ require_relative '../version'
3
+ require_relative '../rewriter'
4
+ require_relative '../exec'
5
+
6
+ module RstFilter
7
+ class CancelRequest < StandardError
8
+ end
9
+
10
+ class LSP
11
+ def initialize input: $stdin, output: $stdout, err: $stderr, indent: 50
12
+ @input = input
13
+ @output = output
14
+ @err = err
15
+ @indent = indent
16
+ @records = {} # {filename => [record, line_record, src]}
17
+ @server_request_id = 0
18
+ @exit_status = 1
19
+ @running = {} # {filename => Thread}
20
+ end
21
+
22
+ def self.reload
23
+ ::RstFilter.send(:remove_const, :LSP)
24
+ $".delete_if{|e| /lsp\/handler\.rb$/ =~ e}
25
+ require __FILE__
26
+ end
27
+
28
+ def start
29
+ # for reload
30
+ trap(:USR1){
31
+ Thread.main.raise "reload"
32
+ }
33
+
34
+ lsp = self
35
+ begin
36
+ lsp.event_loop
37
+ rescue Exception => e
38
+ log e
39
+ log e.backtrace
40
+
41
+ # reload
42
+ lsp.class.reload
43
+ lsp = RstFilter::LSP.new
44
+ retry
45
+ end
46
+ end
47
+
48
+ def event_loop
49
+ while req = recv_message
50
+ if req[:id]
51
+ handle_request req
52
+ else
53
+ handle_notification req
54
+ end
55
+ end
56
+ end
57
+
58
+ def log msg
59
+ @err.puts msg
60
+ end
61
+
62
+ def recv_message
63
+ log "wait from #{@input.inspect}"
64
+ line = @input.gets
65
+ line.match(/Content-Length: (\d+)/) || raise("irregular json-rpc: #{line}")
66
+ @input.gets
67
+ msg = JSON.parse(@input.read($1.to_i), symbolize_names: true)
68
+ log "[recv] #{msg.inspect}"
69
+ msg
70
+ end
71
+
72
+ def send_message type, msg_text
73
+ log "[#{type}] #{msg_text}"
74
+
75
+ text = "Content-Length: #{msg_text.size}\r\n" \
76
+ "\r\n" \
77
+ "#{msg_text}"
78
+ @output.write text
79
+ @output.flush
80
+ if true
81
+ log '----'
82
+ log text
83
+ log '----'
84
+ end
85
+ end
86
+
87
+ def send_response req, kw
88
+ res_text = JSON.dump({
89
+ jsonrpc: "2.0",
90
+ id: req[:id],
91
+ result: kw,
92
+ })
93
+
94
+ send_message 'response', res_text
95
+ end
96
+
97
+ def send_request method, kw = {}
98
+ msg_text = JSON.dump({
99
+ jsonrpc: "2.0",
100
+ method: method,
101
+ id: (@server_request_id+=1),
102
+ params: {
103
+ **kw
104
+ }
105
+ })
106
+
107
+ send_message 'request', msg_text
108
+ end
109
+
110
+ def send_notice method, kw = {}
111
+ msg_text = JSON.dump({
112
+ jsonrpc: "2.0",
113
+ method: method,
114
+ params: {
115
+ **kw
116
+ }
117
+ })
118
+ send_message "notice", msg_text
119
+ end
120
+
121
+ def handle_request req
122
+ if req[:error]
123
+ log "error: #{req.inspect}"
124
+ return
125
+ end
126
+
127
+ case req[:method]
128
+ when 'initialize'
129
+ send_response req, {
130
+ capabilities: {
131
+ textDocumentSync: {
132
+ openClose: true,
133
+ change: 2, # Incremental
134
+ # save: true,
135
+ },
136
+
137
+ inlineValueProvider: true,
138
+
139
+ hoverProvider: true,
140
+
141
+ inlayHintProvider: {
142
+ resolveProvider: true,
143
+ },
144
+
145
+ },
146
+ serverInfo: {
147
+ name: "rstfilter-lsp-server",
148
+ version: '0.0.1',
149
+ }
150
+ }
151
+ when 'textDocument/hover'
152
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
153
+ line = req.dig(:params, :position, :line)
154
+ char = req.dig(:params, :position, :character)
155
+ if (record, _line_record, src = @records[filename])
156
+ line += 1
157
+ rs = record.find_all{|(bl, bc, el, ec), _v| cover?(line, char, bl, bc, el, ec)}
158
+ if rs.empty?
159
+ send_response req, nil
160
+ else
161
+ r = rs.min_by{|(_bl, _bc, _el, ec), _v| ec}
162
+ v = r[1][0].strip
163
+ pos = r[0]
164
+ send_response req, contents: {
165
+ kind: 'markdown',
166
+ value: "```\n#{v}```",
167
+ }, range: {
168
+ start: {line: pos[0]-1, character: pos[1],},
169
+ end: {line: pos[2]-1, character: pos[3],},
170
+ }
171
+ end
172
+ else
173
+ send_response req, nil
174
+ end
175
+ when 'textDocument/codeLens'
176
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
177
+ send_response req, codelens(filename)
178
+ when 'textDocument/inlayHint'
179
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
180
+ send_response req, inlayhints(filename)
181
+ when 'inlayHint/resolve'
182
+ hint = req.dig(:params)
183
+ send_response req, {
184
+ position: hint[:position],
185
+
186
+ label: [{
187
+ tooltip: {
188
+ kind: 'markdown',
189
+ value: hint[:label] + "\n 1. *foo* `bar` _baz_",
190
+ tooltip: hint[:label] + "\n 1. *foo* `bar` _baz_",
191
+ }
192
+ },
193
+ {
194
+ kind: 'markdown',
195
+ value: hint[:label] + "\n# 2. FOO\n*foo* `bar` _baz_",
196
+ }],
197
+ }
198
+ when 'shutdown'
199
+ @exit_status = 0
200
+ send_response req, nil
201
+ when nil
202
+ # reply
203
+ else
204
+ raise "unknown request: #{req.inspect}"
205
+ end
206
+ end
207
+
208
+ def cover? line, char, bl, bc, el, ec
209
+ return false if bl > line
210
+ return false if bl == line && char < bc
211
+ return false if el < line
212
+ return false if el == line && char >= ec
213
+ true
214
+ end
215
+
216
+ def inlayhints filename
217
+ if (_record, line_record, src = @records[filename])
218
+ src_lines = src.lines.to_a
219
+
220
+ line_record.sort_by{|k, v| k}.map do |lineno, r|
221
+ line = src_lines[lineno - 1]
222
+ next unless line
223
+
224
+ {
225
+ position: {
226
+ line: lineno - 1, # 0 origin
227
+ character: line.length,
228
+ },
229
+ label: ' ' * [@indent - line.size, 3].max + "#=> #{r.first.strip}",
230
+ # tooltip: "tooltip of #{lineno}",
231
+ paddingLeft: true,
232
+ kind: 1,
233
+ }
234
+ end.compact
235
+ else
236
+ nil
237
+ end
238
+ end
239
+
240
+ def take_record filename
241
+ send_notice 'rstfilter/started', {
242
+ uri: filename,
243
+ }
244
+ @running[filename] = Thread.new do
245
+ filter = RstFilter::Exec.new
246
+ filter.optparse! ['--pp', '-eruby']
247
+ records, src, _comments = filter.record_records(filename)
248
+ records = records.first # only 1 process results
249
+ @records[filename] = [records, filter.make_line_records(records), src]
250
+ send_notice 'rstfilter/done'
251
+ unless filter.output.empty?
252
+ send_notice 'rstfilter/output', output: "# Output for #{filename}\n\n#{filter.output}"
253
+ end
254
+ send_request 'workspace/inlayHint/refresh'
255
+ rescue CancelRequest
256
+ # canceled
257
+ rescue SyntaxError => e
258
+ send_notice 'rstfilter/output', output: "SyntaxError on #{filename}:\n#{e.inspect}"
259
+ send_notice 'rstfilter/done'
260
+ rescue Exception => e
261
+ send_notice 'rstfilter/output', output: "Error on #{filename}:\n#{e.inspect}\n#{e.backtrace.join("\n")}#{filter.output}"
262
+ send_notice 'rstfilter/done'
263
+ ensure
264
+ @running[filename] = nil
265
+ end
266
+ end
267
+
268
+ def clear_record filename
269
+ @records[filename] = nil
270
+ if th = @running[filename]
271
+ @running[filename] = nil
272
+ th.raise CancelRequest
273
+ end
274
+ end
275
+
276
+ def uri2filename uri
277
+ case uri
278
+ when /^file:\/\/(.+)/
279
+ $1
280
+ else
281
+ raise "unknown uri: #{uri}"
282
+ end
283
+ end
284
+
285
+ def handle_notification req
286
+ case req[:method]
287
+ when 'initialized'
288
+ send_notice 'rstfilter/version', {
289
+ version: "#{::RstFilter::VERSION} on #{RUBY_DESCRIPTION}",
290
+ }
291
+ when 'textDocument/didOpen',
292
+ 'textDocument/didSave'
293
+ #filename = uri2filename req.dig(:params, :textDocument, :uri)
294
+ #take_record filename
295
+ # do nothing
296
+ when 'textDocument/didClose',
297
+ 'textDocument/didChange'
298
+ filename = uri2filename req.dig(:params, :textDocument, :uri)
299
+ clear_record filename
300
+ when 'rstfilter/start'
301
+ filename = req.dig(:params, :path)
302
+ take_record filename
303
+ when '$/cancelRequest'
304
+ # ignore
305
+ when 'exit'
306
+ exit(@exit_code)
307
+ else
308
+ raise
309
+ end
310
+ end
311
+ end
312
+ end
@@ -0,0 +1,178 @@
1
+
2
+ require 'parser/current'
3
+ require 'pp'
4
+
5
+ module RstFilter
6
+ class RecordAll < Parser::TreeRewriter
7
+ def initialize opt
8
+ @decl = opt.show_decl || (opt.show_all_results == false)
9
+
10
+ super()
11
+ end
12
+
13
+ def add_record node
14
+ if le = node&.location&.expression
15
+ pos = [le.begin.line, le.begin.column, le.end.line, le.end.column].join(',')
16
+ insert_before(le.begin, "::RSTFILTER__.record(#{pos}){")
17
+ insert_after(le.end, "}")
18
+ end
19
+ end
20
+
21
+ def add_paren node
22
+ if le = node&.location&.expression
23
+ insert_before(le.begin, '(')
24
+ insert_after(le.end, ")")
25
+ end
26
+ end
27
+
28
+ def process node
29
+ return unless node
30
+
31
+ super
32
+
33
+ case node.type
34
+ when :begin,
35
+ :resbody, :rescue,
36
+ :ensure,
37
+ :return,
38
+ :next,
39
+ :redo,
40
+ :retry,
41
+ :splat,
42
+ :block_pass,
43
+ :lvasgn,
44
+ :when
45
+ # skip
46
+ when :def, :class
47
+ add_record node if @decl
48
+ when :if
49
+ unless node.loc.expression.source.start_with? 'elsif'
50
+ add_record node
51
+ end
52
+ else
53
+ add_record node
54
+ end
55
+ end
56
+
57
+ def on_dstr node
58
+ end
59
+
60
+ def on_regexp node
61
+ end
62
+
63
+ def on_const node
64
+ end
65
+
66
+ def on_masgn node
67
+ _mlhs, rhs = node.children
68
+ if rhs.type == :array
69
+ rhs.children.each{|r| process r}
70
+ end
71
+ end
72
+
73
+ def on_class node
74
+ _name, sup, body = node.children
75
+ process sup
76
+ process body
77
+ end
78
+
79
+ def on_module node
80
+ _name, body = node.children
81
+ process body
82
+ end
83
+
84
+ def process_args args
85
+ args.children.each{|arg|
86
+ case arg.type
87
+ when :optarg
88
+ _name, opexpr = arg.children
89
+ process opexpr
90
+ when :kwoptarg
91
+ _name, kwexpr = arg.children
92
+ process kwexpr
93
+ end
94
+ }
95
+ end
96
+
97
+ def on_def node
98
+ _name, args, body = node.children
99
+ process_args args
100
+ process body
101
+ end
102
+
103
+ def on_defs node
104
+ recv, _name, args, body = node.children
105
+ process recv
106
+ add_paren recv
107
+ process_args args
108
+ process body
109
+ end
110
+
111
+ def process_pairs pairs
112
+ pairs.each{|pair|
113
+ key, val = pair.children
114
+ if key.type != :sym
115
+ process key
116
+ end
117
+ process val
118
+ }
119
+ end
120
+
121
+ def on_hash node
122
+ process_pairs node.children
123
+ end
124
+
125
+ def on_send node
126
+ recv, _name, *args = *node.children
127
+ process recv if recv
128
+
129
+ args.each{|arg|
130
+ if arg.type == :hash
131
+ process_pairs arg.children
132
+ else
133
+ process arg
134
+ end
135
+ }
136
+ end
137
+
138
+ def on_block node
139
+ _send, _args, block = *node.children
140
+ process block
141
+ end
142
+
143
+ def on_numblock node
144
+ on_block node
145
+ end
146
+ end
147
+
148
+ class Rewriter
149
+ def initialize opt
150
+ @opt = opt
151
+ end
152
+
153
+ def rewrite src, filename
154
+ # check syntax
155
+ prev_v, $VERBOSE = $VERBOSE, false
156
+ ast = RubyVM::AbstractSyntaxTree.parse(src)
157
+ $VERBOSE = prev_v
158
+ last_lineno = ast.last_lineno
159
+
160
+ # rewrite
161
+ src = src.lines[0..last_lineno].join # remove __END__ and later
162
+ ast, comments = Parser::CurrentRuby.parse_with_comments(src)
163
+ buffer = Parser::Source::Buffer.new('(example)')
164
+ buffer.source = src
165
+ rewriter = RecordAll.new @opt
166
+ mod_src = rewriter.rewrite(buffer, ast)
167
+
168
+ if @opt.verbose
169
+ pp ast
170
+ puts " #{(0...80).map{|i| i%10}.join}"
171
+ puts mod_src.lines.map.with_index{|line, i| '%4d:%s' % [i+1, line] }
172
+ end
173
+
174
+ return src, mod_src, comments
175
+ end
176
+ end
177
+ end
178
+
@@ -1,3 +1,3 @@
1
- module Rstfilter
2
- VERSION = "0.1.0"
1
+ module RstFilter
2
+ VERSION = "0.4.0"
3
3
  end
data/lib/rstfilter.rb CHANGED
@@ -1,287 +1,11 @@
1
1
  # require "rstfilter/version"
2
2
 
3
- require 'parser/current'
4
- require 'stringio'
5
- require 'optparse'
6
- require 'ostruct'
7
- require 'pp'
8
-
9
- module RstFilter
10
- class RecordAll < Parser::TreeRewriter
11
- def initialize opt
12
- @decl = opt.show_decl || (opt.show_all_results == false)
13
-
14
- super()
15
- end
16
-
17
- def add_record node
18
- if le = node&.location&.expression
19
- insert_before(le.begin, '(')
20
- insert_after(le.end, ").__rst_record__(#{le.end.line}, #{le.end.column})")
21
- end
22
- end
23
-
24
- def process node
25
- return unless node
26
- case node.type
27
- when :resbody, :rescue, :begin
28
- # skip
29
- when :def, :class
30
- add_record node if @decl
31
- else
32
- add_record node
33
- end
34
-
35
- super
36
- end
37
-
38
- def on_class node
39
- _name, sup, body = node.children
40
- process sup
41
- process body
42
- end
43
-
44
- def on_def node
45
- _name, args, body = node.children
46
- args.children.each{|arg|
47
- case arg.type
48
- when :optarg
49
- _name, opexpr = arg.children
50
- process opexpr
51
- when :kwoptarg
52
- _name, kwexpr = arg.children
53
- process kwexpr
54
- end
55
- }
56
- process body
57
- end
58
-
59
- def on_send node
60
- end
61
- end
62
-
63
- class Exec
64
- DEFAULT_SETTING = {
65
- # default setting
66
- show_all_results: true,
67
- show_exceptions: true,
68
- show_output: false,
69
- show_decl: false,
70
- show_specific_line: nil,
71
-
72
- use_pp: false,
73
- comment_indent: 50,
74
- comment_pattern: '#=>',
75
- verbose: false,
76
- }
77
-
78
- ConfigOption = Struct.new(*DEFAULT_SETTING.keys, keyword_init: true)
79
-
80
- def update_opt opt = {}
81
- opt = @opt.to_h.merge(opt)
82
- @opt = ConfigOption.new(**opt)
83
- end
84
-
85
- def initialize
86
- @opt = ConfigOption.new(**DEFAULT_SETTING)
87
- end
88
-
89
- def optparse! argv
90
- opt = {}
91
- o = OptionParser.new
92
- o.on('-c', '--comment', 'Show result only on comment'){
93
- opt[:show_all_results] = false
94
- }
95
- o.on('-o', '--output', 'Show output results'){
96
- opt[:show_output] = true
97
- }
98
- o.on('-d', '--decl', 'Show results on declaration'){
99
- opt[:show_decl] = true
100
- }
101
- o.on('--no-exception', 'Do not show exception'){
102
- opt[:show_exception] = false
103
- }
104
- o.on('-a', '--all', 'Show all results/output'){
105
- opt[:show_output] = true
106
- opt[:show_all_results] = true
107
- opt[:show_exceptions] = true
108
- }
109
- o.on('--pp', 'Use pp to represent objects'){
110
- opt[:use_pp] = true
111
- }
112
- o.on('--comment-indent=NUM', "Specify comment indent size (default: #{DEFAULT_SETTING[:comment_indent]})"){|n|
113
- opt[:comment_indent] = n.to_i
114
- }
115
- o.on('--verbose', 'Verbose mode'){
116
- opt[:verbose] = true
117
- }
118
- o.parse!(argv)
119
- update_opt opt
120
- end
121
-
122
- def capture_out
123
- return yield unless @opt.show_output
124
-
125
- begin
126
- prev_out = $stdout
127
- prev_err = $stderr
128
-
129
- $captured_out = $stdout = StringIO.new
130
- if false # debug
131
- $captured_err = $captured_out
132
- else
133
- $captured_err = $stderr = StringIO.new
134
- end
135
-
136
- yield
137
- ensure
138
- $stdout = prev_out
139
- $stderr = prev_err
140
- end
141
- end
142
-
143
- def record_rescue
144
- if @opt.show_exceptions
145
- TracePoint.new(:raise) do |tp|
146
- caller_locations.each{|loc|
147
- if loc.path == @filename
148
- $__rst_record[loc.lineno][0] = [tp.raised_exception, '', '']
149
- break
150
- end
151
- }
152
- end.enable do
153
- yield
154
- end
155
- else
156
- yield
157
- end
158
- end
159
-
160
- class ::BasicObject
161
- def __rst_record__ line, col
162
- out, err = *[$captured_out, $captured_err].map{|o|
163
- str = o.string
164
- o.string = ''
165
- str
166
- } if $captured_out
167
-
168
- # ::STDERR.puts [line, col, self, out, err].inspect
169
- $__rst_record[line][col] = [self, out, err]
170
-
171
- self
172
- end
173
- end
174
-
175
- $__rst_record = Hash.new{|h, k| h[k] = []}
176
-
177
- def process filename
178
- @filename = filename
179
-
180
- code = File.read(filename)
181
-
182
- begin
183
- ast, comments = Parser::CurrentRuby.parse_with_comments(code)
184
- rescue Parser::SyntaxError => e
185
- puts e
186
- exit 1
187
- end
188
-
189
- end_line = ast.loc.expression.end.line
190
- code = code.lines[0..end_line].join # remove __END__ and later
191
-
192
- buffer = Parser::Source::Buffer.new('(example)')
193
- buffer.source = code
194
- rewriter = RecordAll.new @opt
195
-
196
- mod_src = rewriter.rewrite(buffer, ast)
197
- puts mod_src.lines.map.with_index{|line, i| '%4d: %s' % [i+1, line] } if @opt.verbose
198
-
199
- begin
200
- capture_out do
201
- record_rescue do
202
- ::TOPLEVEL_BINDING.eval(mod_src, filename)
203
- end
204
- end
205
- rescue Exception => e
206
- if @opt.verbose
207
- STDERR.puts e
208
- STDERR.puts e.backtrace
209
- else
210
- STDERR.puts "RstFilter: exit with #{e.inspect}"
211
- end
212
- end
213
-
214
- replace_comments = {}
215
-
216
- comments.each{|c|
217
- next unless c.text.start_with? @opt.comment_pattern
218
- e = c.loc.expression
219
- line, _col = e.begin.line, e.begin.column
220
- if $__rst_record.has_key? line
221
- result = $__rst_record[line].last
222
- replace_comments[line] = result
223
- end
224
- }
225
-
226
- pp $__rst_record if @opt.verbose
227
-
228
- code.each_line.with_index{|line, i|
229
- line_result = $__rst_record[i+1]&.last
230
-
231
- if line_result && line.match(/(.+)#{@opt.comment_pattern}.*$/)
232
- prefix = $1
233
- r = line_result.first
234
- if @opt.use_pp
235
- result_lines = PP.pp(r, '').lines
236
- else
237
- result_lines = r.inspect.lines
238
- end
239
- puts line.sub(/#{@opt.comment_pattern}.*$/, "#{@opt.comment_pattern} #{result_lines.shift.chomp}")
240
- cont_comment = '#' + ' ' * @opt.comment_pattern.size
241
- result_lines.each{|result_line|
242
- puts ' ' * prefix.size + "#{cont_comment}#{result_line}"
243
- }
244
- elsif @opt.show_all_results && line_result
245
- r = line_result.first
246
- indent = ' ' * [@opt.comment_indent - line.chomp.length, 0].max
247
- if @opt.use_pp
248
- result_lines = PP.pp(r, '').lines
249
- else
250
- result_lines = r.inspect.lines
251
- end
252
- prefix = line.chomp.concat "#{indent}"
253
- puts "#{prefix} #=> #{result_lines.shift}"
254
- cont_comment = '#' + ' ' * @opt.comment_pattern.size
255
- result_lines.each{|result_line|
256
- puts ' ' * prefix.size + " #{cont_comment}#{result_line}"
257
- }
258
- else
259
- puts line
260
- end
261
-
262
- if @opt.show_output && line_result
263
- out, err = *line_result[1..2]
264
- if m = line.match(/^\s+/)
265
- indent = ' ' * m[0].size
266
- else
267
- indent = ''
268
- end
269
-
270
- {out: out, err: err}.each{|k, o|
271
- o.strip!
272
- o.each_line{|ol|
273
- puts "#{indent}##{k}: #{ol}"
274
- } unless o.empty?
275
- }
276
- end
277
- }
278
- end
279
- end
280
- end
3
+ require_relative 'rstfilter/rewriter'
4
+ require_relative 'rstfilter/exec'
281
5
 
282
6
  if __FILE__ == $0
283
7
  filter = RstFilter::Exec.new
284
- filter.optparse ARGV
285
- file = ARGV.shift
8
+ filter.optparse! ['-o', '-v']
9
+ file = ARGV.shift || File.expand_path(__dir__ + '/../sample.rb')
286
10
  filter.process File.expand_path(file)
287
11
  end
data/rstfilter.gemspec CHANGED
@@ -2,7 +2,7 @@ require_relative 'lib/rstfilter/version'
2
2
 
3
3
  Gem::Specification.new do |spec|
4
4
  spec.name = "rstfilter"
5
- spec.version = Rstfilter::VERSION
5
+ spec.version = RstFilter::VERSION
6
6
  spec.authors = ["Koichi Sasada"]
7
7
  spec.email = ["ko1@atdot.net"]
8
8
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rstfilter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Koichi Sasada
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-05-26 00:00:00.000000000 Z
11
+ date: 2022-06-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: parser
@@ -29,9 +29,11 @@ email:
29
29
  - ko1@atdot.net
30
30
  executables:
31
31
  - rstfilter
32
+ - rstfilter-lsp
32
33
  extensions: []
33
34
  extra_rdoc_files: []
34
35
  files:
36
+ - ".github/workflows/ruby.yml"
35
37
  - ".gitignore"
36
38
  - ".travis.yml"
37
39
  - Gemfile
@@ -41,7 +43,12 @@ files:
41
43
  - bin/console
42
44
  - bin/setup
43
45
  - exe/rstfilter
46
+ - exe/rstfilter-lsp
44
47
  - lib/rstfilter.rb
48
+ - lib/rstfilter/exec.rb
49
+ - lib/rstfilter/exec_setup.rb
50
+ - lib/rstfilter/lsp/handler.rb
51
+ - lib/rstfilter/rewriter.rb
45
52
  - lib/rstfilter/version.rb
46
53
  - rstfilter.gemspec
47
54
  homepage: https://github.com/ko1/rstfilter
@@ -66,7 +73,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
66
73
  - !ruby/object:Gem::Version
67
74
  version: '0'
68
75
  requirements: []
69
- rubygems_version: 3.1.6
76
+ rubygems_version: 3.3.7
70
77
  signing_key:
71
78
  specification_version: 4
72
79
  summary: Show Ruby script with execution results.