typeprof 0.20.1 → 0.21.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +6 -6
- data/lib/typeprof/analyzer.rb +53 -13
- data/lib/typeprof/block.rb +2 -1
- data/lib/typeprof/container-type.rb +3 -2
- data/lib/typeprof/export.rb +39 -4
- data/lib/typeprof/import.rb +33 -4
- data/lib/typeprof/insns-def.rb +2 -0
- data/lib/typeprof/iseq.rb +2 -2
- data/lib/typeprof/method.rb +25 -4
- data/lib/typeprof/type.rb +4 -4
- data/lib/typeprof/version.rb +1 -1
- data/typeprof.gemspec +2 -2
- data/vscode/package-lock.json +1069 -31
- data/vscode/package.json +3 -3
- data/vscode/src/extension.ts +31 -14
- metadata +4 -12
- data/doc/demo.md +0 -398
- data/doc/doc.ja.md +0 -424
- data/doc/doc.md +0 -437
- data/doc/ide.md +0 -81
- data/doc/ppl2019.pdf +0 -0
- data/doc/todo.md +0 -133
- data/doc/typeprof-for-ide-log.png +0 -0
- data/doc/typeprof-for-ide.png +0 -0
data/doc/doc.md
DELETED
@@ -1,437 +0,0 @@
|
|
1
|
-
# TypeProf: A type analysis tool for Ruby code based on abstract interpretation
|
2
|
-
|
3
|
-
## Show me demo first
|
4
|
-
|
5
|
-
See [demo.md](demo.md).
|
6
|
-
|
7
|
-
## How to use TypeProf
|
8
|
-
|
9
|
-
Analyze app.rb:
|
10
|
-
|
11
|
-
```
|
12
|
-
$ typeprof app.rb
|
13
|
-
```
|
14
|
-
|
15
|
-
Analyze app.rb with sig/app.rbs that specifies some method types:
|
16
|
-
|
17
|
-
```
|
18
|
-
$ typeprof sig/app.rbs app.rb
|
19
|
-
```
|
20
|
-
|
21
|
-
Here is a typical use case:
|
22
|
-
|
23
|
-
```
|
24
|
-
$ typeprof sig/app.rbs app.rb -o sig/app.gen.rbs
|
25
|
-
```
|
26
|
-
|
27
|
-
Here is a list of currently avaiable options:
|
28
|
-
|
29
|
-
* `-o OUTFILE`: Write the analyze result to OUTFILE instead of standard output
|
30
|
-
* `-q`: Hide the progress indicator
|
31
|
-
* `-v`: Alias to `-fshow-errors`
|
32
|
-
* `-d`: Show the analysis log (Currently, the log is just for debugging and may become very huge)
|
33
|
-
* `-I DIR`: Add `DIR` to the file search path of `require`
|
34
|
-
* `-r GEMNAME`: Load the RBS files of `GEMNAME`
|
35
|
-
* `--exclude-dir DIR`: Omit the result of files that are placed under the directory `DIR`. If there are some directory specifications, the latter one is stronger. (Assuming that `--include-dir foo --exclude-dir foo/bar` is specified, the analysis result of foo/bar/baz.rb is omitted, but foo/baz.rb is shown.)
|
36
|
-
* `--include-dir DIR`: Show the result of files that are placed under the directory `DIR`. If there are some directory specifications, the latter one is stronger. (Assuming that `--exclude-dir foo --include-dir foo/bar` is specified, the analysis result of foo/bar/baz.rb is shown, but foo/baz.rb is omitted.)
|
37
|
-
* `--show-errors`: Prints out possible bugs found during execution (often a lot of false positives).
|
38
|
-
* `--show-untyped`: When TypeProf infers a type `A | untyped`, it simply outputs `A` by default. But this option forces to output `A | untyped`.
|
39
|
-
* `--type-depth-limit=NUM`: (undocumented yet)
|
40
|
-
|
41
|
-
## What is a TypeProf?
|
42
|
-
|
43
|
-
TypeProf is a Ruby interpreter that *abstractly* executes Ruby programs at the type level.
|
44
|
-
It executes a given program and observes what types are passed to and returned from methods and what types are assigned to instance variables.
|
45
|
-
All values are, in principle, abstracted to the class to which the object belongs, not the object itself (detailed in the next section).
|
46
|
-
|
47
|
-
Here is an example of a method call.
|
48
|
-
|
49
|
-
```
|
50
|
-
def foo(n)
|
51
|
-
p n #=> Integer
|
52
|
-
n.to_s
|
53
|
-
end
|
54
|
-
|
55
|
-
p foo(42) #=> String
|
56
|
-
```
|
57
|
-
|
58
|
-
The analysis results of TypeProf are as follows.
|
59
|
-
|
60
|
-
```
|
61
|
-
$ ruby exe/typeprof test.rb
|
62
|
-
# Revealed types
|
63
|
-
# test.rb:2 #=> Integer
|
64
|
-
# test.rb:6 #=> String
|
65
|
-
|
66
|
-
# Classes
|
67
|
-
class Object
|
68
|
-
def foo : (Integer) -> String
|
69
|
-
end
|
70
|
-
```
|
71
|
-
|
72
|
-
When the method call `foo(42)` is executed, the type (abstract value) "`Integer`" is passed instead of the `Integer` object 42.
|
73
|
-
The method `foo` executes `n.to_s`.
|
74
|
-
Then, the built-in method `Integer#to_s` is called and you get the type "`String`", which the method `foo` returns.
|
75
|
-
Collecting observations of these execution results, TypeProf outputs, "the method `foo` receives `Integer` and returns `String`" in the RBS format.
|
76
|
-
Also, the argument of `p` is output in the `Revealed types` section.
|
77
|
-
|
78
|
-
Instance variables are stored in each object in Ruby, but are aggregated in class units in TypeProf.
|
79
|
-
|
80
|
-
```
|
81
|
-
class Foo
|
82
|
-
def initialize
|
83
|
-
@a = 42
|
84
|
-
end
|
85
|
-
|
86
|
-
attr_accessor :a
|
87
|
-
end
|
88
|
-
|
89
|
-
Foo.new.a = "str"
|
90
|
-
|
91
|
-
p Foo.new.a #=> Integer | String
|
92
|
-
```
|
93
|
-
|
94
|
-
```
|
95
|
-
$ ruby exe/typeprof test.rb
|
96
|
-
# Revealed types
|
97
|
-
# test.rb:11 #=> Integer | String
|
98
|
-
|
99
|
-
# Classes
|
100
|
-
class Foo
|
101
|
-
attr_accessor a : Integer | String
|
102
|
-
def initialize : -> Integer
|
103
|
-
end
|
104
|
-
```
|
105
|
-
|
106
|
-
|
107
|
-
## Abstract values
|
108
|
-
|
109
|
-
As mentioned above, TypeProf abstracts almost all Ruby values to the type level, with some exceptions like class objects.
|
110
|
-
To avoid confusion with normal Ruby values, we use the word "abstract value" to refer the values that TypeProf handles.
|
111
|
-
|
112
|
-
TypeProf handles the following abstract values.
|
113
|
-
|
114
|
-
* Instance of a class
|
115
|
-
* Class object
|
116
|
-
* Symbol
|
117
|
-
* `untyped`
|
118
|
-
* Union of abstract values
|
119
|
-
* Instance of a container class
|
120
|
-
* Proc object
|
121
|
-
|
122
|
-
Instances of classes are the most common values.
|
123
|
-
A Ruby code `Foo.new` returns an instance of the class `Foo`.
|
124
|
-
This abstract value is represented as `Foo` in the RBS format, though it is a bit confusing.
|
125
|
-
The integer literal `42` generates an instance of `Integer` and the string literal `"str"` generates an instance of `String`.
|
126
|
-
|
127
|
-
A class object is a value that represents the class itself.
|
128
|
-
For example, the constants `Integer` and `String` has class objects.
|
129
|
-
In Ruby semantics, a class object is an instance of the class `Class`, but it is not abstracted into `Class` in TypeProf.
|
130
|
-
This is because, if it is abstracted, TypeProf cannot handle constant references and class methods correctly.
|
131
|
-
|
132
|
-
A symbol is an abstract value returned by Symbol literals like `:foo`.
|
133
|
-
A symbol object is not abstracted to an instance of the class `Symbol` because its concrete vgalue is often required in many cases, such as keyword argumetns, JSON data keys, the argument of `Module#attr_reader`, etc.
|
134
|
-
Note that some Symbol objects are handled as instances of the class `Symbol`, for example, the return value of `String#to_sym` and Symbol literals that contains interpolation like `:"foo_#{ x }"`.
|
135
|
-
|
136
|
-
`untyped` is an abstract value generated when TypeProf fails to trace values due to analysis limits or restrictions.
|
137
|
-
Any operations and method calls on `untyped` are ignored, and the evaluation result is also `untyped`.
|
138
|
-
|
139
|
-
A union of abstract values is a value that represents multiple possibilities.,
|
140
|
-
For (a bit artificial) example, the result of `rand < 0.5 ? 42 : "str"` is a union, `Integer | String`.
|
141
|
-
|
142
|
-
An instance of a container class, such as Array and Hash, is an object that contains other abstract values as elements.
|
143
|
-
At present, only Array, Enumerator and Hash are supported.
|
144
|
-
Details will be described later.
|
145
|
-
|
146
|
-
A Proc object is a closure produced by lambda expressions (`-> {... }`) and block parameters (`&blk`).
|
147
|
-
During the interpretation, these objects are not abstracted but treated as concrete values associated with a piece of code.
|
148
|
-
In the RBS result, they are represented by using anonymous proc type, whose types they accepted and returned.
|
149
|
-
|
150
|
-
|
151
|
-
## Execution
|
152
|
-
|
153
|
-
TypeProf is a kind of Ruby interpreter, but its execution order is quite different from Ruby semantics.
|
154
|
-
|
155
|
-
### Branch
|
156
|
-
|
157
|
-
When it executes a branch, both clauses are executed in parallel. It is unspecified which is evaluated first.
|
158
|
-
|
159
|
-
In the following example, the "then" clause assigns an `Integer` to the variable `x` and the "else" clause assigns a `String` to `x`.
|
160
|
-
|
161
|
-
```ruby
|
162
|
-
if rand <0.5
|
163
|
-
x = 42
|
164
|
-
else
|
165
|
-
x = "str"
|
166
|
-
end
|
167
|
-
|
168
|
-
p x #=> Integer | String
|
169
|
-
```
|
170
|
-
|
171
|
-
TypeProf first evaluates the conditional expression, then does both "then" and "else" clauses (we cannot tell which comes first), and after the branch, evaluates the method call to `p` with `Integer | String`.
|
172
|
-
|
173
|
-
|
174
|
-
### Restart
|
175
|
-
|
176
|
-
If you assign different abstract values to an instance variable, the order of execution may be more complicated.
|
177
|
-
|
178
|
-
```ruby
|
179
|
-
class Foo
|
180
|
-
def initialize
|
181
|
-
@x = 1
|
182
|
-
end
|
183
|
-
|
184
|
-
def get_x
|
185
|
-
@x
|
186
|
-
end
|
187
|
-
|
188
|
-
def update_x
|
189
|
-
@x = "str"
|
190
|
-
end
|
191
|
-
end
|
192
|
-
|
193
|
-
foo = Foo.new
|
194
|
-
|
195
|
-
# ...
|
196
|
-
|
197
|
-
p foo.get_x #=> Integer | String
|
198
|
-
|
199
|
-
# ...
|
200
|
-
|
201
|
-
foo.update_x
|
202
|
-
```
|
203
|
-
|
204
|
-
In the above example, an `Integer` is assigned to the instance variable `@x` in `Foo#initialize`.
|
205
|
-
`Foo#get_x` reads `@x` and returns an `Integer` once.
|
206
|
-
However, when `Foo#update_x` is called, the abstract value of the instance variable `@x` is expanded to `Integer | String`.
|
207
|
-
Therefore, reading `@x` should have returned a `Integer | String` instead of a simple `Integer`, and the access to `@x` in `Foo#get_x` restarts to return `Integer | String`, i.e., retroactively executed again.
|
208
|
-
Therefore, the return value of the call to `Foo#get_x` will eventually be `Integer | String`.
|
209
|
-
|
210
|
-
|
211
|
-
### Method call
|
212
|
-
|
213
|
-
TypeProf does not keep track of the call stack.
|
214
|
-
In other words, there is no concept of "caller" during the execution of the method.
|
215
|
-
Instead, when a method returns, it returns the abstract value to all possible places that may invoke to the method.
|
216
|
-
|
217
|
-
```
|
218
|
-
def fib(n)
|
219
|
-
if n <2
|
220
|
-
return n
|
221
|
-
else
|
222
|
-
fib(n-1) + fib(n-2)
|
223
|
-
end
|
224
|
-
end
|
225
|
-
|
226
|
-
p fib(10) #=> Integer
|
227
|
-
```
|
228
|
-
|
229
|
-
In the above example, the method `fib` is called from three places (including recursive calls).
|
230
|
-
When `return n` is executed, TypeProf returns an `Integer` to all three places.
|
231
|
-
Note that, in Ruby, we cannot statically identify all places that may call to the method (because it depends upon the type of receiver).
|
232
|
-
Therefore, if TypeProf finds a new call to `fib` after `return n` is executed, the call also returns an `Integer` immediately.
|
233
|
-
If a method returns different abstract values, it can lead to retrospective execution.
|
234
|
-
|
235
|
-
|
236
|
-
### Stub execution
|
237
|
-
|
238
|
-
Even after TypeProf traced all programs as possible, there may be methods or blocks that aren't executed.
|
239
|
-
For example, a method is not executed if it is called from nowhere; this is typical for library method that has no test.
|
240
|
-
(Basically, when you use TypeProf, it is recommended to invoke all methods with supposed argument types.)
|
241
|
-
TypeProf forcibly calls these unreachable methods and blocks with `untyped` as arguments.
|
242
|
-
|
243
|
-
```
|
244
|
-
def foo(n)
|
245
|
-
end
|
246
|
-
|
247
|
-
def bar(n)
|
248
|
-
foo(1)
|
249
|
-
end
|
250
|
-
```
|
251
|
-
|
252
|
-
In the above program, neither the method `foo` nor the method `bar` is called.
|
253
|
-
TypeProf stub-calls the `bar` with a `untyped` arugment, so you can get the information that an `Integer` is passed to a method `foo`.
|
254
|
-
|
255
|
-
However, this feature may slow down the analysis and may also brings many wrong guesses, so we plan to allow a user to enable/disable this feature in the configuration.
|
256
|
-
|
257
|
-
|
258
|
-
## Limitations
|
259
|
-
|
260
|
-
Some Ruby language features cannot be handled because they abstract values.
|
261
|
-
|
262
|
-
Basically, it ignores language features whose object identity is important, such as singleton methods for general objects.
|
263
|
-
Note that class method definitions are handled correctly; class objects are not abstracted for the sake.
|
264
|
-
Currently, TypeProf only handles instance methods and class methods; it has no general concept of metaclasses (a class of a class).
|
265
|
-
|
266
|
-
Meta programming is only partially supported.
|
267
|
-
|
268
|
-
* `Module#attr_reader` and `Object#send` handle correctly only when symbol abstract value is passed (for example, when written in a symbol literal).
|
269
|
-
* `Kernel#instance_eval` only supports the function to replace the receiver object when a block is passed (the contents of the string are not tracked).
|
270
|
-
* `Class.new` is not supported; it always returns `untyped`.
|
271
|
-
* `Kernel#require` has a dedicated support only when the argument string is a literal.
|
272
|
-
|
273
|
-
|
274
|
-
## Other features
|
275
|
-
|
276
|
-
### Partial RBS specification
|
277
|
-
|
278
|
-
Sometimes, TypeProf fails to correctly infer the programer's intent due to theoretical or implementation limitations.
|
279
|
-
In such cases, you can manually write a RBS description for some difficult methods to convey your intent to TypeProf.
|
280
|
-
|
281
|
-
For example, TypeProf does not handle a overloaded method.
|
282
|
-
|
283
|
-
```
|
284
|
-
# Programmer Intent: (Integer) -> Integer | (String) -> String
|
285
|
-
# TypeProf : (Integer | String) -> (Integer | String)
|
286
|
-
def foo(n)
|
287
|
-
if n.is_a?(Integer)
|
288
|
-
42
|
289
|
-
else
|
290
|
-
"str"
|
291
|
-
end
|
292
|
-
end
|
293
|
-
|
294
|
-
# Overload intent not respected
|
295
|
-
p foo(42) #=> Integer | String
|
296
|
-
p foo("str") #=> Integer | String
|
297
|
-
```
|
298
|
-
|
299
|
-
Assume that a programmer write the method `foo` as a overloaded method that returns an `Integer` only when an `Integer` is passed, and that returns a `String` only when a `String` is passed.
|
300
|
-
Thus, we expect the result of `foo(42)` to be an `Integer`. However, it's a bit wider result, `Integer | String`.
|
301
|
-
|
302
|
-
If you write the RBS manually to specify the intention of the method `foo`, the result will be as intended.
|
303
|
-
|
304
|
-
```
|
305
|
-
# test.rbs
|
306
|
-
class Object
|
307
|
-
def foo: (Integer) -> Integer | (String) -> String
|
308
|
-
end
|
309
|
-
```
|
310
|
-
|
311
|
-
```
|
312
|
-
# test.rb
|
313
|
-
def foo(n)
|
314
|
-
# Regardless of the contents, the description of test.rbs has priority
|
315
|
-
end
|
316
|
-
|
317
|
-
# Overload is respected correctly
|
318
|
-
p foo(42) #=> Integer
|
319
|
-
p foo("str") #=> String
|
320
|
-
```
|
321
|
-
|
322
|
-
Many of the built-in class methods are also specified by RBS.
|
323
|
-
We plan a feature to load all RBS files of libraries required in Gemfile (but not implemented yet).
|
324
|
-
|
325
|
-
RBS's "interface" type is not supported and is treated as `untyped`.
|
326
|
-
|
327
|
-
### Debug feature
|
328
|
-
|
329
|
-
Unfortunately, understanding the behavior and analysis results of TypeProf is sometimes difficult.
|
330
|
-
|
331
|
-
Currently, you can observe the abstract value of the argument by calling `Kernel#p` in your code, as if you debug your program in Ruby.
|
332
|
-
The only way to get a deeper understanding of the analysis is to watch the debug output with the environment variable `TP_DEBUG=1`.
|
333
|
-
We plan to provide some more useful way to make it easy to understand the analysis result in the future.
|
334
|
-
|
335
|
-
|
336
|
-
### Flow-sensitive analysis
|
337
|
-
|
338
|
-
TypeProf attempts to separate branches if the condition separates a union abstract value.
|
339
|
-
For example, consider that a local variable `var` has an abstract value `Foo | Bar`, and that a branch condition is `var.is_a?(Foo)`.
|
340
|
-
TypeProf will execute the "then" clause with `var` as only a `Foo`, and does the "else" clause with `var` as only a `Bar`.
|
341
|
-
|
342
|
-
Note that it can work well only if the receiver is a local variable defined in the current scope.
|
343
|
-
If the condition is about an instance variable, say `@var.is_a?(Foo)`, or if the variable `var` is defined outside the block, the union is not separated.
|
344
|
-
At present, only the following simple patterns (`is_a?`, `respond_to?`, and `case`/`when`) can be handled well.
|
345
|
-
|
346
|
-
```
|
347
|
-
def foo(x)
|
348
|
-
if x.is_a?(Integer)
|
349
|
-
p x #=> Integer
|
350
|
-
else
|
351
|
-
p x #=> String
|
352
|
-
end
|
353
|
-
end
|
354
|
-
|
355
|
-
foo(42)
|
356
|
-
foo("str")
|
357
|
-
```
|
358
|
-
|
359
|
-
```
|
360
|
-
def foo(x)
|
361
|
-
if x.respond_to?(:times)
|
362
|
-
p x #=> Integer
|
363
|
-
else
|
364
|
-
p x #=> String
|
365
|
-
end
|
366
|
-
end
|
367
|
-
|
368
|
-
foo(42)
|
369
|
-
foo("str")
|
370
|
-
```
|
371
|
-
|
372
|
-
```
|
373
|
-
def foo(x)
|
374
|
-
case x
|
375
|
-
when Integer
|
376
|
-
p x #=> Integer
|
377
|
-
when String
|
378
|
-
p x #=> String
|
379
|
-
end
|
380
|
-
end
|
381
|
-
|
382
|
-
foo(42)
|
383
|
-
foo("str")
|
384
|
-
```
|
385
|
-
|
386
|
-
|
387
|
-
### Container type
|
388
|
-
|
389
|
-
At present, only Array-like containers (Array and Enumerator) and Hash-like containers (Hash) are supported.
|
390
|
-
|
391
|
-
TypeProf keeps the object identity inside a method; the container instances are identified by the place where it is created.
|
392
|
-
You can update the types; this allows the following code to initialize the array:
|
393
|
-
|
394
|
-
```
|
395
|
-
def foo
|
396
|
-
a = []
|
397
|
-
|
398
|
-
100.times {|n| a << n.to_s}
|
399
|
-
|
400
|
-
a
|
401
|
-
end
|
402
|
-
|
403
|
-
p foo #=> Array[String]
|
404
|
-
```
|
405
|
-
|
406
|
-
However, we do not track updates across methods (due to performance reasons).
|
407
|
-
|
408
|
-
```
|
409
|
-
def bar(a)
|
410
|
-
a << "str"
|
411
|
-
end
|
412
|
-
|
413
|
-
def foo
|
414
|
-
a = []
|
415
|
-
|
416
|
-
bar(a)
|
417
|
-
|
418
|
-
a
|
419
|
-
end
|
420
|
-
|
421
|
-
foo #=> [], not Array[String]
|
422
|
-
```
|
423
|
-
|
424
|
-
When a container abstract value is read from an instance variable, an update operation against it will be respected to the instance variable.
|
425
|
-
|
426
|
-
Currently, TypeProf has some limitations about container instances (because of performance).
|
427
|
-
|
428
|
-
* If you put a container type into a key of hash object, the key is replaced with `untyped`.
|
429
|
-
* The maximam depth of nested arrays and hashs is limited to 5.
|
430
|
-
|
431
|
-
We plan to allow them to be configurable, and relax the depth limitation when RBS is manually speficied (mainly for JSON data).
|
432
|
-
|
433
|
-
|
434
|
-
### (Write later)
|
435
|
-
|
436
|
-
* Proc
|
437
|
-
* Struct
|
data/doc/ide.md
DELETED
@@ -1,81 +0,0 @@
|
|
1
|
-
# How to use TypeProf for IDE
|
2
|
-
|
3
|
-
First, try it with an already-configured repository!
|
4
|
-
|
5
|
-
1. `rbenv install 3.1.0-dev`
|
6
|
-
2. `git clone https://github.com/mame/rbswiki`
|
7
|
-
3. `cd rbswiki && bundle install`
|
8
|
-
4. `rbs collection install`
|
9
|
-
5. install [VSCode extension for TypeProf](https://marketplace.visualstudio.com/items?itemName=mame.ruby-typeprof) to your VSCode
|
10
|
-
6. open the `rbswiki` folder by VSCode
|
11
|
-
7. open `lib/rbswiki/wiki.rb`
|
12
|
-
|
13
|
-
If everything goes well, you will see guessed signatures for each method:
|
14
|
-
|
15
|
-
![Screenshot](typeprof-for-ide.png)
|
16
|
-
|
17
|
-
### Troubleshooting
|
18
|
-
|
19
|
-
* Make sure that you are using ruby 3.1.0-dev.
|
20
|
-
|
21
|
-
```
|
22
|
-
$ ruby -v
|
23
|
-
ruby 3.1.0dev (2021-09-28T11:03:54Z master 545e01645f) [x86_64-linux]
|
24
|
-
```
|
25
|
-
|
26
|
-
* Check if typeprof version is 0.20.0 or later.
|
27
|
-
|
28
|
-
```
|
29
|
-
$ bundle exec typeprof --version
|
30
|
-
typeprof 0.20.0
|
31
|
-
```
|
32
|
-
|
33
|
-
* Check if TypeProf can analyze `lib/rbswiki/wiki.rb` within one second.
|
34
|
-
|
35
|
-
```
|
36
|
-
$ bundle exec typeprof lib/rbswiki/wiki.rb
|
37
|
-
warning: rbs collection is experimental, and the behavior may change until RBS v2.0
|
38
|
-
# TypeProf 0.20.0
|
39
|
-
|
40
|
-
# Classes
|
41
|
-
module RBSWiki
|
42
|
-
class Wiki
|
43
|
-
@content: Hash[String, String]
|
44
|
-
...
|
45
|
-
```
|
46
|
-
|
47
|
-
* See the log of "Ruby TypeProf" in vscode's "OUTPUT" panel.
|
48
|
-
|
49
|
-
![Log of TypeProf for IDE](typeprof-for-ide-log.png)
|
50
|
-
|
51
|
-
## How to configure TypeProf for your code
|
52
|
-
|
53
|
-
1. Write `gem "typeprof"` to your `Gemfile`, and run `bundle install`
|
54
|
-
2. Write `rbs_collection.yaml`, and run `rbs collection install`
|
55
|
-
3. Test if TypeProf works well by running `bundle exec typeprof your_code.rb`
|
56
|
-
4. Open your repository with vscode
|
57
|
-
|
58
|
-
### Troubleshooting
|
59
|
-
|
60
|
-
*TBD*
|
61
|
-
|
62
|
-
## Protips, limitation, unimplemented features, ...
|
63
|
-
|
64
|
-
TypeProf for IDE is extremely preliminary! Please give it a try with a warm heart...
|
65
|
-
|
66
|
-
* Write a simple (type-level) test in each file with `if $0 == __FILE__` guard to guide TypeProf to infer method signatures.
|
67
|
-
* Use `require` only for loading gems. To load your code, use `require_relative` instead of `require`. (Or, create `bin/typeprof` and pass `-Ilib`. TBD for details)
|
68
|
-
* Currently, TypeProf for IDE loads `typeprof.rbs` at the top folder. (I'll need to improve it to read `sig/` directory)
|
69
|
-
* TypeProf for IDE analyzes each file within one second. Unfortunately, it takes very long to analyze big code, or complex code, or code that requires a big gem. In this case, TypeProf for IDE cannot show any guesses. Splitting a file or manually writing RBS may make the analysis faster. (TBD for details)
|
70
|
-
* Unfortunately, TypeProf may report some false positive errors that you cannot stop. I'll create an option to configure error level.
|
71
|
-
* TypeProf for IDE tries to invoke TypeProf as follows:
|
72
|
-
* If your repository has `bin/typeprof` executable, it is invoked.
|
73
|
-
* Otherwise, it will try to invoke a command specified in VS`typeprof.server.path`.
|
74
|
-
* Otherwise, it will try to invoke `bundle exec typeprof`.
|
75
|
-
* Otherwise, it will try to invoke `typeprof`.
|
76
|
-
* Otherwise, TypeProf for IDE gives up.
|
77
|
-
* Some people says TypeProf for IDE works with vim! (TBD for details)
|
78
|
-
|
79
|
-
## How to develop TypeProf for IDE
|
80
|
-
|
81
|
-
See `../vscode/development.md`.
|
data/doc/ppl2019.pdf
DELETED
Binary file
|
data/doc/todo.md
DELETED
@@ -1,133 +0,0 @@
|
|
1
|
-
# TypeProf milestones and TODOs
|
2
|
-
|
3
|
-
## Big milestones
|
4
|
-
|
5
|
-
### Rails support
|
6
|
-
|
7
|
-
There are many known issues for the analysis of typical Ruby programs including Rails apps.
|
8
|
-
|
9
|
-
* The main difficulty is that they use many language extensions like `ActiveSupport`.
|
10
|
-
Some features (for example, `blank?` and `Time.now + 1.day`) are trivial to support,
|
11
|
-
but others (for example, `ActiveSupport::Concern` and Zeitwerk) will require special support.
|
12
|
-
* The other difficulty is that they heavily use meta-programming features like `ActiveRecord`.
|
13
|
-
It dynamically defines some methods based on external data (such as DB schema) from the code.
|
14
|
-
|
15
|
-
Currently, projects called [`gem_rbs`](https://github.com/ruby/gem_rbs) and [`rbs_rails`](https://github.com/pocke/rbs_rails) are in progress.
|
16
|
-
The former provides several RBS files for some major gems including Rails.
|
17
|
-
The latter is a tool to generate RBS prototype of a target Rails application by introspection (executing it and monitoring DB schema, etc).
|
18
|
-
TypeProf can use their results to improve analysis precision and performance.
|
19
|
-
|
20
|
-
What we need to do:
|
21
|
-
|
22
|
-
* Experimentally apply TypeProf to some Rails programs and identify problems
|
23
|
-
* Make TypeProf able to work together with `rbs_rails` for supporting trivial core extensions and `ActiveRecord`.
|
24
|
-
* Implement special support for some fundamental language extensions of Rails like `ActiveSupport::Concern`.
|
25
|
-
(It would be best if TypeProf has a plugin system and if we can factor out the special support as a plugin for Rails.)
|
26
|
-
|
27
|
-
### Error detection and diagnosis feature
|
28
|
-
|
29
|
-
At present, TypeProf focuses on generation of RBS prototype from no-type-annotated Ruby code.
|
30
|
-
However, it is possible for TypeProf to report possible errors found during the analysis.
|
31
|
-
In fact, an option `-v` experimentally shows possible errors found.
|
32
|
-
There are some reasons why it is disabled by default:
|
33
|
-
|
34
|
-
* (1) There are too many false positives.
|
35
|
-
* (2) Some kind of error reporting is not implemented yet.
|
36
|
-
* (3) Some reported errors are difficult for a user to understand.
|
37
|
-
|
38
|
-
For (1), we will research how we can avoid false positives to support typical Ruby coding patterns as much as possible.
|
39
|
-
The primary way is to improve the analysis precision, e.g., enhancing flow-sensitive analysis.
|
40
|
-
If the S/N ratio of an error type is too low, we need to consider to suppress the kind of reports.
|
41
|
-
Also, we may try allowing users to guide TypeProf to analyze their program well.
|
42
|
-
(The simplest way is to write inline type casts in the code, but we need to find more Ruby/RBS way.)
|
43
|
-
We may also explore a "TypeProf-friendly coding style" which TypeProf can analyze well.
|
44
|
-
(In principle, the plainer code is, the better TypeProf can analyze.)
|
45
|
-
|
46
|
-
For (2), currently, TypeProf checks the argument types of a call to a method whose type signature is declared in RBS.
|
47
|
-
However, it does not check the return type yet. Redefinition of constants should be warned too.
|
48
|
-
We will survey what errors and warnings TypeProf can print, and evaluate the S/N ratio of each report.
|
49
|
-
|
50
|
-
For (3), since TypeProf uses whole program analysis, an error may be reported at a very different place from its root bug.
|
51
|
-
Thus, if TypeProf shows a possible type error, a diagnosis feature is needed to answer why TypeProf thinks that the error may occur.
|
52
|
-
TypeProf has already implemented a very primitive diagnosis feature, `Kernel#p`, to check what type an expression has.
|
53
|
-
Another idea is to create a pseudo backtrace why TypeProf thought the possible type error may occur.
|
54
|
-
We should consider this feature with LSP support.
|
55
|
-
|
56
|
-
### Performance improvement
|
57
|
-
|
58
|
-
Currently, TypeProf is painfully slow. Even if a target application is small.
|
59
|
-
|
60
|
-
The main reason is that TypeProf analyzes not only the application code but also library code:
|
61
|
-
if an application requires `"foo"`, TypeProf actually loads `foo.rb` even from a gem,
|
62
|
-
and furthermore, if `foo.rb` requires `"bar"`, it loads `bar.rb` recursively.
|
63
|
-
|
64
|
-
RBS will help to stop this cascade;
|
65
|
-
when an application requires `"foo"`, TypeProf loads `sig/foo.rbs` instead of `foo.rb` if the `foo` gem contains both.
|
66
|
-
Such a RBS file is optional for TypeProf but required for Steep.
|
67
|
-
So, we think many gems will eventually equip their RBS declarations.
|
68
|
-
|
69
|
-
That being said, we should continue to improve the analysis performance of TypeProf. We have some ideas.
|
70
|
-
|
71
|
-
* Unfortunately, TypeProf often analyzes one method more than once when it accepts multiple types.
|
72
|
-
As TypeProf squashes the argument types to a union, this duplicated analysis is not necessarily needed.
|
73
|
-
But when TypeProf first analyzes a method, it is difficult to determine if the method will accept another type in further analysis.
|
74
|
-
So, we need good heuristics to guess whether a method accepts multiple types or not, and if so, delay its analysis.
|
75
|
-
* Currently, TypeProf executes the bytecode instructions step by step.
|
76
|
-
This requires creating an environment object after each instruction, which is very heavy.
|
77
|
-
Many environment creations can be omitted by executing each basic block instead of each instruction.
|
78
|
-
(Basic block execution will also make flow-sensitive analysis easier.)
|
79
|
-
* The slowest calculation in TypeProf is to create an instance of a Type class.
|
80
|
-
The creation uses memoization; TypeProf keeps all Type instances created so far, and reuses them if already exist.
|
81
|
-
However, it is very heavy to check if an instance already exists or not.
|
82
|
-
(Currently, it is very simply implemented by a big Hash table.)
|
83
|
-
We've already improved the memoization routine several times but looks like it is still the No.1 bottleneck.
|
84
|
-
We need to investigate and try improving more.
|
85
|
-
* TypeProf heavily uses Hash objects (including above) mainly to represent a set.
|
86
|
-
A union of sets is done by `Hash#merge`, which takes O(n).
|
87
|
-
A more lightweight data structure may make TypeProf faster.
|
88
|
-
(But clever structure often has a big constant term, so we need to evaluate the performance carefully.)
|
89
|
-
* Reusing an old analysis and incrementally updating it will bring a super big improvement.
|
90
|
-
This would be especially helpful for LSP support, so we need to tackle it after the analysis approach is mature.
|
91
|
-
|
92
|
-
### Language Server Protocol (LSP) support
|
93
|
-
|
94
|
-
In the future, we want TypeProf to serve as a language server to show the result in IDE in real-time.
|
95
|
-
However, the current analysis approach is too slow for IDE. So we need to improve the performance first.
|
96
|
-
|
97
|
-
Even if TypeProf becomes fast enough, its approach has a fundamental problem.
|
98
|
-
Since TypeProf uses whole program analysis, one edit may cause a cascade of propagation:
|
99
|
-
if a user write `foo(42)`, an Integer is propagated to a method `foo`,
|
100
|
-
and if `foo` passes its argument to a method `bar`, it is propagated to `bar`, ...
|
101
|
-
So, a breakthrough for LSP may be still needed, e.g, limiting the propagation range in real-time analysis,
|
102
|
-
assuming that a type interface of module boundary is fixed, etc.
|
103
|
-
|
104
|
-
## Relatively smaller TODOs
|
105
|
-
|
106
|
-
* Support more RBS features
|
107
|
-
* TypeProf does not deal with some RBS types well yet.
|
108
|
-
* For example, the `instance` type is handled as `untyped.
|
109
|
-
* The `self` type is handled well only when it is used as a return type.
|
110
|
-
* Using a value of the `void` type should be warned appropriately.
|
111
|
-
* RBS's `interface` is supported just like a module (i.e., `include _Foo` is explicitly required in RBS),
|
112
|
-
but it should be checked structually (i.e., it should be determined as a method set.)
|
113
|
-
* The variance of type parameters is currently ignored.
|
114
|
-
|
115
|
-
* Support more Ruby features
|
116
|
-
* Some meta-programming features like `Class.new`, `Object#method`, etc.
|
117
|
-
* It is possible to support `Class.new` by per-allocation-site approach:
|
118
|
-
e.g., In TypeProf, `A = Class.new; B = Class.new` will create two classes, but `2.times { Class.new }` will create one class.
|
119
|
-
* The analysis precision can be improved more for some Ruby features like pattern matching, keyword arguments, etc.
|
120
|
-
* For example, `foo(*args, k:1)` is currently compiled as if it is `foo(*(args + [{ :k => 1 }]))` into Ruby bytecode.
|
121
|
-
This mixes the keyword arguments to a rest array, and makes it difficult for TypeProf to track the keyword arguments.
|
122
|
-
* Support Enumerator as an Array-type container.
|
123
|
-
* Support `Module#protect` (but RBS does not yet).
|
124
|
-
* More heuristics may help such as `==` returns a bool regardless to its receiver and argument types.
|
125
|
-
|
126
|
-
* Make TypeProf more useful as a tool
|
127
|
-
* Currently, TypeProf provides only the analysis engine and a minimal set of features.
|
128
|
-
* The analysis result would be useful not only to generate RBS prototype
|
129
|
-
but also identifying the source location of a method definition, listing callsites of a method,
|
130
|
-
searching a method call by its argument types, etc.
|
131
|
-
* Sometimes, TypeProf prints very big union type, such as `Integer | Float | Complex | Rational | ...`.
|
132
|
-
Worse, the same big type is printed multiple times.
|
133
|
-
It may be useful to factor out such a long type by using type alias, for example.
|
Binary file
|
data/doc/typeprof-for-ide.png
DELETED
Binary file
|