typeprof 0.10.0 → 0.11.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +5 -5
- data/doc/demo.md +1 -1
- data/doc/todo.md +133 -0
- data/lib/typeprof/cli.rb +3 -0
- data/lib/typeprof/config.rb +2 -0
- data/lib/typeprof/import.rb +7 -3
- data/lib/typeprof/version.rb +1 -1
- data/typeprof.gemspec +1 -1
- metadata +6 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 88521dfb48f311cdb5b651e88afc2627a28040a466a22b50303415531e663cfd
|
4
|
+
data.tar.gz: 4bd17f9d016d6e7a60ed72804eb050a5611af0869275e63d3486700a53befaaf
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f88b069345f02729a09c0516baf4ff31c8aef640adf5a2c53f62f72a9fb8b96b4309993ae79e37ece4ec33972f762005b3898ca7c894cda65c3727513b812b6b
|
7
|
+
data.tar.gz: aa5fec1acd9dd6395d3d16e044f08e7b72fac0f41dc6168683c631fcadca33daf2415b9956ba00cd22c573c6732719fad9eeef3d601a16943d59be6986703c20
|
data/Gemfile.lock
CHANGED
@@ -1,17 +1,17 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
typeprof (0.
|
5
|
-
rbs (>= 0.
|
4
|
+
typeprof (0.11.0)
|
5
|
+
rbs (>= 1.0.0)
|
6
6
|
|
7
7
|
GEM
|
8
8
|
remote: https://rubygems.org/
|
9
9
|
specs:
|
10
10
|
coverage-helpers (1.0.0)
|
11
|
-
docile (1.3.
|
11
|
+
docile (1.3.4)
|
12
12
|
power_assert (1.2.0)
|
13
13
|
rake (13.0.1)
|
14
|
-
rbs (0.
|
14
|
+
rbs (1.0.0)
|
15
15
|
simplecov (0.20.0)
|
16
16
|
docile (~> 1.1)
|
17
17
|
simplecov-html (~> 0.11)
|
@@ -36,4 +36,4 @@ DEPENDENCIES
|
|
36
36
|
typeprof!
|
37
37
|
|
38
38
|
BUNDLED WITH
|
39
|
-
2.2.
|
39
|
+
2.2.3
|
data/doc/demo.md
CHANGED
@@ -39,7 +39,7 @@ class Object
|
|
39
39
|
end
|
40
40
|
```
|
41
41
|
|
42
|
-
|
42
|
+
You can [try this analysis online](https://mame.github.io/typeprof-playground/#rb=def+hello_message%28user%29%0A++%22The+name+is+%22+%2B+user.name%0Aend%0A%0Adef+type_error_demo%28user%29%0A++%22The+age+is+%22+%2B+user.age%0Aend%0A%0Auser+%3D+User.new%28name%3A+%22John%22%2C+age%3A+20%29%0A%0Ahello_message%28user%29%0Atype_error_demo%28user%29&rbs=class+User%0A++attr_reader+name%3A+String%0A++attr_reader+age%3A+Integer%0A%0A++def+initialize%3A+%28name%3A+String%2C+age%3A+Integer%29+-%3E+void%0Aend).
|
43
43
|
|
44
44
|
## A simple demo to generate the signature prototype of "User" class
|
45
45
|
|
data/doc/todo.md
ADDED
@@ -0,0 +1,133 @@
|
|
1
|
+
# TypeProf milestones and TODOs
|
2
|
+
|
3
|
+
## Big milestones
|
4
|
+
|
5
|
+
### Rails support
|
6
|
+
|
7
|
+
There are many known issues for the analysis of typical Ruby programs including Rails apps.
|
8
|
+
|
9
|
+
* The main difficulty is that they use many language extensions like `ActiveSupport`.
|
10
|
+
Some features (for example, `blank?` and `Time.now + 1.day`) are trivial to support,
|
11
|
+
but others (for example, `ActiveSupport::Concern` and Zeitwerk) will require special support.
|
12
|
+
* The other difficulty is that they heavily use meta-programming features like `ActiveRecord`.
|
13
|
+
It dynamically defines some methods based on external data (such as DB schema) from the code.
|
14
|
+
|
15
|
+
Currently, projects called [`gem_rbs`](https://github.com/ruby/gem_rbs) and [`rbs_rails`](https://github.com/pocke/rbs_rails) are in progress.
|
16
|
+
The former provides several RBS files for some major gems including Rails.
|
17
|
+
The latter is a tool to generate RBS prototype of a target Rails application by introspection (executing it and monitoring DB schema, etc).
|
18
|
+
TypeProf can use their results to improve analysis precision and performance.
|
19
|
+
|
20
|
+
What we need to do:
|
21
|
+
|
22
|
+
* Experimentally apply TypeProf to some Rails programs and identify problems
|
23
|
+
* Make TypeProf able to work together with `rbs_rails` for supporting trivial core extensions and `ActiveRecord`.
|
24
|
+
* Implement special support for some fundamental language extensions of Rails like `ActiveSupport::Concern`.
|
25
|
+
(It would be best if TypeProf has a plugin system and if we can factor out the special support as a plugin for Rails.)
|
26
|
+
|
27
|
+
### Error detection and diagnosis feature
|
28
|
+
|
29
|
+
At present, TypeProf focuses on generation of RBS prototype from no-type-annotated Ruby code.
|
30
|
+
However, it is possible for TypeProf to report possible errors found during the analysis.
|
31
|
+
In fact, an option `-v` experimentally shows possible errors found.
|
32
|
+
There are some reasons why it is disabled by default:
|
33
|
+
|
34
|
+
* (1) There are too many false positives.
|
35
|
+
* (2) Some kind of error reporting is not implemented yet.
|
36
|
+
* (3) Some reported errors are difficult for a user to understand.
|
37
|
+
|
38
|
+
For (1), we will research how we can avoid false positives to support typical Ruby coding patterns as much as possible.
|
39
|
+
The primary way is to improve the analysis precision, e.g., enhancing flow-sensitive analysis.
|
40
|
+
If the S/N ratio of an error type is too low, we need to consider to suppress the kind of reports.
|
41
|
+
Also, we may try allowing users to guide TypeProf to analyze their program well.
|
42
|
+
(The simplest way is to write inline type casts in the code, but we need to find more Ruby/RBS way.)
|
43
|
+
We may also explore a "TypeProf-friendly coding style" which TypeProf can analyze well.
|
44
|
+
(In principle, the plainer code is, the better TypeProf can analyze.)
|
45
|
+
|
46
|
+
For (2), currently, TypeProf checks the argument types of a call to a method whose type signature is declared in RBS.
|
47
|
+
However, it does not check the return type yet. Redefinition of constants should be warned too.
|
48
|
+
We will survey what errors and warnings TypeProf can print, and evaluate the S/N ratio of each report.
|
49
|
+
|
50
|
+
For (3), since TypeProf uses whole program analysis, an error may be reported at a very different place from its root bug.
|
51
|
+
Thus, if TypeProf shows a possible type error, a diagnosis feature is needed to answer why TypeProf thinks that the error may occur.
|
52
|
+
TypeProf has already implemented a very primitive diagnosis feature, `Kernel#p`, to check what type an expression has.
|
53
|
+
Another idea is to create a pseudo backtrace why TypeProf thought the possible type error may occur.
|
54
|
+
We should consider this feature with LSP support.
|
55
|
+
|
56
|
+
### Performance improvement
|
57
|
+
|
58
|
+
Currently, TypeProf is painfully slow. Even if a target application is small.
|
59
|
+
|
60
|
+
The main reason is that TypeProf analyzes not only the application code but also library code:
|
61
|
+
if an application requires `"foo"`, TypeProf actually loads `foo.rb` even from a gem,
|
62
|
+
and furthermore, if `foo.rb` requires `"bar"`, it loads `bar.rb` recursively.
|
63
|
+
|
64
|
+
RBS will help to stop this cascade;
|
65
|
+
when an application requires `"foo"`, TypeProf loads `sig/foo.rbs` instead of `foo.rb` if the `foo` gem contains both.
|
66
|
+
Such a RBS file is optional for TypeProf but required for Steep.
|
67
|
+
So, we think many gems will eventually equip their RBS declarations.
|
68
|
+
|
69
|
+
That being said, we should continue to improve the analysis performance of TypeProf. We have some ideas.
|
70
|
+
|
71
|
+
* Unfortunately, TypeProf often analyzes one method more than once when it accepts multiple types.
|
72
|
+
As TypeProf squashes the argument types to a union, this duplicated analysis is not necessarily needed.
|
73
|
+
But when TypeProf first analyzes a method, it is difficult to determine if the method will accept another type in further analysis.
|
74
|
+
So, we need good heuristics to guess whether a method accepts multiple types or not, and if so, delay its analysis.
|
75
|
+
* Currently, TypeProf executes the bytecode instructions step by step.
|
76
|
+
This requires creating an environment object after each instruction, which is very heavy.
|
77
|
+
Many environment creations can be omitted by executing each basic block instead of each instruction.
|
78
|
+
(Basic block execution will also make flow-sensitive analysis easier.)
|
79
|
+
* The slowest calculation in TypeProf is to create an instance of a Type class.
|
80
|
+
The creation uses memoization; TypeProf keeps all Type instances created so far, and reuses them if already exist.
|
81
|
+
However, it is very heavy to check if an instance already exists or not.
|
82
|
+
(Currently, it is very simply implemented by a big Hash table.)
|
83
|
+
We've already improved the memoization routine several times but looks like it is still the No.1 bottleneck.
|
84
|
+
We need to investigate and try improving more.
|
85
|
+
* TypeProf heavily uses Hash objects (including above) mainly to represent a set.
|
86
|
+
A union of sets is done by `Hash#merge`, which takes O(n).
|
87
|
+
A more lightweight data structure may make TypeProf faster.
|
88
|
+
(But clever structure often has a big constant term, so we need to evaluate the performance carefully.)
|
89
|
+
* Reusing an old analysis and incrementally updating it will bring a super big improvement.
|
90
|
+
This would be especially helpful for LSP support, so we need to tackle it after the analysis approach is mature.
|
91
|
+
|
92
|
+
### Language Server Protocol (LSP) support
|
93
|
+
|
94
|
+
In the future, we want TypeProf to serve as a language server to show the result in IDE in real-time.
|
95
|
+
However, the current analysis approach is too slow for IDE. So we need to improve the performance first.
|
96
|
+
|
97
|
+
Even if TypeProf becomes fast enough, its approach has a fundamental problem.
|
98
|
+
Since TypeProf uses whole program analysis, one edit may cause a cascade of propagation:
|
99
|
+
if a user write `foo(42)`, an Integer is propagated to a method `foo`,
|
100
|
+
and if `foo` passes its argument to a method `bar`, it is propagated to `bar`, ...
|
101
|
+
So, a breakthrough for LSP may be still needed, e.g, limiting the propagation range in real-time analysis,
|
102
|
+
assuming that a type interface of module boundary is fixed, etc.
|
103
|
+
|
104
|
+
## Relatively smaller TODOs
|
105
|
+
|
106
|
+
* Support more RBS features
|
107
|
+
* TypeProf does not deal with some RBS types well yet.
|
108
|
+
* For example, the `instance` type is handled as `untyped.
|
109
|
+
* The `self` type is handled well only when it is used as a return type.
|
110
|
+
* Using a value of the `void` type should be warned appropriately.
|
111
|
+
* RBS's `interface` is supported just like a module (i.e., `include _Foo` is explicitly required in RBS),
|
112
|
+
but it should be checked structually (i.e., it should be determined as a method set.)
|
113
|
+
* The variance of type parameters is currently ignored.
|
114
|
+
|
115
|
+
* Support more Ruby features
|
116
|
+
* Some meta-programming features like `Class.new`, `Object#method`, etc.
|
117
|
+
* It is possible to support `Class.new` by per-allocation-site approach:
|
118
|
+
e.g., In TypeProf, `A = Class.new; B = Class.new` will create two classes, but `2.times { Class.new }` will create one class.
|
119
|
+
* The analysis precision can be improved more for some Ruby features like pattern matching, keyword arguments, etc.
|
120
|
+
* For example, `foo(*args, k:1)` is currently compiled as if it is `foo(*(args + [{ :k => 1 }]))` into Ruby bytecode.
|
121
|
+
This mixes the keyword arguments to a rest array, and makes it difficult for TypeProf to track the keyword arguments.
|
122
|
+
* Support Enumerator as an Array-type container.
|
123
|
+
* Support `Module#protect` (but RBS does not yet).
|
124
|
+
* More heuristics may help such as `==` returns a bool regardless to its receiver and argument types.
|
125
|
+
|
126
|
+
* Make TypeProf more useful as a tool
|
127
|
+
* Currently, TypeProf provides only the analysis engine and a minimal set of features.
|
128
|
+
* The analysis result would be useful not only to generate RBS prototype
|
129
|
+
but also identifying the source location of a method definition, listing callsites of a method,
|
130
|
+
searching a method call by its argument types, etc.
|
131
|
+
* Sometimes, TypeProf prints very big union type, such as `Integer | Float | Complex | Rational | ...`.
|
132
|
+
Worse, the same big type is printed multiple times.
|
133
|
+
It may be useful to factor out such a long type by using type alias, for example.
|
data/lib/typeprof/cli.rb
CHANGED
@@ -20,6 +20,7 @@ module TypeProf
|
|
20
20
|
options = {}
|
21
21
|
dir_filter = nil
|
22
22
|
gem_rbs_features = []
|
23
|
+
gem_repo_dirs = []
|
23
24
|
show_version = false
|
24
25
|
max_sec = max_iter = nil
|
25
26
|
|
@@ -31,6 +32,7 @@ module TypeProf
|
|
31
32
|
opt.on("--version", "Display typeprof version") { show_version = true }
|
32
33
|
opt.on("-I DIR", "Add DIR to the load/require path") {|v| $LOAD_PATH << v }
|
33
34
|
opt.on("-r FEATURE", "Require RBS of the FEATURE gem") {|v| gem_rbs_features << v }
|
35
|
+
opt.on("--repo DIR", "Add DIR to the RBS repository") {|v| gem_repo_dirs << v }
|
34
36
|
|
35
37
|
opt.separator ""
|
36
38
|
opt.separator "Analysis output options:"
|
@@ -87,6 +89,7 @@ module TypeProf
|
|
87
89
|
rbs_files: rbs_files,
|
88
90
|
output: output,
|
89
91
|
gem_rbs_features: gem_rbs_features,
|
92
|
+
gem_repo_dirs: gem_repo_dirs,
|
90
93
|
verbose: verbose,
|
91
94
|
dir_filter: dir_filter,
|
92
95
|
max_sec: max_sec,
|
data/lib/typeprof/config.rb
CHANGED
@@ -6,6 +6,7 @@ module TypeProf
|
|
6
6
|
:rbs_files,
|
7
7
|
:output,
|
8
8
|
:gem_rbs_features,
|
9
|
+
:gem_repo_dirs,
|
9
10
|
:verbose,
|
10
11
|
:dir_filter,
|
11
12
|
:max_iter,
|
@@ -25,6 +26,7 @@ module TypeProf
|
|
25
26
|
def initialize(**opt)
|
26
27
|
opt[:output] ||= $stdout
|
27
28
|
opt[:gem_rbs_features] ||= []
|
29
|
+
opt[:gem_repo_dirs] ||= []
|
28
30
|
opt[:dir_filter] ||= DEFAULT_DIR_FILTER
|
29
31
|
opt[:verbose] ||= 0
|
30
32
|
opt[:options] ||= {}
|
data/lib/typeprof/import.rb
CHANGED
@@ -3,6 +3,10 @@ require "rbs"
|
|
3
3
|
module TypeProf
|
4
4
|
class RBSReader
|
5
5
|
def initialize
|
6
|
+
@repo = RBS::Repository.new
|
7
|
+
Config.gem_repo_dirs.each do |dir|
|
8
|
+
@repo.add(Pathname(dir))
|
9
|
+
end
|
6
10
|
@env, @builtin_env_json = RBSReader.get_builtin_env
|
7
11
|
end
|
8
12
|
|
@@ -11,7 +15,7 @@ module TypeProf
|
|
11
15
|
unless @builtin_env
|
12
16
|
@builtin_env = RBS::Environment.new
|
13
17
|
|
14
|
-
loader = RBS::EnvironmentLoader.new
|
18
|
+
loader = RBS::EnvironmentLoader.new(repository: @repo)
|
15
19
|
new_decls = loader.load(env: @builtin_env).map {|decl,| decl }
|
16
20
|
@builtin_env_json = load_rbs(@builtin_env, new_decls)
|
17
21
|
end
|
@@ -24,7 +28,7 @@ module TypeProf
|
|
24
28
|
end
|
25
29
|
|
26
30
|
def load_library(lib)
|
27
|
-
loader = RBS::EnvironmentLoader.new(core_root: nil)
|
31
|
+
loader = RBS::EnvironmentLoader.new(core_root: nil, repository: @repo)
|
28
32
|
loader.add(library: lib)
|
29
33
|
|
30
34
|
case lib
|
@@ -38,7 +42,7 @@ module TypeProf
|
|
38
42
|
end
|
39
43
|
|
40
44
|
def load_path(path)
|
41
|
-
loader = RBS::EnvironmentLoader.new(core_root: nil)
|
45
|
+
loader = RBS::EnvironmentLoader.new(core_root: nil, repository: @repo)
|
42
46
|
loader.add(path: path)
|
43
47
|
new_decls = loader.load(env: @env).map {|decl,| decl }
|
44
48
|
RBSReader.load_rbs(@env, new_decls)
|
data/lib/typeprof/version.rb
CHANGED
data/typeprof.gemspec
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: typeprof
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.11.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Yusuke Endoh
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-12-
|
11
|
+
date: 2020-12-24 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rbs
|
@@ -16,14 +16,14 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: 0.
|
19
|
+
version: 1.0.0
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: 0.
|
26
|
+
version: 1.0.0
|
27
27
|
description: |
|
28
28
|
TypeProf performs a type analysis of non-annotated Ruby code.
|
29
29
|
|
@@ -48,6 +48,7 @@ files:
|
|
48
48
|
- doc/doc.ja.md
|
49
49
|
- doc/doc.md
|
50
50
|
- doc/ppl2019.pdf
|
51
|
+
- doc/todo.md
|
51
52
|
- exe/typeprof
|
52
53
|
- lib/typeprof.rb
|
53
54
|
- lib/typeprof/analyzer.rb
|
@@ -402,7 +403,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
402
403
|
- !ruby/object:Gem::Version
|
403
404
|
version: '0'
|
404
405
|
requirements: []
|
405
|
-
rubygems_version: 3.
|
406
|
+
rubygems_version: 3.2.3
|
406
407
|
signing_key:
|
407
408
|
specification_version: 4
|
408
409
|
summary: TypeProf is a type analysis tool for Ruby code based on abstract interpretation
|