typeprof 0.10.0 → 0.11.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (10) hide show
  1. checksums.yaml +4 -4
  2. data/Gemfile.lock +5 -5
  3. data/doc/demo.md +1 -1
  4. data/doc/todo.md +133 -0
  5. data/lib/typeprof/cli.rb +3 -0
  6. data/lib/typeprof/config.rb +2 -0
  7. data/lib/typeprof/import.rb +7 -3
  8. data/lib/typeprof/version.rb +1 -1
  9. data/typeprof.gemspec +1 -1
  10. metadata +6 -5
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8d68099e4f5d60a45f949d6bfc3a47e8b7d3b7ba66a0794f25f5fe61628de1d1
4
- data.tar.gz: 22792d1acb9b16a4150f1afbaf29d59987bf29772908b23075e6668df81a4309
3
+ metadata.gz: 88521dfb48f311cdb5b651e88afc2627a28040a466a22b50303415531e663cfd
4
+ data.tar.gz: 4bd17f9d016d6e7a60ed72804eb050a5611af0869275e63d3486700a53befaaf
5
5
  SHA512:
6
- metadata.gz: 42faa4e3b5c64fd5976e66067bf13339c530785914caad77bc0a405939bc914e9adf78bfbd0aea57a829cb21aed3c6dfc09f8f57def85b09312facd82d33b5a9
7
- data.tar.gz: bc6f115fbf1d5292286f04f09d745cfb0c13688a71fdae730d0cfdd903ad8f9f3f9019e532c37b4cf23fe41f39c9a1683f92f28af68716ddf7e625a0efca33f8
6
+ metadata.gz: f88b069345f02729a09c0516baf4ff31c8aef640adf5a2c53f62f72a9fb8b96b4309993ae79e37ece4ec33972f762005b3898ca7c894cda65c3727513b812b6b
7
+ data.tar.gz: aa5fec1acd9dd6395d3d16e044f08e7b72fac0f41dc6168683c631fcadca33daf2415b9956ba00cd22c573c6732719fad9eeef3d601a16943d59be6986703c20
data/Gemfile.lock CHANGED
@@ -1,17 +1,17 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- typeprof (0.10.0)
5
- rbs (>= 0.20.1)
4
+ typeprof (0.11.0)
5
+ rbs (>= 1.0.0)
6
6
 
7
7
  GEM
8
8
  remote: https://rubygems.org/
9
9
  specs:
10
10
  coverage-helpers (1.0.0)
11
- docile (1.3.2)
11
+ docile (1.3.4)
12
12
  power_assert (1.2.0)
13
13
  rake (13.0.1)
14
- rbs (0.20.1)
14
+ rbs (1.0.0)
15
15
  simplecov (0.20.0)
16
16
  docile (~> 1.1)
17
17
  simplecov-html (~> 0.11)
@@ -36,4 +36,4 @@ DEPENDENCIES
36
36
  typeprof!
37
37
 
38
38
  BUNDLED WITH
39
- 2.2.1
39
+ 2.2.3
data/doc/demo.md CHANGED
@@ -39,7 +39,7 @@ class Object
39
39
  end
40
40
  ```
41
41
 
42
- Yoy can [try this analysis online](https://mame.github.io/typeprof-playground/#rb=def+hello_message%28user%29%0A++%22The+name+is+%22+%2B+user.name%0Aend%0A%0Adef+type_error_demo%28user%29%0A++%22The+age+is+%22+%2B+user.age%0Aend%0A%0Auser+%3D+User.new%28name%3A+%22John%22%2C+age%3A+20%29%0A%0Ahello_message%28user%29%0Atype_error_demo%28user%29&rbs=class+User%0A++attr_reader+name%3A+String%0A++attr_reader+age%3A+Integer%0A%0A++def+initialize%3A+%28name%3A+String%2C+age%3A+Integer%29+-%3E+void%0Aend).
42
+ You can [try this analysis online](https://mame.github.io/typeprof-playground/#rb=def+hello_message%28user%29%0A++%22The+name+is+%22+%2B+user.name%0Aend%0A%0Adef+type_error_demo%28user%29%0A++%22The+age+is+%22+%2B+user.age%0Aend%0A%0Auser+%3D+User.new%28name%3A+%22John%22%2C+age%3A+20%29%0A%0Ahello_message%28user%29%0Atype_error_demo%28user%29&rbs=class+User%0A++attr_reader+name%3A+String%0A++attr_reader+age%3A+Integer%0A%0A++def+initialize%3A+%28name%3A+String%2C+age%3A+Integer%29+-%3E+void%0Aend).
43
43
 
44
44
  ## A simple demo to generate the signature prototype of "User" class
45
45
 
data/doc/todo.md ADDED
@@ -0,0 +1,133 @@
1
+ # TypeProf milestones and TODOs
2
+
3
+ ## Big milestones
4
+
5
+ ### Rails support
6
+
7
+ There are many known issues for the analysis of typical Ruby programs including Rails apps.
8
+
9
+ * The main difficulty is that they use many language extensions like `ActiveSupport`.
10
+ Some features (for example, `blank?` and `Time.now + 1.day`) are trivial to support,
11
+ but others (for example, `ActiveSupport::Concern` and Zeitwerk) will require special support.
12
+ * The other difficulty is that they heavily use meta-programming features like `ActiveRecord`.
13
+ It dynamically defines some methods based on external data (such as DB schema) from the code.
14
+
15
+ Currently, projects called [`gem_rbs`](https://github.com/ruby/gem_rbs) and [`rbs_rails`](https://github.com/pocke/rbs_rails) are in progress.
16
+ The former provides several RBS files for some major gems including Rails.
17
+ The latter is a tool to generate RBS prototype of a target Rails application by introspection (executing it and monitoring DB schema, etc).
18
+ TypeProf can use their results to improve analysis precision and performance.
19
+
20
+ What we need to do:
21
+
22
+ * Experimentally apply TypeProf to some Rails programs and identify problems
23
+ * Make TypeProf able to work together with `rbs_rails` for supporting trivial core extensions and `ActiveRecord`.
24
+ * Implement special support for some fundamental language extensions of Rails like `ActiveSupport::Concern`.
25
+ (It would be best if TypeProf has a plugin system and if we can factor out the special support as a plugin for Rails.)
26
+
27
+ ### Error detection and diagnosis feature
28
+
29
+ At present, TypeProf focuses on generation of RBS prototype from no-type-annotated Ruby code.
30
+ However, it is possible for TypeProf to report possible errors found during the analysis.
31
+ In fact, an option `-v` experimentally shows possible errors found.
32
+ There are some reasons why it is disabled by default:
33
+
34
+ * (1) There are too many false positives.
35
+ * (2) Some kind of error reporting is not implemented yet.
36
+ * (3) Some reported errors are difficult for a user to understand.
37
+
38
+ For (1), we will research how we can avoid false positives to support typical Ruby coding patterns as much as possible.
39
+ The primary way is to improve the analysis precision, e.g., enhancing flow-sensitive analysis.
40
+ If the S/N ratio of an error type is too low, we need to consider to suppress the kind of reports.
41
+ Also, we may try allowing users to guide TypeProf to analyze their program well.
42
+ (The simplest way is to write inline type casts in the code, but we need to find more Ruby/RBS way.)
43
+ We may also explore a "TypeProf-friendly coding style" which TypeProf can analyze well.
44
+ (In principle, the plainer code is, the better TypeProf can analyze.)
45
+
46
+ For (2), currently, TypeProf checks the argument types of a call to a method whose type signature is declared in RBS.
47
+ However, it does not check the return type yet. Redefinition of constants should be warned too.
48
+ We will survey what errors and warnings TypeProf can print, and evaluate the S/N ratio of each report.
49
+
50
+ For (3), since TypeProf uses whole program analysis, an error may be reported at a very different place from its root bug.
51
+ Thus, if TypeProf shows a possible type error, a diagnosis feature is needed to answer why TypeProf thinks that the error may occur.
52
+ TypeProf has already implemented a very primitive diagnosis feature, `Kernel#p`, to check what type an expression has.
53
+ Another idea is to create a pseudo backtrace why TypeProf thought the possible type error may occur.
54
+ We should consider this feature with LSP support.
55
+
56
+ ### Performance improvement
57
+
58
+ Currently, TypeProf is painfully slow. Even if a target application is small.
59
+
60
+ The main reason is that TypeProf analyzes not only the application code but also library code:
61
+ if an application requires `"foo"`, TypeProf actually loads `foo.rb` even from a gem,
62
+ and furthermore, if `foo.rb` requires `"bar"`, it loads `bar.rb` recursively.
63
+
64
+ RBS will help to stop this cascade;
65
+ when an application requires `"foo"`, TypeProf loads `sig/foo.rbs` instead of `foo.rb` if the `foo` gem contains both.
66
+ Such a RBS file is optional for TypeProf but required for Steep.
67
+ So, we think many gems will eventually equip their RBS declarations.
68
+
69
+ That being said, we should continue to improve the analysis performance of TypeProf. We have some ideas.
70
+
71
+ * Unfortunately, TypeProf often analyzes one method more than once when it accepts multiple types.
72
+ As TypeProf squashes the argument types to a union, this duplicated analysis is not necessarily needed.
73
+ But when TypeProf first analyzes a method, it is difficult to determine if the method will accept another type in further analysis.
74
+ So, we need good heuristics to guess whether a method accepts multiple types or not, and if so, delay its analysis.
75
+ * Currently, TypeProf executes the bytecode instructions step by step.
76
+ This requires creating an environment object after each instruction, which is very heavy.
77
+ Many environment creations can be omitted by executing each basic block instead of each instruction.
78
+ (Basic block execution will also make flow-sensitive analysis easier.)
79
+ * The slowest calculation in TypeProf is to create an instance of a Type class.
80
+ The creation uses memoization; TypeProf keeps all Type instances created so far, and reuses them if already exist.
81
+ However, it is very heavy to check if an instance already exists or not.
82
+ (Currently, it is very simply implemented by a big Hash table.)
83
+ We've already improved the memoization routine several times but looks like it is still the No.1 bottleneck.
84
+ We need to investigate and try improving more.
85
+ * TypeProf heavily uses Hash objects (including above) mainly to represent a set.
86
+ A union of sets is done by `Hash#merge`, which takes O(n).
87
+ A more lightweight data structure may make TypeProf faster.
88
+ (But clever structure often has a big constant term, so we need to evaluate the performance carefully.)
89
+ * Reusing an old analysis and incrementally updating it will bring a super big improvement.
90
+ This would be especially helpful for LSP support, so we need to tackle it after the analysis approach is mature.
91
+
92
+ ### Language Server Protocol (LSP) support
93
+
94
+ In the future, we want TypeProf to serve as a language server to show the result in IDE in real-time.
95
+ However, the current analysis approach is too slow for IDE. So we need to improve the performance first.
96
+
97
+ Even if TypeProf becomes fast enough, its approach has a fundamental problem.
98
+ Since TypeProf uses whole program analysis, one edit may cause a cascade of propagation:
99
+ if a user write `foo(42)`, an Integer is propagated to a method `foo`,
100
+ and if `foo` passes its argument to a method `bar`, it is propagated to `bar`, ...
101
+ So, a breakthrough for LSP may be still needed, e.g, limiting the propagation range in real-time analysis,
102
+ assuming that a type interface of module boundary is fixed, etc.
103
+
104
+ ## Relatively smaller TODOs
105
+
106
+ * Support more RBS features
107
+ * TypeProf does not deal with some RBS types well yet.
108
+ * For example, the `instance` type is handled as `untyped.
109
+ * The `self` type is handled well only when it is used as a return type.
110
+ * Using a value of the `void` type should be warned appropriately.
111
+ * RBS's `interface` is supported just like a module (i.e., `include _Foo` is explicitly required in RBS),
112
+ but it should be checked structually (i.e., it should be determined as a method set.)
113
+ * The variance of type parameters is currently ignored.
114
+
115
+ * Support more Ruby features
116
+ * Some meta-programming features like `Class.new`, `Object#method`, etc.
117
+ * It is possible to support `Class.new` by per-allocation-site approach:
118
+ e.g., In TypeProf, `A = Class.new; B = Class.new` will create two classes, but `2.times { Class.new }` will create one class.
119
+ * The analysis precision can be improved more for some Ruby features like pattern matching, keyword arguments, etc.
120
+ * For example, `foo(*args, k:1)` is currently compiled as if it is `foo(*(args + [{ :k => 1 }]))` into Ruby bytecode.
121
+ This mixes the keyword arguments to a rest array, and makes it difficult for TypeProf to track the keyword arguments.
122
+ * Support Enumerator as an Array-type container.
123
+ * Support `Module#protect` (but RBS does not yet).
124
+ * More heuristics may help such as `==` returns a bool regardless to its receiver and argument types.
125
+
126
+ * Make TypeProf more useful as a tool
127
+ * Currently, TypeProf provides only the analysis engine and a minimal set of features.
128
+ * The analysis result would be useful not only to generate RBS prototype
129
+ but also identifying the source location of a method definition, listing callsites of a method,
130
+ searching a method call by its argument types, etc.
131
+ * Sometimes, TypeProf prints very big union type, such as `Integer | Float | Complex | Rational | ...`.
132
+ Worse, the same big type is printed multiple times.
133
+ It may be useful to factor out such a long type by using type alias, for example.
data/lib/typeprof/cli.rb CHANGED
@@ -20,6 +20,7 @@ module TypeProf
20
20
  options = {}
21
21
  dir_filter = nil
22
22
  gem_rbs_features = []
23
+ gem_repo_dirs = []
23
24
  show_version = false
24
25
  max_sec = max_iter = nil
25
26
 
@@ -31,6 +32,7 @@ module TypeProf
31
32
  opt.on("--version", "Display typeprof version") { show_version = true }
32
33
  opt.on("-I DIR", "Add DIR to the load/require path") {|v| $LOAD_PATH << v }
33
34
  opt.on("-r FEATURE", "Require RBS of the FEATURE gem") {|v| gem_rbs_features << v }
35
+ opt.on("--repo DIR", "Add DIR to the RBS repository") {|v| gem_repo_dirs << v }
34
36
 
35
37
  opt.separator ""
36
38
  opt.separator "Analysis output options:"
@@ -87,6 +89,7 @@ module TypeProf
87
89
  rbs_files: rbs_files,
88
90
  output: output,
89
91
  gem_rbs_features: gem_rbs_features,
92
+ gem_repo_dirs: gem_repo_dirs,
90
93
  verbose: verbose,
91
94
  dir_filter: dir_filter,
92
95
  max_sec: max_sec,
data/lib/typeprof/config.rb CHANGED
@@ -6,6 +6,7 @@ module TypeProf
6
6
  :rbs_files,
7
7
  :output,
8
8
  :gem_rbs_features,
9
+ :gem_repo_dirs,
9
10
  :verbose,
10
11
  :dir_filter,
11
12
  :max_iter,
@@ -25,6 +26,7 @@ module TypeProf
25
26
  def initialize(**opt)
26
27
  opt[:output] ||= $stdout
27
28
  opt[:gem_rbs_features] ||= []
29
+ opt[:gem_repo_dirs] ||= []
28
30
  opt[:dir_filter] ||= DEFAULT_DIR_FILTER
29
31
  opt[:verbose] ||= 0
30
32
  opt[:options] ||= {}
data/lib/typeprof/import.rb CHANGED
@@ -3,6 +3,10 @@ require "rbs"
3
3
  module TypeProf
4
4
  class RBSReader
5
5
  def initialize
6
+ @repo = RBS::Repository.new
7
+ Config.gem_repo_dirs.each do |dir|
8
+ @repo.add(Pathname(dir))
9
+ end
6
10
  @env, @builtin_env_json = RBSReader.get_builtin_env
7
11
  end
8
12
 
@@ -11,7 +15,7 @@ module TypeProf
11
15
  unless @builtin_env
12
16
  @builtin_env = RBS::Environment.new
13
17
 
14
- loader = RBS::EnvironmentLoader.new
18
+ loader = RBS::EnvironmentLoader.new(repository: @repo)
15
19
  new_decls = loader.load(env: @builtin_env).map {|decl,| decl }
16
20
  @builtin_env_json = load_rbs(@builtin_env, new_decls)
17
21
  end
@@ -24,7 +28,7 @@ module TypeProf
24
28
  end
25
29
 
26
30
  def load_library(lib)
27
- loader = RBS::EnvironmentLoader.new(core_root: nil)
31
+ loader = RBS::EnvironmentLoader.new(core_root: nil, repository: @repo)
28
32
  loader.add(library: lib)
29
33
 
30
34
  case lib
@@ -38,7 +42,7 @@ module TypeProf
38
42
  end
39
43
 
40
44
  def load_path(path)
41
- loader = RBS::EnvironmentLoader.new(core_root: nil)
45
+ loader = RBS::EnvironmentLoader.new(core_root: nil, repository: @repo)
42
46
  loader.add(path: path)
43
47
  new_decls = loader.load(env: @env).map {|decl,| decl }
44
48
  RBSReader.load_rbs(@env, new_decls)
data/lib/typeprof/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module TypeProf
2
- VERSION = "0.10.0"
2
+ VERSION = "0.11.0"
3
3
  end
data/typeprof.gemspec CHANGED
@@ -30,5 +30,5 @@ Gem::Specification.new do |spec|
30
30
  spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
31
31
  spec.require_paths = ["lib"]
32
32
 
33
- spec.add_runtime_dependency "rbs", ">= 0.20.1"
33
+ spec.add_runtime_dependency "rbs", ">= 1.0.0"
34
34
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: typeprof
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.10.0
4
+ version: 0.11.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Yusuke Endoh
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2020-12-18 00:00:00.000000000 Z
11
+ date: 2020-12-24 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rbs
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 0.20.1
19
+ version: 1.0.0
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 0.20.1
26
+ version: 1.0.0
27
27
  description: |
28
28
  TypeProf performs a type analysis of non-annotated Ruby code.
29
29
 
@@ -48,6 +48,7 @@ files:
48
48
  - doc/doc.ja.md
49
49
  - doc/doc.md
50
50
  - doc/ppl2019.pdf
51
+ - doc/todo.md
51
52
  - exe/typeprof
52
53
  - lib/typeprof.rb
53
54
  - lib/typeprof/analyzer.rb
@@ -402,7 +403,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
402
403
  - !ruby/object:Gem::Version
403
404
  version: '0'
404
405
  requirements: []
405
- rubygems_version: 3.1.4
406
+ rubygems_version: 3.2.3
406
407
  signing_key:
407
408
  specification_version: 4
408
409
  summary: TypeProf is a type analysis tool for Ruby code based on abstract interpretation