spdx 2.0.11 → 4.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 2666b65e72b1bd12e9fe3d2214ef5177012c7df9ec9a755cf58910c52d6516cd
4
- data.tar.gz: ee8c457edf2d81a51ba5d54904fd56a5dd121f7494403bd591033485d8cd0396
3
+ metadata.gz: 57eb1d8fa593ee4d8502733dd5257f050a5a421e8e272036517f146a362f1eea
4
+ data.tar.gz: e6d33c4be673643a9e052df93fcac4d41a3c8c809fca6293482979f0ffb62fb1
5
5
  SHA512:
6
- metadata.gz: 728c74fa33a14330baab9d8c542c7700200dcc67e88310905404d4631e55a5ba08c80dc1145d7c4c32fccc3a2d8837692541e296465027729930e593ef54b46f
7
- data.tar.gz: 7719ebb561baa9451c90ca440ae548e76c6562bd260589681ecd8416b57b0fb805f6f28917842f05e0d49f5b91d0c52f026a86e7b56618872c83e9de1a8f9297
6
+ metadata.gz: 29d8dd9dd27711bc077d2a6828cfa58a8f378a5bcd2e6a0b3915fc070e2561b03b412f7683027985d50624a013310d6e4bdd4f053e3646c4fb659d4675553fd6
7
+ data.tar.gz: 03ea3f99549c654a817c53077f584feaacd4a85200f008e987a7aebac684435974269ab6edac88a22d253d78f4dd9171ab12f6713c9241a5cf153966ae8b7863
data/README.md CHANGED
@@ -1,7 +1,6 @@
1
- # [Spdx](http://libraries.io/rubygems/spdx) - A SPDX license normalizer
2
-
3
- This gem allows you to find the closest match using [FuzzyMatch](https://github.com/seamusabshere/fuzzy_match) in the [spdx-licenses](https://github.com/domcleal/spdx-licenses) list for similar (not necessarily exact) license names.
1
+ # [Spdx](http://libraries.io/rubygems/spdx) - A SPDX license parser
4
2
 
3
+ This gem allows you validate and parse spdx expressions. It also contains (relatively) up to date license and license exception lists from https://github.com/spdx/license-list-data/tree/master/json
5
4
  ## Installation
6
5
 
7
6
  Add this line to your application's Gemfile:
@@ -21,14 +20,81 @@ Or install it yourself as:
21
20
  ## Usage
22
21
 
23
22
  ```ruby
24
- Spdx.find('Apache 2') # => <SpdxLicenses::License:0x007fa3a2b462c8 @id="Apache-2.0", @name="Apache License 2.0", @osi_approved=true>
23
+ Spdx.valid?("(MIT OR AGPL-3.0+)")
24
+ => true
25
+ ```
26
+
27
+ ```ruby
28
+ Spdx.parse("MIT OR AGPL-3.0+")
29
+ => LogicalOr+OrExpression4 offset=0, "MIT OR AGPL-3.0+":
30
+ License+LicenseId0 offset=0, "MIT" (idstring)
31
+ LicensePlus+SimpleExpression0 offset=7, "AGPL-3.0+" (license_id):
32
+ License+LicenseId0 offset=7, "AGPL-3.0" (idstring)
33
+ ```
34
+
35
+ ```ruby
36
+ Spdx.normalize("Mit OR agpl-3.0+ AND APACHE-2.0")
37
+ => "MIT OR (AGPL-3.0+ AND Apache-2.0)"
38
+
39
+ Spdx.normalize("Mit OR agpl-3.0+ AND APACHE-2.0", top_level_parens: true)
40
+ => "(MIT OR (AGPL-3.0+ AND Apache-2.0))"
41
+ ```
42
+
43
+ ### Nodes
44
+
45
+ Parsed SPDX license expressions can be a number of various nodes. Each of these nodes share a few methods, most notably `text_value` which contains the text that spans that node, and `licenses` which contains an array of individual licenses used in that portion of the expression. Below are the nodes in more detail, and their additional methods.
46
+
47
+ #### `License`
48
+
49
+ This node represents a single license, and is the result of the most simple form of expression, e.g. `Spdx.parse("MIT")`. It can also be found as a child node of other nodes.
50
+
51
+ #### `LicensePlus`
52
+
53
+ This node represents the current version of a license or any later version, e.g. `Spdx.parse("CDDL-1.0+")`. The inner license node can be found via the `child` method.
54
+
55
+ #### `LicenseRef`
56
+
57
+ This node represents a reference to a license not defined in the SPDX license list, e.g. `Spdx.parse("LicenseRef-23")`.
58
+
59
+ #### `DocumentRef`
60
+
61
+ Similar to `LicenseRef`, this node also represents a reference to a license not defined in the SPDX license list, e.g. `Spdx.parse("DocumentRef-spdx-tool-1.2:LicenseRef-MIT-Style-2")`.
62
+
63
+ #### `LicenseException`
64
+
65
+ This node represents an exception to a license. See `With`.
66
+
67
+ #### `With`
68
+
69
+ This node represents a license with an SPDX-defined license exception, e.g. `Spdx.parse("GPL-2.0-or-later WITH Bison-exception-2.2")`. This node has two extra methods, `license` and `exception`, which return the nodes for the license portion of the expression and the exception portion of the expression, respectively.
70
+
71
+ #### `LogicalAnd`
72
+
73
+ This node represents an "AND" expression, e.g. `Spdx.parse("LGPL-2.1-only AND MIT")`. This node has two extra methods, `left` and `right`, which return the node for the left side of the expression and the node for the right side of the expression, respectively. Any node can be the child of `LogicalAnd`, including `LogicalAnd`/`LogicalOr`.
74
+
75
+ `Spdx.parse("(MIT AND GPL-1.0) OR MPL-2.0 AND Apache-2.0")` would result in the following tree:
76
+
77
+ ```txt
78
+ LogicalOr
79
+ ├── LogicalAnd
80
+ │ ├── License (MIT)
81
+ │ └── License (GPL-1.0)
82
+ └── LogicalAnd
83
+ ├── License (MPL-2.0)
84
+ └── License Apache-2.0
25
85
  ```
26
86
 
87
+ #### `LogicalOr`
88
+
89
+ The same as `LogicalAnd`, but for "OR" expressions.
90
+
27
91
  ## Testing
28
92
 
29
93
  Run the tests with:
30
94
 
31
- $ bundle exec rake
95
+ ```sh
96
+ bundle exec rspec
97
+ ```
32
98
 
33
99
  ## Contributing
34
100
 
data/exceptions.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
- "licenseListVersion": "3.8-126-gff8ed78",
3
- "releaseDate": "2020-05-15",
2
+ "licenseListVersion": "3.10-24-gd78ad74",
3
+ "releaseDate": "2020-11-05",
4
4
  "exceptions": [
5
5
  {
6
6
  "reference": "./GCC-exception-2.0.html",
data/lib/spdx.rb CHANGED
@@ -1,187 +1,12 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "spdx/version"
4
- require "fuzzy_match"
5
4
  require "spdx_parser"
6
5
  require "json"
7
6
  require_relative "exception"
8
7
  require_relative "license"
9
8
 
10
- # Fuzzy matcher for licenses to SPDX standard licenses
11
9
  module Spdx
12
- def self.find(name)
13
- name = name.strip
14
- return nil if commercial?(name)
15
- return nil if non_spdx?(name)
16
-
17
- search(name)
18
- end
19
-
20
- def self.search(name)
21
- lookup(name) ||
22
- find_by_special_case(name) ||
23
- closest(name)
24
- end
25
-
26
- def self.commercial?(name)
27
- name.casecmp("commercial").zero?
28
- end
29
-
30
- def self.non_spdx?(name)
31
- ["standard pil license"].include? name.downcase
32
- end
33
-
34
- def self.lookup(name)
35
- return false if name.nil?
36
- return lookup_license(name) if license_exists?(name)
37
-
38
- lowercase = licenses.keys.sort.find { |k| k.casecmp(name).zero? }
39
- lookup_license(lowercase) if lowercase
40
- end
41
-
42
- def self.closest(name)
43
- name.gsub!(/#{stop_words.join('|')}/i, "")
44
- name.gsub!(/(\d)/, ' \1 ')
45
- best_match = fuzzy_match(name)
46
- return nil unless best_match
47
-
48
- lookup(best_match) || find_by_name(best_match)
49
- end
50
-
51
- def self.matches(name, max_distance = 40)
52
- names.map { |key| [key, Text::Levenshtein.distance(name, key)] }
53
- .select { |arr| arr[1] <= max_distance }
54
- .sort_by { |arr| arr[1] }
55
- end
56
-
57
- def self.fuzzy_match(name)
58
- FuzzyMatch.new(names).find(name, must_match_at_least_one_word: true)
59
- end
60
-
61
- def self.stop_words
62
- %w[version software the or right all]
63
- end
64
-
65
- def self.find_by_name(name)
66
- match = licenses.find { |_k, v| v["name"] == name }
67
- lookup(match[0]) if match
68
- end
69
-
70
- def self.find_by_special_case(name)
71
- gpl = gpl_match(name)
72
- return gpl if gpl
73
-
74
- lookup(special_cases[name.downcase.strip])
75
- end
76
-
77
- def self.gpl_match(name)
78
- match = name.match(/^(l|a)?gpl-?\s?_?v?(1|2|3)\.?(\d)?(\+)?$/i)
79
- return unless match
80
-
81
- lookup "#{match[1]}GPL-#{match[2]}.#{match[3] || 0}#{match[4]}"
82
- end
83
-
84
- def self.special_cases
85
- {
86
- "perl_5" => "Artistic-1.0-Perl",
87
- "bsd3" => "BSD-3-Clause",
88
- "bsd" => "BSD-3-Clause",
89
- "bsd license" => "BSD-3-Clause",
90
- "new bsd license" => "BSD-3-Clause",
91
- "gnu gpl v2" => "GPL-2.0-only",
92
- "gpl" => "GPL-2.0+",
93
- "gpl-2 | gpl-3 [expanded from: gpl (≥ 2.0)]" => "GPL-2.0+",
94
- "gpl-2 | gpl-3 [expanded from: gpl]" => "GPL-2.0+",
95
- "gpl-2 | gpl-3 [expanded from: gpl (≥ 2)]" => "GPL-2.0+",
96
- "gpl-2 | gpl-3" => "GPL-2.0+",
97
- "gplv2 or later" => "GPL-2.0+",
98
- "the gpl v3" => "GPL-3.0",
99
- "gpl (≥ 3)" => "GPL-3.0+",
100
- "mpl2.0" => "mpl-2.0",
101
- "mpl1" => "mpl-1.0",
102
- "mpl1.0" => "mpl-1.0",
103
- "mpl1.1" => "mpl-1.1",
104
- "mpl2" => "mpl-2.0",
105
- "gnu lesser general public license" => "LGPL-2.1+",
106
- "lgplv2 or later" => "LGPL-2.1+",
107
- "gpl2 w/ cpe" => "GPL-2.0-with-classpath-exception",
108
- "new bsd license (gpl-compatible)" => "BSD-3-Clause",
109
- "public domain" => "Unlicense",
110
- "cc0" => "CC0-1.0",
111
- "artistic_2" => "Artistic-2.0",
112
- "artistic_1" => "Artistic-1.0",
113
- "alv2" => "Apache-2.0",
114
- "asl" => "Apache-2.0",
115
- "asl 2.0" => "Apache-2.0",
116
- "mpl 2.0" => "MPL-2.0",
117
- "publicdomain" => "Unlicense",
118
- "unlicensed" => "Unlicense",
119
- "psfl" => "Python-2.0",
120
- "psf" => "Python-2.0",
121
- "psf 2" => "Python-2.0",
122
- "psf 2.0" => "Python-2.0",
123
- "asl2" => "Apache-2.0",
124
- "al2" => "Apache-2.0",
125
- "aslv2" => "Apache-2.0",
126
- "apache_2_0" => "Apache-2.0",
127
- "apache_v2" => "Apache-2.0",
128
- "zpl 1.1" => "ZPL-1.1",
129
- "zpl 2.0" => "ZPL-2.0",
130
- "zpl 2.1" => "ZPL-2.1",
131
- "lgpl_2_1" => "LGPL-2.1",
132
- "lgpl_v2_1" => "LGPL-2.1",
133
- "lgpl version 3" => "LGPL-3.0",
134
- "gnu lgpl v3+" => "LGPL-3.0",
135
- "gnu lgpl" => "LGPL-2.1+",
136
- "cc by-sa 4.0" => "CC-BY-SA-4.0",
137
- "cc by-nc-sa 3.0" => "CC-BY-NC-SA-3.0",
138
- "cc by-sa 3.0" => "CC-BY-SA-3.0",
139
- "mpl v2.0" => "MPL-2.0",
140
- "mplv2.0" => "MPL-2.0",
141
- "mplv2" => "MPL-2.0",
142
- "cpal v1.0" => "CPAL-1.0",
143
- "cddl 1.0" => "CDDL-1.0",
144
- "cddl 1.1" => "CDDL-1.1",
145
- "epl" => "EPL-1.0",
146
- "mit-license" => "MIT",
147
- "(mit or x11)" => "MIT",
148
- "iscl" => "ISC",
149
- "wtf" => "WTFPL",
150
- "2-clause bsdl" => "BSD-2-clause",
151
- "3-clause bsdl" => "BSD-3-clause",
152
- "2-clause bsd" => "BSD-2-clause",
153
- "3-clause bsd" => "BSD-3-clause",
154
- "bsd 3-clause" => "BSD-3-clause",
155
- "bsd 2-clause" => "BSD-2-clause",
156
- "two-clause bsd-style license" => "BSD-2-clause",
157
- "bsd style" => "BSD-3-clause",
158
- "cc0 1.0 universal (cc0 1.0) public domain dedication" => "CC0-1.0",
159
- "common development and distribution license 1.0 (cddl-1.0)" => "CDDL-1.0",
160
- "european union public licence 1.0 (eupl 1.0)" => "EUPL-1.0",
161
- "european union public licence 1.1 (eupl 1.1)" => "EUPL-1.1",
162
- "european union public licence 1.2 (eupl 1.2)" => "EUPL-1.2",
163
- "vovida software license 1.0" => "VSL-1.0",
164
- "w3c license" => "W3C",
165
- "zlib/libpng license" => "zlib-acknowledgement",
166
- "gnu general public license (gpl)" => "GPL-2.0+",
167
- "gnu general public license v2 (gplv2)" => "GPL-2.0",
168
- "gnu general public license v2 or later (gplv2+)" => "GPL-2.0+",
169
- "gnu general public license v3 (gplv3)" => "GPL-3.0",
170
- "gnu general public license v3 or later (gplv3+)" => "GPL-3.0+",
171
- "gnu lesser general public license v2 (lgplv2)" => "LGPL-2.0",
172
- "gnu lesser general public license v2 or later (lgplv2+)" => "LGPL-2.0+",
173
- "gnu lesser general public license v3 (lgplv3)" => "LGPL-3.0",
174
- "gnu lesser general public license v3 or later (lgplv3+)" => "LGPL-3.0+",
175
- "gnu library or lesser general public license (lgpl)" => "LGPL-2.0+",
176
- "netscape public License (npl)" => "NPL-1.1",
177
- "apache software license" => "Apache-2.0",
178
- "academic free license (afl)" => "AFL-3.0",
179
- "gnu free documentation license (fdl)" => "GFDL-1.3",
180
- "sun industry standards source license (sissl)" => "SISSL-1.2",
181
- "zope public license" => "ZPL-2.1",
182
- }
183
- end
184
-
185
10
  def self.names
186
11
  (licenses.keys + licenses.map { |_k, v| v["name"] }).sort
187
12
  end
@@ -190,6 +15,7 @@ module Spdx
190
15
  unless defined?(@exceptions)
191
16
  data = JSON.parse(File.read(File.expand_path("../exceptions.json", __dir__)))
192
17
  @exceptions = {}
18
+
193
19
  data["exceptions"].each do |details|
194
20
  id = details.delete("licenseExceptionId")
195
21
  @exceptions[id] = details
@@ -199,27 +25,18 @@ module Spdx
199
25
  end
200
26
 
201
27
  def self.license_exists?(id)
202
- licenses.key?(id.to_s)
203
- end
204
-
205
- def self.lookup_license(id)
206
- json = licenses[id.to_s]
207
- Spdx::License.new(id.to_s, json["name"], json["isOsiApproved"]) if json
208
- end
209
-
210
- def self.lookup_exception(id)
211
- json = exceptions[id.to_s]
212
- Spdx::Exception.new(id.to_s, json["name"], json["isDeprecatedLicenseId"]) if json
28
+ licenses.key?(id.to_s) || licenses_downcase.key?(id.to_s.downcase)
213
29
  end
214
30
 
215
31
  def self.exception_exists?(id)
216
- exceptions.key?(id.to_s)
32
+ exceptions.key?(id.to_s) || exceptions_downcase.key?(id.to_s.downcase)
217
33
  end
218
34
 
219
35
  def self.licenses
220
36
  unless defined?(@licenses)
221
37
  data = JSON.parse(File.read(File.expand_path("../licenses.json", __dir__)))
222
38
  @licenses = {}
39
+
223
40
  data["licenses"].each do |details|
224
41
  id = details.delete("licenseId")
225
42
  @licenses[id] = details
@@ -228,7 +45,70 @@ module Spdx
228
45
  @licenses
229
46
  end
230
47
 
231
- def self.valid_spdx?(spdx_string)
48
+ def self.licenses_downcase
49
+ unless defined?(@licenses_downcase)
50
+ @licenses_downcase = {}
51
+ licenses.keys.each { |key| @licenses_downcase[key.downcase] = key }
52
+ end
53
+ @licenses_downcase
54
+ end
55
+
56
+ def self.exceptions_downcase
57
+ unless defined?(@exceptions_downcase)
58
+ @exceptions_downcase = {}
59
+ exceptions.keys.each { |key| @exceptions_downcase[key.downcase] = key }
60
+ end
61
+ @exceptions_downcase
62
+ end
63
+
64
+ def self.normalize(spdx_string, top_level_parens: false)
65
+ normalize_tree(SpdxParser.parse(spdx_string), parens: top_level_parens)
66
+ end
67
+
68
+ private_class_method def self.normalize_tree(node, parens: true)
69
+ case node
70
+ when SpdxGrammar::LogicalAnd
71
+ left = normalize_tree(node.left)
72
+ right = normalize_tree(node.right)
73
+ if parens
74
+ "(#{left} AND #{right})"
75
+ else
76
+ "#{left} AND #{right}"
77
+ end
78
+ when SpdxGrammar::LogicalOr
79
+ left = normalize_tree(node.left)
80
+ right = normalize_tree(node.right)
81
+ if parens
82
+ "(#{left} OR #{right})"
83
+ else
84
+ "#{left} OR #{right}"
85
+ end
86
+ when SpdxGrammar::With
87
+ license = normalize_tree(node.license)
88
+ exception = normalize_tree(node.exception)
89
+ if parens
90
+ "(#{license} WITH #{exception})"
91
+ else
92
+ "#{license} WITH #{exception}"
93
+ end
94
+ when SpdxGrammar::None
95
+ "NONE"
96
+ when SpdxGrammar::NoAssertion
97
+ "NOASSERTION"
98
+ when SpdxGrammar::License
99
+ licenses_downcase[node.text_value.downcase]
100
+ when SpdxGrammar::LicensePlus
101
+ "#{normalize_tree(node.child)}+"
102
+ when SpdxGrammar::LicenseRef
103
+ node.text_value
104
+ when SpdxGrammar::DocumentRef
105
+ node.text_value
106
+ when SpdxGrammar::LicenseException
107
+ exceptions_downcase[node.text_value.downcase]
108
+ end
109
+ end
110
+
111
+ def self.valid?(spdx_string)
232
112
  return false unless spdx_string.is_a?(String)
233
113
 
234
114
  SpdxParser.parse(spdx_string)
@@ -237,7 +117,7 @@ module Spdx
237
117
  false
238
118
  end
239
119
 
240
- def self.parse_spdx(spdx_string)
120
+ def self.parse(spdx_string)
241
121
  SpdxParser.parse(spdx_string)
242
122
  end
243
123
  end
data/lib/spdx/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Spdx
4
- VERSION = "2.0.11"
4
+ VERSION = "4.0.0"
5
5
  end
data/lib/spdx_grammar.rb CHANGED
@@ -1,19 +1,40 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SpdxGrammar
4
- class CompoundExpression < Treetop::Runtime::SyntaxNode
4
+ class LogicalBinary < Treetop::Runtime::SyntaxNode
5
+ # Used internally
6
+
5
7
  def licenses
6
- elements[0].licenses
8
+ (left.licenses + right.licenses).uniq
9
+ end
10
+
11
+ def left
12
+ elements[0]
13
+ end
14
+
15
+ def right
16
+ elements[1]
7
17
  end
8
18
  end
9
19
 
10
- class LogicalOr < Treetop::Runtime::SyntaxNode
20
+ class LogicalAnd < LogicalBinary
11
21
  end
12
22
 
13
- class LogicalAnd < Treetop::Runtime::SyntaxNode
23
+ class LogicalOr < LogicalBinary
14
24
  end
15
25
 
16
26
  class With < Treetop::Runtime::SyntaxNode
27
+ def licenses
28
+ license.licenses
29
+ end
30
+
31
+ def license
32
+ elements[0]
33
+ end
34
+
35
+ def exception
36
+ elements[1]
37
+ end
17
38
  end
18
39
 
19
40
  class None < Treetop::Runtime::SyntaxNode
@@ -30,20 +51,44 @@ module SpdxGrammar
30
51
 
31
52
  class License < Treetop::Runtime::SyntaxNode
32
53
  def licenses
33
- text_value
54
+ [text_value]
34
55
  end
35
56
  end
36
57
 
37
- class LicenseException < Treetop::Runtime::SyntaxNode
38
- # TODO: actually do license exceptions
58
+ class LicensePlus < Treetop::Runtime::SyntaxNode
59
+ def licenses
60
+ child.licenses
61
+ end
62
+
63
+ def child
64
+ elements[0]
65
+ end
39
66
  end
40
67
 
41
- class Body < Treetop::Runtime::SyntaxNode
68
+ class LicenseRef < Treetop::Runtime::SyntaxNode
42
69
  def licenses
43
- elements.map { |node| node.licenses if node.respond_to?(:licenses) }.flatten.uniq.compact
70
+ [text_value]
44
71
  end
45
72
  end
46
73
 
74
+ class DocumentRef < Treetop::Runtime::SyntaxNode
75
+ def licenses
76
+ [text_value]
77
+ end
78
+ end
79
+
80
+ class LicenseException < Treetop::Runtime::SyntaxNode
81
+ # TODO: actually do license exceptions
82
+ end
83
+
84
+ class GroupedExpression < Treetop::Runtime::SyntaxNode
85
+ # Used internally
86
+ end
87
+
88
+ class Operand < Treetop::Runtime::SyntaxNode
89
+ # Used internally
90
+ end
91
+
47
92
  class SpdxParseError < StandardError
48
93
  end
49
94
  end