earl-report 0.5.1 → 0.7.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +23 -13
- data/VERSION +1 -1
- data/bin/earl-report +4 -2
- data/lib/earl_report.rb +225 -141
- data/lib/earl_report/views/earl_report.html.haml +10 -10
- data/spec/earl_report_spec.rb +127 -31
- data/spec/spec_helper.rb +28 -9
- data/spec/test-files/doap.ttl +11 -11
- data/spec/test-files/foaf.ttl +2 -2
- data/spec/test-files/report-complete.ttl +15 -15
- data/spec/test-files/report-no-doap.ttl +4 -4
- data/spec/test-files/report-no-foaf.ttl +14 -14
- data/spec/test-files/report-no-test.ttl +48 -0
- data/spec/test-files/results.html +33 -33
- data/spec/test-files/results.jsonld +66 -58
- data/spec/test-files/results.ttl +60 -73
- metadata +103 -25
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7879d1953b9802807a57232a01207aa3b63f5ed80c025273ccdfe826e6f86c93
|
4
|
+
data.tar.gz: bf899c9f720aa8b8f67694e7398f693502fef7fb748b3fa5136acd05fe7495ca
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 57c7ec9fff3d0634c35fc96d646febda4c7ce71fdaa9f8c8f78702335fdedb7c0babe6ccddf515cd17065284dc0627d085a3d41ee05dfeea5dd96b73b25b61d5
|
7
|
+
data.tar.gz: fdee07b06ca6344764876c8649f0c906c974326f27427120afb0455ef47fd35d4387e9f64b9f84ecf552bad44581769eec9455e3c27612b66ba26c8ac510a609
|
data/README.md
CHANGED
@@ -2,8 +2,8 @@
|
|
2
2
|
Ruby gem to consolidate multiple EARL report and generate a rollup conformance report.
|
3
3
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/earl-report.png)](http://badge.fury.io/rb/earl-report)
|
5
|
-
[![Build Status](https://
|
6
|
-
[![Coverage Status](https://coveralls.io/repos/gkellogg/earl-report/badge.svg)](https://coveralls.io/r/gkellogg/earl-report)
|
5
|
+
[![Build Status](https://github.com/gkellogg/earl-report/workflows/CI/badge.svg?branch=develop)](https://github.com/gkellogg/earl-report/actions?query=workflow%3ACI)
|
6
|
+
[![Coverage Status](https://coveralls.io/repos/gkellogg/earl-report/badge.svg?branch=develop)](https://coveralls.io/r/gkellogg/earl-report?branch=develop)
|
7
7
|
|
8
8
|
## Description
|
9
9
|
Reads a test manifest in the
|
@@ -17,8 +17,8 @@ may be specified in an any compatible RDF serialization. The report is composed
|
|
17
17
|
in the following form:
|
18
18
|
|
19
19
|
[ a earl:Assertion;
|
20
|
-
earl:assertedBy <
|
21
|
-
earl:subject <
|
20
|
+
earl:assertedBy <https://greggkellogg.net/foaf#me>;
|
21
|
+
earl:subject <https://rubygems.org/gems/rdf-turtle>;
|
22
22
|
earl:test <http://dvcs.w3.org/hg/rdf/raw-file/default/rdf-turtle/tests-ttl/manifest.ttl#turtle-syntax-file-01>;
|
23
23
|
earl:result [
|
24
24
|
a earl:TestResult;
|
@@ -29,9 +29,9 @@ in the following form:
|
|
29
29
|
Additionally, `earl:subject` is expected to reference a [DOAP]() description
|
30
30
|
of the reported software, in the following form:
|
31
31
|
|
32
|
-
<
|
32
|
+
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
33
33
|
doap:name "RDF::Turtle" ;
|
34
|
-
doap:developer <
|
34
|
+
doap:developer <https://greggkellogg.net/foaf#me> ;
|
35
35
|
doap:homepage <http://ruby-rdf.github.com/rdf-turtle> ;
|
36
36
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
37
37
|
doap:release [
|
@@ -49,7 +49,7 @@ The `doap:developer` is expected to reference a [FOAF]() profile for the agent
|
|
49
49
|
(user or organization) responsible for the test subject. It is expected to be
|
50
50
|
of the following form:
|
51
51
|
|
52
|
-
<
|
52
|
+
<https://greggkellogg.net/foaf#me> foaf:name "Gregg Kellogg" .
|
53
53
|
|
54
54
|
If not found, the IRI identified by `doap:developer`
|
55
55
|
will be dereferenced and is presumed to provide a [FOAF]() profile of the developer.
|
@@ -78,11 +78,21 @@ can be used by specifying a customized manifest query, but may require a custom
|
|
78
78
|
The report template is in [ReSpec][] form using [Haml]() to generate individual report elements.
|
79
79
|
|
80
80
|
## Changes from previous versions
|
81
|
-
Version 0.
|
81
|
+
### Version 0.7
|
82
|
+
Version 0.7 creates incompatibilities with previous result formats. Previously, terms were "added" to the EARL vocabulary to help with coordination. This version pulls that back, but does depend on an added `earl:Report` class which acts as an appropriate container for for collections of `earl:Assertion`.
|
83
|
+
|
84
|
+
Within an `earl:TestCase`, the former `earl:assertions` property is replaced with a reverse property on `earl:test` so that, in the JSON-LD representation, an `earl:TestCase` will contain a property representing the related assertions.
|
85
|
+
|
86
|
+
Also, `earl:assertions` had been miss-appropriated to reference the sources of the individual test results provided for each test subject. A new property as appropriated from the Test Manifest vocabulary: `mf:report`. The alias `assertions` continues to be used within the JSON-LD representation, although it now maps to `mf:report` at the top level, and as mentioned, is a reverse property of `earl:test` within an `earl:TestCase`.
|
87
|
+
|
88
|
+
These changes may require updates to customized Haml templates.
|
89
|
+
|
90
|
+
### Version 0.3
|
91
|
+
Version 0.3 and prior re-constructed the test manifest used to create the body of the report, which caused information not described within the query to be lost. Starting with 0.4, all manifests and assertions are read into a single graph, and each test references a list of assertions against it using a list referenced by `mf:assertions`. Additionally, all included manifests are included in a top-level entity referenced via `mf:entries`. For example:
|
82
92
|
|
83
93
|
<> a earl:Software, doap:Project;
|
84
94
|
mf:entries (<http://example/manifest.ttl>);
|
85
|
-
|
95
|
+
mf:assertions (<spec/test-files/report-complete.ttl>) .
|
86
96
|
|
87
97
|
<http://example/manifest.ttl> a mf:Manifest, earl:Report;
|
88
98
|
mf:name "Example Test Cases";
|
@@ -94,16 +104,16 @@ Version 0.3 and prior re-constructed the test manifest used to create the body o
|
|
94
104
|
rdfs:comment "Blank subject";
|
95
105
|
mf:action <http://example/test-00.ttl>;
|
96
106
|
mf:result <http://example/test-00.out>;
|
97
|
-
|
107
|
+
mf:assertions ([
|
98
108
|
a earl:Assertion;
|
99
|
-
earl:assertedBy <
|
109
|
+
earl:assertedBy <https://greggkellogg.net/foaf#me>;
|
100
110
|
earl:mode earl:automatic;
|
101
111
|
earl:result [
|
102
112
|
a earl:TestResult;
|
103
113
|
dc:date "2012-11-06T19:23:29-08:00"^^xsd:dateTime;
|
104
114
|
earl:outcome earl:passed
|
105
115
|
];
|
106
|
-
earl:subject <
|
116
|
+
earl:subject <https://rubygems.org/gems/rdf-turtle>;
|
107
117
|
earl:test <http://example/manifest.ttl#testeval00>
|
108
118
|
]) .
|
109
119
|
|
@@ -132,7 +142,7 @@ Generally, creating a `json` format first is more efficient. Subsequent invocati
|
|
132
142
|
When run, `earl-report` attempts to open the file `.earl` in the current directory. This file is in [YAML][] format with entries for each option. Use the `--rc` option to automatically generate it.
|
133
143
|
|
134
144
|
## Author
|
135
|
-
* [Gregg Kellogg](
|
145
|
+
* [Gregg Kellogg](https://github.com/gkellogg) - <https://greggkellogg.net/>
|
136
146
|
|
137
147
|
## License
|
138
148
|
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.
|
1
|
+
0.7.0
|
data/bin/earl-report
CHANGED
@@ -15,6 +15,7 @@ OPT_ARGS = [
|
|
15
15
|
["--output", "-o", GetoptLong::REQUIRED_ARGUMENT,"Output report to file"],
|
16
16
|
["--query", GetoptLong::REQUIRED_ARGUMENT,"Query, or file containing query for extracting information from Test manifest"],
|
17
17
|
["--rc", GetoptLong::NO_ARGUMENT, "Write options to run-control file"],
|
18
|
+
["--strict", GetoptLong::NO_ARGUMENT, "Fails if any skipped result is found"],
|
18
19
|
["--template", GetoptLong::OPTIONAL_ARGUMENT,"Specify or return default report template"],
|
19
20
|
["--verbose", GetoptLong::NO_ARGUMENT, "Detail on execution"],
|
20
21
|
["--help", "-?", GetoptLong::NO_ARGUMENT, "This message"]
|
@@ -65,6 +66,7 @@ opts.each do |opt, arg|
|
|
65
66
|
when '--name' then options[:name] = arg
|
66
67
|
when '--output' then options[:io] = File.open(arg, "w")
|
67
68
|
when '--rc' then options[:rc] = true
|
69
|
+
when '--strict' then options[:strict] = true
|
68
70
|
when '--template' then options[:template] = (File.open(arg, "r") unless arg.empty?)
|
69
71
|
when '--verbose' then options[:verbose] = true
|
70
72
|
when '--help' then usage
|
@@ -94,6 +96,6 @@ if options.has_key?(:template) && options[:template].nil?
|
|
94
96
|
elsif ARGV.empty?
|
95
97
|
usage
|
96
98
|
else
|
97
|
-
earl = EarlReport.new(*ARGV, options)
|
98
|
-
earl.generate(options)
|
99
|
+
earl = EarlReport.new(*ARGV, **options)
|
100
|
+
earl.generate(**options)
|
99
101
|
end
|
data/lib/earl_report.rb
CHANGED
@@ -1,5 +1,7 @@
|
|
1
1
|
# EARL reporting
|
2
|
-
require '
|
2
|
+
require 'json/ld'
|
3
|
+
require 'rdf/turtle'
|
4
|
+
require 'rdf/vocab'
|
3
5
|
require 'sparql'
|
4
6
|
require 'haml'
|
5
7
|
require 'open-uri'
|
@@ -11,6 +13,7 @@ class EarlReport
|
|
11
13
|
autoload :VERSION, 'earl_report/version'
|
12
14
|
|
13
15
|
attr_reader :graph
|
16
|
+
attr_reader :verbose
|
14
17
|
|
15
18
|
# Return information about each test.
|
16
19
|
# Tests all have an mf:action property.
|
@@ -90,12 +93,20 @@ class EarlReport
|
|
90
93
|
"foaf" => "http://xmlns.com/foaf/0.1/",
|
91
94
|
"rdfs" => "http://www.w3.org/2000/01/rdf-schema#",
|
92
95
|
"assertedBy" => {"@type" => "@id"},
|
93
|
-
"assertions" => {"@type" => "@id", "@container" => "@set"},
|
96
|
+
"assertions" => {"@id" => "mf:report", "@type" => "@id", "@container" => "@set"},
|
94
97
|
"bibRef" => {"@id" => "dc:bibliographicCitation"},
|
95
98
|
"created" => {"@id" => "doap:created", "@type" => "xsd:date"},
|
96
99
|
"description" => {"@id" => "rdfs:comment", "@language" => "en"},
|
97
100
|
"developer" => {"@id" => "doap:developer", "@type" => "@id", "@container" => "@set"},
|
98
101
|
"doapDesc" => {"@id" => "doap:description", "@language" => "en"},
|
102
|
+
"entries" => {
|
103
|
+
"@id" => "mf:entries", "@type" => "@id", "@container" => "@list",
|
104
|
+
"@context" => {
|
105
|
+
"assertions" => {
|
106
|
+
"@reverse" => "earl:test", "@type" => "@id", "@container" => "@set"
|
107
|
+
},
|
108
|
+
}
|
109
|
+
},
|
99
110
|
"generatedBy" => {"@type" => "@id"},
|
100
111
|
"homepage" => {"@id" => "doap:homepage", "@type" => "@id"},
|
101
112
|
"language" => {"@id" => "doap:programming-language"},
|
@@ -111,7 +122,6 @@ class EarlReport
|
|
111
122
|
"testAction" => {"@id" => "mf:action", "@type" => "@id"},
|
112
123
|
"testResult" => {"@id" => "mf:result", "@type" => "@id"},
|
113
124
|
"title" => {"@id" => "mf:name"},
|
114
|
-
"entries" => {"@id" => "mf:entries", "@type" => "@id", "@container" => "@list"},
|
115
125
|
"testSubjects" => {"@type" => "@id", "@container" => "@set"},
|
116
126
|
"xsd" => {"@id" => "http://www.w3.org/2001/XMLSchema#"}
|
117
127
|
},
|
@@ -152,35 +162,70 @@ class EarlReport
|
|
152
162
|
}]
|
153
163
|
}.freeze
|
154
164
|
|
165
|
+
TURTLE_PREFIXES = %(@prefix dc: <http://purl.org/dc/terms/> .
|
166
|
+
@prefix doap: <http://usefulinc.com/ns/doap#> .
|
167
|
+
@prefix earl: <http://www.w3.org/ns/earl#> .
|
168
|
+
@prefix mf: <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#> .
|
169
|
+
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
|
170
|
+
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
|
171
|
+
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
|
172
|
+
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
|
173
|
+
).gsub(/^ /, '')
|
174
|
+
|
175
|
+
TURTLE_SOFTWARE = %(
|
176
|
+
# Report Generation Software
|
177
|
+
<https://rubygems.org/gems/earl-report> a earl:Software, doap:Project;
|
178
|
+
doap:name "earl-report";
|
179
|
+
doap:shortdesc "Earl Report summary generator"@en;
|
180
|
+
doap:description "EarlReport generates HTML+RDFa rollups of multiple EARL reports"@en;
|
181
|
+
doap:homepage <https://github.com/gkellogg/earl-report>;
|
182
|
+
doap:programming-language "Ruby";
|
183
|
+
doap:license <http://unlicense.org>;
|
184
|
+
doap:release <https://github.com/gkellogg/earl-report/tree/#{VERSION}>;
|
185
|
+
doap:developer <https://greggkellogg.net/foaf#me> .
|
186
|
+
|
187
|
+
<https://github.com/gkellogg/earl-report/tree/#{VERSION}> a doap:Version;
|
188
|
+
doap:name "earl-report-#{VERSION}";
|
189
|
+
doap:created "#{File.mtime(File.expand_path('../../VERSION', __FILE__)).strftime('%Y-%m-%d')}"^^xsd:date;
|
190
|
+
doap:revision "#{VERSION}" .
|
191
|
+
).gsub(/^ /, '')
|
192
|
+
|
155
193
|
# Convenience vocabularies
|
156
|
-
class EARL < RDF::Vocabulary("http://www.w3.org/ns/earl#"); end
|
157
194
|
class MF < RDF::Vocabulary("http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#"); end
|
158
195
|
|
159
196
|
##
|
160
197
|
# Load test assertions and look for referenced software and developer information
|
161
|
-
#
|
162
|
-
#
|
163
|
-
# @
|
164
|
-
#
|
165
|
-
#
|
166
|
-
#
|
167
|
-
#
|
168
|
-
#
|
169
|
-
#
|
170
|
-
#
|
171
|
-
#
|
172
|
-
#
|
173
|
-
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
|
198
|
+
#
|
199
|
+
# @param [Array<String>] files Assertions
|
200
|
+
# @param [String] base (nil) Base IRI for loading Manifest
|
201
|
+
# @param [String] bibRef ('Unknown reference')
|
202
|
+
# ReSpec bibliography reference for specification being tested
|
203
|
+
# @param [Boolean] json (false) File is in the JSON format of a report.
|
204
|
+
# @param [String, Array<String>] manifest (nil) Test manifest(s)
|
205
|
+
# @param [String] name ('Unknown') Name of specification
|
206
|
+
# @param [String] query (MANIFEST_QUERY)
|
207
|
+
# Query, or file containing query for extracting information from Test manifests
|
208
|
+
# @param [Boolean] strict (false) Abort on any warning
|
209
|
+
# @param [Boolean] verbose (false)
|
210
|
+
def initialize(*files,
|
211
|
+
base: nil,
|
212
|
+
bibRef: 'Unknown reference',
|
213
|
+
json: false,
|
214
|
+
manifest: nil,
|
215
|
+
name: 'Unknown',
|
216
|
+
query: MANIFEST_QUERY,
|
217
|
+
strict: false,
|
218
|
+
verbose: false,
|
219
|
+
**options)
|
220
|
+
@verbose = verbose
|
221
|
+
raise "Test Manifests must be specified with :manifest option" unless manifest || json
|
178
222
|
raise "Require at least one input file" if files.empty?
|
179
223
|
@files = files
|
180
224
|
@prefixes = {}
|
225
|
+
@warnings = 0
|
181
226
|
|
182
|
-
# If provided
|
183
|
-
if
|
227
|
+
# If provided json, it is used for generating all other output forms
|
228
|
+
if json
|
184
229
|
@json_hash = ::JSON.parse(File.read(files.first))
|
185
230
|
# Add a base_uri so relative subjects aren't dropped
|
186
231
|
JSON::LD::Reader.open(files.first, base_uri: "http://example.org/report") do |r|
|
@@ -195,25 +240,25 @@ class EarlReport
|
|
195
240
|
end
|
196
241
|
|
197
242
|
# Load manifests, possibly with base URI
|
198
|
-
status "read #{
|
243
|
+
status "read #{manifest.inspect}"
|
199
244
|
man_opts = {}
|
200
|
-
man_opts[:base_uri] = RDF::URI(
|
245
|
+
man_opts[:base_uri] = RDF::URI(base) if base
|
201
246
|
@graph = RDF::Graph.new
|
202
|
-
Array(
|
247
|
+
Array(manifest).each do |man|
|
203
248
|
g = RDF::Graph.load(man, unique_bnodes: true, **man_opts)
|
204
249
|
status " loaded #{g.count} triples from #{man}"
|
205
250
|
graph << g
|
206
251
|
end
|
207
252
|
|
208
253
|
# Hash test cases by URI
|
209
|
-
tests = SPARQL.execute(
|
254
|
+
tests = SPARQL.execute(query, graph)
|
210
255
|
.to_a
|
211
256
|
.inject({}) {|memo, soln| memo[soln[:uri]] = soln; memo}
|
212
257
|
|
213
258
|
if tests.empty?
|
214
259
|
raise "no tests found querying manifest.\n" +
|
215
260
|
"Results are found using the following query, this can be overridden using the --query option:\n" +
|
216
|
-
"#{
|
261
|
+
"#{query}"
|
217
262
|
end
|
218
263
|
|
219
264
|
# Manifests in graph
|
@@ -228,14 +273,13 @@ class EarlReport
|
|
228
273
|
end
|
229
274
|
|
230
275
|
assertion_stats = {}
|
231
|
-
release_node_mapper = {}
|
232
276
|
|
233
277
|
# Read test assertion files into assertion graph
|
234
278
|
files.flatten.each do |file|
|
235
279
|
status "read #{file}"
|
236
280
|
file_graph = RDF::Graph.load(file)
|
237
281
|
if file_graph.first_object(predicate: RDF::URI('http://www.w3.org/ns/earl#testSubjects'))
|
238
|
-
|
282
|
+
warn " skip #{file}, which seems to be a previous rollup earl report"
|
239
283
|
@files -= [file]
|
240
284
|
else
|
241
285
|
status " loaded #{file_graph.count} triples"
|
@@ -287,15 +331,15 @@ class EarlReport
|
|
287
331
|
subjects[solution[:uri]] = RDF::URI(file)
|
288
332
|
|
289
333
|
# Add TestSubject information to main graph
|
290
|
-
|
334
|
+
doapName = solution[:name].to_s if solution[:name]
|
291
335
|
language = solution[:language].to_s if solution[:language]
|
292
336
|
doapDesc = solution[:doapDesc] if solution[:doapDesc]
|
293
337
|
doapDesc.language ||= :en if doapDesc
|
294
338
|
devName = solution[:devName].to_s if solution[:devName]
|
295
339
|
graph << RDF::Statement(solution[:uri], RDF.type, RDF::Vocab::DOAP.Project)
|
296
|
-
graph << RDF::Statement(solution[:uri], RDF.type, EARL.TestSubject)
|
297
|
-
graph << RDF::Statement(solution[:uri], RDF.type, EARL.Software)
|
298
|
-
graph << RDF::Statement(solution[:uri], RDF::Vocab::DOAP.name,
|
340
|
+
graph << RDF::Statement(solution[:uri], RDF.type, RDF::Vocab::EARL.TestSubject)
|
341
|
+
graph << RDF::Statement(solution[:uri], RDF.type, RDF::Vocab::EARL.Software)
|
342
|
+
graph << RDF::Statement(solution[:uri], RDF::Vocab::DOAP.name, doapName)
|
299
343
|
graph << RDF::Statement(solution[:uri], RDF::Vocab::DOAP.developer, solution[:developer])
|
300
344
|
graph << RDF::Statement(solution[:uri], RDF::Vocab::DOAP.homepage, solution[:homepage]) if solution[:homepage]
|
301
345
|
graph << RDF::Statement(solution[:uri], RDF::Vocab::DOAP.description, doapDesc) if doapDesc
|
@@ -323,12 +367,12 @@ class EarlReport
|
|
323
367
|
subject = solution[:subject]
|
324
368
|
unless tests[solution[:test]]
|
325
369
|
assertion_stats["Skipped"] = assertion_stats["Skipped"].to_i + 1
|
326
|
-
|
370
|
+
warn "Skipping result for #{solution[:test]} for #{subject}, which is not defined in manifests"
|
327
371
|
next
|
328
372
|
end
|
329
373
|
unless subjects[subject]
|
330
374
|
assertion_stats["Missing Subject"] = assertion_stats["Missing Subject"].to_i + 1
|
331
|
-
|
375
|
+
warn "No test result subject found for #{subject}: in #{subjects.keys.join(', ')}"
|
332
376
|
next
|
333
377
|
end
|
334
378
|
found_solutions ||= true
|
@@ -339,19 +383,19 @@ class EarlReport
|
|
339
383
|
ary = test_assertion_lists[solution[:test]]
|
340
384
|
|
341
385
|
ary[ndx] = a = RDF::Node.new
|
342
|
-
graph << RDF::Statement(a, RDF.type, EARL.Assertion)
|
343
|
-
graph << RDF::Statement(a, EARL.subject, subject)
|
344
|
-
graph << RDF::Statement(a, EARL.test, solution[:test])
|
345
|
-
graph << RDF::Statement(a, EARL.assertedBy, solution[:by])
|
346
|
-
graph << RDF::Statement(a, EARL.mode, solution[:mode]) if solution[:mode]
|
386
|
+
graph << RDF::Statement(a, RDF.type, RDF::Vocab::EARL.Assertion)
|
387
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.subject, subject)
|
388
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.test, solution[:test])
|
389
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.assertedBy, solution[:by])
|
390
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.mode, solution[:mode]) if solution[:mode]
|
347
391
|
r = RDF::Node.new
|
348
|
-
graph << RDF::Statement(a, EARL.result, r)
|
349
|
-
graph << RDF::Statement(r, RDF.type, EARL.TestResult)
|
350
|
-
graph << RDF::Statement(r, EARL.outcome, solution[:outcome])
|
392
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.result, r)
|
393
|
+
graph << RDF::Statement(r, RDF.type, RDF::Vocab::EARL.TestResult)
|
394
|
+
graph << RDF::Statement(r, RDF::Vocab::EARL.outcome, solution[:outcome])
|
351
395
|
end
|
352
396
|
|
353
397
|
# See if subject did not report results, which may indicate a formatting error in the EARL source
|
354
|
-
|
398
|
+
warn "No results found for #{subject} using #{ASSERTION_QUERY}" unless found_solutions
|
355
399
|
end
|
356
400
|
end
|
357
401
|
|
@@ -363,70 +407,49 @@ class EarlReport
|
|
363
407
|
unless a
|
364
408
|
assertion_stats["Untested"] = assertion_stats["Untested"].to_i + 1
|
365
409
|
ary[ndx] = a = RDF::Node.new
|
366
|
-
graph << RDF::Statement(a, RDF.type, EARL.Assertion)
|
367
|
-
graph << RDF::Statement(a, EARL.subject, subjects.keys[ndx])
|
368
|
-
graph << RDF::Statement(a, EARL.test, test)
|
410
|
+
graph << RDF::Statement(a, RDF.type, RDF::Vocab::EARL.Assertion)
|
411
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.subject, subjects.keys[ndx])
|
412
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.test, test)
|
369
413
|
r = RDF::Node.new
|
370
|
-
graph << RDF::Statement(a, EARL.result, r)
|
371
|
-
graph << RDF::Statement(r, RDF.type, EARL.TestResult)
|
372
|
-
graph << RDF::Statement(r, EARL.outcome, EARL.untested)
|
414
|
+
graph << RDF::Statement(a, RDF::Vocab::EARL.result, r)
|
415
|
+
graph << RDF::Statement(r, RDF.type, RDF::Vocab::EARL.TestResult)
|
416
|
+
graph << RDF::Statement(r, RDF::Vocab::EARL.outcome, RDF::Vocab::EARL.untested)
|
373
417
|
end
|
374
|
-
|
375
|
-
# This counts on order being preserved in default repository so we can avoid using an rdf:List
|
376
|
-
graph << RDF::Statement(test, EARL.assertions, a)
|
377
418
|
end
|
378
419
|
end
|
379
420
|
|
380
421
|
assertion_stats.each {|stat, count| status("Assertions #{stat}: #{count}")}
|
381
422
|
|
382
423
|
# Add report wrapper to graph
|
383
|
-
ttl = %(
|
384
|
-
|
385
|
-
|
386
|
-
|
387
|
-
|
388
|
-
|
389
|
-
|
390
|
-
|
391
|
-
|
392
|
-
|
393
|
-
earl:generatedBy <http://rubygems.org/gems/earl-report>;
|
394
|
-
earl:assertions #{subjects.values.map {|f| f.to_ntriples}.join(",\n ")};
|
395
|
-
earl:testSubjects #{subjects.keys.map {|f| f.to_ntriples}.join(",\n ")};
|
396
|
-
mf:entries (#{man_uris.map {|f| f.to_ntriples}.join("\n ")}) .
|
397
|
-
|
398
|
-
<http://rubygems.org/gems/earl-report> a earl:Software, doap:Project;
|
399
|
-
doap:name "earl-report";
|
400
|
-
doap:shortdesc "Earl Report summary generator"@en;
|
401
|
-
doap:description "EarlReport generates HTML+RDFa rollups of multiple EARL reports"@en;
|
402
|
-
doap:homepage <https://github.com/gkellogg/earl-report>;
|
403
|
-
doap:programming-language "Ruby";
|
404
|
-
doap:license <http://unlicense.org>;
|
405
|
-
doap:release <https://github.com/gkellogg/earl-report/tree/#{VERSION}>;
|
406
|
-
doap:developer <http://greggkellogg.net/foaf#me> .
|
407
|
-
|
408
|
-
<https://github.com/gkellogg/earl-report/tree/#{VERSION}> a doap:Version;
|
409
|
-
doap:name "earl-report-#{VERSION}";
|
410
|
-
doap:created "#{File.mtime(File.expand_path('../../VERSION', __FILE__)).strftime('%Y-%m-%d')}"^^xsd:date;
|
411
|
-
doap:revision "#{VERSION}" .
|
412
|
-
).gsub(/^ /, '')
|
424
|
+
ttl = TURTLE_PREFIXES + %(
|
425
|
+
<> a earl:Software, doap:Project;
|
426
|
+
doap:name #{quoted(name)};
|
427
|
+
dc:bibliographicCitation "#{bibRef}";
|
428
|
+
earl:generatedBy <https://rubygems.org/gems/earl-report>;
|
429
|
+
mf:report #{subjects.values.map {|f| f.to_ntriples}.join(",\n ")};
|
430
|
+
earl:testSubjects #{subjects.keys.map {|f| f.to_ntriples}.join(",\n ")};
|
431
|
+
mf:entries (#{man_uris.map {|f| f.to_ntriples}.join("\n ")}) .
|
432
|
+
).gsub(/^ /, '') +
|
433
|
+
TURTLE_SOFTWARE
|
413
434
|
RDF::Turtle::Reader.new(ttl) {|r| graph << r}
|
414
435
|
|
415
436
|
# Each manifest is an earl:Report
|
416
437
|
man_uris.each do |u|
|
417
|
-
graph << RDF::Statement.new(u, RDF.type, EARL.Report)
|
438
|
+
graph << RDF::Statement.new(u, RDF.type, RDF::Vocab::EARL.Report)
|
418
439
|
end
|
419
440
|
|
420
441
|
# Each subject is an earl:TestSubject
|
421
442
|
subjects.keys.each do |u|
|
422
|
-
graph << RDF::Statement.new(u, RDF.type, EARL.TestSubject)
|
443
|
+
graph << RDF::Statement.new(u, RDF.type, RDF::Vocab::EARL.TestSubject)
|
423
444
|
end
|
424
445
|
|
425
446
|
# Each assertion test is a earl:TestCriterion and earl:TestCase
|
426
447
|
test_resources.each do |u|
|
427
|
-
graph << RDF::Statement.new(u, RDF.type, EARL.TestCriterion)
|
428
|
-
graph << RDF::Statement.new(u, RDF.type, EARL.TestCase)
|
448
|
+
graph << RDF::Statement.new(u, RDF.type, RDF::Vocab::EARL.TestCriterion)
|
449
|
+
graph << RDF::Statement.new(u, RDF.type, RDF::Vocab::EARL.TestCase)
|
429
450
|
end
|
451
|
+
|
452
|
+
raise "Warnings issued in strict mode" if strict && @warnings > 0
|
430
453
|
end
|
431
454
|
|
432
455
|
##
|
@@ -434,48 +457,47 @@ class EarlReport
|
|
434
457
|
#
|
435
458
|
# If no `io` option is provided, the output is returned as a string
|
436
459
|
#
|
460
|
+
# @param [Symbol] format (:html)
|
461
|
+
# @param [IO] io (nil)
|
462
|
+
# `IO` to output results
|
437
463
|
# @param [Hash{Symbol => Object}] options
|
438
|
-
# @
|
439
|
-
#
|
440
|
-
# Optional `IO` to output results
|
464
|
+
# @param [String] template
|
465
|
+
# HAML template for generating report
|
441
466
|
# @return [String] serialized graph, if `io` is nil
|
442
|
-
def generate(
|
443
|
-
options = {format: :html}.merge(options)
|
444
|
-
|
445
|
-
io = options[:io]
|
467
|
+
def generate(format: :html, io: nil, template: nil, **options)
|
446
468
|
|
447
|
-
status("generate: #{
|
469
|
+
status("generate: #{format}")
|
448
470
|
##
|
449
471
|
# Retrieve Hashed information in JSON-LD format
|
450
|
-
case
|
472
|
+
case format
|
451
473
|
when :jsonld, :json
|
452
474
|
json = json_hash.to_json(JSON::LD::JSON_STATE)
|
453
475
|
io.write(json) if io
|
454
476
|
json
|
455
477
|
when :turtle, :ttl
|
456
478
|
if io
|
457
|
-
earl_turtle(
|
479
|
+
earl_turtle(io: io)
|
458
480
|
else
|
459
481
|
io = StringIO.new
|
460
|
-
earl_turtle(
|
482
|
+
earl_turtle(io: io)
|
461
483
|
io.rewind
|
462
484
|
io.read
|
463
485
|
end
|
464
486
|
when :html
|
465
|
-
|
466
|
-
when String then
|
467
|
-
when IO, StringIO then
|
487
|
+
haml = case template
|
488
|
+
when String then template
|
489
|
+
when IO, StringIO then template.read
|
468
490
|
else
|
469
491
|
File.read(File.expand_path('../earl_report/views/earl_report.html.haml', __FILE__))
|
470
492
|
end
|
471
493
|
|
472
494
|
# Generate HTML report
|
473
|
-
html = Haml::Engine.new(
|
495
|
+
html = Haml::Engine.new(haml, format: :xhtml).render(self, tests: json_hash)
|
474
496
|
io.write(html) if io
|
475
497
|
html
|
476
498
|
else
|
477
|
-
writer = RDF::Writer.for(
|
478
|
-
writer.dump(@graph, io,
|
499
|
+
writer = RDF::Writer.for(format)
|
500
|
+
writer.dump(@graph, io, standard_prefixes: true, **options)
|
479
501
|
end
|
480
502
|
end
|
481
503
|
|
@@ -493,10 +515,10 @@ class EarlReport
|
|
493
515
|
embed: '@never',
|
494
516
|
pruneBlankNodeIdentifiers: false)
|
495
517
|
# Reorder test subjects by @id
|
496
|
-
framed['testSubjects'] = framed['testSubjects'].sort_by {|t| t['@id']}
|
518
|
+
framed['testSubjects'] = Array(framed['testSubjects']).sort_by {|t| t['@id']}
|
497
519
|
|
498
520
|
# Reorder test assertions to make them consistent with subject order
|
499
|
-
framed['entries'].each do |manifest|
|
521
|
+
Array(framed['entries']).each do |manifest|
|
500
522
|
manifest['entries'].each do |test|
|
501
523
|
test['assertions'] = test['assertions'].sort_by {|a| a['subject']}
|
502
524
|
end
|
@@ -512,61 +534,123 @@ class EarlReport
|
|
512
534
|
|
513
535
|
##
|
514
536
|
# Output consoloated EARL report as Turtle
|
515
|
-
# @param [
|
516
|
-
#
|
537
|
+
# @param [IO] io ($stdout)
|
538
|
+
# `IO` to output results
|
517
539
|
# @return [String]
|
518
|
-
def earl_turtle(
|
519
|
-
|
540
|
+
def earl_turtle(io: $stdout)
|
541
|
+
context = JSON::LD::Context.parse(json_hash['@context'])
|
542
|
+
io.write(TURTLE_PREFIXES + "\n")
|
520
543
|
|
521
|
-
|
544
|
+
# Write project header
|
545
|
+
ttl_entity(io, json_hash, context)
|
522
546
|
|
523
|
-
# Write
|
524
|
-
|
525
|
-
|
547
|
+
# Write out each manifest entry
|
548
|
+
io.puts("# Manifests")
|
549
|
+
json_hash['entries'].each do |man|
|
550
|
+
ttl_entity(io, man, context)
|
526
551
|
|
527
|
-
|
528
|
-
|
529
|
-
|
530
|
-
|
531
|
-
|
552
|
+
# Output each test entry with assertions
|
553
|
+
man['entries'].each do |entry|
|
554
|
+
ttl_entity(io, entry, context)
|
555
|
+
end
|
556
|
+
end
|
532
557
|
|
533
|
-
#
|
534
|
-
|
535
|
-
|
536
|
-
writer.send(:statement, manifest)
|
558
|
+
# Output each DOAP
|
559
|
+
json_hash['testSubjects'].each do |doap|
|
560
|
+
ttl_entity(io, doap, context)
|
537
561
|
|
538
|
-
#
|
539
|
-
|
540
|
-
|
562
|
+
# FOAF
|
563
|
+
dev = doap['developer']
|
564
|
+
dev = [dev] unless dev.is_a?(Array)
|
565
|
+
dev.each do |foaf|
|
566
|
+
ttl_entity(io, foaf, context)
|
541
567
|
end
|
542
568
|
end
|
569
|
+
|
570
|
+
io.write(TURTLE_SOFTWARE)
|
571
|
+
end
|
543
572
|
|
544
|
-
|
545
|
-
io.
|
546
|
-
|
547
|
-
|
573
|
+
def ttl_entity(io, entity, context)
|
574
|
+
io.write(ttl_value(entity) + " " + entity.map do |dk, dv|
|
575
|
+
case dk
|
576
|
+
when '@context', '@id'
|
577
|
+
nil
|
578
|
+
when '@type'
|
579
|
+
"a " + ttl_value(dv)
|
580
|
+
when 'assertions'
|
581
|
+
if dv.all? {|a| a.is_a?(String)}
|
582
|
+
"mf:report #{ttl_value(dv, whitespace: "\n ")}"
|
583
|
+
else
|
584
|
+
"earl:assertions #{dv.map {|a| ttl_assertion(a)}.join(", ")}"
|
585
|
+
end
|
586
|
+
when 'entries'
|
587
|
+
"mf:entries #{ttl_value({'@list' => dv}, whitespace: "\n ")}"
|
588
|
+
when 'release'
|
589
|
+
"doap:release [doap:revision #{quoted(dv['revision'])}]"
|
590
|
+
else
|
591
|
+
dv = [dv] unless dv.is_a?(Array)
|
592
|
+
dv = dv.map {|v| v.is_a?(Hash) ? v : context.expand_value(dk, v)}
|
593
|
+
"#{ttl_value(dk)} #{ttl_value(dv, whitespace: "\n ")}"
|
594
|
+
end
|
595
|
+
end.compact.join(" ;\n ") + " .\n\n")
|
596
|
+
end
|
548
597
|
|
549
|
-
|
550
|
-
|
551
|
-
|
598
|
+
def ttl_value(value, whitespace: " ")
|
599
|
+
if value.is_a?(Array)
|
600
|
+
value.map {|v| ttl_value(v)}.join(",#{whitespace}")
|
601
|
+
elsif value.is_a?(Hash)
|
602
|
+
if value.key?('@list')
|
603
|
+
"(#{value['@list'].map {|vv| ttl_value(vv)}.join(whitespace)})"
|
604
|
+
elsif value.key?('@value')
|
605
|
+
quoted(value['@value'], language: value['@language'], datatype: value['@type'])
|
606
|
+
elsif value.key?('@id')
|
607
|
+
ttl_value(value['@id'])
|
608
|
+
else
|
609
|
+
"[]"
|
552
610
|
end
|
611
|
+
elsif value.start_with?(/https?/) || value.start_with?('/')
|
612
|
+
"<#{value}>"
|
613
|
+
elsif value.include?(':')
|
614
|
+
value
|
615
|
+
elsif json_hash['@context'][value].is_a?(Hash)
|
616
|
+
json_hash['@context'][value].fetch('@id', "earl:#{value}")
|
617
|
+
elsif value.empty?
|
618
|
+
"<>"
|
619
|
+
else
|
620
|
+
"earl:#{value}"
|
553
621
|
end
|
622
|
+
end
|
554
623
|
|
555
|
-
|
556
|
-
|
557
|
-
|
558
|
-
|
624
|
+
def ttl_assertion(value)
|
625
|
+
return ttl_value(value) if value.is_a?(String)
|
626
|
+
block = [
|
627
|
+
"[",
|
628
|
+
" a earl:Assertion ;",
|
629
|
+
" earl:test #{ttl_value(value['test'])} ;",
|
630
|
+
" earl:subject #{ttl_value(value['subject'])} ;",
|
631
|
+
" earl:result [",
|
632
|
+
" a earl:TestResult ;",
|
633
|
+
" earl:outcome #{ttl_value(value['result']['outcome'])}",
|
634
|
+
" ] ;",
|
635
|
+
]
|
636
|
+
block << " earl:assertedBy #{ttl_value(value['assertedBy'])} ;" if value['assertedBy']
|
637
|
+
|
638
|
+
block.join("\n") + "\n ]"
|
559
639
|
end
|
560
640
|
|
561
|
-
def quoted(string)
|
562
|
-
(@turtle_writer ||= RDF::Turtle::Writer.new).send(:quoted, string)
|
641
|
+
def quoted(string, language: nil, datatype: nil)
|
642
|
+
str = (@turtle_writer ||= RDF::Turtle::Writer.new).send(:quoted, string)
|
643
|
+
str += "@#{language}" if language
|
644
|
+
str += "^^#{ttl_value(datatype)}" if datatype
|
645
|
+
str
|
563
646
|
end
|
564
647
|
|
565
648
|
def warn(message)
|
649
|
+
@warnings += 1
|
566
650
|
$stderr.puts message
|
567
651
|
end
|
568
652
|
|
569
653
|
def status(message)
|
570
|
-
$stderr.puts message if
|
654
|
+
$stderr.puts message if verbose
|
571
655
|
end
|
572
656
|
end
|