earl-report 0.7.1 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +13 -12
- data/VERSION +1 -1
- data/lib/earl_report/views/earl_report.html.haml +4 -0
- data/lib/earl_report.rb +22 -8
- data/spec/test-files/doap.ttl +1 -1
- data/spec/test-files/report-complete.ttl +1 -1
- data/spec/test-files/report-no-foaf.ttl +1 -1
- data/spec/test-files/report-no-test.ttl +1 -1
- data/spec/test-files/results.html +448 -417
- data/spec/test-files/results.jsonld +40 -40
- data/spec/test-files/results.ttl +20 -20
- metadata +47 -43
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 96f39fab33a1be32b13d921e1ae02c692a841a9285fc6f79b8b75e0338c395c4
|
|
4
|
+
data.tar.gz: e4f8b52306cc7b25c9602e9f99dab808ae84d27c4b795e97f2185ba0a570213e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 978c03c37788f29f3fed96181f547dc40e7d65cac035331eef8155e10a77612f4917d028a54b063f9ae7ec259837a8e02433de78a7e2282d9724008ff67f4b6f
|
|
7
|
+
data.tar.gz: 836e06dfa27dddc721083926cf443304016e9ac7de7c73003096b673c14ebdcd52deb9d202525057c7042be1c61dbf30754fc922b58302c3e670973ff3a2a19a
|
data/README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
# earl-report
|
|
2
|
-
Ruby gem to consolidate multiple EARL report and generate a rollup conformance report.
|
|
2
|
+
Ruby gem to consolidate multiple [EARL][] report and generate a rollup conformance report.
|
|
3
3
|
|
|
4
4
|
[](http://badge.fury.io/rb/earl-report)
|
|
5
5
|
[](https://github.com/gkellogg/earl-report/actions?query=workflow%3ACI)
|
|
@@ -8,8 +8,9 @@ Ruby gem to consolidate multiple EARL report and generate a rollup conformance r
|
|
|
8
8
|
## Description
|
|
9
9
|
Reads a test manifest in the
|
|
10
10
|
[standard RDF WG format](http://www.w3.org/2011/rdf-wg/wiki/Turtle_Test_Suite)
|
|
11
|
-
along with one or more individual EARL reports and generates a rollup report in
|
|
12
|
-
|
|
11
|
+
along with one or more individual <abbr title="Evaluation and Report Language (EARL) 1.0 Schema">[EARL][]</abbr> reports and generates a rollup report in HTML in [ReSpec][] format.
|
|
12
|
+
|
|
13
|
+
[EARL]() reports use the [Evaluation and Report Language (EARL) 1.0 Schema][EARL] to describe test results in RDF to relate the results for a given _subject_ to a _test_ defined in a _test manifest_.
|
|
13
14
|
|
|
14
15
|
## Individual EARL reports
|
|
15
16
|
Results for individual implementations should be specified in Turtle form, but
|
|
@@ -26,13 +27,13 @@ in the following form:
|
|
|
26
27
|
dc:date "2012-11-17T15:19:11-05:00"^^xsd:dateTime];
|
|
27
28
|
earl:mode earl:automatic ] .
|
|
28
29
|
|
|
29
|
-
Additionally, `earl:subject` is expected to reference a [DOAP]
|
|
30
|
+
Additionally, `earl:subject` is expected to reference a [DOAP][] description
|
|
30
31
|
of the reported software, in the following form:
|
|
31
32
|
|
|
32
33
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
|
33
34
|
doap:name "RDF::Turtle" ;
|
|
34
35
|
doap:developer <https://greggkellogg.net/foaf#me> ;
|
|
35
|
-
doap:homepage <
|
|
36
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
|
36
37
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
|
37
38
|
doap:release [
|
|
38
39
|
doap:name "RDF::Turtle 3.1.0" ;
|
|
@@ -41,26 +42,26 @@ of the reported software, in the following form:
|
|
|
41
42
|
] ;
|
|
42
43
|
doap:programming-language "Ruby" .
|
|
43
44
|
|
|
44
|
-
The [DOAP]
|
|
45
|
+
The [DOAP][] description may be included in the [EARL][] report. If not found,
|
|
45
46
|
the IRI identified by `earl:subject` will be dereferenced and is presumed to
|
|
46
|
-
provide a [DOAP]
|
|
47
|
+
provide a [DOAP][] specification of the test subject.
|
|
47
48
|
|
|
48
|
-
The `doap:developer` is expected to reference a [FOAF]
|
|
49
|
+
The `doap:developer` is expected to reference a [FOAF][] profile for the agent
|
|
49
50
|
(user or organization) responsible for the test subject. It is expected to be
|
|
50
51
|
of the following form:
|
|
51
52
|
|
|
52
53
|
<https://greggkellogg.net/foaf#me> foaf:name "Gregg Kellogg" .
|
|
53
54
|
|
|
54
55
|
If not found, the IRI identified by `doap:developer`
|
|
55
|
-
will be dereferenced and is presumed to provide a [FOAF]
|
|
56
|
+
will be dereferenced and is presumed to provide a [FOAF][] profile of the developer.
|
|
56
57
|
|
|
57
58
|
Assertions are added to each test entry based on that test being referenced from the assertion.
|
|
58
59
|
|
|
59
60
|
## Manifest query
|
|
60
61
|
The test manifest is used to find test entries and a manifest. The built-in
|
|
61
|
-
query is based on the [standard RDF WG format]
|
|
62
|
+
query is based on the [standard RDF WG format][]. Alternative manifest formats
|
|
62
63
|
can be used by specifying a customized manifest query, but may require a custom
|
|
63
|
-
[Haml]
|
|
64
|
+
[Haml][] template for report generation. The default query is the following:
|
|
64
65
|
|
|
65
66
|
PREFIX mf: <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#>
|
|
66
67
|
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
|
|
@@ -75,7 +76,7 @@ can be used by specifying a customized manifest query, but may require a custom
|
|
|
75
76
|
}
|
|
76
77
|
|
|
77
78
|
## Report generation template
|
|
78
|
-
The report template is in [ReSpec][] form using [Haml]
|
|
79
|
+
The report template is in [ReSpec][] form using [Haml][] to generate individual report elements.
|
|
79
80
|
|
|
80
81
|
## Changes from previous versions
|
|
81
82
|
### Version 0.7
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.9.0
|
|
@@ -204,6 +204,7 @@
|
|
|
204
204
|
Test Manifests
|
|
205
205
|
%ul.toc
|
|
206
206
|
- tests['entries'].sort_by {|m| m['title'].to_s.downcase}.each_with_index do |manifest, ndx|
|
|
207
|
+
- manifest['rdfs:comment'] = manifest['rdfs:comment']['@value'] if manifest['rdfs:comment'].is_a?(Hash)
|
|
207
208
|
%li.tocline
|
|
208
209
|
%span.secno="2.#{ndx+1}"
|
|
209
210
|
%a.tocxref{href: "##{manifest['rdfs:comment'].gsub(' ', '-')}"}
|
|
@@ -296,6 +297,7 @@
|
|
|
296
297
|
Test Manifests
|
|
297
298
|
- tests['entries'].each_with_index do |manifest, ndx2|
|
|
298
299
|
- test_cases = manifest['entries']
|
|
300
|
+
- manifest['title'] = manifest['title']['@value'] if manifest['title'].is_a?(Hash)
|
|
299
301
|
%section{id: manifest['rdfs:comment'].gsub(' ', '-'), typeof: manifest['@type'].join(" "), resource: manifest['@id']}
|
|
300
302
|
%h2
|
|
301
303
|
%span.secno="2.#{ndx2+1}"
|
|
@@ -319,6 +321,7 @@
|
|
|
319
321
|
%a{href: '#' + subject_refs[subject['@id']]}<=subject['name']
|
|
320
322
|
- test_cases.each do |test|
|
|
321
323
|
- test['title'] ||= test['rdfs:label']
|
|
324
|
+
- test['title'] = test['title']['@value'] if test['title'].is_a?(Hash)
|
|
322
325
|
- test['title'] = Array(test['title']).first
|
|
323
326
|
- tid = "test_#{Digest::MD5.hexdigest(test['@id'])}"
|
|
324
327
|
- (test_info[tid] ||= []) << test
|
|
@@ -391,6 +394,7 @@
|
|
|
391
394
|
- if subject['developer']
|
|
392
395
|
%dt= "Developer"
|
|
393
396
|
- subject['developer'].each do |dev|
|
|
397
|
+
- dev['foaf:name'] = dev['foaf:name']['@value'] if dev['foaf:name'].is_a?(Hash)
|
|
394
398
|
%dd{rel: "doap:developer"}
|
|
395
399
|
%div{resource: dev['@id'], typeof: Array(dev['@type']).join(" ")}
|
|
396
400
|
- if dev.has_key?('@id')
|
data/lib/earl_report.rb
CHANGED
|
@@ -1,12 +1,18 @@
|
|
|
1
1
|
# EARL reporting
|
|
2
2
|
require 'json/ld'
|
|
3
|
-
require 'rdf/ordered_repo'
|
|
4
3
|
require 'rdf/turtle'
|
|
5
4
|
require 'rdf/vocab'
|
|
6
5
|
require 'sparql'
|
|
7
6
|
require 'haml'
|
|
8
7
|
require 'open-uri'
|
|
9
8
|
|
|
9
|
+
# Optionally, beautify HTML output
|
|
10
|
+
begin
|
|
11
|
+
require 'htmlbeautifier'
|
|
12
|
+
rescue LoadError
|
|
13
|
+
# No beautification
|
|
14
|
+
end
|
|
15
|
+
|
|
10
16
|
##
|
|
11
17
|
# EARL reporting class.
|
|
12
18
|
# Instantiate a new class using one or more input graphs
|
|
@@ -68,16 +74,16 @@ class EarlReport
|
|
|
68
74
|
ASSERTION_QUERY = %(
|
|
69
75
|
PREFIX earl: <http://www.w3.org/ns/earl#>
|
|
70
76
|
|
|
71
|
-
SELECT ?test ?subject ?by ?mode ?outcome
|
|
77
|
+
SELECT ?test ?subject ?by ?mode ?outcome ?info
|
|
72
78
|
WHERE {
|
|
73
79
|
?a a earl:Assertion;
|
|
74
80
|
earl:assertedBy ?by;
|
|
75
|
-
earl:result
|
|
81
|
+
earl:result ?result;
|
|
76
82
|
earl:subject ?subject;
|
|
77
83
|
earl:test ?test .
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
}
|
|
84
|
+
?result earl:outcome ?outcome .
|
|
85
|
+
OPTIONAL {?a earl:mode ?mode .}
|
|
86
|
+
OPTIONAL {?result earl:info ?info .}
|
|
81
87
|
}
|
|
82
88
|
ORDER BY ?subject
|
|
83
89
|
).freeze
|
|
@@ -396,6 +402,7 @@ class EarlReport
|
|
|
396
402
|
graph << RDF::Statement(a, RDF::Vocab::EARL.result, r)
|
|
397
403
|
graph << RDF::Statement(r, RDF.type, RDF::Vocab::EARL.TestResult)
|
|
398
404
|
graph << RDF::Statement(r, RDF::Vocab::EARL.outcome, solution[:outcome])
|
|
405
|
+
graph << RDF::Statement(r, RDF::Vocab::EARL.info, solution[:info]) if solution[:info]
|
|
399
406
|
end
|
|
400
407
|
|
|
401
408
|
# See if subject did not report results, which may indicate a formatting error in the EARL source
|
|
@@ -496,7 +503,14 @@ class EarlReport
|
|
|
496
503
|
end
|
|
497
504
|
|
|
498
505
|
# Generate HTML report
|
|
499
|
-
html = Haml
|
|
506
|
+
html = if Haml.const_defined?(:Template)
|
|
507
|
+
Haml::Template.new(format: :xhtml) {haml}.render(self, tests: json_hash)
|
|
508
|
+
else
|
|
509
|
+
Haml::Engine.new(haml, format: :xhtml).render(self, tests: json_hash)
|
|
510
|
+
end
|
|
511
|
+
if defined?(::HtmlBeautifier)
|
|
512
|
+
html = HtmlBeautifier.beautify(html)
|
|
513
|
+
end
|
|
500
514
|
io.write(html) if io
|
|
501
515
|
html
|
|
502
516
|
else
|
|
@@ -524,7 +538,7 @@ class EarlReport
|
|
|
524
538
|
# Reorder test assertions to make them consistent with subject order
|
|
525
539
|
Array(framed['entries']).each do |manifest|
|
|
526
540
|
manifest['entries'].each do |test|
|
|
527
|
-
test['assertions'] = test['assertions'].sort_by {|a| a['subject']}
|
|
541
|
+
test['assertions'] = test['assertions'].sort_by {|a| a['subject'].to_s}
|
|
528
542
|
end
|
|
529
543
|
end
|
|
530
544
|
framed
|
data/spec/test-files/doap.ttl
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
|
11
11
|
doap:name "RDF::Turtle" ;
|
|
12
|
-
doap:homepage <
|
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
|
11
11
|
doap:name "RDF::Turtle" ;
|
|
12
|
-
doap:homepage <
|
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
|
11
11
|
doap:name "RDF::Turtle" ;
|
|
12
|
-
doap:homepage <
|
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
|
11
11
|
doap:name "RDF::Turtle" ;
|
|
12
|
-
doap:homepage <
|
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|