earl-report 0.7.1 → 0.9.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +13 -12
- data/VERSION +1 -1
- data/lib/earl_report/views/earl_report.html.haml +4 -0
- data/lib/earl_report.rb +22 -8
- data/spec/test-files/doap.ttl +1 -1
- data/spec/test-files/report-complete.ttl +1 -1
- data/spec/test-files/report-no-foaf.ttl +1 -1
- data/spec/test-files/report-no-test.ttl +1 -1
- data/spec/test-files/results.html +448 -417
- data/spec/test-files/results.jsonld +40 -40
- data/spec/test-files/results.ttl +20 -20
- metadata +47 -43
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 96f39fab33a1be32b13d921e1ae02c692a841a9285fc6f79b8b75e0338c395c4
|
4
|
+
data.tar.gz: e4f8b52306cc7b25c9602e9f99dab808ae84d27c4b795e97f2185ba0a570213e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 978c03c37788f29f3fed96181f547dc40e7d65cac035331eef8155e10a77612f4917d028a54b063f9ae7ec259837a8e02433de78a7e2282d9724008ff67f4b6f
|
7
|
+
data.tar.gz: 836e06dfa27dddc721083926cf443304016e9ac7de7c73003096b673c14ebdcd52deb9d202525057c7042be1c61dbf30754fc922b58302c3e670973ff3a2a19a
|
data/README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
# earl-report
|
2
|
-
Ruby gem to consolidate multiple EARL report and generate a rollup conformance report.
|
2
|
+
Ruby gem to consolidate multiple [EARL][] report and generate a rollup conformance report.
|
3
3
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/earl-report.png)](http://badge.fury.io/rb/earl-report)
|
5
5
|
[![Build Status](https://github.com/gkellogg/earl-report/workflows/CI/badge.svg?branch=develop)](https://github.com/gkellogg/earl-report/actions?query=workflow%3ACI)
|
@@ -8,8 +8,9 @@ Ruby gem to consolidate multiple EARL report and generate a rollup conformance r
|
|
8
8
|
## Description
|
9
9
|
Reads a test manifest in the
|
10
10
|
[standard RDF WG format](http://www.w3.org/2011/rdf-wg/wiki/Turtle_Test_Suite)
|
11
|
-
along with one or more individual EARL reports and generates a rollup report in
|
12
|
-
|
11
|
+
along with one or more individual <abbr title="Evaluation and Report Language (EARL) 1.0 Schema">[EARL][]</abbr> reports and generates a rollup report in HTML in [ReSpec][] format.
|
12
|
+
|
13
|
+
[EARL]() reports use the [Evaluation and Report Language (EARL) 1.0 Schema][EARL] to describe test results in RDF to relate the results for a given _subject_ to a _test_ defined in a _test manifest_.
|
13
14
|
|
14
15
|
## Individual EARL reports
|
15
16
|
Results for individual implementations should be specified in Turtle form, but
|
@@ -26,13 +27,13 @@ in the following form:
|
|
26
27
|
dc:date "2012-11-17T15:19:11-05:00"^^xsd:dateTime];
|
27
28
|
earl:mode earl:automatic ] .
|
28
29
|
|
29
|
-
Additionally, `earl:subject` is expected to reference a [DOAP]
|
30
|
+
Additionally, `earl:subject` is expected to reference a [DOAP][] description
|
30
31
|
of the reported software, in the following form:
|
31
32
|
|
32
33
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
33
34
|
doap:name "RDF::Turtle" ;
|
34
35
|
doap:developer <https://greggkellogg.net/foaf#me> ;
|
35
|
-
doap:homepage <
|
36
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
36
37
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
37
38
|
doap:release [
|
38
39
|
doap:name "RDF::Turtle 3.1.0" ;
|
@@ -41,26 +42,26 @@ of the reported software, in the following form:
|
|
41
42
|
] ;
|
42
43
|
doap:programming-language "Ruby" .
|
43
44
|
|
44
|
-
The [DOAP]
|
45
|
+
The [DOAP][] description may be included in the [EARL][] report. If not found,
|
45
46
|
the IRI identified by `earl:subject` will be dereferenced and is presumed to
|
46
|
-
provide a [DOAP]
|
47
|
+
provide a [DOAP][] specification of the test subject.
|
47
48
|
|
48
|
-
The `doap:developer` is expected to reference a [FOAF]
|
49
|
+
The `doap:developer` is expected to reference a [FOAF][] profile for the agent
|
49
50
|
(user or organization) responsible for the test subject. It is expected to be
|
50
51
|
of the following form:
|
51
52
|
|
52
53
|
<https://greggkellogg.net/foaf#me> foaf:name "Gregg Kellogg" .
|
53
54
|
|
54
55
|
If not found, the IRI identified by `doap:developer`
|
55
|
-
will be dereferenced and is presumed to provide a [FOAF]
|
56
|
+
will be dereferenced and is presumed to provide a [FOAF][] profile of the developer.
|
56
57
|
|
57
58
|
Assertions are added to each test entry based on that test being referenced from the assertion.
|
58
59
|
|
59
60
|
## Manifest query
|
60
61
|
The test manifest is used to find test entries and a manifest. The built-in
|
61
|
-
query is based on the [standard RDF WG format]
|
62
|
+
query is based on the [standard RDF WG format][]. Alternative manifest formats
|
62
63
|
can be used by specifying a customized manifest query, but may require a custom
|
63
|
-
[Haml]
|
64
|
+
[Haml][] template for report generation. The default query is the following:
|
64
65
|
|
65
66
|
PREFIX mf: <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#>
|
66
67
|
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
|
@@ -75,7 +76,7 @@ can be used by specifying a customized manifest query, but may require a custom
|
|
75
76
|
}
|
76
77
|
|
77
78
|
## Report generation template
|
78
|
-
The report template is in [ReSpec][] form using [Haml]
|
79
|
+
The report template is in [ReSpec][] form using [Haml][] to generate individual report elements.
|
79
80
|
|
80
81
|
## Changes from previous versions
|
81
82
|
### Version 0.7
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.
|
1
|
+
0.9.0
|
@@ -204,6 +204,7 @@
|
|
204
204
|
Test Manifests
|
205
205
|
%ul.toc
|
206
206
|
- tests['entries'].sort_by {|m| m['title'].to_s.downcase}.each_with_index do |manifest, ndx|
|
207
|
+
- manifest['rdfs:comment'] = manifest['rdfs:comment']['@value'] if manifest['rdfs:comment'].is_a?(Hash)
|
207
208
|
%li.tocline
|
208
209
|
%span.secno="2.#{ndx+1}"
|
209
210
|
%a.tocxref{href: "##{manifest['rdfs:comment'].gsub(' ', '-')}"}
|
@@ -296,6 +297,7 @@
|
|
296
297
|
Test Manifests
|
297
298
|
- tests['entries'].each_with_index do |manifest, ndx2|
|
298
299
|
- test_cases = manifest['entries']
|
300
|
+
- manifest['title'] = manifest['title']['@value'] if manifest['title'].is_a?(Hash)
|
299
301
|
%section{id: manifest['rdfs:comment'].gsub(' ', '-'), typeof: manifest['@type'].join(" "), resource: manifest['@id']}
|
300
302
|
%h2
|
301
303
|
%span.secno="2.#{ndx2+1}"
|
@@ -319,6 +321,7 @@
|
|
319
321
|
%a{href: '#' + subject_refs[subject['@id']]}<=subject['name']
|
320
322
|
- test_cases.each do |test|
|
321
323
|
- test['title'] ||= test['rdfs:label']
|
324
|
+
- test['title'] = test['title']['@value'] if test['title'].is_a?(Hash)
|
322
325
|
- test['title'] = Array(test['title']).first
|
323
326
|
- tid = "test_#{Digest::MD5.hexdigest(test['@id'])}"
|
324
327
|
- (test_info[tid] ||= []) << test
|
@@ -391,6 +394,7 @@
|
|
391
394
|
- if subject['developer']
|
392
395
|
%dt= "Developer"
|
393
396
|
- subject['developer'].each do |dev|
|
397
|
+
- dev['foaf:name'] = dev['foaf:name']['@value'] if dev['foaf:name'].is_a?(Hash)
|
394
398
|
%dd{rel: "doap:developer"}
|
395
399
|
%div{resource: dev['@id'], typeof: Array(dev['@type']).join(" ")}
|
396
400
|
- if dev.has_key?('@id')
|
data/lib/earl_report.rb
CHANGED
@@ -1,12 +1,18 @@
|
|
1
1
|
# EARL reporting
|
2
2
|
require 'json/ld'
|
3
|
-
require 'rdf/ordered_repo'
|
4
3
|
require 'rdf/turtle'
|
5
4
|
require 'rdf/vocab'
|
6
5
|
require 'sparql'
|
7
6
|
require 'haml'
|
8
7
|
require 'open-uri'
|
9
8
|
|
9
|
+
# Optionally, beautify HTML output
|
10
|
+
begin
|
11
|
+
require 'htmlbeautifier'
|
12
|
+
rescue LoadError
|
13
|
+
# No beautification
|
14
|
+
end
|
15
|
+
|
10
16
|
##
|
11
17
|
# EARL reporting class.
|
12
18
|
# Instantiate a new class using one or more input graphs
|
@@ -68,16 +74,16 @@ class EarlReport
|
|
68
74
|
ASSERTION_QUERY = %(
|
69
75
|
PREFIX earl: <http://www.w3.org/ns/earl#>
|
70
76
|
|
71
|
-
SELECT ?test ?subject ?by ?mode ?outcome
|
77
|
+
SELECT ?test ?subject ?by ?mode ?outcome ?info
|
72
78
|
WHERE {
|
73
79
|
?a a earl:Assertion;
|
74
80
|
earl:assertedBy ?by;
|
75
|
-
earl:result
|
81
|
+
earl:result ?result;
|
76
82
|
earl:subject ?subject;
|
77
83
|
earl:test ?test .
|
78
|
-
|
79
|
-
|
80
|
-
}
|
84
|
+
?result earl:outcome ?outcome .
|
85
|
+
OPTIONAL {?a earl:mode ?mode .}
|
86
|
+
OPTIONAL {?result earl:info ?info .}
|
81
87
|
}
|
82
88
|
ORDER BY ?subject
|
83
89
|
).freeze
|
@@ -396,6 +402,7 @@ class EarlReport
|
|
396
402
|
graph << RDF::Statement(a, RDF::Vocab::EARL.result, r)
|
397
403
|
graph << RDF::Statement(r, RDF.type, RDF::Vocab::EARL.TestResult)
|
398
404
|
graph << RDF::Statement(r, RDF::Vocab::EARL.outcome, solution[:outcome])
|
405
|
+
graph << RDF::Statement(r, RDF::Vocab::EARL.info, solution[:info]) if solution[:info]
|
399
406
|
end
|
400
407
|
|
401
408
|
# See if subject did not report results, which may indicate a formatting error in the EARL source
|
@@ -496,7 +503,14 @@ class EarlReport
|
|
496
503
|
end
|
497
504
|
|
498
505
|
# Generate HTML report
|
499
|
-
html = Haml
|
506
|
+
html = if Haml.const_defined?(:Template)
|
507
|
+
Haml::Template.new(format: :xhtml) {haml}.render(self, tests: json_hash)
|
508
|
+
else
|
509
|
+
Haml::Engine.new(haml, format: :xhtml).render(self, tests: json_hash)
|
510
|
+
end
|
511
|
+
if defined?(::HtmlBeautifier)
|
512
|
+
html = HtmlBeautifier.beautify(html)
|
513
|
+
end
|
500
514
|
io.write(html) if io
|
501
515
|
html
|
502
516
|
else
|
@@ -524,7 +538,7 @@ class EarlReport
|
|
524
538
|
# Reorder test assertions to make them consistent with subject order
|
525
539
|
Array(framed['entries']).each do |manifest|
|
526
540
|
manifest['entries'].each do |test|
|
527
|
-
test['assertions'] = test['assertions'].sort_by {|a| a['subject']}
|
541
|
+
test['assertions'] = test['assertions'].sort_by {|a| a['subject'].to_s}
|
528
542
|
end
|
529
543
|
end
|
530
544
|
framed
|
data/spec/test-files/doap.ttl
CHANGED
@@ -9,7 +9,7 @@
|
|
9
9
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
11
11
|
doap:name "RDF::Turtle" ;
|
12
|
-
doap:homepage <
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
@@ -9,7 +9,7 @@
|
|
9
9
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
11
11
|
doap:name "RDF::Turtle" ;
|
12
|
-
doap:homepage <
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
@@ -9,7 +9,7 @@
|
|
9
9
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
11
11
|
doap:name "RDF::Turtle" ;
|
12
|
-
doap:homepage <
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|
@@ -9,7 +9,7 @@
|
|
9
9
|
|
10
10
|
<https://rubygems.org/gems/rdf-turtle> a doap:Project, earl:TestSubject, earl:Software ;
|
11
11
|
doap:name "RDF::Turtle" ;
|
12
|
-
doap:homepage <
|
12
|
+
doap:homepage <https://ruby-rdf.github.io/rdf-turtle> ;
|
13
13
|
doap:license <http://creativecommons.org/publicdomain/zero/1.0/> ;
|
14
14
|
doap:shortdesc "Turtle reader/writer for Ruby."@en ;
|
15
15
|
doap:description "RDF::Turtle is an Turtle reader/writer for the RDF.rb library suite."@en ;
|