earl-report 0.0.3 → 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +37 -37
- data/VERSION +1 -1
- data/bin/earl-report +29 -1
- data/lib/earl_report.rb +208 -164
- data/spec/earl_report_spec.rb +293 -68
- data/spec/test-files/foaf.ttl +3 -0
- data/spec/test-files/results.html +98 -60
- data/spec/test-files/results.jsonld +72 -44
- data/spec/test-files/results.ttl +23 -13
- metadata +2 -2
data/README.md
CHANGED
@@ -1,42 +1,13 @@
|
|
1
1
|
# earl-report
|
2
|
-
============
|
3
|
-
|
4
2
|
Ruby gem to consolidate multiple EARL report and generate a rollup conformance report.
|
5
3
|
|
6
4
|
## Description
|
7
|
-
|
8
5
|
Reads a test manifest in the
|
9
6
|
[standard RDF WG format](http://www.w3.org/2011/rdf-wg/wiki/Turtle_Test_Suite)
|
10
|
-
and generates a rollup report in
|
11
|
-
|
12
|
-
## Test Specifications
|
13
|
-
The test manifest is presumed to be of the following form:
|
14
|
-
|
15
|
-
### Manifest Header
|
16
|
-
|
17
|
-
The manifest header looks like:
|
18
|
-
|
19
|
-
<> rdf:type mf:Manifest ;
|
20
|
-
rdfs:comment "Turtle tests" ;
|
21
|
-
mf:entries
|
22
|
-
(
|
23
|
-
....
|
24
|
-
) .
|
25
|
-
|
26
|
-
where .... is a list of links to test descriptions, one per line.
|
27
|
-
|
28
|
-
### Test description
|
29
|
-
|
30
|
-
This is an example of a syntax test:
|
31
|
-
|
32
|
-
<#turtle-syntax-file-01> rdf:type rdft:TestTurtlePositiveSyntax ;
|
33
|
-
mf:name "turtle-syntax-file-01" ;
|
34
|
-
rdfs:comment "Further description of the test" ;
|
35
|
-
mf:action <turtle-syntax-file-01.ttl> ;
|
36
|
-
mf:result <turtle-eval-struct-01.nt> .
|
7
|
+
along with one or more individual EARL reports and generates a rollup report in
|
8
|
+
HTML+RDFa in [ReSpec][] format.
|
37
9
|
|
38
10
|
## Individual EARL reports
|
39
|
-
|
40
11
|
Results for individual implementations should be specified in Turtle form, but
|
41
12
|
may be specified in an any compatible RDF serialization (JSON-LD is presumed to
|
42
13
|
be a cached rollup report). The report is composed of `Assertion` declarations
|
@@ -75,9 +46,35 @@ of the following form:
|
|
75
46
|
If not found, the IRI identified by `doap:developer`
|
76
47
|
will be dereferenced and is presumed to provide a [FOAF]() profile of the developer.
|
77
48
|
|
78
|
-
##
|
49
|
+
## Manifest query
|
50
|
+
The test manifest is used to generate `earl:TestCase` entries for each test
|
51
|
+
described in the test manifest. It will also summarize each test, including
|
52
|
+
any input and result files associated with the tests. The built-in query
|
53
|
+
is based on the [standard RDF WG format](). Alternative manifest formats
|
54
|
+
can be used by specifying a customized manifest query. The default query
|
55
|
+
is the following:
|
56
|
+
|
57
|
+
PREFIX dc: <http://purl.org/dc/terms/>
|
58
|
+
PREFIX mf: <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#>
|
59
|
+
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
|
60
|
+
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
61
|
+
|
62
|
+
SELECT ?lh ?uri ?title ?description ?testAction ?testResult
|
63
|
+
WHERE {
|
64
|
+
?uri mf:name ?title; mf:action ?testAction.
|
65
|
+
OPTIONAL { ?uri rdfs:comment ?description. }
|
66
|
+
OPTIONAL { ?uri mf:result ?testResult. }
|
67
|
+
OPTIONAL { [ mf:entries ?lh] . ?lh rdf:first ?uri . }
|
68
|
+
}
|
69
|
+
|
70
|
+
If any result has a non-null `?lh`, it is taken as the list head and used
|
71
|
+
to maintain the list order within `earl:tests`.
|
72
|
+
|
73
|
+
## Report generation template
|
74
|
+
The report template is in [ReSpec][] form using [Haml]() to generate individual report elements.
|
79
75
|
|
80
|
-
|
76
|
+
## Usage
|
77
|
+
The `earl-report` command may be used to directly create a report from zero or more input files, which are themselves [EARL][] report.
|
81
78
|
|
82
79
|
gem install earl-report
|
83
80
|
|
@@ -86,12 +83,15 @@ The `earl` command may be used to directly create a report from zero or more inp
|
|
86
83
|
--tempate [FILE] # Location of report template file; returns default if not specified
|
87
84
|
--bibRef # The default ReSpec-formatted bibliographic reference for the report
|
88
85
|
--name # The name of the software being reported upon
|
89
|
-
manifest
|
86
|
+
--manifest FILE # a test manifest used to define test descriptions
|
87
|
+
--base URI # Base URI to by applied when parsing test manifest
|
88
|
+
--query FILE # Alternative SPARQL query for extracting information from manifest
|
90
89
|
report* # one or more EARL report in most RDF formats
|
91
90
|
|
92
|
-
|
93
|
-
|
94
|
-
|
91
|
+
### Initialization File
|
92
|
+
`earl-report` can take defaults for options from an initialization file.
|
93
|
+
When run, `earl-report` attempts to open the file `.earl` in the current directory.
|
94
|
+
This file is in [YAML][] format with entries for each option.
|
95
95
|
|
96
96
|
## License
|
97
97
|
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.0
|
1
|
+
0.1.0
|
data/bin/earl-report
CHANGED
@@ -4,16 +4,21 @@ $:.unshift(File.expand_path(File.join(File.dirname(__FILE__), "..", 'lib')))
|
|
4
4
|
|
5
5
|
require 'earl_report'
|
6
6
|
require 'getoptlong'
|
7
|
+
require 'yaml'
|
7
8
|
|
8
9
|
def run(input, options)
|
9
10
|
end
|
10
11
|
|
11
12
|
OPT_ARGS = [
|
13
|
+
["--base", GetoptLong::REQUIRED_ARGUMENT,"Base URI to use when loading test manifest"],
|
12
14
|
["--bibRef", GetoptLong::REQUIRED_ARGUMENT,"ReSpec BibRef of specification being reported upon"],
|
13
15
|
["--format", "-f", GetoptLong::REQUIRED_ARGUMENT,"Format of output, one of 'ttl', 'json', or 'html'"],
|
14
16
|
["--json", GetoptLong::NO_ARGUMENT, "Input is a JSON-LD formatted result"],
|
17
|
+
["--manifest", GetoptLong::REQUIRED_ARGUMENT,"Test manifest"],
|
15
18
|
["--name", GetoptLong::REQUIRED_ARGUMENT,"Name of specification"],
|
16
19
|
["--output", "-o", GetoptLong::REQUIRED_ARGUMENT,"Output report to file"],
|
20
|
+
["--query", GetoptLong::REQUIRED_ARGUMENT,"Query, or file containing query for extracting information from Test manifest"],
|
21
|
+
["--rc", GetoptLong::NO_ARGUMENT, "Write options to run-control file"],
|
17
22
|
["--template", GetoptLong::OPTIONAL_ARGUMENT,"Specify or return default report template"],
|
18
23
|
["--verbose", GetoptLong::NO_ARGUMENT, "Detail on execution"],
|
19
24
|
["--help", "-?", GetoptLong::NO_ARGUMENT, "This message"]
|
@@ -22,6 +27,9 @@ def usage
|
|
22
27
|
STDERR.puts %{
|
23
28
|
Generate EARL report for mutliple test results against a test manifest.
|
24
29
|
|
30
|
+
Options are initialized by reading optional run-control file '.earl' in the local directory,
|
31
|
+
if it exists.
|
32
|
+
|
25
33
|
Usage: #{$0} [options] test-manifest test-result ...
|
26
34
|
}.gsub(/^ /, '')
|
27
35
|
width = OPT_ARGS.map do |o|
|
@@ -42,13 +50,20 @@ opts = GetoptLong.new(*OPT_ARGS.map {|o| o[0..-2]})
|
|
42
50
|
options = {
|
43
51
|
:format => :html,
|
44
52
|
:io => STDOUT}
|
53
|
+
options.merge!(YAML.load(File.open ".earl")) if File.exist?(".earl")
|
45
54
|
|
46
55
|
opts.each do |opt, arg|
|
47
56
|
case opt
|
57
|
+
when '--base' then options[:base] = arg
|
48
58
|
when '--bibRef' then options[:bibRef] = arg
|
49
59
|
when '--format' then options[:format] = arg.to_sym
|
60
|
+
when '--manifest' then options[:manifest] = arg
|
61
|
+
when '--query' then options[:manifest] = arg
|
62
|
+
when '--base' then options[:base] = arg
|
63
|
+
when '--json' then options[:json] = true
|
50
64
|
when '--name' then options[:name] = arg
|
51
65
|
when '--output' then options[:io] = File.open(arg, "w")
|
66
|
+
when '--rc' then options[:rc] = true
|
52
67
|
when '--template' then options[:template] = arg
|
53
68
|
when '--verbose' then options[:verbose] = true
|
54
69
|
when '--help' then usage
|
@@ -57,6 +72,19 @@ opts.each do |opt, arg|
|
|
57
72
|
end
|
58
73
|
end
|
59
74
|
|
75
|
+
# Replace query from a specified file, with the query itself
|
76
|
+
if options.has_key?(:query) && File.exist?(options[:query])
|
77
|
+
options[:query] = File.read(options[:query])
|
78
|
+
end
|
79
|
+
|
80
|
+
# Write run-control file to output
|
81
|
+
if options.has_key?(:rc)
|
82
|
+
io = options.delete(:io) || STDOUT
|
83
|
+
options.delete_if {|k, v| [:rc, :json, :output, :verbose].include?(k)}
|
84
|
+
io.puts options.to_yaml
|
85
|
+
exit 0
|
86
|
+
end
|
87
|
+
|
60
88
|
# If requesting template, just return it
|
61
89
|
if options.has_key?(:template) && options[:template].empty?
|
62
90
|
File.open(File.expand_path("../../lib/earl_report/views/earl_report.html.haml", __FILE__)) do |f|
|
@@ -64,5 +92,5 @@ if options.has_key?(:template) && options[:template].empty?
|
|
64
92
|
end
|
65
93
|
else
|
66
94
|
earl = EarlReport.new(*ARGV, options)
|
67
|
-
earl.generate(options
|
95
|
+
earl.generate(options)
|
68
96
|
end
|
data/lib/earl_report.rb
CHANGED
@@ -8,19 +8,36 @@ require 'haml'
|
|
8
8
|
# Instantiate a new class using one or more input graphs
|
9
9
|
class EarlReport
|
10
10
|
attr_reader :graph
|
11
|
+
|
12
|
+
MANIFEST_QUERY = %(
|
13
|
+
PREFIX dc: <http://purl.org/dc/terms/>
|
14
|
+
PREFIX mf: <http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#>
|
15
|
+
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
|
16
|
+
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
17
|
+
|
18
|
+
SELECT ?lh ?uri ?title ?description ?testAction ?testResult
|
19
|
+
WHERE {
|
20
|
+
?uri mf:name ?title; mf:action ?testAction.
|
21
|
+
OPTIONAL { ?uri rdfs:comment ?description. }
|
22
|
+
OPTIONAL { ?uri mf:result ?testResult. }
|
23
|
+
OPTIONAL { [ mf:entries ?lh] . ?lh rdf:first ?uri . }
|
24
|
+
}
|
25
|
+
).freeze
|
26
|
+
|
11
27
|
TEST_SUBJECT_QUERY = %(
|
12
28
|
PREFIX doap: <http://usefulinc.com/ns/doap#>
|
13
29
|
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
14
30
|
|
15
|
-
SELECT DISTINCT ?uri ?name ?
|
31
|
+
SELECT DISTINCT ?uri ?name ?doapDesc ?homepage ?language ?developer ?devName ?devType ?devHomepage
|
16
32
|
WHERE {
|
17
33
|
?uri a doap:Project; doap:name ?name .
|
18
34
|
OPTIONAL { ?uri doap:developer ?developer .}
|
19
35
|
OPTIONAL { ?uri doap:homepage ?homepage . }
|
20
|
-
OPTIONAL { ?uri doap:description ?
|
36
|
+
OPTIONAL { ?uri doap:description ?doapDesc . }
|
21
37
|
OPTIONAL { ?uri doap:programming-language ?language . }
|
22
|
-
OPTIONAL { ?developer foaf:name ?
|
23
|
-
OPTIONAL { ?developer a ?
|
38
|
+
OPTIONAL { ?developer foaf:name ?devName .}
|
39
|
+
OPTIONAL { ?developer a ?devType . }
|
40
|
+
OPTIONAL { ?developer foaf:homepage ?devHomepage . }
|
24
41
|
}
|
25
42
|
).freeze
|
26
43
|
|
@@ -35,7 +52,7 @@ class EarlReport
|
|
35
52
|
?subject a doap:Project; doap:name ?name
|
36
53
|
}
|
37
54
|
}
|
38
|
-
)
|
55
|
+
).freeze
|
39
56
|
|
40
57
|
ASSERTION_QUERY = %(
|
41
58
|
PREFIX earl: <http://www.w3.org/ns/earl#>
|
@@ -49,8 +66,39 @@ class EarlReport
|
|
49
66
|
earl:subject ?subject;
|
50
67
|
earl:test ?test ] .
|
51
68
|
}
|
69
|
+
ORDER BY ?subject
|
52
70
|
).freeze
|
53
71
|
|
72
|
+
TEST_CONTEXT = {
|
73
|
+
"@vocab" => "http://www.w3.org/ns/earl#",
|
74
|
+
"foaf:homepage" => {"@type" => "@id"},
|
75
|
+
dc: "http://purl.org/dc/terms/",
|
76
|
+
doap: "http://usefulinc.com/ns/doap#",
|
77
|
+
earl: "http://www.w3.org/ns/earl#",
|
78
|
+
mf: "http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#",
|
79
|
+
foaf: "http://xmlns.com/foaf/0.1/",
|
80
|
+
rdfs: "http://www.w3.org/2000/01/rdf-schema#",
|
81
|
+
assertedBy: {"@type" => "@id"},
|
82
|
+
assertions: {"@type" => "@id", "@container" => "@list"},
|
83
|
+
bibRef: {"@id" => "dc:bibliographicCitation"},
|
84
|
+
description: {"@id" => "dc:description"},
|
85
|
+
developer: {"@id" => "doap:developer", "@type" => "@id", "@container" => "@set"},
|
86
|
+
doapDesc: {"@id" => "doap:description"},
|
87
|
+
homepage: {"@id" => "doap:homepage", "@type" => "@id"},
|
88
|
+
label: {"@id" => "rdfs:label"},
|
89
|
+
language: {"@id" => "doap:programming-language"},
|
90
|
+
mode: {"@type" => "@id"},
|
91
|
+
name: {"@id" => "doap:name"},
|
92
|
+
outcome: {"@type" => "@id"},
|
93
|
+
subject: {"@type" => "@id"},
|
94
|
+
test: {"@type" => "@id"},
|
95
|
+
testAction: {"@id" => "mf:action", "@type" => "@id"},
|
96
|
+
testResult: {"@id" => "mf:result", "@type" => "@id"},
|
97
|
+
tests: {"@type" => "@id", "@container" => "@list"},
|
98
|
+
testSubjects: {"@type" => "@id", "@container" => "@list"},
|
99
|
+
title: {"@id" => "dc:title"}
|
100
|
+
}.freeze
|
101
|
+
|
54
102
|
# Convenience vocabularies
|
55
103
|
class EARL < RDF::Vocabulary("http://www.w3.org/ns/earl#"); end
|
56
104
|
class MF < RDF::Vocabulary("http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#"); end
|
@@ -60,18 +108,36 @@ class EarlReport
|
|
60
108
|
# @param [Array<String>] *files Assertions
|
61
109
|
# @param [Hash{Symbol => Object}] options
|
62
110
|
# @option options [Boolean] :verbose (true)
|
111
|
+
# @option options [String] :base Base IRI for loading Manifest
|
112
|
+
# @option options [String] :bibRef
|
113
|
+
# ReSpec bibliography reference for specification being tested
|
114
|
+
# @option options [String] :json Result of previous JSON-LD generation
|
115
|
+
# @option options [String] :manifest Test manifest
|
116
|
+
# @option options [String] :name Name of specification
|
117
|
+
# @option options [String] :query
|
118
|
+
# Query, or file containing query for extracting information from Test manifest
|
63
119
|
def initialize(*files)
|
64
120
|
@options = files.last.is_a?(Hash) ? files.pop.dup : {}
|
65
|
-
@
|
121
|
+
@options[:query] ||= MANIFEST_QUERY
|
122
|
+
raise "Test Manifest must be specified with :manifest option" unless @options[:manifest] || @options[:json]
|
123
|
+
@files = files
|
66
124
|
@prefixes = {}
|
125
|
+
if @options[:json]
|
126
|
+
@json_hash = ::JSON.parse(File.read(files.first))
|
127
|
+
return
|
128
|
+
end
|
129
|
+
|
130
|
+
# Load manifest, possibly with base URI
|
131
|
+
status "read #{@options[:manifest]}"
|
132
|
+
man_opts = {}
|
133
|
+
man_opts[:base_uri] = RDF::URI(@options[:base]) if @options[:base]
|
134
|
+
@graph = RDF::Graph.load(@options[:manifest], man_opts)
|
135
|
+
status " loaded #{@graph.count} triples"
|
136
|
+
|
137
|
+
# Read test assertion files
|
67
138
|
files.flatten.each do |file|
|
68
139
|
status "read #{file}"
|
69
|
-
file_graph =
|
70
|
-
when /\.jsonld/
|
71
|
-
@json_hash = ::JSON.parse(File.read(file))
|
72
|
-
return
|
73
|
-
else RDF::Graph.load(file)
|
74
|
-
end
|
140
|
+
file_graph = RDF::Graph.load(file)
|
75
141
|
status " loaded #{file_graph.count} triples"
|
76
142
|
@graph << file_graph
|
77
143
|
end
|
@@ -96,7 +162,7 @@ class EarlReport
|
|
96
162
|
# Load developers referenced from Test Subjects
|
97
163
|
SPARQL.execute(TEST_SUBJECT_QUERY, @graph).each do |solution|
|
98
164
|
# Load DOAP definitions
|
99
|
-
if solution[:developer] && !solution[:
|
165
|
+
if solution[:developer] && !solution[:devName] # not loaded
|
100
166
|
status "read description for #{solution[:developer].inspect}"
|
101
167
|
begin
|
102
168
|
foaf_graph = RDF::Graph.load(solution[:developer])
|
@@ -116,37 +182,28 @@ class EarlReport
|
|
116
182
|
#
|
117
183
|
# @param [Hash{Symbol => Object}] options
|
118
184
|
# @option options [Symbol] format (:html)
|
119
|
-
# @option options [String] :bibRef
|
120
|
-
# ReSpec bibliography reference for specification being tested
|
121
|
-
# @option options [Array<String>] :source_files
|
122
|
-
# Used for referencing the files used to generate this report
|
123
185
|
# @option options[IO] :io
|
124
186
|
# Optional `IO` to output results
|
125
187
|
# @return [String] serialized graph, if `io` is nil
|
126
188
|
def generate(options = {})
|
127
|
-
options = {
|
128
|
-
format: :html,
|
129
|
-
bibRef: "[[TURTLE]]",
|
130
|
-
name: "Turtle Test Results",
|
131
|
-
}.merge(options)
|
189
|
+
options = {:format => :html}.merge(options)
|
132
190
|
|
133
191
|
io = options[:io]
|
134
192
|
|
135
193
|
status("generate: #{options[:format]}")
|
136
194
|
##
|
137
195
|
# Retrieve Hashed information in JSON-LD format
|
138
|
-
hash = json_hash(options)
|
139
196
|
case options[:format]
|
140
197
|
when :jsonld, :json
|
141
|
-
json =
|
198
|
+
json = json_hash.to_json(JSON::LD::JSON_STATE)
|
142
199
|
io.write(json) if io
|
143
200
|
json
|
144
201
|
when :turtle, :ttl
|
145
202
|
if io
|
146
|
-
earl_turtle(options
|
203
|
+
earl_turtle(options)
|
147
204
|
else
|
148
205
|
io = StringIO.new
|
149
|
-
earl_turtle(:
|
206
|
+
earl_turtle(options.merge(:io => io))
|
150
207
|
io.rewind
|
151
208
|
io.read
|
152
209
|
end
|
@@ -155,8 +212,7 @@ class EarlReport
|
|
155
212
|
File.read(File.expand_path('../earl_report/views/earl_report.html.haml', __FILE__))
|
156
213
|
|
157
214
|
# Generate HTML report
|
158
|
-
html = Haml::Engine.new(template, :format => :xhtml)
|
159
|
-
.render(self, :tests => hash, :source_files => options.fetch(:source_files, []))
|
215
|
+
html = Haml::Engine.new(template, :format => :xhtml).render(self, :tests => json_hash)
|
160
216
|
io.write(html) if io
|
161
217
|
html
|
162
218
|
else
|
@@ -173,39 +229,16 @@ class EarlReport
|
|
173
229
|
##
|
174
230
|
# Return hashed EARL report in JSON-LD form
|
175
231
|
# @return [Hash]
|
176
|
-
def json_hash
|
232
|
+
def json_hash
|
177
233
|
@json_hash ||= begin
|
178
234
|
# Customized JSON-LD output
|
179
235
|
{
|
180
|
-
"@context" =>
|
181
|
-
dc: "http://purl.org/dc/terms/",
|
182
|
-
doap: "http://usefulinc.com/ns/doap#",
|
183
|
-
earl: "http://www.w3.org/ns/earl#",
|
184
|
-
mf: "http://www.w3.org/2001/sw/DataAccess/tests/test-manifest#",
|
185
|
-
foaf: "http://xmlns.com/foaf/0.1/",
|
186
|
-
rdfs: "http://www.w3.org/2000/01/rdf-schema#",
|
187
|
-
assertedBy: {"@id" => "earl:assertedBy", "@type" => "@id"},
|
188
|
-
bibRef: {"@id" => "dc: bibliographicCitation"},
|
189
|
-
description: {"@id" => "dc:description"},
|
190
|
-
developer: {"@id" => "doap:developer", "@type" => "@id", "@container" => "@set"},
|
191
|
-
homepage: {"@id" => "doap:homepage", "@type" => "@id"},
|
192
|
-
doap_desc: {"@id" => "doap:description"},
|
193
|
-
language: {"@id" => "doap:programming-language"},
|
194
|
-
testAction: {"@id" => "mf:action", "@type" => "@id"},
|
195
|
-
testResult: {"@id" => "mf:result", "@type" => "@id"},
|
196
|
-
label: {"@id" => "rdfs:label"},
|
197
|
-
mode: {"@id" => "earl:mode", "@type" => "@id"},
|
198
|
-
name: {"@id" => "doap:name"},
|
199
|
-
outcome: {"@id" => "earl:outcome", "@type" => "@id"},
|
200
|
-
result: {"@id" => "earl:result"},
|
201
|
-
subject: {"@id" => "earl:subject", "@type" => "@id"},
|
202
|
-
test: {"@id" => "earl:test", "@type" => "@id"},
|
203
|
-
title: {"@id" => "dc:title"}
|
204
|
-
},
|
236
|
+
"@context" => TEST_CONTEXT,
|
205
237
|
"@id" => "",
|
206
238
|
"@type" => %w(earl:Software doap:Project),
|
207
|
-
|
208
|
-
'
|
239
|
+
"assertions" => @files,
|
240
|
+
'name' => @options[:name],
|
241
|
+
'bibRef' => @options[:bibRef],
|
209
242
|
'testSubjects' => json_test_subject_info,
|
210
243
|
'tests' => json_result_info
|
211
244
|
}
|
@@ -217,69 +250,87 @@ class EarlReport
|
|
217
250
|
# @return [Array]
|
218
251
|
def json_test_subject_info
|
219
252
|
# Get the set of subjects
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
226
|
-
|
227
|
-
|
228
|
-
|
229
|
-
|
230
|
-
|
231
|
-
|
232
|
-
|
253
|
+
@subject_info ||= begin
|
254
|
+
ts_info = {}
|
255
|
+
SPARQL.execute(TEST_SUBJECT_QUERY, @graph).each do |solution|
|
256
|
+
status "solution #{solution.to_hash.inspect}"
|
257
|
+
info = ts_info[solution[:uri].to_s] ||= {}
|
258
|
+
%w(name doapDesc homepage language).each do |prop|
|
259
|
+
info[prop] = solution[prop.to_sym].to_s if solution[prop.to_sym]
|
260
|
+
end
|
261
|
+
if solution[:devName]
|
262
|
+
dev_type = solution[:devType].to_s =~ /Organization/ ? "foaf:Organization" : "foaf:Person"
|
263
|
+
dev = {'@type' => dev_type}
|
264
|
+
dev['@id'] = solution[:developer].to_s if solution[:developer].uri?
|
265
|
+
dev['foaf:name'] = solution[:devName].to_s if solution[:devName]
|
266
|
+
dev['foaf:homepage'] = solution[:devHomepage].to_s if solution[:devHomepage]
|
267
|
+
(info['developer'] ||= []) << dev
|
268
|
+
end
|
269
|
+
info['developer'] = info['developer'].uniq
|
233
270
|
end
|
234
|
-
info['developer'] = info['developer'].uniq
|
235
|
-
end
|
236
271
|
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
272
|
+
# Map ids and values to array entries
|
273
|
+
ts_info.keys.sort.map do |id|
|
274
|
+
info = ts_info[id]
|
275
|
+
subject = Hash.ordered
|
276
|
+
subject["@id"] = id
|
277
|
+
subject["@type"] = %w(earl:TestSubject doap:Project)
|
278
|
+
%w(name developer doapDesc homepage language).each do |prop|
|
279
|
+
subject[prop] = info[prop] if info[prop]
|
280
|
+
end
|
281
|
+
subject
|
245
282
|
end
|
246
|
-
subject
|
247
283
|
end
|
248
284
|
end
|
249
|
-
|
285
|
+
|
250
286
|
##
|
251
|
-
# Return result information for each test
|
287
|
+
# Return result information for each test.
|
288
|
+
# This counts on hash maintaining insertion order
|
252
289
|
#
|
253
290
|
# @return [Array]
|
254
291
|
def json_result_info
|
255
292
|
test_cases = {}
|
293
|
+
subjects = json_test_subject_info.map {|s| s['@id']}
|
256
294
|
|
257
|
-
|
258
|
-
|
259
|
-
|
260
|
-
|
261
|
-
|
262
|
-
|
263
|
-
|
264
|
-
|
265
|
-
|
266
|
-
|
267
|
-
|
268
|
-
when MF['name'].to_s
|
269
|
-
tc_hash['title'] = tc_stmt.object.to_s
|
270
|
-
when RDF::RDFS.comment.to_s
|
271
|
-
tc_hash['description'] = tc_stmt.object.to_s
|
272
|
-
when MF.action.to_s
|
273
|
-
tc_hash['testAction'] = tc_stmt.object.to_s
|
274
|
-
when MF.result.to_s
|
275
|
-
tc_hash['testResult'] = tc_stmt.object.to_s
|
276
|
-
else
|
277
|
-
#STDERR.puts "TC soln: #{tc_stmt.inspect}"
|
278
|
-
end
|
279
|
-
end
|
295
|
+
# Hash test cases by URI
|
296
|
+
solutions = SPARQL.execute(@options[:query], @graph)
|
297
|
+
.to_a
|
298
|
+
.inject({}) {|memo, soln| memo[soln[:uri]] = soln; memo}
|
299
|
+
|
300
|
+
# If test cases are in a list, maintain order
|
301
|
+
solution_list = if first_soln = solutions.values.detect {|s| s[:lh]}
|
302
|
+
RDF::List.new(first_soln[:lh], @graph)
|
303
|
+
else
|
304
|
+
solutions.keys # Any order will do
|
305
|
+
end
|
280
306
|
|
281
|
-
|
307
|
+
# Collect each TestCase
|
308
|
+
solution_list.each do |uri|
|
309
|
+
solution = solutions[uri]
|
310
|
+
tc_hash = {
|
311
|
+
'@id' => uri.to_s,
|
312
|
+
'@type' => %w(earl:TestCriterion earl:TestCase),
|
313
|
+
'title' => solution[:title].to_s,
|
314
|
+
'testAction' => solution[:testAction].to_s,
|
315
|
+
'assertions' => []
|
316
|
+
}
|
317
|
+
tc_hash['description'] = solution[:description].to_s if solution[:description]
|
318
|
+
tc_hash['testResult'] = solution[:testResult].to_s if solution[:testResult]
|
319
|
+
|
320
|
+
# Pre-initialize results for each subject to untested
|
321
|
+
subjects.each do |siri|
|
322
|
+
tc_hash['assertions'] << {
|
323
|
+
'@type' => 'earl:Assertion',
|
324
|
+
'test' => uri.to_s,
|
325
|
+
'subject' => siri,
|
326
|
+
'result' => {
|
327
|
+
'@type' => 'earl:TestResult',
|
328
|
+
'outcome' => 'earl:untested'
|
329
|
+
}
|
330
|
+
}
|
282
331
|
end
|
332
|
+
|
333
|
+
test_cases[uri.to_s] = tc_hash
|
283
334
|
end
|
284
335
|
|
285
336
|
raise "No test cases found" if test_cases.empty?
|
@@ -289,33 +340,23 @@ class EarlReport
|
|
289
340
|
SPARQL.execute(ASSERTION_QUERY, @graph).each do |solution|
|
290
341
|
tc = test_cases[solution[:test].to_s]
|
291
342
|
STDERR.puts "No test case found for #{solution[:test]}: #{tc.inspect}" unless tc
|
292
|
-
STDERR.puts "Must have outcome earl:passed or earl:failed: #{solution[:outcome].inspect}" unless
|
293
|
-
[EARL.passed, EARL.failed].include?(solution[:outcome])
|
294
|
-
tc ||= {}
|
295
343
|
subject = solution[:subject].to_s
|
296
|
-
|
297
|
-
ta_hash
|
344
|
+
result_index = subjects.index(subject)
|
345
|
+
ta_hash = tc['assertions'][result_index]
|
298
346
|
ta_hash['assertedBy'] = solution[:by].to_s
|
299
|
-
ta_hash['test'] = solution[:test].to_s
|
300
347
|
ta_hash['mode'] = "earl:#{solution[:mode].to_s.split('#').last || 'automatic'}"
|
301
|
-
ta_hash['
|
302
|
-
ta_hash['result'] = {
|
303
|
-
'@type' => 'earl:TestResult',
|
304
|
-
"outcome" => (solution[:outcome] == EARL.passed ? 'earl:passed' : 'earl:failed')
|
305
|
-
}
|
306
|
-
tc[subject] = ta_hash
|
348
|
+
ta_hash['result']['outcome'] = "earl:#{solution[:outcome].to_s.split('#').last}"
|
307
349
|
end
|
308
350
|
|
309
351
|
test_cases.values
|
310
352
|
end
|
311
|
-
|
353
|
+
|
312
354
|
##
|
313
355
|
# Output consoloated EARL report as Turtle
|
314
356
|
# @param [IO, StringIO] io
|
315
357
|
# @return [String]
|
316
358
|
def earl_turtle(options)
|
317
359
|
io = options[:io]
|
318
|
-
json_hash = options[:json_hash]
|
319
360
|
# Write preamble
|
320
361
|
{
|
321
362
|
:dc => RDF::DC,
|
@@ -334,24 +375,22 @@ class EarlReport
|
|
334
375
|
io.puts
|
335
376
|
|
336
377
|
# Write earl:Software for the report
|
337
|
-
io.puts %
|
338
|
-
io.puts %
|
339
|
-
io.puts %
|
378
|
+
io.puts %{<#{json_hash['@id']}> a earl:Software, doap:Project;}
|
379
|
+
io.puts %{ doap:homepage <#{json_hash['homepage']}>;}
|
380
|
+
io.puts %{ doap:name "#{json_hash['name']}";}
|
381
|
+
io.puts %{ dc:bibliographicCitation "#{json_hash['bibRef']}";}
|
382
|
+
io.puts %{ earl:assertions\n}
|
383
|
+
io.puts %{ } + json_hash['assertions'].map {|a| as_resource(a)}.join(",\n ") + ';'
|
384
|
+
io.puts %{ earl:testSubjects (\n}
|
385
|
+
io.puts %{ } + json_hash['testSubjects'].map {|a| as_resource(a['@id'])}.join("\n ") + ');'
|
386
|
+
io.puts %{ earl:tests (\n}
|
387
|
+
io.puts %{ } + json_hash['tests'].map {|a| as_resource(a['@id'])}.join("\n ") + ') .'
|
340
388
|
|
341
389
|
# Test Cases
|
342
390
|
# also collect each assertion definition
|
343
391
|
test_cases = {}
|
344
392
|
assertions = []
|
345
393
|
|
346
|
-
# Tests
|
347
|
-
json_hash['tests'].each do |test_case|
|
348
|
-
tc_desc = test_cases[test_case['test']] ||= test_case.dup
|
349
|
-
test_case.keys.select {|k| k =~ /^http:/}.each do |ts_uri|
|
350
|
-
tc_desc[ts_uri] = test_case[ts_uri]['@id']
|
351
|
-
assertions << test_case[ts_uri]
|
352
|
-
end
|
353
|
-
end
|
354
|
-
|
355
394
|
# Write out each earl:TestSubject
|
356
395
|
io.puts %(#\n# Subject Definitions\n#)
|
357
396
|
json_hash['testSubjects'].each do |ts_desc|
|
@@ -360,14 +399,8 @@ class EarlReport
|
|
360
399
|
|
361
400
|
# Write out each earl:TestCase
|
362
401
|
io.puts %(#\n# Test Case Definitions\n#)
|
363
|
-
|
364
|
-
io.write(tc_turtle(
|
365
|
-
end
|
366
|
-
|
367
|
-
# Write out each earl:Assertion
|
368
|
-
io.puts %(#\n# Assertions\n#)
|
369
|
-
assertions.sort_by {|a| a['@id']}.each do |as_desc|
|
370
|
-
io.write(as_turtle(as_desc))
|
402
|
+
json_hash['tests'].each do |test_case|
|
403
|
+
io.write(tc_turtle(test_case))
|
371
404
|
end
|
372
405
|
end
|
373
406
|
|
@@ -376,19 +409,24 @@ class EarlReport
|
|
376
409
|
# @param [Hash] desc
|
377
410
|
# @return [String]
|
378
411
|
def test_subject_turtle(desc)
|
379
|
-
developer = desc['developer']
|
380
412
|
res = %(<#{desc['@id']}> a #{desc['@type'].join(', ')};\n)
|
381
413
|
res += %( doap:name "#{desc['name']}";\n)
|
382
|
-
res += %( doap:description """#{desc['
|
414
|
+
res += %( doap:description """#{desc['doapDesc']}""";\n) if desc['doapDesc']
|
383
415
|
res += %( doap:programming-language "#{desc['language']}";\n) if desc['language']
|
384
|
-
|
385
|
-
|
386
|
-
|
387
|
-
|
388
|
-
|
389
|
-
|
390
|
-
|
391
|
-
|
416
|
+
res += %( .\n\n)
|
417
|
+
|
418
|
+
[desc['developer']].flatten.each do |developer|
|
419
|
+
if developer['@id']
|
420
|
+
res += %(<#{desc['@id']}> doap:developer <#{developer['@id']}> .\n\n)
|
421
|
+
res += %(<#{developer['@id']}> a #{[developer['@type']].flatten.join(', ')};\n)
|
422
|
+
res += %( foaf:homepage <#{developer['foaf:homepage']}>;\n) if developer['foaf:homepage']
|
423
|
+
res += %( foaf:name "#{developer['foaf:name']}" .\n\n)
|
424
|
+
else
|
425
|
+
res += %(<#{desc['@id']}> doap:developer\n)
|
426
|
+
res += %( [ a #{developer['@type'] || "foaf:Person"};\n)
|
427
|
+
res += %( foaf:homepage <#{developer['foaf:homepage']}>;\n) if developer['foaf:homepage']
|
428
|
+
res += %( foaf:name "#{developer['foaf:name']}" ] .\n\n)
|
429
|
+
end
|
392
430
|
end
|
393
431
|
res + "\n"
|
394
432
|
end
|
@@ -398,12 +436,16 @@ class EarlReport
|
|
398
436
|
# @prarm[Hash] desc
|
399
437
|
# @return [String]
|
400
438
|
def tc_turtle(desc)
|
401
|
-
res = %
|
402
|
-
res += %
|
403
|
-
res += %
|
404
|
-
res += %
|
405
|
-
res += %
|
406
|
-
res
|
439
|
+
res = %{#{as_resource desc['@id']} a #{[desc['@type']].flatten.join(', ')};\n}
|
440
|
+
res += %{ dc:title "#{desc['title']}";\n}
|
441
|
+
res += %{ dc:description """#{desc['description']}""";\n} if desc.has_key?('description')
|
442
|
+
res += %{ mf:result #{as_resource desc['testResult']};\n} if desc.has_key?('testResult')
|
443
|
+
res += %{ mf:action #{as_resource desc['testAction']};\n}
|
444
|
+
res += %{ earl:assertions (\n}
|
445
|
+
desc['assertions'].each do |as_desc|
|
446
|
+
res += as_turtle(as_desc)
|
447
|
+
end
|
448
|
+
res += %{ ) .\n\n}
|
407
449
|
end
|
408
450
|
|
409
451
|
##
|
@@ -411,16 +453,18 @@ class EarlReport
|
|
411
453
|
# @prarm[Hash] desc
|
412
454
|
# @return [String]
|
413
455
|
def as_turtle(desc)
|
414
|
-
res = %([ a earl:Assertion;\n)
|
415
|
-
res += %(
|
416
|
-
res += %(
|
417
|
-
res += %(
|
418
|
-
res += %(
|
419
|
-
res += %(
|
420
|
-
res += %(\n)
|
421
|
-
res
|
456
|
+
res = %( [ a earl:Assertion;\n)
|
457
|
+
res += %( earl:assertedBy #{as_resource desc['assertedBy']};\n) if desc['assertedBy']
|
458
|
+
res += %( earl:test #{as_resource desc['test']};\n)
|
459
|
+
res += %( earl:subject #{as_resource desc['subject']};\n)
|
460
|
+
res += %( earl:mode #{desc['mode']};\n) if desc['mode']
|
461
|
+
res += %( earl:result [ a earl:TestResult; earl:outcome #{desc['result']['outcome']} ]]\n)
|
422
462
|
end
|
423
463
|
|
464
|
+
def as_resource(resource)
|
465
|
+
resource[0,2] == '_:' ? resource : "<#{resource}>"
|
466
|
+
end
|
467
|
+
|
424
468
|
def status(message)
|
425
469
|
puts message if @options[:verbose]
|
426
470
|
end
|