tapout 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.ruby CHANGED
@@ -1,44 +1,44 @@
1
- ---
2
- spec_version: 1.0.0
3
- replaces: []
4
-
5
- loadpath:
6
- - lib
7
- name: tapout
8
- repositories:
9
- public: git://github.com/proutils/tapout.git
1
+ ---
2
+ authors:
3
+ - name: Thomas Sawyer
4
+ email: transfire@gmail.com
5
+ copyrights:
6
+ - holder: Thomas Sawyer
7
+ year: '2010'
8
+ license: BSD-2-Clause
9
+ replacements: []
10
10
  conflicts: []
11
-
12
- engine_check: []
13
-
14
- title: Tap Out
15
- contact: trans <transfire@gmail.com>
16
- resources:
17
- code: http://github.com/rubyworks/tapout
18
- home: http://rubyworks.github.com/tapout
19
- maintainers: []
20
-
21
- requires:
22
- - group: []
23
-
24
- name: ansi
25
- version: 0+
26
- - group:
27
- - build
28
- name: redline
29
- version: 0+
30
- - group:
11
+ requirements:
12
+ - name: ansi
13
+ - name: detroit
14
+ groups:
31
15
  - build
32
- name: qed
33
- version: 0+
34
- manifest: MANIFEST.txt
35
- version: 0.1.0
36
- licenses:
37
- - Apache 2.0
38
- copyright: Copyright (c) 2010 Thomas Sawyer
39
- authors:
40
- - Thomas Sawyer
41
- organization: RubyWorks
42
- description: Tap Out is a TAP consumer that can take any TAP, TAP-Y or TAP-J stream and output it in a variety of useful formats.
16
+ development: true
17
+ - name: qed
18
+ groups:
19
+ - test
20
+ development: true
21
+ dependencies: []
22
+ repositories:
23
+ - uri: git://github.com/rubyworks/tapout.git
24
+ scm: git
25
+ name: upstream
26
+ resources:
27
+ home: http://rubyworks.github.com/tapout
28
+ code: http://github.com/rubyworks/tapout
29
+ load_path:
30
+ - lib
31
+ extra: {}
32
+ date: '2011-10-06'
33
+ version: 0.2.0
34
+ alternatives: []
35
+ revision: 0
36
+ source:
37
+ - PROFILE
38
+ name: tapout
39
+ title: TapOut
43
40
  summary: Progressive TAP Harness
44
- created: 2010-12-23
41
+ description: Tapout is a TAP consumer that can take any TAP, TAP-Y or TAP-J stream
42
+ and output it in a variety of useful formats.
43
+ organization: RubyWorks
44
+ created: '2010-12-23'
data/.yardopts ADDED
@@ -0,0 +1,7 @@
1
+ --title "Tap Out"
2
+ --readme README.rdoc
3
+ --output-dir site/doc
4
+ --private
5
+ lib/**/*.rb -
6
+ [A-Z][A-Z]*
7
+
data/COPYING.rdoc ADDED
@@ -0,0 +1,54 @@
1
+ = COPYRIGHT NOTICES
2
+
3
+ == TAPOUT
4
+
5
+ Copyright:: (c) 2010 Thomas Sawyer, Rubyworks
6
+ License:: BSD-2-Clause
7
+ Website:: http://rubyworks.github.com/tapout
8
+
9
+ Copyright 2011 Thomas Sawyer. All rights reserved.
10
+
11
+ Redistribution and use in source and binary forms, with or without
12
+ modification, are permitted provided that the following conditions are met:
13
+
14
+ 1. Redistributions of source code must retain the above copyright notice,
15
+ this list of conditions and the following disclaimer.
16
+
17
+ 2. Redistributions in binary form must reproduce the above copyright
18
+ notice, this list of conditions and the following disclaimer in the
19
+ documentation and/or other materials provided with the distribution.
20
+
21
+ THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
22
+ INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
23
+ AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
24
+ COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
25
+ INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
26
+ NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
28
+ OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
29
+ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
30
+ EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31
+
32
+ == Detest
33
+
34
+ Copyright:: (c) 2009 Suraj N. Kurapati
35
+ License:: MIT
36
+ Website:: http://detest.org
37
+
38
+ A small segement of code for outputing error messages was sourced from the
39
+ Detest[http://snk.tuxfamily.org/lib/detest/] project.
40
+
41
+ Copyright 2009 Suraj N. Kurapati <sunaku@gmail.com>
42
+
43
+ Permission to use, copy, modify, and/or distribute this software for any
44
+ purpose with or without fee is hereby granted, provided that the above
45
+ copyright notice and this permission notice appear in all copies.
46
+
47
+ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
48
+ WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
49
+ MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
50
+ ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
51
+ WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
52
+ ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
53
+ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
54
+
data/HISTORY.rdoc CHANGED
@@ -1,9 +1,24 @@
1
1
  = RELEASE HISTORY
2
2
 
3
- == 1.0.0 / 2011-05-21
3
+ == 0.2.0 / 2011-10-06
4
4
 
5
- This is the initial release of Tap Out,
6
- a TAP and TAP-Y/J output formatter.
5
+ This realease revises the specification a bit, primarily
6
+ changing the names of header and footer sections to more
7
+ semantically usefule terms, suite and tally, respectively.
8
+ Also, a `rev` field has been added to the suite entries
9
+ to allow consuming apps to gracfully handle any future
10
+ changes.
11
+
12
+ Changes:
13
+
14
+ * Rename header and footer to suite and tally.
15
+ * Add rev field to suite entry.
16
+
17
+
18
+ == 0.1.0 / 2011-05-21
19
+
20
+ This is the first public release of TapOut, a TAP and TAP-Y/J output
21
+ formatter.
7
22
 
8
23
  Changes:
9
24
 
data/LICENSE.txt ADDED
@@ -0,0 +1,25 @@
1
+ BSD 2 Clause License
2
+
3
+ Copyright 2011 Thomas Sawyer. All rights reserved.
4
+
5
+ Redistribution and use in source and binary forms, with or without
6
+ modification, are permitted provided that the following conditions are met:
7
+
8
+ 1. Redistributions of source code must retain the above copyright notice,
9
+ this list of conditions and the following disclaimer.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright
12
+ notice, this list of conditions and the following disclaimer in the
13
+ documentation and/or other materials provided with the distribution.
14
+
15
+ THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
16
+ INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
17
+ FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
18
+ COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
19
+ INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
20
+ NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
21
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
22
+ OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
23
+ NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
24
+ EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
25
+
data/PROFILE ADDED
@@ -0,0 +1,33 @@
1
+ ---
2
+ name: tapout
3
+ version: 0.2.0
4
+
5
+ title: TapOut
6
+ summary: Progressive TAP Harness
7
+ description:
8
+ Tapout is a TAP consumer that can take any TAP, TAP-Y or
9
+ TAP-J stream and output it in a variety of useful formats.
10
+
11
+ requires:
12
+ - ansi
13
+ - detroit (build)
14
+ - qed (test)
15
+
16
+ resources:
17
+ home: http://rubyworks.github.com/tapout
18
+ code: http://github.com/rubyworks/tapout
19
+
20
+ repositories:
21
+ upstream: git://github.com/rubyworks/tapout.git
22
+
23
+ authors:
24
+ - Thomas Sawyer <transfire@gmail.com>
25
+
26
+ organization: RubyWorks
27
+
28
+ created: 2010-12-23
29
+
30
+ copyrights:
31
+ - Copyright (c) 2010 Thomas Sawyer (BSD-2-Clause)
32
+
33
+ source: PROFILE
data/README.rdoc CHANGED
@@ -1,4 +1,8 @@
1
- = Tap Out
1
+ = TapOut
2
+
3
+ Copyright:: 2010 Thomas Sawyer, Rubyworks
4
+ Author:: Thomas Sawyer
5
+ License:: BSD-2-Clause
2
6
 
3
7
 
4
8
  == RESOURCES
@@ -9,19 +13,14 @@
9
13
 
10
14
  == DESCRIPTION
11
15
 
12
- Tap Out is a TAP handler. Tap Out supports TAP-Y/J as well as traditional TAP.
13
-
14
- TAP-Y/J is an innovation of KO and Tap Out. It is a modernization of TAP using
15
- pure YAML/JSON streams.
16
+ TapOut is a TAP handler. TapOut supports TAP-Y/J as well as traditional TAP.
16
17
 
18
+ TAP-Y/J is a modernization of TAP using pure YAML/JSON streams.
17
19
  Traditional TAP has less detail than TAP-Y/J, but it can still be translated
18
- with fairly good results. Tap Out includes a TAP adapter to handle the
19
- translation transparently. Current Tap Out support TAP v12 with minor
20
+ with fairly good results. TapOut includes a TAP adapter to handle the
21
+ translation transparently. Current TapOut support TAP v12 with minor
20
22
  limitations.
21
23
 
22
- Tap Out fully supports TAP-Y at this time. TAP-J support is still in
23
- development.
24
-
25
24
  To learn about the TAP-Y/J specification, see the TAP-YJ.rdoc file.
26
25
 
27
26
  For information about TAP, see http://testanything.org/wiki/index.php/Main_Page.
@@ -32,17 +31,17 @@ For information about TAP, see http://testanything.org/wiki/index.php/Main_Page.
32
31
  Using a test framework that produces a TAP-Y output stream, simply pipe
33
32
  the stream into `tapout`.
34
33
 
35
- $ ko -y -Ilib test/foo.rb | tapout
34
+ $ ruby-test -y -Ilib test/foo.rb | tapout
36
35
 
37
- Tap Out supports a variety of output formats. These are selectable via the `-f`
36
+ TapOut supports a variety of output formats. These are selectable via the `-f`
38
37
  option.
39
38
 
40
- $ ko -y -Ilib test/foo.rb | tapout -f progessbar
39
+ $ ruby-test -y -Ilib test/foo.rb | tapout -f progessbar
41
40
 
42
- Tap Out is smart enough to match the closest matching format name. So, for
41
+ TapOut is smart enough to match the closest matching format name. So, for
43
42
  example, the above could be written as:
44
43
 
45
- $ ko -y -Ilib test/foo.rb | tapout -f pro
44
+ $ ruby-test -y -Ilib test/foo.rb | tapout -f pro
46
45
 
47
46
  And tapout will know to use the `progressbar` format.
48
47
 
@@ -51,23 +50,15 @@ To see a list of supported formats use the list subcommand:
51
50
  $ tapout list
52
51
 
53
52
  If your test framework does not support TAP-Y, but does support traditional
54
- TAP, Tap Out will automatically recognize the difference by TAP's `1..N` header.
55
-
56
- $ ko -t -Ilib test/foo.rb | tapout -f progressbar
53
+ TAP, TapOut will automatically recognize the difference by TAP's `1..N` header.
57
54
 
55
+ $ ruby-test -ftap -Ilib test/foo.rb | tapout -f progressbar
58
56
 
59
- == COPYRIGHT
60
57
 
61
- Tap Out, Copyright (c) 2010 Thomas Sawyer
58
+ == COPYRIGHTS
62
59
 
63
- Licensed under the Apache License, Version 2.0 (the "License");
64
- you may not use this program except in compliance with the License.
65
- You may obtain a copy of the License at
60
+ TapOut, Copyright (c) 2010 Thomas Sawyer
66
61
 
67
- http://www.apache.org/licenses/LICENSE-2.0
62
+ TapOut is licensed under the terms of the FreeBSD license.
68
63
 
69
- Unless required by applicable law or agreed to in writing, software
70
- distributed under the License is distributed on an "AS IS" BASIS,
71
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
72
- See the License for the specific language governing permissions and
73
- limitations under the License.
64
+ See COPYING.rdoc and LICENSE.txt for details.
data/TAP-YJ.md ADDED
@@ -0,0 +1,346 @@
1
+ # TAP-Y/J Format
2
+
3
+ TAP-Y and TAP-J are test streams. They are essentially the same except
4
+ for the underlying format used, which are YAML and JSON repsectively.
5
+
6
+ The following overview will use the YAML format. Becuase the YAML format
7
+ is a plan format, not using an special YAML tags, it is easy to convert
8
+ to JSON to get the equivalent TAP-J format. TAP-J documents follow all the
9
+ same feild rules as TAP-Y, but are represented as a stream of JSON
10
+ documents, one per line, instead of YAML documents.
11
+
12
+ ## Structure
13
+
14
+ TAP-Y is a plain YAML stream format. Only YAML core types are used:
15
+ scalar, sequence and mapping.
16
+
17
+ A YAML stream is composed a sequence of YAML *documents*, each divided by
18
+ a start document marker (<code>---</code>). Each document MUST have a `type`
19
+ field which designates it a `suite`, `case`, `unit`, `note` or `tally`. Any
20
+ document MAY have an `extra` entry which contains an open mapping for
21
+ extraneous information.
22
+
23
+ ### Suite
24
+
25
+ A `suite` document marks the beginning of a forthcoming stream of tests,
26
+ i.e. a <i>test suite</i>. All TAP-Y streams MUST begin with a suite
27
+ document.
28
+
29
+ ---
30
+ type: suite
31
+ start: 2011-10-10 12:12:32
32
+ count: 2
33
+ rev: 2
34
+
35
+ The `start` field marks the date and time testing began. It MUST be
36
+ an ISO-8601 formated timestamp.
37
+
38
+ The `count` field indicates the total number of test units forethcoming.
39
+ If the number of test units is unknown, the total can be omitted or
40
+ marked as `~` (nil). The total should only indicate the number of
41
+ <i>test units</i>, not any enumeration of <i>test cases</i>.
42
+
43
+ The `rev` field provides the version of the TAP-Y/J format that is
44
+ being used. The specification will change little, if at all, as it
45
+ become more mainstream. But just in case, having a revision field
46
+ ensures things will work if they do change by allowing consuming
47
+ apps to adjust to any future variation.
48
+
49
+ ### Case
50
+
51
+ The `case` type indicates the start of a test case.
52
+
53
+ ---
54
+ type: case
55
+ subtype: feature
56
+ label: Multiplication
57
+ level: 0
58
+
59
+ The case document MAY provide a `class` which is a label for the
60
+ typ of test case. For example, a test framwework that uses Gherkin
61
+ nomenclature would classify a test case as a "feature".
62
+
63
+ The case document SHOULD provide a `label` that is a free-form string
64
+ describing the nature of the test case.
65
+
66
+ The `level` field is used to notate sub-case heiararchies. By default the
67
+ value is assumed to be `0`, which means the case is not a subcase. If `1`
68
+ than the it indicates that the case is a subcase of the previous zero-level
69
+ case, and so on for higher levels. Subcases should proceed sequentially.
70
+ If a case contains both tests and subcases, the tests must come first in the
71
+ document stream.
72
+
73
+ ### Test
74
+
75
+ The `test` type indicates a test procedure. A unit MUST have a `status` with
76
+ one of five possible values: `pass`, `fail`, `error`, `omit` or `todo`.
77
+ Unit documents vary somewhat based on the status. But they all share
78
+ common fields.
79
+
80
+ Here is an example of a passing unit document.
81
+
82
+ ---
83
+ type: test
84
+ subtype: step
85
+ status: pass
86
+ setup: foo instance
87
+ label: multiples of two
88
+ expected: 2
89
+ returned: 2
90
+ file: test/test_foo.rb
91
+ line: 45
92
+ source: ok 1, 2
93
+ snippet:
94
+ - 44: ok 0,0
95
+ - 45: ok 1,2
96
+ - 46: ok 2,4
97
+ coverage:
98
+ file: lib/foo.rb
99
+ line: 11..13
100
+ code: Foo#*
101
+ time: 0.01
102
+
103
+ Besides the `status`, all test documents MUST have a `label`.
104
+
105
+ A test document MAY provide a `setup` field, which is used to describe
106
+ the setup for the unit test.
107
+
108
+ Tests SHOULD also give an `expected` and `returned` value, if relavent
109
+ to the nature of the test. For example, the most common test assertion is
110
+ equality, e.g. <code>assert_equal(4,3)</code>, so `expected` would be `3` and
111
+ `returned` woudl be `4`. Although desirable this can be a difficult piece
112
+ of information for a test framework to provide, so it is the most often
113
+ omitted.
114
+
115
+ A unit SHOULD also have a `file` and `line` number for source file location.
116
+ This is the location of the test definition itself.
117
+
118
+ A unit SHOULD provide the line of `source` code for the test.
119
+ This will be the line of code that `file` and `line` number references.
120
+ Unlike `snippet` lines, the source line should be stripped of whitespace.
121
+
122
+ The `snippet` is like `source` but provides surronding context. It MAY be
123
+ a verbatim string, in which case it MUST have an odd number of lines with
124
+ the source line in the center. Or, it MAY be an ordered map of verbatim
125
+ `- line: source`. Using an ordered map the line numbers may start
126
+ and end wherever, but they MUST be consecutive and the source line MUST
127
+ be among them.
128
+
129
+ [EXPERIMENTAL] The `coverage` subsection MAY be provided, in which can have three optional
130
+ fields: `file`, `line` and `code`. Where `file` specifies the source file being
131
+ targeted by the test, `line` specifies the line number, range of line
132
+ numbers (e.g. `1..4`) or an array of such, and `code` specifices the
133
+ language construct being targeted. For example, `code` might be `Foo#bar`
134
+ if the test targets the `bar` method of the `Foo` class.
135
+
136
+ The `time` is the number of seconds that have elapsed since the
137
+ the suite start time.
138
+
139
+ If a test has a status other than `pass` it MUST also provide a `exception`
140
+ subsection which is used to describe the nature of the failure, error or
141
+ omission.
142
+
143
+ ---
144
+ type: test
145
+ subtype: step
146
+ status: fail
147
+ label: multiples of two
148
+ setup: foo instance
149
+ expected: 2
150
+ returned: 1
151
+ file: test/test_foo.rb
152
+ line: 45
153
+ source: ok 1, 2
154
+ snippet:
155
+ - 44: ok 0,0
156
+ - 45: ok 1,2
157
+ - 46: ok 2,4
158
+ coverage:
159
+ file: lib/foo.rb
160
+ line: 11..13
161
+ code: Foo#*
162
+ exception:
163
+ message: |
164
+ (assertion fail) must_equal
165
+ 1
166
+ 2
167
+ file: test/test_foo.rb
168
+ line: 50
169
+ source: 1.must_equal == v
170
+ snippet:
171
+ - 49: v = 2
172
+ - 50: 1.must_equal == v
173
+ - 51: ''
174
+ backtrace:
175
+ - test/test_foo.rb:50
176
+ - test/test_foo.rb:45
177
+ time: 0.02
178
+
179
+ The `exception` section MUST give the `message`, describing the nature
180
+ of the failure or exception. In this subsection, `file` and `line` indicate
181
+ the location in code that triggered the exception or failed assertion.
182
+
183
+ Like the originating test code, a `source` and code `snippet` SHOULD also
184
+ be provided.
185
+
186
+ It MAY also provide a system `backtrace`.
187
+
188
+ <blockquote>
189
+ Q. Why supply a code snippet when the file and line are already given.
190
+ Can't a test reporter just look up the code itself?
191
+
192
+ A. Of course it can, but if the TAP-Y document is being consumed remotely
193
+ it might not have easy access the file being tested. While this may be of
194
+ rare use it none the less provides the TAP-Y consumer some view of the
195
+ code with having to do additional processing.
196
+ </blockquote>
197
+
198
+ ### Note
199
+
200
+ The `note` type is used to interject a message between tests that
201
+ is not tied to a specific unit or case. It has only a few fields.
202
+
203
+ ---
204
+ type: note
205
+ text:
206
+ This is an example note.
207
+
208
+ The note document is simply used to interject any information the
209
+ tester might want to know, but doesn't properly fit elsewhere in the
210
+ stream. A note cna appear any where in the document stream prior
211
+ to the tally.
212
+
213
+ ### Tally
214
+
215
+ While a running tally can technically occur anywhere in the document
216
+ stream without consequence, it generally incidates the end of a test
217
+ suite, which is strictly complete with the end-document-marker (`...`)
218
+ appears.
219
+
220
+ ---
221
+ type : tally
222
+ time : 0.03
223
+ counts:
224
+ total: 2
225
+ pass : 1
226
+ fail : 1
227
+ error: 0
228
+ omit : 0
229
+ todo : 0
230
+ ...
231
+
232
+ A tally document MUST provide a counts mapping with the `total` number of
233
+ tests (this MUST be same as `count` in the suite document if it was given)
234
+ and the totals for each test status. It SHOULD also give the time elapsed
235
+ since the suite time.
236
+
237
+ As mentioned, the test stream ends when a full ellipsis (<code>...</code>)
238
+ appears.
239
+
240
+ As you can see TAP-Y streams provides a great deal of detail. They are not
241
+ intended for the end-user, but rather to pipe to a consuming app to process
242
+ into a human readable form.
243
+
244
+
245
+ ## Glossery of Fields
246
+
247
+ ### count
248
+
249
+ The `count` field provides the total number of tests being executed. It SHOULD
250
+ be given in the header, if possible, and it MUST be given in the footer.
251
+
252
+ ### extra
253
+
254
+ Additional data, not specifucally designated by this sepecification can
255
+ placed within an `extra` section of any document without worry that future
256
+ versions of the specification will come into conflict with the field name.
257
+ The field MUST be a mapping. The key namespace is a free-for-all, so use it
258
+ with that in mind.
259
+
260
+ ### file
261
+
262
+ The `file` field provides the name of the file in which the test is defined,
263
+ or where th test failed/errored.
264
+
265
+ ### line
266
+
267
+ The `line` field provides the line number of the file on which the
268
+ definition of the test begins, or is the line number of where the
269
+ test failed/errored.
270
+
271
+ ### exception
272
+
273
+ A subsection used to sepcify the nature of a non-passing test.
274
+
275
+ ### message
276
+
277
+ For tests without a `pass` status, the message provides the explination
278
+ for the failure or error. Usually this is just the error message produced by
279
+ the underlying exception. The `pass` type can have the message field too,
280
+ but it will generally be ignored by TAP consumers.
281
+
282
+ ### snippet
283
+
284
+ The `snippet` field is either a verbatim string or an ordered mapping of line
285
+ number mapped to the source code for that line. While `snippet` is
286
+ like `source` it also contains extra lines of code before and after the
287
+ test `line` for context.
288
+
289
+ If `snippet` is a string it MUST consist an odd number of lines, the same
290
+ number before and after the source line in the center, unless the line occurs
291
+ at the begining or the end of the file. The number of lines before and after is
292
+ arbitrary and up to the producer, but should be the same on either side. Three
293
+ to five is generally enough.
294
+
295
+ ### source
296
+
297
+ The `source` field is a verbatim copy of the source code that defines the test.
298
+ This may be just the first line of the definition. In classic TAP this
299
+ is called `raw_test`.
300
+
301
+ ### start
302
+
303
+ The suite decument provides date/time information for when a suite of tests
304
+ began being tests. The filed MUSTbe in ISO standard format `YYYY-MM-DD HH:MM:SS`.
305
+
306
+ ### status
307
+
308
+ The `status` field designates the status of a test document. Valid values
309
+ are `pass`, `fail`, `error`, `omit` and `todo`.
310
+
311
+ In comparison to the classic TAP format, `pass` is equivalent to `ok` and
312
+ `fail` and `error` are akin to `not ok`, where `fail` is "not ok" with regards
313
+ to a test assertion and `error` is "not ok" becuase of a raised coding error.
314
+
315
+ Tests with an `omit` status do not need to be provided in the document stream,
316
+ so this status might not appear often in practice. But if a producer chooses to
317
+ do so this status simply means the test is purposefully being disregarded for
318
+ some reason. The `exception` subsection is used to clarify that reason.
319
+
320
+ On the other hand, `todo` means the test will be used in the future
321
+ but implementation has not been completed. It serves as reminder to developers
322
+ to write a missing test.
323
+
324
+ ### tally
325
+
326
+ The footer MUST provide a tally for all status categories. This is like `count`
327
+ but broken down into status groups.
328
+
329
+ ### time
330
+
331
+ The tests and the footer SHOULD have the `time` elapsed since starting the
332
+ tests given in number of seconds.
333
+
334
+ ### type
335
+
336
+ Each document MUST have a *type*. Valid types are `suite`, `tally`, `case`,
337
+ `test` and `note`.
338
+
339
+ The `suite` type can only occur once, at the start of the stream. All other
340
+ types may occur repeatedly in between, although the `tally` type will
341
+ generally only occur at the end of a stream.
342
+
343
+ The `case` type marks the start of a testcase. All `test` (and `note`)
344
+ documents following it are considered a part of the case until a new case
345
+ document occurs with the same level.
346
+