tomparse 0.3.0 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
data/.index CHANGED
@@ -64,5 +64,5 @@ description: ! 'TomParse provides no other functionality than to take a code com
64
64
  and parse it in to a convenient object-oriented structure in accordance
65
65
 
66
66
  with TomDoc standard.'
67
- version: 0.3.0
68
- date: '2013-01-21'
67
+ version: 0.4.0
68
+ date: '2013-02-11'
data/.yardopts ADDED
@@ -0,0 +1,8 @@
1
+ --title "TomParse"
2
+ --readme README.md
3
+ --plugin tomdoc
4
+ --private
5
+ lib/**/*.rb
6
+ -
7
+ *.md
8
+ *.txt
data/HISTORY.md CHANGED
@@ -1,15 +1,40 @@
1
1
  # Release History
2
2
 
3
+ ## 0.4.0 / 2013-02-10
4
+
5
+ For this release, the document parser has been substantially refactored.
6
+ It should be both more robust and more flexible. Arguments can now
7
+ have an optional `Arguments` heading, and the arguments list can be
8
+ indented. A new `Options` section is supported to handle Ruby 2.0 style
9
+ keyword options. Tag labels ony need to be capitalized. They no longer
10
+ have to be all-caps. And lastly, the `Signature` section has been
11
+ repurposed for describing variant *parameter signatures*, not dynamic
12
+ methods. As such `signature_fields` has been deprecated. Yes, these
13
+ are additions and some deviation from the original TomDoc spec, but
14
+ when purely conceptual specs go into practice they have a tendency to
15
+ adapt to practical requirements.
16
+
17
+ Changes:
18
+
19
+ * Refactor document parser.
20
+ * Add support for optional Arguments section header.
21
+ * Add support for Options section.
22
+ * Modify purpose of Signatures section.
23
+ * Deprecate signature field list.
24
+ * Tag labels only need to be capitialized, not all-caps.
25
+
26
+
3
27
  ## 0.3.0 / 2012-01-21
4
28
 
5
29
  This release fixes a bug which prevented descriptions from having
6
30
  multiple paragraphs. In addition it adds support for section tags.
7
- See the README for more information of tags.
31
+ See the README for more information on tags.
8
32
 
9
33
  Changes:
10
34
 
11
35
  * Fix multi-paragraph description parsing.
12
36
  * Add support for tags.
37
+ * Fix support for option hashes. (Jonas Oberschweiber)
13
38
 
14
39
 
15
40
  ## 0.2.1 / 2012-04-30
data/README.md CHANGED
@@ -13,7 +13,7 @@
13
13
 
14
14
  TomParse is a TomDoc parser for Ruby. It provides no other functionality than
15
15
  to take a code comment and parse it in to a convenient object-oriented
16
- structure in accordance with TomDoc standard. See [TomDoc](https://github.com/mojombo/tomdoc)
16
+ structure in accordance with the TomDoc standard. See [TomDoc](https://github.com/mojombo/tomdoc)
17
17
  for more information about the TomDoc format.
18
18
 
19
19
 
@@ -59,12 +59,59 @@ looks something like this:
59
59
  text * count
60
60
  end
61
61
 
62
- ### Extra Features
62
+ ## Extra Features
63
63
 
64
- Okay, we told a little white lie in the description. TomParse does take a tiny
65
- bit of liberty with the specification to offer up some additional documentation
66
- goodness. In particular, TomParse recoginizes paragraphs starting with an all-caps
67
- word, followed by a colon and a space, as special tags. Here is an example:
64
+ Okay, we told a little white lie in the description. TomParse does take some
65
+ liberties with the specification to offer up some additional documentation
66
+ goodness.
67
+
68
+ ### Arguments
69
+
70
+ The TomDoc specification is rather strict about how arguments are written --they
71
+ have to be the second section just after the description. TomParse offers a little
72
+ more flexability by allowing for an `Arguments` header to be used.
73
+
74
+ ```ruby
75
+ # Method to do fooey.
76
+ #
77
+ # Examples
78
+ # foo(name)
79
+ #
80
+ # Arguments
81
+ # name - Name of the fooey.
82
+ #
83
+ # Returns nothing.
84
+ def foo(name)
85
+ ...
86
+ ```
87
+
88
+ We still recommend putting the arguments right after the desciption in most cases,
89
+ but there are times when its reads better to put them lower, such when using a
90
+ `Signatures` section (see below).
91
+
92
+ ### Options
93
+
94
+ Ruby 2.0 finally introduces *keyword arguments* to the language. TomDoc's current
95
+ support of argument options, as a secondary argument list of a Hash argument,
96
+ doesn't take keyoword arguments into good account. To remedy this TomParse provides
97
+ and `Options` section, and it written just like one would an `Arguments` section.
98
+
99
+ ```ruby
100
+ # Method to do fooey.
101
+ #
102
+ # Options
103
+ # debug - Turn on debug mode? (default: false)
104
+ #
105
+ # Returns nothing.
106
+ def foo(**options)
107
+ ...
108
+ ```
109
+
110
+ ### Tags
111
+
112
+ One really nice new feature of TomParse is it's ability to recoginize sections
113
+ starting with a capitalized word followed by a colon and a space, as special *tags*.
114
+ Here is an example:
68
115
 
69
116
  ```ruby
70
117
  # Method to do something.
@@ -77,7 +124,44 @@ word, followed by a colon and a space, as special tags. Here is an example:
77
124
  ```
78
125
 
79
126
  When this is parsed, rather then lumping the TODO line in with the description,
80
- the TomDoc instance will have a `tags` property containing `{'todo'=>'This is a todo note.'}`.
127
+ the TomDoc instance will have a `tags` entry containing `['TODO', 'This is a todo note.']`.
128
+ It is important for consumer applications to recognize this. They can either just
129
+ add the tags back into the description when generating documentation, or handle
130
+ them separately. But tags don't have to occur right after the description. They
131
+ can occur any place in the documentation.
132
+
133
+ ### Signatures
134
+
135
+ TomParse does not support the Signature sections in exactly the same fashion as
136
+ the TomDoc specification describes. Rather then define dynamic methods, signatures
137
+ are used to specify alternate argument patterns, one signature per line.
138
+
139
+ ```ruby
140
+ # Method to do something.
141
+ #
142
+ # name - The name of the thing.
143
+ # pattern - The pattern of the thing.
144
+ #
145
+ # Signatures
146
+ #
147
+ # dosomething(name)
148
+ # dosomething(name=>pattern)
149
+ #
150
+ # Returns nothing.
151
+ def dosomething(*args)
152
+ ...
153
+ ```
154
+
155
+ Technically the Signatures section can still be used to designate a dynamic method,
156
+ but as you can see by the above example, TomParse does not support the *field* list.
157
+ If needed just use the arguments list for these as well.
158
+
159
+ This choice was made, btw, because support for signatures as defined by the spec,
160
+ leads to very non-optimal code. It requires scanning every chunk of documentation
161
+ for a `Signature` section in order to determine how to treat it. What is needed
162
+ is a more universal syntax, that can be easily recognized by some clear identifier
163
+ at the top of a comment --and one that doesn't confuse dynamic method definitions
164
+ with the more traditional concept of method signatures.
81
165
 
82
166
 
83
167
  ## Resources
@@ -107,6 +191,7 @@ the TomDoc instance will have a `tags` property containing `{'todo'=>'This is a
107
191
 
108
192
  * [Citron](http://rubyworks.github.com/citron) (testing)
109
193
  * [AE](http://rubyworks.github.com/ae) (testing)
194
+ * [Fire](http://detroit.github.com/fire) (building)
110
195
  * [Detroit](http://detroit.github.com/detroit) (building)
111
196
 
112
197
 
data/lib/tomparse.rb CHANGED
@@ -1,4 +1,8 @@
1
1
  module TomParse
2
+ require 'tomparse/parser'
3
+ require 'tomparse/parse_error'
4
+ require 'tomparse/argument'
5
+ require 'tomparse/option'
2
6
 
3
7
  # Main interface to parser.
4
8
  #
@@ -11,9 +15,10 @@ module TomParse
11
15
  # Encapsulate parsed tomdoc documentation.
12
16
  #
13
17
  # TODO: Currently uses lazy evaluation, eventually this should
14
- # be removed and simply parsed at initialization time.
18
+ # be removed and simply parsed all at once.
15
19
  #
16
20
  class TomDoc
21
+
17
22
  attr_accessor :raw
18
23
 
19
24
  # Public: Initialize a TomDoc object.
@@ -22,25 +27,15 @@ module TomParse
22
27
  #
23
28
  # Returns new TomDoc instance.
24
29
  def initialize(text, parse_options={})
25
- @raw = text.to_s.strip
26
-
27
- @arguments = []
28
- @options = [] # TODO
29
- @examples = []
30
- @returns = []
31
- @raises = []
32
- @signatures = []
33
- @signature_fields = []
34
- @tags = {}
35
-
36
- parse unless @raw.empty?
30
+ @parser = Parser.new(text, parse_options)
31
+ @parser.parse
37
32
  end
38
33
 
39
34
  # Raw documentation text.
40
35
  #
41
36
  # Returns String of raw documentation text.
42
37
  def to_s
43
- @raw
38
+ @parser.raw
44
39
  end
45
40
 
46
41
  # Validate given comment text.
@@ -54,9 +49,7 @@ module TomParse
54
49
  #
55
50
  # Returns true if comment is valid, otherwise false.
56
51
  def valid?
57
- return false if !raw.include?('Returns')
58
- return false if sections.size < 2
59
- true
52
+ @parser.valid?
60
53
  end
61
54
 
62
55
  # Validate raw comment.
@@ -64,553 +57,121 @@ module TomParse
64
57
  # Returns true if comment is valid.
65
58
  # Raises ParseError if comment is not valid.
66
59
  def validate
67
- if !raw.include?('Returns')
68
- raise ParseError.new("No `Returns' statement.")
69
- end
70
-
71
- if sections.size < 2
72
- raise ParseError.new("No description section found.")
73
- end
74
-
75
- true
60
+ @parser.validate
76
61
  end
77
62
 
63
+ # TODO: Should we clean the raw documentation here and then pass it on to the parser?
64
+
78
65
  # The raw comment text cleaned-up and ready for section parsing.
79
66
  #
80
67
  # Returns cleaned-up comment String.
81
68
  def tomdoc
82
- lines = raw.split("\n")
83
-
84
- # remove remark symbol
85
- if lines.all?{ |line| /^\s*#/ =~ line }
86
- lines = lines.map do |line|
87
- line =~ /^(\s*#)/ ? line.sub($1, '') : nil
88
- end
89
- end
90
-
91
- # for some reason the first line is coming in without indention
92
- # regardless, so we temporary remove it
93
- first = lines.shift
94
-
95
- # remove indention
96
- spaces = lines.map do |line|
97
- next if line.strip.empty?
98
- md = /^(\s*)/.match(line)
99
- md ? md[1].size : nil
100
- end.compact
101
-
102
- space = spaces.min || 0
103
- lines = lines.map do |line|
104
- if line.strip.empty?
105
- line.strip
106
- else
107
- line[space..-1]
108
- end
109
- end
110
-
111
- # put first line back
112
- lines.unshift(first.sub(/^\s*/,'')) if first
113
-
114
- lines.compact.join("\n")
69
+ return @parser.tomdoc
115
70
  end
116
71
 
117
72
  # List of comment sections. These are divided simply on "\n\n".
118
73
  #
119
74
  # Returns Array of comment sections.
120
75
  def sections
121
- parsed {
122
- @sections
123
- }
76
+ @parser.sections
124
77
  end
125
78
 
126
79
  # Description of method or class/module.
127
80
  #
128
81
  # Returns description String.
129
82
  def description
130
- parsed {
131
- @description
132
- }
83
+ @parser.description
133
84
  end
134
85
 
135
- # Description of method or class/module.
86
+ # Arguments list.
136
87
  #
137
- # Returns description String.
88
+ # Returns list of arguments.
138
89
  def arguments
139
- parsed {
140
- @arguments
141
- }
90
+ @parser.arguments
142
91
  end
143
92
  alias args arguments
144
93
 
94
+ # Keyword arguments, aka Options.
95
+ #
96
+ # Returns list of options.
97
+ def options
98
+ @parser.options
99
+ end
100
+ alias keyword_arguments options
101
+
145
102
  # List of use examples of a method or class/module.
146
103
  #
147
104
  # Returns String of examples.
148
105
  def examples
149
- parsed {
150
- @examples
151
- }
106
+ @parser.examples
152
107
  end
153
108
 
154
109
  # Description of a methods yield procedure.
155
110
  #
156
111
  # Returns String decription of yield procedure.
157
112
  def yields
158
- parsed {
159
- @yields
160
- }
113
+ @parser.yields
161
114
  end
162
115
 
163
116
  # The list of retrun values a method can return.
164
117
  #
165
118
  # Returns Array of method return descriptions.
166
119
  def returns
167
- parsed {
168
- @returns
169
- }
120
+ @parser.returns
170
121
  end
171
122
 
172
123
  # A list of errors a method might raise.
173
124
  #
174
125
  # Returns Array of method raises descriptions.
175
126
  def raises
176
- parsed {
177
- @raises
178
- }
127
+ @parser.raises
179
128
  end
180
129
 
181
130
  # A list of alternate method signatures.
182
131
  #
183
132
  # Returns Array of signatures.
184
133
  def signatures
185
- parsed {
186
- @signatures
187
- }
134
+ @parser.signatures
188
135
  end
189
136
 
190
- # A list of signature fields.
137
+ # Deprecated: A list of signature fields.
138
+ #
139
+ # TODO: Presently this will always return an empty list. It will either
140
+ # be removed or renamed in future version.
191
141
  #
192
142
  # Returns Array of field definitions.
193
143
  def signature_fields
194
- parsed {
195
- @signature_fields
196
- }
144
+ @parser.signature_fields
197
145
  end
198
146
 
199
- # A mapping of tags.
147
+ # List of tags.
200
148
  #
201
- # Returns Hash of tags.
149
+ # Returns an associatve array of tags. [Array<Array<String>>]
202
150
  def tags
203
- parsed {
204
- @tags
205
- }
151
+ @parser.tags
206
152
  end
207
153
 
208
154
  # Check if method is public.
209
155
  #
210
156
  # Returns true if method is public.
211
157
  def public?
212
- parsed {
213
- @status == 'Public'
214
- }
158
+ @parser.public?
215
159
  end
216
160
 
217
161
  # Check if method is internal.
218
162
  #
219
163
  # Returns true if method is internal.
220
164
  def internal?
221
- parsed {
222
- @status == 'Internal'
223
- }
165
+ @parser.internal?
224
166
  end
225
167
 
226
168
  # Check if method is deprecated.
227
169
  #
228
170
  # Returns true if method is deprecated.
229
171
  def deprecated?
230
- parsed {
231
- @status == 'Deprecated'
232
- }
172
+ @parser.deprecated?
233
173
  end
234
174
 
235
- private
236
-
237
- # Has the comment been parsed yet?
238
- def parsed(&block)
239
- parse unless @parsed
240
- block.call
241
- end
242
-
243
- # Internal: Parse the Tomdoc formatted comment.
244
- #
245
- # Returns true if there was a comment to parse.
246
- def parse
247
- @parsed = true
248
-
249
- sections = tomdoc.split("\n\n")
250
-
251
- return false if sections.empty?
252
-
253
- # The description is always the first section, but it may have
254
- # multiple paragraphs. This routine collects those together.
255
- desc = [sections.shift]
256
- loop do
257
- s = sections.first
258
- break if s.nil?
259
- break if s =~ /^\w+\s+\-/m
260
- break if section_type(s) != nil
261
- desc << sections.shift
262
- end
263
- sections = [desc.join("\n\n")] + sections
264
-
265
- @sections = sections.dup
266
-
267
- parse_description(sections.shift)
268
-
269
- if sections.first && sections.first =~ /^\w+\s+\-/m
270
- parse_arguments(sections.shift)
271
- end
272
-
273
- current = sections.shift
274
- while current
275
- case type = section_type(current)
276
- when :examples
277
- parse_examples(current, sections)
278
- when :yields
279
- parse_yields(current)
280
- when :returns
281
- parse_returns(current) # also does raises
282
- when :raises
283
- parse_returns(current) # also does returns
284
- when :signature
285
- parse_signature(current, sections)
286
- when Symbol
287
- parse_tag(current)
288
- end
289
- current = sections.shift
290
- end
291
-
292
- return @parsed
293
- end
294
-
295
- #
296
- #
297
- def section_type(section)
298
- case section
299
- when /^Examples/
300
- :examples
301
- when /^Yields/
302
- :yields
303
- when /^Returns/
304
- :returns
305
- when /^Raises/
306
- :raises
307
- when /^Signature/
308
- :signature
309
- when /^([A-Z]*)\:\ /
310
- $1.downcase.to_sym
311
- else
312
- nil
313
- end
314
- end
315
-
316
- # Recognized description status.
317
- TOMDOC_STATUS = ['Internal', 'Public', 'Deprecated']
318
-
319
- # Parse description.
320
- #
321
- # section - String containig description.
322
- #
323
- # Returns nothing.
324
- def parse_description(section)
325
- if md = /^([A-Z]\w+\:)/.match(section)
326
- @status = md[1].chomp(':')
327
- if TOMDOC_STATUS.include?(@status)
328
- @description = md.post_match.strip
329
- else
330
- @description = section.strip
331
- end
332
- else
333
- @description = section.strip
334
- end
335
- end
336
-
337
- # Parse arguments section. Arguments occur subsequent to
338
- # the description.
339
- #
340
- # section - String containing argument definitions.
341
- #
342
- # Returns nothing.
343
- def parse_arguments(section)
344
- args = []
345
- last_indent = nil
346
-
347
- section.lines.each do |line|
348
- next if line.strip.empty?
349
- indent = line.scan(/^\s*/)[0].to_s.size
350
-
351
- if last_indent && indent > 0 && indent >= last_indent
352
- args.last.description << "\r\n" + line
353
- else
354
- param, desc = line.split(" - ")
355
- args << Argument.new(param.strip, desc.strip) if param && desc
356
- end
357
-
358
- last_indent = indent
359
- end
360
-
361
- args.each do |arg|
362
- arg.parse(arg.description)
363
- end
364
-
365
- @arguments = args
366
- end
367
-
368
- # Parse examples.
369
- #
370
- # section - String starting with `Examples`.
371
- # sections - All sections subsequent to section.
372
- #
373
- # Returns nothing.
374
- def parse_examples(section, sections)
375
- examples = []
376
-
377
- section = section.sub('Examples', '').gsub(/^\s{2}/,'')
378
-
379
- examples << section unless section.empty?
380
- while sections.first && sections.first !~ /^\S/
381
- examples << sections.shift.gsub(/^\s{2}/,'')
382
- end
383
-
384
- @examples = examples
385
- end
386
-
387
- # Parse yields section.
388
- #
389
- # section - String contaning Yields line.
390
- #
391
- # Returns nothing.
392
- def parse_yields(section)
393
- @yields = section.strip
394
- end
395
-
396
- # Parse returns section.
397
- #
398
- # section - String contaning Returns and/or Raises lines.
399
- #
400
- # Returns nothing.
401
- def parse_returns(section)
402
- returns, raises, current = [], [], []
403
-
404
- lines = section.split("\n")
405
- lines.each do |line|
406
- case line
407
- when /^Returns/
408
- returns << line
409
- current = returns
410
- when /^Raises/
411
- raises << line
412
- current = raises
413
- when /^\s+/
414
- current.last << line.squeeze(' ')
415
- else
416
- current << line # TODO: What to do with non-compliant line?
417
- end
418
- end
419
-
420
- @returns.concat(returns)
421
- @raises.concat(raises)
422
- end
423
-
424
- # Parse signature section.
425
- #
426
- # section - String starting with `Signature`.
427
- # sections - All sections subsequent to section.
428
- #
429
- # Returns nothing.
430
- def parse_signature(section, sections=[])
431
- signatures = []
432
-
433
- section = section.sub('Signature', '').strip
434
-
435
- signatures << section unless section.empty?
436
-
437
- while sections.first && sections.first !~ /^\S/
438
- sigs = sections.shift
439
- sigs.split("\n").each do |s|
440
- signatures << s.strip
441
- end
442
- end
443
-
444
- @signatures = signatures
445
-
446
- if sections.first && sections.first =~ /^\w+\s*\-/m
447
- parse_signature_fields(sections.shift)
448
- end
449
- end
450
-
451
- # Subsequent to Signature section there can be field
452
- # definitions.
453
- #
454
- # section - String subsequent to signatures.
455
- #
456
- # Returns nothing.
457
- def parse_signature_fields(section)
458
- args = []
459
- last_indent = nil
460
-
461
- section.split("\n").each do |line|
462
- next if line.strip.empty?
463
- indent = line.scan(/^\s*/)[0].to_s.size
464
-
465
- if last_indent && indent > last_indent
466
- args.last.description << line.squeeze(" ")
467
- else
468
- param, desc = line.split(" - ")
469
- args << Argument.new(param.strip, desc.strip) if param && desc
470
- end
471
-
472
- last_indent = indent
473
- end
474
-
475
- @signature_fields = args
476
- end
477
-
478
- # Tags are arbitrary sections designated by all cap labels and a colon.
479
- #
480
- # label - String name of the tag.
481
- # section - String of the tag section.
482
- #
483
- # Returns nothing.
484
- def parse_tag(section)
485
-
486
- md = /^([A-Z]*)\:\ /.match(section)
487
-
488
- label = md[1]
489
- text = md.post_match
490
-
491
- @tags[label.downcase] = text
492
- end
493
-
494
- end
495
-
496
- # Encapsulate a method argument.
497
- #
498
- class Argument
499
-
500
- attr_accessor :name
501
-
502
- attr_accessor :description
503
-
504
- attr_accessor :options
505
-
506
- # Create new Argument object.
507
- #
508
- # name - name of argument
509
- # description - argument description
510
- #
511
- def initialize(name, description = '')
512
- @name = name.to_s.intern
513
- parse(description)
514
- end
515
-
516
- # Is this an optional argument?
517
- #
518
- # Returns Boolean.
519
- def optional?
520
- @description.downcase.include? 'optional'
521
- end
522
-
523
- # Parse arguments section. Arguments occur subsequent to
524
- # the description.
525
- #
526
- # section - String containing argument definitions.
527
- #
528
- # Returns nothing.
529
- def parse(description)
530
- desc = []
531
- opts = []
532
-
533
- lines = description.lines.to_a
534
-
535
- until lines.empty? or /^\s+\:(\w+)\s+-\s+(.*?)$/ =~ lines.first
536
- desc << lines.shift.chomp.squeeze(" ")
537
- end
538
-
539
- opts = []
540
- last_indent = nil
541
-
542
- lines.each do |line|
543
- next if line.strip.empty?
544
- indent = line.scan(/^\s*/)[0].to_s.size
545
-
546
- if last_indent && indent > last_indent
547
- opts.last.description << line.squeeze(" ")
548
- else
549
- param, d = line.split(" - ")
550
- opts << Option.new(param.strip, d.strip) if param && d
551
- end
552
-
553
- last_indent = indent
554
- end
555
-
556
- @description = desc.join
557
- @options = opts
558
- end
559
-
560
- end
561
-
562
- # Encapsulate a named parameter.
563
- #
564
- class Option
565
-
566
- attr_accessor :name
567
-
568
- attr_accessor :description
569
-
570
- # Create new Argument object.
571
- #
572
- # name - name of option
573
- # description - option description
574
- #
575
- def initialize(name, description = '')
576
- @name = name.to_s.intern
577
- @description = description
578
- end
579
-
580
- # Is this a required option?
581
- #
582
- # Returns Boolean.
583
- def required?
584
- @description.downcase.include? 'required'
585
- end
586
-
587
- end
588
-
589
- # Raised when comment can't be parsed, which means it's most
590
- # likely not valid TomDoc.
591
- #
592
- class ParseError < RuntimeError
593
- # Create new ParseError object.
594
- #
595
- # doc - document string
596
- #
597
- def initialize(doc)
598
- @doc = doc
599
- end
600
-
601
- # Provide access to document string.
602
- #
603
- # Returns String.
604
- def message
605
- @doc
606
- end
607
-
608
- # Provide access to document string.
609
- #
610
- # Returns String.
611
- def to_s
612
- @doc
613
- end
614
175
  end
615
176
 
616
177
  end