lmt 0.1.3 → 0.1.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +4 -4
- data/doc/lmt/{error_reporting.md → include/error_reporting.md} +0 -0
- data/doc/lmt/{lmt_expressions.md → include/lmt_expressions.md} +10 -0
- data/doc/lmt/{lmt_include.md → include/lmt_include.md} +0 -0
- data/doc/lmt/include/lmt_include_path.md +28 -0
- data/doc/lmt/{option_verification.md → include/option_verification.md} +0 -0
- data/doc/lmt/lmt.rb.md +34 -9
- data/doc/lmt/lmw.rb.md +32 -22
- data/lib/lmt/lmt.rb +30 -5
- data/lib/lmt/lmw.rb +45 -21
- data/lib/lmt/version.rb +1 -1
- data/src/lmt/{error_reporting.lmd → include/error_reporting.lmd} +0 -0
- data/src/lmt/{lmt_expressions.lmd → include/lmt_expressions.lmd} +8 -0
- data/src/lmt/{lmt_include.lmd → include/lmt_include.lmd} +0 -0
- data/src/lmt/include/lmt_include_path.lmd +24 -0
- data/src/lmt/{option_verification.lmd → include/option_verification.lmd} +0 -0
- data/src/lmt/lmt.rb.lmd +31 -6
- data/src/lmt/lmw.rb.lmd +29 -17
- metadata +11 -9
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f420500a15df99a0e4ff1a102d69775f9ae1a12397480c4f49ba25953996596b
|
4
|
+
data.tar.gz: 9116d831ae2234dbda149f308b0c69950cb4f547e2bff43188493d965610a676
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 37820ae837377fad18ea955dbdae4eac3c47e964da15fade0171573011761d84bf70b147e3eaf0a897fd957d12c316ea1c01a6bdf7302339b63c2651e28da703
|
7
|
+
data.tar.gz: 9aa7d8d3ab733f9263d0c116930d1e979c2b664002dde544de0e3a36bc4b3841f67d5897d09e8c9fbd7bc90efd52caa2f3cbb2b96ed89b9fb8b4224d21d5f8fc
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -29,13 +29,13 @@ Or install it yourself as:
|
|
29
29
|
The tangle program takes input files and produces tangled output files. It is used as follows:
|
30
30
|
|
31
31
|
``` bash
|
32
|
-
lmt --file {input file} --output {tangled destination}
|
32
|
+
lmt --file {input file} --output {tangled destination} --include-path {comma separated list of include paths}
|
33
33
|
```
|
34
34
|
|
35
35
|
The weave program is similar but produces weaved output files. It does not recurse down include statements, and so will need to be run independently for each included file. An example usage:
|
36
36
|
|
37
37
|
``` bash
|
38
|
-
lmw --file {input file} --output [weaved destination]
|
38
|
+
lmw --file {input file} --output [weaved destination] --include-path {comma separated list of include paths}
|
39
39
|
```
|
40
40
|
|
41
41
|
## Development
|
@@ -53,10 +53,10 @@ bundle exec ruby bin/lmt --file src/lmt/lmt.rb.lmd --output lib/lmt/lmt.rb
|
|
53
53
|
To test the weave you can use the following command which will weave the weaver and write it to the doc directory.
|
54
54
|
|
55
55
|
``` bash
|
56
|
-
bundle exec ruby bin/lmt --file src/lmt/
|
56
|
+
bundle exec ruby bin/lmt --file src/lmt/lmw.rb.lmd --output lib/lmt/lmw.rb; bundle exec ruby bin/lmw --file src/lmt/lmw.rb.lmd --output doc/lmt/lmw.rb.md
|
57
57
|
```
|
58
58
|
|
59
|
-
Since this is a self-bootstraping program, it is both tested and built by running itself on itself. This means that if you add a bug, it won't run. To fix this, check out the most recent version of the output file out of git.
|
59
|
+
Since this is a self-bootstraping program, it is both tested and built by running itself on itself. This means that if you add a bug, it won't run. To fix this, check out the most recent version of the output file out of git. To avoid lost work, it is recommended you commit especially often.
|
60
60
|
|
61
61
|
``` bash
|
62
62
|
git co -- src/lmt/lmt.rb
|
File without changes
|
@@ -12,6 +12,16 @@ The first regular expression handles the detection of include directives. It re
|
|
12
12
|
/^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
13
13
|
```
|
14
14
|
|
15
|
+
## The Include Path Expression
|
16
|
+
|
17
|
+
This regular expression handles the detection of include-path directives. It recognizes lines like `! include-path some-path` and extracts `some-path`.
|
18
|
+
|
19
|
+
###### Code Block: Include Path Expression
|
20
|
+
|
21
|
+
``` ruby
|
22
|
+
/^!\s+include-path\s+(.*)\s*$/
|
23
|
+
```
|
24
|
+
|
15
25
|
## The Block Expressions
|
16
26
|
|
17
27
|
These regular expression recognize if, elseif, else, and end directives. The first group contains the conditional for blocks that have a conditional..
|
File without changes
|
@@ -0,0 +1,28 @@
|
|
1
|
+
# Include Path
|
2
|
+
|
3
|
+
This is the basic algorithm to look up an include on the include path. If the file exists relative to the current file, it includes it. If, not, it goes through the include path looking for the file. When it finds it, it also calculates a relative path for the file from the current file.
|
4
|
+
|
5
|
+
###### Code Block: Includes
|
6
|
+
|
7
|
+
``` ruby
|
8
|
+
require 'pathname'
|
9
|
+
```
|
10
|
+
|
11
|
+
###### Code Block: Resolve Include
|
12
|
+
|
13
|
+
``` ruby
|
14
|
+
def resolve_include(file, current_file)
|
15
|
+
include_file_loc = File.join(File.dirname(current_file), file)
|
16
|
+
if File.exist?(include_file_loc)
|
17
|
+
return [include_file_loc, file]
|
18
|
+
end
|
19
|
+
@include_path.each do |include_dir|
|
20
|
+
include_file_loc = File.join(include_dir, file)
|
21
|
+
if File.exist? (include_file_loc)
|
22
|
+
relative_path = Pathname.new(include_file_loc).relative_path_from(File.dirname(current_file)).to_s
|
23
|
+
return [include_file_loc, relative_path]
|
24
|
+
end
|
25
|
+
end
|
26
|
+
throw "include file: #{file} not found from #{current_file} or in #{@include_path}"
|
27
|
+
end
|
28
|
+
```
|
File without changes
|
data/doc/lmt/lmt.rb.md
CHANGED
@@ -156,7 +156,7 @@ During tangle the link line will be replaced with the lines from the included fi
|
|
156
156
|
included_string = "I am in lmt.lmd"
|
157
157
|
```
|
158
158
|
|
159
|
-
**See include:** [lmt_include.
|
159
|
+
**See include:** [included file](include/lmt_include.md)
|
160
160
|
|
161
161
|
### Extension
|
162
162
|
|
@@ -173,6 +173,14 @@ def parse_hook(main_block, blocks)
|
|
173
173
|
end
|
174
174
|
```
|
175
175
|
|
176
|
+
### Include Path
|
177
|
+
|
178
|
+
It is possible to add directories to the include path using the `! include-path` directive like so:
|
179
|
+
|
180
|
+
! include-path include
|
181
|
+
|
182
|
+
The contents of that directive will be added to the include path.
|
183
|
+
|
176
184
|
### Conditional Output
|
177
185
|
|
178
186
|
Under certain circumstances it is useful to have certain output only happen under certain circumstances. For instance, a file prepared for Windows might have slightly different content than the same file prepared for Linux. In order to enable this, a variable may be set within an extension block and then output may be enabled / disabled using directives based on them.
|
@@ -238,6 +246,7 @@ We need to know where to get the input from and where to send the output to. Fo
|
|
238
246
|
``` ruby
|
239
247
|
on("--file FILE", "-f", "Required: input file")
|
240
248
|
on("--output FILE", "-o", "Required: output file")
|
249
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
241
250
|
on("--dev", "disables self test failure for development")
|
242
251
|
```
|
243
252
|
|
@@ -324,7 +333,8 @@ The main body will first test itself then, invoke the library component, which i
|
|
324
333
|
``` ruby
|
325
334
|
@dev = options[:dev]
|
326
335
|
self_test()
|
327
|
-
|
336
|
+
include_path = (options[:"include-path"] or [])
|
337
|
+
tangler = Tangle::Tangler.new(options[:file], include_path)
|
328
338
|
tangler.tangle()
|
329
339
|
tangler.write(options[:output])
|
330
340
|
```
|
@@ -366,6 +376,8 @@ class Tangler
|
|
366
376
|
end
|
367
377
|
⦅tangle_class_privates⦆
|
368
378
|
|
379
|
+
⦅resolve_include⦆
|
380
|
+
|
369
381
|
⦅conditional_processor⦆
|
370
382
|
end
|
371
383
|
```
|
@@ -377,7 +389,8 @@ The initializer takes in the input file and sets up our state. We are keeping t
|
|
377
389
|
###### Code Block: Initializer
|
378
390
|
|
379
391
|
``` ruby
|
380
|
-
def initialize(input)
|
392
|
+
def initialize(input, include_path = [])
|
393
|
+
@include_path = include_path
|
381
394
|
@extension_context = Context.new()
|
382
395
|
@extension_context.filters = ⦅filter_list⦆
|
383
396
|
@input = input
|
@@ -427,20 +440,32 @@ end
|
|
427
440
|
|
428
441
|
As our specification is a regular language (we do not support any kind of nesting), we will be using regular expressions to process it. Those expressions are detailed in:
|
429
442
|
|
430
|
-
**See include:** [lmt_expressions.
|
443
|
+
**See include:** [included file](include/lmt_expressions.md)
|
444
|
+
|
445
|
+
To resolve include paths we need:
|
446
|
+
|
447
|
+
**See include:** [included file](include/lmt_include_path.md)
|
431
448
|
|
432
449
|
Here we go through each line looking for an include statement. When we find one, we replace it with the lines from that file. Those lines will, of course, need to have includes processed as well.
|
433
450
|
|
451
|
+
When we encounter an include-path directive, it needs to be added to the include path.
|
452
|
+
|
434
453
|
###### Code Block: Include Includes
|
435
454
|
|
436
455
|
``` ruby
|
437
456
|
def include_includes(lines, current_file = @input, depth = 0)
|
438
457
|
raise "too many includes" if depth > 1000
|
439
458
|
include_exp = ⦅include_expression⦆
|
459
|
+
include_path_exp = ⦅include_path_expression⦆
|
440
460
|
lines.map do |line|
|
441
|
-
|
442
|
-
|
443
|
-
|
461
|
+
include_path_match = include_path_exp.match(line)
|
462
|
+
include_match = include_exp.match(line)
|
463
|
+
if include_path_match
|
464
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
465
|
+
@include_path << path
|
466
|
+
[line]
|
467
|
+
elsif include_match
|
468
|
+
file = resolve_include(include_match[1], current_file)[0]
|
444
469
|
include_includes(read_file(file), file, depth + 1)
|
445
470
|
else
|
446
471
|
[line]
|
@@ -864,7 +889,7 @@ end
|
|
864
889
|
|
865
890
|
Option verification is described here:
|
866
891
|
|
867
|
-
**See include:** [option_verification.
|
892
|
+
**See include:** [included file](include/option_verification.md)
|
868
893
|
|
869
894
|
## Self Test, Details
|
870
895
|
|
@@ -872,7 +897,7 @@ So, now we need to go into details of our self test and also include regressions
|
|
872
897
|
|
873
898
|
First, we need a method to report test failures:
|
874
899
|
|
875
|
-
**See include:** [error_reporting.
|
900
|
+
**See include:** [included file](include/error_reporting.md)
|
876
901
|
|
877
902
|
Then we need the tests we are doing. The intentionally empty block is included both at the beginning and end to make sure that we handled all the edge cases related to empty blocks appropriately.
|
878
903
|
|
data/doc/lmt/lmw.rb.md
CHANGED
@@ -35,6 +35,7 @@ We need to know where to get the input from and where to send the output to. Fo
|
|
35
35
|
``` ruby
|
36
36
|
on("--file FILE", "-f", "Required: input file")
|
37
37
|
on("--output FILE", "-o", "Required: output file")
|
38
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
38
39
|
on("--dev", "disables self test failure for development")
|
39
40
|
```
|
40
41
|
|
@@ -115,12 +116,13 @@ The main body will first test itself then, invoke the library component, which i
|
|
115
116
|
|
116
117
|
``` ruby
|
117
118
|
self_test()
|
118
|
-
|
119
|
+
include_path = (options[:"include-path"] or [])
|
120
|
+
weave = Lmw::Weave.from_file(options[:file], include_path)
|
119
121
|
weave.weave()
|
120
122
|
weave.write(options[:output])
|
121
123
|
```
|
122
124
|
|
123
|
-
|
125
|
+
We have the dependencies. Optparse and methadone are used for cli argument handling and other niceties.
|
124
126
|
|
125
127
|
###### Code Block: Includes
|
126
128
|
|
@@ -131,6 +133,10 @@ require 'methadone'
|
|
131
133
|
require 'pry'
|
132
134
|
```
|
133
135
|
|
136
|
+
Finally, The include files are located in
|
137
|
+
|
138
|
+
! include-path include
|
139
|
+
|
134
140
|
There, now we are done with the boilerplate. On to:
|
135
141
|
|
136
142
|
## The Actual Weaver
|
@@ -151,17 +157,10 @@ class Weave
|
|
151
157
|
|
152
158
|
private
|
153
159
|
⦅weave_class_privates⦆
|
160
|
+
⦅resolve_include⦆
|
154
161
|
end
|
155
162
|
```
|
156
163
|
|
157
|
-
There may be some private methods, we need a block for them. They will be inserted where needed.
|
158
|
-
|
159
|
-
###### Code Block: Weave Class Privates
|
160
|
-
|
161
|
-
``` ruby
|
162
|
-
⦅include_includes⦆
|
163
|
-
```
|
164
|
-
|
165
164
|
### Initializer
|
166
165
|
|
167
166
|
The initializer takes the input file and sets up our state.
|
@@ -169,7 +168,8 @@ The initializer takes the input file and sets up our state.
|
|
169
168
|
###### Code Block: Initializer
|
170
169
|
|
171
170
|
``` ruby
|
172
|
-
def initialize(lines, file_name = "")
|
171
|
+
def initialize(lines, file_name = "", include_path = [])
|
172
|
+
@include_path = include_path
|
173
173
|
@file_name = file_name
|
174
174
|
@lines = lines
|
175
175
|
@weaved = false
|
@@ -187,9 +187,9 @@ This is fairly self explanatory, though note, we are storing the file in memory
|
|
187
187
|
###### Code Block: From File
|
188
188
|
|
189
189
|
``` ruby
|
190
|
-
def from_file(file)
|
190
|
+
def from_file(file, include_path = [])
|
191
191
|
File.open(file, 'r') do |f|
|
192
|
-
Weave.new(f.readlines, file)
|
192
|
+
Weave.new(f.readlines, file, include_path)
|
193
193
|
end
|
194
194
|
end
|
195
195
|
|
@@ -216,7 +216,7 @@ end
|
|
216
216
|
|
217
217
|
In order to find the blocks we will need the regular expressions defined in:
|
218
218
|
|
219
|
-
**See include:** [lmt_expressions.
|
219
|
+
**See include:** [included file](include/lmt_expressions.md)
|
220
220
|
|
221
221
|
First, we get the lines from includes. then we filter the lines for only the headers and footers and check for unmatched headers and footers.
|
222
222
|
|
@@ -263,19 +263,28 @@ end
|
|
263
263
|
|
264
264
|
This depends on the expression in [lmt_expressions][lmt_expressions.md#The-Include-Expression]
|
265
265
|
|
266
|
+
To resolve include paths we need:
|
267
|
+
|
268
|
+
**See include:** [included file](include/lmt_include_path.md)
|
269
|
+
|
266
270
|
Here we go through each line looking for an include statement. When we find one, we replace it with the lines from that file. Those lines will, of course, need to have includes processed as well. For each line, we also need to add the file that it came from.
|
267
271
|
|
268
|
-
###### Code Block:
|
272
|
+
###### Code Block: Weave Class Privates
|
269
273
|
|
270
274
|
``` ruby
|
271
275
|
def include_includes(lines, current_file = @file_name, current_path = '', depth = 0)
|
272
276
|
raise "too many includes" if depth > 1000
|
273
277
|
include_exp = ⦅include_expression⦆
|
278
|
+
include_path_exp = ⦅include_path_expression⦆
|
274
279
|
lines.map do |line|
|
275
|
-
|
276
|
-
|
277
|
-
|
278
|
-
path =
|
280
|
+
include_path_match = include_path_exp.match(line)
|
281
|
+
include_match = include_exp.match(line)
|
282
|
+
if include_path_match
|
283
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
284
|
+
@include_path << path
|
285
|
+
[[line,current_path]]
|
286
|
+
elsif include_match
|
287
|
+
file, path = resolve_include(include_match[1], current_file)
|
279
288
|
new_lines = File.open(file, 'r') {|f| f.readlines}
|
280
289
|
include_includes(new_lines, file, path, depth + 1)
|
281
290
|
else
|
@@ -305,7 +314,8 @@ def substitute_directives_and_headers(lines)
|
|
305
314
|
case line
|
306
315
|
when include_expression
|
307
316
|
include_file = $1
|
308
|
-
|
317
|
+
include_path = resolve_include(include_file, @file_name)[1]
|
318
|
+
["**See include:** [included file](#{include_path.gsub(/\.lmd/, ".md")})\n"]
|
309
319
|
when code_block_expression
|
310
320
|
in_block = !in_block
|
311
321
|
if in_block
|
@@ -379,7 +389,7 @@ end
|
|
379
389
|
|
380
390
|
Option verification is described here:
|
381
391
|
|
382
|
-
**See include:** [option_verification.
|
392
|
+
**See include:** [included file](include/option_verification.md)
|
383
393
|
|
384
394
|
## Testing
|
385
395
|
|
@@ -387,7 +397,7 @@ Of course, we will also need a testing procedure. In this case, we will be pass
|
|
387
397
|
|
388
398
|
First, we need a method to report test failures:
|
389
399
|
|
390
|
-
**See include:** [error_reporting.
|
400
|
+
**See include:** [included file](include/error_reporting.md)
|
391
401
|
|
392
402
|
###### Code Block: Self Test
|
393
403
|
|
data/lib/lmt/lmt.rb
CHANGED
@@ -4,6 +4,7 @@
|
|
4
4
|
require 'optparse'
|
5
5
|
require 'methadone'
|
6
6
|
require 'lmt/version'
|
7
|
+
require 'pathname'
|
7
8
|
|
8
9
|
module Lmt
|
9
10
|
|
@@ -18,7 +19,8 @@ class Tangle
|
|
18
19
|
begin
|
19
20
|
@dev = options[:dev]
|
20
21
|
self_test()
|
21
|
-
|
22
|
+
include_path = (options[:"include-path"] or [])
|
23
|
+
tangler = Tangle::Tangler.new(options[:file], include_path)
|
22
24
|
tangler.tangle()
|
23
25
|
tangler.write(options[:output])
|
24
26
|
rescue Exception => e
|
@@ -120,7 +122,8 @@ class Tangle
|
|
120
122
|
end
|
121
123
|
|
122
124
|
class Tangler
|
123
|
-
def initialize(input)
|
125
|
+
def initialize(input, include_path = [])
|
126
|
+
@include_path = include_path
|
124
127
|
@extension_context = Context.new()
|
125
128
|
@extension_context.filters = {
|
126
129
|
'ruby_escape' => LineFilter.new do |line|
|
@@ -169,10 +172,16 @@ class Tangle
|
|
169
172
|
def include_includes(lines, current_file = @input, depth = 0)
|
170
173
|
raise "too many includes" if depth > 1000
|
171
174
|
include_exp = /^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
175
|
+
include_path_exp = /^!\s+include-path\s+(.*)\s*$/
|
172
176
|
lines.map do |line|
|
173
|
-
|
174
|
-
|
175
|
-
|
177
|
+
include_path_match = include_path_exp.match(line)
|
178
|
+
include_match = include_exp.match(line)
|
179
|
+
if include_path_match
|
180
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
181
|
+
@include_path << path
|
182
|
+
[line]
|
183
|
+
elsif include_match
|
184
|
+
file = resolve_include(include_match[1], current_file)[0]
|
176
185
|
include_includes(read_file(file), file, depth + 1)
|
177
186
|
else
|
178
187
|
[line]
|
@@ -327,6 +336,21 @@ class Tangle
|
|
327
336
|
end
|
328
337
|
end
|
329
338
|
|
339
|
+
def resolve_include(file, current_file)
|
340
|
+
include_file_loc = File.join(File.dirname(current_file), file)
|
341
|
+
if File.exist?(include_file_loc)
|
342
|
+
return [include_file_loc, file]
|
343
|
+
end
|
344
|
+
@include_path.each do |include_dir|
|
345
|
+
include_file_loc = File.join(include_dir, file)
|
346
|
+
if File.exist? (include_file_loc)
|
347
|
+
relative_path = Pathname.new(include_file_loc).relative_path_from(File.dirname(current_file)).to_s
|
348
|
+
return [include_file_loc, relative_path]
|
349
|
+
end
|
350
|
+
end
|
351
|
+
throw "include file: #{file} not found from #{current_file} or in #{@include_path}"
|
352
|
+
end
|
353
|
+
|
330
354
|
class ConditionalProcessor
|
331
355
|
|
332
356
|
def initialize(extension_context)
|
@@ -401,6 +425,7 @@ class Tangle
|
|
401
425
|
description "A literate Markdown tangle tool written in Ruby."
|
402
426
|
on("--file FILE", "-f", "Required: input file")
|
403
427
|
on("--output FILE", "-o", "Required: output file")
|
428
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
404
429
|
on("--dev", "disables self test failure for development")
|
405
430
|
required(:file, :output)
|
406
431
|
|
data/lib/lmt/lmw.rb
CHANGED
@@ -5,6 +5,7 @@ require 'optparse'
|
|
5
5
|
require 'methadone'
|
6
6
|
|
7
7
|
require 'pry'
|
8
|
+
require 'pathname'
|
8
9
|
|
9
10
|
module Lmt
|
10
11
|
|
@@ -18,7 +19,8 @@ class Lmw
|
|
18
19
|
check_arguments()
|
19
20
|
begin
|
20
21
|
self_test()
|
21
|
-
|
22
|
+
include_path = (options[:"include-path"] or [])
|
23
|
+
weave = Lmw::Weave.from_file(options[:file], include_path)
|
22
24
|
weave.weave()
|
23
25
|
weave.write(options[:output])
|
24
26
|
rescue Exception => e
|
@@ -50,15 +52,16 @@ class Lmw
|
|
50
52
|
|
51
53
|
class Weave
|
52
54
|
class << self
|
53
|
-
def from_file(file)
|
55
|
+
def from_file(file, include_path = [])
|
54
56
|
File.open(file, 'r') do |f|
|
55
|
-
Weave.new(f.readlines, file)
|
57
|
+
Weave.new(f.readlines, file, include_path)
|
56
58
|
end
|
57
59
|
end
|
58
60
|
|
59
61
|
end
|
60
62
|
|
61
|
-
def initialize(lines, file_name = "")
|
63
|
+
def initialize(lines, file_name = "", include_path = [])
|
64
|
+
@include_path = include_path
|
62
65
|
@file_name = file_name
|
63
66
|
@lines = lines
|
64
67
|
@weaved = false
|
@@ -79,22 +82,6 @@ class Lmw
|
|
79
82
|
|
80
83
|
|
81
84
|
private
|
82
|
-
def include_includes(lines, current_file = @file_name, current_path = '', depth = 0)
|
83
|
-
raise "too many includes" if depth > 1000
|
84
|
-
include_exp = /^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
85
|
-
lines.map do |line|
|
86
|
-
match = include_exp.match(line)
|
87
|
-
if match
|
88
|
-
file = File.dirname(current_file) + '/' + match[1]
|
89
|
-
path = File.dirname(current_path) + '/' + match[1]
|
90
|
-
new_lines = File.open(file, 'r') {|f| f.readlines}
|
91
|
-
include_includes(new_lines, file, path, depth + 1)
|
92
|
-
else
|
93
|
-
[[line, current_path]]
|
94
|
-
end
|
95
|
-
end.flatten(1)
|
96
|
-
end
|
97
|
-
|
98
85
|
def find_blocks(lines)
|
99
86
|
lines_with_includes = include_includes(lines)
|
100
87
|
code_block_exp = /^(\s*)``` ?([\w]*) ?(=?)([-\w]*)?/
|
@@ -120,6 +107,27 @@ class Lmw
|
|
120
107
|
{:count => count, :block_locations => block_locations}
|
121
108
|
end
|
122
109
|
end
|
110
|
+
def include_includes(lines, current_file = @file_name, current_path = '', depth = 0)
|
111
|
+
raise "too many includes" if depth > 1000
|
112
|
+
include_exp = /^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
113
|
+
include_path_exp = /^!\s+include-path\s+(.*)\s*$/
|
114
|
+
lines.map do |line|
|
115
|
+
include_path_match = include_path_exp.match(line)
|
116
|
+
include_match = include_exp.match(line)
|
117
|
+
if include_path_match
|
118
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
119
|
+
@include_path << path
|
120
|
+
[[line,current_path]]
|
121
|
+
elsif include_match
|
122
|
+
file, path = resolve_include(include_match[1], current_file)
|
123
|
+
new_lines = File.open(file, 'r') {|f| f.readlines}
|
124
|
+
include_includes(new_lines, file, path, depth + 1)
|
125
|
+
else
|
126
|
+
[[line, current_path]]
|
127
|
+
end
|
128
|
+
end.flatten(1)
|
129
|
+
end
|
130
|
+
|
123
131
|
def substitute_directives_and_headers(lines)
|
124
132
|
include_expression = /^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
125
133
|
code_block_expression = /^(\s*)``` ?([\w]*) ?(=?)([-\w]*)?/
|
@@ -130,7 +138,8 @@ class Lmw
|
|
130
138
|
case line
|
131
139
|
when include_expression
|
132
140
|
include_file = $1
|
133
|
-
|
141
|
+
include_path = resolve_include(include_file, @file_name)[1]
|
142
|
+
["**See include:** [included file](#{include_path.gsub(/\.lmd/, ".md")})\n"]
|
134
143
|
when code_block_expression
|
135
144
|
in_block = !in_block
|
136
145
|
if in_block
|
@@ -166,6 +175,20 @@ class Lmw
|
|
166
175
|
def replace_markdown_links(line)
|
167
176
|
line
|
168
177
|
end
|
178
|
+
def resolve_include(file, current_file)
|
179
|
+
include_file_loc = File.join(File.dirname(current_file), file)
|
180
|
+
if File.exist?(include_file_loc)
|
181
|
+
return [include_file_loc, file]
|
182
|
+
end
|
183
|
+
@include_path.each do |include_dir|
|
184
|
+
include_file_loc = File.join(include_dir, file)
|
185
|
+
if File.exist? (include_file_loc)
|
186
|
+
relative_path = Pathname.new(include_file_loc).relative_path_from(File.dirname(current_file)).to_s
|
187
|
+
return [include_file_loc, relative_path]
|
188
|
+
end
|
189
|
+
end
|
190
|
+
throw "include file: #{file} not found from #{current_file} or in #{@include_path}"
|
191
|
+
end
|
169
192
|
end
|
170
193
|
|
171
194
|
def self.required(*options)
|
@@ -184,6 +207,7 @@ class Lmw
|
|
184
207
|
description "A literate Markdown weave tool written in Ruby."
|
185
208
|
on("--file FILE", "-f", "Required: input file")
|
186
209
|
on("--output FILE", "-o", "Required: output file")
|
210
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
187
211
|
on("--dev", "disables self test failure for development")
|
188
212
|
required(:file, :output)
|
189
213
|
|
data/lib/lmt/version.rb
CHANGED
File without changes
|
@@ -10,6 +10,14 @@ The first regular expression handles the detection of include directives. It re
|
|
10
10
|
/^!\s+include\s+\[.*\]\((.*)\)\s*$/
|
11
11
|
```
|
12
12
|
|
13
|
+
## The Include Path Expression
|
14
|
+
|
15
|
+
This regular expression handles the detection of include-path directives. It recognizes lines like `! include-path some-path` and extracts `some-path`.
|
16
|
+
|
17
|
+
``` ruby include_path_expression
|
18
|
+
/^!\s+include-path\s+(.*)\s*$/
|
19
|
+
```
|
20
|
+
|
13
21
|
## The Block Expressions
|
14
22
|
|
15
23
|
These regular expression recognize if, elseif, else, and end directives. The first group contains the conditional for blocks that have a conditional..
|
File without changes
|
@@ -0,0 +1,24 @@
|
|
1
|
+
# Include Path
|
2
|
+
|
3
|
+
This is the basic algorithm to look up an include on the include path. If the file exists relative to the current file, it includes it. If, not, it goes through the include path looking for the file. When it finds it, it also calculates a relative path for the file from the current file.
|
4
|
+
|
5
|
+
``` ruby includes
|
6
|
+
require 'pathname'
|
7
|
+
```
|
8
|
+
|
9
|
+
``` ruby resolve_include
|
10
|
+
def resolve_include(file, current_file)
|
11
|
+
include_file_loc = File.join(File.dirname(current_file), file)
|
12
|
+
if File.exist?(include_file_loc)
|
13
|
+
return [include_file_loc, file]
|
14
|
+
end
|
15
|
+
@include_path.each do |include_dir|
|
16
|
+
include_file_loc = File.join(include_dir, file)
|
17
|
+
if File.exist? (include_file_loc)
|
18
|
+
relative_path = Pathname.new(include_file_loc).relative_path_from(File.dirname(current_file)).to_s
|
19
|
+
return [include_file_loc, relative_path]
|
20
|
+
end
|
21
|
+
end
|
22
|
+
throw "include file: #{file} not found from #{current_file} or in #{@include_path}"
|
23
|
+
end
|
24
|
+
```
|
File without changes
|
data/src/lmt/lmt.rb.lmd
CHANGED
@@ -132,7 +132,7 @@ During tangle the link line will be replaced with the lines from the included fi
|
|
132
132
|
included_string = "I am in lmt.lmd"
|
133
133
|
```
|
134
134
|
|
135
|
-
! include [an include](lmt_include.lmd)
|
135
|
+
! include [an include](include/lmt_include.lmd)
|
136
136
|
|
137
137
|
### Extension
|
138
138
|
|
@@ -147,6 +147,14 @@ def parse_hook(main_block, blocks)
|
|
147
147
|
end
|
148
148
|
```
|
149
149
|
|
150
|
+
### Include Path
|
151
|
+
|
152
|
+
It is possible to add directories to the include path using the `! include-path` directive like so:
|
153
|
+
|
154
|
+
! include-path include
|
155
|
+
|
156
|
+
The contents of that directive will be added to the include path.
|
157
|
+
|
150
158
|
### Conditional Output
|
151
159
|
|
152
160
|
Under certain circumstances it is useful to have certain output only happen under certain circumstances. For instance, a file prepared for Windows might have slightly different content than the same file prepared for Linux. In order to enable this, a variable may be set within an extension block and then output may be enabled / disabled using directives based on them.
|
@@ -200,6 +208,7 @@ We need to know where to get the input from and where to send the output to. Fo
|
|
200
208
|
``` ruby options
|
201
209
|
on("--file FILE", "-f", "Required: input file")
|
202
210
|
on("--output FILE", "-o", "Required: output file")
|
211
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
203
212
|
on("--dev", "disables self test failure for development")
|
204
213
|
```
|
205
214
|
|
@@ -280,7 +289,8 @@ The main body will first test itself then, invoke the library component, which i
|
|
280
289
|
``` ruby main_body
|
281
290
|
@dev = options[:dev]
|
282
291
|
self_test()
|
283
|
-
|
292
|
+
include_path = (options[:"include-path"] or [])
|
293
|
+
tangler = Tangle::Tangler.new(options[:file], include_path)
|
284
294
|
tangler.tangle()
|
285
295
|
tangler.write(options[:output])
|
286
296
|
```
|
@@ -318,6 +328,8 @@ class Tangler
|
|
318
328
|
end
|
319
329
|
⦅tangle_class_privates⦆
|
320
330
|
|
331
|
+
⦅resolve_include⦆
|
332
|
+
|
321
333
|
⦅conditional_processor⦆
|
322
334
|
end
|
323
335
|
```
|
@@ -327,7 +339,8 @@ end
|
|
327
339
|
The initializer takes in the input file and sets up our state. We are keeping the unnamed top level block separate from the rest. Then we have a hash of blocks. Finally, we need to make sure we have tangled before we write the output.
|
328
340
|
|
329
341
|
``` ruby initializer
|
330
|
-
def initialize(input)
|
342
|
+
def initialize(input, include_path = [])
|
343
|
+
@include_path = include_path
|
331
344
|
@extension_context = Context.new()
|
332
345
|
@extension_context.filters = ⦅filter_list⦆
|
333
346
|
@input = input
|
@@ -375,16 +388,28 @@ As our specification is a regular language (we do not support any kind of nestin
|
|
375
388
|
|
376
389
|
! include [here](lmt_expressions.lmd)
|
377
390
|
|
391
|
+
To resolve include paths we need:
|
392
|
+
|
393
|
+
! include [this](lmt_include_path.lmd)
|
394
|
+
|
378
395
|
Here we go through each line looking for an include statement. When we find one, we replace it with the lines from that file. Those lines will, of course, need to have includes processed as well.
|
379
396
|
|
397
|
+
When we encounter an include-path directive, it needs to be added to the include path.
|
398
|
+
|
380
399
|
``` ruby include_includes
|
381
400
|
def include_includes(lines, current_file = @input, depth = 0)
|
382
401
|
raise "too many includes" if depth > 1000
|
383
402
|
include_exp = ⦅include_expression⦆
|
403
|
+
include_path_exp = ⦅include_path_expression⦆
|
384
404
|
lines.map do |line|
|
385
|
-
|
386
|
-
|
387
|
-
|
405
|
+
include_path_match = include_path_exp.match(line)
|
406
|
+
include_match = include_exp.match(line)
|
407
|
+
if include_path_match
|
408
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
409
|
+
@include_path << path
|
410
|
+
[line]
|
411
|
+
elsif include_match
|
412
|
+
file = resolve_include(include_match[1], current_file)[0]
|
388
413
|
include_includes(read_file(file), file, depth + 1)
|
389
414
|
else
|
390
415
|
[line]
|
data/src/lmt/lmw.rb.lmd
CHANGED
@@ -31,6 +31,7 @@ We need to know where to get the input from and where to send the output to. Fo
|
|
31
31
|
``` ruby options
|
32
32
|
on("--file FILE", "-f", "Required: input file")
|
33
33
|
on("--output FILE", "-o", "Required: output file")
|
34
|
+
on("--include-path DIRECTORY,DIRECTORY", "-i", Array, "Include path")
|
34
35
|
on("--dev", "disables self test failure for development")
|
35
36
|
```
|
36
37
|
|
@@ -105,12 +106,13 @@ The main body will first test itself then, invoke the library component, which i
|
|
105
106
|
|
106
107
|
``` ruby main_body
|
107
108
|
self_test()
|
108
|
-
|
109
|
+
include_path = (options[:"include-path"] or [])
|
110
|
+
weave = Lmw::Weave.from_file(options[:file], include_path)
|
109
111
|
weave.weave()
|
110
112
|
weave.write(options[:output])
|
111
113
|
```
|
112
114
|
|
113
|
-
|
115
|
+
We have the dependencies. Optparse and methadone are used for cli argument handling and other niceties.
|
114
116
|
|
115
117
|
``` ruby includes
|
116
118
|
require 'optparse'
|
@@ -119,6 +121,10 @@ require 'methadone'
|
|
119
121
|
require 'pry'
|
120
122
|
```
|
121
123
|
|
124
|
+
Finally, The include files are located in
|
125
|
+
|
126
|
+
! include-path include
|
127
|
+
|
122
128
|
There, now we are done with the boilerplate. On to:
|
123
129
|
|
124
130
|
## The Actual Weaver
|
@@ -137,21 +143,17 @@ class Weave
|
|
137
143
|
|
138
144
|
private
|
139
145
|
⦅weave_class_privates⦆
|
146
|
+
⦅resolve_include⦆
|
140
147
|
end
|
141
148
|
```
|
142
149
|
|
143
|
-
There may be some private methods, we need a block for them. They will be inserted where needed.
|
144
|
-
|
145
|
-
``` ruby weave_class_privates
|
146
|
-
⦅include_includes⦆
|
147
|
-
```
|
148
|
-
|
149
150
|
### Initializer
|
150
151
|
|
151
152
|
The initializer takes the input file and sets up our state.
|
152
153
|
|
153
154
|
``` ruby initializer
|
154
|
-
def initialize(lines, file_name = "")
|
155
|
+
def initialize(lines, file_name = "", include_path = [])
|
156
|
+
@include_path = include_path
|
155
157
|
@file_name = file_name
|
156
158
|
@lines = lines
|
157
159
|
@weaved = false
|
@@ -167,9 +169,9 @@ For testing, we want to be able to create an instance with a hard coded set of l
|
|
167
169
|
This is fairly self explanatory, though note, we are storing the file in memory as an array of lines.
|
168
170
|
|
169
171
|
``` ruby from_file
|
170
|
-
def from_file(file)
|
172
|
+
def from_file(file, include_path = [])
|
171
173
|
File.open(file, 'r') do |f|
|
172
|
-
Weave.new(f.readlines, file)
|
174
|
+
Weave.new(f.readlines, file, include_path)
|
173
175
|
end
|
174
176
|
end
|
175
177
|
|
@@ -237,17 +239,26 @@ end
|
|
237
239
|
|
238
240
|
This depends on the expression in [lmt_expressions][lmt_expressions.md#The-Include-Expression]
|
239
241
|
|
242
|
+
To resolve include paths we need:
|
243
|
+
|
244
|
+
! include [this](include/lmt_include_path.lmd)
|
245
|
+
|
240
246
|
Here we go through each line looking for an include statement. When we find one, we replace it with the lines from that file. Those lines will, of course, need to have includes processed as well. For each line, we also need to add the file that it came from.
|
241
247
|
|
242
|
-
``` ruby
|
248
|
+
``` ruby weave_class_privates
|
243
249
|
def include_includes(lines, current_file = @file_name, current_path = '', depth = 0)
|
244
250
|
raise "too many includes" if depth > 1000
|
245
251
|
include_exp = ⦅include_expression⦆
|
252
|
+
include_path_exp = ⦅include_path_expression⦆
|
246
253
|
lines.map do |line|
|
247
|
-
|
248
|
-
|
249
|
-
|
250
|
-
path =
|
254
|
+
include_path_match = include_path_exp.match(line)
|
255
|
+
include_match = include_exp.match(line)
|
256
|
+
if include_path_match
|
257
|
+
path = resolve_include(include_path_match[1], current_file)[0]
|
258
|
+
@include_path << path
|
259
|
+
[[line,current_path]]
|
260
|
+
elsif include_match
|
261
|
+
file, path = resolve_include(include_match[1], current_file)
|
251
262
|
new_lines = File.open(file, 'r') {|f| f.readlines}
|
252
263
|
include_includes(new_lines, file, path, depth + 1)
|
253
264
|
else
|
@@ -275,7 +286,8 @@ def substitute_directives_and_headers(lines)
|
|
275
286
|
case line
|
276
287
|
when include_expression
|
277
288
|
include_file = $1
|
278
|
-
|
289
|
+
include_path = resolve_include(include_file, @file_name)[1]
|
290
|
+
["**See include:** [included file](#{include_path.gsub(/\.lmd/, ".md")})\n"]
|
279
291
|
when code_block_expression
|
280
292
|
in_block = !in_block
|
281
293
|
if in_block
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lmt
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Marty Gentillon
|
@@ -113,23 +113,25 @@ files:
|
|
113
113
|
- bin/lmt
|
114
114
|
- bin/lmw
|
115
115
|
- bin/setup
|
116
|
-
- doc/lmt/error_reporting.md
|
116
|
+
- doc/lmt/include/error_reporting.md
|
117
|
+
- doc/lmt/include/lmt_expressions.md
|
118
|
+
- doc/lmt/include/lmt_include.md
|
119
|
+
- doc/lmt/include/lmt_include_path.md
|
120
|
+
- doc/lmt/include/option_verification.md
|
117
121
|
- doc/lmt/lmt.rb.md
|
118
|
-
- doc/lmt/lmt_expressions.md
|
119
|
-
- doc/lmt/lmt_include.md
|
120
122
|
- doc/lmt/lmw.rb.md
|
121
|
-
- doc/lmt/option_verification.md
|
122
123
|
- lib/lmt.rb
|
123
124
|
- lib/lmt/lmt.rb
|
124
125
|
- lib/lmt/lmw.rb
|
125
126
|
- lib/lmt/version.rb
|
126
127
|
- lmt.gemspec
|
127
|
-
- src/lmt/error_reporting.lmd
|
128
|
+
- src/lmt/include/error_reporting.lmd
|
129
|
+
- src/lmt/include/lmt_expressions.lmd
|
130
|
+
- src/lmt/include/lmt_include.lmd
|
131
|
+
- src/lmt/include/lmt_include_path.lmd
|
132
|
+
- src/lmt/include/option_verification.lmd
|
128
133
|
- src/lmt/lmt.rb.lmd
|
129
|
-
- src/lmt/lmt_expressions.lmd
|
130
|
-
- src/lmt/lmt_include.lmd
|
131
134
|
- src/lmt/lmw.rb.lmd
|
132
|
-
- src/lmt/option_verification.lmd
|
133
135
|
homepage: https://github.com/MartyGentillon/lmt-ruby
|
134
136
|
licenses:
|
135
137
|
- MIT
|