buildlogparser 0.2.0 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/CHANGELOG.md +20 -9
- data/README.md +71 -4
- data/bin/buildlogparser +139 -0
- data/buildlogparser.gemspec +3 -1
- data/lib/buildlogparser.rb +10 -8
- data/lib/buildlogparser/parsers/cmake.rb +4 -2
- data/lib/buildlogparser/parsers/coremark.rb +3 -1
- data/lib/buildlogparser/parsers/ctest.rb +6 -1
- data/lib/buildlogparser/parsers/dhrystone.rb +5 -3
- data/lib/buildlogparser/parsers/lld.rb +105 -0
- data/lib/buildlogparser/parsers/size.rb +3 -1
- data/lib/buildlogparser/registry.rb +41 -0
- data/lib/buildlogparser/version.rb +1 -1
- metadata +37 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 757a90669873992e6241a8400856a00197bd5bea
|
4
|
+
data.tar.gz: db4761b42cbe5a9504cd7d290e058634ac1b9463
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8d63d5828411da6e36c5862d770fc82ca0219e32d8e04c06d8e643af7f376efca0c73a8fe91fd1eb88ff3809f770a6a5b172efe90539ce3d0df89f3dd0e94e63
|
7
|
+
data.tar.gz: 9f5368ec79edbe6f861c2bb6329d0e24f7a367325af8a6eba036cd0e7747798781f1e2cb3a2f63c230ce759fccd3969fe4bb943fb35bd341763bc5bc02df724f
|
data/.gitignore
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,16 +1,27 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## v0.3.0 (09.05.2018)
|
4
|
+
|
5
|
+
* Added a registry which maps string names to parser functions.
|
6
|
+
* Added a simple command line tool named *buildlogparser* that ...
|
7
|
+
- ... applies a parser function on an input file.
|
8
|
+
- ... formats the results either as *yaml* (default), *csv*, or ASCII *table*.
|
9
|
+
- ... writes the results to stdout or to an output file.
|
10
|
+
- ... can load and use external parsers that register themselves via the registry.
|
11
|
+
* Added support for parsing *lld* generated map files.
|
12
|
+
* Removed redundant entries from the path match rules of the bundled parsers.
|
13
|
+
|
3
14
|
## v0.2.0 (28.10.2017)
|
4
15
|
|
5
|
-
* Added support for parsing *coremark* command line output
|
6
|
-
* Added support for parsing *dhrystone* command line output
|
7
|
-
* [ctest] Fixed parsing of regex errors in the command line output
|
8
|
-
* [cmake] Fixed parsing of non empty text which does not match
|
9
|
-
* [size] Fixed parsing of non empty text which does not match
|
16
|
+
* Added support for parsing *coremark* command line output.
|
17
|
+
* Added support for parsing *dhrystone* command line output.
|
18
|
+
* [ctest] Fixed parsing of regex errors in the command line output.
|
19
|
+
* [cmake] Fixed parsing of non empty text which does not match.
|
20
|
+
* [size] Fixed parsing of non empty text which does not match.
|
10
21
|
|
11
22
|
## v0.1.0 (Unreleased)
|
12
23
|
|
13
|
-
* Added support for parsing *cmake* generated makefile command line output
|
14
|
-
* Added support for parsing *ctest* command line output
|
15
|
-
* Added support for parsing *ctest* generated `LastTest.log` files
|
16
|
-
* Added support for parsing *size* command line output in Berkeley format
|
24
|
+
* Added support for parsing *cmake* generated makefile command line output.
|
25
|
+
* Added support for parsing *ctest* command line output.
|
26
|
+
* Added support for parsing *ctest* generated `LastTest.log` files.
|
27
|
+
* Added support for parsing *size* command line output in Berkeley format.
|
data/README.md
CHANGED
@@ -1,14 +1,81 @@
|
|
1
1
|
[![Gem Version](https://badge.fury.io/rb/buildlogparser.svg)](https://rubygems.org/gems/buildlogparser)
|
2
|
-
[![Build Status](https://travis-ci.org/niosHD/buildlogparser.svg?branch=
|
3
|
-
[![codecov](https://codecov.io/gh/niosHD/buildlogparser/branch/
|
2
|
+
[![Build Status](https://travis-ci.org/niosHD/buildlogparser.svg?branch=master)](https://travis-ci.org/niosHD/buildlogparser)
|
3
|
+
[![codecov](https://codecov.io/gh/niosHD/buildlogparser/branch/master/graph/badge.svg)](https://codecov.io/gh/niosHD/buildlogparser)
|
4
|
+
|
5
|
+
Collection of various parsers for the extraction of information from build and execution logs.
|
4
6
|
|
5
7
|
# buildlogparser Gem
|
6
8
|
|
7
|
-
|
9
|
+
To enable easy processing and analysis using automated tools, it is best to have data in a format that is easily machine-readable (e.g., XML, JSON, YAML, CSV, ...). However, not every tool supports the generation of such output formats or using them may not be possible. Therefore, other ways to gather the needed information have to be found. Log files, which are often available, are such an alternative information source.
|
10
|
+
|
11
|
+
This gem provides a collection of simple parsers to extract information from such log files. Of course, gathering the same data as from reading a machine-readable format is probably not possible given that log files are typically only semi-structured and/or optimized for human readability. Still, quite some valuable information can be extracted.
|
12
|
+
|
13
|
+
## Installation
|
14
|
+
|
15
|
+
Release versions of the gem can be installed from rubygems:
|
16
|
+
|
17
|
+
~~~bash
|
18
|
+
$ gem install buildlogparser
|
19
|
+
~~~
|
20
|
+
|
21
|
+
## Usage as Command Line Tool
|
22
|
+
|
23
|
+
Like shown in the following examples, the included `buildlogparser` command line tool is a convenient way to apply one specific parser to an input file.
|
24
|
+
~~~bash
|
25
|
+
# parse coremark output and print the result as yaml to stdout
|
26
|
+
$ buildlogparser -t coremark -f yaml -i test/coremark/example_1.txt
|
27
|
+
# parse size output in Berkeley format and print the result as ASCII table to stdout
|
28
|
+
$ buildlogparser -t sizeBerkeleyStdout -f table -i test/size/berkeley_stdout_multi_line.txt
|
29
|
+
# parse ctest stdout output and write the result as csv into result.csv
|
30
|
+
$ buildlogparser -t ctestStdout -f csv -i test/ctest/stdout_success.txt -o result.csv
|
31
|
+
~~~
|
32
|
+
Note that also the usage of external parsers, which are not bundled with the gem, is supported given that they register themselves. More information on the supported arguments as well as the names for the bundled parsers can be found in the included help (i.e., `buildlogparser -h`).
|
33
|
+
|
34
|
+
## Usage as Library
|
35
|
+
|
36
|
+
It is also possible to use the `buildlogparser` gem as library in situations where more complex parsers are needed or when the extracted data should be directly processed. Simply require the gem and instantiate the desired parsers, either directly or indirectly via the registry, as in the following example.
|
37
|
+
|
38
|
+
~~~ruby
|
39
|
+
require 'rubygems'
|
40
|
+
require 'buildlogparser'
|
41
|
+
|
42
|
+
logtext = """
|
43
|
+
text data bss dec hex filename
|
44
|
+
4960 72 64 5096 13e8 aes
|
45
|
+
12656 72 68 12796 31fc bigen
|
46
|
+
952 72 64 1088 440 indcall
|
47
|
+
"""
|
48
|
+
|
49
|
+
parser = BuildLogParser::SizeParser.new()
|
50
|
+
parse_result = parser.parseBerkeleyStdout(logtext)
|
51
|
+
|
52
|
+
puts "data/bss ratios:"
|
53
|
+
parse_result.each do |line|
|
54
|
+
puts "#{line[:filename]}: #{line[:data].to_f/line[:bss].to_f}"
|
55
|
+
end
|
56
|
+
~~~
|
57
|
+
|
58
|
+
## Developing the gem
|
59
|
+
|
60
|
+
Running tests:
|
61
|
+
~~~bash
|
62
|
+
# execute the full test suite
|
63
|
+
$ rake test
|
64
|
+
# execute the full test suite in verbose mode
|
65
|
+
$ rake test TESTOPTS='-v'
|
66
|
+
# execute only the `RegistryTest#test_get_names` test
|
67
|
+
$ rake test TESTOPTS='-n=/RegistryTest#test_get_names/'
|
68
|
+
~~~
|
69
|
+
|
70
|
+
Executing the command line tool without installation:
|
71
|
+
~~~bash
|
72
|
+
$ RUBYLIB=<repo-path>/lib <repo-path>/bin/buildlogparser <arguments>
|
73
|
+
~~~
|
8
74
|
|
9
|
-
## Tested Program Versions
|
75
|
+
## Tested Program Versions
|
10
76
|
|
11
77
|
* cmake/ctest 3.5.1
|
12
78
|
* coremark 1.0
|
13
79
|
* dhrystone (C Version) 2.1 and 2.2
|
80
|
+
* lld 7.0 (development version)
|
14
81
|
* size (GNU Binutils 2.26.1)
|
data/bin/buildlogparser
ADDED
@@ -0,0 +1,139 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
#
|
3
|
+
# Copyright (C) 2017-2018 Mario Werner <nioshd@gmail.com>
|
4
|
+
#
|
5
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
6
|
+
# this software and associated documentation files (the "Software"), to deal in
|
7
|
+
# the Software without restriction, including without limitation the rights to
|
8
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
9
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
10
|
+
# subject to the following conditions:
|
11
|
+
#
|
12
|
+
# The above copyright notice and this permission notice shall be included in all
|
13
|
+
# copies or substantial portions of the Software.
|
14
|
+
#
|
15
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
17
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
18
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
19
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
20
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
21
|
+
#
|
22
|
+
|
23
|
+
require 'csv'
|
24
|
+
require 'yaml'
|
25
|
+
|
26
|
+
require 'rubygems'
|
27
|
+
require 'buildlogparser'
|
28
|
+
require 'terminal-table'
|
29
|
+
require 'trollop'
|
30
|
+
|
31
|
+
module ApplicationLogic
|
32
|
+
def self.parseComandLine(argv)
|
33
|
+
parsers = BuildLogParser::getParserNames
|
34
|
+
formats = [:yaml, :csv, :table]
|
35
|
+
p = Trollop::Parser.new do
|
36
|
+
banner <<-EOS
|
37
|
+
Extracts information from various log file formats.
|
38
|
+
|
39
|
+
Usage:
|
40
|
+
#{__FILE__} -t <type> -i <inputfile> [-f <outputformat>] [-p <parsermodule>] [-o <outputfile>]
|
41
|
+
where [options] are:
|
42
|
+
EOS
|
43
|
+
|
44
|
+
opt :type, "File type. (supported: #{parsers.join(",")})", :type => String, :required => true, :short => "-t"
|
45
|
+
opt :inputfile, "Path to the log file which should be processed.", :type => String, :required => true, :short => "-i"
|
46
|
+
opt :format, "Output format. (default: #{formats[0]}, supported: #{formats.join(",")}", :default => formats[0].to_s, :short => "-f"
|
47
|
+
opt :outputfile, "Output file.", :type => String, :short => "-o"
|
48
|
+
opt :parsermodule, "Path to ruby file or gem name that implements additional parsers.", :type => String, :multi => true, :short => "-p"
|
49
|
+
end
|
50
|
+
opts = {}
|
51
|
+
Trollop::with_standard_exception_handling p do
|
52
|
+
opts = p.parse argv
|
53
|
+
end
|
54
|
+
|
55
|
+
# load additional parser modules
|
56
|
+
if opts[:parsermodule] then
|
57
|
+
opts[:parsermodule].each do |path|
|
58
|
+
if File.exists?(path) then
|
59
|
+
path = File.realpath(path).chomp(File.extname(path))
|
60
|
+
end
|
61
|
+
require path
|
62
|
+
end
|
63
|
+
end
|
64
|
+
parsers = BuildLogParser::getParserNames
|
65
|
+
|
66
|
+
# check if the parameters are valid
|
67
|
+
opts[:format] = opts[:format].to_sym
|
68
|
+
opts[:type] = opts[:type].to_sym
|
69
|
+
unless formats.include?(opts[:format]) then
|
70
|
+
raise ArgumentError, "Output format '#{opts[:format]}' is not supported"
|
71
|
+
end
|
72
|
+
unless parsers.include?(opts[:type]) then
|
73
|
+
raise ArgumentError, "File type '#{opts[:type]}' is not supported"
|
74
|
+
end
|
75
|
+
unless File.exists?(opts[:inputfile]) then
|
76
|
+
raise ArgumentError, "Inputfile '#{opts[:inputfile]}' does not exist"
|
77
|
+
end
|
78
|
+
return opts
|
79
|
+
end
|
80
|
+
|
81
|
+
def self.formatRow(hash, columns)
|
82
|
+
result = []
|
83
|
+
columns.each do |key|
|
84
|
+
if hash.key?(key) then
|
85
|
+
result.push(hash[key])
|
86
|
+
else
|
87
|
+
result.push('')
|
88
|
+
end
|
89
|
+
end
|
90
|
+
return result
|
91
|
+
end
|
92
|
+
|
93
|
+
def self.formatResult(result, format)
|
94
|
+
# extract columns from result for table based formats
|
95
|
+
columns = []
|
96
|
+
result.each do |line|
|
97
|
+
columns |= line.keys
|
98
|
+
end
|
99
|
+
|
100
|
+
# format the result data
|
101
|
+
if format == :yaml then
|
102
|
+
result = result.to_yaml
|
103
|
+
elsif format == :csv then
|
104
|
+
result = CSV.generate do |csv|
|
105
|
+
csv << columns
|
106
|
+
result.each do |hash|
|
107
|
+
csv << formatRow(hash, columns)
|
108
|
+
end
|
109
|
+
end
|
110
|
+
elsif format == :table then
|
111
|
+
table = Terminal::Table.new
|
112
|
+
table.headings = columns
|
113
|
+
result.each do |hash|
|
114
|
+
table.add_row(formatRow(hash, columns))
|
115
|
+
end
|
116
|
+
result = table.to_s
|
117
|
+
else
|
118
|
+
raise ArgumentError, "Output format '#{format}' is not supported"
|
119
|
+
end
|
120
|
+
return result
|
121
|
+
end
|
122
|
+
end
|
123
|
+
|
124
|
+
if $0 == __FILE__ then
|
125
|
+
opts = ApplicationLogic.parseComandLine(ARGV)
|
126
|
+
|
127
|
+
# load the file into memory and parse it
|
128
|
+
logtext = IO.read(opts[:inputfile])
|
129
|
+
parser = BuildLogParser.getParser(opts[:type])
|
130
|
+
result = BuildLogParser.parse(opts[:type],parser,logtext)
|
131
|
+
|
132
|
+
# save data to the output file
|
133
|
+
result = ApplicationLogic.formatResult(result, opts[:format])
|
134
|
+
if opts[:outputfile] then
|
135
|
+
File.open(opts[:outputfile], 'w') {|f| f.write result }
|
136
|
+
else
|
137
|
+
puts result
|
138
|
+
end
|
139
|
+
end
|
data/buildlogparser.gemspec
CHANGED
@@ -7,7 +7,7 @@ Gem::Specification.new do |spec|
|
|
7
7
|
spec.name = "buildlogparser"
|
8
8
|
spec.version = BuildLogParser::VERSION::STRING
|
9
9
|
spec.authors = ["Mario Werner"]
|
10
|
-
spec.email = ["
|
10
|
+
spec.email = ["nioshd@gmail.com"]
|
11
11
|
spec.summary = "Collection of various parsers for the extraction of information from build and execution logs."
|
12
12
|
spec.homepage = "https://github.com/niosHD/buildlogparser"
|
13
13
|
spec.license = "MIT"
|
@@ -26,4 +26,6 @@ Gem::Specification.new do |spec|
|
|
26
26
|
spec.add_development_dependency "rake", "~> 10.5"
|
27
27
|
spec.add_development_dependency "simplecov", "~> 0.14"
|
28
28
|
spec.add_runtime_dependency "parslet", "~> 1.8"
|
29
|
+
spec.add_runtime_dependency "trollop", "~> 2.1"
|
30
|
+
spec.add_runtime_dependency "terminal-table", "~> 1.8"
|
29
31
|
end
|
data/lib/buildlogparser.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
#
|
2
|
-
# Copyright (C) 2017 Mario Werner <nioshd@gmail.com>
|
2
|
+
# Copyright (C) 2017-2018 Mario Werner <nioshd@gmail.com>
|
3
3
|
#
|
4
4
|
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
5
5
|
# this software and associated documentation files (the "Software"), to deal in
|
@@ -18,11 +18,13 @@
|
|
18
18
|
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
19
19
|
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
20
20
|
#
|
21
|
-
|
22
|
-
|
23
|
-
require "buildlogparser/parsers/coremark"
|
24
|
-
require "buildlogparser/parsers/ctest"
|
25
|
-
require "buildlogparser/parsers/dhrystone"
|
26
|
-
require "buildlogparser/parsers/size"
|
21
|
+
require_relative "buildlogparser/registry"
|
22
|
+
require_relative "buildlogparser/version"
|
27
23
|
|
28
|
-
|
24
|
+
require_relative "buildlogparser/parser"
|
25
|
+
require_relative "buildlogparser/parsers/cmake"
|
26
|
+
require_relative "buildlogparser/parsers/coremark"
|
27
|
+
require_relative "buildlogparser/parsers/ctest"
|
28
|
+
require_relative "buildlogparser/parsers/dhrystone"
|
29
|
+
require_relative "buildlogparser/parsers/lld"
|
30
|
+
require_relative "buildlogparser/parsers/size"
|
@@ -29,7 +29,7 @@ module BuildLogParser
|
|
29
29
|
rule(:newline) { str("\r").maybe >> str("\n") }
|
30
30
|
rule(:restofline){ ( newline.absent? >> any ).repeat }
|
31
31
|
|
32
|
-
rule(:path) { match['[:alnum:]
|
32
|
+
rule(:path) { match['[:alnum:]=\+\.\-_/'].repeat(1) }
|
33
33
|
rule(:integer) { match['0-9'].repeat(1) }
|
34
34
|
|
35
35
|
rule(:percentage) { str("[") >> space? >> integer.as(:percentage) >> str("%]") }
|
@@ -86,7 +86,7 @@ module BuildLogParser
|
|
86
86
|
|
87
87
|
def reset()
|
88
88
|
super()
|
89
|
-
|
89
|
+
@targets = []
|
90
90
|
end
|
91
91
|
|
92
92
|
def parseMakefileStdout(logtext)
|
@@ -97,4 +97,6 @@ module BuildLogParser
|
|
97
97
|
@targets = CMakeMakefileStdout::Transform.new.apply(tree)
|
98
98
|
end
|
99
99
|
end # class CMakeParser
|
100
|
+
|
101
|
+
registerParser(:cmakeMakefileStdout, CMakeParser, :parseMakefileStdout)
|
100
102
|
end # module BuildLogParser
|
@@ -70,7 +70,7 @@ module BuildLogParser
|
|
70
70
|
end
|
71
71
|
|
72
72
|
rule(:letters) { match['[:alnum:]'].repeat(1) }
|
73
|
-
rule(:path) { match['[:alnum:]
|
73
|
+
rule(:path) { match['[:alnum:]=\+\.\-_/'].repeat(1) }
|
74
74
|
rule(:integer) { match['0-9'].repeat(1) }
|
75
75
|
rule(:float) { integer >> (match['\.,'] >> integer).maybe }
|
76
76
|
|
@@ -138,6 +138,7 @@ module BuildLogParser
|
|
138
138
|
@data.each do |event|
|
139
139
|
@errors += 1 unless event[:result] == :passed
|
140
140
|
end
|
141
|
+
return @data
|
141
142
|
end
|
142
143
|
|
143
144
|
def parseLog(logtext)
|
@@ -150,6 +151,10 @@ module BuildLogParser
|
|
150
151
|
@data.each do |event|
|
151
152
|
@errors += 1 unless event[:result] == :passed
|
152
153
|
end
|
154
|
+
return @data
|
153
155
|
end
|
154
156
|
end # class CTestParser
|
157
|
+
|
158
|
+
registerParser(:ctestStdout, CTestParser, :parseStdout)
|
159
|
+
registerParser(:ctestLog, CTestParser, :parseLog)
|
155
160
|
end # module BuildLogParser
|
@@ -29,7 +29,7 @@ module BuildLogParser
|
|
29
29
|
rule(:newline) { str("\r").maybe >> str("\n") }
|
30
30
|
rule(:restofline){ ( newline.absent? >> any ).repeat }
|
31
31
|
|
32
|
-
rule(:path) { match['[:alnum:]
|
32
|
+
rule(:path) { match['[:alnum:]=\+\.\-_/'].repeat(1) }
|
33
33
|
rule(:integer) { match['0-9'].repeat(1) }
|
34
34
|
rule(:float) { integer >> (match['\.,'] >> integer).maybe }
|
35
35
|
|
@@ -68,7 +68,7 @@ module BuildLogParser
|
|
68
68
|
|
69
69
|
def reset()
|
70
70
|
super()
|
71
|
-
|
71
|
+
@data = []
|
72
72
|
end
|
73
73
|
|
74
74
|
def parse(logtext)
|
@@ -78,5 +78,7 @@ module BuildLogParser
|
|
78
78
|
tree = parser.parse(logtext)
|
79
79
|
@data = Dhrystone::Transform.new.apply(tree)
|
80
80
|
end
|
81
|
-
end # class
|
81
|
+
end # class DhrystoneParser
|
82
|
+
|
83
|
+
registerParser(:dhrystone, DhrystoneParser, :parse)
|
82
84
|
end # module BuildLogParser
|
@@ -0,0 +1,105 @@
|
|
1
|
+
#
|
2
|
+
# Copyright (C) 2018 Mario Werner <nioshd@gmail.com>
|
3
|
+
#
|
4
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
5
|
+
# this software and associated documentation files (the "Software"), to deal in
|
6
|
+
# the Software without restriction, including without limitation the rights to
|
7
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
8
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
9
|
+
# subject to the following conditions:
|
10
|
+
#
|
11
|
+
# The above copyright notice and this permission notice shall be included in all
|
12
|
+
# copies or substantial portions of the Software.
|
13
|
+
#
|
14
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
15
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
16
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
17
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
18
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
19
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
20
|
+
#
|
21
|
+
require 'parslet'
|
22
|
+
|
23
|
+
module BuildLogParser
|
24
|
+
module LLDMap
|
25
|
+
class Parser < Parslet::Parser
|
26
|
+
rule(:space) { match['\s'] }
|
27
|
+
rule(:space?) { space.repeat }
|
28
|
+
|
29
|
+
rule(:newline) { str("\r").maybe >> str("\n") }
|
30
|
+
rule(:restofline){ ( newline.absent? >> any ).repeat }
|
31
|
+
|
32
|
+
rule(:path) { match['[:alnum:]=\+\.\-_/<>'].repeat(1) }
|
33
|
+
rule(:integer) { match['0-9'].repeat(1) }
|
34
|
+
rule(:hexnumber) { match['a-fA-F0-9'].repeat(1) }
|
35
|
+
|
36
|
+
rule(:section) { match['[:alnum:]\-_\.'].repeat(1) }
|
37
|
+
rule(:symbol) { match['[:alnum:]_'] >> match['[:alnum:]\-_\.'].repeat }
|
38
|
+
rule(:infile) { }
|
39
|
+
|
40
|
+
rule(:header) do
|
41
|
+
str('Address') >> space? >> str('Size') >> space? >>
|
42
|
+
str('Align') >> space? >> str('Out') >> space? >> str('In') >>
|
43
|
+
space? >> str('Symbol') >> newline
|
44
|
+
end
|
45
|
+
|
46
|
+
rule(:line_prefix) do
|
47
|
+
hexnumber.as(:address) >> space? >> hexnumber.as(:size) >> space? >>
|
48
|
+
integer.as(:align)
|
49
|
+
end
|
50
|
+
|
51
|
+
rule(:sym_entry) do
|
52
|
+
line_prefix >> space? >> symbol.as(:name) >> newline
|
53
|
+
end
|
54
|
+
|
55
|
+
rule(:in_section_entry) do
|
56
|
+
line_prefix >> space? >> path.as(:source_file) >> ( str("(") >>
|
57
|
+
path.as(:source_sub_file) >> str(")") ).maybe >> str(":(") >>
|
58
|
+
section.as(:source_section) >> str(")") >> newline >> sym_entry.repeat.as(:symbols)
|
59
|
+
end
|
60
|
+
|
61
|
+
rule(:out_section_entry) do
|
62
|
+
line_prefix >> match['\s'] >> section.as(:out) >> newline >> in_section_entry.repeat.as(:inputs)
|
63
|
+
end
|
64
|
+
|
65
|
+
rule(:start) { ((header >> out_section_entry.repeat(1)).as(:array) | (restofline >> newline | any).as(:drop)).repeat.as(:array) }
|
66
|
+
root(:start)
|
67
|
+
end # class Parser
|
68
|
+
|
69
|
+
class Transform < Parslet::Transform
|
70
|
+
rule(:drop => subtree(:tree)) { }
|
71
|
+
rule(:address => simple(:address), :size => simple(:size), :align => simple(:align), :name => simple(:name)) do
|
72
|
+
{ :address => String(address).to_i(16), :size => String(size).to_i(16), :align => Integer(align), :name => String(name) }
|
73
|
+
end
|
74
|
+
rule(:address => simple(:address), :size => simple(:size), :align => simple(:align), :source_file => simple(:source_file), :source_section => simple(:source_section), :symbols => subtree(:symbols)) do
|
75
|
+
{ :address => String(address).to_i(16), :size => String(size).to_i(16), :align => Integer(align), :source_file => String(source_file), :source_section => String(source_section), :symbols => symbols }
|
76
|
+
end
|
77
|
+
rule(:address => simple(:address), :size => simple(:size), :align => simple(:align), :source_file => simple(:source_file), :source_sub_file => simple(:source_sub_file), :source_section => simple(:source_section), :symbols => subtree(:symbols)) do
|
78
|
+
{ :address => String(address).to_i(16), :size => String(size).to_i(16), :align => Integer(align), :source_file => String(source_file), :source_sub_file => String(source_sub_file), :source_section => String(source_section), :symbols => symbols }
|
79
|
+
end
|
80
|
+
rule(:address => simple(:address), :size => simple(:size), :align => simple(:align), :out => simple(:out), :inputs => subtree(:inputs)) do
|
81
|
+
{ :address => String(address).to_i(16), :size => String(size).to_i(16), :align => Integer(align), :out => String(out), :inputs => inputs }
|
82
|
+
end
|
83
|
+
rule(:array => subtree(:tree)) { tree.compact.flatten }
|
84
|
+
end # class Transform
|
85
|
+
end # module LLDMap
|
86
|
+
|
87
|
+
class LLDParser < BuildLogParser::Parser
|
88
|
+
attr_reader :data
|
89
|
+
|
90
|
+
def reset()
|
91
|
+
super()
|
92
|
+
@data = []
|
93
|
+
end
|
94
|
+
|
95
|
+
def parseMap(logtext)
|
96
|
+
reset()
|
97
|
+
@logtext = logtext
|
98
|
+
parser = LLDMap::Parser.new
|
99
|
+
tree = parser.parse(logtext)
|
100
|
+
@data = LLDMap::Transform.new.apply(tree)
|
101
|
+
end
|
102
|
+
end # class LLDParser
|
103
|
+
|
104
|
+
registerParser(:lldMap, LLDParser, :parseMap)
|
105
|
+
end # module BuildLogParser
|
@@ -29,7 +29,7 @@ module BuildLogParser
|
|
29
29
|
rule(:newline) { str("\r").maybe >> str("\n") }
|
30
30
|
rule(:restofline){ ( newline.absent? >> any ).repeat }
|
31
31
|
|
32
|
-
rule(:path) { match['[:alnum:]
|
32
|
+
rule(:path) { match['[:alnum:]=\+\.\-_/'].repeat(1) }
|
33
33
|
rule(:integer) { match['0-9'].repeat(1) }
|
34
34
|
rule(:hexnumber) { match['a-fA-F0-9'].repeat(1) }
|
35
35
|
|
@@ -74,4 +74,6 @@ module BuildLogParser
|
|
74
74
|
@data = SizeBerkeleyStdout::Transform.new.apply(tree)
|
75
75
|
end
|
76
76
|
end # class SizeParser
|
77
|
+
|
78
|
+
registerParser(:sizeBerkeleyStdout, SizeParser, :parseBerkeleyStdout)
|
77
79
|
end # module BuildLogParser
|
@@ -0,0 +1,41 @@
|
|
1
|
+
#
|
2
|
+
# Copyright (C) 2017 Mario Werner <nioshd@gmail.com>
|
3
|
+
#
|
4
|
+
# Permission is hereby granted, free of charge, to any person obtaining a copy of
|
5
|
+
# this software and associated documentation files (the "Software"), to deal in
|
6
|
+
# the Software without restriction, including without limitation the rights to
|
7
|
+
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
8
|
+
# the Software, and to permit persons to whom the Software is furnished to do so,
|
9
|
+
# subject to the following conditions:
|
10
|
+
#
|
11
|
+
# The above copyright notice and this permission notice shall be included in all
|
12
|
+
# copies or substantial portions of the Software.
|
13
|
+
#
|
14
|
+
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
15
|
+
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
16
|
+
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
17
|
+
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
18
|
+
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
19
|
+
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
20
|
+
#
|
21
|
+
module BuildLogParser
|
22
|
+
@@parsers = {}
|
23
|
+
|
24
|
+
def self.getParserNames
|
25
|
+
return @@parsers.keys
|
26
|
+
end
|
27
|
+
|
28
|
+
def self.registerParser(parser_name, parser_class, method)
|
29
|
+
@@parsers[parser_name] = [parser_class, method]
|
30
|
+
end
|
31
|
+
|
32
|
+
def self.getParser(parser_name)
|
33
|
+
return nil unless @@parsers.key?(parser_name)
|
34
|
+
return @@parsers[parser_name][0].new
|
35
|
+
end
|
36
|
+
|
37
|
+
def self.parse(parser_name, parser_obj, logtext)
|
38
|
+
return nil unless @@parsers.key?(parser_name)
|
39
|
+
return parser_obj.public_send(@@parsers[parser_name][1],logtext)
|
40
|
+
end
|
41
|
+
end # module BuildLogParser
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: buildlogparser
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mario Werner
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2018-05-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -94,10 +94,39 @@ dependencies:
|
|
94
94
|
- - "~>"
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: '1.8'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: trollop
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - "~>"
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '2.1'
|
104
|
+
type: :runtime
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - "~>"
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '2.1'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: terminal-table
|
113
|
+
requirement: !ruby/object:Gem::Requirement
|
114
|
+
requirements:
|
115
|
+
- - "~>"
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: '1.8'
|
118
|
+
type: :runtime
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - "~>"
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: '1.8'
|
97
125
|
description:
|
98
126
|
email:
|
99
|
-
-
|
100
|
-
executables:
|
127
|
+
- nioshd@gmail.com
|
128
|
+
executables:
|
129
|
+
- buildlogparser
|
101
130
|
extensions: []
|
102
131
|
extra_rdoc_files: []
|
103
132
|
files:
|
@@ -108,6 +137,7 @@ files:
|
|
108
137
|
- LICENSE.txt
|
109
138
|
- README.md
|
110
139
|
- Rakefile
|
140
|
+
- bin/buildlogparser
|
111
141
|
- buildlogparser.gemspec
|
112
142
|
- lib/buildlogparser.rb
|
113
143
|
- lib/buildlogparser/parser.rb
|
@@ -115,7 +145,9 @@ files:
|
|
115
145
|
- lib/buildlogparser/parsers/coremark.rb
|
116
146
|
- lib/buildlogparser/parsers/ctest.rb
|
117
147
|
- lib/buildlogparser/parsers/dhrystone.rb
|
148
|
+
- lib/buildlogparser/parsers/lld.rb
|
118
149
|
- lib/buildlogparser/parsers/size.rb
|
150
|
+
- lib/buildlogparser/registry.rb
|
119
151
|
- lib/buildlogparser/version.rb
|
120
152
|
homepage: https://github.com/niosHD/buildlogparser
|
121
153
|
licenses:
|
@@ -137,7 +169,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
137
169
|
version: '0'
|
138
170
|
requirements: []
|
139
171
|
rubyforge_project:
|
140
|
-
rubygems_version: 2.5.1
|
172
|
+
rubygems_version: 2.5.2.1
|
141
173
|
signing_key:
|
142
174
|
specification_version: 4
|
143
175
|
summary: Collection of various parsers for the extraction of information from build
|