log-analyser 0.1.3.pre.documentation.20201108192110 → 0.1.3.pre.littleimprovements.20201109101712
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +57 -20
- data/lib/parser.rb +8 -3
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: fcc373e60801c889c58051a0d6232beb9dd14fe16fb0048f50eff803048c2418
|
|
4
|
+
data.tar.gz: 722269822aec5c600621d828aa841b78861ea88d92bfa8f0a892a61aee8da323
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 74e0a52bc8c479237418921b8a2ca49b04a771c32d99dca6691485e1e950c0fa85b08fde959e1e026c026eb320da1995d1b55540d7552134bffe554cea0c6a6d
|
|
7
|
+
data.tar.gz: 34eec782db83276dd27cd2410f697a221437609fa0cdc4b6fe69cb18aa965f4e542e75ccee1d807b910d9c59552aa26cca92522271a6f3e4a5a48bbb6c3df6d9
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
+
[](https://app.circleci.com/pipelines/github/DMazzei/log-analyser)
|
|
1
2
|
[](https://coveralls.io/github/DMazzei/log-analyser?branch=master)
|
|
2
3
|
[](https://badge.fury.io/rb/log-analyser)
|
|
3
|
-
[](https://app.circleci.com/pipelines/github/DMazzei/log-analyser)
|
|
4
4
|
|
|
5
5
|
|
|
6
6
|
|
|
@@ -56,14 +56,27 @@ Or install it yourself as:
|
|
|
56
56
|
|
|
57
57
|
#### Gem Usage
|
|
58
58
|
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
59
|
+
```ruby
|
|
60
|
+
#!/usr/bin/env ruby
|
|
61
|
+
|
|
62
|
+
require 'pageviews_log_aggregator'
|
|
63
|
+
|
|
64
|
+
file_path = '/Users/dmazzei/projects/personal/ruby/sp_test/log-analyser/resources/webserver.log'
|
|
65
|
+
log_aggregator = LogAnalyser::PageviewsLogAggregator.new(file_path)
|
|
66
|
+
|
|
67
|
+
puts "\nAll pageviews"
|
|
68
|
+
log_aggregator.all.each do |key, value|
|
|
69
|
+
puts "#{key&.to_s&.ljust(28, '.')} | #{value}"
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
puts "\nUnique pageviews"
|
|
73
|
+
log_aggregator.unique.each do |key, value|
|
|
74
|
+
puts "#{key&.to_s&.ljust(28, '.')} | #{value}"
|
|
75
|
+
end
|
|
65
76
|
```
|
|
66
77
|
|
|
78
|
+
|
|
79
|
+
|
|
67
80
|
### Project
|
|
68
81
|
|
|
69
82
|
Install the Ruby version specified in `.ruby-version` </br>
|
|
@@ -88,12 +101,12 @@ Run the initial setup
|
|
|
88
101
|
|
|
89
102
|
#### Usage
|
|
90
103
|
|
|
91
|
-
Call `./bin/parse_pageview_file.rb` passing a logfile path as argument, will return the pageview count ordered from most to less viewed.</br>
|
|
92
|
-
Check
|
|
104
|
+
Call `./bin/parse_pageview_file.rb` passing a logfile path as argument, it will return the pageview count ordered from most to less viewed.</br>
|
|
105
|
+
Check `--help` for more options
|
|
93
106
|
|
|
94
107
|
|
|
95
108
|
|
|
96
|
-
An example log can be found in
|
|
109
|
+
An example log can be found in :file_folder:`resources` folder:
|
|
97
110
|
|
|
98
111
|
$ ./bin/parse_pageview_file.rb --file 'resources/webserver.log'
|
|
99
112
|
|--------------------------------------------------|
|
|
@@ -147,25 +160,44 @@ A space must separate the page name (first column) from the user identifier (e.g
|
|
|
147
160
|
|
|
148
161
|
#### Start with the project:
|
|
149
162
|
|
|
163
|
+
```
|
|
164
|
+
$ git clone git@github.com:DMazzei/log-analyser.git
|
|
165
|
+
$ cd log-analyser
|
|
166
|
+
$ gem install bundler
|
|
167
|
+
$ bundle install
|
|
168
|
+
```
|
|
150
169
|
|
|
170
|
+
And the world is your oyster...
|
|
151
171
|
|
|
152
|
-
You can also run
|
|
172
|
+
You can also run `$ bundle exec console` for an interactive prompt that will allow you to experiment.
|
|
153
173
|
|
|
154
|
-
To install this gem onto your local machine, run
|
|
155
|
-
To release a new version, update the version number in `version.rb`, and then run
|
|
174
|
+
To install this gem onto your local machine, run `$ bundle exec rake install`.
|
|
175
|
+
To release a new version, update the version number in `version.rb`, and then run `$ bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
156
176
|
|
|
157
177
|
#### Linter (rubocop)
|
|
158
178
|
|
|
159
179
|
_*Rubocop*_ is used as code analyser and maintain code formatting (as well as some best practices).
|
|
160
180
|
|
|
161
|
-
Use
|
|
181
|
+
Use `$ bundle exec rake rubocop` to run the checks.
|
|
162
182
|
|
|
163
183
|
#### Test coverage
|
|
164
184
|
|
|
165
|
-
|
|
185
|
+
[](https://coveralls.io/github/DMazzei/log-analyser?branch=master)
|
|
186
|
+
|
|
187
|
+
Use `$ bundle exec rspec` or `$ bundle exec rake spec:all` to run all the tests.
|
|
188
|
+
|
|
189
|
+
:white_check_mark: To run only unit-tests
|
|
190
|
+
|
|
191
|
+
$ bundle exec rake spec:unit
|
|
166
192
|
|
|
167
|
-
|
|
168
|
-
|
|
193
|
+
:white_check_mark: To run only integration tests
|
|
194
|
+
|
|
195
|
+
$ bundle exec rake spec:integration
|
|
196
|
+
|
|
197
|
+
The test coverage is handled by `rspec`, `simplecov` and `coveralls`.</br>
|
|
198
|
+
After running the tests, a local version of the test coverage report is available [here](http://localhost:63342/log-analyser/coverage/index.html#_AllFiles).
|
|
199
|
+
|
|
200
|
+
Full status and coverage history can be checked online on [coveralls](https://coveralls.io/github/DMazzei/log-analyser).
|
|
169
201
|
|
|
170
202
|
#### Deployment
|
|
171
203
|
|
|
@@ -177,7 +209,7 @@ The _*master branch*_ merge process will trigger the deployment process on Circl
|
|
|
177
209
|
|
|
178
210
|
The whole deployment process will finish by building and tagging a new gem version and pushing it to [rubygems.org](https://rubygems.org/gems/log-analyser).
|
|
179
211
|
|
|
180
|
-
> :warning:
|
|
212
|
+
> :warning: To merge changes into _*master*_, the version must be bumped up, otherwise the deployment will fail!</br>
|
|
181
213
|
> The version must be updated in `version.rb`.
|
|
182
214
|
|
|
183
215
|
## Contributing
|
|
@@ -185,8 +217,13 @@ The whole deployment process will finish by building and tagging a new gem versi
|
|
|
185
217
|
Bug reports and pull requests are welcome on GitHub at https://github.com/DMazzeig/log-analyser.
|
|
186
218
|
|
|
187
219
|
## Next Steps
|
|
188
|
-
|
|
189
|
-
-
|
|
220
|
+
|
|
221
|
+
- One conundrum faced that can be reviewed, deciding between:
|
|
222
|
+
* reading the file whilst aggregation data, preserving memory - e.g. using `Set`;
|
|
223
|
+
* loading data into memory and leaving aggregation and count to be dealt later, gaining flexibility and performance;
|
|
224
|
+
- Extend the accepted logfile format;
|
|
225
|
+
- Add more options for sorting and filtering;
|
|
226
|
+
- Automate library version bump up;
|
|
190
227
|
|
|
191
228
|
## License
|
|
192
229
|
|
data/lib/parser.rb
CHANGED
|
@@ -12,24 +12,29 @@ module LogAnalyser
|
|
|
12
12
|
new.call(file_path)
|
|
13
13
|
end
|
|
14
14
|
|
|
15
|
+
def initialize
|
|
16
|
+
@entries = Hash.new { |h, k| h[k] = [] }
|
|
17
|
+
end
|
|
18
|
+
|
|
15
19
|
def call(file_path)
|
|
16
20
|
raise FileNotFoundError, file_path unless File.exist?(file_path)
|
|
17
21
|
|
|
18
22
|
data = File.open(file_path).map(&:strip)
|
|
19
23
|
data.reject!(&:empty?)
|
|
20
24
|
parse(data)
|
|
25
|
+
|
|
26
|
+
entries
|
|
21
27
|
end
|
|
22
28
|
|
|
23
29
|
private
|
|
24
30
|
|
|
31
|
+
attr_reader :entries
|
|
32
|
+
|
|
25
33
|
def parse(data)
|
|
26
|
-
entries = Hash.new { |h, k| h[k] = [] }
|
|
27
34
|
data.each do |entry|
|
|
28
35
|
key, value = *entry.split(/\s+/)
|
|
29
36
|
entries[key] << value
|
|
30
37
|
end
|
|
31
|
-
|
|
32
|
-
entries
|
|
33
38
|
end
|
|
34
39
|
end
|
|
35
40
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: log-analyser
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.3.pre.
|
|
4
|
+
version: 0.1.3.pre.littleimprovements.20201109101712
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dan Mazzei
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2020-11-
|
|
11
|
+
date: 2020-11-09 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: Log reader and data aggregator for pageviews information.
|
|
14
14
|
email:
|