paxmex 1.0.1 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +54 -2
- data/config/epa.yml +259 -0
- data/lib/paxmex.rb +6 -6
- data/lib/paxmex/parsed_section.rb +25 -0
- data/lib/paxmex/parser.rb +68 -24
- data/lib/paxmex/schema.rb +32 -1
- data/lib/paxmex/schema/field.rb +15 -8
- data/lib/paxmex/schema/section.rb +8 -4
- data/lib/paxmex/version.rb +1 -1
- metadata +10 -9
- data/lib/tasks/paxmex_tasks.rake +0 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: fca1a307922ab3802708452abb03c72c3ae7f278
|
4
|
+
data.tar.gz: 841fb9a2847ac0785c8ee6ab25862294a093a438
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cd732e7e2945b58fd3fe93217439ad0a67194698cdbc9a93d2826d2a837e91e17805acd09caecc6369fecc4a6be296e49a176c1d4dedb7b0b9ec9ff4469ef9c7
|
7
|
+
data.tar.gz: 661e9f998f251c8026b94208031842b60a4933e73f95e91c73bca6626e99e9eccab655784b45b3b3d78a371ce834f716b909fddfc415b52577ea30c654aa2159
|
data/README.md
CHANGED
@@ -16,8 +16,9 @@ This gem parses your Amex data files into human readable data.
|
|
16
16
|
|
17
17
|
* parse_eptrn(file_path)
|
18
18
|
* parse_epraw(file_path)
|
19
|
+
* parse_epa(file_path)
|
19
20
|
|
20
|
-
|
21
|
+
The first two methods return a readable hash in the following format:
|
21
22
|
|
22
23
|
```ruby
|
23
24
|
{
|
@@ -41,12 +42,47 @@ Both methods return a readable hash in the following format:
|
|
41
42
|
}
|
42
43
|
```
|
43
44
|
|
45
|
+
The last method (parse_epa) returns nearly the same, but it contains nested records (according to the schema definition):
|
46
|
+
|
47
|
+
```ruby
|
48
|
+
{
|
49
|
+
"TRAILER_RECORD" => {
|
50
|
+
...
|
51
|
+
},
|
52
|
+
"HEADER_RECORD" => {
|
53
|
+
...
|
54
|
+
},
|
55
|
+
"PAYMENT_SUMMARY" => [
|
56
|
+
{
|
57
|
+
...
|
58
|
+
"SETTLEMENT_SE_ACCOUNT_NUMBER" => "1234567891",
|
59
|
+
"SETTLEMENT_ACCOUNT_NAME_CODE" => "002",
|
60
|
+
:children => {
|
61
|
+
"SUMMARY_OF_CHARGE" => [
|
62
|
+
{
|
63
|
+
...
|
64
|
+
"SOC_DATE" => #<Date: 2014-18-07>,
|
65
|
+
"DISCOUNT_AMOUNT" => #<BigDecimal: '-0.385E1'>,
|
66
|
+
:children => {
|
67
|
+
"RECORD_OF_CHARGE" => [
|
68
|
+
{
|
69
|
+
...
|
70
|
+
"CHARGE_AMOUNT" => #<BigDecimal: '0.135E3'>,
|
71
|
+
"CHARGE_DATE" => #<Date: 2014-17-07>,
|
72
|
+
"6-DIGIT_CHARGE_AUTHORISATION_CODE" => "123456"
|
73
|
+
},
|
74
|
+
...
|
75
|
+
]
|
76
|
+
...
|
77
|
+
```
|
78
|
+
|
44
79
|
Values are parsed from their representation into a corresponding native Ruby type:
|
45
80
|
|
46
81
|
* Alphanumeric values become strings (with whitespace stripped)
|
47
82
|
* Numeric values become Fixnums
|
48
83
|
* Julian strings become Date objects
|
49
84
|
* Date strings become Date objects
|
85
|
+
* Time strings become Time objects
|
50
86
|
* Alphanumerically represented decimals become BigDecimal objects (e.g: ```'0000000011A' -> BigDecimal.new(1.11, 7)```)
|
51
87
|
|
52
88
|
If you'd like the raw values to be returned instead, you can set the ```raw_values``` option to true, e.g.:
|
@@ -54,18 +90,34 @@ If you'd like the raw values to be returned instead, you can set the ```raw_valu
|
|
54
90
|
```ruby
|
55
91
|
Paxmex.parse_eptrn(path_to_file, raw_values: true)
|
56
92
|
Paxmex.parse_epraw(path_to_file, raw_values: true)
|
93
|
+
Paxmex.parse_epa(path_to_file, raw_values: true)
|
57
94
|
```
|
58
95
|
|
96
|
+
## User-defined schema
|
97
|
+
|
98
|
+
If you need to parse a different format (i.e. not EPRAW, EPTRN or EPA), write your own schema definition and use it like this:
|
99
|
+
```ruby
|
100
|
+
parser = Parser.new(path_to_raw_file, path_to_schema_file)
|
101
|
+
result = parser.parse
|
102
|
+
```
|
59
103
|
|
60
104
|
## Example
|
61
105
|
|
62
106
|
```ruby
|
63
107
|
require 'paxmex'
|
108
|
+
|
109
|
+
# Use default schema definitions
|
64
110
|
Paxmex.parse_eptrn('/path/to/amex/eptrn/raw/file')
|
65
111
|
Paxmex.parse_epraw('/path/to/amex/epraw/raw/file')
|
112
|
+
Paxmex.parse_epa('/path/to/amex/epa/raw/file')
|
113
|
+
|
114
|
+
# Use your own schema definition
|
115
|
+
parser = Parser.new('/path/to/raw/file', '/path/to/your/schema.yml')
|
116
|
+
result = parser.parse
|
66
117
|
```
|
67
118
|
|
68
|
-
The input files for either methods
|
119
|
+
The raw input files for either methods are data report files provided by American Express. These files are in either EPRAW, EPTRN or EPA format so use the relevant method to parse them. We have provided dummy EPRAW, EPTRN and EPA files in `spec/support`. Output and key-value pairs vary depending on whether you choose to parse an EPTRN, EPRAW or EPA file.
|
120
|
+
If you need to parse a file in another format, you can write your own YAML schema for this purpose. We would be happy if you help us improving this project by sharing your schemas.
|
69
121
|
|
70
122
|
## Contributing
|
71
123
|
|
data/config/epa.yml
ADDED
@@ -0,0 +1,259 @@
|
|
1
|
+
HEADER_RECORD:
|
2
|
+
FIELDS:
|
3
|
+
- NAME: HEADER_RECORD_TYPE
|
4
|
+
RANGE: [0, 5]
|
5
|
+
TYPE: string
|
6
|
+
- NAME: HEADER_TIME
|
7
|
+
RANGE: [6, 17]
|
8
|
+
TYPE: time(%Y%m%d%H%M)
|
9
|
+
- NAME: HEADER_ID
|
10
|
+
RANGE: [18, 23]
|
11
|
+
TYPE: string
|
12
|
+
- NAME: HEADER_NAME
|
13
|
+
RANGE: [24, 43]
|
14
|
+
TYPE: string
|
15
|
+
DETAIL_RECORD:
|
16
|
+
ABSTRACT: true
|
17
|
+
RECURRING: true
|
18
|
+
TYPE_FIELD: [32, 34]
|
19
|
+
TYPE_MAPPING:
|
20
|
+
'100': PAYMENT_SUMMARY
|
21
|
+
'210': SUMMARY_OF_CHARGE
|
22
|
+
'260': RECORD_OF_CHARGE
|
23
|
+
'230': ADJUSTMENT_RECORD
|
24
|
+
TYPES:
|
25
|
+
PAYMENT_SUMMARY:
|
26
|
+
RECURRING: true
|
27
|
+
FIELDS:
|
28
|
+
- NAME: SETTLEMENT_SE_ACCOUNT_NUMBER
|
29
|
+
RANGE: [0, 9]
|
30
|
+
TYPE: string
|
31
|
+
- NAME: SETTLEMENT_ACCOUNT_NAME_CODE
|
32
|
+
RANGE: [10, 12]
|
33
|
+
TYPE: string
|
34
|
+
- NAME: SETTLEMENT_DATE
|
35
|
+
RANGE: [13, 20]
|
36
|
+
TYPE: date(%Y%m%d)
|
37
|
+
- NAME: SUBMISSION_SE_ACCOUNT_NUMBER
|
38
|
+
RANGE: [22, 31]
|
39
|
+
TYPE: string
|
40
|
+
- NAME: SETTLEMENT_AMOUNT
|
41
|
+
RANGE: [40, 54]
|
42
|
+
TYPE: decimal
|
43
|
+
- NAME: SE_BANK_SORT_CODE
|
44
|
+
RANGE: [55, 69]
|
45
|
+
TYPE: string
|
46
|
+
- NAME: SE_BANK_ACCOUNT_NUMBER
|
47
|
+
RANGE: [70, 89]
|
48
|
+
TYPE: string
|
49
|
+
- NAME: SETTLEMENT_GROSS_AMOUNT
|
50
|
+
RANGE: [90, 104]
|
51
|
+
TYPE: decimal
|
52
|
+
- NAME: TAX_AMOUNT
|
53
|
+
RANGE: [105, 119]
|
54
|
+
TYPE: decimal
|
55
|
+
- NAME: TAX_RATE
|
56
|
+
RANGE: [120, 126]
|
57
|
+
TYPE: decimal
|
58
|
+
- NAME: SERVICE_FEE_AMOUNT
|
59
|
+
RANGE: [127, 141]
|
60
|
+
TYPE: decimal
|
61
|
+
- NAME: SERVICE_FEE_RATE
|
62
|
+
RANGE: [157, 163]
|
63
|
+
TYPE: decimal
|
64
|
+
- NAME: SETTLEMENT_ADJUSTMENT_AMOUNT
|
65
|
+
RANGE: [219, 233]
|
66
|
+
TYPE: decimal
|
67
|
+
- NAME: PAY_PLAN_SHORT_NAME
|
68
|
+
RANGE: [234, 263]
|
69
|
+
TYPE: string
|
70
|
+
- NAME: PAYEE_NAME
|
71
|
+
RANGE: [264, 301]
|
72
|
+
TYPE: string
|
73
|
+
- NAME: SETTLEMENT_ACCOUNT_NAME
|
74
|
+
RANGE: [302, 321]
|
75
|
+
TYPE: string
|
76
|
+
- NAME: SETTLEMENT_CURRENCY_CODE
|
77
|
+
RANGE: [322, 324]
|
78
|
+
TYPE: string
|
79
|
+
- NAME: PREVIOUS_DEBIT_BALANCE
|
80
|
+
RANGE: [325, 339]
|
81
|
+
TYPE: decimal
|
82
|
+
SUMMARY_OF_CHARGE:
|
83
|
+
RECURRING: true
|
84
|
+
PARENT: PAYMENT_SUMMARY
|
85
|
+
FIELDS:
|
86
|
+
- NAME: SETTLEMENT_SE_ACCOUNT_NUMBER
|
87
|
+
RANGE: [0, 9]
|
88
|
+
TYPE: string
|
89
|
+
- NAME: SETTLEMENT_ACCOUNT_NAME_CODE
|
90
|
+
RANGE: [10, 12]
|
91
|
+
TYPE: string
|
92
|
+
- NAME: SETTLEMENT_DATE
|
93
|
+
RANGE: [13, 20]
|
94
|
+
TYPE: date(%Y%m%d)
|
95
|
+
- NAME: SUBMISSION_SE_ACCOUNT_NUMBER
|
96
|
+
RANGE: [22, 31]
|
97
|
+
TYPE: string
|
98
|
+
- NAME: SOC_DATE
|
99
|
+
RANGE: [40, 47]
|
100
|
+
TYPE: date(%Y%m%d)
|
101
|
+
- NAME: SUBMISSION_CALCULATED_GROSS_AMOUNT
|
102
|
+
RANGE: [48, 62]
|
103
|
+
TYPE: decimal
|
104
|
+
- NAME: SUBMISSION_DECLARED_GROSS_AMOUNT
|
105
|
+
RANGE: [63, 77]
|
106
|
+
TYPE: decimal
|
107
|
+
- NAME: DISCOUNT_AMOUNT
|
108
|
+
RANGE: [78, 92]
|
109
|
+
TYPE: decimal
|
110
|
+
- NAME: SETTLEMENT_NET_AMOUNT
|
111
|
+
RANGE: [108, 122]
|
112
|
+
TYPE: decimal
|
113
|
+
- NAME: SERVICE_FEE_RATE
|
114
|
+
RANGE: [123, 128]
|
115
|
+
TYPE: decimal
|
116
|
+
- NAME: SETTLEMENT_GROSS_AMOUNT
|
117
|
+
RANGE: [170, 184]
|
118
|
+
TYPE: decimal
|
119
|
+
- NAME: ROC_CALCULATED_COUNT
|
120
|
+
RANGE: [185, 189]
|
121
|
+
TYPE: decimal
|
122
|
+
- NAME: TERMINAL_ID
|
123
|
+
RANGE: [205, 214]
|
124
|
+
TYPE: string
|
125
|
+
- NAME: SETTLEMENT_TAX_AMOUNT
|
126
|
+
RANGE: [215, 229]
|
127
|
+
TYPE: decimal
|
128
|
+
- NAME: SETTLEMENT_TAX_RATE
|
129
|
+
RANGE: [230, 236]
|
130
|
+
TYPE: decimal
|
131
|
+
- NAME: SUBMISSION_CURRENCY_CODE
|
132
|
+
RANGE: [237, 239]
|
133
|
+
TYPE: string
|
134
|
+
- NAME: SUBMISSION_NUMBER
|
135
|
+
RANGE: [240, 254]
|
136
|
+
TYPE: numeric
|
137
|
+
- NAME: SUBMISSION_SE_BRANCH_NUMBER
|
138
|
+
RANGE: [255, 264]
|
139
|
+
TYPE: string
|
140
|
+
- NAME: SUBMISSION_METHOD_CODE
|
141
|
+
RANGE: [265, 266]
|
142
|
+
TYPE: string
|
143
|
+
- NAME: EXCHANGE_RATE
|
144
|
+
RANGE: [292, 306]
|
145
|
+
TYPE: decimal
|
146
|
+
RECORD_OF_CHARGE:
|
147
|
+
RECURRING: true
|
148
|
+
PARENT: SUMMARY_OF_CHARGE
|
149
|
+
FIELDS:
|
150
|
+
- NAME: SETTLEMENT_SE_ACCOUNT_NUMBER
|
151
|
+
RANGE: [0, 9]
|
152
|
+
TYPE: string
|
153
|
+
- NAME: SETTLEMENT_ACCOUNT_NAME_CODE
|
154
|
+
RANGE: [10, 12]
|
155
|
+
TYPE: string
|
156
|
+
- NAME: SUBMISSION_SE_ACCOUNT_NUMBER
|
157
|
+
RANGE: [22, 31]
|
158
|
+
TYPE: string
|
159
|
+
- NAME: CHARGE_AMOUNT
|
160
|
+
RANGE: [40, 50]
|
161
|
+
TYPE: decimal
|
162
|
+
- NAME: CHARGE_DATE
|
163
|
+
RANGE: [51, 58]
|
164
|
+
TYPE: date(%Y%m%d)
|
165
|
+
- NAME: ROC_REFERENCE_NUMBER
|
166
|
+
RANGE: [59, 70]
|
167
|
+
TYPE: string
|
168
|
+
- NAME: ROC_REFERENCE_NUMBER_CPC
|
169
|
+
RANGE: [71, 85]
|
170
|
+
TYPE: string
|
171
|
+
- NAME: 3-DIGIT_CHARGE_AUTHORISATION_CODE
|
172
|
+
RANGE: [86, 88]
|
173
|
+
TYPE: string
|
174
|
+
- NAME: CARD_MEMBER_ACCOUNT_NUMBER
|
175
|
+
RANGE: [89, 103]
|
176
|
+
TYPE: string
|
177
|
+
- NAME: AIRLINE_TICKET_NUMBER
|
178
|
+
RANGE: [104, 117]
|
179
|
+
TYPE: string
|
180
|
+
- NAME: 6-DIGIT_CHARGE_AUTHORISATION_CODE
|
181
|
+
RANGE: [118, 123]
|
182
|
+
TYPE: string
|
183
|
+
ADJUSTMENT_DETAIL_RECORD:
|
184
|
+
FIELDS:
|
185
|
+
- NAME: SETTLEMENT_SE_ACCOUNT_NUMBER
|
186
|
+
RANGE: [0, 9]
|
187
|
+
TYPE: string
|
188
|
+
- NAME: SETTLEMENT_ACCOUNT_NAME_CODE
|
189
|
+
RANGE: [10, 12]
|
190
|
+
TYPE: string
|
191
|
+
- NAME: SETTLEMENT_DATE
|
192
|
+
RANGE: [13, 20]
|
193
|
+
TYPE: date(%Y%m%d)
|
194
|
+
- NAME: SUBMISSION_SE_ACCOUNT_NUMBER
|
195
|
+
RANGE: [22, 31]
|
196
|
+
TYPE: decimal
|
197
|
+
- NAME: SUPPORTING_REFERENCE_NUMBER
|
198
|
+
RANGE: [40, 50]
|
199
|
+
TYPE: string
|
200
|
+
- NAME: SETTLEMENT_GROSS_AMOUNT
|
201
|
+
RANGE: [51, 65]
|
202
|
+
TYPE: decimal
|
203
|
+
- NAME: SETTLEMENT_DISCOUNT_AMOUNT
|
204
|
+
RANGE: [66, 80]
|
205
|
+
TYPE: decimal
|
206
|
+
- NAME: SETTLEMENT_NET_AMOUNT
|
207
|
+
RANGE: [96, 110]
|
208
|
+
TYPE: decimal
|
209
|
+
- NAME: SERVICE_FEE_RATE
|
210
|
+
RANGE: [111, 117]
|
211
|
+
TYPE: decimal
|
212
|
+
- NAME: SETTLEMENT_TAX_AMOUNT
|
213
|
+
RANGE: [153, 167]
|
214
|
+
TYPE: decimal
|
215
|
+
- NAME: SETTLEMENT_TAX_RATE
|
216
|
+
RANGE: [168, 174]
|
217
|
+
TYPE: decimal
|
218
|
+
- NAME: CARDMEMBER_ACCOUNT_NUMBER
|
219
|
+
RANGE: [190, 204]
|
220
|
+
TYPE: decimal
|
221
|
+
- NAME: ADJUSTMENT_RECORD_CODE
|
222
|
+
RANGE: [205, 214]
|
223
|
+
TYPE: string
|
224
|
+
- NAME: ADJUSTMENT_MESSAGE_DESCRIPTION
|
225
|
+
RANGE: [215, 278]
|
226
|
+
TYPE: string
|
227
|
+
- NAME: SUBMISSION_SE_BRANCH_NUMBER
|
228
|
+
RANGE: [282, 291]
|
229
|
+
TYPE: string
|
230
|
+
- NAME: SUBMISSION_GROSS_AMOUNT
|
231
|
+
RANGE: [292, 306]
|
232
|
+
TYPE: decimal
|
233
|
+
- NAME: SUBMISSION_CURRENCY_CODE
|
234
|
+
RANGE: [307, 309]
|
235
|
+
TYPE: string
|
236
|
+
- NAME: ADJUSTMENT_MESSAGE_REFERENCE
|
237
|
+
RANGE: [310, 324]
|
238
|
+
TYPE: string
|
239
|
+
TRAILER_RECORD:
|
240
|
+
TRAILER: true
|
241
|
+
FIELDS:
|
242
|
+
- NAME: TRAILER_RECORD_TYPE
|
243
|
+
RANGE: [0, 5]
|
244
|
+
TYPE: string
|
245
|
+
- NAME: TRAILER_TIME
|
246
|
+
RANGE: [6, 17]
|
247
|
+
TYPE: time(%Y%m%d%H%M)
|
248
|
+
- NAME: TRAILER_ID
|
249
|
+
RANGE: [18, 23]
|
250
|
+
TYPE: string
|
251
|
+
- NAME: TRAILER_NAME
|
252
|
+
RANGE: [24, 43]
|
253
|
+
TYPE: string
|
254
|
+
- NAME: TRAILER_RECIPIENT_KEY
|
255
|
+
RANGE: [44, 83]
|
256
|
+
TYPE: string
|
257
|
+
- NAME: TRAILER_RECORD_COUNT
|
258
|
+
RANGE: [84, 90]
|
259
|
+
TYPE: numeric
|
data/lib/paxmex.rb
CHANGED
@@ -1,11 +1,11 @@
|
|
1
1
|
module Paxmex
|
2
2
|
require 'paxmex/parser'
|
3
3
|
|
4
|
-
def self.
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
4
|
+
def self.method_missing(method_sym, *arguments, &block)
|
5
|
+
if method_sym.to_s =~ /^parse_(.*)$/
|
6
|
+
Parser.new(arguments.shift, $1).parse(*arguments)
|
7
|
+
else
|
8
|
+
super
|
9
|
+
end
|
10
10
|
end
|
11
11
|
end
|
@@ -0,0 +1,25 @@
|
|
1
|
+
class Paxmex::ParsedSection < Hash
|
2
|
+
attr_reader :section
|
3
|
+
|
4
|
+
def initialize(section, raw_result = {}, &blk)
|
5
|
+
@section = section
|
6
|
+
@children = {}
|
7
|
+
|
8
|
+
super(&blk) if block_given?
|
9
|
+
raw_result.each { |k, v| self[k] = v }
|
10
|
+
end
|
11
|
+
|
12
|
+
def key
|
13
|
+
section.key
|
14
|
+
end
|
15
|
+
|
16
|
+
def children
|
17
|
+
self[:children]
|
18
|
+
end
|
19
|
+
|
20
|
+
def add_child(key, child)
|
21
|
+
self[:children] ||= {}
|
22
|
+
self[:children][key] ||= []
|
23
|
+
self[:children][key] << child
|
24
|
+
end
|
25
|
+
end
|
data/lib/paxmex/parser.rb
CHANGED
@@ -1,17 +1,24 @@
|
|
1
1
|
require 'yaml'
|
2
2
|
require 'paxmex/schema'
|
3
|
+
require 'paxmex/parsed_section'
|
3
4
|
|
4
5
|
class Paxmex::Parser
|
5
|
-
SCHEMATA = %w(epraw eptrn).reduce({}) do |h, fn|
|
6
|
+
SCHEMATA = %w(epraw eptrn epa).reduce({}) do |h, fn|
|
6
7
|
file = File.expand_path("../../config/#{fn}.yml", File.dirname(__FILE__))
|
7
8
|
h.merge(fn => Paxmex::Schema.new(YAML.load(File.open(file))))
|
8
9
|
end
|
9
10
|
|
10
|
-
attr_reader :schema
|
11
|
+
attr_reader :schema, :path
|
11
12
|
|
12
|
-
def initialize(path,
|
13
|
+
def initialize(path, schema)
|
13
14
|
@path = path
|
14
|
-
@
|
15
|
+
@parent_chain = []
|
16
|
+
|
17
|
+
if File.file?(schema)
|
18
|
+
@schema = Paxmex::Schema.new(YAML.load_file(schema))
|
19
|
+
else
|
20
|
+
@schema = SCHEMATA.fetch(schema)
|
21
|
+
end
|
15
22
|
end
|
16
23
|
|
17
24
|
def raw
|
@@ -27,25 +34,17 @@ class Paxmex::Parser
|
|
27
34
|
# to consider it when parsing recurring sections
|
28
35
|
trailer_section = schema.sections.detect(&:trailer?)
|
29
36
|
trailer_content = [content.slice!(-1)]
|
30
|
-
@parsed = parse_section(
|
31
|
-
|
32
|
-
schema.sections.reject(&:trailer?).each do |section|
|
33
|
-
@parsed.merge!(
|
34
|
-
parse_section(
|
35
|
-
content: section.recurring? ? content : [content.slice!(0)],
|
36
|
-
section: section,
|
37
|
-
raw: opts[:raw_values]))
|
38
|
-
end
|
37
|
+
@parsed = parse_section(trailer_section, trailer_content, raw: opts[:raw_values])
|
39
38
|
|
40
|
-
@parsed
|
39
|
+
schema.sections.reject(&:trailer?).each.with_object(@parsed) do |s, o|
|
40
|
+
section_content = s.recurring? ? content : [content.slice!(0)]
|
41
|
+
o.update(parse_section(s, section_content, raw: opts[:raw_values]))
|
42
|
+
end
|
41
43
|
end
|
42
44
|
|
43
45
|
private
|
44
46
|
|
45
|
-
def parse_section(opts = {})
|
46
|
-
raise 'Content must be provided' unless content = opts[:content]
|
47
|
-
raise 'Section must be provided' unless section = opts[:section]
|
48
|
-
|
47
|
+
def parse_section(section, content, opts = {})
|
49
48
|
result = {}
|
50
49
|
abstract_section = section if section.abstract?
|
51
50
|
|
@@ -56,21 +55,66 @@ class Paxmex::Parser
|
|
56
55
|
section = abstract_section.section_for_type(section_type)
|
57
56
|
end
|
58
57
|
|
59
|
-
|
60
|
-
|
61
|
-
p = {}
|
58
|
+
p = Paxmex::ParsedSection.new(section)
|
62
59
|
section.fields.each do |field|
|
63
60
|
raw_value = section_content[field.start..field.final]
|
64
61
|
p[field.name] = opts[:raw] ? raw_value : field.parse(raw_value)
|
65
62
|
end
|
66
63
|
|
67
|
-
if section.
|
68
|
-
|
64
|
+
if section.child?
|
65
|
+
trim_parent_chain(section.parent_key)
|
66
|
+
add_child_result(@parent_chain.last, section, p)
|
69
67
|
else
|
70
|
-
|
68
|
+
trim_parent_chain
|
69
|
+
add_root_result(result, section, p)
|
70
|
+
end
|
71
|
+
|
72
|
+
if @schema.parent_section?(section.key)
|
73
|
+
@parent_chain << p
|
71
74
|
end
|
72
75
|
end
|
73
76
|
|
74
77
|
result
|
75
78
|
end
|
79
|
+
|
80
|
+
# Search right-to-left and compare each parent with the section's parent:
|
81
|
+
# - If the match fails, remove parent from the chain and try the next one
|
82
|
+
# - If the match succeeds, stop iterating
|
83
|
+
#
|
84
|
+
# This makes sure cases like this are handled:
|
85
|
+
# + parent1 # set chain to []
|
86
|
+
# ++ child1 # set chain to ['parent']
|
87
|
+
# +++ grandchild1 # set chain to ['parent', 'child1']
|
88
|
+
# +++ grandchild2
|
89
|
+
# ++ child2 # <- set chain to ['parent'] again
|
90
|
+
# + parent2 # <- set chain to [] again
|
91
|
+
#
|
92
|
+
def trim_parent_chain(key = nil)
|
93
|
+
new_chain = []
|
94
|
+
@parent_chain.each.with_index do |parent, i|
|
95
|
+
if key == parent.key
|
96
|
+
new_chain = @parent_chain[0..i]
|
97
|
+
break
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
@parent_chain = new_chain
|
102
|
+
end
|
103
|
+
|
104
|
+
|
105
|
+
def add_child_result(parent, section, parsed)
|
106
|
+
fail "Orphaned child #{section.key}" if parent.nil?
|
107
|
+
parent.add_child(section.key, parsed)
|
108
|
+
end
|
109
|
+
|
110
|
+
def add_root_result(root, section, parsed)
|
111
|
+
@parent_chain.clear
|
112
|
+
|
113
|
+
if section.recurring?
|
114
|
+
root[section.key] ||= []
|
115
|
+
root[section.key] << parsed
|
116
|
+
else
|
117
|
+
root[section.key] = parsed
|
118
|
+
end
|
119
|
+
end
|
76
120
|
end
|
data/lib/paxmex/schema.rb
CHANGED
@@ -3,13 +3,44 @@ class Paxmex::Schema
|
|
3
3
|
|
4
4
|
def initialize(schema_hash)
|
5
5
|
@schema_hash = schema_hash
|
6
|
+
@parents = []
|
6
7
|
end
|
7
8
|
|
8
9
|
def sections
|
9
|
-
@sections ||= @schema_hash.map { |k, v|
|
10
|
+
@sections ||= @schema_hash.map { |k, v| build_section(k,v) }
|
10
11
|
end
|
11
12
|
|
12
13
|
def to_h
|
13
14
|
@schema_hash
|
14
15
|
end
|
16
|
+
|
17
|
+
def parent_section?(key)
|
18
|
+
sections.any? do |s|
|
19
|
+
if s.abstract?
|
20
|
+
s.sections_for_types.any? { |ss| ss.parent_key == key }
|
21
|
+
else
|
22
|
+
s.parent_key == key
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
27
|
+
private
|
28
|
+
|
29
|
+
def build_section(key, data)
|
30
|
+
section = Section.new(key, data)
|
31
|
+
mark_abstract if section.abstract?
|
32
|
+
add_parent(section) if section.child?
|
33
|
+
|
34
|
+
section
|
35
|
+
end
|
36
|
+
|
37
|
+
def mark_abstract
|
38
|
+
fail 'Cannot have more than one abstract section' if @have_abstract
|
39
|
+
@have_abstract = true
|
40
|
+
end
|
41
|
+
|
42
|
+
def add_parent(section)
|
43
|
+
fail 'Recursive parent definition' if @parents.include?(section.key)
|
44
|
+
@parents << parent
|
45
|
+
end
|
15
46
|
end
|
data/lib/paxmex/schema/field.rb
CHANGED
@@ -12,25 +12,32 @@ class Paxmex::Schema::Field
|
|
12
12
|
end
|
13
13
|
|
14
14
|
def parse(raw_value)
|
15
|
+
date_pattern = /^date\((.+)\)$/
|
16
|
+
time_pattern = /^time\((.+)\)$/
|
17
|
+
|
15
18
|
case type
|
19
|
+
when 'string' then raw_value.rstrip
|
16
20
|
when 'julian' then parse_julian_date(raw_value)
|
17
21
|
when 'date' then Date.strptime(raw_value, '%m%d%Y')
|
18
22
|
when 'numeric' then raw_value.strip.to_i
|
19
23
|
when 'decimal' then parse_decimal(raw_value)
|
20
|
-
|
24
|
+
when date_pattern then Date.strptime(raw_value, date_pattern.match(type).captures.first)
|
25
|
+
when time_pattern then Time.strptime(raw_value, time_pattern.match(type).captures.first)
|
26
|
+
else fail "Could not parse field type #{type}. Supported types: string, julian, date, numeric, decimal, date(format), time(format)"
|
21
27
|
end
|
22
28
|
end
|
23
29
|
|
24
30
|
def parse_decimal(value)
|
31
|
+
# fields _may_ end with a letter
|
32
|
+
unless value.match(/^[0-9]*[0-9A-R{}]$/)
|
33
|
+
fail "Unexpected value '#{value}' for field '#{name}'"
|
34
|
+
end
|
35
|
+
|
25
36
|
is_credit = !!(value =~ /[JKLMNOPQR}]/)
|
26
37
|
value = value.gsub(/[ABCDEFGHIJKLMNOPQR{}]/,
|
27
|
-
'A'
|
28
|
-
'
|
29
|
-
'
|
30
|
-
'J' => 1, 'K' => 2, 'L' => 3,
|
31
|
-
'M' => 4, 'N' => 5, 'O' => 6,
|
32
|
-
'P' => 7, 'Q' => 8, 'R' => 9,
|
33
|
-
'{' => 0, '}' => 0)
|
38
|
+
'A'=>1, 'B'=>2, 'C'=>3, 'D'=>4, 'E'=>5, 'F'=>6, 'G'=>7, 'H'=>8, 'I'=>9,
|
39
|
+
'J'=>1, 'K'=>2, 'L'=>3, 'M'=>4, 'N'=>5, 'O'=>6, 'P'=>7, 'Q'=>8, 'R'=>9,
|
40
|
+
'{'=>0, '}'=>0)
|
34
41
|
|
35
42
|
parsed_value = value.to_i * (is_credit ? -1 : 1) / 100.0
|
36
43
|
BigDecimal.new(parsed_value.to_s, 7)
|
@@ -3,9 +3,10 @@ require 'paxmex/schema/field'
|
|
3
3
|
class Paxmex::Schema::Section
|
4
4
|
BLOCK_LENGTH = 450
|
5
5
|
|
6
|
-
attr_reader :key, :data
|
6
|
+
attr_reader :key, :data, :parent_key
|
7
7
|
|
8
8
|
def initialize(key, data)
|
9
|
+
@parent_key = data.delete('PARENT') { nil }
|
9
10
|
@key = key
|
10
11
|
@data = data
|
11
12
|
end
|
@@ -22,13 +23,18 @@ class Paxmex::Schema::Section
|
|
22
23
|
!!data['TRAILER']
|
23
24
|
end
|
24
25
|
|
26
|
+
def child?
|
27
|
+
!!@parent_key
|
28
|
+
end
|
29
|
+
|
25
30
|
def fields
|
26
31
|
@fields ||= data['FIELDS'].map do |field|
|
27
32
|
Paxmex::Schema::Field.new(
|
28
33
|
name: field['NAME'],
|
29
34
|
start: field['RANGE'].first,
|
30
35
|
final: field['RANGE'].last,
|
31
|
-
type: field['TYPE']
|
36
|
+
type: field['TYPE']
|
37
|
+
)
|
32
38
|
end
|
33
39
|
end
|
34
40
|
|
@@ -52,8 +58,6 @@ class Paxmex::Schema::Section
|
|
52
58
|
BLOCK_LENGTH
|
53
59
|
end
|
54
60
|
|
55
|
-
private
|
56
|
-
|
57
61
|
def sections_for_types
|
58
62
|
@sections_for_types ||= types.map { |k, v| self.class.new(k, v) }
|
59
63
|
end
|
data/lib/paxmex/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: paxmex
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Daryl Yeo
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-
|
12
|
+
date: 2014-07-31 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: rspec
|
@@ -47,18 +47,19 @@ executables: []
|
|
47
47
|
extensions: []
|
48
48
|
extra_rdoc_files: []
|
49
49
|
files:
|
50
|
+
- MIT-LICENSE
|
51
|
+
- README.md
|
52
|
+
- Rakefile
|
53
|
+
- config/epa.yml
|
50
54
|
- config/epraw.yml
|
51
55
|
- config/eptrn.yml
|
56
|
+
- lib/paxmex.rb
|
57
|
+
- lib/paxmex/parsed_section.rb
|
52
58
|
- lib/paxmex/parser.rb
|
59
|
+
- lib/paxmex/schema.rb
|
53
60
|
- lib/paxmex/schema/field.rb
|
54
61
|
- lib/paxmex/schema/section.rb
|
55
|
-
- lib/paxmex/schema.rb
|
56
62
|
- lib/paxmex/version.rb
|
57
|
-
- lib/paxmex.rb
|
58
|
-
- lib/tasks/paxmex_tasks.rake
|
59
|
-
- MIT-LICENSE
|
60
|
-
- Rakefile
|
61
|
-
- README.md
|
62
63
|
homepage: https://github.com/lumoslabs/paxmex
|
63
64
|
licenses: []
|
64
65
|
metadata: {}
|
@@ -78,7 +79,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
78
79
|
version: '0'
|
79
80
|
requirements: []
|
80
81
|
rubyforge_project:
|
81
|
-
rubygems_version: 2.
|
82
|
+
rubygems_version: 2.4.1
|
82
83
|
signing_key:
|
83
84
|
specification_version: 4
|
84
85
|
summary: This gem parses your Amex data files into human readable data.
|
data/lib/tasks/paxmex_tasks.rake
DELETED