ydim 1.1.5 → 1.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0e5f53b42fbdc66ca0b05da68957b0621fef14c1c5123939e663895fc52b094f
4
- data.tar.gz: 0b0377f58780313293f21072e5fea737b2c9fe08a555db9e2498f01954115644
3
+ metadata.gz: cf1f40eabcc24aeef2f6c0e5eb3a1c6d207297dc5e3009aadbf073a9ce07233d
4
+ data.tar.gz: 01ebd91434ee1121c038e1dfd2fc7601f354d0175d08c0c8028055cbcd04c1bf
5
5
  SHA512:
6
- metadata.gz: 18304fab8af6cc62469f6443569ddd0ab667bede93f6d752d9e341cc2e18386542acd86ec056f3d8f40da93418137704f4ef18407b7633986eece916260386a8
7
- data.tar.gz: f7930455ae2ce1ca4c71d979ab605f5c11d7c629e48306519a41f9dd14834c78754ab410fbf5423e60347851cc63e8eacf248d093ed5978a7cfc8eafc6db9940
6
+ metadata.gz: d8c31ab23b9df6679605589ba8a02e742415a0dad1fedc35b1ab307b68694024c622200da569fbaf55f74c24380fd2b19fad9bfdb5440581b4a244a380de00b7
7
+ data.tar.gz: 443601812147a9c983bf692deb8a4fc67c66379a2dccc21bc5eba77afb6789ccd8d9f79d65790d9486bc087223842c3844d28265ae1462707b0a5dc1c8f8fbd3
@@ -20,7 +20,7 @@ jobs:
20
20
  fail-fast: false
21
21
  matrix:
22
22
  os: [ubuntu-latest]
23
- ruby: [ '2.7', '3.0', '3.1', '3.2']
23
+ ruby: ['3.0', '3.1', '3.2']
24
24
  runs-on: ${{ matrix.os }}
25
25
  steps:
26
26
  - uses: actions/checkout@v3
@@ -29,3 +29,5 @@ jobs:
29
29
  ruby-version: ${{ matrix.ruby }}
30
30
  bundler-cache: true # runs 'bundle install' and caches installed gems automatically
31
31
  - run: bundle exec rake test
32
+ env:
33
+ MT_COMPAT: '1'
data/CLAUDE.md ADDED
@@ -0,0 +1,147 @@
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ ## Project
6
+
7
+ `ydim` — ywesee distributed invoice manager. A Ruby gem (GPLv2) providing a DRb daemon that
8
+ stores debitors/invoices in PostgreSQL via ODBA, renders PDF invoices, and mails them.
9
+ Version lives in `lib/ydim/version.rb`; changes are recorded in `History.txt`.
10
+
11
+ ## Commands
12
+
13
+ ```bash
14
+ bundle install # gems come from ydim.gemspec via Gemfile
15
+ MT_COMPAT=1 bundle exec rake test # full suite (test/suite.rb); this is what CI runs
16
+ MT_COMPAT=1 bundle exec ruby -Ilib -Itest test/test_invoice.rb # a single test file
17
+ MT_COMPAT=1 bundle exec ruby -Ilib -Itest test/test_invoice.rb -n test_add_item # one test
18
+ bundle exec rake # default: clobber + test + build gem into pkg/
19
+ ```
20
+
21
+ `MT_COMPAT=1` is required — the tests use `flexmock/test_unit`, which needs minitest's
22
+ `Minitest::Unit::TestCase` compatibility shim. Without it (or with a too-new
23
+ flexmock/minitest) loading a test file dies with `undefined method 'teardown'`.
24
+ CI (`.github/workflows/ruby.yml`) runs Ruby 3.0/3.1/3.2 on ubuntu; `.travis.yml` is dead.
25
+ `Gemfile.lock` is gitignored — a stale one in the working tree will make `bundle exec` fail
26
+ with `Bundler::GemNotFound`; delete it and re-run `bundle install`.
27
+
28
+ `rake spec` exists (RSpec task) but there are no spec files.
29
+
30
+ ## Architecture
31
+
32
+ Client/server over **DRb**, authenticated by DSA key challenge (`rrba`), persisted with
33
+ **ODBA** (object database over `ydbi`/`ydbd-pg` → PostgreSQL).
34
+
35
+ **Server side** (`lib/ydim/ydimd` is the daemon entry point, not `bin/`):
36
+ `ydimd` wires `ODBA.storage.dbi` to a connection pool, installs `ODBA::DRbIdConv`, then
37
+ serves a `YDIM::Server` over `druby://`. `Server#initialize` builds a **Needle registry**
38
+ (`@serv`) — the service locator threaded through nearly every class. Registered services:
39
+ `:auth_server`, `:clients`, `:config`, `:currency_converter`, `:factory`, `:id_server`,
40
+ `:logger`. Anything that takes a `serv` argument (`Factory`, `AutoInvoicer`,
41
+ `CurrencyUpdater`) reads its collaborators from this registry, so tests mock `serv` rather
42
+ than the individual dependencies.
43
+
44
+ `Server` also spawns daily background threads via `repeat_at(hour)`: AutoInvoicer,
45
+ CurrencyUpdater, and a StatusUpdater that re-saves every invoice so derived `status` stays
46
+ current in the DB.
47
+
48
+ **Session/API surface**: `Server#login` → `RootUser#new_session` → `RootSession` wrapped in
49
+ `ODBA::DRbWrapper`. **`RootSession` is the entire remote API** — every method a client can
50
+ call (`create_invoice`, `add_items`, `send_invoice`, `debitors`, `collect_garbage`,
51
+ `mark_paid`, `reconcile_camt`, …) is a public method there. Adding a client-callable
52
+ operation means adding it to `RootSession`. `Client#method_missing` forwards everything to
53
+ the session, so clients need no stubs.
54
+
55
+ **Domain**: `Debitor` (1→n `Invoice` and `AutoInvoice`) → `Item`. `Invoice#status` is
56
+ computed, not stored (`is_trash` / `is_paid` / `is_due` / `is_open`). `AutoInvoice` is a
57
+ recurring template: `AutoInvoicer#run` walks all debitors daily and, when `auto.date` is
58
+ today, calls `Factory#generate_invoice` to materialise a real `Invoice` from it (copying
59
+ items, stamping `expiry_time`, applying the current `vat_rate`) and mails it; a month ahead
60
+ it sends a reminder instead. `Factory` is the only place invoice IDs are allocated
61
+ (`id_server.next_id(:invoice, config.invoice_number_start)`).
62
+
63
+ **Persistence is declared in one file**: `lib/ydim/odba.rb` reopens the domain classes to
64
+ include `ODBA::Persistable`, list `ODBA_SERIALIZABLE` ivars, and declare `odba_index`
65
+ (e.g. `Debitor` by email/name/unique_id, `Invoice` by status/unique_id). Those indexes are
66
+ what power `find_by_unique_id` / `search_by_status` / `search_by_exact_email` used in
67
+ `RootSession`. New persisted classes or lookups must be registered here — and each
68
+ `odba_index` needs a matching `ydim_<class>_<attr>` table, see `set_initial_ydim_db.sql`.
69
+
70
+ **VAT**: `config.vat_rate` (currently 8.1, in `server_config.rb`) is applied when items are
71
+ added (`RootSession#add_items`), when autoinvoices are materialised (`Factory`), and reset
72
+ by `Invoice#suppress_vat=`. `Debitor#foreign?` (country != `config.home_country`) makes
73
+ `Factory` suppress VAT at creation time. Changing the rate touches all of these plus the
74
+ `texts.tax` string in pdfinvoice config.
75
+
76
+ **Payment reconciliation**: `lib/ydim/camt.rb` parses ISO-20022 camt.052/053/054 statements
77
+ (rexml only, no DB — the namespace is read off the document root, so any minor version
78
+ works), and `lib/ydim/reconciler.rb` matches booked credits against invoices. The CLI
79
+ (`ydim-camt`) parses client-side and sends `Camt::Entry` objects to
80
+ `RootSession#reconcile_camt`, so the daemon never touches the files. Results carry
81
+ `Reconciler::InvoiceRef`, not `Invoice::Info`, so a client can unmarshal them with
82
+ `ydim/reconciler` alone instead of loading the whole server.
83
+
84
+ Three rules that the real UBS data forces and that are easy to break:
85
+ - **Filter by IBAN.** An e-banking download holds every account the login sees, private ones
86
+ included; `config.camt_accounts` says which are ydim's, and `Reconciler#reconcile` raises
87
+ rather than run without it.
88
+ - **Dedup by `AcctSvcrRef`.** UBS re-sends the same day under a new `MsgId`; in the sample
89
+ set 60 of 142 entries were redeliveries.
90
+ - **Whole digit runs only** (`Camt::Entry::TOKEN_PATTERN`) — never a substring, and never a
91
+ run touching a letter. TWINT credits carry the payer's phone number, and bank
92
+ `EndToEndId`s are hex that happens to contain 5-digit sequences.
93
+
94
+ Only `:exact` and `:split` matches (invoice named *and* amount equal to the cent) are ever
95
+ applied; everything else is reported for review.
96
+
97
+ **PDF**: `lib/pdfinvoice/` is a vendored sub-library (PDF::Writer based) with its own config;
98
+ `Invoice#pdf_invoice` maps ydim items onto it and overrides `formats`/`texts.tax` per invoice.
99
+
100
+ **Mail**: `lib/ydim/mail.rb` configures `::Mail.defaults` with SMTP settings at load time.
101
+
102
+ ## Configuration
103
+
104
+ Three independent `rclconf` config objects, all merging ARGV over defaults:
105
+
106
+ | Object | Defaults defined in | YAML read from |
107
+ | --- | --- | --- |
108
+ | `YDIM::Server.config` (daemon) | `lib/ydim/server_config.rb` | `/etc/ydim/ydimd.yml` |
109
+ | `YDIM::Client::CONFIG` | `lib/ydim/config.rb` | `/etc/ydim/ydim.yml` |
110
+ | `PdfInvoice.config` | `lib/pdfinvoice/config.rb` | `~/.pdfinvoice/config.yml`, `/etc/pdfinvoice/config.yml` |
111
+
112
+ Gotcha: `server_config.rb` only builds `CONFIG` with defaults — the YAML file is actually
113
+ loaded by the `config.load(config.config)` call at the top of `lib/ydim/mail.rb`, which is
114
+ why `mail` is required early in the daemon's require chain. Don't reorder those requires.
115
+ Since 1.1.4/1.1.5 all server paths are rooted at `/etc/ydim` by design; keep them there.
116
+
117
+ ## Executables
118
+
119
+ `bin/ydim-edit` and `bin/ydim-inject` are **byte-identical copies** of `lib/ydim/ydim-edit`
120
+ and `lib/ydim/ydim-inject` (the gemspec takes executables from `bin/`, but the working
121
+ copies live in `lib/ydim/` — see History.txt 1.0.3). Edit both, or the gem and the checkout
122
+ diverge. Note `ydim-edit` carries its own duplicated defaults hash, separate from
123
+ `server_config.rb`.
124
+
125
+ - `ydim-edit` — IRB console with a live `$server` / `$needle` (Needle registry) against the DB.
126
+ - `ydim-inject` — reads a YAML invoice on stdin, creates and mails it through a `Client`.
127
+ - `ydim-camt` — reconciles a camt.053 zip/directory/file against the open invoices; reports
128
+ by default, books only with `--apply`. Config overrides are RCLConf's `key=value` form,
129
+ not `--key value`, so the OptionParser call filters those out of the file arguments.
130
+ - `ydim_migrate_to_utf_8` — one-off LATIN1→UTF-8 DB migration (repo root, not `bin/`).
131
+ - `get_db_ydim` — shell script pulling a nightly Postgres dump and restoring it locally.
132
+
133
+ `install.rb` is a generated setup.rb-style installer; `Manifest.txt` is stale (lists
134
+ `bin/ydimd`, `lib/ydim/smtp_tls.rb`, `README.txt` — none exist).
135
+
136
+ ## Testing conventions
137
+
138
+ Minitest + FlexMock, no DB. `test/stub/odba.rb` replaces `ODBA.transaction` with a plain
139
+ yield and `odba_store` with a counter, so tests that touch persistence require it
140
+ (`require 'stub/odba'`) *before* the class under test. Tests mock the Needle registry with a
141
+ bare `FlexMock` and stub `serv.config` / `serv.logger` individually. `test/suite.rb` just
142
+ globs `test_*.rb`; SimpleCov is present but disabled (`if false`).
143
+
144
+ ## Style
145
+
146
+ Existing code is hard-tabbed in the older files and two-space indented in the newer ones,
147
+ often mixed within a single file. Match the surrounding block rather than reformatting.
data/History.txt CHANGED
@@ -1,3 +1,15 @@
1
+ === 1.1.6 / 09.08.2026
2
+
3
+ * Reconcile payments from ISO-20022 camt.053 bank statements (UBS Z53)
4
+ * ydim-camt reads a statement zip, directory or single xml and reports
5
+ which open invoices the credits settle; --apply books them
6
+ * RootSession#reconcile_camt and #mark_paid make this available over DRb
7
+ * new config camt_accounts: the IBANs ydim invoices are paid into. The
8
+ e-banking download also contains the private accounts, and reconciling
9
+ without this set is refused rather than guessed
10
+ * Pin minitest below 6, which dropped the Minitest::Unit::TestCase shim that
11
+ flexmock/test_unit needs
12
+
1
13
  === 1.1.5/ 21.01.2023
2
14
 
3
15
  * Read ALL configuration values only from /etc/ydim
data/bin/ydim-camt ADDED
@@ -0,0 +1,120 @@
1
+ #!/usr/bin/env ruby
2
+ # encoding: utf-8
3
+ # ydim-camt -- ydim -- 09.08.2026 -- zdavatz@ywesee.com
4
+ #
5
+ # Reconciles bank statements (ISO-20022 camt.053, as downloaded from UBS
6
+ # e-banking) against the open invoices. Reports by default; only books a
7
+ # payment when told to with --apply, and even then only when the payer named
8
+ # the invoice and the amount matches to the cent.
9
+ #
10
+ # ydim-camt ~/Downloads/statements.zip
11
+ # ydim-camt --apply ~/Downloads/statements.zip
12
+ # ydim-camt --account CH87... /var/ydim/camt/
13
+
14
+ require 'openssl'
15
+ require 'optparse'
16
+ require 'rrba/error'
17
+ require 'ydim/camt'
18
+ require 'ydim/client'
19
+ require 'ydim/config'
20
+ require 'ydim/reconciler'
21
+
22
+ options = { :apply => false, :verbose => false, :accounts => [] }
23
+ parser = OptionParser.new { |opt|
24
+ opt.banner = "Usage: ydim-camt [options] <statements.zip|directory|file.xml>"
25
+ opt.on('--apply', 'mark unambiguously matched invoices as paid') {
26
+ options[:apply] = true }
27
+ opt.on('--account IBAN', 'account to reconcile (repeatable); ',
28
+ 'defaults to camt_accounts from the config') { |iban|
29
+ options[:accounts].push(iban) }
30
+ opt.on('-v', '--verbose', 'also list credits that matched nothing') {
31
+ options[:verbose] = true }
32
+ opt.on('-h', '--help') { puts opt; exit }
33
+ }
34
+ # Every ydim executable also takes its configuration on the command line, in
35
+ # RCLConf's "key=value" form -- those are not statements to read, they were
36
+ # already picked up from ARGV when ydim/config was required.
37
+ paths = parser.parse(ARGV.dup).reject { |arg| arg =~ /\A[\w.]+=/ }
38
+ if paths.empty?
39
+ warn parser
40
+ exit 1
41
+ end
42
+ missing = paths.reject { |path| File.exist?(path) }
43
+ unless missing.empty?
44
+ warn "no such file: #{missing.join(', ')}"
45
+ exit 1
46
+ end
47
+
48
+ entries = paths.flat_map { |path| YDIM::Camt.entries(path) }
49
+ if entries.empty?
50
+ warn "no camt entries found in #{paths.join(', ')}"
51
+ exit 1
52
+ end
53
+
54
+ config = YDIM::Client::CONFIG
55
+ accounts = options[:accounts]
56
+ accounts = [config.camt_accounts].flatten.compact if accounts.empty?
57
+ if accounts.empty?
58
+ warn "no account to reconcile: pass --account IBAN, or set camt_accounts " \
59
+ "in the config to the IBAN of the ywesee business account. The private " \
60
+ "accounts in the same download must not be matched against invoices."
61
+ exit 1
62
+ end
63
+
64
+ def show(match, marker)
65
+ ids = match.invoice_ids
66
+ printf(" %-11s %10.2f %s %s %s\n", ids.first || '?',
67
+ match.entry.amount.to_f, match.entry.currency,
68
+ match.entry.booking_date, match.entry.counterparty.to_s[0, 30])
69
+ ids[1..-1].to_a.each { |id| printf(" %-11s\n", id) }
70
+ puts " #{' ' * 11} #{marker} #{match.reason}"
71
+ end
72
+
73
+ server = DRb::DRbObject.new(nil, config.server_url)
74
+ client = YDIM::Client.new(config)
75
+ key = OpenSSL::PKey::DSA.new(File.read(config.private_key))
76
+
77
+ DRb.start_service
78
+ client.login(server, key)
79
+ begin
80
+ result = client.reconcile_camt(entries,
81
+ :apply => options[:apply], :accounts => accounts)
82
+
83
+ applicable = result.applicable
84
+ puts(options[:apply] ? "BOOKED AS PAID" \
85
+ : "MATCHED (would be marked paid with --apply)")
86
+ if applicable.empty?
87
+ puts " none"
88
+ else
89
+ applicable.each { |match| show(match, match.applied? ? '+' : '~') }
90
+ end
91
+
92
+ review = result.review
93
+ unless review.empty?
94
+ puts
95
+ puts "REVIEW (never booked automatically)"
96
+ review.each { |match| show(match, '?') }
97
+ end
98
+
99
+ unmatched = result.unmatched
100
+ if options[:verbose] && !unmatched.empty?
101
+ puts
102
+ puts "UNMATCHED CREDITS"
103
+ unmatched.each { |match| show(match, '-') }
104
+ end
105
+
106
+ puts
107
+ summary = ["#{applicable.size} matched", "#{review.size} to review",
108
+ "#{unmatched.size} unmatched"]
109
+ summary.push("#{result.applied.size} booked") if options[:apply]
110
+ result.skipped.sort_by { |reason, count| reason.to_s }.each { |reason, count|
111
+ summary.push("#{count} #{reason.to_s.tr('_', ' ')}") if count > 0
112
+ }
113
+ puts summary.join(', ')
114
+ unless options[:apply] || applicable.empty?
115
+ puts "run again with --apply to mark the #{applicable.size} matched " \
116
+ "invoice(s) paid"
117
+ end
118
+ ensure
119
+ client.logout
120
+ end
data/bin/ydim-edit CHANGED
@@ -44,7 +44,7 @@ defaults = {
44
44
  'root_key' => 'root_dsa',
45
45
  'smtp_from' => '',
46
46
  'smtp_server' => 'localhost',
47
- 'vat_rate' => 8.0,
47
+ 'vat_rate' => 8.1,
48
48
  }
49
49
  config = RCLConf::RCLConf.new(ARGV, defaults)
50
50
  config.load(config.config)
data/lib/ydim/camt.rb ADDED
@@ -0,0 +1,219 @@
1
+ #!/usr/bin/env ruby
2
+ # encoding: utf-8
3
+ # YDIM::Camt -- ydim -- 09.08.2026 -- zdavatz@ywesee.com
4
+
5
+ require 'date'
6
+ require 'fileutils'
7
+ require 'rexml/document'
8
+ require 'tmpdir'
9
+
10
+ module YDIM
11
+ # Reader for ISO-20022 bank-to-customer statements (camt.052/053/054), as
12
+ # delivered by UBS and the other Swiss banks. This is pure parsing: no DB and
13
+ # no ODBA, so it can run on the client side. The matching of entries against
14
+ # invoices lives in YDIM::Reconciler.
15
+ module Camt
16
+ class Error < StandardError; end
17
+
18
+ # A single booked movement on an account (a camt <Ntry>).
19
+ class Entry
20
+ attr_accessor :account_iban, :account_currency, :amount, :currency,
21
+ :credit, :status, :booking_date, :value_date, :reference,
22
+ :counterparty, :additional_info, :source
23
+ attr_reader :remittance, :end_to_end_ids, :creditor_references
24
+ def initialize
25
+ @remittance = []
26
+ @end_to_end_ids = []
27
+ @creditor_references = []
28
+ @credit = false
29
+ end
30
+ def credit?
31
+ @credit
32
+ end
33
+ # ydim invoices are only ever paid into the ywesee business account --
34
+ # the private accounts share the same e-banking download, so entries have
35
+ # to be filtered by IBAN before anything is matched. An empty filter
36
+ # accepts everything here; refusing to run without one is the
37
+ # Reconciler's job, since only it knows the entries are about to be
38
+ # matched against invoices.
39
+ def account?(ibans)
40
+ ibans = [ibans].flatten.compact
41
+ ibans.empty? || ibans.any? { |iban|
42
+ normalize_iban(iban) == normalize_iban(@account_iban)
43
+ }
44
+ end
45
+ # Only BOOK entries have actually hit the account; PDNG ones can still
46
+ # disappear, so they must never mark an invoice as paid.
47
+ def booked?
48
+ @status.nil? || @status == 'BOOK'
49
+ end
50
+ # Money as an integer so that amounts can be compared without float
51
+ # rounding surprises.
52
+ def amount_cents
53
+ (@amount.to_f * 100).round
54
+ end
55
+ # Every string a payer might have put an invoice number into.
56
+ def texts
57
+ (@remittance + @end_to_end_ids + @creditor_references \
58
+ + [@additional_info]).compact
59
+ end
60
+ # Runs of digits found in the remittance information. Whole runs only --
61
+ # never substrings, or the phone number in a TWINT payment
62
+ # ("+41796723413") would match half the invoices in the database. A run
63
+ # touching a letter is machine noise rather than something a human typed:
64
+ # this drops the hex EndToEndIds that banks generate
65
+ # ("0ebf22116f364fc394da3e2776a05643" would otherwise offer up "22116"),
66
+ # while "RG 13363 VOM 26.6.2026" and "ISO 13368" still yield their
67
+ # invoice number.
68
+ TOKEN_PATTERN = /(?<![0-9A-Za-z])\d+(?![0-9A-Za-z])/
69
+ def numeric_tokens
70
+ texts.flat_map { |text| text.scan(TOKEN_PATTERN) }.uniq
71
+ end
72
+ # Two statements can deliver the same entry (UBS re-sends a day under a
73
+ # new MsgId), so entries are identified by the bank's own reference.
74
+ def dedup_key
75
+ [@account_iban, @reference || texts.join('|'), amount_cents, @credit,
76
+ @booking_date]
77
+ end
78
+ def to_s
79
+ sprintf("%s %s %s %s %s", @booking_date, @credit ? 'CRDT' : 'DBIT',
80
+ @currency, sprintf('%.2f', @amount.to_f), @counterparty)
81
+ end
82
+ private
83
+ def normalize_iban(iban)
84
+ iban.to_s.gsub(/\s+/, '').upcase
85
+ end
86
+ end
87
+
88
+ # A camt <Stmt> -- one account over one period.
89
+ class Statement
90
+ attr_accessor :id, :account_iban, :currency, :owner, :created_at,
91
+ :opening_balance, :closing_balance, :source
92
+ attr_reader :entries
93
+ def initialize
94
+ @entries = []
95
+ end
96
+ end
97
+
98
+ class << self
99
+ # Reads whatever the bank handed you: a single xml file, a directory of
100
+ # them, or the zip you downloaded from e-banking. Returns Statements.
101
+ def read(path)
102
+ if File.directory?(path)
103
+ read_files(Dir[File.join(path, '**', '*.[xX][mM][lL]')].sort)
104
+ elsif path =~ /\.zip\z/i
105
+ read_zip(path)
106
+ else
107
+ read_files([path])
108
+ end
109
+ end
110
+ def read_files(paths)
111
+ paths.flat_map { |path|
112
+ parse(File.read(path), File.basename(path))
113
+ }
114
+ end
115
+ def read_zip(path)
116
+ Dir.mktmpdir('ydim-camt') { |dir|
117
+ unless system('unzip', '-q', '-o', path, '-d', dir)
118
+ raise Error,
119
+ "unable to unzip #{path} -- extract it and pass the directory"
120
+ end
121
+ read_files(Dir[File.join(dir, '**', '*.[xX][mM][lL]')].sort)
122
+ }
123
+ end
124
+ # Parses one camt document into its Statements.
125
+ def parse(xml, source = nil)
126
+ doc = REXML::Document.new(xml)
127
+ root = doc.root or raise Error, "#{source}: not an XML document"
128
+ # camt.053.001.02 through .08 differ only in the namespace URI for
129
+ # everything we read, so take whatever the document declares.
130
+ ns = { 'c' => root.namespace }
131
+ REXML::XPath.match(doc, '//c:Stmt | //c:Rpt | //c:Ntfctn', ns).collect { |node|
132
+ parse_statement(node, ns, source)
133
+ }
134
+ end
135
+ # All entries of all statements below path, duplicates included -- the
136
+ # Reconciler drops those, and it can only report how many deliveries were
137
+ # doubled up if it gets to see them.
138
+ def entries(path)
139
+ read(path).flat_map { |stmt| stmt.entries }
140
+ end
141
+ def dedup(entries)
142
+ seen = {}
143
+ entries.select { |entry| !seen.key?(entry.dedup_key) \
144
+ && seen[entry.dedup_key] = true }
145
+ end
146
+
147
+ private
148
+ def parse_statement(node, ns, source)
149
+ stmt = Statement.new
150
+ stmt.source = source
151
+ stmt.id = text(node, 'c:Id', ns)
152
+ stmt.account_iban = text(node, 'c:Acct/c:Id/c:IBAN', ns)
153
+ stmt.currency = text(node, 'c:Acct/c:Ccy', ns)
154
+ stmt.owner = text(node, 'c:Acct/c:Ownr/c:Nm', ns)
155
+ stmt.created_at = date(text(node, 'c:CreDtTm', ns))
156
+ stmt.opening_balance = balance(node, ns, 'OPBD')
157
+ stmt.closing_balance = balance(node, ns, 'CLBD')
158
+ REXML::XPath.each(node, 'c:Ntry', ns) { |entry_node|
159
+ stmt.entries.push(parse_entry(entry_node, ns, stmt))
160
+ }
161
+ stmt
162
+ end
163
+ def parse_entry(node, ns, stmt)
164
+ entry = Entry.new
165
+ entry.source = stmt.source
166
+ entry.account_iban = stmt.account_iban
167
+ entry.account_currency = stmt.currency
168
+ amount = REXML::XPath.first(node, 'c:Amt', ns)
169
+ entry.amount = amount && amount.text.to_f
170
+ entry.currency = (amount && amount.attributes['Ccy']) || stmt.currency
171
+ entry.credit = text(node, 'c:CdtDbtInd', ns) == 'CRDT'
172
+ entry.status = text(node, 'c:Sts/c:Cd', ns) || text(node, 'c:Sts', ns)
173
+ entry.booking_date = date(text(node, 'c:BookgDt/c:Dt', ns) \
174
+ || text(node, 'c:BookgDt/c:DtTm', ns))
175
+ entry.value_date = date(text(node, 'c:ValDt/c:Dt', ns) \
176
+ || text(node, 'c:ValDt/c:DtTm', ns))
177
+ entry.reference = text(node, 'c:AcctSvcrRef', ns) \
178
+ || text(node, 'c:NtryRef', ns)
179
+ entry.additional_info = text(node, 'c:AddtlNtryInf', ns)
180
+ # The party on the other side: for money coming in that is the debtor,
181
+ # for money going out the creditor. Either can be given as a name or as
182
+ # loose address lines.
183
+ side = entry.credit? ? 'Dbtr' : 'Cdtr'
184
+ entry.counterparty = collect(node,
185
+ ".//c:#{side}/c:Pty/c:Nm | .//c:#{side}/c:Pty/c:PstlAdr/c:AdrLine",
186
+ ns).first
187
+ # An entry can bundle several transactions (a batch booking), so the
188
+ # remittance information of all of them counts.
189
+ entry.remittance.concat(collect(node, './/c:RmtInf/c:Ustrd', ns))
190
+ entry.remittance.concat(collect(node, './/c:Strd/c:AddtlRmtInf', ns))
191
+ entry.end_to_end_ids.concat(collect(node, './/c:EndToEndId', ns)\
192
+ .reject { |id| id == 'NOTPROVIDED' })
193
+ entry.creditor_references.concat(
194
+ collect(node, './/c:CdtrRefInf/c:Ref', ns))
195
+ entry
196
+ end
197
+ def balance(node, ns, code)
198
+ amount = REXML::XPath.first(node,
199
+ "c:Bal[c:Tp/c:CdOrPrtry/c:Cd='#{code}']/c:Amt", ns)
200
+ amount && amount.text.to_f
201
+ end
202
+ def collect(node, path, ns)
203
+ REXML::XPath.match(node, path, ns).collect { |el|
204
+ el.text.to_s.strip
205
+ }.reject { |str| str.empty? }.uniq
206
+ end
207
+ def text(node, path, ns)
208
+ el = REXML::XPath.first(node, path, ns)
209
+ str = el && el.text.to_s.strip
210
+ str unless str.nil? || str.empty?
211
+ end
212
+ def date(str)
213
+ Date.parse(str) if str
214
+ rescue ArgumentError
215
+ nil
216
+ end
217
+ end
218
+ end
219
+ end
data/lib/ydim/config.rb CHANGED
@@ -13,6 +13,11 @@ module YDIM
13
13
  File.join(ydim_default_dir, 'ydim.yml'),
14
14
  ]
15
15
  defaults = {
16
+ # IBANs of the accounts ydim invoices are paid into -- see
17
+ # camt_accounts in server_config.rb. ydim-camt reconciles these and
18
+ # nothing else, so that the private accounts in the same e-banking
19
+ # download are left alone.
20
+ 'camt_accounts' => [],
16
21
  'client_url' => 'druby://localhost:0',
17
22
  'config' => default_config_files,
18
23
  'private_key' => File.join(home_dir, '.ssh', 'id_dsa'),