blz 0.2.0.20180305 → 0.2.0.20180604

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 23d2e9f80c8d7f92347738f5df1997372ea32617
4
- data.tar.gz: 829f4a0f0ace3c8929a94f845c14975a2b7e7ae2
3
+ metadata.gz: f56ab0ea1e3baa5ffbd5cc9fac34376f96949a00
4
+ data.tar.gz: 2d761008414baa0d5ba0cd324a11652c648f144a
5
5
  SHA512:
6
- metadata.gz: 20302a1fd1d0eb6b1944b116798cbc41b1c38dc173b07faf313b7be934aab3f9e3bb276aef2daca68ba08d223ef7bfd88ac366de715c438f021e1c65cf7c22a7
7
- data.tar.gz: 285e92bb3165ad5b48ecedb4f665c84693cfc541f2dd6797dadefb079beaf40fb6cad27431fbcb51e268dd61d713757efe0a49a2f66a39eb33484d09c83896ae
6
+ metadata.gz: e38a7c35894455fc7439692292bdc64b570d52cdc42336f8a8c4963cdf80e28bc51abc7d4d96a2ef6fa97fd983656fdaa4a268d477a3708cbac4e0d37f4844ed
7
+ data.tar.gz: 55f222f01e38faa765c81da67de9fb24c7a6af4d404ed2398d1be257a1eaefdb7941709af9915e396793d81bf817d8b4243689df93ebae133321332f1b25cfdd
@@ -1,8 +1,11 @@
1
+ # 0.2.0.20180604, released 2018-05-04
2
+
3
+ * BLZ data file for cycle 2018-06-04 to 2018-08-02 added
4
+
1
5
  # 0.2.0.20180305, released 2018-03-16
2
6
 
3
7
  * BLZ data file for cycle 2018-03-05 to 2018-06-03 added
4
8
 
5
-
6
9
  # 0.2.0.20171204, released 2017-05-08
7
10
 
8
11
  * BLZ data file for cycle 2017-12-04 to 2018-03-04 added
data/Gemfile CHANGED
@@ -6,3 +6,7 @@ group :test do
6
6
  gem 'test-unit'
7
7
  end
8
8
 
9
+ group :development do
10
+ gem "mechanize"
11
+ gem "nokogiri"
12
+ end
data/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  ## Description
4
4
 
5
5
  BLZ or Bankleitzahlen is a bank identifier code system used by German
6
- and Austrian banks (see [Wikipedia](http://en.wikipedia.org/wiki/Bankleitzahl)).
6
+ and Austrian banks (see [Wikipedia](https://en.wikipedia.org/wiki/Bankleitzahl)).
7
7
 
8
8
  This gem provides for searching and returning the information
9
9
  represented by the BLZ, such as the name of the bank etc.
@@ -16,45 +16,50 @@ represented by the BLZ, such as the name of the bank etc.
16
16
 
17
17
  I provided a command line tool to search for BLZ:
18
18
 
19
- blz exact 70150000
20
-
21
- 70150000, Stadtsparkasse München, 80791 München, SSKMDEMMXXX
19
+ ```sh
20
+ $ blz exact 70150000
21
+ 70150000, Stadtsparkasse München, 80791 München, SSKMDEMMXXX
22
+ ```
22
23
 
23
24
  Or search for BIC matches:
24
25
 
25
- blz bic SSKMDE
26
-
27
- 70150000, Stadtsparkasse München, 80791 München, SSKMDEMMXXX
26
+ ```sh
27
+ $ blz bic SSKMDE
28
+ 70150000, Stadtsparkasse München, 80791 München, SSKMDEMMXXX
29
+ ```
28
30
 
29
31
  Or use substring matches:
30
32
 
31
- blz match 4945
32
-
33
- 49450120, Sparkasse Herford, 32045 Herford, WLAHDE44XXX
34
- 49450120, Sparkasse Herford, 32285 Rödinghausen, Westf
35
- 49450120, Sparkasse Herford, 32132 Spenge
36
- 49450120, Sparkasse Herford, 32269 Kirchlengern
37
- 49450120, Sparkasse Herford, 32122 Enger, Westf
38
- 49450120, Sparkasse Herford, 32591 Vlotho
39
- 49450120, Sparkasse Herford, 32120 Hiddenhausen
40
- 49450120, Sparkasse Herford, 32221 Bünde
41
- 49450120, Sparkasse Herford, 32556 Löhne
42
- 49451210, Sparkasse Bad Salzuflen -alt-, 32102 Bad Salzuflen, WELADED1BSU
33
+ ```sh
34
+ $ blz match 4945
35
+ 49450120, Sparkasse Herford, 32045 Herford, WLAHDE44XXX
36
+ 49450120, Sparkasse Herford, 32285 Rödinghausen, Westf
37
+ 49450120, Sparkasse Herford, 32132 Spenge
38
+ 49450120, Sparkasse Herford, 32269 Kirchlengern
39
+ 49450120, Sparkasse Herford, 32122 Enger, Westf
40
+ 49450120, Sparkasse Herford, 32591 Vlotho
41
+ 49450120, Sparkasse Herford, 32120 Hiddenhausen
42
+ 49450120, Sparkasse Herford, 32221 Bünde
43
+ 49450120, Sparkasse Herford, 32556 Löhne
44
+ 49451210, Sparkasse Bad Salzuflen -alt-, 32102 Bad Salzuflen, WELADED1BSU
45
+ ```
43
46
 
44
47
  Or find by city (rather uncommon, but useful):
45
48
 
46
- blz city Münstereifel
47
-
48
- 37069627, Raiffeisenbank Rheinbach Voreifel, 53902 Bad Münstereifel
49
- 37070024, Deutsche Bank Privat und Geschäftskunden, 53902 Bad Münstereifel, DEUTDEDB379
50
- 37070060, Deutsche Bank, 53902 Bad Münstereifel, DEUTDEDK379
51
- 38250110, Kreissparkasse Euskirchen, 53896 Bad Münstereifel
52
- 38260082, Volksbank Euskirchen, 53895 Bad Münstereifel
49
+ ```sh
50
+ $ blz city Münstereifel
51
+ 37069627, Raiffeisenbank Rheinbach Voreifel, 53902 Bad Münstereifel
52
+ 37070024, Deutsche Bank Privat und Geschäftskunden, 53902 Bad Münstereifel, DEUTDEDB379
53
+ 37070060, Deutsche Bank, 53902 Bad Münstereifel, DEUTDEDK379
54
+ 38250110, Kreissparkasse Euskirchen, 53896 Bad Münstereifel
55
+ 38260082, Volksbank Euskirchen, 53895 Bad Münstereifel
56
+ ```
53
57
 
54
58
  ## Downloads
55
59
 
56
60
  You can download the current list of BLZ (free of charge)
57
- [from the German Bundesbank](http://www.bundesbank.de/Redaktion/DE/Standardartikel/Kerngeschaeftsfelder/Unbarer_Zahlungsverkehr/bankleitzahlen_download.html).
61
+ [from the German Bundesbank](http://www.bundesbank.de/Redaktion/DE/Standardartikel/Kerngeschaeftsfelder/Unbarer_Zahlungsverkehr/bankleitzahlen_download.html),
62
+ however, you need to complete their [registration process](https://extranet.bundesbank.de/bsvpub/).
58
63
 
59
64
  The Bundesbank releases a new list every three months. The last part of the
60
65
  version number of this gem mirrors the last date the provided data is valid.
@@ -63,25 +68,19 @@ Now go and build your own BLZ gem ;-)
63
68
 
64
69
  ### Note on converting
65
70
 
66
- One of the easiest ways to contribute is by updating the `data/*.tsv.gz`
67
- files. Currently, this is a 3-monthly manual process of:
68
-
69
- 1. Downloading the file linked above (it helps to be subscribed to the
70
- Bundesbank ExtraNet INTBA mailing list to get notifications, but the
71
- [registration process](https://extranet.bundesbank.de/bsvpub/)
72
- is a bit cumbersome...).
73
-
74
- 2. Converting the `BLZ_yyyymmdd.xlsx` file into a tab seperated format
75
- (plain old CSV format with `\t` as field seperator; omit the header).
76
- Save it as `data/yyyy_mm_dd.tsv`
71
+ If you have an Extranet account, you may use the `scripts/fetch.rb` helper,
72
+ which will perform login, download, conversion (and logout). You need to
73
+ provide it with your credentials, like so:
77
74
 
78
- 3. Gzip'ing the `.tsv` file (i.e. `gzip -9 yyyy_mm_dd.tsv`, resulting
79
- in `data/yyyy_mm_dd.tsv.gz`).
75
+ ```sh
76
+ $ EXTRANET_USERNAME="..." EXTRANET_PASSWORD="..." scripts/fetch.rb
77
+ ```
80
78
 
81
- 4. Run the tests (`rake`).
79
+ It will create a new `data/*.tsv.gz` file or do nothing if the download
80
+ provided already exists.
82
81
 
83
- Obviously, this process tends to be error prone. A pull request with a
84
- Rake task completing these steps would be nice to have.
82
+ Note, that you'll need to clone this repository and execute `bundle` in
83
+ the repository directory to fetch development dependencies.
85
84
 
86
85
  ## Contributors
87
86
 
@@ -4,7 +4,7 @@ extra_rdoc_files = ['CHANGELOG.md', 'LICENSE', 'README.md']
4
4
 
5
5
  Gem::Specification.new do |s|
6
6
  s.name = 'blz'
7
- s.version = '0.2.0.20180305'
7
+ s.version = '0.2.0.20180604'
8
8
  s.platform = Gem::Platform::RUBY
9
9
  s.summary = "BLZ (Bankleitzahlen) for Ruby"
10
10
 
Binary file
@@ -0,0 +1,20 @@
1
+ # see https://github.com/olivere/blz/issues/14
2
+ TEST_SOURCE_URL = https://github.com/olivere/blz/files/1820218/BLZ_20180305.txt
3
+ TEST_TARGET_URL = https://github.com/olivere/blz/files/1820224/2018_03_05.tsv.gz
4
+
5
+ .PHONY: test
6
+ test: target.tsv compiled.tsv
7
+ diff -a target.tsv compiled.tsv && echo "SUCCESS: files identical" || echo "ERROR: file not identical"
8
+
9
+ .PHONY: clean
10
+ clean:
11
+ rm -f source.txt target.tsv compiled.tsv
12
+
13
+ source.txt:
14
+ curl -sSL $(TEST_SOURCE_URL) >$@
15
+
16
+ target.tsv:
17
+ curl -sSL $(TEST_TARGET_URL) | gunzip --stdout | sort >$@
18
+
19
+ compiled.tsv: source.txt
20
+ ./convert.rb $< | sort >$@
@@ -0,0 +1,46 @@
1
+ #!/usr/bin/env ruby
2
+ #
3
+ # Helper to convert an Extranet BLZ file download in a more sane format.
4
+ #
5
+
6
+ if ARGV[0].nil?
7
+ $stderr.puts "Usage: #{$0} FILE|-"
8
+ exit 1
9
+ end
10
+
11
+ # length (and name as comment) of each column
12
+ METRIC = [
13
+ 8, # BLZ
14
+ 1, # Merkmal
15
+ 58, # Bezeichnung
16
+ 5, # PLZ
17
+ 35, # Ort
18
+ 27, # Kurzbezeichnung
19
+ 5, # PAN
20
+ 11, # BIC
21
+ 2, # Prüfzifferberechnungsmethode
22
+ 6, # Datensatznummer
23
+ 1, # Änderungskennzeichen
24
+ 1, # BLZ-Löschung
25
+ 8, # Nachfolge-BLZ
26
+ ]
27
+
28
+ METRIC_LEN = METRIC.inject(:+)
29
+
30
+ # read from stdin if arg is "-", and setup encoding transformation
31
+ SOURCE = (ARGV[0] == "-" ? $stdin : File.open(ARGV[0], "rb"))
32
+ .set_encoding(Encoding::ISO_8859_15, Encoding::UTF_8)
33
+
34
+ while line = SOURCE.gets
35
+ line.chomp!
36
+ if line.length != METRIC_LEN
37
+ $stderr.puts "expected line length #{METRIC_LEN}, got #{line.length} in '#{line}'"
38
+ next
39
+ end
40
+
41
+ i = 0
42
+ puts METRIC.inject([]) {|s, m|
43
+ # extract columns, and strip leading/trailing whitespace
44
+ s << line[i ... (i+=m)].strip
45
+ }.join("\t")
46
+ end
@@ -0,0 +1,89 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "mechanize"
4
+ require "logger"
5
+ require "zlib"
6
+
7
+ if (f = File.expand_path("~/.extranet")) && File.exist?(f)
8
+ username, password = File.read(f).strip.split(":", 2)
9
+ else
10
+ username = ENV.fetch("EXTRANET_USERNAME")
11
+ password = ENV.fetch("EXTRANET_PASSWORD")
12
+ end
13
+
14
+ # from ./convert.rb
15
+ METRIC = [8, 1, 58, 5, 35, 27, 5, 11, 2, 6, 1, 1, 8]
16
+ METRIC_LEN = METRIC.inject(:+)
17
+
18
+ NAVIGATION = {
19
+ login: "https://extranet.bundesbank.de/FT/",
20
+ table: "https://extranet.bundesbank.de/FT/dirHTML.do?dirKind=DOWN&ftclient=browser&tableView=kompakt",
21
+ logout: "https://extranet.bundesbank.de/FT/logout.do",
22
+ }
23
+
24
+ # don't double download already existing files
25
+ last = Dir[ File.expand_path('../data/*.tsv.gz', __dir__) ].sort.last
26
+ last_match = last.match(/(?<y>\d{4})_(?<m>\d\d)_(?<d>\d\d)\.tsv\.gz$/)
27
+
28
+ logger = Logger.new($stderr).tap{|l| l.level = Logger::INFO }
29
+ agent = Mechanize.new do |m|
30
+ # m.log = logger
31
+ m.user_agent_alias = "Windows IE 11"
32
+ m.pluggable_parser["application/octet-stream"] = Mechanize::Download
33
+ end
34
+
35
+ begin
36
+ logger.info("login")
37
+ page = agent.get NAVIGATION[:login]
38
+ form = page.form_with action: "/pkmslogin.form"
39
+ form["username"] = username
40
+ form["password"] = password
41
+ agent.submit(form)
42
+
43
+ logger.info("get list of downloads")
44
+ sleep 1
45
+ page = agent.get NAVIGATION[:table]
46
+
47
+ target_link = page.links.find do |link|
48
+ !!(link.text.strip =~ /BLZ_\d{8}.txt/)
49
+ end
50
+
51
+ if !target_link
52
+ logger.info("no new download found")
53
+ exit 0
54
+ end
55
+
56
+ name = target_link.text.strip
57
+ if name == format("BLZ_%4d%02d%02d.txt", last_match[:y], last_match[:m], last_match[:d])
58
+ logger.info("no matching link found")
59
+ exit 0
60
+ end
61
+
62
+ logger.info("downloading new file #{name}")
63
+ sleep 1
64
+ blz = agent.get(target_link.href)
65
+
66
+ name_match = name.match(/BLZ_(?<y>\d{4})(?<m>\d\d)(?<d>\d\d)\.txt$/)
67
+ target_name = format("../data/%4d_%02d_%02d.tsv.gz", name_match[:y], name_match[:m], name_match[:d])
68
+ target_file = File.expand_path(target_name, __dir__)
69
+
70
+ logger.info("reformatting, saving as #{target_file}")
71
+ Zlib::GzipWriter.open(target_file, Zlib::BEST_COMPRESSION) do |gz|
72
+ while line = blz.body_io.gets
73
+ line = line.encode(Encoding::UTF_8, Encoding::ISO_8859_15).chomp!
74
+ if line.length != METRIC_LEN
75
+ logger.error("expected line length #{METRIC_LEN}, got #{line.length} in '#{line}'")
76
+ next
77
+ end
78
+
79
+ i = 0
80
+ gz.puts METRIC.inject([]) {|s, m| s << line[i ... (i+=m)].strip }.join("\t")
81
+ end
82
+ end
83
+
84
+ logger.info("suggestion new version: 0.2.0.#{name_match[:y]}#{name_match[:m]}#{name_match[:d]}")
85
+ ensure
86
+ logger.info("logout")
87
+ sleep 1
88
+ agent.get NAVIGATION[:logout]
89
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: blz
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0.20180305
4
+ version: 0.2.0.20180604
5
5
  platform: ruby
6
6
  authors:
7
7
  - Oliver Eilhard
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2018-03-16 00:00:00.000000000 Z
12
+ date: 2018-05-04 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler
@@ -84,8 +84,12 @@ files:
84
84
  - data/2017_06_05.tsv.gz
85
85
  - data/2017_12_04.tsv.gz
86
86
  - data/2018_03_05.tsv.gz
87
+ - data/2018_06_04.tsv.gz
87
88
  - lib/blz.rb
88
89
  - lib/blz/bank.rb
90
+ - scripts/Makefile
91
+ - scripts/convert.rb
92
+ - scripts/fetch.rb
89
93
  - test/test_bank.rb
90
94
  homepage: http://github.com/olivere/blz
91
95
  licenses: