ptj 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/Gemfile CHANGED
@@ -2,6 +2,18 @@ source "http://rubygems.org"
2
2
  # Add dependencies required to use your gem here.
3
3
  # Example:
4
4
  # gem "activesupport", ">= 2.3.5"
5
+ gem "dm-core"
6
+ gem "dm-migrations"
7
+ gem "dm-types"
8
+ gem "dm-transactions"
9
+ gem "dm-aggregates"
10
+ gem "dm-validations"
11
+ gem "dm-serializer"
12
+ gem "dm-timestamps"
13
+ gem "dm-sqlite-adapter"
14
+ gem "dm-postgres-adapter"
15
+ gem "progressbar"
16
+ gem "bundler", "~> 1.0.0"
5
17
 
6
18
  # Add dependencies to develop your gem here.
7
19
  # Include everything needed to run rake, tests, features, etc.
@@ -21,4 +33,5 @@ group :development do
21
33
  gem "dm-timestamps"
22
34
  gem "dm-sqlite-adapter"
23
35
  gem "dm-postgres-adapter"
36
+ gem "progressbar"
24
37
  end
@@ -49,6 +49,7 @@ GEM
49
49
  git (>= 1.2.5)
50
50
  rake
51
51
  json (1.4.6)
52
+ progressbar (0.9.1)
52
53
  rake (0.9.2)
53
54
  rcov (0.9.10)
54
55
  rspec (2.3.0)
@@ -79,6 +80,7 @@ DEPENDENCIES
79
80
  dm-types
80
81
  dm-validations
81
82
  jeweler (~> 1.6.2)
83
+ progressbar
82
84
  rcov
83
85
  rspec (~> 2.3.0)
84
86
  shoulda
@@ -1,12 +1,51 @@
1
1
  = PTJ
2
2
 
3
- Minimalistic database for the analyzing and storing of passwords.
3
+ Minimalistic database for the analyzing and storing of passwords. This project
4
+ came to be out of the need to quickly analyze a large number of passwords in a
5
+ short timeframe. As I, like most of you I'm sure, got tired of a simple
6
+ ruby/python/perl/bash/<insert language> script against a text file, the idea of
7
+ storing everything in a database came to be. I know, it's nothing revolutionary.
8
+
9
+ I'm using DataMapper in order to provide a database-agnostic front-end for users
10
+ to easily (well, somewhat easily) query and extract analytics against the entire
11
+ database, or a subset of passwords within the database.
12
+
13
+ 'Tags' are used as a way to categorize subsets of passwords. Think of them like
14
+ you would tags on a blog post. You might categorize a subset as 'public', or
15
+ 'internal', or perhaps 'company_name'. The number of tags which can be assigned
16
+ to groups of passwords is unlimited, and provides an easy way to ensure the
17
+ wrong passwords are not analyzed. This comes in handy when you want to, say,
18
+ do an analysis on every password you cracked on an internal network every
19
+ month, quarter, or year.
20
+
21
+ The following stats of a given password are currently being stored in the
22
+ Password table:
23
+
24
+ id - Sequential number used as the database key.
25
+ password - Plaintext password
26
+ pw_hash - Hash of the password, if known.
27
+ created_at - Time of when the password was added to the database.
28
+ upper - Boolean value telling you if an upper-case character is present.
29
+ lower - Boolean value telling you if a lower-case character is present.
30
+ number - Boolean value telling you if a number is present.
31
+ special - Boolean value telling you if a special character is present.
32
+ size - Length of the password.
33
+ sequence - Sequence of characters found in the password.
34
+ u - upper-case
35
+ l - lower-case
36
+ n - number
37
+ s - special
38
+ Example: Password123! => ulllllllnnns
39
+
40
+ Any number of combinations of these values, along with the tags can be used to
41
+ look at subsets of data.
4
42
 
5
43
  == Using PTJ
6
44
 
7
45
  === Setup
8
46
 
9
- First, we want to ensure our configuration file is correct.
47
+ Grab the repo, or grab the gem. The only difference should be the path where
48
+ everything is stored. You can find the path of the gem like so,
10
49
 
11
50
  gem which ptj
12
51
  /some/path/gems/ptj-0.1.0/lib/ptj.rb
@@ -15,24 +54,25 @@ Navigate to the 'etc' directory
15
54
 
16
55
  cd /some/path/gems/ptj-0.1.0/etc/
17
56
 
18
- Modify the configuration file to point to the correct database (default is a
19
- local SQLite database).
57
+ Modify the configuration file to point to the correct database.
20
58
 
21
59
  When this is complete, you can setup a new project as such:
22
60
 
23
- require 'ptj'
24
- PTJ::Model.setup!
25
- DataMapper.auto_upgrade!
61
+ rake db:init
62
+
63
+ In order to test that this was successful, attempt to get the count of all items
64
+ in the database.
65
+
66
+ rake db:count
67
+ Passwords: 0
68
+ Tags: 0
69
+ Time Taken: 0.03307 seconds.
26
70
 
27
71
  === Importing Data
28
72
 
29
73
  There is a provided script which will allow you to import data into the PTJ
30
74
  database.
31
75
 
32
- Working with our PTJ path above, it can be found in
33
-
34
- /some/path/gems/ptj-0.1.0/scripts/import.rb
35
-
36
76
  Running it, you can see the following options:
37
77
 
38
78
  ruby scripts/import.rb -h
@@ -50,16 +90,18 @@ Running it, you can see the following options:
50
90
  -a, --hash HASH Hash to import (Use in conjunction with -p).
51
91
  -h, --help Show this message.
52
92
 
93
+
53
94
  If you feel like doing it the hard way, feel free to view the source of this
54
- script.
95
+ script. Honestly, it's not terribly difficult. However, the script will allow
96
+ you to see the status of the import via a progress bar.
55
97
 
56
98
  === Analying Data
57
99
 
58
100
  Again, there is a handy, included, script which can be used to quickly generate
59
- and 'analysis' of a subset of passwords. It can be found in the 'scripts'
101
+ an 'analysis' of a subset of passwords. It can be found in the 'scripts'
60
102
  directory as well.
61
103
 
62
- ruby script/analyze.rb -h
104
+ ruby scripts/analyze.rb -h
63
105
 
64
106
  Usage: analyze.rb [opts]
65
107
  -t, --tags TAGS Tags to be used to when querying passwords (separated by a comma)
@@ -71,6 +113,36 @@ directory as well.
71
113
  --[no-]number Query based on numbers
72
114
  -h, --help Show this message
73
115
 
116
+ == Generating Wordlists
117
+
118
+ Guess what? There's another script that we can use to generate a customized
119
+ wordlist, sorted by number of occurrances. This really comes in handy once the
120
+ PTJ database becomes full of rich data, and we're looking for the top 'X'
121
+ passwords which meets a certain criteria. For example, let's say that we're on
122
+ an internal penetration test, and we need the top 20 passwords which include a
123
+ special character. We can simply run:
124
+
125
+ ruby scripts/generate_wordlist.rb -t internal --special
126
+
127
+
128
+ == Other Queries
129
+
130
+ Other queries can easily be made into a script, or simply from within irb. In
131
+ order to do this, you really should take a look at DataMapper's documentation:
132
+ http://datamapper.org/docs/
133
+
134
+ A few examples:
135
+
136
+ Querying all samples with the tag of 'internal':
137
+ PTJ::Tag.get('internal').passwords
138
+
139
+ Querying all samples with upper-case and lower-case characters with a tag of
140
+ 'internal':
141
+ PTJ::Tag.get('internal').passwords.all(:upper => true, :lower => true)
142
+
143
+ Get a list of the top sequences used by passwords:
144
+ PTJ::Password.aggregate(:sequence, :sequence.count).sort{|x,y| y[1] <=> x[1]}.each{|x| p x}
145
+
74
146
  == Contributing to ptj
75
147
 
76
148
  * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
@@ -87,10 +159,4 @@ Copyright (c) 2011 Josh Grunzweig. See LICENSE.txt for
87
159
  further details.
88
160
 
89
161
 
90
- == TODO
91
-
92
- * Batch imports?
93
- * specs / unit tests :/
94
- * benchmark unit tests with various actions
95
-
96
162
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.1.0
1
+ 0.1.1
@@ -4,7 +4,7 @@
4
4
  ## This connection string will cause the sqlite database to be located in
5
5
  ## the root directory of the ptj project under 'data/ptj.db'
6
6
 
7
- db_conn: "sqlite://$DATADIR$/ptj.db"
7
+ #db_conn: "sqlite://$DATADIR$/ptj.db"
8
8
 
9
9
  ## ... Or you can specify a hash of seperated parameters.
10
10
  ## The configuration below is loosely based on a production config.yml
@@ -37,6 +37,10 @@ module PTJ
37
37
  property :size, Integer,
38
38
  :default => lambda{|this,p| this.password.size }
39
39
 
40
+ # sequence of upper, lower, special, and number
41
+ property :sequence, String,
42
+ :default => lambda{|this,p| this.password.gsub(/[a-z]/,"l").gsub(/[A-Z]/,"u").gsub(/[0-9]/,"n").gsub(/[^uln]/,"s") }
43
+
40
44
  # Tags associated with a sample
41
45
  has n, :tags, :through => Resource
42
46
 
@@ -91,26 +95,6 @@ module PTJ
91
95
  return {:lower => lower, :upper => upper, :special => special, :number => number}
92
96
  end
93
97
 
94
- # Add a single password/hash to the database.
95
- #
96
- # @param mypass
97
- # Password to add.
98
- #
99
- # @param myhash
100
- # Password hash to add.
101
- #
102
- # @return DataMapper::Password
103
- def self.add_single(mypass, myhash = "")
104
- begin
105
- return if mypass.to_s.empty?
106
- pass = Password.create!(:password => mypass, :pw_hash => myhash)
107
- pass.save!
108
- pass
109
- rescue
110
- return
111
- end
112
- end
113
-
114
98
  end
115
99
  end
116
100
 
@@ -13,6 +13,10 @@ module PTJ
13
13
  raise(NotImplementedError, "This is an abstract implementation, you must override parse_line")
14
14
  end
15
15
 
16
+ def total_count(file)
17
+ raise(NotImplementedError, "This is an abstract implementation, you must override parse_line")
18
+ end
19
+
16
20
  end
17
21
  end
18
22
  end
@@ -19,7 +19,31 @@ module PTJ
19
19
  pass = $~[2]
20
20
  hash = nil
21
21
  end
22
- {:mypass => pass, :myhash => hash, :count => count}
22
+ ret_ary = []
23
+ count.to_i.times do
24
+ ret_ary << {:mypass => pass, :myhash => hash}
25
+ end
26
+ ret_ary
27
+ end
28
+
29
+ # Method used to return the total number of passwords that will be added
30
+ # to PTJ
31
+ #
32
+ # @param file
33
+ # File path which will be read
34
+ #
35
+ # @return Integer
36
+ #
37
+ def total_count(file)
38
+ file_obj = File.new(file,'r')
39
+ counter = 0
40
+ while (line = file_obj.gets)
41
+ line = line.force_encoding("BINARY")
42
+ if line =~ /^\s*(\d+)\s*(\S+)\s*$/
43
+ counter += $~[1].to_i
44
+ end
45
+ end
46
+ counter
23
47
  end
24
48
 
25
49
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -5,11 +5,11 @@
5
5
 
6
6
  Gem::Specification.new do |s|
7
7
  s.name = %q{ptj}
8
- s.version = "0.1.0"
8
+ s.version = "0.1.1"
9
9
 
10
10
  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
11
  s.authors = ["Josh Grunzweig"]
12
- s.date = %q{2011-08-29}
12
+ s.date = %q{2011-11-04}
13
13
  s.description = %q{An easy way to collect and analyze data about password
14
14
  databases.}
15
15
  s.email = %q{jgrunzweig@gmail.com}
@@ -67,6 +67,18 @@ Gem::Specification.new do |s|
67
67
  s.specification_version = 3
68
68
 
69
69
  if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
70
+ s.add_runtime_dependency(%q<dm-core>, [">= 0"])
71
+ s.add_runtime_dependency(%q<dm-migrations>, [">= 0"])
72
+ s.add_runtime_dependency(%q<dm-types>, [">= 0"])
73
+ s.add_runtime_dependency(%q<dm-transactions>, [">= 0"])
74
+ s.add_runtime_dependency(%q<dm-aggregates>, [">= 0"])
75
+ s.add_runtime_dependency(%q<dm-validations>, [">= 0"])
76
+ s.add_runtime_dependency(%q<dm-serializer>, [">= 0"])
77
+ s.add_runtime_dependency(%q<dm-timestamps>, [">= 0"])
78
+ s.add_runtime_dependency(%q<dm-sqlite-adapter>, [">= 0"])
79
+ s.add_runtime_dependency(%q<dm-postgres-adapter>, [">= 0"])
80
+ s.add_runtime_dependency(%q<progressbar>, [">= 0"])
81
+ s.add_runtime_dependency(%q<bundler>, ["~> 1.0.0"])
70
82
  s.add_development_dependency(%q<shoulda>, [">= 0"])
71
83
  s.add_development_dependency(%q<bundler>, ["~> 1.0.0"])
72
84
  s.add_development_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -82,7 +94,20 @@ Gem::Specification.new do |s|
82
94
  s.add_development_dependency(%q<dm-timestamps>, [">= 0"])
83
95
  s.add_development_dependency(%q<dm-sqlite-adapter>, [">= 0"])
84
96
  s.add_development_dependency(%q<dm-postgres-adapter>, [">= 0"])
97
+ s.add_development_dependency(%q<progressbar>, [">= 0"])
85
98
  else
99
+ s.add_dependency(%q<dm-core>, [">= 0"])
100
+ s.add_dependency(%q<dm-migrations>, [">= 0"])
101
+ s.add_dependency(%q<dm-types>, [">= 0"])
102
+ s.add_dependency(%q<dm-transactions>, [">= 0"])
103
+ s.add_dependency(%q<dm-aggregates>, [">= 0"])
104
+ s.add_dependency(%q<dm-validations>, [">= 0"])
105
+ s.add_dependency(%q<dm-serializer>, [">= 0"])
106
+ s.add_dependency(%q<dm-timestamps>, [">= 0"])
107
+ s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
108
+ s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
109
+ s.add_dependency(%q<progressbar>, [">= 0"])
110
+ s.add_dependency(%q<bundler>, ["~> 1.0.0"])
86
111
  s.add_dependency(%q<shoulda>, [">= 0"])
87
112
  s.add_dependency(%q<bundler>, ["~> 1.0.0"])
88
113
  s.add_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -98,8 +123,21 @@ Gem::Specification.new do |s|
98
123
  s.add_dependency(%q<dm-timestamps>, [">= 0"])
99
124
  s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
100
125
  s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
126
+ s.add_dependency(%q<progressbar>, [">= 0"])
101
127
  end
102
128
  else
129
+ s.add_dependency(%q<dm-core>, [">= 0"])
130
+ s.add_dependency(%q<dm-migrations>, [">= 0"])
131
+ s.add_dependency(%q<dm-types>, [">= 0"])
132
+ s.add_dependency(%q<dm-transactions>, [">= 0"])
133
+ s.add_dependency(%q<dm-aggregates>, [">= 0"])
134
+ s.add_dependency(%q<dm-validations>, [">= 0"])
135
+ s.add_dependency(%q<dm-serializer>, [">= 0"])
136
+ s.add_dependency(%q<dm-timestamps>, [">= 0"])
137
+ s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
138
+ s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
139
+ s.add_dependency(%q<progressbar>, [">= 0"])
140
+ s.add_dependency(%q<bundler>, ["~> 1.0.0"])
103
141
  s.add_dependency(%q<shoulda>, [">= 0"])
104
142
  s.add_dependency(%q<bundler>, ["~> 1.0.0"])
105
143
  s.add_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -115,6 +153,7 @@ Gem::Specification.new do |s|
115
153
  s.add_dependency(%q<dm-timestamps>, [">= 0"])
116
154
  s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
117
155
  s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
156
+ s.add_dependency(%q<progressbar>, [">= 0"])
118
157
  end
119
158
  end
120
159
 
@@ -9,7 +9,6 @@ DataMapper::Model.raise_on_save_failure = true if $DEBUG
9
9
 
10
10
  include PTJ
11
11
 
12
- #FILTER = {:fields => [:id, :password, :upper, :lower, :special, :number, :size]}
13
12
  FILTER = {}
14
13
  CFG = {
15
14
  :tags => [],
@@ -34,11 +33,11 @@ opts = OptionParser.new do |o|
34
33
  end
35
34
 
36
35
  o.on("--max-size SIZE", Integer, "Maximum size of the resulting passords") do |f|
37
- FILTER[:size.lt] = f
36
+ FILTER[:size.lte] = f
38
37
  end
39
38
 
40
39
  o.on("--min-size SIZE", Integer, "Minimum size of the resulting passwords") do |f|
41
- FILTER[:size.gt] = f
40
+ FILTER[:size.gte] = f
42
41
  end
43
42
 
44
43
  o.on("--[no-]upper", "Query based on upper-case letters") do |f|
@@ -71,6 +70,12 @@ def size_count(object, my_hash)
71
70
  return object.aggregate(:size, :size.count, my_hash).sort {|x,y| x[0] <=> y[0]}
72
71
  end
73
72
 
73
+ def top_sequences(object, my_hash)
74
+ my_hash.delete(:fields) if my_hash[:fields]
75
+ my_hash.delete(:order) if my_hash[:order]
76
+ return object.aggregate(:sequence, :sequence.count, my_hash).sort {|x,y| y[1] <=> x[1]}.first(5)
77
+ end
78
+
74
79
  def cat_result(object, my_hash)
75
80
  my_hash.delete(:fields) if my_hash[:fields]
76
81
  my_hash.delete(:order) if my_hash[:order]
@@ -93,15 +98,19 @@ def cat_result(object, my_hash)
93
98
  return_hash
94
99
  end
95
100
 
101
+
102
+
96
103
  time_now = Time.now
97
- if CFG[:tags]
104
+ if CFG[:tags].empty?
105
+ top5 = top5_pass(PTJ::Password.all, FILTER)
106
+ top_seq = top_sequences(PTJ::Password.all, FILTER)
107
+ my_count = size_count(PTJ::Password.all, FILTER)
108
+ split_up = cat_result(PTJ::Password.all, FILTER)
109
+ else
98
110
  top5 = top5_pass(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
111
+ top_seq = top_sequences(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
99
112
  my_count = size_count(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
100
113
  split_up = cat_result(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
101
- else
102
- top5 = top5_pass(PTJ::Passwords.all, FILTER)
103
- my_count = size_count(PTJ::Passwords.all, FILTER)
104
- split_up = cat_result(PTJ::Passwords.all, FILTER)
105
114
  end
106
115
 
107
116
 
@@ -112,7 +121,9 @@ top5.each do |pass, count|
112
121
  end
113
122
 
114
123
  total_size = 0
124
+
115
125
  my_count.each{|result| total_size += result[1] }
126
+
116
127
  puts "\n-=-=-=-=-=- Password Length -=-=-=-=-=-"
117
128
  my_count.each do |result|
118
129
  percent = "%.2f" % ((result[1].to_f/total_size.to_f)*100).to_f
@@ -126,4 +137,10 @@ split_up.sort_by{|k,v| k.length}.each do |result|
126
137
  printf("Type: %-30s %s", result[0], ("Result: #{result[1]}"))
127
138
  puts ""
128
139
  end
129
- p "Time taken: #{Time.now - time_now}"
140
+
141
+ puts "\n-=-=-=-=-=- Top Sequences -=-=-=-=-=-"
142
+ top_seq.each do |seq, count|
143
+ printf("%-30s : %d", seq, count)
144
+ puts ""
145
+ end
146
+ puts "\nTime taken: #{Time.now - time_now}"
@@ -1,6 +1,7 @@
1
1
  #!/usr/bin/env ruby
2
2
  require 'pp'
3
3
  require 'optparse'
4
+ require 'progressbar'
4
5
 
5
6
  require_relative('ptj_libpath')
6
7
  require 'ptj/default_setup'
@@ -71,41 +72,57 @@ end.parse!(ARGV)
71
72
 
72
73
  raise(OptionParser::MissingArgument, "Must specify file with -f or password with -p") if (CFG[:file].nil? and CFG[:password].nil?)
73
74
 
74
- #o_pass_count = Password.all(:fields => [:id]).size
75
-
76
-
77
75
  def import_file
78
76
  file = Pathname.new(CFG[:file])
79
77
  parser = CFG[:parser]
80
78
  tags = CFG[:tags]
81
- file = File.open(CFG[:file], "r")
82
- lines = file.readlines
83
- lines.each do |line|
79
+ file_obj = File.new(file, "r")
80
+ total_count = parser.total_count(file)
81
+ prog = ProgressBar.new("Importing...", total_count+1)
82
+ counter = 0
83
+ threshold = 2000
84
+ queue_array = []
85
+ while (line = file_obj.gets)
84
86
  begin
85
87
  line = line.force_encoding("BINARY")
86
88
  parsed = parser.parse_line(line)
87
- mypass = parsed[:mypass]
88
- myhash = parsed[:myhash]
89
- next if mypass.to_s.empty?
90
- if parsed[:count]
91
- parsed[:count].to_i.times do
92
- pass = Password.add_single(mypass, myhash)
93
- tags.each{|tag| pass.tags << tag}
94
- pass.save
95
- #puts "Adding #{mypass}"
89
+ queue_array = queue_array + parsed
90
+ while (queue_array.size > threshold)
91
+ Password.transaction do
92
+ queue_array[0..(threshold-1)].each do |myhash|
93
+ next if myhash[:mypass].to_s.empty?
94
+ pass = Password.create(:password => myhash[:mypass], :pw_hash => myhash[:myhash]||"")
95
+ prog.set(counter)
96
+ counter = counter+1
97
+ pass.save
98
+ tags.each{|tag| pass.tags << tag}
99
+ pass.save
100
+ end
101
+ queue_array = queue_array[(threshold)..queue_array.size]
96
102
  end
97
- else
98
- pass = Password.add_single(mypass, myhash)
103
+ end
104
+ rescue StandardError => e
105
+ p e.message
106
+ next
107
+ end
108
+ end
109
+
110
+ begin
111
+ Password.transaction do
112
+ queue_array.each do |myhash|
113
+ next if myhash[:mypass].to_s.empty?
114
+ pass = Password.create(:password => myhash[:mypass], :pw_hash => myhash[:myhash]||"")
115
+ prog.set(counter)
116
+ counter = counter+1
117
+ pass.save
99
118
  tags.each{|tag| pass.tags << tag}
100
119
  pass.save
101
- #puts "Adding #{mypass}"
102
120
  end
103
- rescue
104
- next
105
121
  end
122
+ rescue StandardError => e
123
+ p e.message
106
124
  end
107
125
  end
108
126
 
109
127
  import_file
110
128
 
111
-
@@ -13,7 +13,7 @@ describe PTJ::Password do
13
13
  context "basic functionality" do
14
14
  before :each do
15
15
  @ctime = Time.now
16
- @obj = PTJ::Password.add_single("password", "abchash")
16
+ @obj = PTJ::Password.create(:password => "password", :pw_hash => "abchash")
17
17
  end
18
18
 
19
19
  it_should_behave_like "a valid ptj model"
@@ -14,22 +14,13 @@ namespace :db do
14
14
  puts "Database initialized"
15
15
  end
16
16
 
17
- namespace :size do
18
- desc "Dump db record counts"
19
- task :default do
20
- require 'ptj/default_setup'
21
- puts "Passwords: #{PTJ::Password.all.size}"
22
- puts "Tags: #{PTJ::Tag.all.size}"
23
- end
24
-
25
- desc "Dump db record counts with debugging"
26
- task :debug do
27
- require 'ptj/default_setup'
28
- time_now = Time.now
29
- puts "Passwords: #{PTJ::Password.all.size}"
30
- puts "Tags: #{PTJ::Tag.all.size}"
31
- puts "Time Taken: #{Time.now - time_now} seconds."
32
- end
17
+ desc "Dump db record counts with debugging"
18
+ task :count do
19
+ require 'ptj/default_setup'
20
+ time_now = Time.now
21
+ puts "Passwords: #{PTJ::Password.count}"
22
+ puts "Tags: #{PTJ::Tag.count}"
23
+ puts "Time Taken: #{Time.now - time_now} seconds."
33
24
  end
34
25
 
35
26
  desc "Backup the development database"
metadata CHANGED
@@ -2,7 +2,7 @@
2
2
  name: ptj
3
3
  version: !ruby/object:Gem::Version
4
4
  prerelease:
5
- version: 0.1.0
5
+ version: 0.1.1
6
6
  platform: ruby
7
7
  authors:
8
8
  - Josh Grunzweig
@@ -10,34 +10,166 @@ autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
12
 
13
- date: 2011-08-29 00:00:00 -05:00
13
+ date: 2011-11-04 00:00:00 -05:00
14
14
  default_executable:
15
15
  dependencies:
16
16
  - !ruby/object:Gem::Dependency
17
- name: shoulda
17
+ name: dm-core
18
18
  requirement: &id001 !ruby/object:Gem::Requirement
19
19
  none: false
20
20
  requirements:
21
21
  - - ">="
22
22
  - !ruby/object:Gem::Version
23
23
  version: "0"
24
- type: :development
24
+ type: :runtime
25
25
  prerelease: false
26
26
  version_requirements: *id001
27
27
  - !ruby/object:Gem::Dependency
28
- name: bundler
28
+ name: dm-migrations
29
29
  requirement: &id002 !ruby/object:Gem::Requirement
30
+ none: false
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: "0"
35
+ type: :runtime
36
+ prerelease: false
37
+ version_requirements: *id002
38
+ - !ruby/object:Gem::Dependency
39
+ name: dm-types
40
+ requirement: &id003 !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ">="
44
+ - !ruby/object:Gem::Version
45
+ version: "0"
46
+ type: :runtime
47
+ prerelease: false
48
+ version_requirements: *id003
49
+ - !ruby/object:Gem::Dependency
50
+ name: dm-transactions
51
+ requirement: &id004 !ruby/object:Gem::Requirement
52
+ none: false
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ version: "0"
57
+ type: :runtime
58
+ prerelease: false
59
+ version_requirements: *id004
60
+ - !ruby/object:Gem::Dependency
61
+ name: dm-aggregates
62
+ requirement: &id005 !ruby/object:Gem::Requirement
63
+ none: false
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: "0"
68
+ type: :runtime
69
+ prerelease: false
70
+ version_requirements: *id005
71
+ - !ruby/object:Gem::Dependency
72
+ name: dm-validations
73
+ requirement: &id006 !ruby/object:Gem::Requirement
74
+ none: false
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ version: "0"
79
+ type: :runtime
80
+ prerelease: false
81
+ version_requirements: *id006
82
+ - !ruby/object:Gem::Dependency
83
+ name: dm-serializer
84
+ requirement: &id007 !ruby/object:Gem::Requirement
85
+ none: false
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: "0"
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: *id007
93
+ - !ruby/object:Gem::Dependency
94
+ name: dm-timestamps
95
+ requirement: &id008 !ruby/object:Gem::Requirement
96
+ none: false
97
+ requirements:
98
+ - - ">="
99
+ - !ruby/object:Gem::Version
100
+ version: "0"
101
+ type: :runtime
102
+ prerelease: false
103
+ version_requirements: *id008
104
+ - !ruby/object:Gem::Dependency
105
+ name: dm-sqlite-adapter
106
+ requirement: &id009 !ruby/object:Gem::Requirement
107
+ none: false
108
+ requirements:
109
+ - - ">="
110
+ - !ruby/object:Gem::Version
111
+ version: "0"
112
+ type: :runtime
113
+ prerelease: false
114
+ version_requirements: *id009
115
+ - !ruby/object:Gem::Dependency
116
+ name: dm-postgres-adapter
117
+ requirement: &id010 !ruby/object:Gem::Requirement
118
+ none: false
119
+ requirements:
120
+ - - ">="
121
+ - !ruby/object:Gem::Version
122
+ version: "0"
123
+ type: :runtime
124
+ prerelease: false
125
+ version_requirements: *id010
126
+ - !ruby/object:Gem::Dependency
127
+ name: progressbar
128
+ requirement: &id011 !ruby/object:Gem::Requirement
129
+ none: false
130
+ requirements:
131
+ - - ">="
132
+ - !ruby/object:Gem::Version
133
+ version: "0"
134
+ type: :runtime
135
+ prerelease: false
136
+ version_requirements: *id011
137
+ - !ruby/object:Gem::Dependency
138
+ name: bundler
139
+ requirement: &id012 !ruby/object:Gem::Requirement
30
140
  none: false
31
141
  requirements:
32
142
  - - ~>
33
143
  - !ruby/object:Gem::Version
34
144
  version: 1.0.0
145
+ type: :runtime
146
+ prerelease: false
147
+ version_requirements: *id012
148
+ - !ruby/object:Gem::Dependency
149
+ name: shoulda
150
+ requirement: &id013 !ruby/object:Gem::Requirement
151
+ none: false
152
+ requirements:
153
+ - - ">="
154
+ - !ruby/object:Gem::Version
155
+ version: "0"
35
156
  type: :development
36
157
  prerelease: false
37
- version_requirements: *id002
158
+ version_requirements: *id013
159
+ - !ruby/object:Gem::Dependency
160
+ name: bundler
161
+ requirement: &id014 !ruby/object:Gem::Requirement
162
+ none: false
163
+ requirements:
164
+ - - ~>
165
+ - !ruby/object:Gem::Version
166
+ version: 1.0.0
167
+ type: :development
168
+ prerelease: false
169
+ version_requirements: *id014
38
170
  - !ruby/object:Gem::Dependency
39
171
  name: jeweler
40
- requirement: &id003 !ruby/object:Gem::Requirement
172
+ requirement: &id015 !ruby/object:Gem::Requirement
41
173
  none: false
42
174
  requirements:
43
175
  - - ~>
@@ -45,10 +177,10 @@ dependencies:
45
177
  version: 1.6.2
46
178
  type: :development
47
179
  prerelease: false
48
- version_requirements: *id003
180
+ version_requirements: *id015
49
181
  - !ruby/object:Gem::Dependency
50
182
  name: rcov
51
- requirement: &id004 !ruby/object:Gem::Requirement
183
+ requirement: &id016 !ruby/object:Gem::Requirement
52
184
  none: false
53
185
  requirements:
54
186
  - - ">="
@@ -56,10 +188,10 @@ dependencies:
56
188
  version: "0"
57
189
  type: :development
58
190
  prerelease: false
59
- version_requirements: *id004
191
+ version_requirements: *id016
60
192
  - !ruby/object:Gem::Dependency
61
193
  name: rspec
62
- requirement: &id005 !ruby/object:Gem::Requirement
194
+ requirement: &id017 !ruby/object:Gem::Requirement
63
195
  none: false
64
196
  requirements:
65
197
  - - ~>
@@ -67,10 +199,10 @@ dependencies:
67
199
  version: 2.3.0
68
200
  type: :development
69
201
  prerelease: false
70
- version_requirements: *id005
202
+ version_requirements: *id017
71
203
  - !ruby/object:Gem::Dependency
72
204
  name: dm-core
73
- requirement: &id006 !ruby/object:Gem::Requirement
205
+ requirement: &id018 !ruby/object:Gem::Requirement
74
206
  none: false
75
207
  requirements:
76
208
  - - ">="
@@ -78,10 +210,10 @@ dependencies:
78
210
  version: "0"
79
211
  type: :development
80
212
  prerelease: false
81
- version_requirements: *id006
213
+ version_requirements: *id018
82
214
  - !ruby/object:Gem::Dependency
83
215
  name: dm-migrations
84
- requirement: &id007 !ruby/object:Gem::Requirement
216
+ requirement: &id019 !ruby/object:Gem::Requirement
85
217
  none: false
86
218
  requirements:
87
219
  - - ">="
@@ -89,10 +221,10 @@ dependencies:
89
221
  version: "0"
90
222
  type: :development
91
223
  prerelease: false
92
- version_requirements: *id007
224
+ version_requirements: *id019
93
225
  - !ruby/object:Gem::Dependency
94
226
  name: dm-types
95
- requirement: &id008 !ruby/object:Gem::Requirement
227
+ requirement: &id020 !ruby/object:Gem::Requirement
96
228
  none: false
97
229
  requirements:
98
230
  - - ">="
@@ -100,10 +232,10 @@ dependencies:
100
232
  version: "0"
101
233
  type: :development
102
234
  prerelease: false
103
- version_requirements: *id008
235
+ version_requirements: *id020
104
236
  - !ruby/object:Gem::Dependency
105
237
  name: dm-transactions
106
- requirement: &id009 !ruby/object:Gem::Requirement
238
+ requirement: &id021 !ruby/object:Gem::Requirement
107
239
  none: false
108
240
  requirements:
109
241
  - - ">="
@@ -111,10 +243,10 @@ dependencies:
111
243
  version: "0"
112
244
  type: :development
113
245
  prerelease: false
114
- version_requirements: *id009
246
+ version_requirements: *id021
115
247
  - !ruby/object:Gem::Dependency
116
248
  name: dm-aggregates
117
- requirement: &id010 !ruby/object:Gem::Requirement
249
+ requirement: &id022 !ruby/object:Gem::Requirement
118
250
  none: false
119
251
  requirements:
120
252
  - - ">="
@@ -122,10 +254,10 @@ dependencies:
122
254
  version: "0"
123
255
  type: :development
124
256
  prerelease: false
125
- version_requirements: *id010
257
+ version_requirements: *id022
126
258
  - !ruby/object:Gem::Dependency
127
259
  name: dm-validations
128
- requirement: &id011 !ruby/object:Gem::Requirement
260
+ requirement: &id023 !ruby/object:Gem::Requirement
129
261
  none: false
130
262
  requirements:
131
263
  - - ">="
@@ -133,10 +265,10 @@ dependencies:
133
265
  version: "0"
134
266
  type: :development
135
267
  prerelease: false
136
- version_requirements: *id011
268
+ version_requirements: *id023
137
269
  - !ruby/object:Gem::Dependency
138
270
  name: dm-serializer
139
- requirement: &id012 !ruby/object:Gem::Requirement
271
+ requirement: &id024 !ruby/object:Gem::Requirement
140
272
  none: false
141
273
  requirements:
142
274
  - - ">="
@@ -144,10 +276,10 @@ dependencies:
144
276
  version: "0"
145
277
  type: :development
146
278
  prerelease: false
147
- version_requirements: *id012
279
+ version_requirements: *id024
148
280
  - !ruby/object:Gem::Dependency
149
281
  name: dm-timestamps
150
- requirement: &id013 !ruby/object:Gem::Requirement
282
+ requirement: &id025 !ruby/object:Gem::Requirement
151
283
  none: false
152
284
  requirements:
153
285
  - - ">="
@@ -155,10 +287,10 @@ dependencies:
155
287
  version: "0"
156
288
  type: :development
157
289
  prerelease: false
158
- version_requirements: *id013
290
+ version_requirements: *id025
159
291
  - !ruby/object:Gem::Dependency
160
292
  name: dm-sqlite-adapter
161
- requirement: &id014 !ruby/object:Gem::Requirement
293
+ requirement: &id026 !ruby/object:Gem::Requirement
162
294
  none: false
163
295
  requirements:
164
296
  - - ">="
@@ -166,10 +298,10 @@ dependencies:
166
298
  version: "0"
167
299
  type: :development
168
300
  prerelease: false
169
- version_requirements: *id014
301
+ version_requirements: *id026
170
302
  - !ruby/object:Gem::Dependency
171
303
  name: dm-postgres-adapter
172
- requirement: &id015 !ruby/object:Gem::Requirement
304
+ requirement: &id027 !ruby/object:Gem::Requirement
173
305
  none: false
174
306
  requirements:
175
307
  - - ">="
@@ -177,7 +309,18 @@ dependencies:
177
309
  version: "0"
178
310
  type: :development
179
311
  prerelease: false
180
- version_requirements: *id015
312
+ version_requirements: *id027
313
+ - !ruby/object:Gem::Dependency
314
+ name: progressbar
315
+ requirement: &id028 !ruby/object:Gem::Requirement
316
+ none: false
317
+ requirements:
318
+ - - ">="
319
+ - !ruby/object:Gem::Version
320
+ version: "0"
321
+ type: :development
322
+ prerelease: false
323
+ version_requirements: *id028
181
324
  description: |-
182
325
  An easy way to collect and analyze data about password
183
326
  databases.
@@ -242,7 +385,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
242
385
  requirements:
243
386
  - - ">="
244
387
  - !ruby/object:Gem::Version
245
- hash: 536136881655303577
388
+ hash: 1240768379415668793
246
389
  segments:
247
390
  - 0
248
391
  version: "0"