ptj 0.1.0 → 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
data/Gemfile CHANGED
@@ -2,6 +2,18 @@ source "http://rubygems.org"
2
2
  # Add dependencies required to use your gem here.
3
3
  # Example:
4
4
  # gem "activesupport", ">= 2.3.5"
5
+ gem "dm-core"
6
+ gem "dm-migrations"
7
+ gem "dm-types"
8
+ gem "dm-transactions"
9
+ gem "dm-aggregates"
10
+ gem "dm-validations"
11
+ gem "dm-serializer"
12
+ gem "dm-timestamps"
13
+ gem "dm-sqlite-adapter"
14
+ gem "dm-postgres-adapter"
15
+ gem "progressbar"
16
+ gem "bundler", "~> 1.0.0"
5
17
 
6
18
  # Add dependencies to develop your gem here.
7
19
  # Include everything needed to run rake, tests, features, etc.
@@ -21,4 +33,5 @@ group :development do
21
33
  gem "dm-timestamps"
22
34
  gem "dm-sqlite-adapter"
23
35
  gem "dm-postgres-adapter"
36
+ gem "progressbar"
24
37
  end
@@ -49,6 +49,7 @@ GEM
49
49
  git (>= 1.2.5)
50
50
  rake
51
51
  json (1.4.6)
52
+ progressbar (0.9.1)
52
53
  rake (0.9.2)
53
54
  rcov (0.9.10)
54
55
  rspec (2.3.0)
@@ -79,6 +80,7 @@ DEPENDENCIES
79
80
  dm-types
80
81
  dm-validations
81
82
  jeweler (~> 1.6.2)
83
+ progressbar
82
84
  rcov
83
85
  rspec (~> 2.3.0)
84
86
  shoulda
@@ -1,12 +1,51 @@
1
1
  = PTJ
2
2
 
3
- Minimalistic database for the analyzing and storing of passwords.
3
+ Minimalistic database for the analyzing and storing of passwords. This project
4
+ came to be out of the need to quickly analyze a large number of passwords in a
5
+ short timeframe. As I, like most of you I'm sure, got tired of a simple
6
+ ruby/python/perl/bash/<insert language> script against a text file, the idea of
7
+ storing everything in a database came to be. I know, it's nothing revolutionary.
8
+
9
+ I'm using DataMapper in order to provide a database-agnostic front-end for users
10
+ to easily (well, somewhat easily) query and extract analytics against the entire
11
+ database, or a subset of passwords within the database.
12
+
13
+ 'Tags' are used as a way to categorize subsets of passwords. Think of them like
14
+ you would tags on a blog post. You might categorize a subset as 'public', or
15
+ 'internal', or perhaps 'company_name'. The number of tags which can be assigned
16
+ to groups of passwords is unlimited, and provides an easy way to ensure the
17
+ wrong passwords are not analyzed. This comes in handy when you want to, say,
18
+ do an analysis on every password you cracked on an internal network every
19
+ month, quarter, or year.
20
+
21
+ The following stats of a given password are currently being stored in the
22
+ Password table:
23
+
24
+ id - Sequential number used as the database key.
25
+ password - Plaintext password
26
+ pw_hash - Hash of the password, if known.
27
+ created_at - Time of when the password was added to the database.
28
+ upper - Boolean value telling you if an upper-case character is present.
29
+ lower - Boolean value telling you if a lower-case character is present.
30
+ number - Boolean value telling you if a number is present.
31
+ special - Boolean value telling you if a special character is present.
32
+ size - Length of the password.
33
+ sequence - Sequence of characters found in the password.
34
+ u - upper-case
35
+ l - lower-case
36
+ n - number
37
+ s - special
38
+ Example: Password123! => ulllllllnnns
39
+
40
+ Any number of combinations of these values, along with the tags can be used to
41
+ look at subsets of data.
4
42
 
5
43
  == Using PTJ
6
44
 
7
45
  === Setup
8
46
 
9
- First, we want to ensure our configuration file is correct.
47
+ Grab the repo, or grab the gem. The only difference should be the path where
48
+ everything is stored. You can find the path of the gem like so,
10
49
 
11
50
  gem which ptj
12
51
  /some/path/gems/ptj-0.1.0/lib/ptj.rb
@@ -15,24 +54,25 @@ Navigate to the 'etc' directory
15
54
 
16
55
  cd /some/path/gems/ptj-0.1.0/etc/
17
56
 
18
- Modify the configuration file to point to the correct database (default is a
19
- local SQLite database).
57
+ Modify the configuration file to point to the correct database.
20
58
 
21
59
  When this is complete, you can setup a new project as such:
22
60
 
23
- require 'ptj'
24
- PTJ::Model.setup!
25
- DataMapper.auto_upgrade!
61
+ rake db:init
62
+
63
+ In order to test that this was successful, attempt to get the count of all items
64
+ in the database.
65
+
66
+ rake db:count
67
+ Passwords: 0
68
+ Tags: 0
69
+ Time Taken: 0.03307 seconds.
26
70
 
27
71
  === Importing Data
28
72
 
29
73
  There is a provided script which will allow you to import data into the PTJ
30
74
  database.
31
75
 
32
- Working with our PTJ path above, it can be found in
33
-
34
- /some/path/gems/ptj-0.1.0/scripts/import.rb
35
-
36
76
  Running it, you can see the following options:
37
77
 
38
78
  ruby scripts/import.rb -h
@@ -50,16 +90,18 @@ Running it, you can see the following options:
50
90
  -a, --hash HASH Hash to import (Use in conjunction with -p).
51
91
  -h, --help Show this message.
52
92
 
93
+
53
94
  If you feel like doing it the hard way, feel free to view the source of this
54
- script.
95
+ script. Honestly, it's not terribly difficult. However, the script will allow
96
+ you to see the status of the import via a progress bar.
55
97
 
56
98
  === Analying Data
57
99
 
58
100
  Again, there is a handy, included, script which can be used to quickly generate
59
- and 'analysis' of a subset of passwords. It can be found in the 'scripts'
101
+ an 'analysis' of a subset of passwords. It can be found in the 'scripts'
60
102
  directory as well.
61
103
 
62
- ruby script/analyze.rb -h
104
+ ruby scripts/analyze.rb -h
63
105
 
64
106
  Usage: analyze.rb [opts]
65
107
  -t, --tags TAGS Tags to be used to when querying passwords (separated by a comma)
@@ -71,6 +113,36 @@ directory as well.
71
113
  --[no-]number Query based on numbers
72
114
  -h, --help Show this message
73
115
 
116
+ == Generating Wordlists
117
+
118
+ Guess what? There's another script that we can use to generate a customized
119
+ wordlist, sorted by number of occurrances. This really comes in handy once the
120
+ PTJ database becomes full of rich data, and we're looking for the top 'X'
121
+ passwords which meets a certain criteria. For example, let's say that we're on
122
+ an internal penetration test, and we need the top 20 passwords which include a
123
+ special character. We can simply run:
124
+
125
+ ruby scripts/generate_wordlist.rb -t internal --special
126
+
127
+
128
+ == Other Queries
129
+
130
+ Other queries can easily be made into a script, or simply from within irb. In
131
+ order to do this, you really should take a look at DataMapper's documentation:
132
+ http://datamapper.org/docs/
133
+
134
+ A few examples:
135
+
136
+ Querying all samples with the tag of 'internal':
137
+ PTJ::Tag.get('internal').passwords
138
+
139
+ Querying all samples with upper-case and lower-case characters with a tag of
140
+ 'internal':
141
+ PTJ::Tag.get('internal').passwords.all(:upper => true, :lower => true)
142
+
143
+ Get a list of the top sequences used by passwords:
144
+ PTJ::Password.aggregate(:sequence, :sequence.count).sort{|x,y| y[1] <=> x[1]}.each{|x| p x}
145
+
74
146
  == Contributing to ptj
75
147
 
76
148
  * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
@@ -87,10 +159,4 @@ Copyright (c) 2011 Josh Grunzweig. See LICENSE.txt for
87
159
  further details.
88
160
 
89
161
 
90
- == TODO
91
-
92
- * Batch imports?
93
- * specs / unit tests :/
94
- * benchmark unit tests with various actions
95
-
96
162
 
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.1.0
1
+ 0.1.1
@@ -4,7 +4,7 @@
4
4
  ## This connection string will cause the sqlite database to be located in
5
5
  ## the root directory of the ptj project under 'data/ptj.db'
6
6
 
7
- db_conn: "sqlite://$DATADIR$/ptj.db"
7
+ #db_conn: "sqlite://$DATADIR$/ptj.db"
8
8
 
9
9
  ## ... Or you can specify a hash of seperated parameters.
10
10
  ## The configuration below is loosely based on a production config.yml
@@ -37,6 +37,10 @@ module PTJ
37
37
  property :size, Integer,
38
38
  :default => lambda{|this,p| this.password.size }
39
39
 
40
+ # sequence of upper, lower, special, and number
41
+ property :sequence, String,
42
+ :default => lambda{|this,p| this.password.gsub(/[a-z]/,"l").gsub(/[A-Z]/,"u").gsub(/[0-9]/,"n").gsub(/[^uln]/,"s") }
43
+
40
44
  # Tags associated with a sample
41
45
  has n, :tags, :through => Resource
42
46
 
@@ -91,26 +95,6 @@ module PTJ
91
95
  return {:lower => lower, :upper => upper, :special => special, :number => number}
92
96
  end
93
97
 
94
- # Add a single password/hash to the database.
95
- #
96
- # @param mypass
97
- # Password to add.
98
- #
99
- # @param myhash
100
- # Password hash to add.
101
- #
102
- # @return DataMapper::Password
103
- def self.add_single(mypass, myhash = "")
104
- begin
105
- return if mypass.to_s.empty?
106
- pass = Password.create!(:password => mypass, :pw_hash => myhash)
107
- pass.save!
108
- pass
109
- rescue
110
- return
111
- end
112
- end
113
-
114
98
  end
115
99
  end
116
100
 
@@ -13,6 +13,10 @@ module PTJ
13
13
  raise(NotImplementedError, "This is an abstract implementation, you must override parse_line")
14
14
  end
15
15
 
16
+ def total_count(file)
17
+ raise(NotImplementedError, "This is an abstract implementation, you must override parse_line")
18
+ end
19
+
16
20
  end
17
21
  end
18
22
  end
@@ -19,7 +19,31 @@ module PTJ
19
19
  pass = $~[2]
20
20
  hash = nil
21
21
  end
22
- {:mypass => pass, :myhash => hash, :count => count}
22
+ ret_ary = []
23
+ count.to_i.times do
24
+ ret_ary << {:mypass => pass, :myhash => hash}
25
+ end
26
+ ret_ary
27
+ end
28
+
29
+ # Method used to return the total number of passwords that will be added
30
+ # to PTJ
31
+ #
32
+ # @param file
33
+ # File path which will be read
34
+ #
35
+ # @return Integer
36
+ #
37
+ def total_count(file)
38
+ file_obj = File.new(file,'r')
39
+ counter = 0
40
+ while (line = file_obj.gets)
41
+ line = line.force_encoding("BINARY")
42
+ if line =~ /^\s*(\d+)\s*(\S+)\s*$/
43
+ counter += $~[1].to_i
44
+ end
45
+ end
46
+ counter
23
47
  end
24
48
 
25
49
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -21,6 +21,19 @@ module PTJ
21
21
  {:mypass => pass, :myhash => hash}
22
22
  end
23
23
 
24
+ # Method used to return the total number of passwords that will be added
25
+ # to PTJ
26
+ #
27
+ # @param file
28
+ # File path which will be read
29
+ #
30
+ # @return Integer
31
+ #
32
+ def total_count(file)
33
+ file_obj = File.new(file,'r')
34
+ file_obj.readlines.size
35
+ end
36
+
24
37
  end
25
38
  end
26
39
  end
@@ -5,11 +5,11 @@
5
5
 
6
6
  Gem::Specification.new do |s|
7
7
  s.name = %q{ptj}
8
- s.version = "0.1.0"
8
+ s.version = "0.1.1"
9
9
 
10
10
  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
11
  s.authors = ["Josh Grunzweig"]
12
- s.date = %q{2011-08-29}
12
+ s.date = %q{2011-11-04}
13
13
  s.description = %q{An easy way to collect and analyze data about password
14
14
  databases.}
15
15
  s.email = %q{jgrunzweig@gmail.com}
@@ -67,6 +67,18 @@ Gem::Specification.new do |s|
67
67
  s.specification_version = 3
68
68
 
69
69
  if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
70
+ s.add_runtime_dependency(%q<dm-core>, [">= 0"])
71
+ s.add_runtime_dependency(%q<dm-migrations>, [">= 0"])
72
+ s.add_runtime_dependency(%q<dm-types>, [">= 0"])
73
+ s.add_runtime_dependency(%q<dm-transactions>, [">= 0"])
74
+ s.add_runtime_dependency(%q<dm-aggregates>, [">= 0"])
75
+ s.add_runtime_dependency(%q<dm-validations>, [">= 0"])
76
+ s.add_runtime_dependency(%q<dm-serializer>, [">= 0"])
77
+ s.add_runtime_dependency(%q<dm-timestamps>, [">= 0"])
78
+ s.add_runtime_dependency(%q<dm-sqlite-adapter>, [">= 0"])
79
+ s.add_runtime_dependency(%q<dm-postgres-adapter>, [">= 0"])
80
+ s.add_runtime_dependency(%q<progressbar>, [">= 0"])
81
+ s.add_runtime_dependency(%q<bundler>, ["~> 1.0.0"])
70
82
  s.add_development_dependency(%q<shoulda>, [">= 0"])
71
83
  s.add_development_dependency(%q<bundler>, ["~> 1.0.0"])
72
84
  s.add_development_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -82,7 +94,20 @@ Gem::Specification.new do |s|
82
94
  s.add_development_dependency(%q<dm-timestamps>, [">= 0"])
83
95
  s.add_development_dependency(%q<dm-sqlite-adapter>, [">= 0"])
84
96
  s.add_development_dependency(%q<dm-postgres-adapter>, [">= 0"])
97
+ s.add_development_dependency(%q<progressbar>, [">= 0"])
85
98
  else
99
+ s.add_dependency(%q<dm-core>, [">= 0"])
100
+ s.add_dependency(%q<dm-migrations>, [">= 0"])
101
+ s.add_dependency(%q<dm-types>, [">= 0"])
102
+ s.add_dependency(%q<dm-transactions>, [">= 0"])
103
+ s.add_dependency(%q<dm-aggregates>, [">= 0"])
104
+ s.add_dependency(%q<dm-validations>, [">= 0"])
105
+ s.add_dependency(%q<dm-serializer>, [">= 0"])
106
+ s.add_dependency(%q<dm-timestamps>, [">= 0"])
107
+ s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
108
+ s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
109
+ s.add_dependency(%q<progressbar>, [">= 0"])
110
+ s.add_dependency(%q<bundler>, ["~> 1.0.0"])
86
111
  s.add_dependency(%q<shoulda>, [">= 0"])
87
112
  s.add_dependency(%q<bundler>, ["~> 1.0.0"])
88
113
  s.add_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -98,8 +123,21 @@ Gem::Specification.new do |s|
98
123
  s.add_dependency(%q<dm-timestamps>, [">= 0"])
99
124
  s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
100
125
  s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
126
+ s.add_dependency(%q<progressbar>, [">= 0"])
101
127
  end
102
128
  else
129
+ s.add_dependency(%q<dm-core>, [">= 0"])
130
+ s.add_dependency(%q<dm-migrations>, [">= 0"])
131
+ s.add_dependency(%q<dm-types>, [">= 0"])
132
+ s.add_dependency(%q<dm-transactions>, [">= 0"])
133
+ s.add_dependency(%q<dm-aggregates>, [">= 0"])
134
+ s.add_dependency(%q<dm-validations>, [">= 0"])
135
+ s.add_dependency(%q<dm-serializer>, [">= 0"])
136
+ s.add_dependency(%q<dm-timestamps>, [">= 0"])
137
+ s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
138
+ s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
139
+ s.add_dependency(%q<progressbar>, [">= 0"])
140
+ s.add_dependency(%q<bundler>, ["~> 1.0.0"])
103
141
  s.add_dependency(%q<shoulda>, [">= 0"])
104
142
  s.add_dependency(%q<bundler>, ["~> 1.0.0"])
105
143
  s.add_dependency(%q<jeweler>, ["~> 1.6.2"])
@@ -115,6 +153,7 @@ Gem::Specification.new do |s|
115
153
  s.add_dependency(%q<dm-timestamps>, [">= 0"])
116
154
  s.add_dependency(%q<dm-sqlite-adapter>, [">= 0"])
117
155
  s.add_dependency(%q<dm-postgres-adapter>, [">= 0"])
156
+ s.add_dependency(%q<progressbar>, [">= 0"])
118
157
  end
119
158
  end
120
159
 
@@ -9,7 +9,6 @@ DataMapper::Model.raise_on_save_failure = true if $DEBUG
9
9
 
10
10
  include PTJ
11
11
 
12
- #FILTER = {:fields => [:id, :password, :upper, :lower, :special, :number, :size]}
13
12
  FILTER = {}
14
13
  CFG = {
15
14
  :tags => [],
@@ -34,11 +33,11 @@ opts = OptionParser.new do |o|
34
33
  end
35
34
 
36
35
  o.on("--max-size SIZE", Integer, "Maximum size of the resulting passords") do |f|
37
- FILTER[:size.lt] = f
36
+ FILTER[:size.lte] = f
38
37
  end
39
38
 
40
39
  o.on("--min-size SIZE", Integer, "Minimum size of the resulting passwords") do |f|
41
- FILTER[:size.gt] = f
40
+ FILTER[:size.gte] = f
42
41
  end
43
42
 
44
43
  o.on("--[no-]upper", "Query based on upper-case letters") do |f|
@@ -71,6 +70,12 @@ def size_count(object, my_hash)
71
70
  return object.aggregate(:size, :size.count, my_hash).sort {|x,y| x[0] <=> y[0]}
72
71
  end
73
72
 
73
+ def top_sequences(object, my_hash)
74
+ my_hash.delete(:fields) if my_hash[:fields]
75
+ my_hash.delete(:order) if my_hash[:order]
76
+ return object.aggregate(:sequence, :sequence.count, my_hash).sort {|x,y| y[1] <=> x[1]}.first(5)
77
+ end
78
+
74
79
  def cat_result(object, my_hash)
75
80
  my_hash.delete(:fields) if my_hash[:fields]
76
81
  my_hash.delete(:order) if my_hash[:order]
@@ -93,15 +98,19 @@ def cat_result(object, my_hash)
93
98
  return_hash
94
99
  end
95
100
 
101
+
102
+
96
103
  time_now = Time.now
97
- if CFG[:tags]
104
+ if CFG[:tags].empty?
105
+ top5 = top5_pass(PTJ::Password.all, FILTER)
106
+ top_seq = top_sequences(PTJ::Password.all, FILTER)
107
+ my_count = size_count(PTJ::Password.all, FILTER)
108
+ split_up = cat_result(PTJ::Password.all, FILTER)
109
+ else
98
110
  top5 = top5_pass(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
111
+ top_seq = top_sequences(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
99
112
  my_count = size_count(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
100
113
  split_up = cat_result(PTJ::Tag.all(:tag => CFG[:tags]).passwords, FILTER)
101
- else
102
- top5 = top5_pass(PTJ::Passwords.all, FILTER)
103
- my_count = size_count(PTJ::Passwords.all, FILTER)
104
- split_up = cat_result(PTJ::Passwords.all, FILTER)
105
114
  end
106
115
 
107
116
 
@@ -112,7 +121,9 @@ top5.each do |pass, count|
112
121
  end
113
122
 
114
123
  total_size = 0
124
+
115
125
  my_count.each{|result| total_size += result[1] }
126
+
116
127
  puts "\n-=-=-=-=-=- Password Length -=-=-=-=-=-"
117
128
  my_count.each do |result|
118
129
  percent = "%.2f" % ((result[1].to_f/total_size.to_f)*100).to_f
@@ -126,4 +137,10 @@ split_up.sort_by{|k,v| k.length}.each do |result|
126
137
  printf("Type: %-30s %s", result[0], ("Result: #{result[1]}"))
127
138
  puts ""
128
139
  end
129
- p "Time taken: #{Time.now - time_now}"
140
+
141
+ puts "\n-=-=-=-=-=- Top Sequences -=-=-=-=-=-"
142
+ top_seq.each do |seq, count|
143
+ printf("%-30s : %d", seq, count)
144
+ puts ""
145
+ end
146
+ puts "\nTime taken: #{Time.now - time_now}"
@@ -1,6 +1,7 @@
1
1
  #!/usr/bin/env ruby
2
2
  require 'pp'
3
3
  require 'optparse'
4
+ require 'progressbar'
4
5
 
5
6
  require_relative('ptj_libpath')
6
7
  require 'ptj/default_setup'
@@ -71,41 +72,57 @@ end.parse!(ARGV)
71
72
 
72
73
  raise(OptionParser::MissingArgument, "Must specify file with -f or password with -p") if (CFG[:file].nil? and CFG[:password].nil?)
73
74
 
74
- #o_pass_count = Password.all(:fields => [:id]).size
75
-
76
-
77
75
  def import_file
78
76
  file = Pathname.new(CFG[:file])
79
77
  parser = CFG[:parser]
80
78
  tags = CFG[:tags]
81
- file = File.open(CFG[:file], "r")
82
- lines = file.readlines
83
- lines.each do |line|
79
+ file_obj = File.new(file, "r")
80
+ total_count = parser.total_count(file)
81
+ prog = ProgressBar.new("Importing...", total_count+1)
82
+ counter = 0
83
+ threshold = 2000
84
+ queue_array = []
85
+ while (line = file_obj.gets)
84
86
  begin
85
87
  line = line.force_encoding("BINARY")
86
88
  parsed = parser.parse_line(line)
87
- mypass = parsed[:mypass]
88
- myhash = parsed[:myhash]
89
- next if mypass.to_s.empty?
90
- if parsed[:count]
91
- parsed[:count].to_i.times do
92
- pass = Password.add_single(mypass, myhash)
93
- tags.each{|tag| pass.tags << tag}
94
- pass.save
95
- #puts "Adding #{mypass}"
89
+ queue_array = queue_array + parsed
90
+ while (queue_array.size > threshold)
91
+ Password.transaction do
92
+ queue_array[0..(threshold-1)].each do |myhash|
93
+ next if myhash[:mypass].to_s.empty?
94
+ pass = Password.create(:password => myhash[:mypass], :pw_hash => myhash[:myhash]||"")
95
+ prog.set(counter)
96
+ counter = counter+1
97
+ pass.save
98
+ tags.each{|tag| pass.tags << tag}
99
+ pass.save
100
+ end
101
+ queue_array = queue_array[(threshold)..queue_array.size]
96
102
  end
97
- else
98
- pass = Password.add_single(mypass, myhash)
103
+ end
104
+ rescue StandardError => e
105
+ p e.message
106
+ next
107
+ end
108
+ end
109
+
110
+ begin
111
+ Password.transaction do
112
+ queue_array.each do |myhash|
113
+ next if myhash[:mypass].to_s.empty?
114
+ pass = Password.create(:password => myhash[:mypass], :pw_hash => myhash[:myhash]||"")
115
+ prog.set(counter)
116
+ counter = counter+1
117
+ pass.save
99
118
  tags.each{|tag| pass.tags << tag}
100
119
  pass.save
101
- #puts "Adding #{mypass}"
102
120
  end
103
- rescue
104
- next
105
121
  end
122
+ rescue StandardError => e
123
+ p e.message
106
124
  end
107
125
  end
108
126
 
109
127
  import_file
110
128
 
111
-
@@ -13,7 +13,7 @@ describe PTJ::Password do
13
13
  context "basic functionality" do
14
14
  before :each do
15
15
  @ctime = Time.now
16
- @obj = PTJ::Password.add_single("password", "abchash")
16
+ @obj = PTJ::Password.create(:password => "password", :pw_hash => "abchash")
17
17
  end
18
18
 
19
19
  it_should_behave_like "a valid ptj model"
@@ -14,22 +14,13 @@ namespace :db do
14
14
  puts "Database initialized"
15
15
  end
16
16
 
17
- namespace :size do
18
- desc "Dump db record counts"
19
- task :default do
20
- require 'ptj/default_setup'
21
- puts "Passwords: #{PTJ::Password.all.size}"
22
- puts "Tags: #{PTJ::Tag.all.size}"
23
- end
24
-
25
- desc "Dump db record counts with debugging"
26
- task :debug do
27
- require 'ptj/default_setup'
28
- time_now = Time.now
29
- puts "Passwords: #{PTJ::Password.all.size}"
30
- puts "Tags: #{PTJ::Tag.all.size}"
31
- puts "Time Taken: #{Time.now - time_now} seconds."
32
- end
17
+ desc "Dump db record counts with debugging"
18
+ task :count do
19
+ require 'ptj/default_setup'
20
+ time_now = Time.now
21
+ puts "Passwords: #{PTJ::Password.count}"
22
+ puts "Tags: #{PTJ::Tag.count}"
23
+ puts "Time Taken: #{Time.now - time_now} seconds."
33
24
  end
34
25
 
35
26
  desc "Backup the development database"
metadata CHANGED
@@ -2,7 +2,7 @@
2
2
  name: ptj
3
3
  version: !ruby/object:Gem::Version
4
4
  prerelease:
5
- version: 0.1.0
5
+ version: 0.1.1
6
6
  platform: ruby
7
7
  authors:
8
8
  - Josh Grunzweig
@@ -10,34 +10,166 @@ autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
12
 
13
- date: 2011-08-29 00:00:00 -05:00
13
+ date: 2011-11-04 00:00:00 -05:00
14
14
  default_executable:
15
15
  dependencies:
16
16
  - !ruby/object:Gem::Dependency
17
- name: shoulda
17
+ name: dm-core
18
18
  requirement: &id001 !ruby/object:Gem::Requirement
19
19
  none: false
20
20
  requirements:
21
21
  - - ">="
22
22
  - !ruby/object:Gem::Version
23
23
  version: "0"
24
- type: :development
24
+ type: :runtime
25
25
  prerelease: false
26
26
  version_requirements: *id001
27
27
  - !ruby/object:Gem::Dependency
28
- name: bundler
28
+ name: dm-migrations
29
29
  requirement: &id002 !ruby/object:Gem::Requirement
30
+ none: false
31
+ requirements:
32
+ - - ">="
33
+ - !ruby/object:Gem::Version
34
+ version: "0"
35
+ type: :runtime
36
+ prerelease: false
37
+ version_requirements: *id002
38
+ - !ruby/object:Gem::Dependency
39
+ name: dm-types
40
+ requirement: &id003 !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ">="
44
+ - !ruby/object:Gem::Version
45
+ version: "0"
46
+ type: :runtime
47
+ prerelease: false
48
+ version_requirements: *id003
49
+ - !ruby/object:Gem::Dependency
50
+ name: dm-transactions
51
+ requirement: &id004 !ruby/object:Gem::Requirement
52
+ none: false
53
+ requirements:
54
+ - - ">="
55
+ - !ruby/object:Gem::Version
56
+ version: "0"
57
+ type: :runtime
58
+ prerelease: false
59
+ version_requirements: *id004
60
+ - !ruby/object:Gem::Dependency
61
+ name: dm-aggregates
62
+ requirement: &id005 !ruby/object:Gem::Requirement
63
+ none: false
64
+ requirements:
65
+ - - ">="
66
+ - !ruby/object:Gem::Version
67
+ version: "0"
68
+ type: :runtime
69
+ prerelease: false
70
+ version_requirements: *id005
71
+ - !ruby/object:Gem::Dependency
72
+ name: dm-validations
73
+ requirement: &id006 !ruby/object:Gem::Requirement
74
+ none: false
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ version: "0"
79
+ type: :runtime
80
+ prerelease: false
81
+ version_requirements: *id006
82
+ - !ruby/object:Gem::Dependency
83
+ name: dm-serializer
84
+ requirement: &id007 !ruby/object:Gem::Requirement
85
+ none: false
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: "0"
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: *id007
93
+ - !ruby/object:Gem::Dependency
94
+ name: dm-timestamps
95
+ requirement: &id008 !ruby/object:Gem::Requirement
96
+ none: false
97
+ requirements:
98
+ - - ">="
99
+ - !ruby/object:Gem::Version
100
+ version: "0"
101
+ type: :runtime
102
+ prerelease: false
103
+ version_requirements: *id008
104
+ - !ruby/object:Gem::Dependency
105
+ name: dm-sqlite-adapter
106
+ requirement: &id009 !ruby/object:Gem::Requirement
107
+ none: false
108
+ requirements:
109
+ - - ">="
110
+ - !ruby/object:Gem::Version
111
+ version: "0"
112
+ type: :runtime
113
+ prerelease: false
114
+ version_requirements: *id009
115
+ - !ruby/object:Gem::Dependency
116
+ name: dm-postgres-adapter
117
+ requirement: &id010 !ruby/object:Gem::Requirement
118
+ none: false
119
+ requirements:
120
+ - - ">="
121
+ - !ruby/object:Gem::Version
122
+ version: "0"
123
+ type: :runtime
124
+ prerelease: false
125
+ version_requirements: *id010
126
+ - !ruby/object:Gem::Dependency
127
+ name: progressbar
128
+ requirement: &id011 !ruby/object:Gem::Requirement
129
+ none: false
130
+ requirements:
131
+ - - ">="
132
+ - !ruby/object:Gem::Version
133
+ version: "0"
134
+ type: :runtime
135
+ prerelease: false
136
+ version_requirements: *id011
137
+ - !ruby/object:Gem::Dependency
138
+ name: bundler
139
+ requirement: &id012 !ruby/object:Gem::Requirement
30
140
  none: false
31
141
  requirements:
32
142
  - - ~>
33
143
  - !ruby/object:Gem::Version
34
144
  version: 1.0.0
145
+ type: :runtime
146
+ prerelease: false
147
+ version_requirements: *id012
148
+ - !ruby/object:Gem::Dependency
149
+ name: shoulda
150
+ requirement: &id013 !ruby/object:Gem::Requirement
151
+ none: false
152
+ requirements:
153
+ - - ">="
154
+ - !ruby/object:Gem::Version
155
+ version: "0"
35
156
  type: :development
36
157
  prerelease: false
37
- version_requirements: *id002
158
+ version_requirements: *id013
159
+ - !ruby/object:Gem::Dependency
160
+ name: bundler
161
+ requirement: &id014 !ruby/object:Gem::Requirement
162
+ none: false
163
+ requirements:
164
+ - - ~>
165
+ - !ruby/object:Gem::Version
166
+ version: 1.0.0
167
+ type: :development
168
+ prerelease: false
169
+ version_requirements: *id014
38
170
  - !ruby/object:Gem::Dependency
39
171
  name: jeweler
40
- requirement: &id003 !ruby/object:Gem::Requirement
172
+ requirement: &id015 !ruby/object:Gem::Requirement
41
173
  none: false
42
174
  requirements:
43
175
  - - ~>
@@ -45,10 +177,10 @@ dependencies:
45
177
  version: 1.6.2
46
178
  type: :development
47
179
  prerelease: false
48
- version_requirements: *id003
180
+ version_requirements: *id015
49
181
  - !ruby/object:Gem::Dependency
50
182
  name: rcov
51
- requirement: &id004 !ruby/object:Gem::Requirement
183
+ requirement: &id016 !ruby/object:Gem::Requirement
52
184
  none: false
53
185
  requirements:
54
186
  - - ">="
@@ -56,10 +188,10 @@ dependencies:
56
188
  version: "0"
57
189
  type: :development
58
190
  prerelease: false
59
- version_requirements: *id004
191
+ version_requirements: *id016
60
192
  - !ruby/object:Gem::Dependency
61
193
  name: rspec
62
- requirement: &id005 !ruby/object:Gem::Requirement
194
+ requirement: &id017 !ruby/object:Gem::Requirement
63
195
  none: false
64
196
  requirements:
65
197
  - - ~>
@@ -67,10 +199,10 @@ dependencies:
67
199
  version: 2.3.0
68
200
  type: :development
69
201
  prerelease: false
70
- version_requirements: *id005
202
+ version_requirements: *id017
71
203
  - !ruby/object:Gem::Dependency
72
204
  name: dm-core
73
- requirement: &id006 !ruby/object:Gem::Requirement
205
+ requirement: &id018 !ruby/object:Gem::Requirement
74
206
  none: false
75
207
  requirements:
76
208
  - - ">="
@@ -78,10 +210,10 @@ dependencies:
78
210
  version: "0"
79
211
  type: :development
80
212
  prerelease: false
81
- version_requirements: *id006
213
+ version_requirements: *id018
82
214
  - !ruby/object:Gem::Dependency
83
215
  name: dm-migrations
84
- requirement: &id007 !ruby/object:Gem::Requirement
216
+ requirement: &id019 !ruby/object:Gem::Requirement
85
217
  none: false
86
218
  requirements:
87
219
  - - ">="
@@ -89,10 +221,10 @@ dependencies:
89
221
  version: "0"
90
222
  type: :development
91
223
  prerelease: false
92
- version_requirements: *id007
224
+ version_requirements: *id019
93
225
  - !ruby/object:Gem::Dependency
94
226
  name: dm-types
95
- requirement: &id008 !ruby/object:Gem::Requirement
227
+ requirement: &id020 !ruby/object:Gem::Requirement
96
228
  none: false
97
229
  requirements:
98
230
  - - ">="
@@ -100,10 +232,10 @@ dependencies:
100
232
  version: "0"
101
233
  type: :development
102
234
  prerelease: false
103
- version_requirements: *id008
235
+ version_requirements: *id020
104
236
  - !ruby/object:Gem::Dependency
105
237
  name: dm-transactions
106
- requirement: &id009 !ruby/object:Gem::Requirement
238
+ requirement: &id021 !ruby/object:Gem::Requirement
107
239
  none: false
108
240
  requirements:
109
241
  - - ">="
@@ -111,10 +243,10 @@ dependencies:
111
243
  version: "0"
112
244
  type: :development
113
245
  prerelease: false
114
- version_requirements: *id009
246
+ version_requirements: *id021
115
247
  - !ruby/object:Gem::Dependency
116
248
  name: dm-aggregates
117
- requirement: &id010 !ruby/object:Gem::Requirement
249
+ requirement: &id022 !ruby/object:Gem::Requirement
118
250
  none: false
119
251
  requirements:
120
252
  - - ">="
@@ -122,10 +254,10 @@ dependencies:
122
254
  version: "0"
123
255
  type: :development
124
256
  prerelease: false
125
- version_requirements: *id010
257
+ version_requirements: *id022
126
258
  - !ruby/object:Gem::Dependency
127
259
  name: dm-validations
128
- requirement: &id011 !ruby/object:Gem::Requirement
260
+ requirement: &id023 !ruby/object:Gem::Requirement
129
261
  none: false
130
262
  requirements:
131
263
  - - ">="
@@ -133,10 +265,10 @@ dependencies:
133
265
  version: "0"
134
266
  type: :development
135
267
  prerelease: false
136
- version_requirements: *id011
268
+ version_requirements: *id023
137
269
  - !ruby/object:Gem::Dependency
138
270
  name: dm-serializer
139
- requirement: &id012 !ruby/object:Gem::Requirement
271
+ requirement: &id024 !ruby/object:Gem::Requirement
140
272
  none: false
141
273
  requirements:
142
274
  - - ">="
@@ -144,10 +276,10 @@ dependencies:
144
276
  version: "0"
145
277
  type: :development
146
278
  prerelease: false
147
- version_requirements: *id012
279
+ version_requirements: *id024
148
280
  - !ruby/object:Gem::Dependency
149
281
  name: dm-timestamps
150
- requirement: &id013 !ruby/object:Gem::Requirement
282
+ requirement: &id025 !ruby/object:Gem::Requirement
151
283
  none: false
152
284
  requirements:
153
285
  - - ">="
@@ -155,10 +287,10 @@ dependencies:
155
287
  version: "0"
156
288
  type: :development
157
289
  prerelease: false
158
- version_requirements: *id013
290
+ version_requirements: *id025
159
291
  - !ruby/object:Gem::Dependency
160
292
  name: dm-sqlite-adapter
161
- requirement: &id014 !ruby/object:Gem::Requirement
293
+ requirement: &id026 !ruby/object:Gem::Requirement
162
294
  none: false
163
295
  requirements:
164
296
  - - ">="
@@ -166,10 +298,10 @@ dependencies:
166
298
  version: "0"
167
299
  type: :development
168
300
  prerelease: false
169
- version_requirements: *id014
301
+ version_requirements: *id026
170
302
  - !ruby/object:Gem::Dependency
171
303
  name: dm-postgres-adapter
172
- requirement: &id015 !ruby/object:Gem::Requirement
304
+ requirement: &id027 !ruby/object:Gem::Requirement
173
305
  none: false
174
306
  requirements:
175
307
  - - ">="
@@ -177,7 +309,18 @@ dependencies:
177
309
  version: "0"
178
310
  type: :development
179
311
  prerelease: false
180
- version_requirements: *id015
312
+ version_requirements: *id027
313
+ - !ruby/object:Gem::Dependency
314
+ name: progressbar
315
+ requirement: &id028 !ruby/object:Gem::Requirement
316
+ none: false
317
+ requirements:
318
+ - - ">="
319
+ - !ruby/object:Gem::Version
320
+ version: "0"
321
+ type: :development
322
+ prerelease: false
323
+ version_requirements: *id028
181
324
  description: |-
182
325
  An easy way to collect and analyze data about password
183
326
  databases.
@@ -242,7 +385,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
242
385
  requirements:
243
386
  - - ">="
244
387
  - !ruby/object:Gem::Version
245
- hash: 536136881655303577
388
+ hash: 1240768379415668793
246
389
  segments:
247
390
  - 0
248
391
  version: "0"