hivemind-ruby 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 0aadfd31a3b3cd254d99798feb136cc48ff56c6c
4
+ data.tar.gz: 4d36c10d1159ee476f38e6ed95cd6086f5ce4310
5
+ SHA512:
6
+ metadata.gz: fdc2b61b8a77f7ee6c1c62ceb70231b6c078fd93afbf8b2dddf72ffb73eed7202fde083904872b1e7270ebfdeeb7a59d00dfd03e998d418e7c4465d8d246d4e2
7
+ data.tar.gz: cf68ddcdb0f3b003b53b6e7a32cd693e0869276524f644384b7b3e6993fdc50217cce19836c066eb74c0018dbdb1d56c1f47b594f8f1b9a8f52de7f4debe60a2
data/.gitignore ADDED
@@ -0,0 +1,10 @@
1
+ /.bundle/
2
+ Gemfile.lock
3
+ /.yardoc
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /tmp/
9
+ *.gem
10
+ **/.DS_Store
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+ git_source(:github) { |repo| "https://github.com/#{repo}.git" }
3
+
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright 2018 Anthony Martin
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,219 @@
1
+ [![Gem Version](https://badge.fury.io/rb/hivemind-ruby.svg)](https://badge.fury.io/rb/hivemind-ruby)
2
+ [![Inline docs](http://inch-ci.org/github/steemit/hivemind-ruby.svg?branch=master&style=shields)](http://inch-ci.org/github/steemit/hivemind-ruby)
3
+
4
+ # `hivemind-ruby` - Object Relational Mapping for Hivemind
5
+
6
+ If you run your own `hivemind` node, you can leverage your local subset of the blockchain you've synchronized to Postgres using ActiveRecord.
7
+
8
+ What does that mean? It means if, for example, you have an existing Rails application, you can use this gem to access your `hivemind` node and access all of the data stored there.
9
+
10
+ Full documentation: http://www.rubydoc.info/gems/hivemind-ruby
11
+
12
+ ## Overview
13
+
14
+ ##### What problem does this tool solve?
15
+
16
+ It allows you to bypass `steemd` and access a representation of the blockchain maintained by `hivemind` for certain types of queries.
17
+
18
+ ##### Why would I want to bypass `steemd`? Isn't that the authoritative way to access the blockchain?
19
+
20
+ Yes, `steemd` is authoritative. But for certain kinds of queries, there are alternatives. For example, if you wanted to find all users that have the letter `z` in their account name, `steemd` is not the best tool.
21
+
22
+ Instead, we can use an SQL statement like:
23
+
24
+ ```sql
25
+ SELECT "hive_accounts".*
26
+ FROM "hive_accounts"
27
+ WHERE (name LIKE '%z%');
28
+ ```
29
+
30
+ But, you don't have to write this SQL. You can let ActiveRecord do it instead:
31
+
32
+ ```ruby
33
+ Hive::Account.where("name LIKE ?", '%z%')
34
+ ```
35
+
36
+ To do this query comprehensively with `steemd` would require hours of API calls to find accounts that match. But using SQL, it takes less than a second.
37
+
38
+ ##### What can I query with `hivemind`?
39
+
40
+ The focus of `hivemind` is on communities:
41
+
42
+ [Developer-friendly microservice powering social networks on the Steem blockchain.
43
+ ](https://github.com/steemit/hivemind#developer-friendly-microservice-powering-social-networks-on-the-steem-blockchain)
44
+
45
+ > Hive is a "consensus interpretation" layer for the Steem blockchain, maintaining the state of social features such as post feeds, follows, and communities. Written in Python, it synchronizes an SQL database with chain state, providing developers with a more flexible/extensible alternative to the raw steemd API.
46
+
47
+ You should refer to the main `hivemind` [purpose](https://github.com/steemit/hivemind#purpose) to determine specifically what blockchain data it will index.
48
+
49
+ This tool will allow rubyists to access the same database that `hivemind` maintains.
50
+
51
+ ## Getting Started
52
+
53
+ The hivemind-ruby gem is compatible with Ruby 2.2.5 or later. Also targets [ActiveRecord 5.2](https://github.com/rails/rails/blob/v5.2.0/activerecord/CHANGELOG.md#rails-520-april-09-2018), but older versions may work as well.
54
+
55
+ ## Installation
56
+
57
+ *(Assuming that [Ruby is installed](https://www.ruby-lang.org/en/downloads/) on your computer, as well as [RubyGems](http://rubygems.org/pages/download))*
58
+
59
+ First, you need your own [`hivemind`](https://github.com/steemit/hivemind) node. A `hivemind` node requires at least 310GB of HDD storage for the database (as of August of 2018).
60
+
61
+ Also install the minimal set of `PostgreSQL` binaries and headers requried for building 3rd-party applications:
62
+
63
+ ### Linux
64
+
65
+ ```bash
66
+ $ apt-get install libpq-dev
67
+ ```
68
+
69
+ ### macOS
70
+
71
+ Should already be provided if you have `PostgreSQL` provided by homebrew.
72
+
73
+ ### Verify psql
74
+
75
+ Once it's running and all synced, just make sure you can access Postgres locally, e.g.:
76
+
77
+ ```bash
78
+ psql hive
79
+ ```
80
+
81
+ If that works, you can use this gem. Keep in mind, if you stop the sync, your Postgres database will fall behind the head block. Once you resume, `hivemind` will pick up where it left off.
82
+
83
+ *It's that easy!*
84
+
85
+ Add this line to your application's Gemfile:
86
+
87
+ ```ruby
88
+ gem 'hivemind-ruby', require: 'hive'
89
+ ```
90
+
91
+ And then execute:
92
+ ```bash
93
+ $ bundle install
94
+ ```
95
+
96
+ Or install it yourself as:
97
+ ```bash
98
+ $ gem install hivemind-ruby
99
+ ```
100
+
101
+ ## Usage
102
+
103
+ If you've already installed the gem, to check on your configuration, use this command from the terminal:
104
+
105
+ ```bash
106
+ export DATABASE_URL=postgresql://user:pass@localhost:5432/hive
107
+ hivmind-ruby
108
+ ```
109
+
110
+ Or, if you'd like to use the interactive ruby console with `hive` already required:
111
+
112
+ ```bash
113
+ hivmind-ruby console
114
+ ```
115
+
116
+ Here are a bunch of ActiveRecord queries you can do in your app (or from the interactive console).
117
+
118
+ This will return the number of accounts currently synced:
119
+
120
+ ```ruby
121
+ Hive::Account.count
122
+ ```
123
+
124
+ To do the same as accounts, but for posts, this counts everything, including root posts and comments:
125
+
126
+ ```ruby
127
+ Hive::Post.count
128
+ ```
129
+
130
+ This counts just the number of root posts (not comments):
131
+
132
+ ```ruby
133
+ Hive::Post.root_posts.count
134
+ ```
135
+
136
+ This counts just the number of comments (not root posts):
137
+
138
+ ```ruby
139
+ Hive::Post.replies.count
140
+ Hive::Post.replies(parent_author: 'alice').count # just replies to alice
141
+ Hive::Post.replies(parent_author: %w(alice bob)).count # just replies to alice or bob
142
+ ```
143
+
144
+ This will report the number of accounts with `z` in their name:
145
+
146
+ ```ruby
147
+ accounts = Hive::Account.where("name LIKE ?", '%z%')
148
+ accounts.count
149
+ ```
150
+
151
+ This will show you the number of root posts by `alice`:
152
+
153
+ ```ruby
154
+ alice = Hive::Account.find_by_name 'alice'
155
+ alice.posts.root_posts.count
156
+ ```
157
+
158
+ This will show you the number of reblogs by `alice`:
159
+
160
+ ```ruby
161
+ alice = Hive::Account.find_by_name 'alice'
162
+ alice.reblogged_posts.count
163
+ ```
164
+
165
+ This is the number of accounts that `alice` follows:
166
+
167
+ ```ruby
168
+ alice = Hive::Account.find_by_name 'alice'
169
+ alice.following.count
170
+ ```
171
+
172
+ This is the number of accounts that follow `alice`:
173
+
174
+ ```ruby
175
+ alice = Hive::Account.find_by_name 'alice'
176
+ alice.followers.count
177
+ ```
178
+
179
+ The entire feed for `alice` (all content created by accounts `alice` follows, roughly analogous to https://steemit.com/@alice/feed):
180
+
181
+ ```ruby
182
+ alice = Hive::Account.find_by_name 'alice'
183
+ alice.feed.count
184
+ ```
185
+
186
+ This is the entire discussion for the first post on the Steem platform (e.g.: https://steemit.com/@steemit/firstpost):
187
+
188
+ ```ruby
189
+ firstpost = Hive::Post.first
190
+ firstpost.discussion.count
191
+ ```
192
+
193
+ To get the direct children of a post:
194
+
195
+ ```ruby
196
+ firstpost = Hive::Post.first
197
+ firstpost.children.count
198
+ ```
199
+
200
+ ### Tests
201
+
202
+ * Clone the client repository into a directory of your choice:
203
+ * `git clone https://github.com/steemit/hivemind-ruby.git`
204
+ * Navigate into the new folder
205
+ * `cd hivemind-ruby`
206
+ * All tests can be invoked as follows:
207
+ * `DATABASE_URL=postgresql://user:pass@localhost:5432/hive bundle exec rake test`
208
+
209
+ ## Contributions
210
+
211
+ Patches are welcome! Contributors are listed in the `hivemind-ruby.gemspec` file. Please run the tests (`rake test`) before opening a pull request and make sure that you are passing all of them. If you would like to contribute, but don't know what to work on, check the issues list.
212
+
213
+ ## Issues
214
+
215
+ When you find issues, please report them!
216
+
217
+ ## License
218
+
219
+ MIT
data/Rakefile ADDED
@@ -0,0 +1,20 @@
1
+ require 'bundler/gem_tasks'
2
+ require 'rake/testtask'
3
+ require 'hive'
4
+
5
+ Rake::TestTask.new(:test) do |t|
6
+ t.libs << 'test'
7
+ t.libs << 'lib'
8
+ t.test_files = FileList['test/**/*_test.rb']
9
+ t.ruby_opts << if ENV['HELL_ENABLED']
10
+ '-W2'
11
+ else
12
+ '-W1'
13
+ end
14
+ end
15
+
16
+ task default: :test
17
+
18
+ task :console do
19
+ exec 'irb -r hive -I ./lib'
20
+ end
data/bin/hivemind-ruby ADDED
@@ -0,0 +1,31 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'hive'
4
+
5
+ config = Hive::Base.connection_config.dup
6
+ config[:password] = '*' * 8
7
+
8
+ puts "hivemind-ruby-#{Hive::VERSION}"
9
+
10
+ puts "Database config:"
11
+ config.each do |k, v|
12
+ puts "\t#{k}: #{v}"
13
+ end
14
+
15
+ case ARGV[0]
16
+ when 'console' then exec 'irb -r hive -I #{__FILE__}/../lib'
17
+ else
18
+ last_block = begin
19
+ Hive::Block.last
20
+ rescue => e
21
+ puts e.inspect
22
+ exit
23
+ end
24
+
25
+ puts "Totals synced so far:"
26
+ puts "\tBlocks: #{last_block.num} (#{last_block.created_at})"
27
+ puts "\tAccounts: #{Hive::Account.count}"
28
+ puts "\tPosts: #{Hive::Post.count}"
29
+ puts "\tReblogs: #{Hive::Reblog.count}"
30
+ puts "\tFollows: #{Hive::Follow.count}"
31
+ end
@@ -0,0 +1,49 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+
5
+ require 'hive/version'
6
+
7
+ Gem::Specification.new do |s|
8
+ s.name = 'hivemind-ruby'
9
+ s.version = Hive::VERSION
10
+ s.authors = ['Anthony Martin']
11
+ s.email = ['anthony@steemit.com']
12
+
13
+ s.summary = 'STEEM Hivemind for Ruby.'
14
+ s.description = 'If you run your own `hivemind` node, you can leverage your local subset of the blockchain you\'ve synchronied to Postgres using ActiveRecord.'
15
+ s.homepage = 'https://github.com/steemit/hivemind-ruby'
16
+ s.license = 'MIT'
17
+
18
+ s.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test)/}) }
19
+ s.bindir = 'bin'
20
+ s.executables = 'hivemind-ruby'
21
+ s.require_paths = ['lib']
22
+
23
+ # Ruby Make (interprets the Rakefile DSL).
24
+ s.add_development_dependency 'rake', '~> 12.3', '>= 12.3.1'
25
+
26
+ # Minitest is a simple unittest suite.
27
+ s.add_development_dependency 'minitest', '~> 5.10', '>= 5.10.3'
28
+
29
+ # Provies a way to run an individual test by giving a line number argument.
30
+ s.add_development_dependency 'minitest-line', '~> 0.6', '>= 0.6.4'
31
+
32
+ # Forces all tests to do at least one assert/refute.
33
+ s.add_development_dependency 'minitest-proveit', '~> 1.0', '>= 1.0.0'
34
+
35
+ # Simple Code Coverage.
36
+ s.add_development_dependency 'simplecov', '~> 0.15', '>= 0.15.1'
37
+
38
+ # Rails style Object Relational Mapping
39
+ s.add_dependency 'activerecord', ['>= 4', '< 6']
40
+
41
+ # Postgres AR driver.
42
+ s.add_dependency 'pg', '~> 0.21'
43
+
44
+ # Deals with hive schema use of AR keywords in columns like hive_blocks.hash.
45
+ s.add_dependency 'safe_attributes'
46
+
47
+ # Deals with hive schema use of composite primary keys.
48
+ s.add_dependency 'composite_primary_keys'
49
+ end
data/lib/hive.rb ADDED
@@ -0,0 +1,21 @@
1
+ require 'hive/version'
2
+
3
+ require 'hive/models/base'
4
+ require 'hive/models/account'
5
+ require 'hive/models/block'
6
+ require 'hive/models/community'
7
+ require 'hive/models/feed_cache'
8
+ require 'hive/models/flag'
9
+ require 'hive/models/follow'
10
+ require 'hive/models/member'
11
+ require 'hive/models/modlog'
12
+ require 'hive/models/payment'
13
+ require 'hive/models/post_tag'
14
+ require 'hive/models/post'
15
+ require 'hive/models/posts_cache'
16
+ require 'hive/models/reblog'
17
+ require 'hive/models/state'
18
+
19
+ # Access a Hivemind database with ActiveRecord.
20
+ module Hive
21
+ end
@@ -0,0 +1,138 @@
1
+ module Hive
2
+
3
+ # To find an {Hive::Account}, use the following idiom:
4
+ #
5
+ # alice = Hive::Account.find_by_name 'alice'
6
+ #
7
+ # An account has many {Hive::Post} records. To access associated posts, use:
8
+ #
9
+ # alice.posts
10
+ #
11
+ # Like posts, account has many {Hive::PostsCache} records. There are two ways
12
+ # to access an account's related {Hive::PostsCache} records.
13
+ #
14
+ # alice.posts.joins(:cache) # which includes posts_cache fields
15
+ # alice.posts_cache # automatically joins posts to get posts_cache
16
+ #
17
+ # An account also has many {Hive::Reblog} records that can be used to access
18
+ # reblogged posts, which are the posts that the account has reblogged
19
+ # (aka re-steemed):
20
+ #
21
+ # alice.reblogged_posts
22
+ #
23
+ # Accounts also have access to {Hive::Follow} for various things like "follow"
24
+ # and "mute".
25
+ #
26
+ # alice.following # accounts that this account is following
27
+ # alice.followers # accounts that follow this account
28
+ # alice.muting # accounts that this account is muting
29
+ # alice.muters # accounts that mute this account
30
+ #
31
+ # Post promotions are tracked by {Hive::Payment} and associated with accounts
32
+ # as well. To get a list of posts that this account has promoted:
33
+ #
34
+ # alice.promoted_posts
35
+ #
36
+ # Also, you can get a list of all accounts that this account has promoted at
37
+ # some point.
38
+ #
39
+ # alice.promoted_authors
40
+ #
41
+ # This is the sum of all post promotion by this account, grouped by the author
42
+ # being promoted:
43
+ #
44
+ # puts JSON.pretty_generate alice.payments.
45
+ # joins(:post).group(:author).sum(:amount)
46
+ #
47
+ # This scope will limit the number of accounts to those who have only ever
48
+ # posted *n* times:
49
+ #
50
+ # Hive::Account.root_posts_count(1)
51
+ # Hive::Account.root_posts_count(1..5) # accounts with between 1 and 5 posts
52
+ class Account < Base
53
+ self.table_name = :hive_accounts
54
+
55
+ has_many :posts, primary_key: :name, foreign_key: :author, inverse_of: :author_account
56
+ has_many :posts_cache, through: :posts, source: :cache
57
+
58
+ has_many :reblogs, primary_key: :name, foreign_key: :account, inverse_of: :reblogger
59
+ has_many :reblogged_posts, through: :reblogs, source: :post
60
+ has_many :inverse_rebloggers, through: :reblogged_posts, source: :rebloggers
61
+
62
+ has_many :raw_follows, foreign_key: :follower, class_name: 'Follow'
63
+ has_many :inverse_raw_follows, foreign_key: :following, class_name: 'Follow'
64
+
65
+ has_many :follows, -> { state(:follow) }, foreign_key: :follower, class_name: 'Follow'
66
+ has_many :following, through: :follows, source: :following_account
67
+ has_many :inverse_follows, -> { state(:follow) }, foreign_key: :following, class_name: 'Follow'
68
+ has_many :followers, through: :inverse_follows, source: :follower_account
69
+
70
+ has_many :mutes, -> { state(:mute) }, foreign_key: :follower, class_name: 'Follow'
71
+ has_many :muting, through: :mutes, source: :following_account
72
+ has_many :inverse_mutes, -> { state(:mute) }, foreign_key: :following, class_name: 'Follow'
73
+ has_many :muters, through: :inverse_mutes, source: :follower_account
74
+
75
+ has_many :payments, -> { to_null.token('SBD') }, foreign_key: :from_account, source: :from
76
+ has_many :promoted_posts, through: :payments, source: :post
77
+ has_many :promoted_authors, -> { distinct }, through: :promoted_posts, source: :author_account
78
+
79
+ has_many :memberships, primary_key: :name, foreign_key: :account, class_name: 'Member'
80
+
81
+ has_many :feed_cache
82
+ has_many :feed_posts, through: :feed_cache, source: :post
83
+
84
+ scope :root_posts_count, lambda { |count, options = {invert: false}|
85
+ clause = <<~DONE
86
+ (
87
+ SELECT COUNT(hive_posts.id) FROM hive_posts
88
+ WHERE hive_posts.author = hive_accounts.name
89
+ AND hive_posts.depth = 0
90
+ AND hive_posts.parent_id IS NULL
91
+ )
92
+ DONE
93
+
94
+ clause += 'NOT ' if !!options[:invert]
95
+
96
+ r = case count
97
+ when Range
98
+ count = [count.first, count.last]
99
+ clause += 'BETWEEN ? AND ?'
100
+
101
+ where(clause, *count)
102
+ else
103
+ count = [count].flatten
104
+ clause += 'IN (?)'
105
+
106
+ where(clause, count)
107
+ end
108
+ }
109
+
110
+ # The entire feed for this account, as in, all content created/reblogged by
111
+ # authors that this account follows.
112
+ #
113
+ # @return {ActiveRecord::Relation}
114
+ def feed_cache; Post.feed_cache(following); end
115
+
116
+ # The entire feed for this account, as in, all content created/reblogged by
117
+ # authors that this account follows.
118
+ #
119
+ # @return {ActiveRecord::Relation}
120
+ def feed; Post.feed(following); end
121
+
122
+ # The entire ignred feed for this account, as in, all content
123
+ # created/reblogged by authors that this account mutes.
124
+ #
125
+ # @return {ActiveRecord::Relation}
126
+ def ignored_feed_cache; Post.feed_cache(muting); end
127
+
128
+ # The entire ignred feed for this account, as in, all content
129
+ # created/reblogged by authors that this account mutes.
130
+ #
131
+ # @return {ActiveRecord::Relation}
132
+ def ignored_feed; Post.feed(muting); end
133
+
134
+ # All comments that have replied to this account, as in, all content that
135
+ # has the parent_author as this account.
136
+ def replies; Post.replies(parent_author: self); end
137
+ end
138
+ end