wgit 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: wgit
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.0
4
+ version: 0.9.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michael Telford
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-01-27 00:00:00.000000000 Z
11
+ date: 2020-07-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: addressable
@@ -66,6 +66,20 @@ dependencies:
66
66
  - - "~>"
67
67
  - !ruby/object:Gem::Version
68
68
  version: '1.3'
69
+ - !ruby/object:Gem::Dependency
70
+ name: ferrum
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '0.8'
76
+ type: :runtime
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '0.8'
69
83
  - !ruby/object:Gem::Dependency
70
84
  name: byebug
71
85
  requirement: !ruby/object:Gem::Requirement
@@ -184,14 +198,10 @@ dependencies:
184
198
  - - "<"
185
199
  - !ruby/object:Gem::Version
186
200
  version: '1.0'
187
- description: 'Fundamentally, Wgit is a HTTP indexer/scraper which crawls URL''s to
188
- retrieve and serialise their page contents for later use. You can use Wgit to scrape
189
- entire websites if required. Wgit also provides a means to search indexed documents
190
- stored in a database. Therefore, this library provides the main components of a
191
- WWW search engine. The Wgit API is easily extended allowing you to pull out the
192
- parts of a webpage that are important to you, the code snippets or tables for example.
193
- As Wgit is a library, it supports many different use cases including data mining,
194
- analytics, web indexing and URL parsing to name a few.
201
+ description: 'Wgit was primarily designed to crawl static HTML websites to index and
202
+ search their content - providing the basis of any search engine; but Wgit is suitable
203
+ for many application domains including: URL parsing, data mining and statistical
204
+ analysis.
195
205
 
196
206
  '
197
207
  email: michael.telford@live.com
@@ -202,12 +212,14 @@ extra_rdoc_files: []
202
212
  files:
203
213
  - "./lib/wgit.rb"
204
214
  - "./lib/wgit/assertable.rb"
215
+ - "./lib/wgit/base.rb"
205
216
  - "./lib/wgit/core_ext.rb"
206
217
  - "./lib/wgit/crawler.rb"
207
218
  - "./lib/wgit/database/database.rb"
208
219
  - "./lib/wgit/database/model.rb"
209
220
  - "./lib/wgit/document.rb"
210
- - "./lib/wgit/document_extensions.rb"
221
+ - "./lib/wgit/document_extractors.rb"
222
+ - "./lib/wgit/dsl.rb"
211
223
  - "./lib/wgit/indexer.rb"
212
224
  - "./lib/wgit/logger.rb"
213
225
  - "./lib/wgit/response.rb"
@@ -246,9 +258,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
246
258
  - !ruby/object:Gem::Version
247
259
  version: '0'
248
260
  requirements: []
249
- rubygems_version: 3.0.6
261
+ rubygems_version: 3.1.2
250
262
  signing_key:
251
263
  specification_version: 4
252
- summary: Wgit is a Ruby library primarily used for crawling, indexing and searching
253
- HTML webpages.
264
+ summary: Wgit is a HTML web crawler, written in Ruby, that allows you to programmatically
265
+ extract the data you want from the web.
254
266
  test_files: []