ReadRobotstxt 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/lib/Robots/version.rb +1 -1
- data/lib/Robots.rb +0 -1
- metadata +3 -6
- data/README.md +0 -34
- data/Rakefile +0 -4
- data/sig/Robots.rbs +0 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 77e4a680402a0213053b6cc3b7fa825a332a9e6d8c61a42842fa9461dd814c79
|
|
4
|
+
data.tar.gz: 66811f73150550f72f935e19823c29a639b5a1c4d79302ae90e1b0b7884e892a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3a35d73d66bc13cc565c920601508ad591e6c7b68559713419a4e395e722e7803bcd742c1539853f2ec4ed6b441a568fb01fb2795d2f4d4fd07abcb6d73c6067
|
|
7
|
+
data.tar.gz: bffeb4a05a3fe498e14eee5648d7933799c88ebd167ee594d17c7016d352df369c688293d110f918d96ff77bf98e727b23d34ce616fe04574df7d18f1ae534d1
|
data/lib/Robots/version.rb
CHANGED
data/lib/Robots.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ReadRobotstxt
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Michael-Meade
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2025-
|
|
11
|
+
date: 2025-11-01 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: Read Robots.txt files
|
|
14
14
|
email:
|
|
@@ -17,11 +17,8 @@ executables: []
|
|
|
17
17
|
extensions: []
|
|
18
18
|
extra_rdoc_files: []
|
|
19
19
|
files:
|
|
20
|
-
- README.md
|
|
21
|
-
- Rakefile
|
|
22
20
|
- lib/Robots.rb
|
|
23
21
|
- lib/Robots/version.rb
|
|
24
|
-
- sig/Robots.rbs
|
|
25
22
|
homepage:
|
|
26
23
|
licenses: []
|
|
27
24
|
metadata: {}
|
|
@@ -33,7 +30,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
33
30
|
requirements:
|
|
34
31
|
- - ">="
|
|
35
32
|
- !ruby/object:Gem::Version
|
|
36
|
-
version:
|
|
33
|
+
version: '0'
|
|
37
34
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
38
35
|
requirements:
|
|
39
36
|
- - ">="
|
data/README.md
DELETED
|
@@ -1,34 +0,0 @@
|
|
|
1
|
-
# Robots
|
|
2
|
-
Get robots.txt information from a site.
|
|
3
|
-
|
|
4
|
-
## Installation
|
|
5
|
-
|
|
6
|
-
```ruby
|
|
7
|
-
gem install Robots-txt
|
|
8
|
-
```
|
|
9
|
-
|
|
10
|
-
```bash
|
|
11
|
-
bundle add Robots-txt
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
If bundler is not being used to manage dependencies, install the gem by executing:
|
|
15
|
-
|
|
16
|
-
```bash
|
|
17
|
-
gem install Robots-txt
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
## Usage
|
|
21
|
-
|
|
22
|
-
```ruby
|
|
23
|
-
require './lib/robots'
|
|
24
|
-
|
|
25
|
-
u = Robots::Url.new('https://www.ebay.com')
|
|
26
|
-
u.allow
|
|
27
|
-
u.disallow
|
|
28
|
-
u.disallow_removed
|
|
29
|
-
u.allow_removed
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
## Contributing
|
|
33
|
-
|
|
34
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/Robots.
|
data/Rakefile
DELETED
data/sig/Robots.rbs
DELETED