haveapi-fs 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.yardopts +1 -0
- data/CHANGELOG +2 -0
- data/Gemfile +3 -0
- data/LICENSE.txt +21 -0
- data/README.md +338 -0
- data/Rakefile +1 -0
- data/assets/css/bootstrap.min.css +6 -0
- data/assets/css/local.css +11 -0
- data/bin/haveapi-fs +5 -0
- data/haveapi-fs.gemspec +30 -0
- data/lib/core_ext/string.rb +9 -0
- data/lib/haveapi/fs/auth/base.rb +56 -0
- data/lib/haveapi/fs/auth/basic.rb +29 -0
- data/lib/haveapi/fs/auth/noauth.rb +9 -0
- data/lib/haveapi/fs/auth/token.rb +39 -0
- data/lib/haveapi/fs/cache.rb +71 -0
- data/lib/haveapi/fs/cleaner.rb +56 -0
- data/lib/haveapi/fs/component.rb +237 -0
- data/lib/haveapi/fs/components/action_dir.rb +106 -0
- data/lib/haveapi/fs/components/action_errors.rb +49 -0
- data/lib/haveapi/fs/components/action_exec.rb +12 -0
- data/lib/haveapi/fs/components/action_exec_edit.rb +90 -0
- data/lib/haveapi/fs/components/action_input.rb +40 -0
- data/lib/haveapi/fs/components/action_message.rb +19 -0
- data/lib/haveapi/fs/components/action_output.rb +79 -0
- data/lib/haveapi/fs/components/action_status.rb +32 -0
- data/lib/haveapi/fs/components/cache_stats.rb +19 -0
- data/lib/haveapi/fs/components/component_list.rb +24 -0
- data/lib/haveapi/fs/components/create_action_dir.rb +15 -0
- data/lib/haveapi/fs/components/delete_action_dir.rb +22 -0
- data/lib/haveapi/fs/components/directory.rb +45 -0
- data/lib/haveapi/fs/components/directory_reset.rb +8 -0
- data/lib/haveapi/fs/components/executable.rb +75 -0
- data/lib/haveapi/fs/components/file.rb +43 -0
- data/lib/haveapi/fs/components/groff_help_file.rb +9 -0
- data/lib/haveapi/fs/components/help_file.rb +28 -0
- data/lib/haveapi/fs/components/html_help_file.rb +24 -0
- data/lib/haveapi/fs/components/index_filter.rb +63 -0
- data/lib/haveapi/fs/components/info_files.rb +19 -0
- data/lib/haveapi/fs/components/instance_create.rb +20 -0
- data/lib/haveapi/fs/components/instance_edit.rb +49 -0
- data/lib/haveapi/fs/components/list_item.rb +28 -0
- data/lib/haveapi/fs/components/md_help_file.rb +24 -0
- data/lib/haveapi/fs/components/meta_dir.rb +42 -0
- data/lib/haveapi/fs/components/meta_file.rb +21 -0
- data/lib/haveapi/fs/components/parameter.rb +132 -0
- data/lib/haveapi/fs/components/pry.rb +9 -0
- data/lib/haveapi/fs/components/remote_control_file.rb +92 -0
- data/lib/haveapi/fs/components/resource_action_dir.rb +72 -0
- data/lib/haveapi/fs/components/resource_dir.rb +161 -0
- data/lib/haveapi/fs/components/resource_id.rb +15 -0
- data/lib/haveapi/fs/components/resource_instance_dir.rb +146 -0
- data/lib/haveapi/fs/components/rfuse_check.rb +3 -0
- data/lib/haveapi/fs/components/root.rb +75 -0
- data/lib/haveapi/fs/components/save_instance.rb +11 -0
- data/lib/haveapi/fs/components/unsaved_list.rb +24 -0
- data/lib/haveapi/fs/components/update_action_dir.rb +31 -0
- data/lib/haveapi/fs/context.rb +43 -0
- data/lib/haveapi/fs/exceptions.rb +0 -0
- data/lib/haveapi/fs/factory.rb +59 -0
- data/lib/haveapi/fs/fs.rb +198 -0
- data/lib/haveapi/fs/help.rb +91 -0
- data/lib/haveapi/fs/main.rb +134 -0
- data/lib/haveapi/fs/remote_control.rb +29 -0
- data/lib/haveapi/fs/version.rb +5 -0
- data/lib/haveapi/fs/worker.rb +77 -0
- data/lib/haveapi/fs.rb +65 -0
- data/templates/help/html/action_dir.erb +33 -0
- data/templates/help/html/action_errors.erb +4 -0
- data/templates/help/html/action_input.erb +40 -0
- data/templates/help/html/action_output.erb +21 -0
- data/templates/help/html/index_filter.erb +16 -0
- data/templates/help/html/layout.erb +45 -0
- data/templates/help/html/resource_action_dir.erb +18 -0
- data/templates/help/html/resource_dir.erb +18 -0
- data/templates/help/html/resource_instance_dir.erb +64 -0
- data/templates/help/html/root.erb +42 -0
- data/templates/help/md/action_dir.erb +29 -0
- data/templates/help/md/action_errors.erb +2 -0
- data/templates/help/md/action_input.erb +23 -0
- data/templates/help/md/action_output.erb +11 -0
- data/templates/help/md/index_filter.erb +11 -0
- data/templates/help/md/layout.erb +14 -0
- data/templates/help/md/resource_action_dir.erb +10 -0
- data/templates/help/md/resource_dir.erb +15 -0
- data/templates/help/md/resource_instance_dir.erb +42 -0
- data/templates/help/md/root.erb +34 -0
- metadata +231 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 6a14a50210ec4ce355b9ecc5dae75ea593a86036
|
4
|
+
data.tar.gz: dac3cc874105f07b4ebae90e9b9642f4c1640979
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 7c671e57fc944d0b69bb3e965302250931bfb5fd3c2e79a131bd4e3fba0bf0ed96b87966f40bf7606af6ff050a050fc55b980db26c6d3dd42c888034c6515188
|
7
|
+
data.tar.gz: 19e2183b8864263c8efe4499d64854e739aca10169ad8f952b5ac1a6c02dd0ef637a4bc24d3aa8d3dcd2b0a7c377f9b20d30ba783de2f10cf34945ef3f7b6148
|
data/.yardopts
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
--files 'lib/**/*.rb' --output-dir=html_doc --markup markdown
|
data/CHANGELOG
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2016 Jakub Skokan
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,338 @@
|
|
1
|
+
HaveAPI Filesystem
|
2
|
+
==================
|
3
|
+
`haveapi-fs` is a virtual read-write file system created using FUSE. It works
|
4
|
+
with any API based on [HaveAPI](https://github.com/vpsfreecz/haveapi) and
|
5
|
+
allows it to be browsed and interacted with as directories and files. The file
|
6
|
+
system can be easily extended to add custom directories/files or modify
|
7
|
+
behaviour for your API's needs.
|
8
|
+
|
9
|
+
## Requirements
|
10
|
+
FUSE has to be enabled in kernel and userspace utilities installed, e.g. on
|
11
|
+
Debian:
|
12
|
+
|
13
|
+
$ apt-get install fuse libfuse-dev
|
14
|
+
|
15
|
+
## Installation
|
16
|
+
|
17
|
+
$ gem install haveapi-fs
|
18
|
+
|
19
|
+
## Usage
|
20
|
+
|
21
|
+
$ haveapi-fs -h
|
22
|
+
Usage:
|
23
|
+
haveapi-fs api_url mountpoint [-h] [-d] [-o [opt,optkey=value,...]]
|
24
|
+
|
25
|
+
Fuse options: (2.9)
|
26
|
+
-h help - print this help output
|
27
|
+
-d |-o debug enable internal FUSE debug output
|
28
|
+
|
29
|
+
Filesystem options:
|
30
|
+
version=VERSION API version to use
|
31
|
+
auth_method=METHOD Authentication method (basic, token, noauth)
|
32
|
+
user Username
|
33
|
+
password Password
|
34
|
+
token Authentication token
|
35
|
+
nodaemonize Stay in the foreground
|
36
|
+
log Enable logging while daemonized
|
37
|
+
index_limit=LIMIT Limit number of objects in resource directory
|
38
|
+
|
39
|
+
## `/etc/fstab` entry
|
40
|
+
|
41
|
+
There are two ways in which `haveapi-fs` can be added to `/etc/fstab`. Both
|
42
|
+
require that the `haveapi-fs` executable is in `$PATH`. The first approach is
|
43
|
+
to use `mount.fuse`:
|
44
|
+
|
45
|
+
# <fs> <mountpoint> <type> <opts> <dump/pass>
|
46
|
+
haveapi-fs#https://api.domain.tld /mnt/api.domain.tld fuse user 0 0
|
47
|
+
|
48
|
+
The other way is to link `/sbin/mount.haveapi-fs` to `haveapi-fs`, e.g.:
|
49
|
+
|
50
|
+
$ ln -s `which haveapi-fs` /sbin/mount.haveapi-fs
|
51
|
+
|
52
|
+
Then we can use filesystem type `haveapi-fs` in fstab directly:
|
53
|
+
|
54
|
+
# <fs> <mountpoint> <type> <opts> <dump/pass>
|
55
|
+
https://api.domain.tld /mnt/api.domain.tld haveapi-fs user 0 0
|
56
|
+
|
57
|
+
## Example
|
58
|
+
The following example uses [vpsadmin-api](https://github.com/vpsfreecz/vpsadmin-api),
|
59
|
+
which requires users to be authenticated.
|
60
|
+
|
61
|
+
`haveapi-fs` supports all available authentication methods, it defaults to
|
62
|
+
HTTP basic and it will prompt the user to input credentials, if they are not
|
63
|
+
supplied as options using `-o`.
|
64
|
+
|
65
|
+
$ haveapi-fs https://api.vpsfree.cz /mnt/api.vpsfree.cz
|
66
|
+
User name: <username>
|
67
|
+
Password:
|
68
|
+
|
69
|
+
The root directory contains a list of top-level resources in the API
|
70
|
+
represented by directories. `help.{html,man,md,txt}` files are to be found in
|
71
|
+
every directory in this file system and contain information about the current
|
72
|
+
directory.
|
73
|
+
|
74
|
+
$ cd /mnt/api.vpsfree.cz
|
75
|
+
$ tree -L 1
|
76
|
+
.
|
77
|
+
├── location/
|
78
|
+
├── environment/
|
79
|
+
├── node/
|
80
|
+
├── vps/
|
81
|
+
├── help.html
|
82
|
+
├── help.man
|
83
|
+
├── help.md
|
84
|
+
├── help.txt
|
85
|
+
├── .client_version
|
86
|
+
├── .fs_version
|
87
|
+
├── .protocol_version
|
88
|
+
├── .reset
|
89
|
+
└── .unsaved
|
90
|
+
|
91
|
+
Inside a resource directory we can see the objects themselves as directories
|
92
|
+
whose name is their id.
|
93
|
+
|
94
|
+
$ tree -L 1 vps
|
95
|
+
vps
|
96
|
+
├── 198/
|
97
|
+
├── 199/
|
98
|
+
├── 202/
|
99
|
+
├── actions/
|
100
|
+
├── by-environment/
|
101
|
+
├── by-limit/
|
102
|
+
├── by-location/
|
103
|
+
├── by-node/
|
104
|
+
├── by-object_state/
|
105
|
+
├── by-offset/
|
106
|
+
├── by-os_template/
|
107
|
+
├── create.yml
|
108
|
+
├── help.html
|
109
|
+
├── help.man
|
110
|
+
├── help.md
|
111
|
+
└── help.txt
|
112
|
+
|
113
|
+
Directory `actions/` contains resource-level actions like `Index` and `Create`.
|
114
|
+
Directories named as `by-<input_param>` represent input parameters of action
|
115
|
+
`Index` and serve as a quick way to filter by them, e.g.:
|
116
|
+
|
117
|
+
$ tree -L 1 vps/by-environment/3/by-node/5
|
118
|
+
vps/by-environment/3/by-node/5
|
119
|
+
├── 202/
|
120
|
+
├── actions/
|
121
|
+
├── by-environment/
|
122
|
+
├── by-limit/
|
123
|
+
├── by-location/
|
124
|
+
├── by-node/
|
125
|
+
├── by-object_state/
|
126
|
+
├── by-offset/
|
127
|
+
├── by-os_template/
|
128
|
+
├── create.yml
|
129
|
+
├── help.html
|
130
|
+
├── help.man
|
131
|
+
├── help.md
|
132
|
+
└── help.txt
|
133
|
+
|
134
|
+
Only VPS #202 matches the filters.
|
135
|
+
|
136
|
+
Object directory contains a list of attributes, instance-level actions and
|
137
|
+
subresources. Associated resources can be browsed as directories.
|
138
|
+
|
139
|
+
$ tree -L 1 vps/199/
|
140
|
+
vps/199/
|
141
|
+
├── actions/
|
142
|
+
├── id
|
143
|
+
├── hostname
|
144
|
+
├── node/
|
145
|
+
├── node_id
|
146
|
+
├── os_template/
|
147
|
+
├── os_template_id
|
148
|
+
├── ...
|
149
|
+
├── edit.yml
|
150
|
+
├── help.html
|
151
|
+
├── help.man
|
152
|
+
├── help.md
|
153
|
+
├── help.txt
|
154
|
+
└── save
|
155
|
+
|
156
|
+
Now let's look at the action directory:
|
157
|
+
|
158
|
+
$ tree -L 1 vps/199/actions/update/
|
159
|
+
vps/199/actions/update/
|
160
|
+
├── errors/
|
161
|
+
├── exec
|
162
|
+
├── exec.yml
|
163
|
+
├── help.html
|
164
|
+
├── help.man
|
165
|
+
├── help.md
|
166
|
+
├── help.txt
|
167
|
+
├── input/
|
168
|
+
├── message
|
169
|
+
├── output/
|
170
|
+
├── reset
|
171
|
+
└── status
|
172
|
+
|
173
|
+
On the lowest level, actions are always invoked using file `exec`. All
|
174
|
+
executable files can be run in two ways, either write `1` into them or execute
|
175
|
+
them, e.g.:
|
176
|
+
|
177
|
+
$ echo 1 > vps/199/actions/restart/exec
|
178
|
+
|
179
|
+
is the same as
|
180
|
+
|
181
|
+
$ ./vps/199/actions/restart/exec
|
182
|
+
|
183
|
+
The success of this operation can be checked in files `status`, `message` and
|
184
|
+
directory `errors`. Use executable file `reset` to reset the state of these
|
185
|
+
files.
|
186
|
+
|
187
|
+
Input and output parameters are accessible as files in directories `input`
|
188
|
+
and `output`.
|
189
|
+
|
190
|
+
## Authentication
|
191
|
+
Authentication method is selected using option `auth_method`. Accepted values
|
192
|
+
are `basic`, `token` and `noauth`. If the option is not specified, config file
|
193
|
+
of [haveapi-client](https://github.com/vpsfreecz/haveapi-client) is checked,
|
194
|
+
otherwise it defaults to HTTP basic.
|
195
|
+
|
196
|
+
If needed parameters are not provided as options, the program prompts for them
|
197
|
+
on stdin.
|
198
|
+
|
199
|
+
## Executables
|
200
|
+
All executables can be called either by writing `1` to them or executing them.
|
201
|
+
|
202
|
+
## Run actions using YAML files
|
203
|
+
Resource directory has `create.yml` to create a new instance, instance
|
204
|
+
directory has `edit.yml` to update an instance and actions have `exec.yml`.
|
205
|
+
|
206
|
+
All these files contain a hash of input parameters and their values for
|
207
|
+
respective actions. The action is called when this file is saved, closed and
|
208
|
+
is not empty.
|
209
|
+
|
210
|
+
## Unsaved data
|
211
|
+
The filesystem tracks changed and unsaved files and takes care not to lose
|
212
|
+
them. Normal directories and files are freed from memory after some period of
|
213
|
+
inactivity, but unsaved files are held forever. A list of such files can be
|
214
|
+
found in a hidden file `.unsaved` located in every directory. The file always
|
215
|
+
contains paths to unsaved files in the current directory and all its
|
216
|
+
descendants.
|
217
|
+
|
218
|
+
To drop these unsaved files, use executable `.reset`, which is also located in
|
219
|
+
every directory.
|
220
|
+
|
221
|
+
## Access time, modification time and creation time
|
222
|
+
Access time is always updated, it is used to decide which directories/files may
|
223
|
+
be freed from memory. Components not accessed within the last 10 minutes are
|
224
|
+
regularly freed.
|
225
|
+
|
226
|
+
Modification time is changed only for files representing input parameters or
|
227
|
+
instance attributes.
|
228
|
+
|
229
|
+
Creation time is advertised as the time at which the component (directory/file)
|
230
|
+
was created in memory, i.e. for some components that is when they were fetched
|
231
|
+
from the API. As of now, components with creation time older than 30 minutes
|
232
|
+
are regularly freed from memory to ensure that the files and data you see are
|
233
|
+
still actually in the API, or have not been modified.
|
234
|
+
|
235
|
+
## Limiting number of fetched objects
|
236
|
+
By default, resource dir contains all its objects. For some APIs, it may be
|
237
|
+
undesirable, as they may contain too many objects and it is useless and slow to
|
238
|
+
fetch them all. For this reason, there is option `index_limit`, e.g.
|
239
|
+
`index_limit=2000` to fetch 2000 objects from every resource at most.
|
240
|
+
|
241
|
+
## Extending the file system
|
242
|
+
Internally, every directory and file is represented by a `Component` object.
|
243
|
+
Components are arranged in a tree structure, where directories are branches and
|
244
|
+
files are leaves. Any component can be modified or replaced.
|
245
|
+
|
246
|
+
The following example demonstrates how to add a custom file to an existing
|
247
|
+
component.
|
248
|
+
|
249
|
+
```ruby
|
250
|
+
#!/usr/bin/env ruby
|
251
|
+
require 'haveapi/fs'
|
252
|
+
|
253
|
+
module CustomComponents
|
254
|
+
# Extends ResourceInstanceDir and adds file `extended.txt`
|
255
|
+
class InstanceDir < HaveAPI::Fs::Components::ResourceInstanceDir
|
256
|
+
# If not specified, class name is used as a help file name
|
257
|
+
help_file :resource_instance_dir
|
258
|
+
|
259
|
+
# This method returns directory contents. Return entries from the superclass
|
260
|
+
# and add our own file.
|
261
|
+
def contents
|
262
|
+
super + %w(extended.txt)
|
263
|
+
end
|
264
|
+
|
265
|
+
protected
|
266
|
+
# `new_child` is called whenever a path in the file system is accessed for
|
267
|
+
# the first time. It returns the class and arguments that will represent
|
268
|
+
# a component under `name`.
|
269
|
+
#
|
270
|
+
# First, we let the superclass to try to find the component. If it does not
|
271
|
+
# exist, we check whether it's our custom file.
|
272
|
+
def new_child(name)
|
273
|
+
if child = super
|
274
|
+
child
|
275
|
+
|
276
|
+
elsif name == :'extended.txt'
|
277
|
+
SomeFile
|
278
|
+
|
279
|
+
else
|
280
|
+
nil
|
281
|
+
end
|
282
|
+
end
|
283
|
+
end
|
284
|
+
|
285
|
+
# Component representing `extended.txt`
|
286
|
+
class SomeFile < HaveAPI::Fs::Components::File
|
287
|
+
# This method returns file contents. File size is deduced from the return
|
288
|
+
# value and access times are managed by the file system by default, so this
|
289
|
+
# method is all that is needed. Default permissions make this file
|
290
|
+
# read-only.
|
291
|
+
def read
|
292
|
+
"you have been extended!\n"
|
293
|
+
end
|
294
|
+
end
|
295
|
+
end
|
296
|
+
|
297
|
+
# Replace ResourceInstanceDir with our InstanceDir in component factory.
|
298
|
+
# Whenever the file system would create ResourceInstanceDir instance, it will
|
299
|
+
# create InstanceDir instead.
|
300
|
+
HaveAPI::Fs::Factory.replace(
|
301
|
+
HaveAPI::Fs::Components::ResourceInstanceDir,
|
302
|
+
CustomComponents::InstanceDir
|
303
|
+
)
|
304
|
+
|
305
|
+
# Mount the file system
|
306
|
+
HaveAPI::Fs.main
|
307
|
+
```
|
308
|
+
|
309
|
+
## API requirements
|
310
|
+
For the file system to work correctly, the API has comply with the following
|
311
|
+
requirements:
|
312
|
+
|
313
|
+
- `Index` and `Show` actions must return parameter `id`
|
314
|
+
- `Update` action should return the same parameters as `Show`
|
315
|
+
- If the API uses authentication, it has to have resource `User` and action
|
316
|
+
`User.current` that is like a `Show` for the current user, not knowing his
|
317
|
+
`id` yet
|
318
|
+
- `Index` input parameter `limit` has the meaning from
|
319
|
+
`HaveAPI::Actions::Paginable`
|
320
|
+
|
321
|
+
## Troubleshooting
|
322
|
+
Whenever `haveapi-fs` crashes, throws IO errors or misbehaves, helpful
|
323
|
+
information can be found in the log file. Logging is disabled by default when
|
324
|
+
daemonized, as it can grow large. It can be enabled using option `log`. The log
|
325
|
+
file is located at `~/.haveapi-fs/<api domain>/haveapi-fs.log`.
|
326
|
+
|
327
|
+
If `haveapi-fs` is run in the foreground using option `nodaemonize`, it logs
|
328
|
+
to standard output.
|
329
|
+
|
330
|
+
Whenever reporting an error, send also contents of the log file or search it
|
331
|
+
for a relevant backtrace.
|
332
|
+
|
333
|
+
## Known issues
|
334
|
+
- On some distributions, `man` cannot directly open `help.man` files, because
|
335
|
+
it uses suid bit to run as another user. Other users by default cannot
|
336
|
+
access this file system. This can be changed in `/etc/fuse.conf` and then by
|
337
|
+
using mount option `allow_other`. A workaround is to open the manual
|
338
|
+
indirectly, e.g. `cat help.man | man -l -`
|
data/Rakefile
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
require 'bundler/gem_tasks'
|