@opentermsarchive/engine 0.16.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +7 -1
- package/.env.example +0 -3
- package/.eslintrc.yaml +0 -116
- package/.github/workflows/deploy.yml +0 -50
- package/.github/workflows/release.yml +0 -71
- package/.github/workflows/test.yml +0 -77
- package/CHANGELOG.md +0 -14
- package/CODE_OF_CONDUCT.md +0 -128
- package/CONTRIBUTING.md +0 -143
- package/MIGRATING.md +0 -42
- package/Vagrantfile +0 -38
- package/ansible.cfg +0 -13
- package/decision-records/0001-service-name-and-id.md +0 -73
- package/decision-records/0002-service-history.md +0 -212
- package/decision-records/0003-snapshots-database.md +0 -123
- package/ops/README.md +0 -280
- package/ops/app.yml +0 -5
- package/ops/infra.yml +0 -6
- package/ops/inventories/dev.yml +0 -7
- package/ops/inventories/production.yml +0 -27
- package/ops/roles/infra/defaults/main.yml +0 -2
- package/ops/roles/infra/files/.gitconfig +0 -3
- package/ops/roles/infra/files/mongod.conf +0 -18
- package/ops/roles/infra/files/ota-bot-key.private_key +0 -26
- package/ops/roles/infra/tasks/main.yml +0 -78
- package/ops/roles/infra/tasks/mongo.yml +0 -40
- package/ops/roles/infra/templates/ssh_config.j2 +0 -5
- package/ops/roles/ota/defaults/main.yml +0 -14
- package/ops/roles/ota/files/.env +0 -21
- package/ops/roles/ota/tasks/database.yml +0 -65
- package/ops/roles/ota/tasks/main.yml +0 -110
- package/ops/site.yml +0 -6
- package/pm2.config.cjs +0 -20
- package/test/fixtures/service_A.js +0 -22
- package/test/fixtures/service_A_terms.md +0 -10
- package/test/fixtures/service_A_terms_snapshot.html +0 -14
- package/test/fixtures/service_B.js +0 -22
- package/test/fixtures/service_with_declaration_history.js +0 -65
- package/test/fixtures/service_with_filters_history.js +0 -155
- package/test/fixtures/service_with_history.js +0 -188
- package/test/fixtures/service_with_multipage_document.js +0 -100
- package/test/fixtures/service_without_history.js +0 -31
- package/test/fixtures/services.js +0 -19
- package/test/fixtures/terms.pdf +0 -0
- package/test/fixtures/termsFromPDF.md +0 -25
- package/test/fixtures/termsModified.pdf +0 -0
- package/test/services/service_A.json +0 -9
- package/test/services/service_B.json +0 -9
- package/test/services/service_with_declaration_history.filters.js +0 -7
- package/test/services/service_with_declaration_history.history.json +0 -17
- package/test/services/service_with_declaration_history.json +0 -13
- package/test/services/service_with_filters_history.filters.history.js +0 -29
- package/test/services/service_with_filters_history.filters.js +0 -7
- package/test/services/service_with_filters_history.json +0 -13
- package/test/services/service_with_history.filters.history.js +0 -29
- package/test/services/service_with_history.filters.js +0 -7
- package/test/services/service_with_history.history.json +0 -26
- package/test/services/service_with_history.json +0 -17
- package/test/services/service_with_multipage_document.filters.js +0 -7
- package/test/services/service_with_multipage_document.history.json +0 -37
- package/test/services/service_with_multipage_document.json +0 -28
- package/test/services/service_without_history.filters.js +0 -7
- package/test/services/service_without_history.json +0 -13
package/ops/README.md
DELETED
|
@@ -1,280 +0,0 @@
|
|
|
1
|
-
# Open Terms Archive Ops
|
|
2
|
-
|
|
3
|
-
Recipes to set up the infrastructure of and deploy Open Terms Archive.
|
|
4
|
-
|
|
5
|
-
## Requirements
|
|
6
|
-
|
|
7
|
-
1. Install [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html).
|
|
8
|
-
2. Install [Vagrant](https://www.vagrantup.com/downloads).
|
|
9
|
-
3. Install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) to manage virtual machines. If you prefer Docker, or have an Apple Silicon machine, install [Docker](https://docs.docker.com/get-docker/) instead.
|
|
10
|
-
4. Create a dedicated SSH key with no password: `ssh-keygen -f ~/.ssh/ota-vagrant -q -N ""`. This key will be automatically used by Vagrant.
|
|
11
|
-
|
|
12
|
-
> VirtualBox is not compatible with Apple Silicon (M1…) processors. If you have such a machine, you will need to use the Docker provider. Since MongoDB cannot be installed on ARM, it is skipped in the infrastructure installation process. This means you cannot test the MongoDB storage repository with Vagrant with an Apple Silicon processor.
|
|
13
|
-
|
|
14
|
-
## Usage
|
|
15
|
-
|
|
16
|
-
**You should never apply changes to production from your machine.** We use continuous deployment to apply changes. To avoid making changes on the production server by mistake, we use [Vagrant](https://www.vagrantup.com) to describe and spawn virtual machines. By default all commands will only affect the Vagrant development virtual machine (VM).
|
|
17
|
-
|
|
18
|
-
### Launch
|
|
19
|
-
|
|
20
|
-
If you’re on an Apple Silicon processor or want to use Docker instead of VirtualBox, use `vagrant up --provider=docker`.
|
|
21
|
-
|
|
22
|
-
In all other cases, use `vagrant up` 🙂
|
|
23
|
-
|
|
24
|
-
You can then deploy the code to the running machine with all the options described below.
|
|
25
|
-
|
|
26
|
-
### Main commands
|
|
27
|
-
|
|
28
|
-
- To set up a full [(phoenix)](https://martinfowler.com/bliki/PhoenixServer.html) server:
|
|
29
|
-
|
|
30
|
-
```
|
|
31
|
-
ansible-playbook ops/site.yml
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
- To setup the infrastructure only:
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
ansible-playbook ops/infra.yml
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
- To setup the `Open Terms Archive` app only:
|
|
41
|
-
|
|
42
|
-
```
|
|
43
|
-
ansible-playbook ops/app.yml
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
### Vagrant quick reference
|
|
47
|
-
|
|
48
|
-
#### Connect to the virtual machine
|
|
49
|
-
|
|
50
|
-
```
|
|
51
|
-
vagrant up
|
|
52
|
-
vagrant ssh # use "vagrant" as password
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
#### Start again with a clean virtual machine
|
|
56
|
-
|
|
57
|
-
```
|
|
58
|
-
vagrant halt # stop machine
|
|
59
|
-
vagrant destroy # remove machine
|
|
60
|
-
vagrant up
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
#### Troubleshooting: Remote host identification has changed
|
|
64
|
-
|
|
65
|
-
In case you get that kind of error:
|
|
66
|
-
|
|
67
|
-
```
|
|
68
|
-
fatal: [127.0.0.1]: UNREACHABLE! => changed=false
|
|
69
|
-
msg: |-
|
|
70
|
-
Failed to connect to the host via ssh: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
|
71
|
-
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
|
|
72
|
-
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
|
73
|
-
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
|
|
74
|
-
…
|
|
75
|
-
unreachable: true
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
It may be because you already have a `known_host` registered with the same IP and port. To solve this, remove it from the entries using `ssh-keygen -R [127.0.0.1]:2222`.
|
|
79
|
-
|
|
80
|
-
#### Troubleshooting: Connection refused
|
|
81
|
-
|
|
82
|
-
If you have the following error:
|
|
83
|
-
|
|
84
|
-
```
|
|
85
|
-
Failed to connect to the host via ssh: ssh: connect to host 127.0.0.1 port 2222: Connection refused
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
You may have a collision on the default port `2222` used by Vagrant to forward SSH commands.
|
|
89
|
-
Run the following command to know which ports are forwarded for the virtual machine:
|
|
90
|
-
|
|
91
|
-
```
|
|
92
|
-
vagrant port
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
It should display something like that:
|
|
96
|
-
|
|
97
|
-
```
|
|
98
|
-
The forwarded ports for the machine are listed below. Please note that
|
|
99
|
-
these values may differ from values configured in the Vagrantfile if the
|
|
100
|
-
provider supports automatic port collision detection and resolution.
|
|
101
|
-
|
|
102
|
-
22 (guest) => 2200 (host)
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
Modify the Ansible SSH options in the `ops/inventories/dev.yml` file with the proper `ansible_ssh_port`:
|
|
106
|
-
|
|
107
|
-
```
|
|
108
|
-
all:
|
|
109
|
-
children:
|
|
110
|
-
vagrant:
|
|
111
|
-
hosts:
|
|
112
|
-
127.0.0.1:
|
|
113
|
-
[…]
|
|
114
|
-
ansible_ssh_port: 2200
|
|
115
|
-
[…]
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
### Logs
|
|
119
|
-
|
|
120
|
-
You can obtain logs from the process manager over SSH:
|
|
121
|
-
|
|
122
|
-
```
|
|
123
|
-
ssh <user>@<instance_hostname> pm2 logs ota
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
### Tags
|
|
127
|
-
|
|
128
|
-
Some tags are available to refine what will happen, use them with `--tags`:
|
|
129
|
-
|
|
130
|
-
- `setup`: to only setup system dependencies required by the app (cloning repo, installing app dependencies, all config files, and so on…)
|
|
131
|
-
- `start`: to start the app
|
|
132
|
-
- `stop`: to stop the app
|
|
133
|
-
- `restart`: to restart the app
|
|
134
|
-
- `update`: to update the app (pull code, install dependencies and restart app)
|
|
135
|
-
- `update-declarations`: to update services declarations (pull declarations, install dependencies and restart app)
|
|
136
|
-
|
|
137
|
-
For example, if you have changes to the core engine to deploy but no infrastructure changes, you can update the app only by running:
|
|
138
|
-
|
|
139
|
-
```
|
|
140
|
-
ansible-playbook ops/app.yml --tags update --limit <instance_name>
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
## Production
|
|
144
|
-
|
|
145
|
-
### Applying changes
|
|
146
|
-
|
|
147
|
-
To test locally your changes to the playbook before opening a pull request:
|
|
148
|
-
|
|
149
|
-
- Remove all traces of previous tests to ensure that your changes do not work by coincidence: `vagrant destroy && vagrant up`.
|
|
150
|
-
- Start by applying your changes on the virtual machine: `ansible-playbook ops/site.yml`.
|
|
151
|
-
- Connect through SSH to the virtual machine and check that everything works as intended: `vagrant ssh`, `pm2 logs`…
|
|
152
|
-
- Open a pull request and wait for it to be reviewed and merged. The continuous deployment process will take care of applying your changes to every production instance.
|
|
153
|
-
|
|
154
|
-
### Deploying manually from your machine
|
|
155
|
-
|
|
156
|
-
**You should not be doing this.** If something terrible is happening in production, did you try just stopping the instance? Any fix should be applied through a PR and deployed in CD to ensure reproducibility.
|
|
157
|
-
|
|
158
|
-
Note that executing the playbook on the `production` inventory will affect **all** production servers. Unless you know exactly what you are doing, you should always execute a playbook on a specific server only, add the `--limit` option with the instance name defined in `ops/inventories/production.yml` as parameter:
|
|
159
|
-
|
|
160
|
-
```
|
|
161
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/site.yml --limit <instance_name>
|
|
162
|
-
```
|
|
163
|
-
|
|
164
|
-
### Allowed keys
|
|
165
|
-
|
|
166
|
-
Setting up the production infrastructure for publishing on the shared versions repository entails decrypting a private key managed with [Ansible Vault](https://docs.ansible.com/ansible/latest/user_guide/vault.html). It is decrypted with a password stored in the passwords database.
|
|
167
|
-
|
|
168
|
-
In case the instance you're deploying on is operated by the Core team, you should use the `OTA-bot` SSH private key instead of your personal one. You can thus run any of the commands with the `--private-key` option, passing it the path to the bot SSH private key. This key can be found in the passwords database.
|
|
169
|
-
|
|
170
|
-
### Commands examples
|
|
171
|
-
|
|
172
|
-
- Check deployment without actually applying changes for the `dating` instance:
|
|
173
|
-
|
|
174
|
-
```
|
|
175
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/app.yml --limit dating --check --diff
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
- Update the Open Terms Archive application only on the `dating` instance, without applying changes to the infrastructure:
|
|
179
|
-
|
|
180
|
-
```
|
|
181
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/app.yml --limit dating --tag update
|
|
182
|
-
```
|
|
183
|
-
|
|
184
|
-
- Update services declarations only on the `france` instance:
|
|
185
|
-
|
|
186
|
-
```
|
|
187
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/app.yml --limit france --tag update-declarations
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
- Stop the Open Terms Archive application only on the `france` instance:
|
|
191
|
-
|
|
192
|
-
```
|
|
193
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/app.yml --limit france --tag stop
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
- Update the infrastructure and the Open Terms Archive application on all servers:
|
|
197
|
-
|
|
198
|
-
```
|
|
199
|
-
ansible-playbook --inventory ops/inventories/production.yml ops/site.yml
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
## Set up a new instance
|
|
203
|
-
|
|
204
|
-
### Provision a server
|
|
205
|
-
|
|
206
|
-
#### With [OVH Horizon](https://horizon.cloud.ovh.net/project/instances/)
|
|
207
|
-
|
|
208
|
-
Click on the `Launch Instance` button. Then fill in at least the following fields:
|
|
209
|
-
|
|
210
|
-
- `Instance name`.
|
|
211
|
-
- `Source`. Suggested: `Debian 11`.
|
|
212
|
-
- `Flavor`. Suggested: `b-7-flex`.
|
|
213
|
-
- `Key pair`. Suggested: Your own personal SSH key, to allow you to connect to the freshly created server.
|
|
214
|
-
|
|
215
|
-
#### Recommended specs
|
|
216
|
-
|
|
217
|
-
The following setup is sufficient to track 20 services:
|
|
218
|
-
|
|
219
|
-
- 1 vCore @ 1.8GHz
|
|
220
|
-
- 2 GB RAM
|
|
221
|
-
- 1 MBps bandwidth
|
|
222
|
-
- 20 GB disk space
|
|
223
|
-
|
|
224
|
-
The major factor for performance is bandwidth.
|
|
225
|
-
|
|
226
|
-
Disk space is used up linearily with time as the archive grows. The number of services, their frequency of change and the chosen storage mechanism will all influence the speed at which disk space is used. You can start with 20GB but will have to consider expansion in the future. You should be safe for a longer time period with 100GB.
|
|
227
|
-
|
|
228
|
-
We suggest using a dedicated attached volume for storage, independently from the main VM drive, so that you can more easily upgrade or format it.
|
|
229
|
-
|
|
230
|
-
### Define host
|
|
231
|
-
|
|
232
|
-
Add an entry to the production inventory file `ops/inventories/production.yml` for the created host with the server address and proper variables.
|
|
233
|
-
|
|
234
|
-
The host name can not contain dashes. Use snake_case.
|
|
235
|
-
|
|
236
|
-
### Configure instance
|
|
237
|
-
|
|
238
|
-
Create a JSON file in the `config` folder with the name of the instance.
|
|
239
|
-
|
|
240
|
-
### Create repositories
|
|
241
|
-
|
|
242
|
-
Create the `snapshot` and `version` repositories, with:
|
|
243
|
-
|
|
244
|
-
- A `main` branch.
|
|
245
|
-
- The `main` branch should be the default branch.
|
|
246
|
-
- At least one commit on this branch with some content (`README.md` and `LICENSE`).
|
|
247
|
-
|
|
248
|
-
Templates are provided to that end, for [declarations](https://github.com/OpenTermsArchive/template-declarations/), [snapshots](https://github.com/OpenTermsArchive/template-snapshots/) and [versions](https://github.com/OpenTermsArchive/template-versions/).
|
|
249
|
-
|
|
250
|
-
### Set up permissions
|
|
251
|
-
|
|
252
|
-
The @OTA-Bot GitHub user should have write access to all three (declarations, snapshots, versions) repositories, so it can publish data, create issues, and publish dataset releases.
|
|
253
|
-
|
|
254
|
-
Each instance should have a responsible entity, which we currently model as a [“team” in the @OpenTermsArchive](https://github.com/orgs/OpenTermsArchive/teams) GitHub organisation. Each team has write access to the three repositories, and @OTA-Bot should be added to that team along with the human maintainers.
|
|
255
|
-
|
|
256
|
-
## Optimise performance
|
|
257
|
-
|
|
258
|
-
### MongoDB
|
|
259
|
-
|
|
260
|
-
If you use MongoDB as storage, hosting the database on an XFS-formatted volume significantly improves performance.
|
|
261
|
-
|
|
262
|
-
The following instructions assume [OVH Horizon](https://horizon.cloud.ovh.net/project/instances/) for volume creation, but can be adapted for any cloud provider.
|
|
263
|
-
|
|
264
|
-
#### Mounting
|
|
265
|
-
|
|
266
|
-
- Create a volume with the highest speed possible.
|
|
267
|
-
- Attach the volume to the server that runs your Open Terms Archive instance.
|
|
268
|
-
- On the machine, check what is your volume with `lsblk` (it should be one with no partition).
|
|
269
|
-
- Then use `sudo fdisk /dev/sd$N` (where `$N` is the identifier of the volume) and answer `n`, `p`, `1`, `w`.
|
|
270
|
-
- Install XFS utilities `sudo apt-get install xfsprogs`
|
|
271
|
-
- Format the disk to XFS: `sudo mkfs.xfs -f /dev/sd$N1`/
|
|
272
|
-
- Finally, create a folder (for example in `/mnt`) and mount the volume in it: `sudo mount -t auto /dev/sd$N1 /mnt/disk`.
|
|
273
|
-
|
|
274
|
-
#### Unmounting
|
|
275
|
-
|
|
276
|
-
To remove a volume:
|
|
277
|
-
|
|
278
|
-
- Unmount it with `sudo umount /mnt/disk`.
|
|
279
|
-
- Unattach it from the Horizon console.
|
|
280
|
-
- Remove the volume from the Horizon console.
|
package/ops/app.yml
DELETED
package/ops/infra.yml
DELETED
package/ops/inventories/dev.yml
DELETED
|
@@ -1,27 +0,0 @@
|
|
|
1
|
-
contrib:
|
|
2
|
-
hosts:
|
|
3
|
-
198.244.153.104:
|
|
4
|
-
ed25519_fingerprint: AAAAC3NzaC1lZDI1NTE5AAAAIITN8hTCst7+6mHNzeo465crCZwHrc/SzUL1410mb9Lv
|
|
5
|
-
dating:
|
|
6
|
-
hosts:
|
|
7
|
-
vps-99ae1d89.vps.ovh.net:
|
|
8
|
-
ed25519_fingerprint: AAAAC3NzaC1lZDI1NTE5AAAAIClFdaZhaXFmxdQI+rNSOsZaSlrgPlK9UzyGvi66u88V
|
|
9
|
-
france:
|
|
10
|
-
hosts:
|
|
11
|
-
198.244.142.9:
|
|
12
|
-
ed25519_fingerprint: AAAAC3NzaC1lZDI1NTE5AAAAIKH7P9SCnnSiVOhGMNvHIjWw5+3TYlmgmTK45Y9d1aCu
|
|
13
|
-
pga:
|
|
14
|
-
hosts:
|
|
15
|
-
134.102.58.70:
|
|
16
|
-
ansible_user: pga
|
|
17
|
-
ed25519_fingerprint: AAAAC3NzaC1lZDI1NTE5AAAAIDmKHW4LMOEIxnBHkdNzwvSrzjmfhQkx5n2lFtJdraOy
|
|
18
|
-
p2b_compliance:
|
|
19
|
-
hosts:
|
|
20
|
-
vps-463f0baf.vps.ovh.net:
|
|
21
|
-
ansible_user: ota
|
|
22
|
-
ed25519_fingerprint: AAAAC3NzaC1lZDI1NTE5AAAAIDOrkEl2aR2gJe0XmLy4j+0/51G/kAlkupfU4S2Qv0dJ
|
|
23
|
-
config_file_name: p2b-compliance
|
|
24
|
-
|
|
25
|
-
all:
|
|
26
|
-
vars:
|
|
27
|
-
ansible_user: debian
|
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
# For documentation of all options, see: http://docs.mongodb.org/manual/reference/configuration-options/
|
|
2
|
-
|
|
3
|
-
storage:
|
|
4
|
-
dbPath: /mnt/disk/mongodb
|
|
5
|
-
journal:
|
|
6
|
-
enabled: true
|
|
7
|
-
|
|
8
|
-
systemLog:
|
|
9
|
-
destination: file
|
|
10
|
-
logAppend: true
|
|
11
|
-
path: /var/log/mongodb/mongod.log
|
|
12
|
-
|
|
13
|
-
net:
|
|
14
|
-
port: 27017
|
|
15
|
-
bindIp: 127.0.0.1
|
|
16
|
-
|
|
17
|
-
processManagement:
|
|
18
|
-
timeZoneInfo: /usr/share/zoneinfo
|
|
@@ -1,26 +0,0 @@
|
|
|
1
|
-
$ANSIBLE_VAULT;1.1;AES256
|
|
2
|
-
62323034623265653135396533383461663035373132353462393965626661663763616338663364
|
|
3
|
-
6137323531393331613165656234623365393531343136310a313439336336663337663838386465
|
|
4
|
-
61376563313330653330313232656262386439666364363436316561323934313064643333323365
|
|
5
|
-
3663616163636133610a363332616537366535343737646332373239663434646339333561653365
|
|
6
|
-
32346630323336373035323435343733383539663661316166383065316334616165353166386532
|
|
7
|
-
38363233646537353566623536353561663839313564363833623835663139616430643637373439
|
|
8
|
-
30663634653466353862326333363663376634343838343262323039363165653336616666613461
|
|
9
|
-
34646430393031376163646334353438383439383164616338373961356361333534383565303164
|
|
10
|
-
36373663623630356338356239376265616632313933353332643036343134366534363066363039
|
|
11
|
-
34323532386631643861616162376438343835656264366334666562303337303333623335343739
|
|
12
|
-
33303138636665363061633733666538326664313561363834646562323066353532316363393132
|
|
13
|
-
36306565363165653637613435323234656361336535343164396339343330666263656636326339
|
|
14
|
-
66346132346363633664643763326638366331623232616430393838653631623233303865316437
|
|
15
|
-
37363039376131316134306264383332663839633130616364636461366131323835323237636261
|
|
16
|
-
61633232646533633131363461383239653732373530333131376265663832636464396134303961
|
|
17
|
-
34336435396131303531616666376666643762386438623732386634316465383938356233646436
|
|
18
|
-
62376532663136323137336365343064383137333739666433323365616638343661346564636362
|
|
19
|
-
37613436353662656137323332343936613134306133653361336135366532626338633363306664
|
|
20
|
-
30366566323965623262383266663331386565326461643566383938333266343366353730373232
|
|
21
|
-
31346434613463653261313430326634323931333062663836333933333635393265616232363031
|
|
22
|
-
36376239326137636430373564633534393830386463376632383731333537366533343030373937
|
|
23
|
-
66396366613331623838316533623163333662313430666562306639353665616430303735356163
|
|
24
|
-
30643738383366616633393438366235666631393033616639613938363939646138323934396631
|
|
25
|
-
36633734313936383566393934316263636363616231343139633164303864646164306337643063
|
|
26
|
-
34343664333031303737646232346234643131306565333265373539616537373864
|
|
@@ -1,78 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
- name: Load app config
|
|
3
|
-
ansible.builtin.include_vars:
|
|
4
|
-
name: app_config
|
|
5
|
-
file: '../config/{{ config_file_name }}.json'
|
|
6
|
-
|
|
7
|
-
- name: Install common required packages
|
|
8
|
-
apt:
|
|
9
|
-
pkg:
|
|
10
|
-
- build-essential
|
|
11
|
-
- curl
|
|
12
|
-
- git
|
|
13
|
-
- zip
|
|
14
|
-
update_cache: yes
|
|
15
|
-
state: latest
|
|
16
|
-
|
|
17
|
-
- name: Add the NodeSource repository to the system
|
|
18
|
-
shell: curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
|
|
19
|
-
|
|
20
|
-
- name: Install NodeJS and NPM
|
|
21
|
-
apt:
|
|
22
|
-
name: nodejs
|
|
23
|
-
update_cache: yes
|
|
24
|
-
state: latest
|
|
25
|
-
|
|
26
|
-
- name: Update NPM to latest version
|
|
27
|
-
command: npm install -g npm
|
|
28
|
-
|
|
29
|
-
- name: Install pm2
|
|
30
|
-
command: npm install -g pm2 --production=true
|
|
31
|
-
|
|
32
|
-
- name: Add global git config
|
|
33
|
-
copy:
|
|
34
|
-
src: .gitconfig
|
|
35
|
-
dest: '/home/{{ ansible_user }}/.gitconfig'
|
|
36
|
-
|
|
37
|
-
- name: Add GitHub bot account SSH key
|
|
38
|
-
copy:
|
|
39
|
-
src: ota-bot-key.private_key
|
|
40
|
-
dest: '/home/{{ ansible_user }}/.ssh/ota-bot-key'
|
|
41
|
-
owner: '{{ ansible_user }}'
|
|
42
|
-
group: '{{ ansible_user }}'
|
|
43
|
-
mode: 0600
|
|
44
|
-
|
|
45
|
-
- name: Configure SSH to use GitHub bot account key on github.com
|
|
46
|
-
template:
|
|
47
|
-
src: ssh_config.j2
|
|
48
|
-
dest: '/home/{{ ansible_user }}/.ssh/config'
|
|
49
|
-
owner: '{{ ansible_user }}'
|
|
50
|
-
group: '{{ ansible_user }}'
|
|
51
|
-
mode: 0644
|
|
52
|
-
|
|
53
|
-
- name: Install Chromium — Debian
|
|
54
|
-
apt:
|
|
55
|
-
pkg:
|
|
56
|
-
- chromium
|
|
57
|
-
update_cache: yes
|
|
58
|
-
state: latest
|
|
59
|
-
when: ansible_distribution == 'Debian'
|
|
60
|
-
|
|
61
|
-
- name: Install Chromium — Ubuntu
|
|
62
|
-
apt:
|
|
63
|
-
pkg:
|
|
64
|
-
- chromium-browser
|
|
65
|
-
update_cache: yes
|
|
66
|
-
state: latest
|
|
67
|
-
when: ansible_distribution == 'Ubuntu'
|
|
68
|
-
|
|
69
|
-
# See https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#recommended-enable-user-namespace-cloning
|
|
70
|
-
- name: Enable user namespace cloning to allow running Chromium in a sandbox
|
|
71
|
-
command: sysctl -w kernel.unprivileged_userns_clone=1
|
|
72
|
-
when: ansible_facts['architecture'] != 'aarch64'
|
|
73
|
-
|
|
74
|
-
- include_tasks: mongo.yml
|
|
75
|
-
when:
|
|
76
|
-
- (app_config.recorder.versions.storage.type is defined and app_config.recorder.versions.storage.type == 'mongo') or (app_config.recorder.snapshots.storage.type is defined and app_config.recorder.snapshots.storage.type == 'mongo')
|
|
77
|
-
# Skip Debian 11 with ARM architecture as it is not currently supported by MongoDB. See https://www.mongodb.com/docs/manual/installation/#supported-platforms
|
|
78
|
-
- ansible_distribution != 'Debian' or (ansible_distribution == 'Debian' and ansible_facts['architecture'] != 'aarch64')
|
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
# See https://docs.mongodb.com/manual/tutorial/install-mongodb-on-debian/
|
|
3
|
-
- name: Import the MongoDB public key used by the package management system
|
|
4
|
-
shell: wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -
|
|
5
|
-
|
|
6
|
-
- name: Create an apt list file for MongoDB — Debian
|
|
7
|
-
shell: echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/5.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list
|
|
8
|
-
when: ansible_distribution == 'Debian'
|
|
9
|
-
|
|
10
|
-
- name: Create an apt list file for MongoDB — Ubuntu
|
|
11
|
-
shell: echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list
|
|
12
|
-
when: ansible_distribution == 'Ubuntu'
|
|
13
|
-
|
|
14
|
-
- name: Install MongoDB
|
|
15
|
-
apt:
|
|
16
|
-
name: mongodb-org
|
|
17
|
-
update_cache: yes
|
|
18
|
-
state: latest
|
|
19
|
-
|
|
20
|
-
- name: Add mongod.conf
|
|
21
|
-
copy:
|
|
22
|
-
src: mongod.conf
|
|
23
|
-
dest: '/etc/mongod.conf'
|
|
24
|
-
|
|
25
|
-
- name: Create data directory
|
|
26
|
-
file:
|
|
27
|
-
path: /mnt/disk/mongodb
|
|
28
|
-
state: directory
|
|
29
|
-
|
|
30
|
-
- name: Set database files permissions
|
|
31
|
-
ansible.builtin.file:
|
|
32
|
-
path: /mnt/disk/mongodb
|
|
33
|
-
owner: mongodb
|
|
34
|
-
group: mongodb
|
|
35
|
-
recurse: yes
|
|
36
|
-
|
|
37
|
-
- name: Start MongoDB service
|
|
38
|
-
service:
|
|
39
|
-
name: mongod
|
|
40
|
-
state: restarted
|
|
@@ -1,14 +0,0 @@
|
|
|
1
|
-
# Try out experimental features by deploying alternative versions of the engine, configuration or databases
|
|
2
|
-
ota_repository: https://github.com/ambanum/OpenTermsArchive.git
|
|
3
|
-
ota_branch: main
|
|
4
|
-
declarations_branch: main
|
|
5
|
-
snapshots_branch: main
|
|
6
|
-
versions_branch: main
|
|
7
|
-
config_file_name: '{{ instance_name }}'
|
|
8
|
-
|
|
9
|
-
# Avoid collisions in case of multi-tenancy (running several instances on the same hosts)
|
|
10
|
-
instance_name: '{{ group_names[0] }}' # assume there is only one named group per host; override if a single server runs several instances
|
|
11
|
-
ota_directory: ota
|
|
12
|
-
declarations_directory: declarations
|
|
13
|
-
snapshots_directory: snapshots
|
|
14
|
-
versions_directory: versions
|
package/ops/roles/ota/files/.env
DELETED
|
@@ -1,21 +0,0 @@
|
|
|
1
|
-
$ANSIBLE_VAULT;1.1;AES256
|
|
2
|
-
65653333633465346264373663643062363265383439323061633338323166393262663332646534
|
|
3
|
-
3034646331313536623831616466653564396663636166650a643831383438613765323264386266
|
|
4
|
-
37383663326333636366653537383339636231313261323239663164393933366535623730313337
|
|
5
|
-
3230323166306336630a646239643135326335393364366534663535316334333531353431323331
|
|
6
|
-
62376464333830643539333764646534613935386263633634623962653537363139363265353864
|
|
7
|
-
34623031333535313930633964626438343432363366633263396466323830323035343333373763
|
|
8
|
-
31303962353238383666343834653539663536623561393966373763393535663038333833396433
|
|
9
|
-
61326137363133373861343732336335373037613134336362396563653563646364376438366363
|
|
10
|
-
61613437393364396332313833356138303835633739653937656631313737623762346235316164
|
|
11
|
-
30353961393663663635393630653535353064353031623962333038366638376333646130663263
|
|
12
|
-
35346637303534663639613537316538663134356166323566613333343236383631653563643433
|
|
13
|
-
65373533656236613631653665383039303564613830343530363130663462343665343865666637
|
|
14
|
-
61353766623634303537326163623564663433343835366463613837303361393538656365633365
|
|
15
|
-
64306532626533373561633066616336663439653734313832366333323461383065326166313635
|
|
16
|
-
38343261656638396235326266376531633662343235633336363361313266643166346433356539
|
|
17
|
-
38363833336562623631306432656531326465306535623164353035366339336663626161613364
|
|
18
|
-
39616539616532643935623939313834343964303035383334666261363336303034346439316663
|
|
19
|
-
65306530333965353766363336386564383932333234303334633438333839316362666533363330
|
|
20
|
-
34336232623333353366363232656461646236383061383634653137623661376639323162383162
|
|
21
|
-
38653230353662316366
|
|
@@ -1,65 +0,0 @@
|
|
|
1
|
-
- name: Check if {{ name }} base data has already been obtained from {{ repository }}
|
|
2
|
-
git:
|
|
3
|
-
repo: '{{ repository }}'
|
|
4
|
-
version: '{{ branch }}'
|
|
5
|
-
dest: '/home/{{ ansible_user }}/{{ directory }}'
|
|
6
|
-
clone: no
|
|
7
|
-
update: no
|
|
8
|
-
accept_hostkey: yes
|
|
9
|
-
key_file: '/home/{{ ansible_user }}/.ssh/ota-bot-key'
|
|
10
|
-
register: existing_repository # the `before` property of the return value can tell us if the repository has been cloned already or not, see <https://docs.ansible.com/ansible/latest/collections/ansible/builtin/git_module.html#return-values>
|
|
11
|
-
tags:
|
|
12
|
-
- restart
|
|
13
|
-
- start
|
|
14
|
-
- update
|
|
15
|
-
- setup
|
|
16
|
-
|
|
17
|
-
- name: Obtain {{ name }} initial data from branch {{ branch }} of {{ repository }}
|
|
18
|
-
git:
|
|
19
|
-
repo: '{{ repository }}'
|
|
20
|
-
version: '{{ branch }}'
|
|
21
|
-
dest: '/home/{{ ansible_user }}/{{ directory }}'
|
|
22
|
-
accept_hostkey: yes
|
|
23
|
-
key_file: '/home/{{ ansible_user }}/.ssh/ota-bot-key'
|
|
24
|
-
when: existing_repository.before is defined and not existing_repository.before # if existing_repository.before is null, then the repository is new
|
|
25
|
-
tags:
|
|
26
|
-
- setup
|
|
27
|
-
|
|
28
|
-
- name: Remove existing locks in {{ name }}
|
|
29
|
-
file:
|
|
30
|
-
path: '/home/{{ ansible_user }}/{{ directory }}/.git/index.lock'
|
|
31
|
-
state: absent
|
|
32
|
-
tags:
|
|
33
|
-
- restart
|
|
34
|
-
- start
|
|
35
|
-
- update
|
|
36
|
-
|
|
37
|
-
- name: Get latest data from {{ repository }}
|
|
38
|
-
command:
|
|
39
|
-
cmd: git fetch origin
|
|
40
|
-
chdir: '/home/{{ ansible_user }}/{{ directory }}'
|
|
41
|
-
tags:
|
|
42
|
-
- restart
|
|
43
|
-
- start
|
|
44
|
-
- update
|
|
45
|
-
- setup
|
|
46
|
-
|
|
47
|
-
- name: Clean {{ name }} local copy
|
|
48
|
-
command:
|
|
49
|
-
cmd: git reset --hard origin/{{ branch }}
|
|
50
|
-
chdir: '/home/{{ ansible_user }}/{{ directory }}'
|
|
51
|
-
tags:
|
|
52
|
-
- restart
|
|
53
|
-
- start
|
|
54
|
-
- update
|
|
55
|
-
- setup
|
|
56
|
-
|
|
57
|
-
- name: Ensure {{ name }} is on branch {{ branch }}
|
|
58
|
-
command:
|
|
59
|
-
cmd: git checkout {{ branch }}
|
|
60
|
-
chdir: '/home/{{ ansible_user }}/{{ directory }}'
|
|
61
|
-
tags:
|
|
62
|
-
- restart
|
|
63
|
-
- start
|
|
64
|
-
- update
|
|
65
|
-
- setup
|