@opentermsarchive/engine 0.16.0 → 0.17.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (82) hide show
  1. package/README.md +240 -232
  2. package/package.json +7 -1
  3. package/scripts/dataset/README.md +2 -2
  4. package/scripts/dataset/assets/README.template.js +5 -5
  5. package/scripts/dataset/export/test/fixtures/dataset/README.md +5 -5
  6. package/scripts/import/README.md +1 -1
  7. package/scripts/rewrite/README.md +2 -2
  8. package/scripts/rewrite/rewrite-versions.js +1 -1
  9. package/scripts/utils/renamer/README.md +5 -5
  10. package/scripts/utils/renamer/index.js +2 -2
  11. package/src/archivist/recorder/index.js +2 -2
  12. package/src/archivist/recorder/index.test.js +3 -3
  13. package/src/archivist/recorder/repositories/git/dataMapper.js +1 -1
  14. package/src/archivist/recorder/repositories/git/index.test.js +5 -5
  15. package/src/archivist/recorder/repositories/interface.js +2 -2
  16. package/src/archivist/recorder/repositories/mongo/index.test.js +4 -4
  17. package/src/archivist/services/index.test.js +2 -2
  18. package/src/archivist/services/service.test.js +1 -1
  19. package/src/main.js +1 -1
  20. package/.env.example +0 -3
  21. package/.eslintrc.yaml +0 -116
  22. package/.github/workflows/deploy.yml +0 -50
  23. package/.github/workflows/release.yml +0 -71
  24. package/.github/workflows/test.yml +0 -77
  25. package/CHANGELOG.md +0 -14
  26. package/CODE_OF_CONDUCT.md +0 -128
  27. package/CONTRIBUTING.md +0 -143
  28. package/MIGRATING.md +0 -42
  29. package/README.fr.md +0 -110
  30. package/Vagrantfile +0 -38
  31. package/ansible.cfg +0 -13
  32. package/decision-records/0001-service-name-and-id.md +0 -73
  33. package/decision-records/0002-service-history.md +0 -212
  34. package/decision-records/0003-snapshots-database.md +0 -123
  35. package/ops/README.md +0 -280
  36. package/ops/app.yml +0 -5
  37. package/ops/infra.yml +0 -6
  38. package/ops/inventories/dev.yml +0 -7
  39. package/ops/inventories/production.yml +0 -27
  40. package/ops/roles/infra/defaults/main.yml +0 -2
  41. package/ops/roles/infra/files/.gitconfig +0 -3
  42. package/ops/roles/infra/files/mongod.conf +0 -18
  43. package/ops/roles/infra/files/ota-bot-key.private_key +0 -26
  44. package/ops/roles/infra/tasks/main.yml +0 -78
  45. package/ops/roles/infra/tasks/mongo.yml +0 -40
  46. package/ops/roles/infra/templates/ssh_config.j2 +0 -5
  47. package/ops/roles/ota/defaults/main.yml +0 -14
  48. package/ops/roles/ota/files/.env +0 -21
  49. package/ops/roles/ota/tasks/database.yml +0 -65
  50. package/ops/roles/ota/tasks/main.yml +0 -110
  51. package/ops/site.yml +0 -6
  52. package/pm2.config.cjs +0 -20
  53. package/test/fixtures/service_A.js +0 -22
  54. package/test/fixtures/service_A_terms.md +0 -10
  55. package/test/fixtures/service_A_terms_snapshot.html +0 -14
  56. package/test/fixtures/service_B.js +0 -22
  57. package/test/fixtures/service_with_declaration_history.js +0 -65
  58. package/test/fixtures/service_with_filters_history.js +0 -155
  59. package/test/fixtures/service_with_history.js +0 -188
  60. package/test/fixtures/service_with_multipage_document.js +0 -100
  61. package/test/fixtures/service_without_history.js +0 -31
  62. package/test/fixtures/services.js +0 -19
  63. package/test/fixtures/terms.pdf +0 -0
  64. package/test/fixtures/termsFromPDF.md +0 -25
  65. package/test/fixtures/termsModified.pdf +0 -0
  66. package/test/services/service_A.json +0 -9
  67. package/test/services/service_B.json +0 -9
  68. package/test/services/service_with_declaration_history.filters.js +0 -7
  69. package/test/services/service_with_declaration_history.history.json +0 -17
  70. package/test/services/service_with_declaration_history.json +0 -13
  71. package/test/services/service_with_filters_history.filters.history.js +0 -29
  72. package/test/services/service_with_filters_history.filters.js +0 -7
  73. package/test/services/service_with_filters_history.json +0 -13
  74. package/test/services/service_with_history.filters.history.js +0 -29
  75. package/test/services/service_with_history.filters.js +0 -7
  76. package/test/services/service_with_history.history.json +0 -26
  77. package/test/services/service_with_history.json +0 -17
  78. package/test/services/service_with_multipage_document.filters.js +0 -7
  79. package/test/services/service_with_multipage_document.history.json +0 -37
  80. package/test/services/service_with_multipage_document.json +0 -28
  81. package/test/services/service_without_history.filters.js +0 -7
  82. package/test/services/service_without_history.json +0 -13
@@ -1,77 +0,0 @@
1
- name: Test
2
-
3
- on:
4
- push:
5
- branches-ignore:
6
- - main # tests will be launched by workflow_call from the deploy workflow
7
- pull_request:
8
- types: [ opened, reopened ]
9
- workflow_call:
10
- outputs:
11
- release_type:
12
- description: "The release type extracted from changelog"
13
- value: ${{ jobs.check_changelog.outputs.release_type }}
14
-
15
- jobs:
16
- check_changelog:
17
- runs-on: [ubuntu-latest]
18
- outputs:
19
- release_type: ${{ steps.set_release_type_output.outputs.release_type }}
20
- steps:
21
- - uses: actions/checkout@v2
22
- - name: Get release type in changelog
23
- run: echo "RELEASE_TYPE=$(cat CHANGELOG.md | grep -E '^## Unreleased \[(patch|minor|major)\]$' | grep -E -w -o "patch|minor|major" | tr -d '\n')" >> $GITHUB_ENV
24
- - name: Make release type available to subsequent jobs
25
- if: env.RELEASE_TYPE
26
- id: set_release_type_output
27
- run: |
28
- echo "Found release type '${{ env.RELEASE_TYPE }}'"
29
- echo "release_type=${{ env.RELEASE_TYPE }}" >> $GITHUB_OUTPUT
30
- - name: Fail and display error if no proper release type is found in changelog
31
- if: env.RELEASE_TYPE == ''
32
- run: echo "No valid release type found in changelog. The title of the 'Unreleased' section must contain one of the following tags '[patch]', '[minor]', '[major]'. For example, '## Unreleased [minor]'."; exit 1
33
-
34
-
35
- test:
36
- strategy:
37
- matrix:
38
- operating_system: [ ubuntu-20.04, windows-latest, macos-latest ]
39
- fail-fast: false # run tests on other operating systems even if one fails
40
-
41
- runs-on: ${{ matrix.operating_system }}
42
-
43
- steps:
44
- - run: |
45
- git config --global core.autocrlf false
46
- - uses: actions/checkout@v2
47
- - uses: ankane/setup-mongodb@ce30d9041565cb469945895d5bde3384a254dd2e # use commit ID until action is versioned, see https://github.com/ankane/setup-mongodb/issues/2
48
- with:
49
- mongodb-version: 5.0
50
- - uses: actions/setup-node@v2
51
- with:
52
- node-version: 16
53
- - run: npm ci
54
- - run: npm test
55
-
56
- validate_declarations:
57
- strategy:
58
- matrix:
59
- operating_system: [ ubuntu-latest, windows-latest, macos-latest ]
60
- fail-fast: false # run tests on other operating systems even if one fails
61
-
62
- runs-on: ${{ matrix.operating_system }}
63
-
64
- steps:
65
- - run: |
66
- git config --global core.autocrlf false
67
- - uses: actions/checkout@v2
68
- - uses: actions/checkout@v2
69
- with:
70
- repository: OpenTermsArchive/contrib-declarations
71
- path: ./contrib-declarations
72
- - uses: actions/setup-node@v2
73
- with:
74
- node-version: 16.x
75
- - run: cd ./contrib-declarations && npm install
76
- - run: npm ci
77
- - run: ./node_modules/.bin/cross-env NODE_ENV=ci npm run declarations:validate:schema
package/CHANGELOG.md DELETED
@@ -1,14 +0,0 @@
1
- # Changelog
2
-
3
- All notable changes to this project will be documented in this file.
4
-
5
- This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and the format is based on [Common Changelog](https://common-changelog.org).\
6
- Unlike Common Changelog, the `unreleased` section of [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) is preserved with the addition of a tag to specify which type of release should be published and to foster discussions about it inside pull requests. This tag should be one of the names mandated by SemVer, within brackets: `[patch]`, `[minor]` or `[major]`. For example: `## Unreleased [minor]`.
7
-
8
- ## Unreleased
9
-
10
- ## 0.16.0 - 2022-12-05
11
- ### Added
12
- - Publish package on NPM under name `@opentermsarchive/engine`.
13
- - Export `filter`, `pageDeclaration` and `fetch` in NPM module.
14
- - Add changelog.
@@ -1,128 +0,0 @@
1
- # Contributor Covenant Code of Conduct
2
-
3
- ## Our Pledge
4
-
5
- We as members, contributors, and leaders pledge to make participation in our
6
- community a harassment-free experience for everyone, regardless of age, body
7
- size, visible or invisible disability, ethnicity, sex characteristics, gender
8
- identity and expression, level of experience, education, socio-economic status,
9
- nationality, personal appearance, race, religion, or sexual identity
10
- and orientation.
11
-
12
- We pledge to act and interact in ways that contribute to an open, welcoming,
13
- diverse, inclusive, and healthy community.
14
-
15
- ## Our Standards
16
-
17
- Examples of behavior that contributes to a positive environment for our
18
- community include:
19
-
20
- * Demonstrating empathy and kindness toward other people
21
- * Being respectful of differing opinions, viewpoints, and experiences
22
- * Giving and gracefully accepting constructive feedback
23
- * Accepting responsibility and apologizing to those affected by our mistakes,
24
- and learning from the experience
25
- * Focusing on what is best not just for us as individuals, but for the
26
- overall community
27
-
28
- Examples of unacceptable behavior include:
29
-
30
- * The use of sexualized language or imagery, and sexual attention or
31
- advances of any kind
32
- * Trolling, insulting or derogatory comments, and personal or political attacks
33
- * Public or private harassment
34
- * Publishing others' private information, such as a physical or email
35
- address, without their explicit permission
36
- * Other conduct which could reasonably be considered inappropriate in a
37
- professional setting
38
-
39
- ## Enforcement Responsibilities
40
-
41
- Community leaders are responsible for clarifying and enforcing our standards of
42
- acceptable behavior and will take appropriate and fair corrective action in
43
- response to any behavior that they deem inappropriate, threatening, offensive,
44
- or harmful.
45
-
46
- Community leaders have the right and responsibility to remove, edit, or reject
47
- comments, commits, code, wiki edits, issues, and other contributions that are
48
- not aligned to this Code of Conduct, and will communicate reasons for moderation
49
- decisions when appropriate.
50
-
51
- ## Scope
52
-
53
- This Code of Conduct applies within all community spaces, and also applies when
54
- an individual is officially representing the community in public spaces.
55
- Examples of representing our community include using an official e-mail address,
56
- posting via an official social media account, or acting as an appointed
57
- representative at an online or offline event.
58
-
59
- ## Enforcement
60
-
61
- Instances of abusive, harassing, or otherwise unacceptable behavior may be
62
- reported to the community leaders responsible for enforcement at
63
- moderators@opentermsarchive.org.
64
- All complaints will be reviewed and investigated promptly and fairly.
65
-
66
- All community leaders are obligated to respect the privacy and security of the
67
- reporter of any incident.
68
-
69
- ## Enforcement Guidelines
70
-
71
- Community leaders will follow these Community Impact Guidelines in determining
72
- the consequences for any action they deem in violation of this Code of Conduct:
73
-
74
- ### 1. Correction
75
-
76
- **Community Impact**: Use of inappropriate language or other behavior deemed
77
- unprofessional or unwelcome in the community.
78
-
79
- **Consequence**: A private, written warning from community leaders, providing
80
- clarity around the nature of the violation and an explanation of why the
81
- behavior was inappropriate. A public apology may be requested.
82
-
83
- ### 2. Warning
84
-
85
- **Community Impact**: A violation through a single incident or series
86
- of actions.
87
-
88
- **Consequence**: A warning with consequences for continued behavior. No
89
- interaction with the people involved, including unsolicited interaction with
90
- those enforcing the Code of Conduct, for a specified period of time. This
91
- includes avoiding interactions in community spaces as well as external channels
92
- like social media. Violating these terms may lead to a temporary or
93
- permanent ban.
94
-
95
- ### 3. Temporary Ban
96
-
97
- **Community Impact**: A serious violation of community standards, including
98
- sustained inappropriate behavior.
99
-
100
- **Consequence**: A temporary ban from any sort of interaction or public
101
- communication with the community for a specified period of time. No public or
102
- private interaction with the people involved, including unsolicited interaction
103
- with those enforcing the Code of Conduct, is allowed during this period.
104
- Violating these terms may lead to a permanent ban.
105
-
106
- ### 4. Permanent Ban
107
-
108
- **Community Impact**: Demonstrating a pattern of violation of community
109
- standards, including sustained inappropriate behavior, harassment of an
110
- individual, or aggression toward or disparagement of classes of individuals.
111
-
112
- **Consequence**: A permanent ban from any sort of public interaction within
113
- the community.
114
-
115
- ## Attribution
116
-
117
- This Code of Conduct is adapted from the [Contributor Covenant][homepage],
118
- version 2.0, available at
119
- https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
120
-
121
- Community Impact Guidelines were inspired by [Mozilla's code of conduct
122
- enforcement ladder](https://github.com/mozilla/diversity).
123
-
124
- [homepage]: https://www.contributor-covenant.org
125
-
126
- For answers to common questions about this code of conduct, see the FAQ at
127
- https://www.contributor-covenant.org/faq. Translations are available at
128
- https://www.contributor-covenant.org/translations.
package/CONTRIBUTING.md DELETED
@@ -1,143 +0,0 @@
1
- First of all, thanks for taking the time to contribute! 🎉👍
2
-
3
- ## Table of Contents
4
-
5
- - [Workflow](#workflow)
6
- - [Pull requests](#pull-requests)
7
- - [Commit messages](#commit-messages)
8
- - [Continuous delivery](#continuous-delivery)
9
- - [Changelog](#changelog)
10
- - [Development](#development)
11
- - [Documentation](#documentation)
12
- - [Naming](#naming)
13
- - [Instances and repositories](#instances-and-repositories)
14
- - [Namespaces](#namespaces)
15
- - [Practices](#practices)
16
- - [Errors handling](#errors-handling)
17
-
18
- ---
19
-
20
- ## Workflow
21
-
22
- ### Pull requests
23
-
24
- We follow the [GitHub Flow](https://guides.github.com/introduction/flow/): all code contributions are submitted via a pull request towards the `main` branch.
25
-
26
- Opening a pull request means you want that code to be merged. If you want to only discuss it, send a link to your branch along with your questions through whichever communication channel you prefer.
27
-
28
- #### Peer reviews
29
-
30
- All pull requests must be reviewed by at least one person who is not their original author.
31
-
32
- To help reviewers, make sure to describe your pull request with a **clear text explanation** of your changes.
33
-
34
- ### Continuous delivery
35
-
36
- GitHub Actions is used to deploy the application on every merge to the main branch.
37
-
38
- Branch protection is active, meaning that a merge to the main branch can only take place once all tests pass in CI, and that the peer review policy has been fulfilled.
39
-
40
- ### Commit messages
41
-
42
- We strive to follow this [recommendation](https://chris.beams.io/posts/git-commit) to write our commit messages, which contains the following rules:
43
-
44
- - [Separate subject from body with a blank line](https://chris.beams.io/posts/git-commit/#separate).
45
- - [Limit the subject line to 50 characters](https://chris.beams.io/posts/git-commit/#limit-50).
46
- - [Capitalize the subject line](https://chris.beams.io/posts/git-commit/#capitalize).
47
- - [Do not end the subject line with a period](https://chris.beams.io/posts/git-commit/#end).
48
- - [Use the imperative mood in the subject line](https://chris.beams.io/posts/git-commit/#imperative).
49
- - [Wrap the body at 72 characters](https://chris.beams.io/posts/git-commit/#wrap-72).
50
- - [Use the body to explain what and why vs. how](https://chris.beams.io/posts/git-commit/#why-not-how).
51
-
52
- We add this additional rule:
53
-
54
- - Do not rely on GitHub issue reference numbers in commit messages, as we have no guarantee the host system and its autolinking will be stable in time. Make sure the context is self-explanatory. If an external reference is given, use its full URL.
55
-
56
- ### Changelog
57
-
58
- All changes to the codebase that impact users must be documented in the [`CHANGELOG.md`](./CHANGELOG.md) file. The format to use is documented in the file itself.
59
-
60
- ## Development
61
-
62
- ### Documentation
63
-
64
- #### CLI
65
-
66
- ##### docopt
67
-
68
- For command-line examples and documentation, we follow the [docopt usage patterns](http://docopt.org) syntax. Quick recap of the main points:
69
-
70
- - mandatory arguments are given between `<` and `>`;
71
- - optional elements are given between `[` and `]`;
72
- - mutually exclusive elements are given between `(` and `)` and separated by `|`.
73
-
74
- ```diff
75
- - npm start [ $service_id ] [, $service_id, ...]
76
- + npm start [service_id]...
77
- ```
78
-
79
- ##### Long options
80
-
81
- In order to improve the understandability of commands, we document all CLI options and examples with the long version of the options.
82
-
83
-
84
- ```diff
85
- - ansible-playbook -i production.yml app.yml -l france -t stop
86
- + ansible-playbook --inventory production.yml app.yml --limit france --tag stop
87
- ```
88
-
89
- ## Naming
90
-
91
- ### Instances and repositories
92
-
93
- An “instance” of Open Terms Archive is comprised of a server running Open Terms Archive and up to three repositories. An instance has a _name_ describing the scope of services it aims at tracking. This scope is defined by one or several dimensions: jurisdiction, language, industry…
94
-
95
- > For example, the `france` instance tracks documents in the French jurisdiction and French language, while the `dating` instance tracks services from the dating industry.
96
-
97
- The instance name is written in lowercase and is made of one word for each dimension it focuses on, separated by dashes.
98
-
99
- > For example, the `france-elections` instance tracks services in the French jurisdiction and French language that could impact the French electoral processes.
100
-
101
- This name is used consistently in all communication, written references, and in the inventory of instances that are managed automatically. It is also used as the base for naming the database repositories, by suffixing it with each type:
102
-
103
- - The repository containing the declarations of services to be tracked is named `$instance_name-declarations`. You can [create it from a template](https://github.com/OpenTermsArchive/template-declarations/generate).
104
- - The repository containing the snapshots of the tracked documents (unless the instance is storing them in an alternative database) is named `$instance_name-snapshots`. You can [create it from a template](https://github.com/OpenTermsArchive/template-snapshots/generate).
105
- - The repository containing the versions of the tracked documents (unless the instance is storing them in an alternative database) is named `$instance_name-versions`. You can [create it from a template](https://github.com/OpenTermsArchive/template-versions/generate).
106
-
107
- ### Namespaces
108
-
109
- We deploy identifiers for packages and namespaces across different universes: package managers, social networks, URLs… In order to unify these names across constraints, we reserve everywhere the name `opentermsarchive`, with no space, no dash, no underscore, no capital.
110
-
111
- > For example, NPM does not allow uppercase and spaces; Ansible does not allow dashes and spaces; Twitter does not allow spaces. The name `ota` is too unlikely to be available everywhere.
112
-
113
- ## Practices
114
-
115
- ### Errors handling
116
-
117
- First of all it's important to distinguish two fundamentally different kinds of errors: **operational** and **programmer** errors.
118
-
119
- - **Operational errors represent run-time problems experienced by correctly-written programs**. These are not bugs in the program. These are usually problems with something else: the system itself (e.g. out of memory), the system’s configuration (e.g. no route to a remote host), the network (e.g. socket hang-up), or a remote service (e.g. a 500 error).
120
-
121
- - **Programmer errors are bugs in the program**. These are things that can always be avoided by changing the code. They can never be handled properly, since by definition the code in question is broken (e.g. tried to read property of `undefined`, or forget to `await` an asynchronous function).
122
-
123
- So the very important distinction is that operational errors are part of the **normal operation of a program** whereas programmer errors are **bugs**.
124
-
125
- Also noteworthy, failure to handle an operational error is itself a programmer error.
126
-
127
- #### Handling operational errors
128
-
129
- There are five ways to handle operational errors:
130
- - **Deal with the failure directly**. For example, create directory if it's missing.
131
- - **Propagate the failure**. If you don’t know how to deal with the error, the simplest thing to do is to abort whatever operation you’re trying to do, clean up whatever you’ve started, and propagate the error.
132
- - **Retry the operation**. For example, try to reconnect if the connection is lost.
133
- - **Log the error — and do nothing else**. If it's a minor error and there’s nothing you can do about, and there is no reason to stop the whole process.
134
- - **Crash immediately**. If the error cannot be handled and can affect data integrity.
135
-
136
- In our case, we consider all `fetch`-related errors as expected, so as operational errors and we handle them by logging but we do not stop the whole process. We handle errors related to the `notifier` in the same way.
137
- In contrast, we consider errors from the `recorder` module as fatal, and we crash immediately.
138
-
139
- #### Handling programmer errors
140
-
141
- **The best way to handle programmer errors is to crash immediately.** Indeed, it is not recommended to attempt to recover from programmer errors — that is, allow the current operation to fail, but keep handling requests. Consider that a programmer error is a case that you didn’t think about when you wrote the original code. How can you be sure that the problem won’t affect the program itself and the data integrity?
142
-
143
- This section is highly inspired, and in part extracted, from [this error handling guide](https://console.joyent.com/node-js/production/design/errors).
package/MIGRATING.md DELETED
@@ -1,42 +0,0 @@
1
- ## 2022-03-29 Unify repository names
2
-
3
- All services declarations, snapshots and versions repo has been renamed and moved to the [OpenTermArchive organization](https://github.com/OpenTermsArchive) on github.
4
-
5
- Documentation on the new naming system can be found [here](https://github.com/ambanum/OpenTermsArchive/blob/main/CONTRIBUTING.md#instances-and-repositories-names)
6
-
7
- If you already have some of those projects locally, you can run
8
-
9
- ```
10
- mv services-all contrib-declarations
11
- mv services-dating dating-declarations
12
- mv declarations-france-elections france-elections-declarations
13
- mv declarations-france france-elections
14
- mv declarations-france-elections-experiment france-elections-experiment-declarations
15
- ```
16
-
17
- ## 2021-11-25 Externalize services declarations
18
-
19
- Services declarations are now declared in their [own repository](https://github.com/OpenTermsArchive/services-all).
20
-
21
- In order to use all the defined services locally, you now have to clone that new repository in the parent folder of the OpenTermsArchive one:
22
-
23
- ```
24
- # From your local clone of the OpenTermsArchive repository
25
- cd ..
26
- git clone git@github.com:OpenTermsArchive/contrib-declarations.git
27
- ```
28
-
29
- ## 2021-02-24 Name change
30
-
31
- `CGUs` is now called `OpenTermsArchive`.
32
-
33
- To reflect this change, if you already have this project locally, execute:
34
-
35
- ```
36
- # From your local clone of the OpenTermsArchive repository
37
- cd ..
38
- mv CGUs OpenTermsArchive
39
- cd OpenTermsArchive
40
- git remote set-url origin https://github.com/ambanum/OpenTermsArchive.git
41
- ```
42
-
package/README.fr.md DELETED
@@ -1,110 +0,0 @@
1
- <img src="https://disinfo.quaidorsay.fr/assets/img/logo.png" width="140">
2
-
3
- # Open Terms Archive
4
-
5
- Les services en ligne ont des conditions générales qui évoluent dans le temps. _Open Terms Archive_ permet aux défenseurs des droits des utilisateurs, aux régulateurs et à toute personne intéressée de suivre les évolutions de ces conditions générales en étant notifiée à chaque publication d'une nouvelle version, et en explorant leur historique.
6
-
7
- ## Table des matières
8
-
9
- - [Fonctionnement](#fonctionnement)
10
- - [Naviguer dans l'historique des versions](#naviguer-dans-lhistorique-des-versions)
11
- - [Remarques](#remarques)
12
- - [Recevoir des notifications](#recevoir-des-notifications)
13
- - [Par courriel](#par-courriel)
14
- - [Recevoir les mises à jour de services ou documents spécifiques](#recevoir-les-mises-%C3%A0-jour-de-services-ou-documents-sp%C3%A9cifiques)
15
- - [Par RSS](#par-rss)
16
- - [Récapitulatif des flux RSS disponibles](#r%C3%A9capitulatif-des-flux-rss-disponibles)
17
- - [Désabonnement](#désabonnement)
18
- - [Contribuer](#contribuer)
19
- - [Ajouter un nouveau service](#ajouter-un-nouveau-service)
20
-
21
- ## Fonctionnement
22
-
23
- _Note: Les mots en gras sont les [termes du domaine](https://fr.wikipedia.org/wiki/Conception_pilot%C3%A9e_par_le_domaine)._
24
-
25
- Les **services** sont **déclarés** dans l'outil _Open Terms Archive_ grâce à un **fichier de déclaration** listant les **documents** qui forment l'ensemble des **conditions** régissant l'usage du **service**. Ces **documents** peuvent être de plusieurs **types** : « conditions d'utilisation », « politique de confidentialité », « contrat de développeur »…
26
-
27
- Afin de **suivre** leurs **évolutions**, les **documents** sont régulièrement mis à jour, en les **téléchargeant** depuis une **adresse** web et en **sélectionnant leur contenu** dans la **page web** pour supprimer le **bruit** (publicités, menus de navigation, champs de connexion…). En plus de simplement sélectionner une zone de la page, certains documents possèdent du **bruit** supplémentaire (hashs dans des liens, jetons CSRF...) créant de faux positifs en terme d'**évolutions**. En conséquence, _Open Terms Archive_ supporte des **filtres** spécifiques pour chaque **document**.
28
-
29
- Néanmoins, le **bruit** peut changer de forme avec le temps. Afin d'éviter des pertes d'information irrécupérables pendant l'étape de **filtrage du bruit**, un **instantané** de la page Web est **enregistré** à chaque **évolution**. Après avoir **filtré l'instantané** de son **bruit**, si le **document** résultant a changé par rapport à sa **version** précédente, une nouvelle **version** est **enregistrée**.
30
-
31
- Vous pouvez disposer de votre propre instance **privée** de l'outil _Open Terms Archive_ et suivre vous-même les **évolutions**. Néanmoins, nous **publions** chaque **version** sur une [instance **publique**](https://github.com/OpenTermsArchive/contrib-versions) facilitant l'exploration de l'**historique** et **notifiant** par courriels l'**enregistrement** de nouvelles **versions**. Les **utilisateurs** peuvent [**s'abonner** aux **notifications**](#recevoir-des-notifications).
32
-
33
- _Note: Actuellement, nous ne suivons que les **conditions** rédigées en anglais et concernant la juridiction européenne._
34
-
35
- ## Naviguer dans l'historique des versions
36
-
37
- À partir de la **page d'accueil du dépôt** [contrib-versions](https://github.com/OpenTermsArchive/contrib-versions), ouvrez le dossier du **service de votre choix** (prenons par exemple [WhatsApp](https://github.com/OpenTermsArchive/contrib-versions/tree/main/WhatsApp)).
38
-
39
- L'**ensemble des documents suivis** pour ce service s'affichent, cliquez ensuite sur **celui dont vous souhaitez suivre l'historique** (par exemple la [politique d'utilisation des données de WhatsApp](https://github.com/OpenTermsArchive/contrib-versions/blob/main/WhatsApp/Privacy%20Policy.md)). Le document s'affiche alors dans sa **dernière version** (il est actualisé toutes les heures).
40
-
41
- Pour afficher l'**historique des modifications** subies par ce document, cliquez sur **History** en haut à droite du document (pour l'exemple précédent nous arrivons [ici](https://github.com/OpenTermsArchive/contrib-versions/commits/main/WhatsApp/Privacy%20Policy.md)). Les **modifications** sont affichées **par dates**, de la plus récente à la plus ancienne.
42
-
43
- Cliquez sur une modification pour voir en quoi elle consiste (par exemple [celle-ci](https://github.com/OpenTermsArchive/contrib-versions/commit/58a1d2ae4187a3260ac58f3f3c7dcd3aeacaebcd)). Vous disposez de **deux types d'affichage**, sélectionnables à partir des icônes dans la barre grisée qui chapeaute le document.
44
-
45
- - Le premier, appelé _source diff_ (bouton avec des chevrons) permet d'**afficher côte-à-côte l'ancienne version et la nouvelle** (pour notre [exemple](https://github.com/OpenTermsArchive/contrib-versions/commit/58a1d2ae4187a3260ac58f3f3c7dcd3aeacaebcd#diff-e8bdae8692561f60aeac9d27a55e84fc)). Cet affichage a le mérite de **montrer explicitement** l'ensemble des ajouts/suppressions.
46
- - Le second, appelé _rich diff_ (bouton avec l'icône document) permet d'**unifier l'ensemble des modifications sur un seul document** (pour notre [exemple](https://github.com/OpenTermsArchive/contrib-versions/commit/58a1d2ae4187a3260ac58f3f3c7dcd3aeacaebcd?short_path=e8bdae8#diff-e8bdae8692561f60aeac9d27a55e84fc)). La couleur **rouge** montre les éléments **supprimés**, la couleur **jaune** montre les paragraphes **modifiés**, et la couleur **verte** montrent les éléments **ajoutés**. Attention, cet affichage **ne montre pas certaines modifications** comme le changement des hyperliens et le style du texte.
47
-
48
- ### Remarques
49
-
50
- - Pour les longs documents, les **paragraphes inchangés ne seront pas affichés par défaut**. Vous pouvez manuellement les faire apparaître en cliquant sur les petites flèches juste au-dessus ou juste en-dessous des paragraphes affichés.
51
- - Vous pouvez utiliser le bouton **History n'importe où** dans le dépôt contrib-versions, qui affichera alors l'**historique des modifications subies par tous les documents se trouvant dans le dossier** où vous vous trouvez (y compris dans les sous-dossiers).
52
-
53
- ## Recevoir des notifications
54
-
55
- ### Par courriel
56
-
57
- #### Pour tous les documents d'un coup
58
-
59
- Vous pouvez [vous abonner](https://59692a77.sibforms.com/serve/MUIEAKuTv3y67e27PkjAiw7UkHCn0qVrcD188cQb-ofHVBGpvdUWQ6EraZ5AIb6vJqz3L8LDvYhEzPb2SE6eGWP35zXrpwEFVJCpGuER9DKPBUrifKScpF_ENMqwE_OiOZ3FdCV2ra-TXQNxB2sTEL13Zj8HU7U0vbbeF7TnbFiW8gGbcOa5liqmMvw_rghnEB2htMQRCk6A3eyj) pour recevoir un courriel à chaque modification d'un document dans l'ensemble de la base.
60
-
61
- **Attention, vous risquez de recevoir de nombreuses notifications !** Vous pourrez vous désabonner en répondant à n'importe quel courriel reçu.
62
-
63
- #### Recevoir les mises à jour de services ou documents spécifiques
64
-
65
- Vous pouvez vous rendre sur le site officiel [opentermsarchive.org] (https://opentermsarchive.org). De là, vous pouvez sélectionner un service, puis le type de document correspondant.
66
- Après avoir entré votre adresse électronique et cliqué sur "S'inscrire", nous ajouterons votre adresse à la liste de diffusion correspondante dans [SendInBlue](https://www.sendinblue.com/) et nous ne la conserverons nulle part ailleurs.
67
- Ensuite, chaque fois qu'une modification sera trouvée sur le document correspondant, nous vous enverrons un e-mail.
68
-
69
- Vous pouvez vous désinscrire à tout moment en cliquant sur le lien "désinscription" en bas de l'email reçu.
70
-
71
- ### Par RSS
72
-
73
- Vous pouvez recevoir une notification pour un service ou un document spécifique en vous abonnant à des flux RSS.
74
-
75
- > Un flux RSS est un type de page accessible en ligne qui contient des informations sur les derniers contenus publiés par un site web comme leur date de publication et l'adresse pour les consulter. Lorsque cette ressource est mise à jour, une application de type lecteur de flux vous notifie automatiquement et vous pouvez ainsi consulter la mise à jour.
76
-
77
- Pour obtenir l'adresse du flux RSS auquel vous abonner :
78
-
79
- 1. [Naviguez](#naviguer-dans-lhistorique-des-versions) jusqu’à la page qui présente l’historique des modifications qui vous intéressent. _Dans l'exemple de WhatsApp donné plus haut, il s’agit de [cette page](https://github.com/OpenTermsArchive/contrib-versions/commits/main/WhatsApp/Privacy%20Policy.md)._
80
- 2. Copiez l’adresse de cette page depuis la barre d’adresse de votre navigateur. _Dans l’exemple de WhatsApp, il s’agit de `https://github.com/OpenTermsArchive/contrib-versions/commits/main/WhatsApp/Privacy%20Policy.md`._
81
- 3. Ajoutez `.atom` à la fin de cette adresse. _Dans l’exemple de WhatsApp, cela donnerait `https://github.com/OpenTermsArchive/contrib-versions/commits/main/WhatsApp/Privacy%20Policy.md.atom`._
82
- 4. Abonnez votre lecteur de flux RSS à l’adresse résultante.
83
-
84
- #### Récapitulatif des flux RSS disponibles
85
-
86
- | Mis à jour pour | URL |
87
- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
88
- | l'ensemble des services et documents | `https://github.com/OpenTermsArchive/contrib-versions/commits.atom` |
89
- | l'ensemble des documents d'un service | Remplacer `$serviceId` par l'identifiant du service :<br>`https://github.com/OpenTermsArchive/contrib-versions/commits/main/$serviceId.atom` |
90
- | un document spécifique d'un service | Remplacer `$serviceId` par l'identifiant du service et `$documentType` par le type du document :<br>`https://github.com/OpenTermsArchive/contrib-versions/commits/main/$serviceId/$documentType.md.atom` |
91
-
92
- Par exemple :
93
-
94
- - Pour recevoir toutes les mises à jour des documents de `Facebook`, abonnez-vous à `https://github.com/OpenTermsArchive/contrib-versions/commits/main/Facebook.atom`.
95
- - Pour recevoir toutes les mises à jour des `Privacy Policy` de `Google`, abonnez-vous à `https://github.com/OpenTermsArchive/contrib-versions/commits/main/Google/Privacy%20Policy.md.atom`.
96
-
97
- ### Désabonnement
98
-
99
- Afin de ne plus recevoir d'e-mails de mise à jour des services, deux liens sont inclus dans chaque e-mail reçu :
100
-
101
- - un pour ne plus recevoir tous les e-mails de bot@opentermsarchive.org
102
- - un pour ne plus recevoir les e-mails d'un document particulier
103
-
104
- Ce dernier lien consiste à envoyer un courriel à contact@opentermsarchive.org pour être retiré manuellement de la liste correspondante.
105
-
106
- ## Contribuer
107
-
108
- ### Ajouter un nouveau service
109
-
110
- Voir le fichier [CONTRIBUTING](CONTRIBUTING.md) (en anglais).
package/Vagrantfile DELETED
@@ -1,38 +0,0 @@
1
- # -*- mode: ruby -*-
2
- # vi: set ft=ruby :
3
-
4
- Vagrant.configure("2") do |config|
5
- config.vm.hostname = "vagrant"
6
-
7
- config.vm.box = "debian/bullseye64" # Unable to locate package mongodb-org
8
-
9
- # in order to have the same config for both Docker and VirtualBox providers, we load the key manually
10
- # if necessary, create the key with `ssh-keygen -f ~/.ssh/ota-vagrant -q -N ""`
11
- # CAUTION: use of `~` in path causes problems with ssh
12
- config.vm.provision "file", source: File.join(ENV['HOME'], ".ssh", "ota-vagrant.pub"), destination: "/home/vagrant/.ssh/authorized_keys"
13
-
14
- # based on https://github.com/rofrano/vagrant-docker-provider#example-vagrantfile
15
- config.vm.provider :docker do |docker, override|
16
- override.vm.box = nil
17
- docker.image = "rofrano/vagrant-provider:debian"
18
- docker.remains_running = true
19
- docker.has_ssh = true
20
- docker.privileged = true
21
- docker.volumes = ["/sys/fs/cgroup:/sys/fs/cgroup:rw"]
22
- docker.create_args = ["--cgroupns=host"]
23
-
24
- # python is not installed by default in the vagrant-provider image
25
- # and deploying results in /bin/sh: 1: /usr/bin/python: not found
26
- # use a provision to fix that
27
- # only with debian, no need with ubuntu
28
- # Also need to name the provisioner, so that it runs only once https://github.com/hashicorp/vagrant/issues/7685#issuecomment-308281283
29
- config.vm.provision "install_python3", type: "shell", inline: $installPython3
30
- end
31
- end
32
-
33
- $installPython3 = <<-SCRIPT
34
- echo Updating apt...
35
- sudo apt-get update --fix-missing # Needed to fix "No package matching 'chromium' is available"
36
- echo Installing python...
37
- sudo apt-get --assume-yes install python3 python3-pip
38
- SCRIPT
package/ansible.cfg DELETED
@@ -1,13 +0,0 @@
1
- [defaults]
2
-
3
- inventory = ops/inventories/dev.yml
4
- roles_path = ops/roles
5
-
6
-
7
- # The two following lines allow to have human readable output
8
- # Use the YAML callback plugin.
9
- stdout_callback = yaml
10
- # Use the stdout_callback when running ad-hoc commands.
11
- bin_ansible_callbacks = true
12
-
13
- vault_password_file = vault.key
@@ -1,73 +0,0 @@
1
- # Choosing service name and service ID
2
-
3
- - Date: 2020-10-14
4
-
5
- ## Context and Problem Statement
6
-
7
- To scale up from 50 to 5,000 services, we need a clear way for choosing the service name and the service ID.
8
-
9
- ### We need
10
-
11
- A name that reflects the common name used by the provider itself, to be exposed in a GUI. This name is currently exposed as the name property in the service declaration.
12
- An ID of sorts that can be represented in the filesystem. This ID is currently exposed as the filename of the service declaration, without the .json extension.
13
-
14
- ### Use cases
15
-
16
- The service name is presented to end users. It should reflect as closely as possible the official service name, so that it can be identified easily.
17
- The ID is used internally and exposed for analysis. It should be easy to handle with scripts and other tools.
18
-
19
- ### Constraints for the ID
20
-
21
- As long as this ID is stored in the filesystem:
22
-
23
- - No `/` for UNIX.
24
- - No `\` for Windows.
25
- - No `:` for APFS and HFS.
26
- - No case-sensitive duplicates to support case-insensitive filesystems.
27
- - No more than 255 characters to support transfer over [FAT32](https://en.wikipedia.org/wiki/File_Allocation_Table#FAT32).
28
-
29
- UTF, spaces and capitals are all supported, even on case-insensitive filesystems.
30
-
31
- ### However
32
-
33
- - UTF in filenames can be [a (fixable) problem with Git and HFS+](https://stackoverflow.com/questions/5581857/git-and-the-umlaut-problem-on-mac-os-x).
34
- - UTF in filenames is by default quoted in Git, leading for example `été.txt` to be displayed as `"\303\251t\303\251.txt"`.
35
- - Most online services align their brand name with their domain name. Even though UTF is now officially supported in domain names, support is limited and most services, even non-Western, have an official ASCII transliteration used at least in their domain name (e.g. “qq” by Tencent, “rzd.ru” for “РЖД”, “yahoo” for “Yahoo!”).
36
- - We currently use GitHub as a GUI, so the service ID is presented to the user instead of the service name. The name is used in email notifications.
37
-
38
- ## Decision Outcome
39
-
40
- 1. The service name should be the one used by the service itself, no matter the alphabet.
41
-
42
- - _Example: `туту.ру`_.
43
-
44
- 2. We don't support non-ASCII characters in service IDs, at least as long as the database is Git and the filesystem, in order to minimise risk. Service IDs are derived from the service name through normalization into ASCII.
45
-
46
- - _Example: `туту.ру` → `tutu.ru`_.
47
- - _Example: `historielærer.dk` → `historielaerer.dk`_.
48
- - _Example: `RTÉ` → `RTE`_.
49
-
50
- 3. We support punctuation, except characters that have meaning at filesystem level (`:`, `/`, `\`). These are replaced with a dash (`-`).
51
-
52
- - _Example: `Yahoo!` → `Yahoo!`_.
53
- - _Example: `Last.fm` → `Last.fm`_.
54
- - _Example: `re:start` → `re-start`_.
55
- - _Example: `we://` → `we---`_.
56
-
57
- 4. We support capitals. Casing is expected to reflect the official service name casing.
58
-
59
- - _Example: `hi5` → `hi5`_.
60
- - _Example: `DeviantArt` → `DeviantArt`_.
61
- - _Example: `LINE` → `LINE`_.
62
-
63
- 5. We support spaces. Spaces are expected to reflect the official service name spacing.
64
-
65
- - _Example: `App Store` → `App Store`_.
66
- - _Example: `DeviantArt` → `DeviantArt`_.
67
-
68
- 6. We prefix the service name by the provider name when self-references are ambiguous, separated by a space. For example, Facebook refers to their Self-serve Ads service simply as “Ads”, which we cannot use in a shared database. We thus call the service “Facebook Ads”.
69
-
70
- - _Example: `Ads` (by Facebook) → `Facebook Ads`_.
71
- - _Example: `Analytics` (by Google) → `Google Analytics`_.
72
- - _Example: `Firebase` (by Google) → `Firebase`_.
73
- - _Example: `App Store` (by Apple) → `App Store`_.