@tricoteuses/assemblee 2.2.1 → 2.3.0

package/README.md

@@ -42,53 +42,74 @@ npm install
 
 ### Basic usage
 
-Create a folder where the data will be downloaded and run the following command to download, reorganize and clean the data.
-
+Create a directory to store the data, then run the following command to download, reorganize and clean the data.
 ```bash
 mkdir ../assemblee-data/
-
-# Download and clean open data
 npm run data:download ../assemblee-data
 ```
 
-Data from other sources is also available :
-
-```bash
-# Retrieval of députés' pictures from Assemblée nationale's website
-npm run data:retrieve_deputes_photos ../assemblee-data
+### Available Commands
 
-# Retrieval of sénateurs' pictures from Assemblée nationale's website
-npm run data:retrieve_senateurs_photos ../assemblee-data
+- `npm run data:download <dir>`: Download, reorganize, and clean data
+- `npm run data:retrieve_open_data <dir>`: Download raw data files.
+- `npm run data:reorganize_data <dir>`: Reorganize raw files by entity.
+- `npm run data:clean_data <dir>`: Clean and validate reorganized files.
+- `npm run data:retrieve_deputes_photos <dir>`: Retrieval of députés' pictures from Assemblée nationale's website
+- `npm run data:retrieve_senateurs_photos <dir>`: Retrieval of sénateurs' pictures from Assemblée nationale's website
+- `npm run data:retrieve_documents <dir>`: Retrieval of legislative documents from Assemblée nationale's website
+- `npm run data:retrieve_pending_amendements <dir>`: Retrieval of pending amendments from Assemblée nationale's website (waiting to be processed by Assemblée services)
 
-# Retrieval of pending amendments from Assemblée nationale's website (waiting to be processed by Assemblée services)
-npm run data:retrieve_pending_amendements ../assemblee-data
-```
 
 _Notes_:
 
 - Reorganized files (generated by the _data:reorganize_data_ command) are also available in [Tricoteuses / Data / Données brutes de l'Assemblée](https://git.en-root.org/tricoteuses/data/assemblee-brut). They are updated on a regular basis.
 - Split & cleaned files (generated by the _data:clean_data_ command) are also available in [Tricoteuses / Data / Données nettoyées de l'Assemblée](https://git.en-root.org/tricoteuses/data/assemblee-nettoye) with the `_nettoye` suffix. They are updated on a regular basis.
 
-### Filtering options
+### Filtering Options
 
 Downloading and cleaning all the data is long and takes up a lot of disk space. It is possible to choose the type of data that you want to retrieve to reduce the load.
 
-To download only a type of dataset, use the _--categories_ option (shortcut _-k_) :
+Examples:
 
 ```bash
-# Available options : ActeursEtOrganes, Agendas, Amendements, DossiersLegislatifs, Photos, Scrutins, Questions, ComptesRendusSeances
-npm run data:download ../assemblee-data -- --categories Amendements
-```
+# Only download amendments
+npm run data:download ../assemblee-data -- -k Amendements
 
-To download a specific or multiple legislatures, use the *--legislature* option (shortcut *-l*):
-```bash
-# Available options : 14, 15, 16, 17
+# Only process 16th and 17th legislatures
 npm run data:download ../assemblee-data -- -l 16 -l 17
-
 ```
 
+### Common Options
+
+- `--categories` or `-k <name>`: Filter by dataset categories (Available options : `ActeursEtOrganes`, `Agendas`, `Amendements`, `DossiersLegislatifs`, `Photos`, `Scrutins`, `Questions`, `ComptesRendusSeances`)
+
+- `--legislature` or `-l <number>`: Specify one or more legislatures to process (e.g., `-l 15 -l 16`)
+- `--dataDir <path>` (Mandatory): Path to the working directory where all data is stored (required)
+- `--silent` or `-s`: Disable logging
+- `--verbose` or `-v`: Enable verbose logging
+- `--fetch` or `-f`: Force re-download of data even if already present
+- `--commit` or `-c`: Automatically commit cleaned data
+- `--pull` or `-p`: Pull repositories before starting
+- `--clone` or `-C <url>`: Clone Git repositories from a remote group or organization
+- `--remote` or `-r <name>`: Push commits to specified Git remote(s)
+
 If you use such options, use them in all subsequent commands too (_data:regorganize_data_ and _data:clean_data_).
 
+### Options for Cleaning Data
+
+- `--dataset` or `-d <name>`: Clean a specific dataset only
+- `--no-reset-after-commit`: Skip Git reset after committing (useful to preserve local changes)
+- `--no-validate` or `-V`: Skip schema validation during cleaning
+- `--fullCompteRenduCommissions`: Force reprocessing of commission reports
+- `--fetchDocuments` : Specify to retrieve documents like reports, videos metadata files
+- `--parseDocuments`: Specify to parse documents into cleaned json
+
+### Options for Retrieving Documents
+
+- `--full` or `-f`: Retrieve all documents, even those already downloaded
+- `--document-type` or `-T <type>`: Restrict to specific document types (e.g., `PION`)
+
+
 ## Download using Docker
 
 A Docker image that downloads and cleans the data all at once is available. Build it locally or run it from the container registry.

package/lib/cleaners/scrutins.d.ts

@@ -1 +1,2 @@
+export declare function cleanDecompteVoix(decompteVoix: any): void;
 export declare function cleanScrutin(scrutin: any): void;