appwrite-utils-cli 0.0.32 → 0.0.34

package/README.md

@@ -1,97 +1,103 @@
-# appwrite-utils-cli
-
-## Overview
-
-`appwrite-utils-cli` is a powerful command-line interface tool designed for Appwrite developers who need to manage database migrations, schema generation, data import, and much more. This CLI tool facilitates complex tasks like setting up databases, running migrations, generating schemas, and managing backups efficiently, making it an indispensable part of your Appwrite project management.
-
-## Features
-
-- **Easy Configuration**: Initialize your Appwrite project configurations interactively directly from the command line.
-- **Database Migrations**: Control the migration process with options to target production, staging, or development environments.
-- **Schema Generation**: Generate and manage TypeScript schemas directly from your Appwrite database schemas.
-- **Data Import**: Facilitate the import of data into your Appwrite databases with comprehensive command-line support.
-- **Backup Management**: Create backups of your Appwrite databases to ensure data integrity and safety.
-- **Flexible Database Management**: Includes commands to wipe databases, documents, or user data, providing flexibility in managing your database state during development or testing.
-
-## Installation
-
-To use `appwrite-utils-cli`, you can install it globally via npm to make it accessible from anywhere in your command line:
-
-```bash
-npm install -g appwrite-utils-cli
-```
-
-However, due to the nature of the speed at which I am developing this project, I would recommend the following command:
-
-```bash
-npx --package=appwrite-utils-cli@latest appwrite-migrate --arg1 --arg2 --arg3
-```
-
-**DO NOT INSTALL THIS LOCALLY INTO YOUR PROJECT, IT IS MEANT TO BE USED AS A COMMAND LINE TOOL ONLY**
-
-## Usage
-
-After installation, you can access the tool directly from your command line using the provided commands. Here's how you can use the different functionalities:
-
-### Initialization
-
-Interactively set up your Appwrite project with necessary configurations:
-
-```bash
-npx --package=appwrite-utils-cli@latest appwrite-init
-```
-
-### Running Migrations and Tasks
-
-Run migration and management tasks with specific flags according to your needs:
-
-```bash
-npx --package=appwrite-utils-cli@latest appwrite-migrate --args
-```
-
-Replace `--args` with the appropriate options:
-
-- `--prod`: Run tasks in the production environment.
-- `--staging`: Run tasks in the staging environment.
-- `--dev`: Run tasks in the development environment.
-- `--wipe`: Wipe all databases.
-- `--wipe-docs` or `--wipeDocs`: Wipe all documents in the databases.
-- `--generate`: Generate TypeScript schemas from database schemas.
-- `--import`: Import data into your databases.
-- `--backup`: Perform a backup of your databases.
-- `--wipe-users` or `--wipeUsers`: Wipe all user data.
-- `--write-data` or `--writeData`: Write converted imported data to file
-- `--sync`: Synchronize your project's config and generate schema for your database
-- `--endpoint`: Set a different endpoint for the migration target
-- `--project`: Set a different project ID for the migration target
-- `--key`: Set a different API key for the migration target
-
-### OpenAPI Generation (almost done, in progress)
-
-Recently, I have also added an optional OpenAPI generation for each attribute in the schema. This is because I needed it and because I felt it would be nice to have. This is done using [this package](https://github.com/asteasolutions/zod-to-openapi), many thanks to them.
-
-To use it, add a `description` to any attribute or collection, and it will export that schema to the `appwrite/openapi` folder
-
-This setup ensures that developers have robust tools at their fingertips to manage complex Appwrite projects effectively from the command line. I also have added logging automatically for information and errors as the console can be hard to keep up with.
-
-### Roadmap
-
-- Syncing configuration (DONE)
-- Better file format for config (potentially)
-- Separation of collections and import configuration from main config
-
-### Changelog
-
-- 0.0.29: If you use the `description` variable in an attribute and collection, it'll add that description to the generated schemas. This assumes you have `zod-to-openpi`
-- 0.0.275: THINGS ARE NOW IN TYPESCRIPT WOOHOO. No but for reaal, super happy to report that everything has been converted to TypeScript, just way too many changes, I hope you enjoy it!
-- 0.0.274: Small improvement for attribute handling, rather than getting it every attribute, I check the collections attributes
-- 0.0.273: Small fix for relationship attribute comparisons
-- 0.0.272: That's what I get for not testing lmao, also updated logic for checking for existing attributes to take the `format` into consideration from the database (URL's are not of `type: "url"`, they are of `format: "url"`)
-- 0.0.271: Small change to update attributes that are different from each other by deleting the attribute and recreating, as we cannot update most things
-- 0.0.270: Fixed enums in `--sync`, added optional OpenAPI generation (in progress, almost done, but wanted to push other changes), added `--endpoint`, `--project`, `--key` as optional parameters to change the target destination (shoutout to [pingu24k](https://github.com/pingu2k4) for pointing out these bugs and suggesting those changes for endpoint customization)
-- 0.0.254: Added `--sync` to synchronize your Appwrite instance with your local `appwriteConfig.yaml` and generate schemas
-- 0.0.253: Added `--writeData` (or `--write-data`) to command to write the output of the import data to a file called dataLoaderOutput in your root dir
-- 0.0.23: Added batching to user deletion
-- 0.0.22: Converted all import processes except `postImportActions` and Relationship Resolution to the local data import, so it should be much faster.
-- 0.0.6: Added `setTargetFieldFromOtherCollectionDocumentsByMatchingField` for the below, but setting a different field than the field you matched. The names are long, but at least you know what's going on lmao.
-- 0.0.5: Added `setFieldFromOtherCollectionDocuments` to set an array of ID's for instance from another collection as a `postImportAction`
+# appwrite-utils-cli
+
+## Overview
+
+`appwrite-utils-cli` is a powerful command-line interface tool designed for Appwrite developers who need to manage database migrations, schema generation, data import, and much more. This CLI tool facilitates complex tasks like setting up databases, running migrations, generating schemas, and managing backups efficiently, making it an indispensable part of your Appwrite project management.
+
+## Features
+
+- **Easy Configuration**: Initialize your Appwrite project configurations interactively directly from the command line.
+- **Database Migrations**: Control the migration process with options to target production, staging, or development environments.
+- **Schema Generation**: Generate and manage TypeScript schemas directly from your Appwrite database schemas.
+- **Data Import**: Facilitate the import of data into your Appwrite databases with comprehensive command-line support.
+- **Backup Management**: Create backups of your Appwrite databases to ensure data integrity and safety.
+- **Flexible Database Management**: Includes commands to wipe databases, documents, or user data, providing flexibility in managing your database state during development or testing.
+
+## Installation
+
+To use `appwrite-utils-cli`, you can install it globally via npm to make it accessible from anywhere in your command line:
+
+```bash
+npm install -g appwrite-utils-cli
+```
+
+However, due to the nature of the speed at which I am developing this project, I would recommend the following command:
+
+```bash
+npx --package=appwrite-utils-cli@latest appwrite-migrate --arg1 --arg2 --arg3
+```
+
+**DO NOT INSTALL THIS LOCALLY INTO YOUR PROJECT, IT IS MEANT TO BE USED AS A COMMAND LINE TOOL ONLY**
+
+## Usage
+
+After installation, you can access the tool directly from your command line using the provided commands. Here's how you can use the different functionalities:
+
+### Initialization
+
+Interactively set up your Appwrite project with necessary configurations:
+
+```bash
+npx --package=appwrite-utils-cli@latest appwrite-init
+```
+
+### Running Migrations and Tasks
+
+Run migration and management tasks with specific flags according to your needs:
+
+```bash
+npx --package=appwrite-utils-cli@latest appwrite-migrate --args
+```
+
+Replace `--args` with the appropriate options:
+
+- `--prod`: Run tasks in the production environment.
+- `--staging`: Run tasks in the staging environment.
+- `--dev`: Run tasks in the development environment.
+- `--wipe`: Wipe all databases.
+- `--wipe-docs`: Wipe all documents in the databases.
+- `--generate`: Generate TypeScript schemas from database schemas.
+- `--import`: Import data into your databases.
+- `--backup`: Perform a backup of your databases.
+- `--wipe-users`: Wipe all user data.
+- `--write-data`: Write converted imported data to file
+- `--sync`: Synchronize your project's config and generate schema for your database
+- `--endpoint`: Set a different endpoint for the migration target
+- `--project`: Set a different project ID for the migration target
+- `--key`: Set a different API key for the migration target
+
+## If you run out of RAM
+
+This happens because Node only allocates 4 GB, it'll only happen if you're importing a ton of data, but you can use `export NODE_OPTIONS="--max-old-space-size=16384"` or the relevant command for your system if that doesn't work, and it'll set the env var to whatever. That's 16 GB, I would recommend `8192` for most.
+
+### OpenAPI Generation (almost done, in progress)
+
+Recently, I have also added an optional OpenAPI generation for each attribute in the schema. This is because I needed it and because I felt it would be nice to have. This is done using [this package](https://github.com/asteasolutions/zod-to-openapi), many thanks to them.
+
+To use it, add a `description` to any attribute or collection, and it will export that schema to the `appwrite/openapi` folder
+
+This setup ensures that developers have robust tools at their fingertips to manage complex Appwrite projects effectively from the command line. I also have added logging automatically for information and errors as the console can be hard to keep up with.
+
+### Roadmap
+
+- Syncing configuration (DONE)
+- Better file format for config (potentially)
+- Separation of collections and import configuration from main config
+
+### Changelog
+
+- 0.0.34: Fixed the `bin` section of the package.json, apparently you can't use `node` to run it
+- 0.0.33: Fixed `idMappings`, if you are importing data and use the `idMappings` functionality, you can set a `fieldToSet` based on the value of a `sourceField` in the current imported items data (whether it's in the final data or the original), in order to match another field in another collection. So if you had a store, and it had items and the items have a Region ID for instance. You can then, in your regionId of the items, setup an `idMapping` that will allow you to map the value of the `targetField` based on the value of the `targetFieldToMatch` in the `targetCollection`. Sounds complex, but it's very useful. Like psuedo-relationship resolution, without the relationships.
+- 0.0.29: If you use the `description` variable in an attribute and collection, it'll add that description to the generated schemas. This assumes you have `zod-to-openpi`
+- 0.0.275: THINGS ARE NOW IN TYPESCRIPT WOOHOO. No but for reaal, super happy to report that everything has been converted to TypeScript, just way too many changes, I hope you enjoy it!
+- 0.0.274: Small improvement for attribute handling, rather than getting it every attribute, I check the collections attributes
+- 0.0.273: Small fix for relationship attribute comparisons
+- 0.0.272: That's what I get for not testing lmao, also updated logic for checking for existing attributes to take the `format` into consideration from the database (URL's are not of `type: "url"`, they are of `format: "url"`)
+- 0.0.271: Small change to update attributes that are different from each other by deleting the attribute and recreating, as we cannot update most things
+- 0.0.270: Fixed enums in `--sync`, added optional OpenAPI generation (in progress, almost done, but wanted to push other changes), added `--endpoint`, `--project`, `--key` as optional parameters to change the target destination (shoutout to [pingu24k](https://github.com/pingu2k4) for pointing out these bugs and suggesting those changes for endpoint customization)
+- 0.0.254: Added `--sync` to synchronize your Appwrite instance with your local `appwriteConfig.yaml` and generate schemas
+- 0.0.253: Added `--writeData` (or `--write-data`) to command to write the output of the import data to a file called dataLoaderOutput in your root dir
+- 0.0.23: Added batching to user deletion
+- 0.0.22: Converted all import processes except `postImportActions` and Relationship Resolution to the local data import, so it should be much faster.
+- 0.0.6: Added `setTargetFieldFromOtherCollectionDocumentsByMatchingField` for the below, but setting a different field than the field you matched. The names are long, but at least you know what's going on lmao.
+- 0.0.5: Added `setFieldFromOtherCollectionDocuments` to set an array of ID's for instance from another collection as a `postImportAction`

package/dist/migrations/backup.d.ts

@@ -316,6 +316,7 @@ export declare const getMigrationCollectionSchemas: () => {
                     targetField: string;
                     targetCollection: string;
                     fieldToSet?: string | undefined;
+                    targetFieldToMatch?: string | undefined;
                 }[] | undefined;
                 updateMapping?: {
                     targetField: string;
@@ -568,6 +569,7 @@ export declare const getMigrationCollectionSchemas: () => {
                     targetField: string;
                     targetCollection: string;
                     fieldToSet?: string | undefined;
+                    targetFieldToMatch?: string | undefined;
                 }[] | undefined;
                 updateMapping?: {
                     targetField: string;

package/dist/migrations/dataLoader.d.ts

@@ -1,5 +1,5 @@
 import type { ImportDataActions } from "./importDataActions.js";
-import { type AppwriteConfig, type AttributeMappings, type CollectionCreate, type ConfigDatabase, type IdMapping, type ImportDef } from "appwrite-utils";
+import { type AppwriteConfig, type AttributeMappings, type CollectionCreate, type ConfigDatabase, type ImportDef } from "appwrite-utils";
 import { z } from "zod";
 import { type Databases } from "node-appwrite";
 export declare const CollectionImportDataSchema: z.ZodObject<{
@@ -61,6 +61,14 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
             required: z.ZodOptional<z.ZodDefault<z.ZodBoolean>>;
             array: z.ZodOptional<z.ZodDefault<z.ZodBoolean>>;
             min: z.ZodOptional<z.ZodNumber>;
+            /**
+             * Prepares the data for creating documents in a collection.
+             * This involves loading the data, transforming it, and handling ID mappings.
+             *
+             * @param db - The database configuration.
+             * @param collection - The collection configuration.
+             * @param importDef - The import definition containing the attribute mappings and other relevant info.
+             */
             max: z.ZodOptional<z.ZodNumber>;
             xdefault: z.ZodOptional<z.ZodNullable<z.ZodNumber>>;
             description: z.ZodOptional<z.ZodNullable<z.ZodUnion<[z.ZodString, z.ZodRecord<z.ZodString, z.ZodString>]>>>;
@@ -359,10 +367,21 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
             type: z.ZodOptional<z.ZodDefault<z.ZodEnum<["create", "update"]>>>;
             filePath: z.ZodString;
             basePath: z.ZodOptional<z.ZodString>;
+            /**
+             * Generates attribute mappings with post-import actions based on the provided attribute mappings.
+             * This method checks each mapping for a fileData attribute and adds a post-import action to create a file
+             * and update the field with the file's ID if necessary.
+             *
+             * @param attributeMappings - The attribute mappings from the import definition.
+             * @param context - The context object containing information about the database, collection, and document.
+             * @param item - The item being imported, used for resolving template paths in fileData mappings.
+             * @returns The attribute mappings updated with any necessary post-import actions.
+             */
             primaryKeyField: z.ZodDefault<z.ZodString>;
             idMappings: z.ZodOptional<z.ZodArray<z.ZodObject<{
                 sourceField: z.ZodString;
                 fieldToSet: z.ZodOptional<z.ZodString>;
+                targetFieldToMatch: z.ZodOptional<z.ZodString>;
                 targetField: z.ZodString;
                 targetCollection: z.ZodString;
             }, "strip", z.ZodTypeAny, {
@@ -370,11 +389,13 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }, {
                 sourceField: string;
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }>, "many">>;
             updateMapping: z.ZodOptional<z.ZodObject<{
                 originalIdField: z.ZodString;
@@ -484,6 +505,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -517,6 +539,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -665,6 +688,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -819,6 +843,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -839,6 +864,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
             idMappings: z.ZodOptional<z.ZodArray<z.ZodObject<{
                 sourceField: z.ZodString;
                 fieldToSet: z.ZodOptional<z.ZodString>;
+                targetFieldToMatch: z.ZodOptional<z.ZodString>;
                 targetField: z.ZodString;
                 targetCollection: z.ZodString;
             }, "strip", z.ZodTypeAny, {
@@ -846,11 +872,13 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }, {
                 sourceField: string;
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }>, "many">>;
             updateMapping: z.ZodOptional<z.ZodObject<{
                 originalIdField: z.ZodString;
@@ -960,6 +988,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -993,6 +1022,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1031,6 +1061,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1069,6 +1100,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1109,6 +1141,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1257,6 +1290,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1301,6 +1335,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1452,6 +1487,7 @@ export declare const CollectionImportDataSchema: z.ZodObject<{
                 targetField: string;
                 targetCollection: string;
                 fieldToSet?: string | undefined;
+                targetFieldToMatch?: string | undefined;
             }[] | undefined;
             updateMapping?: {
                 targetField: string;
@@ -1501,6 +1537,7 @@ export declare class DataLoader {
                     targetField: string;
                     targetCollection: string;
                     fieldToSet?: string | undefined;
+                    targetFieldToMatch?: string | undefined;
                 }[] | undefined;
                 updateMapping?: {
                     targetField: string;
@@ -1649,6 +1686,7 @@ export declare class DataLoader {
                     targetField: string;
                     targetCollection: string;
                     fieldToSet?: string | undefined;
+                    targetFieldToMatch?: string | undefined;
                 }[] | undefined;
                 updateMapping?: {
                     targetField: string;
@@ -1697,10 +1735,8 @@ export declare class DataLoader {
     setupMaps(dbId: string): Promise<void>;
     getAllUsers(): Promise<import("node-appwrite").Models.User<import("node-appwrite").Models.Preferences>[]>;
     start(dbId: string): Promise<void>;
-    dealWithMergedUsers(): Promise<void>;
+    dealWithMergedUsers(): void;
     updateOldReferencesForNew(): Promise<void>;
-    updateReferencesInRelatedCollections(): Promise<void>;
-    findNewIdForOldId(oldId: string, idMapping: IdMapping, importDef: ImportDef): any;
     private writeMapsToJsonFile;
     /**
      * Prepares user data by checking for duplicates based on email or phone, adding to a duplicate map if found,