data-primals-engine 1.7.2 → 1.7.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (78) hide show
  1. package/README.md +160 -160
  2. package/client/package-lock.json +8430 -8430
  3. package/client/src/APIInfo.jsx +1 -1
  4. package/client/src/App.jsx +2 -1
  5. package/client/src/App.scss +1635 -1626
  6. package/client/src/AssistantChat.jsx +1 -0
  7. package/client/src/CalendarView.jsx +1 -0
  8. package/client/src/ContentView.jsx +3 -3
  9. package/client/src/Dashboard.jsx +5 -2
  10. package/client/src/DashboardChart.jsx +1 -0
  11. package/client/src/DashboardFlexViewItem.jsx +1 -0
  12. package/client/src/DashboardHtmlViewItem.jsx +1 -0
  13. package/client/src/DashboardView.jsx +2 -0
  14. package/client/src/DataEditor.jsx +0 -1
  15. package/client/src/DataImporter.jsx +489 -468
  16. package/client/src/DataLayout.jsx +6 -3
  17. package/client/src/DataTable.jsx +4 -3
  18. package/client/src/Dialog.jsx +92 -90
  19. package/client/src/Dialog.scss +122 -116
  20. package/client/src/DisplayFlexNodeRenderer.jsx +1 -0
  21. package/client/src/FlexBuilderControls.jsx +1 -0
  22. package/client/src/FlexBuilderPreview.jsx +1 -1
  23. package/client/src/FlexNode.jsx +1 -1
  24. package/client/src/HistoryDialog.jsx +3 -2
  25. package/client/src/KPIWidget.jsx +1 -1
  26. package/client/src/KanbanView.jsx +1 -0
  27. package/client/src/ModelCreator.jsx +4 -0
  28. package/client/src/ModelList.jsx +1 -0
  29. package/client/src/PackGallery.jsx +5 -4
  30. package/client/src/RTETrans.jsx +1 -1
  31. package/client/src/RelationField.jsx +2 -0
  32. package/client/src/RelationSelectorWidget.jsx +2 -0
  33. package/client/src/RelationValue.jsx +1 -0
  34. package/client/src/RestoreDialog.jsx +1 -0
  35. package/client/src/WorkflowEditor.jsx +3 -0
  36. package/client/src/contexts/CommandContext.jsx +2 -1
  37. package/client/src/contexts/ModelContext.jsx +3 -3
  38. package/client/src/hooks/data.js +1 -0
  39. package/client/src/hooks/useValidation.js +1 -0
  40. package/client/src/translations.js +24 -24
  41. package/doc/AI-assistance.md +87 -63
  42. package/doc/Concepts.md +122 -0
  43. package/doc/Custom-Endpoints.md +31 -0
  44. package/doc/Event-system.md +13 -14
  45. package/doc/Home.md +33 -0
  46. package/doc/Modules.md +83 -0
  47. package/doc/Packs-gallery.md +8 -23
  48. package/doc/Workflows.md +32 -0
  49. package/doc/automation-workflows.md +141 -102
  50. package/doc/dashboards-kpis-charts.md +47 -49
  51. package/doc/data-management.md +126 -120
  52. package/doc/data-models.md +68 -75
  53. package/doc/roles-permissions.md +144 -43
  54. package/doc/sharding-replication.md +158 -0
  55. package/doc/users.md +54 -30
  56. package/package.json +1 -1
  57. package/server.js +37 -37
  58. package/src/constants.js +7 -8
  59. package/src/data.js +521 -520
  60. package/src/email.js +157 -154
  61. package/src/engine.js +117 -7
  62. package/src/gameObject.js +6 -0
  63. package/src/i18n.js +0 -1
  64. package/src/modules/auth-google/index.js +53 -50
  65. package/src/modules/bucket.js +346 -339
  66. package/src/modules/data/data.backup.js +400 -376
  67. package/src/modules/data/data.cluster.js +410 -42
  68. package/src/modules/data/data.operations.js +3666 -3635
  69. package/src/modules/data/data.replication.js +106 -64
  70. package/src/modules/data/data.routes.js +87 -64
  71. package/src/modules/data/data.scheduling.js +2 -1
  72. package/src/modules/file.js +248 -247
  73. package/src/modules/user.js +11 -1
  74. package/src/sso.js +91 -25
  75. package/test/cluster.test.js +221 -0
  76. package/test/core.test.js +0 -2
  77. package/test/replication.test.js +163 -0
  78. package/doc/core-concepts.md +0 -33
@@ -1,75 +1,68 @@
1
- # Data Models: Structuring Your Information
2
-
3
- Data Models are the fundamental building blocks in `data-primals-engine`, defining how your application's data is structured and stored. They are essentially blueprints for collections of documents, enabling you to organize and manage diverse types of information.
4
-
5
- ## What is a Data Model?
6
-
7
- In `data-primals-engine`, a data model is a flexible definition of a data entity. Unlike traditional relational databases, models here are schema-less (thanks to MongoDB), meaning you can evolve their structure over time without rigid migration processes. Each model corresponds to a collection in your MongoDB database.
8
-
9
- ## Defining a Model
10
-
11
- Models can be defined either through the platform's user interface or by providing a JSON schema. Key attributes of a model include:
12
-
13
- - **`name`** (string, unique): The technical name of the model (e.g., `product`, `user`, `order`). This is used for API interactions and internal references.
14
- - **`description`** (string): A brief explanation of the model's purpose.
15
- - **`icon`** (string): An icon (e.g., a Font Awesome class) to visually represent the model in the UI.
16
- - **`tags`** (array of strings): Keywords for categorization and filtering.
17
- - **`locked`** (boolean): Indicates if the model is a system model and cannot be modified or deleted via the standard UI (e.g., `user`, `permission`).
18
- - **`fields`** (array of objects): The core of the model, defining its attributes.
19
-
20
- ## Fields: The Attributes of a Model
21
-
22
- Each field within a model defines a specific piece of data. Fields have various properties to control their type, behavior, and validation:
23
-
24
- - **`name`** (string, unique within model): The name of the attribute (e.g., `title`, `price`, `email`).
25
- - **`type`** (string, required): The data type of the field. Common types include:
26
- - `string`, `string_t` (translatable string)
27
- - `number`
28
- - `boolean`
29
- - `datetime`, `date`
30
- - `email`, `url`, `phone`, `password`
31
- - `richtext`, `richtext_t` (translatable rich text)
32
- - `enum` (predefined list of values)
33
- - `file` (for file uploads)
34
- - `relation` (to link to another model)
35
- - `array` (for lists of values or sub-documents)
36
- - `code` (for storing code snippets, e.g., JSON, JavaScript)
37
- - **`required`** (boolean): If `true`, the field must have a value.
38
- - **`unique`** (boolean): If `true`, the field's value must be unique across all documents in the collection.
39
- - **`default`**: A default value for the field.
40
- - **`min`, `max`**: Minimum and maximum values for `number` fields, or length for `string` fields.
41
- - **`relation`** (string, for `relation` type): The name of the model this field relates to.
42
- - **`multiple`** (boolean, for `relation` type): If `true`, the field can relate to multiple documents in the target model.
43
- - **`hint`** (string): A helpful description displayed in the UI.
44
-
45
- ## Example: The `product` Model
46
-
47
- The `product` model, defined in `defaultModels.js`, illustrates how fields are used to structure information about a product:
48
-
49
- ```javascript
50
- product: {
51
- name: 'product',
52
- "icon": "FaShoppingBag",
53
- "description": "",
54
- "tags": ["ecommerce", "products"],
55
- fields: [
56
- { name: 'name', type: 'string_t', required: true },
57
- { name: 'image', type: 'array', itemsType: 'file', mimeTypes: ['image/jpeg', 'image/png', 'image/gif', 'image/webp'] },
58
- { name: 'description', type: 'richtext_t' },
59
- { name: 'price', type: 'number', required: true },
60
- { name: 'currency', type: 'relation', relation: 'currency', required: true },
61
- { name: 'billingFrequency', type: 'enum', items: ['none', 'monthly', 'yearly'] },
62
- { name: 'slug', type: 'string', required: true, unique: true },
63
- { name: 'brand', type: 'relation', relation: 'brand' },
64
- { name: 'category', type: 'relation', relation: 'taxonomy' },
65
- { name: 'seoTitle', type: 'string_t' },
66
- { name: 'seoDescription', type: 'string_t' }
67
- ]
68
- },
69
- ```
70
-
71
- This example shows how a `product` can have a translatable `name`, multiple `image` files, a `price` with a `currency` relation, and be linked to a `brand` and `category` (taxonomy).
72
-
73
- By defining models and their fields, you create a robust and adaptable data foundation for your `data-primals-engine` application.
74
-
75
- **Next: Data Management**
1
+ # Data Models: Structuring Your Information
2
+
3
+ Data Models are the fundamental building blocks in `data-primals-engine`, defining how your application's data is structured and stored. They are essentially blueprints for collections of documents, enabling you to organize and manage diverse types of information.
4
+
5
+ ## What is a Data Model?
6
+
7
+ In `data-primals-engine`, a data model is a flexible definition of a data entity. Unlike traditional relational databases, models here are schema-less (thanks to MongoDB), meaning you can evolve their structure over time without rigid migration processes. Each model corresponds to a collection in your MongoDB database.
8
+
9
+ ## Defining a Model
10
+
11
+ Models can be defined either through the platform's user interface or by providing a JSON schema. Key attributes of a model include:
12
+
13
+ | Attribute | Type | Description |
14
+ |:--------------|:--------------------|:--------------------------------------------------------------------------------------------------------|
15
+ | **`name`** | string, unique | The technical name of the model (e.g., `product`, `user`). Used for API and internal references. |
16
+ | **`description`** | string | A brief explanation of the model's purpose. |
17
+ | **`icon`** | string | An icon (e.g., a Font Awesome class) to visually represent the model in the UI. |
18
+ | **`tags`** | array of strings | Keywords for categorization and filtering. |
19
+ | **`locked`** | boolean | If `true`, the model is a system model and cannot be modified or deleted via the UI. |
20
+ | **`fields`** | array of objects | The core of the model, defining its attributes. |
21
+
22
+ ## Fields: The Attributes of a Model
23
+
24
+ Each field within a model defines a specific piece of data. Fields have various properties to control their type, behavior, and validation:
25
+
26
+ | Attribute | Type | Description -
27
+ |:--------------|:--------------------|:--------------------------------------------------------------------------------------------------------|
28
+ | **`name`** | string | The name of the attribute (e.g., `title`, `price`, `email`). Unique within the model. -
29
+ | **`type`** | string | The data type of the field. Common types include `string`, `number`, `boolean`, `datetime`, `email`, `richtext`, `enum`, `file`, `relation`, `array`, and `code`. -
30
+ | **`required`** | boolean | If `true`, the field must have a value. -
31
+ | **`unique`** | boolean | If `true`, the field's value must be unique across all documents in the collection. -
32
+ | **`default`** | any | A default value for the field. -
33
+ | **`min`, `max`** | number | Minimum and maximum values for `number` fields, or length for `string` fields. -
34
+ | **`relation`** | string | The name of the model this field relates to (for `relation` type). -
35
+ | **`multiple`** | boolean | If `true`, the field can relate to multiple documents in the target model (for `relation` type). -
36
+ | **`hint`** | string | A helpful description displayed in the UI. -
37
+
38
+ ## Example: The `product` Model
39
+
40
+ The `product` model, defined in `defaultModels.js`, illustrates how fields are used to structure information about a product:
41
+
42
+ ```javascript
43
+ product: {
44
+ name: 'product',
45
+ "icon": "FaShoppingBag",
46
+ "description": "",
47
+ "tags": ["ecommerce", "products"],
48
+ fields: [
49
+ { name: 'name', type: 'string_t', required: true },
50
+ { name: 'image', type: 'array', itemsType: 'file', mimeTypes: ['image/jpeg', 'image/png', 'image/gif', 'image/webp'] },
51
+ { name: 'description', type: 'richtext_t' },
52
+ { name: 'price', type: 'number', required: true },
53
+ { name: 'currency', type: 'relation', relation: 'currency', required: true },
54
+ { name: 'billingFrequency', type: 'enum', items: ['none', 'monthly', 'yearly'] },
55
+ { name: 'slug', type: 'string', required: true, unique: true },
56
+ { name: 'brand', type: 'relation', relation: 'brand' },
57
+ { name: 'category', type: 'relation', relation: 'taxonomy' },
58
+ { name: 'seoTitle', type: 'string_t' },
59
+ { name: 'seoDescription', type: 'string_t' }
60
+ ]
61
+ },
62
+ ```
63
+
64
+ This example shows how a `product` can have a translatable `name`, multiple `image` files, a `price` with a `currency` relation, and be linked to a `brand` and `category` (taxonomy).
65
+
66
+ By defining models and their fields, you create a robust and adaptable data foundation for your `data-primals-engine` application.
67
+
68
+ **[Next: Data Management](data-management)**
@@ -1,43 +1,144 @@
1
- # Roles and Permissions: Define Who Can See and Do What
2
-
3
- The `data-primals-engine` implements a robust Role-Based Access Control (RBAC) system to manage user authorizations. This system relies on two core models: `role` and `permission`, allowing for granular control over what users can access and perform within the platform.
4
-
5
- ## Permissions
6
-
7
- A **Permission** defines a specific action or access right within the system. Permissions can be very broad (e.g., "admin.full_access") or highly specific (e.g., "product.edit").
8
-
9
- ### Key Fields of the `permission` Model:
10
-
11
- - **`name`** (string, required): A unique identifier for the permission (e.g., `model.create`, `product.read`, `user.delete`).
12
- - **`description`** (richtext, optional): A detailed explanation of what the permission grants.
13
- - **`filter`** (code - JSON, optional): A JSON filter that restricts the scope of this permission. This is a powerful feature for implementing granular access control. For example, a `product.edit` permission could have a filter `{ "owner": "{_user}" }` to allow a user to only edit products they own. The target model is typically deduced from the permission name (e.g., `product.edit` implies the `product` model).
14
-
15
- ## Roles
16
-
17
- A **Role** is a collection of permissions. Instead of assigning individual permissions to each user, you assign roles, which simplifies management. A user can have multiple roles.
18
-
19
- ### Key Fields of the `role` Model:
20
-
21
- - **`name`** (string, required, unique): The name of the role (e.g., `Administrator`, `Editor`, `Viewer`).
22
- - **`permissions`** (multiple relation to `permission` model): A list of `permission` documents associated with this role. Any user assigned this role will inherit all its permissions.
23
-
24
- ## How RBAC Works
25
-
26
- 1. **Define Permissions**: Create specific `permission` documents for every action or resource you want to control. Use the `filter` field for fine-grained control.
27
- 2. **Create Roles**: Group relevant permissions into `role` documents.
28
- 3. **Assign Roles to Users**: Assign one or more `role` documents to each `user` document.
29
-
30
- When a user attempts an action (e.g., accessing an API endpoint or modifying a data entry), the system checks if the user's assigned roles grant them the necessary permissions. If a permission has a `filter`, that filter is applied to the data access query, ensuring the user only interacts with allowed subsets of data.
31
-
32
- ### User Permissions (Exceptions)
33
-
34
- The `userPermission` model allows for exceptions to the standard role-based permissions. This model can grant or revoke specific permissions for an individual user, either permanently or temporarily.
35
-
36
- - **`user`** (relation to `user` model): The user for whom the exception applies.
37
- - **`permission`** (relation to `permission` model): The specific permission being granted or revoked.
38
- - **`isGranted`** (boolean, required): `true` to grant the permission, `false` to explicitly revoke it.
39
- - **`expiresAt`** (datetime, optional): If set, the exception is temporary.
40
-
41
- This comprehensive RBAC system ensures that your `data-primals-engine` application remains secure and that users only have access to the functionalities and data they are authorized to use.
42
-
43
- **Next: Automation with Workflows**
1
+ # Roles and Permissions: Define Who Can See and Do What
2
+
3
+ The `data-primals-engine` implements a robust Role-Based Access Control (RBAC) system to manage user authorizations. This system relies on two core models: `role` and `permission`, allowing for granular control over what users can access and perform within the platform.
4
+
5
+ ## Understanding the RBAC Flow
6
+
7
+ The core idea is to assign **Roles** to your **Users**. Each **Role** is a collection of specific **Permissions**. This makes managing rights scalable and easy to understand.
8
+
9
+ ```mermaid
10
+ graph TD
11
+ subgraph "Core RBAC Flow"
12
+ U[<i class='fa fa-user'></i> User] -- "Is assigned" --> R(<i class='fa fa-users'></i> Role)
13
+ R -- "Contains" --> P1[<i class='fa fa-key'></i> Permission: product.create]
14
+ R -- "Contains" --> P2[<i class='fa fa-key'></i> Permission: product.read]
15
+ end
16
+
17
+ subgraph "Exception Mechanism"
18
+ U -- "Can have direct" --> UP(<i class='fa fa-user-check'></i> userPermission)
19
+ UP -- "Grants/Revokes" --> P3[<i class='fa fa-key'></i> Permission: product.delete]
20
+ end
21
+
22
+ style U fill:#cde,color:#000,stroke:#333,stroke-width:2px
23
+ style R fill:#dcf,color:#000,stroke:#333,stroke-width:2px
24
+ style P1 fill:#fcd,color:#000,stroke:#333,stroke-width:2px
25
+ style P2 fill:#fcd,color:#000,stroke:#333,stroke-width:2px
26
+ style P3 fill:#fcd,color:#000,stroke:#333,stroke-width:2px
27
+ style UP fill:#fec,color:#000,stroke:#333,stroke-width:2px
28
+ ```
29
+
30
+ ## How to Set Up Permissions: A Step-by-Step Guide
31
+
32
+ Here is a practical guide to setting up your access control using API calls.
33
+
34
+ ### Step 1: Define Your Permissions
35
+
36
+ A **Permission** defines a specific action. The engine uses a clear naming convention for API access, including explicit denials (`NOT_`) which override broader permissions:
37
+ - `API_ADD_DATA`: Allows creating data in *any* model.
38
+ - `API_ADD_DATA_product`: Allows creating data *only* in the `product` model.
39
+ - `API_ADD_DATA_NOT_user`: Explicitly denies creating data in the `user` model, even if `API_ADD_DATA` is granted.
40
+ - `API_SEARCH_DATA`: Allows reading data from *any* model.
41
+ - `API_SEARCH_DATA_NOT_user`: Explicitly denies searching data in the `user` model.
42
+ - `API_EDIT_DATA_order`: Allows editing data *only* in the `order` model.
43
+ - `API_EDIT_DATA_NOT_user`: Explicitly denies editing data in the `user` model.
44
+ - `API_DELETE_DATA`: Allows deleting data from *any* model.
45
+ - `API_DELETE_DATA_NOT_user`: Explicitly denies deleting data in the `user` model.
46
+ - `API_ADMIN`: A super-permission that grants all rights.
47
+
48
+ **Example: Creating permissions for a "Product Manager"**
49
+
50
+ ```javascript
51
+ await insertData('permission', [
52
+ // General permission to view products
53
+ { name: 'API_SEARCH_DATA_product', description: 'Allows viewing all products.' },
54
+ // Permission to create new products
55
+ { name: 'API_ADD_DATA_product', description: 'Allows creating new products.' },
56
+ // Permission to edit ONLY products they own
57
+ {
58
+ name: 'API_EDIT_DATA_product',
59
+ description: 'Allows editing of one\'s own products.',
60
+ filter: { "$eq": ["$createdBy", "{user._id}"] } // Dynamic filter
61
+ }
62
+ ]);
63
+ ```
64
+
65
+ ### Step 2: Create Roles and Assign Permissions
66
+
67
+ A **Role** is a collection of permissions. Instead of assigning dozens of individual permissions to each user, you assign them a role.
68
+
69
+ **Example: Creating the "Product Manager" role**
70
+
71
+ We use the `$link` operator to associate the permissions we created in the previous step.
72
+
73
+ ```javascript
74
+ await insertData('role', {
75
+ name: 'Product Manager',
76
+ permissions: {
77
+ "$link": {
78
+ // Find permissions whose name is in the following list
79
+ "$in": ["$name", [
80
+ "API_SEARCH_DATA_product",
81
+ "API_ADD_DATA_product",
82
+ "API_EDIT_DATA_product"
83
+ ]],
84
+ "_model": "permission" // Specify the model to search in
85
+ }
86
+ }
87
+ });
88
+ ```
89
+
90
+ ### Step 3: Assign a Role to a User
91
+
92
+ Finally, you assign the newly created role to a user by updating their `roles` field.
93
+
94
+ **Example: Making "John Doe" a Product Manager**
95
+
96
+ ```javascript
97
+ await editData('user',
98
+ { username: 'john.doe' }, // Filter to find the user
99
+ {
100
+ // Use $link to find the role by its name
101
+ roles: { "$link": { "name": "Product Manager", "_model": "role" } }
102
+ }
103
+ );
104
+ ```
105
+
106
+ Now, `john.doe` has all the permissions defined in the "Product Manager" role.
107
+
108
+ ## Advanced: Exceptions with `userPermission`
109
+
110
+ Sometimes, you need to grant or revoke a specific permission for a single user without creating a whole new role. The `userPermission` model is perfect for this.
111
+
112
+ ### Key Fields of the `userPermission` Model
113
+
114
+ | Attribute | Type | Description |
115
+ |:---|:---|:---|
116
+ | **`user`** | relation to `user` | The user for whom the exception applies. |
117
+ | **`permission`** | relation to `permission` | The specific permission being granted or revoked. |
118
+ | **`isGranted`** | boolean | `true` to grant the permission, `false` to explicitly revoke it. |
119
+ | **`expiresAt`** | datetime | If set, the exception is temporary and will automatically expire. |
120
+
121
+ **Example: Temporarily grant "John Doe" the right to delete products for 24 hours.**
122
+
123
+ ```javascript
124
+ // First, ensure the 'API_DELETE_DATA_product' permission exists
125
+ await insertData('permission', {
126
+ name: 'API_DELETE_DATA_product',
127
+ description: 'Allows deleting products.'
128
+ });
129
+
130
+ // Now, create the temporary exception
131
+ const expirationDate = new Date();
132
+ expirationDate.setDate(expirationDate.getDate() + 1); // Expires tomorrow
133
+
134
+ await insertData('userPermission', {
135
+ user: { "$link": { "username": "john.doe", "_model": "user" } },
136
+ permission: { "$link": { "name": "API_DELETE_DATA_product", "_model": "permission" } },
137
+ isGranted: true,
138
+ expiresAt: expirationDate.toISOString()
139
+ });
140
+ ```
141
+
142
+ This comprehensive RBAC system ensures that your application remains secure and that users only have access to the functionalities and data they are authorized to use.
143
+
144
+ **[Next: Automation with Workflows](automation-workflows)**