data-primals-engine 1.1.7 → 1.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,4 +1,9 @@
1
1
  # Data Primals Engine
2
+ [![Node.js CI](https://github.com/anonympins/data-primals-engine/actions/workflows/node.js.yml/badge.svg?branch=main)](https://github.com/anonympins/data-primals-engine/actions/workflows/node.js.yml)
3
+ ![](https://img.shields.io/npm/dw/data-primals-engine)
4
+ ![](https://img.shields.io/npm/last-update/data-primals-engine)
5
+ ![](https://img.shields.io/github/v/release/anonympins/data-primals-engine)
6
+ ![](https://img.shields.io/github/license/anonympins/data-primals-engine)
2
7
 
3
8
  **data-primals-engine** is a powerful and flexible **Node.js** backend framework designed to accelerate development of complex data-driven applications. Built on **Express.js** and **MongoDB**, it offers dynamic data modeling, automation workflows, advanced user management, and more.
4
9
 
@@ -35,7 +40,7 @@ npm i data-primals-engine
35
40
  ```
36
41
  or
37
42
  ```bash
38
- git clone https://your-repo/data-primals-engine.git
43
+ git clone https://github.com/anonympins/data-primals-engine.git
39
44
  cd data-primals-engine
40
45
  npm install
41
46
  ```
@@ -78,7 +83,7 @@ By default, the app runs on port **7633**.
78
83
 
79
84
  ## 🧠 Concepts
80
85
 
81
- ### 1. Models
86
+ ### Models
82
87
  Define schemas using JSON:
83
88
  ```json
84
89
  {
@@ -115,23 +120,15 @@ Define schemas using JSON:
115
120
  | model | Stores a model by name | – |
116
121
  | modelField | Stores a model field path | – |
117
122
 
118
- ### 2. Modules
123
+ ### Modules
119
124
  Activatable features:
120
125
  - `mongodb`, `data`, `user`, `workflow`, `file`, `assistant`, `swagger`
121
126
 
122
- ### 3. Starter Packs
127
+ ### Starter Packs
123
128
  - **E-commerce**: Products, orders, KPIs
124
129
  - **CRM**: Contacts, leads, interactions
125
130
  - **Website/blog**: Pages, posts, i18n
126
131
 
127
- ### 4. Workflows
128
- Automate business processes:
129
- - **Triggers**: `DataAdded`, `DataUpdated`, `Scheduled`, `Manual`
130
- - **Actions**: `CreateData`, `UpdateData`, `SendEmail`, `ApiCall`
131
-
132
- Example:
133
- > When a new order is created, email the customer, update stock, and notify logistics.
134
-
135
132
  ---
136
133
 
137
134
  ## 🔌 API Examples (using `curl`)
@@ -440,7 +437,7 @@ data-primals-engine/
440
437
  │ ├── engine.js // The Express engine that serves the API
441
438
  │ ├── constants.js // The inner-application constants definitions
442
439
  │ ├── packs.js // The packs that will be loaded and available with installPack() method
443
- │ ├── defaultModels.js // The default models available to import
440
+ │ ├── defaultModels.js // The default models available at startup.
444
441
  │ ├── ...
445
442
  └── server.js
446
443
  ```
@@ -473,8 +470,93 @@ A workflow is composed of two main parts: **Triggers** and **Actions**.
473
470
 
474
471
  See the details of the workflow models for more details.
475
472
 
473
+ ## ⚡ Dynamic API Endpoints
474
+ Beyond standard CRUD operations, data-primals-engine allows you to create your own custom API endpoints directly from the UI. This feature acts like a built-in serverless function environment, enabling you to write custom business logic, integrate with third-party services, or create complex data aggregations on the fly.
475
+
476
+ Your code is executed in a secure, isolated sandbox, with access to the core data functions and the incoming request context.
477
+
478
+ ### How It Works
479
+ Define an Endpoint: You create a new document in the endpoint model.
480
+ Write Your Logic: In the code field, you write the JavaScript that will be executed.
481
+ Activate: The engine automatically listens for requests on /api/actions/:path that match your endpoint's definition.
482
+
483
+ ### The endpoint Model
484
+ To create a custom endpoint, you need to define a document with the following structure:
485
+ ```json
486
+ {
487
+ "name": "GetContactPostCount",
488
+ "path": "postCount/:name",
489
+ "method": "GET",
490
+ "code": "const posts = await db.find('content', { author: { $find: { $eq: ['$lastName', request.params.name]}}}); return { postCount: posts.length };",
491
+ "isActive": true
492
+ }
493
+ ```
494
+ | Field | Type | Description |
495
+ |:---------|:--------|:---------------------------------------------------------------------|
496
+ | name | string | A descriptive name for your endpoint (e.g., "Calculate User Stats"). |
497
+ | path | string | The URL path. It can include parameters like :id. |
498
+ | method | enum | The HTTP method: GET, POST, PUT, PATCH, or DELETE. |
499
+ | code | code | The JavaScript code to execute when the endpoint is called. |
500
+ | isActive | boolean | A flag to enable or disable the endpoint without deleting it. |
476
501
  ---
477
502
 
503
+ ### The Execution Context
504
+ Your JavaScript code runs in an async context and has access to several global objects that are securely injected into the sandbox:
505
+
506
+ #### The context Object
507
+ > This object contains all the information about the incoming HTTP request.
508
+ - context.request.**body**: The parsed request body (for POST, PUT, PATCH).
509
+ - context.request.**query**: The URL query parameters as an object.
510
+ - context.request.**params**: The URL path parameters (e.g., username from /user-summary/:username).
511
+ - context.request.**headers**: The incoming request headers.
512
+
513
+ #### The db Object
514
+ > A secure API to interact with the database. All methods are async and must be awaited. They automatically operate within the authenticated user's permissions.
515
+ - await db.**create**(modelName, dataObject): Inserts a new document.
516
+ - await db.**find**(modelName, filter): Finds multiple documents. Returns an array.
517
+ - await db.**findOne**(modelName, filter): Finds a single document. Returns an object or null.
518
+ - await db.**update**(modelName, filter, updateObject): Partially updates documents matching the filter (similar to a PATCH).
519
+ - await db.**delete**(modelName, filter): Deletes documents matching the filter.
520
+
521
+ #### The logger Object
522
+ > A safe way to log messages from your script. These logs will be collected and can be returned in the API response if an error occurs, which is very useful for debugging.
523
+ - logger.**info**(...args)
524
+ - logger.**warn**(...args)
525
+ - logger.**error**(...args)
526
+
527
+ #### The env Object
528
+ > Provides access to the user-defined variables stored in the env model, not the server's process.env.
529
+ - await env.**get**(variableName): Retrieves a single environment variable's value.
530
+ - await env.**getAll**(): Retrieves all user environment variables as an object.
531
+
532
+ #### Example: Creating a User Summary Endpoint
533
+ Let's create an endpoint that fetches a user's profile and counts how many posts they have published.
534
+ 1. Create the Endpoint Document
535
+ Create a new document in the endpoint model with the following data:
536
+ ```json
537
+ {
538
+ "name": "Get User Summary",
539
+ "path": "user-summary/:username",
540
+ "method": "GET",
541
+ "isActive": true,
542
+ "code": "const { username } = context.request.params;\n\n if (!username) {\n logger.error('Username parameter is required.');\n return { success: false, error: 'Username is required.' };\n }\n\n logger.info(`Fetching summary for user: ${username}`);\n\n // Fetch the user profile using the sandboxed db API\n const userProfile = await db.findOne('userProfile', { username: username });\n\n if (!userProfile) {\n return { success: false, error: 'User not found' };\n }\n\n // Count the user's published posts\n const posts = await db.find('post', { \n authorId: userProfile._id.toString(), \n status: 'published' \n });\n\n return {\n profile: userProfile,\n publishedPosts: posts.length\n };\n});",
543
+ }
544
+ ```
545
+ 2. Call the New Endpoint
546
+ You can now call this custom endpoint like any other API route:
547
+
548
+ Expected response :
549
+ ```json
550
+ {
551
+ "profile": {
552
+ "_id": "60d0fe4f5311236168a109ca",
553
+ "username": "demo",
554
+ "bio": "A demo user profile.",
555
+ "...": "..."
556
+ },
557
+ "publishedPosts": 15
558
+ }
559
+ ```
478
560
  ## 🤝 Contributing
479
561
 
480
562
  1. Fork the repo
@@ -492,4 +574,4 @@ Distributed under the **MIT License**. See `LICENSE` file.
492
574
 
493
575
  ---
494
576
 
495
- ## 🔼 Back to Top
577
+ ## [🔼](#) Back to Top
@@ -1,16 +1,16 @@
1
- import React, { useState, useEffect, useRef, useCallback } from 'react';
1
+ import React, { useState, useEffect, useRef } from 'react';
2
2
  import { useQuery, useQueryClient } from 'react-query';
3
3
  import { useModelContext } from './contexts/ModelContext.jsx';
4
4
  import { TextField } from './Field.jsx';
5
5
  import { useAuthContext } from './contexts/AuthContext.jsx';
6
- import {conditionToApiSearchFilter, getDataAsString, getUserId} from 'data-primals-engine/data';
6
+ import { getDataAsString, getUserId } from 'data-primals-engine/data';
7
7
  import { FaEdit, FaTrash } from 'react-icons/fa';
8
8
  import Button from './Button.jsx';
9
- import {mainFieldsTypes} from "../../src/constants.js";
9
+ import { mainFieldsTypes } from "../../src/constants.js";
10
10
  import Draggable from "./Draggable.jsx";
11
- import {useTranslation} from "react-i18next";
11
+ import { useTranslation } from "react-i18next";
12
12
 
13
- const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) => {
13
+ const RelationField = ({ field, help, onFocus, onBlur, onChange, value = null }) => {
14
14
  const { models, dataByModel, setOnSuccessCallbacks, relationIds, setRelationIds } = useModelContext();
15
15
  const { name, relation: modelName } = field;
16
16
  const queryClient = useQueryClient();
@@ -19,45 +19,67 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
19
19
  const [searchValue, setSearchValue] = useState('');
20
20
  const [history, setHistory] = useState([]);
21
21
  const { me } = useAuthContext();
22
- const tr = useTranslation()
23
- const {t, i18n} = tr
22
+ const tr = useTranslation();
23
+ const { t, i18n } = tr;
24
24
  const model = models?.find(f => f.name === modelName && f._user === me?.username);
25
25
 
26
26
  // Fetch related data based on search value
27
27
  const { data: results = [], isError, refetch } = useQuery(
28
- ['api/search', model?.name, field?.name, searchValue],
28
+ // La clé de la requête inclut maintenant le filtre de relation pour une mise en cache correcte
29
+ ['api/search', model?.name, field?.name, searchValue, JSON.stringify(field.relationFilter)],
29
30
  async ({ signal }) => {
30
31
  if (!model) return [];
31
- const orFilter = [];
32
- let filter = {};
33
- if( field.relationFilter ){
34
- filter = {"$and": field.relationFilter.filter};
35
- }else if ( searchValue) {
32
+
33
+ // --- DÉBUT DE LA NOUVELLE LOGIQUE DE FILTRAGE ---
34
+
35
+ // 1. On récupère le filtre permanent défini dans le modèle.
36
+ // S'il n'y en a pas, on utilise un objet vide qui n'aura aucun effet.
37
+ const permanentFilter = field.relationFilter || {};
38
+
39
+ // 2. On construit le filtre basé sur la recherche de l'utilisateur.
40
+ const searchFilter = {};
41
+ if (searchValue) {
42
+ const orConditions = [];
43
+ // Recherche sur les champs principaux (asMain)
36
44
  model.fields.forEach(f => {
37
- if (f.asMain) {
38
- if( f.type !== 'relation' && mainFieldsTypes.includes(f.type))
39
- orFilter.push({"$regexMatch": { input: '$'+f.name, regex: searchValue}});
45
+ if (f.asMain && mainFieldsTypes.includes(f.type)) {
46
+ orConditions.push({ [f.name]: { "$regex": searchValue, "$options": "i" } });
40
47
  }
41
- })
42
- if (!orFilter.length) {
48
+ });
49
+ // Si aucun champ principal, recherche sur les champs texte
50
+ if (orConditions.length === 0) {
43
51
  model.fields.forEach(f => {
44
- if (["string", "string_t", "richtext", "url"]?.includes(f.type)) {
45
- orFilter.push({"$regexMatch": { input: '$'+f.name, regex: searchValue}});
52
+ if (["string", "string_t", "richtext", "url"].includes(f.type)) {
53
+ orConditions.push({ [f.name]: { "$regex": searchValue, "$options": "i" } });
46
54
  }
47
55
  });
48
- if (!orFilter.length) {
49
- orFilter.push({"$eq": ['_id', searchValue]});
50
- }
51
56
  }
52
- filter = {'$or':orFilter};
57
+ // Si toujours rien, on cherche sur l'ID (utile pour les développeurs)
58
+ if (orConditions.length === 0) {
59
+ orConditions.push({ "_id": searchValue });
60
+ }
61
+ searchFilter['$or'] = orConditions;
53
62
  }
63
+
64
+ // 3. On combine les deux filtres avec un opérateur $and.
65
+ // Un document devra correspondre au filtre permanent ET au filtre de recherche.
66
+ const finalFilter = {
67
+ "$and": [
68
+ permanentFilter,
69
+ searchFilter
70
+ ]
71
+ };
72
+
73
+ // --- FIN DE LA NOUVELLE LOGIQUE DE FILTRAGE ---
74
+
54
75
  const params = new URLSearchParams();
55
- params.append('_user', getUserId(me));
56
76
  params.append('model', field.relation);
57
- params.append('limit', '1000');
58
- params.append('depth', '2'); // Fetch related data with depth 2
77
+ params.append('limit', '100'); // Limite raisonnable pour les suggestions
78
+ params.append('depth', '2');
79
+
59
80
  return fetch(`/api/data/search?${params.toString()}`, {
60
- body: JSON.stringify({ filter }),
81
+ // On envoie le filtre final et complet
82
+ body: JSON.stringify({ filter: finalFilter }),
61
83
  method: 'POST',
62
84
  signal: signal,
63
85
  headers: { 'Content-Type': 'application/json' },
@@ -65,9 +87,11 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
65
87
  .then(e => e.json())
66
88
  .then(e => e.data);
67
89
  },
68
- { enabled: !!model }
90
+ { enabled: !!model && showResults } // La requête ne s'exécute que si le panneau de résultats est visible
69
91
  );
70
92
 
93
+ // ... (le reste du composant reste identique) ...
94
+
71
95
  useEffect(() => {
72
96
  setSearchValue('');
73
97
  setHistory([]);
@@ -100,7 +124,6 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
100
124
  }
101
125
  }, [value]);
102
126
 
103
- //console.log(field, value);
104
127
  const updateValue = () => {
105
128
  if (!field.multiple) {
106
129
  const v = history.find(d => d._id === value);
@@ -108,7 +131,7 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
108
131
  setSearchValue(getDataAsString(model, v, tr, models) || '');
109
132
  onChange({ name, value });
110
133
  } else {
111
- //onChange({ name, value: null });
134
+ // Ne rien faire si la valeur n'est pas dans l'historique pour éviter d'effacer
112
135
  }
113
136
  } else {
114
137
  if (Array.isArray(value)) {
@@ -133,7 +156,7 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
133
156
  return value;
134
157
  });
135
158
  } else {
136
- onChange({ name, value });
159
+ onChange({ name, value: selectedValues });
137
160
  }
138
161
  }
139
162
  setResultsVisible(false);
@@ -141,10 +164,6 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
141
164
  e.preventDefault();
142
165
  };
143
166
 
144
- useEffect(() => {
145
- if (showResults) refetch();
146
- }, [showResults]);
147
-
148
167
  const handleRemove = (element) => {
149
168
  if (!field.multiple) return;
150
169
  setSelectedValues(values => {
@@ -162,6 +181,7 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
162
181
  inputRef.current.ref.setAttribute('autocomplete', 'off');
163
182
  }
164
183
  }, [inputRef]);
184
+
165
185
  return (
166
186
  <div onFocus={onFocus} onBlur={onBlur} className="field field-relation flex flex-row flex-start flex-1">
167
187
  {help && <div className={"flex help"}>{help}</div>}
@@ -176,17 +196,15 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
176
196
  id={field.name}
177
197
  onFocus={e => {
178
198
  setResultsVisible(true);
179
- queryClient.invalidateQueries(['api/search', field.name, searchValue]);
180
199
  }}
181
200
  value={searchValue}
182
201
  onChange={e => {
183
- if (!e.target.value) onChange({ name, value: '' });
184
- else onChange({ name, value: '' });
202
+ if (!e.target.value) onChange({ name, value: null });
185
203
  setResultsVisible(true);
186
204
  setSearchValue(e.target.value);
187
205
  }}
188
206
  onBlur={(e) => {
189
- setResultsVisible(false);
207
+ setTimeout(() => setResultsVisible(false), 150); // Léger délai pour permettre le clic sur un résultat
190
208
  }}
191
209
  />
192
210
  <Button
@@ -200,11 +218,6 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
200
218
  <div className="results" onKeyDown={e => {
201
219
  if( e.key === 'Escape' )
202
220
  setResultsVisible(false);
203
- }} onBlur={e => {
204
- const t = e.target.parentNode.children[e.target.parentNode.childElementCount-1];
205
- if(t === e.target){
206
- setResultsVisible(false);
207
- }
208
221
  }} >
209
222
  {!isError &&
210
223
  (results || []).map(r => {
@@ -224,7 +237,7 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
224
237
  {field.multiple && selectedValues.length > 0 && (
225
238
  <div ref={ref} tabIndex={0} className="selected-values flex flex-border flex-row flex-no-gap flex-start flex-1">
226
239
  <Draggable items={selectedValues} renderItem={(id,i) =>{
227
- const val = history.find(f => f._id === id) || dataByModel[modelName].find(f => f._id === id);
240
+ const val = history.find(f => f._id === id) || dataByModel[modelName]?.find(f => f._id === id);
228
241
  if (!val) {
229
242
  return <div className="flex" key={id}>data non chargée<FaTrash onClick={() => handleRemove(id)} /></div>;
230
243
  }
@@ -239,7 +252,6 @@ const RelationField = ({ field, help, onFocus, onBlur, onChange, value=null }) =
239
252
  }} />
240
253
  </div>
241
254
  )}
242
-
243
255
  </div>
244
256
  );
245
257
  };
@@ -523,7 +523,7 @@ export const websiteTranslations = {
523
523
  "prez1.5": "Support des traductions et langues",
524
524
  "links.deploy_api": "Déployez votre API",
525
525
  "links.documentation": "Documentation",
526
- "links.demo": "Essayez maintenant !",
526
+ "links.demo": "Démarrer la démo",
527
527
  "filterstringfield.placeholder.regex": 'Expression régulière'
528
528
 
529
529
  }
@@ -1483,7 +1483,7 @@ export const websiteTranslations = {
1483
1483
  "dataimporter.title": "Import data into {{model}}",
1484
1484
  "dataimporter.hasCsvHeaders": "Does the CSV file contain a header row?",
1485
1485
  "dataimporter.info": "Import your JSON or CSV file containing the required data for your model. Your data will be copied to your personal database. The transfer rate is {{constante}}",
1486
- "links.demo": "Try the demo!",
1486
+ "links.demo": "Start the demo",
1487
1487
  "filterstringfield.placeholder.regex": 'Regular expression',
1488
1488
  "cgu.h1": "Terms of Use (TOU)",
1489
1489
  "cgu.h1.1": "1. Introduction",
@@ -2887,7 +2887,7 @@ export const websiteTranslations = {
2887
2887
  "dataimporter.title": "Importar datos a {{model}}",
2888
2888
  "dataimporter.hasCsvHeaders": "¿El archivo CSV contiene una fila de encabezado?",
2889
2889
  "dataimporter.info": "Importa tu archivo JSON o CSV con los datos necesarios para tu modelo. Tus datos se copiarán a tu base de datos personal. La velocidad de transferencia es {{constante}}",
2890
- "links.demo": "¡Prueba la demo!",
2890
+ "links.demo": "Iniciar la demostración",
2891
2891
  "filterstringfield.placeholder.regex": 'Expresión regular',
2892
2892
  "cgu.h1": "Condiciones Generales de Uso (CGU)",
2893
2893
  "cgu.h1.1": "1. Introducción",
@@ -4298,7 +4298,7 @@ export const websiteTranslations = {
4298
4298
  "dataimporter.title": "Importar dados para {{model}}",
4299
4299
  "dataimporter.hasCsvHeaders": "O ficheiro CSV contém uma linha de cabeçalho?",
4300
4300
  "dataimporter.info": "Importe o seu ficheiro JSON ou CSV contendo os dados necessários para o seu modelo. Os seus dados serão copiados para a sua base de dados pessoal. A taxa de transferência é {{constante}}",
4301
- "links.demo": "Experimente a demonstração!",
4301
+ "links.demo": "Iniciar a demonstração",
4302
4302
  "filterstringfield.placeholder.regex": 'Expressão regular',
4303
4303
  "cgu.h1": "Termos e Condições Gerais de Uso (TCGU)",
4304
4304
  "cgu.h1.1": "1. Introdução",
@@ -5681,7 +5681,7 @@ export const websiteTranslations = {
5681
5681
  "dataimporter.title": "Daten in {{model}} importieren",
5682
5682
  "dataimporter.hasCsvHeaders": "Enthält die CSV-Datei eine Kopfzeile?",
5683
5683
  "dataimporter.info": "Importieren Sie Ihre JSON- oder CSV-Datei mit den benötigten Daten für Ihr Modell. Ihre Daten werden in Ihre persönliche Datenbank kopiert. Die Übertragungsrate beträgt {{constante}}",
5684
- "links.demo": "Probieren Sie die Demo aus!",
5684
+ "links.demo": "Demo starten",
5685
5685
  "filterstringfield.placeholder.regex": 'Regulärer Ausdruck',
5686
5686
  "cgu.h1": "Nutzungsbedingungen (TOU)",
5687
5687
  "cgu.h1.1": "1. Einleitung",
@@ -7087,7 +7087,7 @@ export const websiteTranslations = {
7087
7087
  "dataimporter.title": "Importa dati in {{model}}",
7088
7088
  "dataimporter.hasCsvHeaders": "Il file CSV contiene una riga di intestazione?",
7089
7089
  "dataimporter.info": "Importa il tuo file JSON o CSV contenente i dati necessari per il tuo modello. I tuoi dati verranno copiati nel tuo database personale. La velocità di trasferimento è {{constante}}",
7090
- "links.demo": "Prova la demo!",
7090
+ "links.demo": "Avvia la demo",
7091
7091
  "filterstringfield.placeholder.regex": 'Espressione regolare',
7092
7092
  "cgu.h1": "Termini di Utilizzo (TOU)",
7093
7093
  "cgu.h1.1": "1. Introduzione",
@@ -8483,7 +8483,7 @@ export const websiteTranslations = {
8483
8483
  "dataimporter.title": "Importovat data do {{model}}",
8484
8484
  "dataimporter.hasCsvHeaders": "Obsahuje soubor CSV řádek záhlaví?",
8485
8485
  "dataimporter.info": "Importujte soubor JSON nebo CSV obsahující požadovaná data pro váš model. Vaše data budou zkopírována do vaší osobní databáze. Přenosová rychlost je {{constante}}",
8486
- "links.demo": "Vyzkoušejte demo!",
8486
+ "links.demo": "Spustit demo",
8487
8487
  "filterstringfield.placeholder.regex": 'Regulární výraz',
8488
8488
  "cgu.h1": "Podmínky použití (TOU)",
8489
8489
  "cgu.h1.1": "1. Úvod",
@@ -9887,7 +9887,7 @@ export const websiteTranslations = {
9887
9887
  "dataimporter.title": "Импортировать данные в {{model}}",
9888
9888
  "dataimporter.hasCsvHeaders": "Содержит ли CSV-файл строку заголовка?",
9889
9889
  "dataimporter.info": "Импортируйте файл JSON или CSV, содержащий необходимые данные для вашей модели. Ваши данные будут скопированы в вашу личную базу данных. Скорость передачи составляет {{constante}}",
9890
- "links.demo": "Попробуйте демо!",
9890
+ "links.demo": "Запустить демо",
9891
9891
  "filterstringfield.placeholder.regex": 'Регулярное выражение',
9892
9892
  "cgu.h1": "Условия использования (TOU)",
9893
9893
  "cgu.h1.1": "1. Введение",
@@ -11311,7 +11311,7 @@ export const websiteTranslations = {
11311
11311
  "dataimporter.title": "استيراد البيانات إلى {{model}}",
11312
11312
  "dataimporter.hasCsvHeaders": "هل يحتوي ملف CSV على صف رأس؟",
11313
11313
  "dataimporter.info": "استورد ملف JSON أو CSV الذي يحتوي على البيانات المطلوبة لنموذجك. سيتم نسخ بياناتك إلى قاعدة بياناتك الشخصية. معدل النقل ثابت ({{constante}}).",
11314
- "links.demo": "جرب النسخة التجريبية!",
11314
+ "links.demo": "ابدأ العرض التوضيحي",
11315
11315
  "filterstringfield.placeholder.regex": 'تعبير عادي',
11316
11316
  "cgu.h1": "الشروط العامة للاستخدام (GTU)",
11317
11317
  "cgu.h1.1": "1. المقدمة",
@@ -12711,7 +12711,7 @@ export const websiteTranslations = {
12711
12711
  "dataimporter.title": "Importera data till {{model}}",
12712
12712
  "dataimporter.hasCsvHeaders": "Innehåller CSV-filen en rubrikrad?",
12713
12713
  "dataimporter.info": "Importera din JSON- eller CSV-fil som innehåller den data som krävs för din modell. Dina data kommer att kopieras till din personliga databas. Överföringshastigheten är {{constante}}",
12714
- "links.demo": "Testa demon!",
12714
+ "links.demo": "Starta demon",
12715
12715
  "filterstringfield.placeholder.regex": 'Reguljärt uttryck',
12716
12716
  "cgu.h1": "Användarvillkor (Användarvillkor)",
12717
12717
  "cgu.h1.1": "1. Introduktion",
@@ -14112,7 +14112,7 @@ export const websiteTranslations = {
14112
14112
  "dataimporter.title": "Εισαγωγή δεδομένων στο {{model}}",
14113
14113
  "dataimporter.hasCsvHeaders": "Περιέχει το αρχείο CSV μια γραμμή κεφαλίδας;",
14114
14114
  "dataimporter.info": "Εισαγάγετε το αρχείο JSON που περιέχει τα απαιτούμενα δεδομένα για το μοντέλο σας. Τα δεδομένα σας θα αντιγραφούν στην προσωπική σας βάση δεδομένων. Ο ρυθμός μεταφοράς είναι {{constante}}",
14115
- "links.demo": "Δοκιμάστε την επίδειξη!",
14115
+ "links.demo": "Έναρξη της επίδειξης",
14116
14116
  "filterstringfield.placeholder.regex": "Κανονική έκφραση",
14117
14117
  "cgu.h1": "Γενικοί όροι χρήσης (GTU)",
14118
14118
  "cgu.h1.1": "1. Εισαγωγή",
@@ -14561,6 +14561,7 @@ export const websiteTranslations = {
14561
14561
  },
14562
14562
  fa: {
14563
14563
  "translation": {
14564
+ "links.demo": "شروع نسخهٔ نمایشی",
14564
14565
  "cb.searchOperator": "جستجوی یک عملگر...",
14565
14566
  "cb.changeOperator": "تغییر عملگر",
14566
14567
  "cb.deleteBlock": "حذف این بلوک",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "data-primals-engine",
3
- "version": "1.1.7",
3
+ "version": "1.1.8",
4
4
  "description": "data-primals-engine is the package responsible from handling large amount of data in a practical and performant way. It can handle large amount of data in a practical and performant way. It can also get workflow models working (for automation), and fully supports internationalisation.",
5
5
  "main": "src/engine.js",
6
6
  "type": "module",
@@ -43,9 +43,9 @@
43
43
  "./*": "./src/*.js"
44
44
  },
45
45
  "peerDependencies": {
46
+ "express": "^5.1.0",
46
47
  "react": ">=18.0.0",
47
- "react-query": ">=3.0.0",
48
- "express": "^5.1.0"
48
+ "react-query": ">=3.0.0"
49
49
  },
50
50
  "dependencies": {
51
51
  "@langchain/core": "^0.3.66",
@@ -69,6 +69,7 @@
69
69
  "express-rate-limit": "^8.0.1",
70
70
  "express-session": "^1.18.2",
71
71
  "i18next-browser-languagedetector": "^8.2.0",
72
+ "isolated-vm": "^4.7.2",
72
73
  "juice": "^11.0.1",
73
74
  "mathjs": "^14.6.0",
74
75
  "mongodb": "^6.18.0",
@@ -88,7 +89,6 @@
88
89
  "tar": "^7.4.3",
89
90
  "uniqid": "^5.4.0",
90
91
  "vitest": "^3.2.4",
91
- "isolated-vm": "^4.7.2",
92
92
  "yaml": "^2.8.0"
93
93
  },
94
94
  "devDependencies": {
@@ -102,13 +102,12 @@
102
102
  "client"
103
103
  ],
104
104
  "keywords": [
105
- "mongodb",
106
- "data",
107
- "primals",
105
+ "data-driven engine",
106
+ "headless CMS",
107
+ "backend",
108
108
  "automation",
109
- "aws",
110
- "bucket",
111
- "S3"
109
+ "AWS S3",
110
+ "MongoDB"
112
111
  ],
113
112
  "author": "anonympins",
114
113
  "license": "MIT",
package/src/core.js CHANGED
@@ -340,12 +340,13 @@ export const event_off = (name, callback) => {
340
340
  };
341
341
 
342
342
 
343
- export function slugify(str) {
343
+ export function slugify(str,replacer='-', replaceUnicode=false) {
344
344
  str = str.replace(/^\s+|\s+$/g, ''); // trim leading/trailing white space
345
345
  str = str.toLowerCase(); // convert string to lowercase
346
- str = str.replace(/[^a-z0-9 -]/g, '') // remove any non-alphanumeric characters
347
- .replace(/\s+/g, '-') // replace spaces with hyphens
348
- .replace(/-+/g, '-'); // remove consecutive hyphens
346
+ if( replaceUnicode)
347
+ str = str.replace(/[^a-z0-9 -]/g, ''); // remove any non-alphanumeric characters
348
+ str = str.replace(/\s+/g, replacer) // replace spaces with hyphens
349
+ .replace(/-+/g, replacer); // remove consecutive hyphens
349
350
  return str;
350
351
  }
351
352
 
@@ -388,4 +389,8 @@ export function object_equals( x, y ) {
388
389
  // allows x[ p ] to be set to undefined
389
390
 
390
391
  return true;
391
- }
392
+ }
393
+
394
+ export const isValidPath = (path) =>{
395
+ return /^(?:[a-z]:)?[\/\\]{0,2}(?:[.\/\\ ](?![.\/\\\n])|[^<>:"|?*.\/\\ \n])+$/gmi.test(path);
396
+ }
package/src/engine.js CHANGED
@@ -41,17 +41,35 @@ const secureContext = tls.createSecureContext({
41
41
  ca: caFile, cert: certFile, key: keyFile
42
42
  });
43
43
 
44
+ export const dbUrl = process.env.CI ? 'mongodb://mongodb:27017' : (process.env.MONGO_DB_URL || 'mongodb://127.0.0.1:27017');
45
+
44
46
  const isTlsActive = !(!process.env.TLS || ["0", "false"].includes(process.env.TLS.toLowerCase()));
45
47
 
46
- // Connection URL
47
- export const dbUrl = process.env.CI ? 'mongodb://mongodb:27017' : (process.env.MONGO_DB_URL || 'mongodb://127.0.0.1:27017');
48
- export const MongoClient = new InternalMongoClient(dbUrl, {
49
- maxPoolSize: databasePoolSize,
50
- tls: isTlsActive,
51
- secureContext,
52
- tlsAllowInvalidCertificates,
53
- tlsAllowInvalidHostnames
54
- });
48
+ const clientOptions = {
49
+ maxPoolSize: databasePoolSize
50
+ };
51
+ // On ajoute les options TLS si elles sont activées
52
+ if (isTlsActive) {
53
+ clientOptions.tls = true;
54
+ // Chemin vers le certificat de l'autorité de certification (pour faire confiance au serveur)
55
+ if (process.env.CA_CERT) {
56
+ clientOptions.tlsCAFile = process.env.CA_CERT;
57
+ }
58
+ // Chemin vers le certificat et la clé du CLIENT (pour que le serveur vous fasse confiance)
59
+ if (process.env.CERT_KEY) {
60
+ clientOptions.tlsCertificateKeyFile = process.env.CERT_KEY;
61
+ }
62
+ // Options pour le développement (à utiliser avec prudence)
63
+ if (tlsAllowInvalidCertificates) {
64
+ clientOptions.tlsAllowInvalidCertificates = true;
65
+ }
66
+ if (tlsAllowInvalidHostnames) {
67
+ clientOptions.tlsAllowInvalidHostnames = true;
68
+ }
69
+ }
70
+
71
+ export const MongoClient = new InternalMongoClient(dbUrl, clientOptions);
72
+
55
73
 
56
74
  // Database Name
57
75
  export const MongoDatabase = MongoClient.db(dbName);
@@ -112,7 +112,7 @@ function sanitize(target, options = {}) {
112
112
  export function middleware(options = {}) {
113
113
  const hasOnSanitize = typeof options.onSanitize === 'function';
114
114
  return function (req, res, next) {
115
- ['body', 'params', 'headers', 'query'].forEach(function (key) {
115
+ ['body', 'fields', 'params', 'headers', 'query'].forEach(function (key) {
116
116
  if (req[key]) {
117
117
  // The _sanitize function mutates the req[key] object in-place.
118
118
  // We only need to capture whether it was sanitized to call the hook.
@@ -3,7 +3,7 @@ import {BSON, ObjectId} from "mongodb";
3
3
  import * as util from 'node:util';
4
4
  import {promisify} from 'node:util';
5
5
  import crypto from "node:crypto";
6
- import {exec} from 'node:child_process';
6
+ import {exec,execFile} from 'node:child_process';
7
7
  import sanitizeHtml from 'sanitize-html';
8
8
  import * as tar from "tar";
9
9
  import process from "node:process";
@@ -105,6 +105,7 @@ const delay = ms => new Promise(res => setTimeout(res, ms));
105
105
 
106
106
  const getBackupDir = () => process.env.BACKUP_DIR || './backups'; // Répertoire de stockage des sauvegardes
107
107
  const execAsync = promisify(exec);
108
+ const execFileAsync = promisify(execFile);
108
109
 
109
110
  let importJobs = {};
110
111
  const IMPORT_CHUNK_SIZE = 100; // Nombre d'enregistrements à traiter par lot
@@ -3489,6 +3490,24 @@ async function insertAndResolveRelations(doc, model, collection, me, idMap) {
3489
3490
  return existingDoc._id;
3490
3491
  }
3491
3492
 
3493
+ for (const field of model.fields) {
3494
+ if (field.type === 'relation' && field.relationFilter && docToProcess[field.name]) {
3495
+ const relatedIds = Array.isArray(docToProcess[field.name]) ? docToProcess[field.name] : [docToProcess[field.name]];
3496
+ for (const id of relatedIds) {
3497
+ const targetCollection = await getCollectionForUser(me, field.targetModel);
3498
+ const validationQuery = {
3499
+ _id: new ObjectId(id), // L'ID doit correspondre
3500
+ ...field.relationFilter // ET le document doit respecter le filtre
3501
+ };
3502
+ const relatedDoc = await targetCollection.findOne(validationQuery);
3503
+ if (!relatedDoc) {
3504
+ // Si on ne trouve rien, c'est que l'ID est invalide ou ne respecte pas le filtre.
3505
+ throw new Error(`La valeur '${id}' pour le champ '${field.name}' ne respecte pas le filtre de relation défini.`);
3506
+ }
3507
+ }
3508
+ }
3509
+ }
3510
+
3492
3511
  // Insertion en conservant éventuellement l'ID original
3493
3512
  const result = docToProcess._id
3494
3513
  ? await collection.insertOne(docToProcess)
@@ -3708,6 +3727,33 @@ const internalEditOrPatchData = async (modelName, filter, data, files, user, isP
3708
3727
  }
3709
3728
  }
3710
3729
 
3730
+ for (const field of model.fields) {
3731
+ // On ne vérifie que si un champ de relation avec un filtre est en cours de modification.
3732
+ if (field.type === 'relation' && field.relationFilter && updateData[field.name] !== undefined) {
3733
+ const relatedIds = Array.isArray(updateData[field.name])
3734
+ ? updateData[field.name]
3735
+ : (updateData[field.name] ? [updateData[field.name]] : []);
3736
+
3737
+ for (const id of relatedIds) {
3738
+ if (!id || !isObjectId(id)) continue; // Ignorer les valeurs null/invalides
3739
+
3740
+ const targetCollection = await getCollectionForUser(user, field.relation);
3741
+
3742
+ const validationQuery = {
3743
+ _id: new ObjectId(id),
3744
+ ...field.relationFilter
3745
+ };
3746
+
3747
+ const relatedDoc = await targetCollection.findOne(validationQuery);
3748
+
3749
+ if (!relatedDoc) {
3750
+ // Si on ne trouve rien, c'est que l'ID est invalide ou ne respecte pas le filtre.
3751
+ throw new Error(`La valeur '${id}' pour le champ '${field.name}' ne respecte pas le filtre de relation défini.`);
3752
+ }
3753
+ }
3754
+ }
3755
+ }
3756
+
3711
3757
  // 8. Calcul du nouveau hash et préparation des données finales
3712
3758
  const finalStateForHash = { ...existingDocs[0], ...updateData };
3713
3759
  const newHash = getFieldValueHash(model, finalStateForHash);
@@ -5172,6 +5218,7 @@ const handleFields = async (model, data, user, isRecursiveCall = false) => {
5172
5218
  }
5173
5219
  };
5174
5220
 
5221
+ let restoreRequests = {};
5175
5222
  export const validateRestoreRequest = (username, token) => {
5176
5223
  const request = restoreRequests[username];
5177
5224
  if (!request) {
@@ -5202,20 +5249,21 @@ export const loadFromDump = async (user, options = {}) => {
5202
5249
  throw new Error("No encryption key found for this user. Cannot restore.");
5203
5250
  }
5204
5251
 
5252
+ const userId = getObjectHash({user: user.username});
5205
5253
  // --- La logique pour trouver le fichier de backup (local ou S3) reste la même ---
5206
5254
  // (le code existant pour trouver/télécharger le backupFilePath va ici)
5207
5255
  // ...
5208
5256
  let backupFilePath; // Assurez-vous que cette variable est bien définie avec le chemin du fichier .tar.gz
5209
5257
  // Exemple simplifié :
5210
5258
  const backupDir = getBackupDir();
5211
- const backupFilenameRegex = new RegExp(`^backup_${user.username}_(\\d+)\\.tar\\.gz$`);
5259
+ const backupFilenameRegex = new RegExp(`^backup_${userId}_(\\d+)\\.tar\\.gz$`);
5212
5260
  const backupFiles = fs.readdirSync(backupDir).filter(filename => backupFilenameRegex.test(filename));
5213
5261
  if (backupFiles.length === 0) throw new Error(`Aucun fichier de sauvegarde local trouvé pour l'utilisateur ${user.username}.`);
5214
5262
  const latestBackupFile = backupFiles.sort((a, b) => parseInt(b.match(backupFilenameRegex)[1], 10) - parseInt(a.match(backupFilenameRegex)[1], 10))[0];
5215
5263
  backupFilePath = path.join(backupDir, latestBackupFile);
5216
5264
  // --- Fin de la logique de recherche de fichier ---
5217
5265
 
5218
- const tmpRestoreDir = path.join(backupDir, `tmp_restore_${user.username}_${Date.now()}`);
5266
+ const tmpRestoreDir = path.join(backupDir, `tmp_restore_${userId}_${Date.now()}`);
5219
5267
 
5220
5268
  try {
5221
5269
  await runCryptoWorkerTask('decrypt', { filePath: backupFilePath, password: encryptedKey });
@@ -5252,17 +5300,25 @@ export const loadFromDump = async (user, options = {}) => {
5252
5300
  }
5253
5301
 
5254
5302
  let command;
5303
+ const args = [
5304
+ '--uri', dbUrl,
5305
+ '--db', dbName
5306
+ ];
5307
+
5255
5308
  if (modelsOnly) {
5256
- // Restaure uniquement la collection des modèles
5257
- command = `mongorestore --uri="${dbUrl}" --db=${dbName} --nsInclude="${dbName}.models" "${restoreSourceDir}"`;
5309
+ args.push('--nsInclude', `${dbName}.models`);
5258
5310
  } else {
5259
- // Restauration complète
5260
- command = `mongorestore --uri="${dbUrl}" --db=${dbName} --nsInclude="${dbName}.datas" --nsInclude="${dbName}.models" "${restoreSourceDir}"`;
5311
+ // mongorestore accepte plusieurs fois l'option --nsInclude
5312
+ args.push('--nsInclude', `${dbName}.datas`);
5313
+ args.push('--nsInclude', `${dbName}.models`);
5261
5314
  }
5315
+ // Le répertoire source est le dernier argument
5316
+ args.push(restoreSourceDir);
5262
5317
 
5263
- logger.info(`[${action}] Executing restore command: ${command}`);
5264
- await execAsync(command);
5265
5318
 
5319
+ logger.info(`[${action}] Executing restore command: ${command}`);
5320
+ await execFileAsync('mongorestore', args);
5321
+
5266
5322
  // --- Tâches Post-Restauration ---
5267
5323
  await scheduleAlerts();
5268
5324
  await scheduleWorkflowTriggers();
@@ -5301,121 +5357,90 @@ const readKeyFromFile = (user) => {
5301
5357
  return null;
5302
5358
  };
5303
5359
 
5360
+ // C:/Dev/data-primals-engine/src/modules/data.js
5361
+
5304
5362
  export const dumpUserData = async (user) => {
5305
- // Déterminer la clé de chiffrement
5306
- // Pour cet exemple, on simule la config S3. Remplace par la vraie récupération.
5307
- const s3Config = user.configS3; // Supposons que l'objet 'user' passé contient déjà 'configS3'
5363
+ const s3Config = user.configS3;
5308
5364
  const backupDir = getBackupDir();
5365
+ const userId = getObjectHash({ user: user.username });
5366
+ const backupFilename = `backup_${userId}`;
5367
+ const timestamp = Date.now();
5368
+
5369
+ // Déclarer les chemins ici pour qu'ils soient accessibles dans tout le scope de la fonction
5370
+ const localTempDumpDir = path.join(backupDir, `${backupFilename}_${timestamp}_temp`);
5371
+ const finalArchiveName = `${backupFilename}_${timestamp}.tar.gz`;
5372
+ const localArchivePath = path.join(backupDir, finalArchiveName);
5309
5373
 
5310
5374
  let encryptedKey = readKeyFromFile(user);
5311
5375
  if (!encryptedKey) {
5312
5376
  encryptedKey = generateAndStoreKey(user);
5313
- logger.info('Encryption key générée.');
5314
- } else {
5315
- logger.info('Encryption key chargée.');
5316
5377
  }
5317
5378
 
5318
5379
  try {
5319
- // Déterminer la fréquence de la sauvegarde
5320
5380
  const backupFrequency = await engine.userProvider.getBackupFrequency(user);
5321
-
5322
5381
  logger.info(`Fréquence de sauvegarde : ${backupFrequency}.`);
5323
5382
 
5324
- // Définir le nom du fichier de sauvegarde et les chemins
5325
- const backupFilename = `backup_${user.username}`; //nom corrigé
5326
- const backupFileBasePath = path.join(backupDir, backupFilename); //chemin corrigé
5327
- const backupFilePath = `${backupFileBasePath}_${Date.now()}`;
5328
- const backupFilenameBase = `backup_${user.username}`;
5329
- const timestamp = Date.now();
5330
- const localTempDumpDir = path.join(backupDir, `${backupFilenameBase}_${timestamp}_temp`); // Répertoire temporaire local
5331
- const finalArchiveName = `${backupFilenameBase}_${timestamp}.tar.gz`; // Nom de l'archive finale
5332
- const localArchivePath = path.join(backupDir, finalArchiveName); // Chemin
5333
-
5334
- logger.info(`Chemin du fichier de sauvegarde : ${backupFilePath}.`);
5335
-
5336
- // Ajoute les filtres sur les collections et l'utilisateur
5337
5383
  const collections = await MongoDatabase.listCollections().toArray();
5338
- let query;
5339
- let col;
5340
5384
  for (const collection of collections) {
5341
-
5342
- const colls = [await getUserCollectionName(user), 'models'];
5343
- if( colls.includes(collection.name) ){
5344
-
5345
- // Exécuter mongodump avec les filtres appropriés
5346
- let command = `mongodump --uri="${dbUrl}" --db=${dbName} --out=${localTempDumpDir} `;
5347
-
5348
- query = JSON.stringify({_user : user.username }); // Filtrer par _user
5349
- col = ` --collection=${collection.name}`;
5350
- command += `${col} --query ${JSON.stringify(query)}`;
5351
-
5352
- logger.info(`Exécution de la commande : ${command}`);
5353
- await execAsync(command);
5385
+ const collsToBackup = [await getUserCollectionName(user), 'models'];
5386
+ if (collsToBackup.includes(collection.name)) {
5387
+ const query = { _user: user.username };
5388
+ const args = [
5389
+ '--uri', dbUrl,
5390
+ '--db', dbName,
5391
+ '--out', localTempDumpDir,
5392
+ '--collection', collection.name,
5393
+ '--query', JSON.stringify(query)
5394
+ ];
5395
+ logger.info(`Exécution de la commande : mongodump ${args.join(' ')}`);
5396
+ await execFileAsync('mongodump', args);
5354
5397
  }
5355
-
5356
5398
  }
5357
5399
 
5358
-
5359
- // Zipper le contenu du répertoire de dump temporaire
5360
- // Le répertoire zippé sera `localTempDumpDir/${dbName}` si mongodump a créé ce sous-dossier
5361
- const dumpSourceDir = path.join(localTempDumpDir, dbName); // Le répertoire que mongodump a créé
5400
+ const dumpSourceDir = path.join(localTempDumpDir, dbName);
5362
5401
  if (fs.existsSync(dumpSourceDir)) {
5363
5402
  await tar.create({ gzip: true, file: localArchivePath, C: localTempDumpDir }, [dbName]);
5364
5403
  logger.info(`Archive de sauvegarde locale créée : ${localArchivePath}`);
5365
5404
  } else {
5366
- throw new Error(`Le répertoire de dump ${dumpSourceDir} n'a pas été trouvé après mongodump.`);
5405
+ logger.warn(`Le répertoire de dump ${dumpSourceDir} était vide. Aucune archive n'a été créée.`);
5406
+ // On s'arrête ici car il n'y a rien à traiter
5407
+ return Promise.resolve();
5367
5408
  }
5368
5409
 
5369
- // Chiffrer le fichier local avant l'upload si tu le souhaites (optionnel si le bucket S3 est déjà chiffré)
5370
- await encryptFile(localArchivePath, encryptedKey); // Ta fonction existante
5410
+ await encryptFile(localArchivePath, encryptedKey);
5371
5411
 
5372
5412
  if (s3Config && s3Config.bucketName) {
5373
- logger.info(`Téléversement de la sauvegarde vers S3 pour l'utilisateur ${user.username}. Bucket: ${s3Config.bucketName}`);
5374
5413
  await uploadToS3(s3Config, localArchivePath, finalArchiveName);
5375
- logger.info(`Sauvegarde téléversée avec succès vers S3 pour ${user.username}.`);
5376
- // Optionnel: supprimer l'archive locale après l'upload S3 réussi
5377
- fs.unlinkSync(localArchivePath);
5378
- logger.info(`Archive locale ${localArchivePath} supprimée après l'upload S3.`);
5414
+ fs.unlinkSync(localArchivePath); // Supprime l'archive locale après l'upload
5379
5415
  } else {
5380
- logger.info(`Aucune configuration S3 trouvée pour ${user.username}. La sauvegarde reste locale : ${localArchivePath}.`);
5381
- // Si pas de S3, le fichier chiffré localement (si encryptFile a été appelé) reste.
5382
- // Si tu ne chiffres pas localement et pas de S3, c'est l'archive .tar.gz qui reste.
5383
- }
5384
-
5385
- // Supprimer le répertoire de dump temporaire
5386
- if (fs.existsSync(localTempDumpDir)) {
5387
- fs.rmSync(localTempDumpDir, { recursive: true, force: true });
5388
- logger.info(`Répertoire de dump temporaire ${localTempDumpDir} supprimé.`);
5416
+ logger.info(`Aucune configuration S3 trouvée. La sauvegarde reste locale : ${localArchivePath}.`);
5389
5417
  }
5390
5418
 
5391
5419
  logger.info(`Sauvegarde réussie pour l'utilisateur ${user.username}.`);
5392
- // La gestion de la rotation des sauvegardes devra être adaptée
5393
- // si les sauvegardes sont sur S3 (lister depuis S3, supprimer depuis S3).
5394
- await manageBackupRotation(user, backupFrequency, s3Config); // Passer s3Config
5420
+ await manageBackupRotation(user, backupFrequency, s3Config);
5395
5421
 
5396
- return Promise.resolve();
5397
5422
  } catch (error) {
5398
5423
  logger.error(`Erreur lors de la sauvegarde pour l'utilisateur ${user.username}:`, error);
5399
- // Nettoyage en cas d'erreur
5400
- const localTempDumpDir = path.join(backupDir, `backup_${user.username}_${Date.now()}_temp`); // Recalculer pour être sûr
5401
- if (fs.existsSync(localTempDumpDir)) {
5402
- fs.rmSync(localTempDumpDir, { recursive: true, force: true });
5403
- }
5404
- const localArchivePath = path.join(backupDir, `backup_${user.username}_${Date.now()}.tar.gz`); // Recalculer
5424
+ // Nettoyage de l'archive si elle a été créée avant l'erreur
5405
5425
  if (fs.existsSync(localArchivePath)) {
5406
5426
  fs.unlinkSync(localArchivePath);
5407
5427
  }
5408
- throw error; // Relancer l'erreur
5428
+ throw error; // Relancer l'erreur pour que l'appelant soit informé
5429
+ } finally {
5430
+ // --- NETTOYAGE GARANTI ---
5431
+ // Ce bloc s'exécute toujours, que la sauvegarde réussisse ou échoue.
5432
+ if (fs.existsSync(localTempDumpDir)) {
5433
+ fs.rmSync(localTempDumpDir, { recursive: true, force: true });
5434
+ logger.info(`Répertoire de dump temporaire ${localTempDumpDir} supprimé.`);
5435
+ }
5409
5436
  }
5410
- }
5411
-
5437
+ };
5412
5438
  async function manageBackupRotation(user, backupFrequency, s3Config = null) { // Accepter s3Config
5413
- const userId = user.username; // ou user._id selon ce que tu utilises dans les noms de fichiers
5414
-
5439
+ const userId = getObjectHash({user:user.username});
5415
5440
  let filesToManage = [];
5416
5441
 
5417
5442
  if (s3Config && s3Config.bucketName) {
5418
- logger.info(`Gestion de la rotation des sauvegardes S3 pour ${userId}.`);
5443
+ logger.info(`Gestion de la rotation des sauvegardes S3 pour ${userid}.`);
5419
5444
  const s3Backups = await listS3Backups(s3Config);
5420
5445
  // Filtrer pour ne garder que les backups de cet utilisateur et trier
5421
5446
  filesToManage = s3Backups
@@ -86,7 +86,7 @@ afterAll(async () => {
86
86
  fs.readdirSync(backupDir).forEach(file => {
87
87
  fs.unlinkSync(path.join(backupDir, file));
88
88
  });
89
- // Optional: fs.rmdirSync(backupDir); // Remove the directory itself
89
+ fs.rmdirSync(backupDir); // Remove the directory itself
90
90
  }
91
91
  });
92
92
 
@@ -132,5 +132,7 @@ describe('Data Backup and Restore Integration', () => {
132
132
  expect(docAfterRestore.testField).toBe('Initial Value');
133
133
  expect(docAfterRestore.optionalField).toBe(123);
134
134
 
135
+ console.log("Tests passed for backup and restore");
136
+
135
137
  }, 15000); // Timeout augmenté pour les opérations de fichiers
136
138
  });
@@ -1,13 +1,13 @@
1
1
  // __tests__/data.integration.test.js
2
2
  import { ObjectId } from 'mongodb';
3
- import {expect, describe, it, beforeEach, beforeAll} from 'vitest';
3
+ import {expect, describe, it, beforeEach, beforeAll, afterAll} from 'vitest';
4
4
  import { Config } from '../src/config.js';
5
5
 
6
6
  import {
7
7
  insertData,
8
8
  editData,
9
9
  deleteData,
10
- searchData, installPack
10
+ searchData, installPack, deleteModels, createModel
11
11
  } from 'data-primals-engine/modules/data';
12
12
 
13
13
  import {
@@ -120,11 +120,12 @@ async function setupTestContext() {
120
120
  };
121
121
  }
122
122
 
123
+ let engine;
123
124
  describe('Intégration des fonctions CRUD de données avec validation complète', () => {
124
125
 
125
126
  beforeAll(async () =>{
126
127
  Config.Set("modules", ["mongodb", "data", "file", "bucket", "workflow","user", "assistant"]);
127
- await initEngine();
128
+ engine = await initEngine();
128
129
 
129
130
  })
130
131
 
@@ -788,5 +789,103 @@ describe('Intégration des fonctions CRUD de données avec validation complète'
788
789
  expect(validModel).not.toBeNull();
789
790
  });
790
791
  });
792
+ // In test/data.integration.test.js
793
+
794
+ describe('relationFilter validation', () => {
795
+ let user;
796
+ let activeProductId, inactiveProductId;
797
+
798
+ // Model names are kept in French to match the database, but variables are in English.
799
+ const productModel = {
800
+ name: 'produitTestFiltre',
801
+ description: '',
802
+ fields: [
803
+ { name: 'name', type: 'string' },
804
+ { name: 'actif', type: 'boolean', default: false }
805
+ ]
806
+ };
807
+ const orderModel = {
808
+ name: 'commandeTestFiltre',
809
+ description: '',
810
+ fields: [
811
+ { name: 'ref', type: 'string' },
812
+ {
813
+ name: 'produit',
814
+ type: 'relation',
815
+ relation: 'produitTestFiltre',
816
+ relationFilter: { actif: true } // Only link active products
817
+ }
818
+ ]
819
+ };
820
+
821
+ // Set up the context once for all tests in this describe block.
822
+ beforeAll(async () => {
823
+ user = await engine.userProvider.findUserByUsername('demo');
824
+ // Cleanup before starting to ensure a clean state
825
+ await deleteModels({ name: productModel.name, _user: user.username });
826
+ await deleteModels({ name: orderModel.name, _user: user.username });
827
+ await deleteData(productModel.name, [], {}, user);
828
+ await deleteData(orderModel.name, [], {}, user);
829
+
830
+ // Create models
831
+ await createModel({ ...productModel, _user: user.username });
832
+ await createModel({ ...orderModel, _user: user.username });
833
+
834
+ // Create test data
835
+ const activeProduct = await insertData(productModel.name, { name: 'Active Product', actif: true }, {}, user);
836
+ const inactiveProduct = await insertData(productModel.name, { name: 'Inactive Product', actif: false }, {}, user);
837
+
838
+ activeProductId = activeProduct.insertedIds[0];
839
+ inactiveProductId = inactiveProduct.insertedIds[0];
840
+ });
841
+
842
+ // Clean up everything after all tests in this block have run.
843
+ afterAll(async () => {
844
+ await deleteModels({ name: productModel.name, _user: user.username });
845
+ await deleteModels({ name: orderModel.name, _user: user.username });
846
+ await deleteData(productModel.name, [], {}, user);
847
+ await deleteData(orderModel.name, [], {}, user);
848
+ });
849
+
850
+ it('should ALLOW inserting data with a valid relation', async () => {
851
+ const result = await insertData(orderModel.name, { ref: 'CMD-OK', produit: activeProductId }, {}, user);
852
+ expect(result.success).toBe(true);
853
+ expect(result.insertedIds).toHaveLength(1);
854
+ // Cleanup the created order for test isolation
855
+ await deleteData(orderModel.name, result.insertedIds, {}, user);
856
+ });
857
+
858
+ it('should REJECT inserting data with a relation that does not respect the filter', async () => {
859
+ const result = await insertData(orderModel.name, { ref: 'CMD-FAIL', produit: inactiveProductId }, {}, user);
860
+ expect(result.success).toBe(false);
861
+ expect(result.error).toContain('produit');
862
+ });
863
+
864
+ it('should ALLOW updating data with a valid relation', async () => {
865
+ // First, create a valid order
866
+ const initialOrder = await insertData(orderModel.name, { ref: 'CMD-TO-EDIT', produit: activeProductId }, {}, user);
791
867
 
868
+ // Update it (even with the same value, this tests the code path)
869
+ const result = await editData(orderModel.name, { _id: initialOrder.insertedIds[0] }, { produit: activeProductId }, {}, user);
870
+
871
+ expect(result.success).toBe(true);
872
+ // The hash might not change if only metadata like _updatedAt changes, so modifiedCount can be 0 or 1.
873
+ expect(result.modifiedCount).toBeGreaterThanOrEqual(0);
874
+
875
+ // Cleanup
876
+ await deleteData(orderModel.name, initialOrder.insertedIds, {}, user);
877
+ });
878
+
879
+ it('should REJECT updating data with a relation that does not respect the filter', async () => {
880
+ // First, create a valid order
881
+ const initialOrder = await insertData(orderModel.name, { ref: 'CMD-TO-EDIT-FAIL', produit: activeProductId }, {}, user);
882
+
883
+ // Attempt to link it to an inactive product, expecting it to fail.
884
+ const result = await editData(orderModel.name, { _id: initialOrder.insertedIds[0] }, { produit: inactiveProductId }, {}, user);
885
+ expect(result.success).toBe(false);
886
+ expect(result.error).toContain('produit');
887
+ // Cleanup
888
+ await deleteData(orderModel.name, initialOrder.insertedIds, {}, user);
889
+ });
890
+ });
792
891
  });