@servicelabsco/nestjs-utility-services 2.0.5 β 2.0.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +96 -0
- package/CODEBASE-REVIEW.md +227 -0
- package/CODEBASE_REFERENCE.md +568 -0
- package/dist/app.controller.d.ts +3 -4
- package/dist/app.controller.js +7 -23
- package/dist/app.controller.js.map +1 -1
- package/dist/app.module.js +37 -4
- package/dist/app.module.js.map +1 -1
- package/dist/auth/auth.module.js +4 -1
- package/dist/auth/auth.module.js.map +1 -1
- package/dist/auth/controllers/admin.auth.controller.js +6 -1
- package/dist/auth/controllers/admin.auth.controller.js.map +1 -1
- package/dist/auth/middlewares/client.connect.middleware.js +0 -2
- package/dist/auth/middlewares/client.connect.middleware.js.map +1 -1
- package/dist/auth/middlewares/internal.middleware.d.ts +1 -0
- package/dist/auth/middlewares/internal.middleware.js +43 -1
- package/dist/auth/middlewares/internal.middleware.js.map +1 -1
- package/dist/auth/middlewares/jwt.middleware.js +1 -1
- package/dist/auth/middlewares/jwt.middleware.js.map +1 -1
- package/dist/auth/services/auth.service.js +1 -1
- package/dist/auth/services/auth.service.js.map +1 -1
- package/dist/auth/services/refresh.token.service.js +35 -1
- package/dist/auth/services/refresh.token.service.js.map +1 -1
- package/dist/auth/services/user.service.d.ts +6 -0
- package/dist/auth/services/user.service.js +41 -2
- package/dist/auth/services/user.service.js.map +1 -1
- package/dist/common/dtos/capture.record.index.dto.js.map +1 -1
- package/dist/common/dtos/index.params.dto.js.map +1 -1
- package/dist/common/dtos/record.data.manager.dto.d.ts +2 -2
- package/dist/common/dtos/report.data.manager.dto.d.ts +2 -2
- package/dist/common/es6.classes.js +10 -1
- package/dist/common/es6.classes.js.map +1 -1
- package/dist/common/exceptions/index.d.ts +1 -0
- package/dist/common/exceptions/index.js +1 -0
- package/dist/common/exceptions/index.js.map +1 -1
- package/dist/common/exceptions/too.many.attempts.exception.d.ts +4 -0
- package/dist/common/exceptions/too.many.attempts.exception.js +11 -0
- package/dist/common/exceptions/too.many.attempts.exception.js.map +1 -0
- package/dist/common/libraries/auth.js +34 -1
- package/dist/common/libraries/auth.js.map +1 -1
- package/dist/common/libraries/custom.crypt.js +34 -1
- package/dist/common/libraries/custom.crypt.js.map +1 -1
- package/dist/common/libraries/custom.logger.d.ts +1 -6
- package/dist/common/libraries/custom.logger.js +12 -34
- package/dist/common/libraries/custom.logger.js.map +1 -1
- package/dist/common/libraries/generic.index.parser.js +8 -2
- package/dist/common/libraries/generic.index.parser.js.map +1 -1
- package/dist/common/libraries/hash.d.ts +2 -2
- package/dist/common/libraries/hash.js +52 -10
- package/dist/common/libraries/hash.js.map +1 -1
- package/dist/common/libraries/index.d.ts +1 -0
- package/dist/common/libraries/index.js +1 -0
- package/dist/common/libraries/index.js.map +1 -1
- package/dist/common/libraries/list.manager.js +6 -1
- package/dist/common/libraries/list.manager.js.map +1 -1
- package/dist/common/libraries/platform.utility.d.ts +1 -0
- package/dist/common/libraries/platform.utility.js +17 -2
- package/dist/common/libraries/platform.utility.js.map +1 -1
- package/dist/common/libraries/record.manager.d.ts +2 -2
- package/dist/common/libraries/record.manager.js +4 -1
- package/dist/common/libraries/record.manager.js.map +1 -1
- package/dist/common/libraries/report.list.manager.js +6 -1
- package/dist/common/libraries/report.list.manager.js.map +1 -1
- package/dist/common/libraries/sql.safety.d.ts +5 -0
- package/dist/common/libraries/sql.safety.js +72 -0
- package/dist/common/libraries/sql.safety.js.map +1 -0
- package/dist/config/cors.config.d.ts +1 -1
- package/dist/config/cors.config.js +11 -2
- package/dist/config/cors.config.js.map +1 -1
- package/dist/config/rate.limiter.config.d.ts +2 -0
- package/dist/config/rate.limiter.config.js +2 -0
- package/dist/config/rate.limiter.config.js.map +1 -1
- package/dist/config/read.typeorm.config.js +34 -1
- package/dist/config/read.typeorm.config.js.map +1 -1
- package/dist/config/typeorm.config.js +34 -1
- package/dist/config/typeorm.config.js.map +1 -1
- package/dist/main.js +61 -10
- package/dist/main.js.map +1 -1
- package/dist/migrations/1619296224563-AddMasterSeedersDataSeederTable.js +35 -2
- package/dist/migrations/1619296224563-AddMasterSeedersDataSeederTable.js.map +1 -1
- package/dist/migrations/1778754675764-AlterColumnValueTypeSysPropertiesTable.d.ts +5 -0
- package/dist/migrations/1778754675764-AlterColumnValueTypeSysPropertiesTable.js +13 -0
- package/dist/migrations/1778754675764-AlterColumnValueTypeSysPropertiesTable.js.map +1 -0
- package/dist/platformUtility/consumers/job.consumer.js +4 -1
- package/dist/platformUtility/consumers/job.consumer.js.map +1 -1
- package/dist/platformUtility/dtos/index.d.ts +1 -0
- package/dist/platformUtility/dtos/index.js +1 -0
- package/dist/platformUtility/dtos/index.js.map +1 -1
- package/dist/platformUtility/dtos/r2.config.dto.d.ts +7 -0
- package/dist/platformUtility/dtos/r2.config.dto.js +7 -0
- package/dist/platformUtility/dtos/r2.config.dto.js.map +1 -0
- package/dist/platformUtility/dtos/set.sms.payload.dto.d.ts +1 -1
- package/dist/platformUtility/dtos/set.sms.payload.dto.js.map +1 -1
- package/dist/platformUtility/es6.classes.d.ts +4 -2
- package/dist/platformUtility/es6.classes.js +4 -0
- package/dist/platformUtility/es6.classes.js.map +1 -1
- package/dist/platformUtility/jobs/process.tracking.webhook.job.js +1 -1
- package/dist/platformUtility/jobs/process.tracking.webhook.job.js.map +1 -1
- package/dist/platformUtility/libraries/common.sqs.polling.js +34 -1
- package/dist/platformUtility/libraries/common.sqs.polling.js.map +1 -1
- package/dist/platformUtility/libraries/create.entity.constants.file.js +34 -1
- package/dist/platformUtility/libraries/create.entity.constants.file.js.map +1 -1
- package/dist/platformUtility/libraries/create.es6.classes.file.js +34 -1
- package/dist/platformUtility/libraries/create.es6.classes.file.js.map +1 -1
- package/dist/platformUtility/libraries/create.es6.jobs.file.js +34 -1
- package/dist/platformUtility/libraries/create.es6.jobs.file.js.map +1 -1
- package/dist/platformUtility/libraries/create.es6.service.file.js +34 -1
- package/dist/platformUtility/libraries/create.es6.service.file.js.map +1 -1
- package/dist/platformUtility/libraries/create.index.file.js +34 -1
- package/dist/platformUtility/libraries/create.index.file.js.map +1 -1
- package/dist/platformUtility/libraries/file.system.utility.js +34 -1
- package/dist/platformUtility/libraries/file.system.utility.js.map +1 -1
- package/dist/platformUtility/libraries/process.common.mail.js +4 -1
- package/dist/platformUtility/libraries/process.common.mail.js.map +1 -1
- package/dist/platformUtility/libraries/process.kaleyra.sms.js +34 -1
- package/dist/platformUtility/libraries/process.kaleyra.sms.js.map +1 -1
- package/dist/platformUtility/libraries/process.ses.raw.mail.js +18 -12
- package/dist/platformUtility/libraries/process.ses.raw.mail.js.map +1 -1
- package/dist/platformUtility/libraries/process.smtp.mail.js +36 -3
- package/dist/platformUtility/libraries/process.smtp.mail.js.map +1 -1
- package/dist/platformUtility/platform.utility.module.js +4 -1
- package/dist/platformUtility/platform.utility.module.js.map +1 -1
- package/dist/platformUtility/services/index.d.ts +1 -0
- package/dist/platformUtility/services/index.js +1 -0
- package/dist/platformUtility/services/index.js.map +1 -1
- package/dist/platformUtility/services/local.property.service.js +4 -1
- package/dist/platformUtility/services/local.property.service.js.map +1 -1
- package/dist/platformUtility/services/mail.service.js +1 -1
- package/dist/platformUtility/services/mail.service.js.map +1 -1
- package/dist/platformUtility/services/maintenance.service.js +35 -2
- package/dist/platformUtility/services/maintenance.service.js.map +1 -1
- package/dist/platformUtility/services/r2.config.service.d.ts +11 -0
- package/dist/platformUtility/services/r2.config.service.js +51 -0
- package/dist/platformUtility/services/r2.config.service.js.map +1 -0
- package/dist/platformUtility/services/redis.service.js +4 -1
- package/dist/platformUtility/services/redis.service.js.map +1 -1
- package/dist/platformUtility/services/remote.request.service.js +4 -1
- package/dist/platformUtility/services/remote.request.service.js.map +1 -1
- package/dist/platformUtility/services/s3.service.js +34 -1
- package/dist/platformUtility/services/s3.service.js.map +1 -1
- package/dist/platformUtility/services/ses.mail.notification.service.js +1 -1
- package/dist/platformUtility/services/ses.mail.notification.service.js.map +1 -1
- package/dist/platformUtility/services/sql.service.d.ts +4 -0
- package/dist/platformUtility/services/sql.service.js +88 -27
- package/dist/platformUtility/services/sql.service.js.map +1 -1
- package/dist/platformUtility/services/zip.service.d.ts +3 -1
- package/dist/platformUtility/services/zip.service.js +3 -3
- package/dist/platformUtility/services/zip.service.js.map +1 -1
- package/dist/security/security.module.js +4 -1
- package/dist/security/security.module.js.map +1 -1
- package/dist/system/constants/index.d.ts +1 -0
- package/dist/system/constants/index.js +18 -0
- package/dist/system/constants/index.js.map +1 -0
- package/dist/system/constants/storage.constants.d.ts +3 -0
- package/dist/system/constants/storage.constants.js +7 -0
- package/dist/system/constants/storage.constants.js.map +1 -0
- package/dist/system/controllers/upload.controller.d.ts +0 -1
- package/dist/system/controllers/upload.controller.js +6 -12
- package/dist/system/controllers/upload.controller.js.map +1 -1
- package/dist/system/dtos/engine.feature.support.dto.d.ts +8 -0
- package/dist/system/dtos/engine.feature.support.dto.js +3 -0
- package/dist/system/dtos/engine.feature.support.dto.js.map +1 -0
- package/dist/system/dtos/index.d.ts +3 -0
- package/dist/system/dtos/index.js +3 -0
- package/dist/system/dtos/index.js.map +1 -1
- package/dist/system/dtos/s3.upload.options.dto.d.ts +2 -7
- package/dist/system/dtos/s3.upload.options.dto.js +2 -43
- package/dist/system/dtos/s3.upload.options.dto.js.map +1 -1
- package/dist/system/dtos/storage.ref.dto.d.ts +37 -0
- package/dist/system/dtos/storage.ref.dto.js +3 -0
- package/dist/system/dtos/storage.ref.dto.js.map +1 -0
- package/dist/system/dtos/storage.upload.options.dto.d.ts +22 -0
- package/dist/system/dtos/storage.upload.options.dto.js +113 -0
- package/dist/system/dtos/storage.upload.options.dto.js.map +1 -0
- package/dist/system/es6.classes.d.ts +4 -2
- package/dist/system/es6.classes.js +4 -0
- package/dist/system/es6.classes.js.map +1 -1
- package/dist/system/index.d.ts +1 -0
- package/dist/system/index.js +1 -0
- package/dist/system/index.js.map +1 -1
- package/dist/system/interceptors/sentry.interceptor.js +34 -1
- package/dist/system/interceptors/sentry.interceptor.js.map +1 -1
- package/dist/system/jobs/sqs.polling.job.js +4 -1
- package/dist/system/jobs/sqs.polling.job.js.map +1 -1
- package/dist/system/jobs/sync.all.code.job.js +4 -1
- package/dist/system/jobs/sync.all.code.job.js.map +1 -1
- package/dist/system/libraries/business.rule.filter.validator.js +1 -1
- package/dist/system/libraries/business.rule.filter.validator.js.map +1 -1
- package/dist/system/libraries/business.rule.query.evaluator.js +1 -1
- package/dist/system/libraries/business.rule.query.evaluator.js.map +1 -1
- package/dist/system/libraries/column.manager.d.ts +1 -1
- package/dist/system/libraries/column.manager.js.map +1 -1
- package/dist/system/libraries/common.storage.service.d.ts +26 -2
- package/dist/system/libraries/common.storage.service.js +88 -1
- package/dist/system/libraries/common.storage.service.js.map +1 -1
- package/dist/system/libraries/execute.code.fix.js +1 -1
- package/dist/system/libraries/execute.code.fix.js.map +1 -1
- package/dist/system/libraries/file.type.detector.d.ts +7 -0
- package/dist/system/libraries/file.type.detector.js +113 -0
- package/dist/system/libraries/file.type.detector.js.map +1 -0
- package/dist/system/libraries/generate.downloadable.report.file.js +36 -3
- package/dist/system/libraries/generate.downloadable.report.file.js.map +1 -1
- package/dist/system/libraries/get.report.query.d.ts +1 -0
- package/dist/system/libraries/get.report.query.js +28 -6
- package/dist/system/libraries/get.report.query.js.map +1 -1
- package/dist/system/libraries/index.d.ts +1 -0
- package/dist/system/libraries/index.js +1 -0
- package/dist/system/libraries/index.js.map +1 -1
- package/dist/system/libraries/model.sync.js +4 -1
- package/dist/system/libraries/model.sync.js.map +1 -1
- package/dist/system/libraries/security.rule.evaluator.js +1 -1
- package/dist/system/libraries/security.rule.evaluator.js.map +1 -1
- package/dist/system/services/aws.s3.service.d.ts +16 -1
- package/dist/system/services/aws.s3.service.js +56 -2
- package/dist/system/services/aws.s3.service.js.map +1 -1
- package/dist/system/services/base.service.js +4 -1
- package/dist/system/services/base.service.js.map +1 -1
- package/dist/system/services/client.credential.service.js +0 -2
- package/dist/system/services/client.credential.service.js.map +1 -1
- package/dist/system/services/cloudflare.r2.service.d.ts +22 -8
- package/dist/system/services/cloudflare.r2.service.js +76 -38
- package/dist/system/services/cloudflare.r2.service.js.map +1 -1
- package/dist/system/services/document.service.js +38 -2
- package/dist/system/services/document.service.js.map +1 -1
- package/dist/system/services/event.detail.service.js +4 -1
- package/dist/system/services/event.detail.service.js.map +1 -1
- package/dist/system/services/index.d.ts +1 -0
- package/dist/system/services/index.js +1 -0
- package/dist/system/services/index.js.map +1 -1
- package/dist/system/services/internal.server.connect.service.d.ts +1 -1
- package/dist/system/services/list.service.d.ts +2 -2
- package/dist/system/services/storage.service.d.ts +65 -0
- package/dist/system/services/storage.service.js +342 -0
- package/dist/system/services/storage.service.js.map +1 -0
- package/dist/system/services/upload.service.d.ts +6 -22
- package/dist/system/services/upload.service.js +71 -243
- package/dist/system/services/upload.service.js.map +1 -1
- package/dist/system/services/user.preference.service.js.map +1 -1
- package/dist/system/subscribers/code.fix.script.subscriber.d.ts +1 -0
- package/dist/system/subscribers/code.fix.script.subscriber.js +10 -1
- package/dist/system/subscribers/code.fix.script.subscriber.js.map +1 -1
- package/dist/system/subscribers/material.view.subscriber.d.ts +1 -0
- package/dist/system/subscribers/material.view.subscriber.js +9 -0
- package/dist/system/subscribers/material.view.subscriber.js.map +1 -1
- package/dist/system/subscribers/recurring.query.subscriber.d.ts +1 -0
- package/dist/system/subscribers/recurring.query.subscriber.js +9 -0
- package/dist/system/subscribers/recurring.query.subscriber.js.map +1 -1
- package/dist/system/subscribers/scheduled.event.subscriber.d.ts +1 -0
- package/dist/system/subscribers/scheduled.event.subscriber.js +9 -0
- package/dist/system/subscribers/scheduled.event.subscriber.js.map +1 -1
- package/dist/system/system.module.js +18 -3
- package/dist/system/system.module.js.map +1 -1
- package/package.json +9 -13
package/CLAUDE.md
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## Repository nature
|
|
6
|
+
|
|
7
|
+
This is a **publishable npm library** (`@servicelabsco/nestjs-utility-services`) that ships compiled output from `dist/` (`main: dist/index.js`). The `src/index.ts`-equivalent (see `dist/index.d.ts`) re-exports `auth`, `common`, `platformUtility`, `security`, `system`, and `worker.service`. Consuming apps install this package and import its modules/services/entities.
|
|
8
|
+
|
|
9
|
+
It is also runnable as a **standalone NestJS app** (`main.ts` / `worker.ts`) for local development of the library itself β running migrations, exercising endpoints, validating wiring. Treat changes as library changes first: anything exported is part of the public API.
|
|
10
|
+
|
|
11
|
+
## Common commands
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
yarn build # nest build β dist/
|
|
15
|
+
yarn start:dev # watch mode standalone app (main.ts)
|
|
16
|
+
yarn start:prod # node dist/main
|
|
17
|
+
yarn lint # eslint --fix on src/, apps/, libs/, test/
|
|
18
|
+
yarn format # prettier on src/ and test/
|
|
19
|
+
yarn test # jest (rootDir=src, *.spec.ts)
|
|
20
|
+
yarn test -- path/to/file.spec.ts # run a single test file
|
|
21
|
+
yarn test -- -t "matches test name" # run tests by name
|
|
22
|
+
yarn test:cov # coverage β ../coverage
|
|
23
|
+
yarn test:e2e # jest --config ./test/jest-e2e.json (*.e2e-spec.ts)
|
|
24
|
+
|
|
25
|
+
# Migrations (TypeORM, src/config/orm.config.ts)
|
|
26
|
+
yarn m:g <name> # generate from entity diff
|
|
27
|
+
yarn m:c <name> # create empty
|
|
28
|
+
yarn m:r # run pending (ts-node, dev)
|
|
29
|
+
yarn m:rd # run pending against compiled dist
|
|
30
|
+
yarn m:rev # revert last
|
|
31
|
+
|
|
32
|
+
yarn runtime-check # curl /health/runtime-check (cron-parser + cache sanity)
|
|
33
|
+
yarn save # repo-specific commit helper (see commit.sh)
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`yarn save` runs `commit.sh`: pulls, optionally builds, optionally runs `slnu syncClassess`, prettier-formats, then commits as `<JIRA-ID> #comment <message>` extracting the JIRA ID from the current branch name. The recent commit history follows this pattern β preserve it for any commits made through this script.
|
|
37
|
+
|
|
38
|
+
## High-level architecture
|
|
39
|
+
|
|
40
|
+
### Five top-level NestJS modules under `src/`
|
|
41
|
+
|
|
42
|
+
- **`auth/`** β users, sessions, JWT, refresh tokens, social login, device tokens, role/permission assignment. Owns `JwtMiddleware`, `BasicAuthMiddleware`, `InternalMiddleware`, `RestrictedMiddleware`.
|
|
43
|
+
- **`common/`** β framework-level base classes and utilities. The module itself is empty (`@Module({})`); its value is the exported classes. Most other modules and consumers extend these. **`CommonEntity`** (audit/who-columns, soft delete, `firstOrNew`, helpers β all postgres entities extend this), `CommonMongoEntity`, `CommonService`, `CommonSubscriber`, `CommonJob`, `CommonConsumer`, `CommonMapperJob`, `MigrationUtility` / `BaseMigrationUtility` / `ReverseMigrationUtility` (wrap TypeORM migrations), `Auth` (static accessor over express-http-context2 β `Auth.user()`, `Auth.check()`, `Auth.id()`, `Auth.login()`), exception classes (`OperationException`, `AccessException`, `FormException`, `MaintenanceException`, `NoLoggedUserException`, `SubscriptionException`), and pure helpers (`Hash`, `CustomCrypt`, `DateUtil`, `PlatformUtility`, `JsonEvaluator`, `EntityEvaluator`, `DatabaseEventEvaluator`, `GenericIndexParser`, `GenericShowParser`, `ReportBodyParser`, `DataManager`, `RecordManager`, `ListManager`, `ReportListManager`, `ReportDataManager`, `ClassMapper`, `CustomLogger`, `SeederUtility`).
|
|
44
|
+
- **`platformUtility/`** β infra-adjacent services: `S3Service`, `SqsService`, `DynamoService`, `RedisService`, `CacheService` (Redis-backed), `QueueService` (BullMQ), `MailService` (with SMTP/SES variants in `libraries/`), `SmsService`, `FcmNotificationService`, `RemoteRequestService`, `AwsConfigService`, `AwsSecretService`, `SqlService`, `ZipService`, `MaintenanceService`, `AuditService`, `StartupService`, `ShutdownService`. Hosts `JobConsumer` (the BullMQ consumer), `MaintenanceMiddleware`, `TrimPipe`.
|
|
45
|
+
- **`security/`** β registers security-related entities (TypeORM `forFeature`); module is otherwise pass-through.
|
|
46
|
+
- **`system/`** β the bulk of the domain: ~70 entities for menus, roles, permissions, models, columns, relationships, properties, lookups, forms, reports, charts, scripts, scheduled events, mail/sms/whatsapp templates, document storage. Controllers (`BaseController`, `DataController`, `FormController`, `MenuController`, `PreferenceController`, `ReportController`, `UploadController`, `UserPreferenceController`), services for each domain area, jobs that drive recurring/material-view/code-fix/scheduled execution, subscribers that react to entity events, and a `SentryInterceptor`.
|
|
47
|
+
|
|
48
|
+
### Module wiring convention β `es6.classes.ts`
|
|
49
|
+
|
|
50
|
+
Every module folder has an `es6.classes.ts` that imports every controller/service/dto/entity/job/subscriber/middleware/library/consumer in that module and exports a single object with arrays. The module file then spreads those arrays into `@Module({ providers, controllers, exports, imports: [TypeOrmModule.forFeature(es6Classes.entities), ...] })`. **When adding a new class to a module, you must register it in that module's `es6.classes.ts` β otherwise it will not be wired into Nest's DI.** The `slnu syncClassess` step in `commit.sh` regenerates these files via libraries in `platformUtility/libraries/create.es6.*.ts` and `create.entity.constants.file.ts`; prefer running it instead of editing by hand when you've added several files.
|
|
51
|
+
|
|
52
|
+
### `app.module.ts` composition
|
|
53
|
+
|
|
54
|
+
Top-level wiring:
|
|
55
|
+
- Three TypeORM connections: **`default`** (postgres write β `src/config/typeorm.config.ts`), **`read`** (postgres read replica β `src/config/read.typeorm.config.ts`), **`mongodb`** (`src/config/mongo.config.ts`).
|
|
56
|
+
- BullMQ via `src/config/queue.config.ts`; queue name from `BULL_QUEUE_NAME`. `PlatformUtilityModule` registers consumers only when `BULL_QUEUE_WORKER=true` (otherwise `es6Classes.consumers = []`).
|
|
57
|
+
- TerminusModule for `/health` (master + read replica + redis + filesystem checks in `health.check.controller.ts`; `/health/runtime-check` validates cron-parser v5 and cache round-trip).
|
|
58
|
+
- Middleware order applied to all routes: `MaintenanceMiddleware` β `JwtMiddleware` β `BasicAuthMiddleware`. `RestrictedMiddleware` on `api/*` (rejects unauthenticated). `InternalMiddleware` on `internal/*` (validates `x-internal-token` against `internal.server.key` property, then logs in user id 2).
|
|
59
|
+
|
|
60
|
+
### Entity / migration discovery
|
|
61
|
+
|
|
62
|
+
Both `typeorm.config.ts` (write) and `mongo.config.ts` glob entities from **the current build dir AND `node_modules/@servicelabsco/**`**:
|
|
63
|
+
|
|
64
|
+
```ts
|
|
65
|
+
entities: [
|
|
66
|
+
join(__dirname, '/../**/**/*.entity.{ts,js}'),
|
|
67
|
+
join(__dirname, '..', '..', 'node_modules/@servicelabsco/**/*.entity.{ts,js}'),
|
|
68
|
+
],
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Mongo uses `*.mentity.{ts,js}` and `*.mongo.subscriber.{ts,js}`. Migrations are similarly merged from this repo's `src/migrations/` and any `@servicelabsco/*` package's `migrations/` dir. `migrationsTableName` is `sys_migrations`. **This means migrations from sibling packages run in this app's database** β be aware when generating new ones, and don't duplicate names across packages.
|
|
72
|
+
|
|
73
|
+
A snake_case naming strategy (`typeorm-naming-strategies`) maps camelCase entity columns to snake_case in postgres. Postgres `bigint` (oid 20) and `decimal` (oid 1700) are auto-cast to JS `number`/`float` in `pg.types.setTypeParser`.
|
|
74
|
+
|
|
75
|
+
### Worker vs API
|
|
76
|
+
|
|
77
|
+
`main.ts` is the HTTP entrypoint (helmet, CORS, rate limit, compression, sentry, global ValidationPipe with `whitelist: true, transform: true`, optional graceful shutdown via `SERVER_COOL_DOWN_PERIOD`). `worker.ts` is the queue-worker entrypoint β same `AppModule`, no `listen()`, just `init()` + shutdown hooks; in this mode `BULL_QUEUE_WORKER=true` is required for consumers to register. `WorkerService.onModuleInit` calls `MaintenanceService.disableQueueIfRequired()` before processing starts.
|
|
78
|
+
|
|
79
|
+
### Auth context
|
|
80
|
+
|
|
81
|
+
Request-scoped user is stored via `express-http-context2` and accessed everywhere through the static `Auth` class (`src/common/libraries/auth.ts`). `JwtMiddleware` sets it from a Bearer token; `InternalMiddleware` sets it to user id 2 after validating the internal token. **Never inject the user from the request manually** β use `Auth.user()` / `Auth.id()` / `Auth.check()` so behavior is consistent with the rest of the codebase, and so it works in jobs/consumers that propagate the context.
|
|
82
|
+
|
|
83
|
+
### Tests
|
|
84
|
+
|
|
85
|
+
Jest config lives in `package.json` (`rootDir: src`, `testRegex: .*\\.spec\\.ts$`, `ts-jest`). E2E uses `test/jest-e2e.json` with `*.e2e-spec.ts`. Tests sit alongside source as `*.spec.ts` (e.g. `auth.service.spec.ts`, `cache.service.spec.ts`, `zip.service.spec.ts`, all of `common/libraries/`, all of `common/exceptions/`).
|
|
86
|
+
|
|
87
|
+
## Conventions to follow
|
|
88
|
+
|
|
89
|
+
- **4-space indent, single quotes, 150-char line width** (`.prettierrc`). Run `yarn format` before committing.
|
|
90
|
+
- New entities extend `CommonEntity` (or `CommonMongoEntity` for mongo) β never `BaseEntity` directly. Filename suffix is `.entity.ts` (postgres) or `.mentity.ts` (mongo) β the loader globs depend on this.
|
|
91
|
+
- New subscribers: filename `*.subscriber.ts` (postgres) or `*.mongo.subscriber.ts` (mongo).
|
|
92
|
+
- New services extend `CommonService`. New jobs extend `CommonJob` / `CommonMapperJob`. New consumers extend `CommonConsumer`.
|
|
93
|
+
- Throw the project's exception classes (`OperationException`, `AccessException`, etc.) rather than raw `HttpException` / `Error` β they integrate with the response shape and (where applicable) sentry.
|
|
94
|
+
- Migrations: instantiate `MigrationUtility` (or `ReverseMigrationUtility`) in the generated migration class so the `up`/`down` semantics handle the reverse-migration flag correctly. Migration filenames follow `<timestamp>-<PascalCaseDescription>.ts`.
|
|
95
|
+
- After adding files to a module, regenerate or update that module's `es6.classes.ts` so DI picks it up.
|
|
96
|
+
- Library/exported public API: anything added under the five top-level modules is reachable by consumer apps via `@servicelabsco/nestjs-utility-services`. Avoid breaking renames/removals of exports without considering downstream impact, and rebuild (`yarn build`) before publishing.
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# Codebase audit β `@servicelabsco/nestjs-utility-services`
|
|
2
|
+
|
|
3
|
+
**Branch:** `main` Β· **HEAD:** `fa2fa14a7` Β· **Date:** 2026-05-18
|
|
4
|
+
**Scope:** 613 TS files / 39,595 LOC across `auth/`, `common/`, `platformUtility/`, `system/`, `security/`, `config/`, `migrations/`.
|
|
5
|
+
**Method:** 6 parallel module-bucket reviewers (`superpowers:code-reviewer`) using `codebase-audit` skill; findings consolidated and ID-tagged here.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Part A β Architecture in one page
|
|
10
|
+
|
|
11
|
+
This is a **publishable npm library** that ships compiled `dist/` and is consumed by other NestJS apps via `entities: [..., 'node_modules/@servicelabsco/**/*.entity.{ts,js}']`. It is also runnable as a standalone Nest app (`main.ts` HTTP + `worker.ts` BullMQ worker) for local dev. Every export is therefore part of the public API surface that ships into every consumer's runtime.
|
|
12
|
+
|
|
13
|
+
Five top-level modules:
|
|
14
|
+
|
|
15
|
+
- **`common/`** β framework primitives (`CommonEntity`, `CommonService`, `CommonJob`, `CommonSubscriber`, `MigrationUtility`, static `Auth` accessor over `express-http-context2`, exception classes, helpers: `Hash`, `CustomCrypt`, `DateUtil`, `JsonEvaluator`, `EntityEvaluator`, `DataManager`/`RecordManager`/`ListManager`/`ReportDataManager`).
|
|
16
|
+
- **`auth/`** β users, sessions, JWT, refresh tokens, social login, device tokens, role/permission, four globally-mounted middlewares (`JwtMiddleware`, `BasicAuthMiddleware`, `InternalMiddleware`, `RestrictedMiddleware`).
|
|
17
|
+
- **`platformUtility/`** β infra adapters: S3, SQS, Dynamo, Redis, Cache, Queue (BullMQ), Mail (SMTP/SES), SMS, FCM, RemoteRequest, AwsConfig, AwsSecret, Sql, Zip (Lambda), Maintenance/Startup/Shutdown, `JobConsumer`.
|
|
18
|
+
- **`security/`** β pass-through (entity registration only).
|
|
19
|
+
- **`system/`** β the bulk of the domain (~70 entities for menus, roles, permissions, models, columns, lookups, forms, reports, charts, **scripts**, scheduled events, mail/sms/whatsapp templates, document storage).
|
|
20
|
+
|
|
21
|
+
Three TypeORM connections (`default` write, `read` replica, `mongodb`); BullMQ via `JobConsumer`; cron-driven scheduled events; subscribers wired to lifecycle hooks dispatching `*Job.delayedDispatch()`. Multi-app migration table (`sys_migrations`) shared across sibling `@servicelabsco/*` packages β duplicate names collide.
|
|
22
|
+
|
|
23
|
+
**Strongest design:** the recent storage refactor (`StorageService` + `CommonStorageService` + S3/R2 adapters) β clean facade-adapter-registry, eager warm-up, well-documented public-API contracts. Use as the template for refactoring elsewhere.
|
|
24
|
+
|
|
25
|
+
**Biggest concentrations of risk:**
|
|
26
|
+
1. `system/libraries/*evaluator*.ts`, `system/libraries/process.script.execution.ts`, `system/libraries/execute.code.fix.ts` β six `eval()` call sites running stored DB strings.
|
|
27
|
+
2. `system/controllers/base.controller.ts` β `plainToInstance(entity, req.body)` mass-assignment.
|
|
28
|
+
3. `auth/services/social.service.ts` + `refresh.token.service.ts` + JWT verify config β multiple high-severity gaps.
|
|
29
|
+
4. `platformUtility/consumers/job.consumer.ts` + `maintenance.service.ts` β shared-singleton mutation, broken worker-mode detection, lock-never-released typo.
|
|
30
|
+
5. `config/typeorm.config.ts` β `pg.types.setTypeParser(1700, parseFloat)` library-wide precision loss on every `numeric`/`decimal` column.
|
|
31
|
+
6. `common/libraries/{custom.crypt,data.manager,common.subscriber}.ts` β weak crypto, bypassed user-binding cache key, transactional dispatch hazard for all 27 subscribers.
|
|
32
|
+
|
|
33
|
+
**Per-module grades:** `common/` C+ Β· `auth/` C+ Β· `platformUtility/` Cβ Β· `system` services+controllers+libraries D+ Β· `system` data layer C+ Β· migrations+config+bootstrap Bβ.
|
|
34
|
+
|
|
35
|
+
**Overall grade:** **Cβ.** Architecture skeleton is uniform and disciplined; the team clearly cares about subtle issues (constant-time compare, helmet hardening, storage refactor). But the eval surface, mass-assignment, plaintext refresh tokens, fail-open security defaults, SSRF, and a library-wide float-precision coercion are each independently shippable-stoppers if a downstream consumer handles real user data or money. Most fixes are localized.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Part B β Findings
|
|
40
|
+
|
|
41
|
+
Severity legend: π΄ Critical Β· π‘ Important Β· π’ Minor
|
|
42
|
+
ID prefix: **S** security Β· **M** money/precision Β· **B** confirmed bug Β· **A** systemic anti-pattern Β· **E** enhancement
|
|
43
|
+
|
|
44
|
+
### S β Security (π΄ unless noted)
|
|
45
|
+
|
|
46
|
+
- [ ] **S1 π΄ Six `eval()` call sites executing stored DB scripts.**
|
|
47
|
+
`system/libraries/code.evaluator.ts:27`, `system/libraries/process.script.execution.ts:39`, `system/libraries/execute.code.fix.ts:58`, `system/libraries/business.rule.query.evaluator.ts:53`, `system/libraries/business.rule.filter.validator.ts:74`, `system/libraries/security.rule.evaluator.ts:53`. Scripts run with full `process`/`require`/fs/network access; security decisions are themselves evaluated through `eval`. β Move to `isolated-vm` / sandboxed `worker_thread` with frozen globals, or a constrained expression DSL (`jsep`/`expr-eval`). Audit-log every script write.
|
|
48
|
+
- [ ] **S2 π΄ Social login does not verify identity binding.** `auth/controllers/social.controller.ts:31-48`, `auth/services/social.service.ts:23-54`. No `aud`/`client_id` check, no `email_verified`, no `(provider, provider_user_id) β local user` mapping. β Validate audience, require verified email, look up/create local user keyed on provider+id, only then issue JWT.
|
|
49
|
+
- [ ] **S3 π΄ JWT verification doesn't pin algorithm/issuer/audience.** `auth/services/auth.service.ts:96-102`. β `jwtService.verify(token, { algorithms: ['HS256'], issuer, audience })`.
|
|
50
|
+
- [ ] **S4 π΄ Refresh tokens stored plaintext, no rotation, no reuse detection.** `auth/services/refresh.token.service.ts:30-49`, `auth/entities/refresh.token.entity.ts:18-19`, `auth/controllers/admin.auth.controller.ts:125-156`. DB leak β durable session creds. β Store sha256, rotate on use, family-chain reuse detection.
|
|
51
|
+
- [ ] **S5 π΄ SSRF/DoS in RemoteRequestService.** `platformUtility/services/remote.request.service.ts:36-71`. No allow-list, no timeout, no `maxContentLength`, no `maxRedirects` cap, no IP/scheme filter. β Default timeout + content cap + hostname guard + explicit opt-in for internal hosts.
|
|
52
|
+
- [ ] **S6 π΄ SQL injection in `security.rule.service.ts:73-104`.** `operation`, `identifier`, `id`, `sourceType` interpolated raw. Public API β consumer caller with user input gets dropped. β Parameterize via `sqlService.read(sql, params)`.
|
|
53
|
+
- [ ] **S7 π΄ SQL injection in `scheduled.event.service.ts:38`.** `source_type` from DTO interpolated raw. β Parameterize.
|
|
54
|
+
- [ ] **S8 π΄ SQL injection in `sync.all.code.job.ts:191, :265`.** `WHERE table_name = '${model.table_name}'`. β Parameterize.
|
|
55
|
+
- [ ] **S9 π΄ SQL injection in `report.column.sync.job.ts:94`.** `query.replace(new RegExp(key, 'ig'), param.default_value)` β param value inlined into SQL. β Parameterize.
|
|
56
|
+
- [ ] **S10 π΄ BaseController mass-assignment.** `system/controllers/base.controller.ts:107, :171`. `plainToInstance(entityObject, body)` lets clients write `created_by`, `updated_by`, `deleted_at`, `password_hash`, any column. β Allow-list from `ColumnManager.allowed`, strip body keys not in list, null out audit/soft-delete cols.
|
|
57
|
+
- [ ] **S11 π΄ Stored-XSS via originalname-derived S3 keys + missing MIME validation.** `system/controllers/upload.controller.ts:42`, `system/services/storage.service.ts:449`, `system/services/upload.service.ts:96`. `FileTypeDetector` exists but isn't used in `UploadController`. SVG/HTML lands in public bucket served by CDN. β Use `FileTypeDetector` in controller, reject HTML/SVG for public folder, force `Content-Disposition: attachment`.
|
|
58
|
+
- [ ] **S12 π΄ `TrimPipe` mutates body in-place + throws on non-body input.** `platformUtility/middlewares/trim.pipe.ts:24-31`. Registered globally β every query/param throws `BadRequest('Validation failed')`. Arrays not handled. β Clone input, handle `Array.isArray`, short-circuit non-body without throwing.
|
|
59
|
+
- [ ] **S13 π΄ `CustomCrypt` PBKDF2 fixed salt + 1000 iterations.** `common/libraries/custom.crypt.ts:45`. Salt is literal `'salt'`; all historical ciphertexts share one key. β Per-installation random salt, β₯600k iterations.
|
|
60
|
+
- [ ] **S14 π΄ `CustomCrypt` AES-256-CBC unauthenticated + plaintext fallback.** `common/libraries/custom.crypt.ts:80-241`. CBC malleable (padding-oracle); `tryPlaintextFallback()` returns attacker-controlled JSON on decryption failure. β Switch to AES-GCM, gate fallback behind explicit migration flag with metric, retire after backfill.
|
|
61
|
+
- [ ] **S15 π΄ `Math.random()` for tokens/OTPs.** `common/libraries/platform.utility.ts:46-49, 96-101` β `generateRandomNumber`, `generateRandomAlpha`, `generateRandomAlphaNumeric`, `generateBase32`, `generateChars` all public statics. β `crypto.randomBytes`/`randomUUID`; separate non-security `pseudoRandom` helper if needed.
|
|
62
|
+
- [ ] **S16 π΄ `InternalMiddleware` key stored in writable DB row.** `auth/middlewares/internal.middleware.ts:39-54`. `x-internal-token` compared (constant-time, good) against `properties.internal.server.key`, which any admin can edit. Header trivially forgeable via proxy header injection. β Read key from `AwsSecretService`/env, IP allow-list, audit-log every internal call.
|
|
63
|
+
- [ ] **S17 π΄ `Auth.login` merges JWT claims over DB.** `auth/middlewares/jwt.middleware.ts:58` β `{...auth, ...accessObject, req}`. If JWT-encoded roles/permissions land in `auth` and aren't returned by `getUserObject`, they survive the merge. β Drop `...auth`, pass only `id`/`session_identifier`.
|
|
64
|
+
- [ ] **S18 π΄ Fail-open security defaults.** `system/libraries/security.rule.evaluator.ts` returns `true` on missing `filter_condition`/empty `condition`/eval error; `business.rule.filter.validator.ts:71` defaults to `true`. β Default-deny.
|
|
65
|
+
- [ ] **S19 π΄ Prototype-pollution-via-`props` + shared `jobInstance` mutation.** `platformUtility/consumers/job.consumer.ts:67-89`. `Object.assign(jobInstance, data.props)` on a long-lived singleton from BullMQ-JSON payloads β `__proto__` pollution + concurrent jobs overwriting each other's state. β Instantiate per execution, sanitize keys.
|
|
66
|
+
- [ ] **S20 π΄ `pg.types.setTypeParser(1700, parseFloat)` library-wide.** `config/typeorm.config.ts:7-12`, `config/read.typeorm.config.ts`, `config/orm.config.ts`. Every `numeric`/`decimal` in every consumer becomes IEEE-754 float. Money precision lost invisibly. (Also tagged **M1**.) β Default to `string`, wrap arithmetic in `Decimal.js` / `big.js`, make opt-out explicit.
|
|
67
|
+
- [ ] **S21 π‘ AWS credentials static long-lived keys via PropertyService.** `platformUtility/services/aws.config.service.ts:51-63`. Bypasses default credential chain (IRSA/instance profile). β Allow SDK chain when properties absent.
|
|
68
|
+
- [ ] **S22 π‘ Sentry `includeLocalVariables: true` + no `beforeSend` PII scrubber.** `config/sentry.config.ts:5`. Local stack-frame vars include JWTs, password hashes, internal tokens. `SentryInterceptor` (`system/interceptors/sentry.interceptor.ts:14-21`) forwards `req` + `Auth.user()` unscrubbed. β Add `beforeSend` redactor, drop `includeLocalVariables` (or scrub).
|
|
69
|
+
- [ ] **S23 π‘ `/health` endpoints unauthenticated, expose recon info.** `health.check.controller.ts:152-209` (Redis write probe on every call) + `/health/detailed` exposes `process.version`/`platform`/`uptime`. β Gate `detailed` behind `InternalMiddleware`; `/health` should just `PING`.
|
|
70
|
+
- [ ] **S24 π‘ `/health/runtime-check` returns 200 on failure.** `health.check.controller.ts:361-429`. Monitors miss the down state. β Throw or return 503.
|
|
71
|
+
- [ ] **S25 π‘ `CORS_ORIGINS` unset β `origin: true` (reflects any origin).** `main.ts:53`, `config/cors.config.ts:27`. β Deny-by-default with explicit opt-in env.
|
|
72
|
+
- [ ] **S26 π‘ Rate-limit default `max: 10000/min` + `trust proxy 1` hop assumption.** `config/rate.limiter.config.ts:19`, `main.ts:57`. Effectively no limit + bypassable behind 2-hop proxy chain via `X-Forwarded-For` spoof. β Document hop count from env; lower default.
|
|
73
|
+
- [ ] **S27 π‘ Mongo `logging: 'all'` default writes query bodies to file.** `config/mongo.config.ts:22`. No env gate. β Make env-driven like Postgres.
|
|
74
|
+
- [ ] **S28 π‘ Real PII seeded into every consumer's DB.** `migrations/1619193418460-AddDefaultUserDataSeederTable.ts` β real email + phone for user id 1. β Replace with placeholder.
|
|
75
|
+
- [ ] **S29 π‘ RemoteRequestService logs full request headers including `Authorization`.** `platformUtility/services/remote.request.service.ts:104-111`. β Redact auth headers before persist.
|
|
76
|
+
- [ ] **S30 π‘ `InternalServerConnectService` logs full `options` including `x-client-secret`.** `system/services/internal.server.connect.service.ts:75`. β Drop log.
|
|
77
|
+
- [ ] **S31 π‘ `ClientCredentialService` logs `'client secret validated', validated` on every auth.** `system/services/client.credential.service.ts:36`. β Drop log.
|
|
78
|
+
- [ ] **S32 π‘ `refresh.property.cache.job` caches decrypted secrets plaintext in Redis 15min.** `system/jobs/refresh.property.cache.job.ts:24-28`. β Cache encrypted + decrypt on read, or document trust model.
|
|
79
|
+
- [ ] **S33 π‘ `BasicAuthMiddleware` has no throttle/lockout.** `auth/middlewares/basic.auth.middleware.ts:50-57`. Brute-force surface; `UserService.assertCanAttemptLogin` not wired here. β Wire the same throttle as `AdminAuthController.login`.
|
|
80
|
+
- [ ] **S34 π‘ `validateSession` creates session rows on verify path.** `auth/services/auth.service.ts:144-152`. `firstOrCreate` β DB-fill DoS via session_identifier scraping. β Use `findOne`.
|
|
81
|
+
- [ ] **S35 π‘ `getToken` device check bypassable when no device bound.** `auth/controllers/admin.auth.controller.ts:138-139`. β Make device binding mandatory.
|
|
82
|
+
- [ ] **S36 π‘ `logout` doesn't revoke active JWT.** `auth/controllers/admin.auth.controller.ts:164-184`. β Call `revokeSession()`.
|
|
83
|
+
- [ ] **S37 π‘ `logout` returns soft-deleted refresh tokens (PII).** Same controller line 183. β Don't return tokens.
|
|
84
|
+
- [ ] **S38 π‘ `ReportColumnSyncJob` HTTP-triggerable without report-edit permission.** `system/controllers/report.controller.ts:50`. β Add `reportService.hasAccess` check.
|
|
85
|
+
- [ ] **S39 π‘ `JWT_SECRET` read directly via `process.env` without validation.** `config/jwt.config.ts:5-6`. Missing env β signs with `undefined`. β `ConfigService` with required-non-empty validation.
|
|
86
|
+
- [ ] **S40 π‘ `CacheService.getStoreInfo` exposed publicly (`publicExposed = true`), leaks Redis host/port.** `platformUtility/services/cache.service.ts:128-182`. β Admin-only.
|
|
87
|
+
- [ ] **S41 π‘ ZipService hardcoded Lambda URLs in ap-south-1.** `platformUtility/services/zip.service.ts:36, 72`. β Read from properties.
|
|
88
|
+
- [ ] **S42 π‘ S3Service hardcoded `ContentType: 'text/plain'`** for `uploadFile`/`uploadContent`. `platformUtility/services/s3.service.ts:72, 102`. No `ServerSideEncryption`, no path/key sanitization. β Pass detected content-type, add SSE default, sanitize keys.
|
|
89
|
+
|
|
90
|
+
### M β Money / precision (π΄ unless noted)
|
|
91
|
+
|
|
92
|
+
- [ ] **M1 π΄ `pg.types.setTypeParser(1700, parseFloat)` library-wide** (also **S20**). Every `numeric`/`decimal` column silently loses precision in every consumer.
|
|
93
|
+
- [ ] **M2 π‘ `@Column({ type: 'float', precision: 20, scale: 3 })` for duration columns.** `system/entities/event.log.entity.ts:24`, `system/entities/system.log.entity.ts:25`. Postgres `float` ignores `precision`/`scale` (only `numeric(p,s)` honors them) β duration drifts. β Switch to `numeric(20,3)` or integer ms.
|
|
94
|
+
|
|
95
|
+
### B β Confirmed bugs (π‘ unless noted)
|
|
96
|
+
|
|
97
|
+
- [ ] **B1 π΄ `data.manager.ts:336` β `Auth.id` (function reference) stored, not `Auth.id()`.** Cache user-binding check `data.user_id !== user.id` is always true β cache effectively dead, or worse if comparison ever matches. β Call `Auth.id()`.
|
|
98
|
+
- [ ] **B2 π΄ `DatabaseEventEvaluator.isNewRecord` always false.** `common/libraries/database.event.evaluator.ts:28` β dead condition `!entity[col] && entity[col]`. Affects every subscriber via this evaluator. β Fix to `!this.event.databaseEntity?.[column]`.
|
|
99
|
+
- [ ] **B3 π΄ `QueueService.isWorkerInstance()` returns `... || true` β always truthy.** `platformUtility/services/queue.service.ts:349-351`. Same bug in `StartupService.checkForWorkerQueue` (`startup.service.ts:40`). Pause path unreachable. β `=== 'true'`.
|
|
100
|
+
- [ ] **B4 π΄ `MaintenanceService.removeRuntimeLock` sets `runtimeLock = true`** (typo for `false`). `platformUtility/services/maintenance.service.ts:141-144`. Lock never releases programmatically.
|
|
101
|
+
- [ ] **B5 π΄ `JobConsumer.failedJob` writes `FailedBullJobEntity` on every retry attempt.** `platformUtility/consumers/job.consumer.ts:193-216`. No `attemptsMade >= maxAttempts` guard β table fills with retry duplicates.
|
|
102
|
+
- [ ] **B6 π΄ `sync.all.code.job.ts:147` `allowed_permissions: 'raed'`** β typo for `'read'`. Affects every auto-created model.
|
|
103
|
+
- [ ] **B7 π΄ `sync.all.code.job.ts:212-240 setColumn` inverts NOT NULL semantics** β `required: data.is_nullable` should be `!data.is_nullable`.
|
|
104
|
+
- [ ] **B8 π΄ SQS polling deletes message before processing returns.** `system/jobs/sqs.polling.job.ts:33-56`. Process crash mid-handler = lost message. β Delete after success.
|
|
105
|
+
- [ ] **B9 π΄ `report.column.sync.job.ts:99-104` catch returns `{}` β downstream `removeAllUnusedColumns` soft-deletes every column on the report.** Silent destructive failure.
|
|
106
|
+
- [ ] **B10 π΄ `event.queue.service.ts:80-89, :105-114` swallow all errors silently.** Failed event triggers indistinguishable from success in logs.
|
|
107
|
+
- [ ] **B11 π΄ `DeviceTokenService.setDeviceToken` infinite-recursion-on-save-error.** `auth/services/device.token.service.ts:38-42`. Persistent constraint violation blows the stack.
|
|
108
|
+
- [ ] **B12 π΄ `ShutdownService` does not drain in-flight jobs.** `platformUtility/services/shutdown.service.ts:42-52` β `pause()` then sleep then exit; missing `worker.close(false)`. Active jobs killed mid-execution.
|
|
109
|
+
- [ ] **B13 π΄ `CommonEntity.firstOrNew` / `firstOrCreate` do not honor soft delete.** `common/libraries/common.entity.ts:78-105`. Returns soft-deleted rows, skip create. β `withDeleted: false` or soft-delete-aware repo.
|
|
110
|
+
- [ ] **B14 π‘ `JobConsumer.handleNextJob` swallows all errors with `console.log`.** `job.consumer.ts:121`. Chained jobs silently lost.
|
|
111
|
+
- [ ] **B15 π‘ `set.event.queue.job.ts:40-49 while(true)` no break condition.** Poison row spins until BullMQ timeout. β Max iterations + poison-pill detection.
|
|
112
|
+
- [ ] **B16 π‘ `cleanRecord` non-transactional delete+save.** `system/jobs/scheduled.event.job.ts:32-39`. Crash between = lost cursor.
|
|
113
|
+
- [ ] **B17 π‘ `PreferenceController` missing `await preference.save()`.** `system/controllers/preference.controller.ts:81, :103`.
|
|
114
|
+
- [ ] **B18 π‘ `writeRows` shadowed `row` variable.** `system/libraries/generate.downloadable.report.file.ts:248,254` β inner `let row` shadows outer; final `row.commit()` is dead code.
|
|
115
|
+
- [ ] **B19 π‘ Report `OFFSET` pagination on raw SQL is O(NΒ²)** on large reports. `system/libraries/generate.downloadable.report.file.ts:186-211`. β Keyset.
|
|
116
|
+
- [ ] **B20 π‘ `entity.evaluator.ts` falsy-as-null** β `equals('flag', false)` returns false; same with `0` and `''`. β Explicit `=== null || === undefined`.
|
|
117
|
+
- [ ] **B21 π‘ `CommonSubscriber.handleFloatColumn` rounds updated value but compares against unrounded original** β asymmetric, near-scale-boundary "no change". `common/libraries/common.subscriber.ts:195-202`.
|
|
118
|
+
- [ ] **B22 π‘ `DateUtil` hard-codes IST offset `330` minutes.** `common/libraries/date.util.ts:172-188, 308-311`. Library ships to multi-tenant consumers. β Require offset or read from config.
|
|
119
|
+
- [ ] **B23 π‘ `json.evaluator.ts:188, 194` `rule.value.split(',')` crashes if value is already an array.** Same code coerces with `Number()` β silent `NaN` ranges.
|
|
120
|
+
- [ ] **B24 π‘ `common.job.ts:106-107` `parseInt(env)` with no fallback β `NaN` passed to BullMQ `removeOnComplete`.** Version-dependent behavior.
|
|
121
|
+
- [ ] **B25 π‘ `getReportBaseQuery` regex injection.** `common/libraries/data.manager.ts`, `record.manager.ts`, `list.manager.ts`, `report.data.manager.ts` β `query.replace(new RegExp(key, 'ig'), value)` where key/value are unescaped. β Escape key, neutralize `$`-sequences, parameterize.
|
|
122
|
+
- [ ] **B26 π‘ `DynamoService.updateItem` RMW without `ConditionExpression`** β lost updates. `platformUtility/services/dynamo.service.ts:139-145`.
|
|
123
|
+
- [ ] **B27 π‘ `DynamoService.queryItem` returns `undefined` (not `[]`) when `Items` missing.** Same file:106. Callers iterating throw.
|
|
124
|
+
- [ ] **B28 π‘ `SqsService.add` mutates caller's message; FIFO `MessageGroupId = 'group-1'` hardcoded** β serializes all messages. `platformUtility/services/sqs.service.ts:73`.
|
|
125
|
+
- [ ] **B29 π‘ `MailService.send` returns `undefined` when `NOTIFICATION_DISABLED`** + no idempotency on redelivery β duplicate emails. `platformUtility/services/mail.service.ts:56-61`.
|
|
126
|
+
- [ ] **B30 π‘ `AccessService.hasRoleIds([2])` short-circuits before user check.** `auth/services/access.service.ts:87`. β Decide whether public means anonymous; document or fix.
|
|
127
|
+
- [ ] **B31 π‘ `UserService.LOGIN_FAIL_LIMIT/WINDOW` read `process.env` at class-load.** `auth/services/user.service.ts:29-30`. Library consumers loading env later get `NaN`. β Read via `ConfigService` at runtime.
|
|
128
|
+
- [ ] **B32 π‘ `Hash.generateSalt(len)` passes `len` as bcrypt rounds.** `common/libraries/hash.ts:82-83`. Misleading name + dangerous if called with high "len" (DoS).
|
|
129
|
+
- [ ] **B33 π‘ `MaintenanceMiddleware` mounts on `*` blocks `/health/*`.** `app.module.ts:51-55`. Ops can't detect maintenance via health probe.
|
|
130
|
+
- [ ] **B34 π‘ `worker.ts` `app.init()` not awaited.** Fire-and-forget bootstrap.
|
|
131
|
+
- [ ] **B35 π‘ `BaseMigrationUtility.primary()` defaults to `int`, not `bigint`.** `common/libraries/base.migration.utility.ts:158-172`. Log/audit tables (`sys_mail_logs`, `sys_data_logs`, `sys_event_queues`) will hit 2.1B ceiling. β Default `bigint`.
|
|
132
|
+
- [ ] **B36 π‘ `getDisplayName` uses `str.substr(0, -3)` which returns `''`.** `system/jobs/model.scanner.job.ts:259-266`, `report.column.sync.job.ts:114-121`. `lastChar === '_id'` always false. Duplicated.
|
|
133
|
+
- [ ] **B37 π‘ `model.scanner.job.ts:23` silently returns `model_hash` when model missing.** Caller can't distinguish success from not-found.
|
|
134
|
+
- [ ] **B38 π‘ `refresh.material.view.job` doesn't check `now > date`** like the sibling scheduled-event job does. Past-dated events get enqueued.
|
|
135
|
+
- [ ] **B39 π‘ `AwsConfigService.getProperty` truthy-drops `'0'`/`''`.** `platformUtility/services/aws.config.service.ts:74`.
|
|
136
|
+
- [ ] **B40 π‘ `AuditService.setAuditRecord` uses `return` (not `continue`) inside `for` loop.** `platformUtility/services/audit.service.ts:71`. Skips remaining columns.
|
|
137
|
+
- [ ] **B41 π‘ `AwsSecretService.getItem` swallows errors with `console.log` + logs full error object** (may contain SecretId). `platformUtility/services/aws.secret.service.ts:62-68`.
|
|
138
|
+
- [ ] **B42 π‘ `PropertyService.set firstOrNew` without transaction β race on concurrent writes.** `system/services/property.service.ts`.
|
|
139
|
+
- [ ] **B43 π‘ `DocumentService.setDocument` race; no DB unique on `(source_type, source_id, file_hash)`.**
|
|
140
|
+
- [ ] **B44 π‘ `LoggerService` singleton state leaks across requests** β concurrent requests share `logs[]`. `system/services/logger.service.ts`. β Request-scope or context-store.
|
|
141
|
+
- [ ] **B45 π‘ `ReportEntity.first(...)` result passed to manager without null guard.** `system/controllers/report.controller.ts:87, :118`.
|
|
142
|
+
- [ ] **B46 π‘ Migration `1599567001962-AddUserSequence.ts:5` `queryRunner.query()` not awaited.**
|
|
143
|
+
- [ ] **B47 π‘ `getIpFromRequest` returns unparsed `x-forwarded-for` (CSV string).** `common/libraries/platform.utility.ts:312-323`.
|
|
144
|
+
- [ ] **B48 π‘ `Hash.hashMD5(${date.now()}-${8-digit-int})` collision risk in same ms.** Used as request identifier in `data.manager.ts:333`, `report.data.manager.ts:252`. β `crypto.randomUUID()`.
|
|
145
|
+
- [ ] **B49 π’ `UserController` constructor parameter named `s` instead of `userService`.** Readability.
|
|
146
|
+
- [ ] **B50 π’ `getStoreInfo` debug method publicly exposed via `publicExposed = true`.** (Also **S40**.)
|
|
147
|
+
|
|
148
|
+
### A β Systemic anti-patterns (these recur across the codebase)
|
|
149
|
+
|
|
150
|
+
- [ ] **A1 π΄ Subscribers dispatch BullMQ jobs synchronously inside the source DB transaction.** `common/libraries/common.subscriber.ts:35-46, 55-61`. 27 subscribers carry this hazard: if outer transaction rolls back, the job is already in Redis processing a stale/nonexistent record. β Enqueue inside `queryRunner.afterTransactionCommit` or buffer-and-flush-on-commit. **Fix once in `CommonSubscriber`, fixes all 27.**
|
|
151
|
+
- [ ] **A2 π΄ No idempotency keys on side-effecting jobs.** `UserGroupMemberJob`, `UserGroupPermissionJob`, `RefreshMaterialViewJob`, `SetScheduledEventJob`, `SetEventQueueJob`, `SyncAllCodeJob`. At-least-once queue + run-once assumptions β duplicates on redelivery. β DB unique constraints + `INSERT ... ON CONFLICT DO NOTHING` / `firstOrCreate({source_type, source_id, ...})`.
|
|
152
|
+
- [ ] **A3 π΄ 25+ entities use `@Column('json')` typed as `any`** instead of `jsonb` + typed DTO. `report.entity.ts:60-63`, `event.queue.entity.ts:30,47`, `mail.log.entity.ts:45-49`, `column.entity.ts:28`, etc. Postgres `json` has no GIN; library consumers can't enumerate the shape. β Migrate all to `jsonb`; type each `attributes`/`payload`/`data` with a DTO (`DocumentAttributesDto` is the template).
|
|
153
|
+
- [ ] **A4 π΄ Six `eval()` call sites for business / security / code-fix rules** (also **S1**). Single highest-risk pattern in the codebase.
|
|
154
|
+
- [ ] **A5 π΄ Manual SQL string building with regex placeholder substitution** in 5 manager classes (`data`/`record`/`list`/`report.data`/`report.list` manager). Soft-delete filter, join definitions, parameter substitution drift between managers. Tripwire (`common/libraries/sql.safety.ts`) exists but isn't applied at building sites β only at executing service. β Collapse onto a single internal query builder; wire tripwire at every manager exit.
|
|
155
|
+
- [ ] **A6 π΄ User-controlled strings flow into raw SQL templates in 7+ places** (S6βS10, B25). Escape helper `escapeSqlString` exists; use it everywhere or parameterize.
|
|
156
|
+
- [ ] **A7 π΄ Fail-open security / business rule evaluators** (also **S18**, **B**). Default-deny baseline missing.
|
|
157
|
+
- [ ] **A8 π‘ Error swallowing with `console.log`.** `aws.secret.service.ts:67`, `job.consumer.ts:121`, `cache.service.ts:104`, `custom.crypt.ts`, `event.queue.service.ts:80,105`, `report.column.sync.job.ts:99`, `sync.all.code.job.ts:321-335`. β Route through `CustomLogger`, never swallow.
|
|
158
|
+
- [ ] **A9 π‘ Active Record entity access (`Entity.first/find/firstOrNew`) in controllers and libraries** β no repository abstraction; couples to TypeORM forever; untestable without DB. The storage refactor shows the team can do DI well; extend pattern to data access.
|
|
159
|
+
- [ ] **A10 π‘ Polymorphic FKs (`source_type` MD5-hash + `source_id`) with no DB-level referential integrity, no orphan-cleanup job.** 12+ entities (`column`, `comment`, `document`, `event_queue`, `mail_log`, β¦). β Document contract + add sweeper job.
|
|
160
|
+
- [ ] **A11 π‘ Static singletons holding context** (`Auth`, `PlatformUtility.setEntity/Service/Job` mutable global registries, `CustomLogger` env-cached at first call). Multi-tenant / cross-request fragility.
|
|
161
|
+
- [ ] **A12 π‘ Cryptography assembled from primitives, not a hardened library.** `CustomCrypt` (PBKDF2-fixed-salt + AES-CBC + JSON.parse fallback) and `Hash` (bcrypt + MD5 + xxhash). Migration comments show team knows about legacy formats β productionize a one-shot re-encrypt job and remove fallback.
|
|
162
|
+
- [ ] **A13 π‘ Library-as-app coupling.** `common/` imports from `system/entities/*`, `platformUtility/services/*`, `auth/dtos/*`. Consumers pulling `common` transitively get everything. β Split `common` into true-primitive layer (no system/auth imports) + higher-level managers layer.
|
|
163
|
+
- [ ] **A14 π‘ 27 subscribers are 17-line stubs of identical shape.** β Decorator factory `@SubscribeFor(Entity, JobClass)` + registry; drops to 1 file.
|
|
164
|
+
- [ ] **A15 π‘ Magic IDs scattered.** Role id 1 = superadmin, role id 2 = public, user id 2 = internal middleware login, source-type MD5 literals inlined in `column.mapper.job.ts:11-24`, `relationship.mapper.job.ts:11-20`, `model.service.ts:49-67`. β `Role.SUPER_ADMIN`, `Role.PUBLIC`, `User.INTERNAL`, use `SourceHash.*` consistently.
|
|
165
|
+
- [ ] **A16 π‘ Single-file migrations bundle schema + data fill + index in one transaction.** No two-phase pattern (deploy column β backfill β flip NOT NULL β drop old). Migration DSL has no `CREATE INDEX CONCURRENTLY` helper. β Add DSL helpers; document multi-stage pattern.
|
|
166
|
+
- [ ] **A17 π‘ `PropertyService` is the de-facto config oracle for everything** (URLs, secrets, gateway names, FCM topics). Every service hits it; no centralized cache or rotation. β Extract `CachedProperty<T>` with TTL.
|
|
167
|
+
- [ ] **A18 π‘ PII not redacted before Sentry / logs** (also **S22**, **S29**, **S30**, **S31**, **S32**). β Single redaction list applied at all three exit points (Sentry beforeSend, RemoteRequestService.saveLog, CommonJob.saveLogs).
|
|
168
|
+
- [ ] **A19 π‘ Inheritance for a flag.** `CommonService.publicExposed = false` and several subclasses extend just to flip the flag. β Decorator or composition.
|
|
169
|
+
- [ ] **A20 π‘ Falsy-as-null conflation** (also **B20**). `Auth.id()` returns `boolean | number`; `EntityEvaluator` treats `0`/`''`/`false` as null; soft-delete checks via `if (deleted_at)`. β Tighten return types to `T | null`, explicit `=== null` checks.
|
|
170
|
+
- [ ] **A21 π‘ Float/decimal precision** (also **M1**, **M2**). Library-wide.
|
|
171
|
+
- [ ] **A22 π’ No `*.spec.ts` for the most load-bearing classes:** all 5 managers (`data`/`record`/`list`/`report.data`/`report.list`), `CommonSubscriber`, `CommonJob`, `BaseMigrationUtility`, every middleware, all subscribers, all jobs. β Backfill priority: managers β middlewares β subscribers β jobs.
|
|
172
|
+
|
|
173
|
+
### E β Enhancements
|
|
174
|
+
|
|
175
|
+
- [ ] **E1** Extract `AwsClientFactory` across S3/SQS/Dynamo/Secrets/SES β single credential-chain + retry config.
|
|
176
|
+
- [ ] **E2** Storage refactor template β apply same adapter-registry pattern to SMS (Twilio/MessageBird), Push (FCM/APNs), Secrets backends.
|
|
177
|
+
- [ ] **E3** Sandboxed script execution (`isolated-vm` or DSL) for the 6 eval sites. **Required to land S1/A4 safely.**
|
|
178
|
+
- [ ] **E4** Migration DSL: add `concurrentIndex()`, `bigint`-default `primary()`, two-phase migration helper, `CREATE INDEX CONCURRENTLY` lint.
|
|
179
|
+
- [ ] **E5** CI lint script: `synchronize: true`, `DROP COLUMN`, `NOT NULL` without backfill, `CREATE INDEX` without `CONCURRENTLY`, `UPDATE`/`DELETE` without `WHERE`, `eval(`, template-string SQL, `@Column('json')`, `Math.random`. Pre-commit hook.
|
|
180
|
+
- [ ] **E6** Centralize role/source-hash/user-id constants.
|
|
181
|
+
- [ ] **E7** `Auth.requireUser()` helper that throws `NoLoggedUserException`; tighten `Auth.id()` to `number | null`.
|
|
182
|
+
- [ ] **E8** Backfill `*.spec.ts` for managers / middlewares / crypto / subscribers / jobs (priority order).
|
|
183
|
+
- [ ] **E9** Single PII redaction list applied at Sentry beforeSend + RemoteRequestService.saveLog + CommonJob.saveLogs + LoggerService.
|
|
184
|
+
- [ ] **E10** Remove ~15 leftover `console.log` debug statements in prod code.
|
|
185
|
+
- [ ] **E11** Move all secrets to `AwsSecretService` (JWT_SECRET, internal.server.key, AWS keys).
|
|
186
|
+
- [ ] **E12** Repository / data-access abstraction for entity access (mirror storage adapter pattern).
|
|
187
|
+
- [ ] **E13** Migrate from `pg.types` decimalβfloat to decimalβstring + Decimal.js arithmetic wrapper. Document as breaking change for consumers.
|
|
188
|
+
- [ ] **E14** Migrate all `@Column('json')` β `'jsonb'` + typed DTOs (single multi-table migration).
|
|
189
|
+
- [ ] **E15** Refresh-token rotation + family-chain reuse detection.
|
|
190
|
+
- [ ] **E16** Remove `test.mentity.ts` and `test.mongo.job.ts` from production module.
|
|
191
|
+
- [ ] **E17** Two-phase migration pattern doc + examples.
|
|
192
|
+
- [ ] **E18** Sweeper job for orphan rows on polymorphic FKs.
|
|
193
|
+
- [ ] **E19** Subscriber decorator + registry β drop 27 stubs to 1 file.
|
|
194
|
+
- [ ] **E20** Migration filename cleanup β 60+ files end in `.ts.ts`.
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## Part C β Phase plan
|
|
199
|
+
|
|
200
|
+
| Phase | Goal | Risk to prod | Items |
|
|
201
|
+
|---|---|---|---|
|
|
202
|
+
| **0 β Safety nets** | Observability, rollback playbook, feature-flag pattern | None | E5 (CI lint), E10 (drop debug logs), E16 (drop test mentity), E20 (filename cleanup) |
|
|
203
|
+
| **1 β Foundations** | Helpers, TS strictness, audit hooks | None | E4 (DSL helpers), E6 (constants), E7 (Auth helpers), E9 (PII redactor) |
|
|
204
|
+
| **2 β Pure bugs (B-series)** | One PR per item; quick wins | Low | B1, B2, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14βB50 |
|
|
205
|
+
| **3 β Security (S-series)** | Discussion per item, staging verify | Medium | S2, S3, S4, S5, S6βS9, S10, S11, S12, S13, S14, S15, S16, S17, S18, S19, plus S21βS42 |
|
|
206
|
+
| **4 β Money / precision (M-series)** | Multi-week, careful migration | High | M1 (pg.types β breaking), M2 (floatβnumeric duration cols) |
|
|
207
|
+
| **5 β Systemic patterns (A-series)** | Refactor with flags | Medium | A1 (subscriber dispatch), A2 (idempotency), A3 (jsonβjsonb), A5 (manager SQL), A6 (template SQL), A7 (fail-closed), A8 (no swallow), A10 (FK orphans), A12 (crypto), A14 (subscriber decorator), A15 (constants), A16 (two-phase migration), A17 (cached property), A18 (PII), A20 (falsy/null), A22 (tests) |
|
|
208
|
+
| **6 β Architecture (E-series)** | Long horizon; incremental | Low | E1, E2, E3 (sandboxed eval β blocks A4/S1), E11, E12, E13 (pg.types), E14 (jsonβjsonb), E15 (refresh tokens), E17, E18, E19 |
|
|
209
|
+
|
|
210
|
+
**Suggested kickoff (recommended):** Do **B1** as the trial fix end-to-end (one-line: `Auth.id` β `Auth.id()`, add a spec, ship). Establishes the per-change template: ID-tagged PR title, regression test, Linear issue closed. Then take the rest of phase 2 in any order β they're independent.
|
|
211
|
+
|
|
212
|
+
**Critical blockers if shipping to a new consumer that handles money or real PII:** S1/A4 (eval), S2/S3/S4 (auth), S5 (SSRF), S10/S11 (mass-assignment + upload XSS), S13/S14 (crypto), M1 (precision), A1 (subscriber dispatch), A2 (idempotency).
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## Per-module grades
|
|
217
|
+
|
|
218
|
+
| Module | Files | LOC | Grade | Headline |
|
|
219
|
+
|---|---:|---:|:---:|---|
|
|
220
|
+
| `common/` | 81 | 9,198 | **C+** | Solid breadth, but `Auth.id` ref bug, PBKDF2 fixed-salt, plaintext-fallback decrypt, `Math.random` for tokens, falsy-as-null systemic |
|
|
221
|
+
| `auth/` | 39 | 2,550 | **C+** | Constant-time compare β, bcrypt-12 β, but social-login trust, JWT-alg-not-pinned, plaintext refresh tokens, InternalMiddleware writable key |
|
|
222
|
+
| `platformUtility/` | 85 | 8,179 | **Cβ** | `isWorkerInstance()||true`, `removeRuntimeLock=true`, shared `jobInstance` mutation, SSRF, retry-amplified failed-job rows, no idempotency |
|
|
223
|
+
| `system/` services + controllers + libraries | 61 | 8,400 | **D+** | 6 `eval()` RCE sites, mass-assignment, fail-open security, stored-XSS via originalname; storage subsystem is B+ |
|
|
224
|
+
| `system/` data layer + jobs + subscribers | ~178 | ~6,600 | **C+** | Subscriber-dispatch-in-tx hazard Γ 27, idempotency gaps Γ 6, json-not-jsonb Γ 25, `'raed'` typo |
|
|
225
|
+
| migrations + config + bootstrap | ~163 | ~3,800 | **Bβ** | `pg.types` precision coercion, default-int PKs, no concurrent-index DSL, real-PII seeded; bootstrap is mostly hardened |
|
|
226
|
+
|
|
227
|
+
**Overall: Cβ.** Localized fixes can lift to **B** within a focused 4-6 week effort; reaching **A** requires the eval-sandbox migration (E3) and the pg.types-precision change (M1/E13), both multi-month projects with downstream impact.
|