@venizia/ignis-docs 0.2.0 → 0.2.1-1

package/content/references/base/connectors.md

@@ -1,178 +1,121 @@
 ---
-title: Connectors Reference
-description: How IGNIS separates engine-neutral base classes from per-engine connectors, dual-door exports, and adding a new engine
-difficulty: advanced
+title: Connectors
+description: How IGNIS supports multiple database and search engines behind one engine-neutral contract
+difficulty: intermediate
 ---
 
-# Deep Dive: Connectors
+# Connectors
 
-IGNIS's persistence layer is split into two layers: an **engine-neutral root** and a set of **connectors**, one per storage engine. This page documents that architecture, the dual-door export strategy, naming symmetry between engines, and how to add a new connector.
+A connector is how IGNIS adds a storage engine - PostgreSQL, Typesense, Meilisearch - behind one engine-neutral contract. `DataSource`, `Entity`, and `Repository` mean the same thing no matter which engine backs them.
 
-**Files:** `packages/core/src/base/**` (neutral) and `packages/core/src/connectors/{postgres,typesense,meilisearch,search}/**` (per-engine)
+"A connector" (this page) means an engine-integration module. "The connector" means the Drizzle instance exposed as `this.connector` on a datasource or repository. The two senses are unrelated despite the shared word.
 
-## Why the split
+## In one example
 
-Before the restructure, `AbstractDataSource`/`AbstractRepository`/`AbstractEntity` were PostgreSQL-and-Drizzle-aware from the start - `pool`, `connector` (Drizzle instance), and SQL-shaped `TWhere`/`TFilter` lived directly in `src/base`. Adding typesense (a document search engine, not a relational database) exposed the coupling: it has no connection pool, doesn't speak SQL, and doesn't support transactions or row-level locking.
-
-The fix moves everything engine-specific out of `src/base` and into `src/connectors/<engine>/`, leaving `src/base` with only what every engine can implement:
-
-| Concept | Neutral (`src/base`) | PostgreSQL (`connectors/postgres`) | Typesense (`connectors/typesense`) |
-|---|---|---|---|
-| DataSource | `AbstractDataSource` | `AbstractPostgresDataSource` → `BasePostgresDataSource` | `AbstractSearchDataSource` → `BaseSearchDataSource` → `TypesenseDataSource` |
-| Entity | `AbstractEntity` | `BasePostgresEntity` (alias `BaseEntity`) | `BaseSearchEntity` |
-| Repository | `AbstractRepository` | `PostgresBaseRepository` → `ReadableRepository`/`PersistableRepository`/`DefaultCRUDRepository`/`SoftDeletableRepository` | `TypesenseBaseRepository` → `ReadableSearchRepository`/`PersistableSearchRepository`/`DefaultSearchRepository` |
-| Transactions | throws `NotSupported` by default | real transactions, 3 isolation levels | throws `NotSupported` |
-| Filtering | neutral `TWhere`/`TFilter` | SQL WHERE via Drizzle | typesense query DSL via `TypesenseQueryDialect` |
-
-## The neutral contract
-
-### `AbstractDataSource`
-
-`packages/core/src/base/datasources/abstract.ts` declares:
+The neutral `AbstractDataSource` has no SQL, no pool, no Drizzle. The postgres connector's `BasePostgresDataSource` adds all of that, and `@datasource({ driver })` names the concrete client class:
 
 ```typescript
-abstract class AbstractDataSource<
-  Settings extends object = {},
-  Schema extends TAnyDataSourceSchema = TAnyDataSourceSchema,
-  ConfigurableOptions extends object = {},
-> extends BaseHelper implements IDataSource<Settings, Schema, ConfigurableOptions> {
-  abstract configure(opts?: ConfigurableOptions): ValueOrPromise<void>;
-
-  getCapabilities(): IDataSourceCapabilities {
-    return { transactions: false };
-  }
-
-  async beginTransaction(_opts?: ITransactionOptions): Promise<ITransaction> {
-    return throwNotSupported({
-      scope: this.constructor.name,
-      feature: 'Transactions',
-      logger: this.logger,
-    });
-  }
+import { Pool } from 'pg';
+import { datasource } from '@venizia/ignis';
+import { BasePostgresDataSource } from '@venizia/ignis/postgres';
+import { NodePostgresDriver } from '@venizia/ignis/postgres/node-postgres';
+
+interface IDataSourceConfigs {
+  host: string;
+  port: number;
+  database: string;
+  user: string;
+  password: string;
 }
-```
-
-No `pool`, no `connector`, no SQL types. Every engine either overrides `getCapabilities()`/`beginTransaction()` (PostgreSQL) or accepts the neutral default (typesense).
-
-### `AbstractEntity`
 
-`packages/core/src/base/models/base.ts` declares the minimal entity contract, including `getIdType()`:
+@datasource({ driver: NodePostgresDriver })
+export class PostgresDataSource extends BasePostgresDataSource<IDataSourceConfigs> {
+  override configure(): void {
+    this.client = new Pool(this.settings);
+  }
 
-```typescript
-abstract class AbstractEntity extends BaseHelper {
-  getIdType(): TIdSchemaType {
-    return 'string';
+  override getConnectionString(): string {
+    const { host, port, user, password, database } = this.settings;
+    return `postgresql://${user}:${password}@${host}:${port}/${database}`;
   }
 }
 ```
 
-`BasePostgresEntity` overrides `getIdType()` based on the Drizzle column's actual data type; other engines can override it per their own ID representation.
+A Typesense datasource follows the same shape but extends the search connector's `BaseSearchDataSource` instead - no pool, no `getConnectionString()`, no transactions.
 
-### `AbstractRepository`
+## How it works
 
-`packages/core/src/base/repositories/core/abstract.ts`:
-
-```typescript
-abstract class AbstractRepository<
-  TDataObject extends object,
-  TPersistObject extends object = TDataObject,
-  TOptions extends IExtraOptions = IExtraOptions,
-> extends BaseHelper implements IPersistableRepository<TDataObject, TPersistObject, TOptions>
-```
+- **Three engine-neutral roots.** `packages/core-server/src/base` declares three roots every connector implements:
 
-Notice the generic parameters are named for their role (`TDataObject`, `TPersistObject`, `TOptions`), not for Drizzle concepts (`EntitySchema`, `TTableObject`) - those Drizzle-flavored names now belong to `PostgresBaseRepository` and its subclasses, which narrow `AbstractRepository`'s generics into a `TTableSchemaWithId`-based signature. `AbstractRepository` does **not** compose `FieldsVisibilityMixin`/`DefaultFilterMixin` - see [Repository Mixins](./repositories/mixins) for what happened to those.
+| Root | Purpose | Neutral default |
+|---|---|---|
+| `AbstractDataSource` | Connection ownership | No `pool`, no Drizzle `connector`; `beginTransaction()` throws `NotSupported` |
+| `AbstractEntity` | Model/schema contract | `name`, `getSchema()`, `getIdType()` |
+| `AbstractRepository` | Data access contract | Generics named for role (data/persist/options), not any one engine's vocabulary |
 
-### Neutral `TFilter`/`TWhere` and `ITransaction`
+- **Connectors narrow the roots into a real engine.** `packages/core-server/src/connectors/<engine>` adds engine-specific members:
 
-`packages/core/src/base/repositories/common/types.ts` and `packages/core/src/base/datasources/common/types.ts` define engine-neutral filter and transaction shapes shared by both connectors. `ITransaction` has no `connector` field - each engine that supports transactions (currently only PostgreSQL) narrows it with `IDatabaseTransaction`, whose `connector` is a `TRelationalConnector<Schema>` bound to the transaction's dedicated connection (the older `TNodePostgresTransactionConnector` name is kept as a `@deprecated` alias). Its `commit()`/`rollback()` **throw** on failure (a failed `COMMIT` no longer resolves as success) and destroy the poisoned connection rather than pooling it under the `node-postgres` driver; a failed `BEGIN` likewise destroys the acquired connection instead of leaking it. See [Transactions](/guides/core-concepts/persistent/transactions).
+| Connector | Adds | `getCapabilities()` |
+|---|---|---|
+| **postgres** (`connectors/postgres`) | `pool`, Drizzle `connector`, SQL-shaped `TWhere`/`TFilter`, real transactions with isolation levels | `{ transactions: true }` |
+| **search** (`connectors/search`) | Engine-neutral search base - no SQL, no transactions. Both typesense and meilisearch extend it (not `AbstractDataSource` directly), sharing one query-dialect and capability shape | inherited neutral default |
 
-## Dual-door exports
+- **`@datasource({ driver })` picks the concrete client, within an engine.** Postgres takes a driver **class** (`NodePostgresDriver` or `PostgresJsDriver`), never a driver-name string. A bundler packages values, not text, so a string would leave the peer dependency uninstalled.
+- **All engine clients stay optional peer dependencies.** Importing `@venizia/ignis/postgres` alone loads zero client libraries.
+- **Search connectors are subpath-only.** `@venizia/ignis/typesense` and `@venizia/ignis/meilisearch` are excluded from the root `@venizia/ignis` barrel. An app that never touches search never pulls a search client into its bundle.
 
-IGNIS ships both a **root barrel** (backward-compatible default) and **per-engine subpaths** (opt-in, tree-shakeable). Both resolve to the exact same compiled classes - there is no duplicated implementation.
+**Canonical names and aliases**
 
-```json
-// packages/core/package.json - exports map (persistence-relevant entries)
-{
-  ".": "dist/index.js",                                        // framework + postgres (compatibility default)
-  "./postgres": "dist/connectors/postgres/index.js",           // relational connector - loads ZERO client library
-  "./postgres/node-postgres": "dist/connectors/postgres/drivers/node-postgres.js", // concrete driver (value-imports `pg`)
-  "./postgres/postgres-js": "dist/connectors/postgres/drivers/postgres-js.js",     // concrete driver (value-imports `postgres`)
-  "./postgres/supabase": "dist/connectors/postgres/supabase/index.js",             // Supabase pooler + RLS helpers
-  "./search": "dist/connectors/search/index.js",               // engine-neutral search paradigm (no engine client)
-  "./typesense": "dist/connectors/typesense/index.js",         // subpath-only, optional peer
-  "./meilisearch": "dist/connectors/meilisearch/index.js"      // subpath-only, optional peer
-}
-```
+| Layer | Canonical (`Relational` paradigm) | Compatibility alias (`Postgres`-prefixed) |
+|---|---|---|
+| DataSource | `BaseRelationalDataSource` | `BasePostgresDataSource` |
+| Entity | `BaseRelationalEntity` | `BasePostgresEntity` |
+| Repository | `RelationalBaseRepository` | `PostgresBaseRepository` |
 
-> [!IMPORTANT] Concrete Postgres drivers are sub-path only
-> `pg` and `postgres` are **both optional peer dependencies** (`peerDependenciesMeta.pg.optional` / `.postgres.optional`). The root barrel, `@venizia/ignis/postgres`, and the drivers barrel (`connectors/postgres/drivers/index.ts`) load **zero** `pg`/`postgres` modules - the drivers barrel exports only the neutral `IRelationalDriver` contract. A concrete driver value-imports its own client, so reach it by its own sub-path only: `@venizia/ignis/postgres/node-postgres` or `@venizia/ignis/postgres/postgres-js`. There is no structural client sniffing: `@datasource({ driver })` names the driver **class** directly (`NodePostgresDriver`/`PostgresJsDriver`), and the base datasource's `wireDriverFromMetadata()` instantiates that named class over `this.client` lazily, on first `getConnector()`/`beginTransaction()`. Naming a class rather than a driver-name string is what carries `pg`/`postgres` into the app's bundle - a bundler only packages a real value reference, never text, and a bare side-effect import would not survive `sideEffects: false`. This is what keeps IGNIS from forcing a database client on a project that does not use one. See [Postgres Drivers & Supabase](/guides/core-concepts/persistent/postgres-drivers).
+Both names resolve to the same class - existing imports keep working.
 
-> [!NOTE] `internal/` modules are not public API
-> The search connectors (`search`/`typesense`/`meilisearch`) no longer re-export their `internal/` barrels (`SearchConnectorInternal`, `TypesenseInternal`, `MeilisearchInternal`) from the connector barrel. There is no sub-path export for them either, and package `exports` blocks deep imports - so no specifier reaches them from outside the package. They are implementation detail; if you need what they do, ask for a supported API.
+## Common tasks
 
-```typescript
-// packages/core/src/index.ts - root barrel
-export * from './base';
-export * from './common';
-export * from './components';
-export * from './connectors';   // -> connectors/index.ts: export * from './postgres';
-export * from './helpers';
-export * from './utilities';
-```
+### Pick an engine
 
-- **Root (`@venizia/ignis`)** re-exports the framework plus the `postgres` connector - existing code that imports `BaseDataSource`/`DefaultCRUDRepository` from the root keeps working unchanged.
-- **`@venizia/ignis/postgres`** re-exports the same connector in isolation, for code that wants to be explicit about which engine it depends on.
-- **`@venizia/ignis/typesense`** is **subpath-only** - it is deliberately excluded from the root barrel because `typesense` is an optional peer dependency (`"peerDependenciesMeta": { "typesense": { "optional": true } }`). Importing from the root must never force-install the typesense client for apps that don't use search.
+Import the connector for the engine you need - `@venizia/ignis/postgres` for a relational database with transactions, `@venizia/ignis/typesense` or `@venizia/ignis/meilisearch` for document search. The root `@venizia/ignis` barrel re-exports postgres for backward compatibility; search engines are always subpath-only:
 
 ```typescript
-// Both of these resolve to the same BasePostgresDataSource class:
+// Postgres: available at the root or the subpath - same class either way
 import { BaseDataSource } from '@venizia/ignis';
 import { BasePostgresDataSource } from '@venizia/ignis/postgres';
 
-// Typesense must be imported from its subpath - not available at the root:
-import { TypesenseDataSource, defineSearchCollection } from '@venizia/ignis/typesense';
+// Search engines: subpath only, never at the root
+import { TypesenseDataSource } from '@venizia/ignis/typesense';
+import { MeilisearchDataSource } from '@venizia/ignis/meilisearch';
 ```
 
-## Naming symmetry and compatibility aliases
-
-Each connector's canonical class names carry the engine name (`BasePostgresDataSource`, `BasePostgresEntity`), matching the pattern typesense follows (`TypesenseDataSource`). PostgreSQL additionally re-exports the pre-restructure names as aliases, so existing imports do not break:
-
-```typescript
-// packages/core/src/connectors/postgres/datasources/index.ts
-export { BasePostgresDataSource as BaseDataSource } from './base-datasource';
-
-// packages/core/src/connectors/postgres/models/index.ts
-export { BasePostgresEntity as BaseEntity } from './base';
-```
-
-| Canonical (new, engine-carrying) | Alias (compatibility) | Notes |
-|---|---|---|
-| `BasePostgresDataSource` | `BaseDataSource` | Same class, re-exported |
-| `BasePostgresEntity` | `BaseEntity` | Same class, re-exported |
-
-Write new code against the canonical names - they make it unambiguous which engine a class belongs to once you have more than one connector in play. Existing code using the aliases needs no changes.
+### Know what lives in base vs a connector
 
-## Adding a new engine connector
+- **Engine-specific -> connector.** A connection pool, query dialect, isolation level, or transaction lives in a connector, not `src/base`.
+- **Engine-universal -> base.** `src/base` only ever grows members every engine can implement, such as `getSchema()` or `getIdType()`.
+- **Litmus test.** Could typesense (no pool, no SQL, no transactions) implement it? If not, it belongs in the postgres connector, not the neutral root.
 
-To add a third connector (e.g. a Redis-backed cache connector), follow the shape the two existing connectors share:
+### Add a new engine connector
 
-1. **`src/connectors/<engine>/datasources/`** - `Abstract<Engine>DataSource extends AbstractDataSource`, then `Base<Engine>DataSource` (or a single concrete `<Engine>DataSource` if there's no useful abstract/base split) implementing `configure()`/`getConnectionString()`. Override `getCapabilities()` and `beginTransaction()` only if the engine truly supports transactions - otherwise inherit the neutral `NotSupported` default.
-2. **`src/connectors/<engine>/models/`** (if the engine needs entity definitions beyond plain objects) - extend `AbstractEntity`, override `getIdType()` if the engine's ID representation differs from the neutral default.
-3. **`src/connectors/<engine>/repositories/core/`** - `<Engine>BaseRepository extends AbstractRepository`, narrowing the generics to the engine's data/persist/options shapes, followed by a `Readable`/`Persistable`/`DefaultCRUD`-style tier ladder mirroring the postgres and typesense connectors. Any operation the engine cannot support (transactions, locks) should call `throwNotSupported({ scope, feature, logger: this.logger })` for consistency.
-4. **`src/connectors/<engine>/index.ts`** - barrel export for the connector.
-5. **`src/connectors/index.ts`** - add `export * from './<engine>'` **only if** the engine has no optional runtime peer dependency (mirroring `postgres`). If the engine driver is an optional peer (like `typesense`), leave it out of `connectors/index.ts` and register it as a subpath-only export in `package.json`.
-6. **`package.json` `exports`** - add `"./​<engine>": "dist/connectors/<engine>/index.js"`. If the driver is an optional peer, add it to `peerDependencies` + `peerDependenciesMeta.<driver>.optional: true`.
+- **Mirror the shape.** Under `src/connectors/<engine>/`, add a `datasources/` extending `AbstractDataSource` and a `repositories/core/` extending `AbstractRepository` with a `Readable`/`Persistable`/`DefaultCRUD`-style tier ladder. Add a `models/` extending `AbstractEntity` too, if the engine needs entity definitions.
+- **Override transactions only if supported.** Override `getCapabilities()` and `beginTransaction()` only if the engine truly supports transactions - otherwise inherit the neutral `NotSupported` default.
+- **Export as a subpath.** Add the connector to `package.json` `exports` as `./<engine>`. If its driver is an optional peer dependency, keep it out of `connectors/index.ts` and register it as a subpath-only export instead, mirroring typesense and meilisearch.
 
-## See Also
+## See also
 
-- **Related Concepts:**
-  - [DataSources](./datasources) - Full DataSource reference (neutral contract + PostgreSQL connector)
-  - [Models & Enrichers](./models) - Entity reference (neutral `AbstractEntity` + PostgreSQL `BasePostgresEntity`)
-  - [Repositories](./repositories/) - Repository hierarchy reference
-  - [Repository Mixins](./repositories/mixins) - Legacy mixins, now orphaned from the public API
+- [DataSources](./datasources) - the engine-neutral contract and the PostgreSQL connector in depth
+- [Models & Enrichers](./models) - the engine-neutral entity contract and the PostgreSQL connector's entity
+- [Repositories](/references/base/repositories/) - the CRUD layer built on top of a connector's datasource
+- [Filter System](/references/base/filter-system/) - querying through a connector's repository
+- [Persistent Layer](/guides/core-concepts/persistent/) - the guide these reference pages support
+- [Search & Typesense](/guides/core-concepts/persistent/search-typesense) - the typesense connector in depth
+- [Search & Meilisearch](/guides/core-concepts/persistent/search-meilisearch) - the meilisearch connector in depth
 
-- **Guides:**
-  - [Search & Typesense](/guides/core-concepts/persistent/search-typesense) - The typesense connector in depth
+**Files:**
 
-- **Best Practices:**
-  - [Architectural Patterns](/best-practices/architectural-patterns) - Layered architecture and separation of concerns
+- [`packages/core-server/src/base/datasources/abstract.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/core-server/src/base/datasources/abstract.ts) - neutral `AbstractDataSource`
+- [`packages/core-server/src/base/models/base.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/core-server/src/base/models/base.ts) - neutral `AbstractEntity`
+- [`packages/core-server/src/base/repositories/core/abstract.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/core-server/src/base/repositories/core/abstract.ts) - neutral `AbstractRepository`
+- [`packages/core-server/src/connectors/postgres/datasources/base.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/core-server/src/connectors/postgres/datasources/base.ts) - `BaseRelationalDataSource` (alias `BasePostgresDataSource`)
+- [`packages/core-server/src/connectors/search/datasources/base.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/core-server/src/connectors/search/datasources/base.ts) - `BaseSearchDataSource`, shared by typesense and meilisearch

package/content/references/base/controllers.md

@@ -12,14 +12,14 @@ Technical reference for REST controller classes - the foundation for creating HT
 > This page covers **REST controllers** (HTTP/JSON). For gRPC controllers using ConnectRPC, see the [gRPC Controllers Reference](./grpc-controllers.md).
 
 **Files:**
-- `packages/core/src/base/controllers/rest/abstract.ts` - Abstract base class
-- `packages/core/src/base/controllers/rest/base.ts` - Concrete base class
-- `packages/core/src/base/controllers/common/types.ts` - Shared types and interfaces
-- `packages/core/src/base/controllers/common/constants.ts` - Transport constants and headers
-- `packages/core/src/base/metadata/routes/rest.ts` - Route decorators (`@api`, `@get`, `@post`, etc.)
-- `packages/core/src/base/metadata/routes/controller.ts` - `@controller` decorator
-- `packages/core/src/base/controllers/factory/controller.ts` - CRUD controller factory
-- `packages/core/src/components/controller/rest/rest.component.ts` - RestComponent
+- `packages/core-server/src/base/controllers/rest/abstract.ts` - Abstract base class
+- `packages/core-server/src/base/controllers/rest/base.ts` - Concrete base class
+- `packages/core-server/src/base/controllers/common/types.ts` - Shared types and interfaces
+- `packages/core-server/src/base/controllers/common/constants.ts` - Transport constants and headers
+- `packages/core-server/src/base/metadata/routes/rest.ts` - Route decorators (`@api`, `@get`, `@post`, etc.)
+- `packages/core-server/src/base/metadata/routes/controller.ts` - `@controller` decorator
+- `packages/core-server/src/base/controllers/factory/controller.ts` - CRUD controller factory
+- `packages/core-server/src/components/controller/rest/rest.component.ts` - RestComponent
 
 ## Quick Reference
 
@@ -89,7 +89,7 @@ During `registerControllers()`, the application creates a `RestComponent` for RE
 
 ## `RestComponent`
 
-**File:** `packages/core/src/components/controller/rest/rest.component.ts`
+**File:** `packages/core-server/src/components/controller/rest/rest.component.ts`
 
 The `RestComponent` is responsible for discovering, configuring, and mounting all REST controllers onto the application's root Hono router. It is automatically instantiated by `BaseApplication.registerControllers()` when the REST transport is enabled.
 
@@ -400,7 +400,7 @@ interface ICustomizableRoutes<
 
 ## Route Decorators
 
-**File:** `packages/core/src/base/metadata/routes/rest.ts`
+**File:** `packages/core-server/src/base/metadata/routes/rest.ts`
 
 ### `@controller` Decorator
 
@@ -618,7 +618,7 @@ request: {
 
 ## Standard Headers and Constants
 
-**File:** `packages/core/src/base/controllers/common/constants.ts`
+**File:** `packages/core-server/src/base/controllers/common/constants.ts`
 
 ### `RestPaths`
 
@@ -644,7 +644,7 @@ class RestPaths {
 
 The `ControllerFactory` provides a static method `defineCrudController` to quickly generate a pre-configured CRUD controller for any given `AbstractEntity` subclass (e.g. `BaseEntity` for Postgres) and its corresponding repository.
 
-**File:** `packages/core/src/base/controllers/factory/controller.ts`
+**File:** `packages/core-server/src/base/controllers/factory/controller.ts`
 
 ### `static defineCrudController<TDataObject, TPersistObject = TDataObject, Routes extends ICustomizableRoutes = ICustomizableRoutes>(opts: ICrudControllerOptions<Routes>)`