cloesce 0.0.4-unstable.1 → 0.0.4-unstable.10

package/README.md

@@ -2,28 +2,27 @@
 
 Cloesce is a full stack compiler for the Cloudflare developer platform, allowing class definitions in high level languages to serve as a metadata basis to create a database schema, backend REST API, frontend API client, and Cloudflare infrastructure (as of v0.0.4, D1 + Workers).
 
-Cloesce is working towards an alpha MVP (v0.1.0), with the general milestones being [here](https://cloesce.pages.dev/schreiber/v0.1.0_milestones/).
+Cloesce is working towards a stable alpha MVP (v0.1.0), with the general milestones being [here](https://cloesce.pages.dev/schreiber/v0.1.0_milestones/).
 
 Internal documentation going over design decisions and general thoughts for each milestone can be found [here](https://cloesce.pages.dev/).
 
-# Documentation `v0.0.4`
+# Documentation
 
 ## Getting Started
 
 `v0.0.4` supports only Typescript-to-Typescript projects. An example project is shown [here](https://github.com/bens-schreiber/cloesce/tree/main/examples).
 
-1. NPM
+### 1) NPM
 
-- Create an NPM project and install cloesce
+Create an NPM project and install cloesce
 
 ```sh
-# check https://www.npmjs.com/package/cloesce/v/0.0.4-unstable.0?activeTab=versions for the most recent patch
-npm i cloesce@0.0.4-unstable.0
+npm i cloesce@0.0.4-unstable.10
 ```
 
-2. TypeScript
+### 2) TypeScript
 
-- Create a `tsconfig.json` with the following values:
+Create a `tsconfig.json` with the following values:
 
 ```json
 {
@@ -39,9 +38,9 @@ npm i cloesce@0.0.4-unstable.0
 }
 ```
 
-3. Cloesce Config
+### 3) Cloesce Config
 
-- Create a `cloesce.config.json` with the following keys:
+Create a `cloesce.config.json` with your desired configuration:
 
 ```json
 {
@@ -51,7 +50,7 @@ npm i cloesce@0.0.4-unstable.0
 }
 ```
 
-4. Vite
+### 4) Vite
 
 To prevent CORS issues, a Vite proxy can be used for the frontend:
 
@@ -71,9 +70,11 @@ export default defineConfig({
 });
 ```
 
-5. Wrangler Config
+Middleware support for CORS is also supported (see Middleware section).
 
-- `v0.0.4` will generate the required areas of your wrangler config. A full config looks like this:
+### 5) Wrangler Config
+
+Cloesce will generate any missing `wrangler.toml` values (or the file if missing). A minimal `wrangler.toml` looks like this:
 
 ```toml
 compatibility_date = "2025-10-02"
@@ -86,12 +87,11 @@ database_id = "..."
 database_name = "example"
 ```
 
-## A Simple Model
+## Cloesce Models
 
 A model is a type which represents:
 
 - a database table,
-- database views
 - REST API
 - Client API
 - Cloudflare infrastructure (D1 + Workers)
@@ -121,13 +121,18 @@ export class Horse {
 - `@POST` reveals the method as an API endpoint with the `POST` HTTP Verb.
 - All Cloesce models need to be under a `.cloesce.ts` file.
 
-To compile this model into a working full stack application, Cloesce must undergo both **compilation** and **migrations**. Compilation is the process of extracting the metadata language that powers Cloesce, ensuring it is a valid program, and then producing code to orchestrate the program across different domains (database, backend, frontend, cloud). Migrations utilize the history of validated metadata to create SQL code, translating the evolution of your models.
+To compile this model into a working full stack application, Cloesce must undergo both **compilation** and **migrations**.
+
+Compilation is the process of extracting the metadata language that powers Cloesce, ensuring it is a valid program, and then producing code to orchestrate the program across different domains (database, backend, frontend, cloud).
 
-To compile, run `npx cloesce compile`.
+Migrations utilize the history of validated metadata to create SQL code, translating the evolution of your models.
 
-To create a migration, run `npx cloesce migrate <name>`.
+### Compiling
 
-After running the above commands, you will have a full project capable of being ran with:
+- `npx cloesce compile`
+- `npx cloesce migrate <name>`.
+
+After running the above commands, you will have a full project capable of being ran with Wrangler:
 
 ```sh
 # Apply the generated migrations
@@ -137,17 +142,36 @@ npx wrangler d1 migrations apply <db-name>
 npx wrangler dev
 ```
 
-Note the output in the `.generated/` dir. These values should not be committed to git, as they depend on the file system of the machine running it.
+### Compiled Artifacts
+
+#### `.generated/`
+
+These values should not be committed to git, as they depend on the file system of the machine running it.
 
 - `client.ts` is an importable API with all of your backend types and endpoints
 - `workers.ts` is the workers entrypoint.
 - `cidl.json` is the working metadata for the project
 
-Note the output in `migrations`, ex after running `npx cloesce migrate Initial`
+#### `migrations`
+
+After running `npx cloesce migrate <name>`, a new migration will be created in the `migrations/` folder. For example, after creating a migration called `Initial`, you will see:
 
 - `<date>_Initial.json` contains all model information necessary from SQL
 - `<date>_Initial.sql` contains the acual SQL migrations. In this early version of Cloesce, it's important to check migrations every time.
 
+#### Supported Column Types
+
+Model columns must directly map to SQLite columns. The supported TypeScript types are:
+
+- `number` => `Real` not null
+- `string` => `Text` not null
+- `boolean` and `Boolean` => `Integer` not null
+- `Integer` => `Integer` not null
+- `Date` => `Text` (ISO formatted) not null
+- `N | null` => `N` (nullable)
+
+Blob types will be added in v0.0.5 when R2 support is added.
+
 ## Features
 
 ### Wrangler Environment
@@ -155,12 +179,10 @@ Note the output in `migrations`, ex after running `npx cloesce migrate Initial`
 In order to interact with your database, you will need to define a WranglerEnv
 
 ```ts
-// horse.cloesce.ts
-
 import { WranglerEnv } from "cloesce/backend";
 
 @WranglerEnv
-export class Env {
+export class MyEnv {
   db: D1Database; // only one DB is supported for now-- make sure it matches the name in `wrangler.toml`
 
   // you can also define values in the toml under [[vars]]
@@ -168,7 +190,7 @@ export class Env {
 }
 ```
 
-The wrangler environment is dependency injected into your method calls:
+Your WranglerEnv can then be injected into any model method using the `@Inject` decorator:
 
 ```ts
 @D1
@@ -177,7 +199,7 @@ export class Horse {
   id: number;
 
   @POST
-  async neigh(@Inject env: WranglerEnv): Promise<string> {
+  async neigh(@Inject env: MyEnv): Promise<string> {
     await env.db.prepare(...);
 
     return `i am ${this.name}, this is my horse noise`;
@@ -185,10 +207,9 @@ export class Horse {
 }
 ```
 
-### Foreign Keys, One to One, Data Sources
+### Foreign Key Column
 
-Complex model relationships are permitted via the `@ForeignKey`, `@OneToOne / @OneToMany @ManyToMany` and `@DataSource` decorators.
-Foreign keys are scalar attributes which must reference some other model's primary key:
+Reference another model via a foreign key using the `@ForeignKey` decorator:
 
 ```ts
 @D1
@@ -207,7 +228,9 @@ export class Person {
 }
 ```
 
-This representation is true to the underlying SQL table: `Person` has a column `dogId` which is a foreign key to `Dog`. Cloesce allows you to actually join these tables together in your model representation:
+### One to One
+
+Cloesce allows you to relate models via `1:1` relationships using the `@OneToOne` decorator. It requires that a foreign key already exists on the model.
 
 ```ts
 @D1
@@ -229,7 +252,7 @@ export class Person {
 }
 ```
 
-In `v0.0.4`, there are no defaults, only very explicit decisons. Because of that, navigation properties won't exist at runtime unless you tell them to. Cloesce does this via a `DataSource`, which describes the foreign key dependencies you wish to include. All scalar properties are included by default and cannot be excluded.
+In `v0.0.4`, there are no defaults, only very explicit decisons. Because of that, navigation properties won't exist at runtime unless you tell them to. Cloesce does this via a `Data Source`, which describes the foreign key dependencies you wish to include. All scalar properties are included by default and cannot be excluded.
 
 ```ts
 @D1
@@ -256,39 +279,9 @@ export class Person {
 }
 ```
 
-Data sources are just SQL views and can be invoked in your queries. They are aliased in such a way that its similiar to object properties. The frontend chooses which datasource to use in it's API client. `null` is a valid option, meaning no joins will occur.
-
-```ts
-@D1
-export class Person {
-  @PrimaryKey
-  id: number;
-
-  @ForeignKey(Dog)
-  dogId: number;
-
-  @OneToOne("dogId")
-  dog: Dog | undefined;
-
-  @DataSource
-  static readonly default: IncludeTree<Person> = {
-    dog: {},
-  };
-
-  @GET
-  static async get(id: number, @Inject env: WranglerEnv): Promise<Person> {
-    let records = await env.db
-      .prepare("SELECT * FROM [Person.default] WHERE [Person.id] = ?") // Person.default is the SQL view generated from the IncludeTree
-      .bind(id)
-      .run();
-
-    let persons = Orm.fromSql(Person, records.results, Person.default);
-    return persons.value[0];
-  }
-}
-```
+Data sources describe how foreign keys should be joined on model hydration (i.e. when invoking any instantiated method). They are composed of an `IncludeTree<T>`, a recursive type composed of the relationships you wish to include. All scalar properties are always included.
 
-Note that the `get` code can be simplified using CRUD methods or ORM primitives.
+Note that `DataSourceOf` is added implicitly to all instantiated methods if no data source parameter is defined.
 
 ### One to Many
 
@@ -352,6 +345,24 @@ export class Course {
 }
 ```
 
+### DataSourceOf<T>
+
+If it is important to determine what data source the frontend called the instantiated method with, the type `DataSourceOf<T>` allows explicit data source parameters:
+
+```ts
+@D1
+class Foo {
+  ...
+
+  @POST
+  bar(ds: DataSourceOf<Foo>) {
+    // ds = "DataSource1" | "DataSource2" | ... | "none"
+  }
+}
+```
+
+Data sources are implicitly added to all instantiated methods if no data source parameter is defined.
+
 ### ORM Methods
 
 Cloesce provides a suite of ORM methods for getting, listing, updating and inserting models.
@@ -374,6 +385,44 @@ class Horse {
 
 #### List, Get
 
+Both methods take an optional `IncludeTree<T>` parameter to specify what relationships in the generated CTE.
+
+```ts
+@D1
+export class Person {
+  @PrimaryKey
+  id: number;
+
+  @ForeignKey(Dog)
+  dogId: number;
+
+  @OneToOne("dogId")
+  dog: Dog | undefined;
+
+  @DataSource
+  static readonly default: IncludeTree<Person> = {
+    dog: {},
+  };
+}
+```
+
+Running `Orm.listQuery` with the data source `Person.default` would produce the CTE:
+
+```sql
+WITH "Person_view" AS (
+  SELECT
+      "Person"."id"      AS "id",
+      "Person"."dogId"   AS "dogId",
+      "Dog"."id"         AS "dog.id"
+  FROM
+      "Person"
+  LEFT JOIN
+      "Dog" ON "Person"."dogId" = "Dog"."id"
+) SELECT * FROM "Person_view"
+```
+
+Example usages:
+
 ```ts
 @D1
 class Horse {
@@ -381,23 +430,47 @@ class Horse {
   @GET
   static async get(@Inject { db }: Env, id: number): Promise<Horse> {
     const orm = Orm.fromD1(db);
-    return (await orm.get(Horse, id, "default")).value;
+    return (await orm.get(Horse, id, Horse.default)).value;
   }
 
   @GET
   static async list(@Inject { db }: Env): Promise<Horse[]> {
     const orm = Orm.fromD1(db);
-    return (await orm.list(Horse, "default")).value;
+    return (await orm.list(Horse, {})).value;
   }
 }
 ```
 
+`list` takes an optional `from` parameter to modify the source of the list query. This is useful in filtering / limiting results.
+
+```ts
+await orm.list(
+  Horse,
+  Horse.default,
+  "SELECT * FROM Horse ORDER BY name LIMIT 10"
+);
+```
+
+produces SQL
+
+```sql
+WITH "Horse_view" AS (
+  SELECT
+      "Horse"."id"      AS "id",
+      "Horse"."name"    AS "name"
+  FROM
+      (SELECT * FROM Horse ORDER BY name LIMIT 10) as "Horse"
+) SELECT * FROM "Horse_view"
+```
+
 ### CRUD Methods
 
-Generic GET, POST, PATCH (and in a future version, DEL) boilerplate methods do not need to be copied around. Cloesce supports CRUD generation, a syntactic sugar that adds the methods to the compiler output.
+Generic `GET, POST, PATCH` (and in a future version, DEL) boilerplate methods do not need to be copied around. Cloesce supports CRUD generation, a syntactic sugar that adds the methods to the compiler output.
+
+The `SAVE` method is an `upsert`, meaning it both inserts and updates in the same query.
 
 ```ts
-@CRUD(["POST", "GET", "LIST"])
+@CRUD(["SAVE", "GET", "LIST"])
 @D1
 export class CrudHaver {
   @PrimaryKey
@@ -415,6 +488,76 @@ static async get(
 ): Promise<HttpResult<CrudHaver>>
 ```
 
+### Middleware
+
+Cloesce supports middleware at the global level (before routing to a model+method), the model level (before validation) and the method level (before hydration). Middleware also exposes read/write access to the dependency injection instance that all models use.
+
+Middleware is capable of exiting from the Cloesce Router early with an HTTP Result.
+
+An example of all levels of middleware is below. All middleware must be defined in the file `app.cloesce.ts` which exports a `CloesceApp` instance as default.
+
+```ts
+@PlainOldObject
+export class InjectedThing {
+  value: string;
+}
+
+@WranglerEnv
+export class Env {
+  db: D1Database;
+}
+
+@D1
+@CRUD(["POST"])
+export class Model {
+  @PrimaryKey
+  id: number;
+
+  @GET
+  static blockedMethod() {}
+
+  @GET
+  static getInjectedThing(@Inject thing: InjectedThing): InjectedThing {
+    return thing;
+  }
+}
+
+const app: CloesceApp = new CloesceApp();
+
+app.onRequest((request: Request, env, ir) => {
+  if (request.method === "POST") {
+    return { ok: false, status: 401, message: "POST methods aren't allowed." };
+  }
+});
+
+app.onModel(Model, (request, env, ir) => {
+  ir.set(InjectedThing.name, {
+    value: "hello world",
+  });
+});
+
+app.onMethod(Model, "blockedMethod", (request, env, ir) => {
+  return { ok: false, status: 401, message: "Blocked method" };
+});
+
+app.onResponse(async (request, env, di, response: Response) => {
+  // basic CORS, allow all origins
+  response.headers.set("Access-Control-Allow-Origin", "*");
+  response.headers.set(
+    "Access-Control-Allow-Methods",
+    "GET, POST, PUT, DELETE, OPTIONS"
+  );
+  response.headers.set(
+    "Access-Control-Allow-Headers",
+    "Content-Type, Authorization"
+  );
+});
+
+export default app;
+```
+
+With this middleware, all POST methods will be blocked, and all methods for the model `Model` will be able to inject `InjectedThing`,and `blockedMethod` will return a 401. Additionally, all responses will have CORS headers.
+
 ### Plain Old Objects
 
 Simple non-model objects can be returned and serialized from a model method:
@@ -467,10 +610,11 @@ class Foo {
 ## Integration Tests
 
 - Regression tests: `cargo run --bin test regression`
-- Pass fail extractor tests: `cargo run --bin test run-fail`
 
 Optionally, pass `--check` if new snapshots should not be created.
 
+To target a specific fixture, pass `--fixture folder_name`
+
 To update integration snapshots, run:
 
 - `cargo run --bin update`

package/dist/cli.js

@@ -22,30 +22,28 @@ const cmds = subcommands({
             },
             handler: async (args) => {
                 const config = loadCloesceConfig(process.cwd(), args.debug);
-                if (!config.workersUrl || !config.clientUrl) {
-                    console.error("Error: `workersUrl` and `clientUrl` must be defined in cloesce.config.json");
+                if (!config.workersUrl) {
+                    console.error("Error: `workersUrl`` must be defined in cloesce.config.json");
                     process.exit(1);
                 }
                 // Creates a `cidl.json` file. Exits the process on failure.
                 await extract({ debug: args.debug });
                 const outputDir = config.outputDir ?? ".generated";
-                const allConfig = {
-                    name: "all",
+                const generateConfig = {
+                    name: "generate",
                     wasmFile: "generator.wasm",
                     args: [
                         "generate",
-                        "all",
                         path.join(outputDir, "cidl.pre.json"),
                         path.join(outputDir, "cidl.json"),
                         "wrangler.toml",
                         path.join(outputDir, "workers.ts"),
                         path.join(outputDir, "client.ts"),
-                        config.clientUrl,
                         config.workersUrl,
                     ],
                 };
                 // Runs a generator command. Exits the process on failure.
-                await generate(allConfig);
+                await generate(generateConfig);
             },
         }),
         extract: command({
@@ -83,6 +81,10 @@ const cmds = subcommands({
                     short: "d",
                     description: "Show debug output",
                 }),
+                skipTsCheck: flag({
+                    long: "skipTsCheck",
+                    description: "Skip TypeScript compilation checks",
+                }),
             },
             handler: async (args) => {
                 await extract({ ...args });
@@ -138,20 +140,22 @@ const cmds = subcommands({
         }),
     },
 });
-async function extract(opts) {
+async function extract(args) {
     const root = process.cwd();
     const projectRoot = process.cwd();
-    const config = loadCloesceConfig(projectRoot, opts.debug);
-    const searchPaths = opts.inp ? [opts.inp] : (config.paths ?? [root]);
+    const config = loadCloesceConfig(projectRoot, args.debug);
+    const searchPaths = args.inp ? [args.inp] : (config.paths ?? [root]);
     const outputDir = config.outputDir ?? ".generated";
-    const outPath = opts.out ?? path.join(outputDir, "cidl.pre.json");
-    const truncate = opts.truncateSourcePaths ?? config.truncateSourcePaths ?? false;
-    const cloesceProjectName = opts.projectName ??
+    const outPath = args.out ?? path.join(outputDir, "cidl.pre.json");
+    const truncate = args.truncateSourcePaths ?? config.truncateSourcePaths ?? false;
+    const cloesceProjectName = args.projectName ??
         config.projectName ??
         readPackageJsonProjectName(projectRoot);
     const project = new Project({
         compilerOptions: {
             strictNullChecks: true,
+            experimentalDecorators: true,
+            emitDecoratorMetadata: true,
         },
     });
     findCloesceProject(root, searchPaths, project);
@@ -159,16 +163,25 @@ async function extract(opts) {
     if (fileCount === 0) {
         new ExtractorError(ExtractorErrorCode.MissingFile);
     }
-    if (opts.debug)
+    if (args.debug)
         console.log(`Found ${fileCount} .cloesce.ts files`);
+    // Run typescript compiler checks to before extraction
+    if (!args.skipTsCheck) {
+        const diagnostics = project.getPreEmitDiagnostics();
+        if (diagnostics.length > 0) {
+            console.error("TypeScript errors detected in provided files:");
+            console.error(project.formatDiagnosticsWithColorAndContext(diagnostics));
+            process.exit(1);
+        }
+    }
     try {
-        const extractor = new CidlExtractor(cloesceProjectName, "v0.0.3");
+        const extractor = new CidlExtractor(cloesceProjectName, "v0.0.4");
         const result = extractor.extract(project);
-        if (!result.ok) {
+        if (result.isLeft()) {
             console.error(formatErr(result.value));
             process.exit(1);
         }
-        let ast = result.value;
+        let ast = result.unwrap();
         if (truncate) {
             ast.wrangler_env.source_path =
                 "./" + path.basename(ast.wrangler_env.source_path);
@@ -189,11 +202,11 @@ async function extract(opts) {
         const json = JSON.stringify(ast, null, 4);
         fs.mkdirSync(path.dirname(outPath), { recursive: true });
         fs.writeFileSync(outPath, json);
-        console.log(`CIDL generated successfully at ${outPath}`);
+        console.log(`CIDL extracted to ${outPath}`);
         return { outPath, projectName: cloesceProjectName };
     }
     catch (err) {
-        console.error("Critical uncaught error. Submit a ticket to https://github.com/bens-schreiber/cloesce: ", err?.message ?? err);
+        console.error("Critical uncaught error in generator. \nSubmit a ticket to https://github.com/bens-schreiber/cloesce\n\n", err?.message ?? "No error message.", "\n", err?.stack ?? "No error stack.");
         process.exit(1);
     }
 }
@@ -293,10 +306,13 @@ function formatErr(e) {
     const { description, suggestion } = getErrorInfo(e.code);
     const contextLine = e.context ? `Context: ${e.context}\n` : "";
     const snippetLine = e.snippet ? `${e.snippet}\n\n` : "";
-    return `==== CLOESCE ERROR ====
+    return `
+==== CLOESCE ERROR ====
 Error [${ExtractorErrorCode[e.code]}]: ${description}
 Phase: TypeScript IDL Extraction
-${contextLine}${snippetLine}Suggested fix: ${suggestion}`;
+${contextLine}${snippetLine}Suggested fix: ${suggestion}
+
+`;
 }
 run(cmds, process.argv.slice(2)).catch((err) => {
     console.error(err);