@travetto/compiler 3.0.0-rc.32 → 3.0.0-rc.34

package/README.md

@@ -1,28 +1,82 @@
 <!-- This file was generated by @travetto/doc and should not be modified directly -->
 <!-- Please modify https://github.com/travetto/travetto/tree/main/module/compiler/DOC.ts and execute "npx trv doc" to rebuild -->
 # Compiler
-## Compiler
+## The compiler infrastructure for the Travetto framework
 
 **Install: @travetto/compiler**
 ```bash
 npm install @travetto/compiler
+
+# or
+
+yarn add @travetto/compiler
 ```
 
-This module expands upon [Typescript](https://typescriptlang.org), with supplemental functionality:
+This module expands upon the [Typescript](https://typescriptlang.org) compiler, with the additional features:
    
-   *  Read `tsconfig.json` from the project directory to provide 
-   *  Supports on-the-fly compilation, nothing needs to be compiled ahead of time
-   *  Enhanced AST transformations, and transformer registration [object Object]
-   *  Intelligent caching of source files to minimize recompilation
-   *  Support for detecting changes in sources files at runtime
-   *  Allows for hot-reloading of classes during development    
-      *  Utilizes `es2015` `Proxy`s to allow for swapping out implementation at runtime
-Additionally, there is support for common AST transformations via [Transformation](https://github.com/travetto/travetto/tree/main/module/transformer#readme "Functionality for AST transformations, with transformer registration, and general utils")
-## Debugging
-When dealing with transformers, logging is somewhat tricky as the compiler executes before the code is loaded.  To that end, the file `compiler.log` is created in the cache directory during the compilation process. This is a location that transformers should be free to log to, for debugging, and any additional feedback.
+   *  Integration with the [Transformation](https://github.com/travetto/travetto/tree/main/module/transformer#readme "Functionality for AST transformations, with transformer registration, and general utils") module, allowing for rich, type-aware transformations
+   *  Automatic conversion to either [Ecmascript Module](https://nodejs.org/api/esm.html) or [CommonJS](https://nodejs.org/api/modules.html) based on the [Package JSON](https://docs.npmjs.com/cli/v9/configuring-npm/package-json) `type` value
+   *  Removal of type only imports which can break [Ecmascript Module](https://nodejs.org/api/esm.html)-style output
+   *  Automatic addition of `.js` extension to imports to also support  [Ecmascript Module](https://nodejs.org/api/esm.html)-style output
+
+Beyond the [Typescript](https://typescriptlang.org) compiler functionality, the module provides the primary entry point into the development process.  
 
 ## CLI
 
-The module provides the ability to clear the compilation cache to handle any inconsistencies that may arise.
+The cli, [trv](https://github.com/travetto/travetto/tree/main/module/compiler/bin/trv.js#L60) is a compilation aware entry point, that has the ability to check for active builds, and ongoing watch operations to ensure only one process is building at a time.  Within the framework, regardless of mono-repo or not, always builds the entire project.  With the efficient caching behavior, this leads to generally a minimal overhead but allows for centralization of all operations.
+
+The CLI supports the following operations:
+
+   
+   *  `clean` - Removes the output folder, and if `-a` is also passed, will also clean out the compiler folder
+   *  `build` - Will attempt to build the project.  If the project is already built, will return immediately.  If the project is being built somewhere else, will wait until a build is completed.
+   *  `watch` - If nothing else is watching, will start the watch operation.  Otherwise will return immediately.
+   *  `manifest` - Will produce a manifest. If no file is passed in the command line arguments, will output to stdout
+   *  `<other>` - Will be delegated to the [Command Line Interface](https://github.com/travetto/travetto/tree/main/module/cli#readme "CLI infrastructure for Travetto framework") entry point after a successful build.
+
+In addition to the normal output, the compiler supports an environment variable `TRV_BUILD` that supports the following values: `debug`, `info`, `warn` or `none`.  This provides different level of logging during the build process which is helpful to diagnose any odd behaviors.  When invoking an unknown command (e.g. `<other>` from above), the default level is `warn`.  Otherwise the default logging level is `info`.
+
+**Terminal: Sample trv output with debug logging**
+```bash
+$ TRV_BUILD=debug trv build
+
+2029-03-14T04:00:00.618Z [lock           ] watch pid=000000 Started
+2029-03-14T04:00:00.837Z [lock           ] watch pid=000000 Already running, and has built
+2029-03-14T04:00:01.510Z [lock           ] watch pid=000000 Completed
+2029-03-14T04:00:02.450Z [build          ] Successfully built
+```
+
+**Terminal: Sample trv output with default log level**
+```bash
+$ trv build
+```
+
+## Compilation Architecture
+
+The compiler will move through the following phases on a given compilation execution:
+   
+   *  `Bootstrapping` - Initial compilation of [Compiler](https://github.com/travetto/travetto/tree/main/module/compiler#readme "The compiler infrastructure for the Travetto framework")'s `support/*.ts` files
+   *  `Lock Management` - Manages cross-process interaction to ensure single compiler
+   *  `Build Compiler` - Leverages [Typescript](https://typescriptlang.org) to build files needed to execute compiler
+   *  `Build Manifest` - Produces the manifest for the given execution
+   *  `Build Transformers` - Leverages [Typescript](https://typescriptlang.org) to compile all transformers defined in the manifest
+   *  `Produce Manifest Delta` - Compare the output file system with the manifest to determine what needs to be compiled
+   *  `Clear all output if needed` - When the compiler source or transformers change, invalidate the entire output
+   *  `Persist Manifest(s)` - Ensure the manifest is available for the compiler to leverage. Multiple will be written if in a monorepo
+   *  `Invoke Compiler` - Run [Typescript](https://typescriptlang.org) compiler with the aforementioned enhancements
+
+### Bootstrapping
+
+Given that the framework is distributed as [Typescript](https://typescriptlang.org) only files, there is a bootstrapping problem that needs to be mitigated.  The [trv](https://github.com/travetto/travetto/tree/main/module/compiler/bin/trv.js#L60) entrypoint, along with a small context utility in [Manifest](https://github.com/travetto/travetto/tree/main/module/manifest#readme "Support for project indexing, manifesting, along with file watching") are the only [Javascript](https://developer.mozilla.org/en-US/docs/Web/JavaScript) files needed to run the project.  The [trv](https://github.com/travetto/travetto/tree/main/module/compiler/bin/trv.js#L60) entry point will compile `@travetto/compiler/support/*` files as the set that is used at startup.  These files are also accessible to the compiler as they get re-compiled after the fact.  
+
+### Lock Management
+
+The compiler supports invocation from multiple locations at the same time, and provides a layer of orchestration to ensure a single process is building at a time.  For a given project, there are four main states:
+
+   
+   *  No Watch - Building
+   *  Watch    - No Build
+   *  Watch    - Building
+   *  Inactive / Stale
 
-TODO: Describe cli behavior
+Depending on what state the project is in (depending on various processes), will influence what the supporting tooling should do. [LockManager](https://github.com/travetto/travetto/tree/main/module/compiler/support/lock.ts#L25) represents the majority of the logic for tracking various states, and informing what action should happen when in the above states.

package/bin/trv.js

@@ -16,6 +16,10 @@ const COMPILER_FILES = [...['launcher', 'transpile', 'lock', 'log', 'lock-pinger
  * @return {Promise<import('@travetto/compiler/support/launcher').launch>}
  */
 const $getLauncher = async (ctx) => {
+  const tsconfigFile = path.resolve(ctx.workspacePath, 'tsconfig.json');
+  if (!(await fs.stat(tsconfigFile).catch(() => undefined))) {
+    await fs.writeFile(tsconfigFile, JSON.stringify({ extends: '@travetto/compiler/tsconfig.trv.json' }), 'utf8');
+  }
   const compPkg = createRequire(path.resolve(ctx.workspacePath, 'node_modules')).resolve('@travetto/compiler/package.json');
   const files = [];
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@travetto/compiler",
-  "version": "3.0.0-rc.32",
-  "description": "Compiler",
+  "version": "3.0.0-rc.34",
+  "description": "The compiler infrastructure for the Travetto framework",
   "keywords": [
     "compiler",
     "travetto",
@@ -31,12 +31,12 @@
   },
   "dependencies": {
     "@parcel/watcher": "^2.1.0",
-    "@travetto/manifest": "^3.0.0-rc.17",
-    "@travetto/terminal": "^3.0.0-rc.10",
-    "@travetto/transformer": "^3.0.0-rc.21"
+    "@travetto/manifest": "^3.0.0-rc.19",
+    "@travetto/terminal": "^3.0.0-rc.11",
+    "@travetto/transformer": "^3.0.0-rc.23"
   },
   "peerDependencies": {
-    "@travetto/cli": "^3.0.0-rc.22"
+    "@travetto/cli": "^3.0.0-rc.25"
   },
   "peerDependenciesMeta": {
     "@travetto/cli": {

package/src/watch.ts

@@ -44,10 +44,8 @@ export class CompilerWatcher {
     for (const ctx of [...contexts, this.#state.manifest]) {
       const newManifest = await ManifestUtil.buildManifest(ctx);
       for (const file of files) {
-        if (
-          file.folderKey && file.moduleFile && file.type &&
-          file.mod in newManifest.modules && file.folderKey in newManifest.modules[file.mod].files
-        ) {
+        if (file.folderKey && file.moduleFile && file.type && file.mod in newManifest.modules) {
+          newManifest.modules[file.mod].files[file.folderKey] ??= [];
           newManifest.modules[file.mod].files[file.folderKey]!.push([file.moduleFile, file.type, Date.now()]);
         }
       }
@@ -74,7 +72,9 @@ export class CompilerWatcher {
 
     return async ({ file: sourceFile, action }: WatchEvent, folder: string): Promise<void> => {
       const mod = mods[folder];
-      const moduleFile = sourceFile.includes(mod.sourceFolder) ? sourceFile.split(`${mod.sourceFolder}/`)[1] : sourceFile;
+      const moduleFile = mod.sourceFolder ?
+        (sourceFile.includes(mod.sourceFolder) ? sourceFile.split(`${mod.sourceFolder}/`)[1] : sourceFile) :
+        sourceFile.replace(`${this.#state.manifest.workspacePath}/`, '');
       switch (action) {
         case 'create': {
           const fileType = ManifestModuleUtil.getFileType(moduleFile);
@@ -82,8 +82,8 @@ export class CompilerWatcher {
             mod: mod.name,
             modFolder: folder,
             moduleFile,
-            folderKey: ManifestModuleUtil.getFolderKey(sourceFile),
-            type: ManifestModuleUtil.getFileType(sourceFile)
+            folderKey: ManifestModuleUtil.getFolderKey(moduleFile),
+            type: ManifestModuleUtil.getFileType(moduleFile)
           });
           if (CompilerUtil.validFile(fileType)) {
             await this.#rebuildManifestsIfNeeded();