@hanseltime/template-repo-sync 1.0.1 → 1.2.0

package/CHANGELOG.md

@@ -1,3 +1,17 @@
+# [1.2.0](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.1.0...v1.2.0) (2024-03-11)
+
+
+### Features
+
+* adding comment-json support for merges ([c4393aa](https://github.com/HanseltimeIndustries/template-repo-sync/commit/c4393aaa6bf3bb985670f4377f2a370338450618))
+
+# [1.1.0](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.0.1...v1.1.0) (2024-03-04)
+
+
+### Features
+
+* adding updateRef Option ([b553e65](https://github.com/HanseltimeIndustries/template-repo-sync/commit/b553e65ebce59777ccb80728d540c5734d41cab7))
+
 ## [1.0.1](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.0.0...v1.0.1) (2024-03-03)
 
 

package/README.md

@@ -1,4 +1,4 @@
-# Template Sync Action
+# Template Sync
 
 This npm package seeks to provide further granularity for people hoping to maintain a base template repo in github that
 is either imported or used as a literal template repo.
@@ -13,6 +13,16 @@ best practice development patterns, then we naturally want to have a way to allo
 changes, while also having control over things that may be specifically changed due to their need to support something beyond the
 orgnaization standard.
 
+- [Template Sync](#template-sync)
+- [How to use this](#how-to-use-this)
+  - [Config file](#config-file)
+    - [File format](#file-format)
+    - [Example 1 - Using a custom plugin](#example-1---using-a-custom-plugin)
+    - [Example 2 - Using a custom plugin for some paths](#example-2---using-a-custom-plugin-for-some-paths)
+  - [From SHA/Tag directive](#from-shatag-directive)
+  - [Programmatic API](#programmatic-api)
+  <!-- Created with Markdown All In One VsCode Entension, rerun to update -->
+
 # How to use this
 
 This repository publishes a github action that can be used for ease of use in github. It also provides itself as an npm package
@@ -32,6 +42,10 @@ This library will always respect the overrides of the local template sync file i
 templates and their repos, will also provide a list of all files whose template sync behavior was either ignored or overridden by the local
 file. In this way, teams should be able to track (with a little extra CI/CD wiring) or at the very least, explicitly acknowledge a deviation.
 
+All config files have the ability to write custom merge plugins either in repo or published as packages for larger use.
+
+Please see the [plugins](./docs/merge-plugins/) documentation for more information beyond the simple examples in this readme.
+
 ### File format
 
 ```typescript
@@ -61,7 +75,11 @@ export interface Config {
 }
 ```
 
-### Example 1
+### Example 1 - Using a custom plugin
+
+In this scenario, you have installed a package that exposes the correct plugin interface for handling .ini file contents in
+your implementing repository and set up this templatesync.local config file. Because of this, we can be assured that .ini files
+in the local repo will be merged using the plugin we specified.
 
 ```typescript
 {
@@ -70,7 +88,34 @@ export interface Config {
         ".ini": {
             // If you are running under pacakge manager like yarn or npm,
             // you can provide a valid pacakge or .js fil from your package to run
-            driver: 'my-installed-npm-package',
+            plugin: 'my-installed-npm-package',
+        }
+    }
+}
+```
+
+### Example 2 - Using a custom plugin for some paths
+
+Just like in example 1, we have installed a plugin that exposes the correct plugin interface. Now though,
+instead of applying that plugin to all '.ini' files, we are saying that, for this particular set of .ini
+files, only the ones in custom-configs/ will use this merge operator.
+
+```typescript
+{
+
+    merge: {
+        ".ini": {
+            // If you are running under pacakge manager like yarn or npm,
+            // you can provide a valid pacakge or .js fil from your package to run
+            plugin: 'my-installed-npm-package',
+            rule: [
+                {
+                    glob: 'custom-configs/**',
+                    options: {
+                        myPluginParam: 'some parameter',
+                    }
+                }
+            ]
         }
     }
 }
@@ -91,3 +136,35 @@ but it does mean that you will only see the changes to files that are newer than
 
 As always, you can remove the SHA/Tag from your local config and this will trigger a full sync in the event that you made the wrong
 assumption about merging templates correctly.
+
+```typescript
+{
+    afterRef: 'git sha',
+    merge: {
+        ".ini": {
+            // If you are running under pacakge manager like yarn or npm,
+            // you can provide a valid pacakge or .js fil from your package to run
+            plugin: 'my-installed-npm-package',
+            rule: [
+                {
+                    glob: 'custom-configs/**',
+                    options: {
+                        myPluginParam: 'some parameter',
+                    }
+                }
+            ]
+        }
+    }
+}
+```
+
+## Programmatic API
+
+The programmatic api for this package is centered around `templateSync`. The way the function is written, we try to
+allow escape hatches for other styles of comparison in the form of "TemplateDriver" functions. As part of the current
+implementation, all of these `drivers` represent git actions, but for the sake of expandability, may be set up to evaluate
+things like helm chart renderings (at least that is the hope). If you write a driver, please consider contributing it back.
+
+Please see [template-sync](./src/template-sync.ts) for the most up to date options.
+
+TODO: we should update use case examples

package/docs/merge-plugins/CURRENT_PLUGINS.md

@@ -0,0 +1,168 @@
+# Current Merge Plugins
+
+This document lists all merge plugins that are provided provided as defaults for certain file extensions.
+These should be found in the [plugins folder](src/plugins)
+
+- [Current Merge Plugins](#current-merge-plugins)
+  - [Json Merge Plugin](#json-merge-plugin)
+  - [Configuration Options:](#configuration-options)
+    - [Simple merge spec](#simple-merge-spec)
+    - [JsonPath config](#jsonpath-config)
+      - [Example](#example)
+    - [About Comments](#about-comments)
+
+<!-- Created with Markdown All In One VsCode Extension -->
+
+## Json Merge Plugin
+
+The json merge plugin allows you to configure jsonpath based merges on any .json file.
+
+## Configuration Options:
+
+### Simple merge spec
+
+At it's simplest, you can take advantage of lodash merge behavior by just specifying one of:
+
+- overwrite - the template completely overwrites the file
+- merge-template - keys are merged together with the template overwriting any matching keys on local file
+- merge-current - keys are merged together with the local file keeping any keys that match in the template
+
+Example config:
+
+```json
+{
+  "merge": {
+    ".json": {
+      "rules": [
+        {
+          "glob": "metadata.json",
+          "options": "merge-template"
+        },
+        {
+          "glob": "template-lock.json",
+          "options": "overwrite"
+        },
+        {
+          "glob": "package.json",
+          "options": "merge-current"
+        }
+      ]
+    }
+  }
+}
+```
+
+### JsonPath config
+
+If you would like further control over what merges within a .json file, you can actually specify, via way of jsonpath operators,
+the level of merge per field.
+
+A few rules:
+
+- Once you have provided jsonpath options, only the json path options (or new fields if the option is enabled) will be merged
+- jsonpaths are run from first to last. This means you can layer merges.
+
+```typescript
+interface Options {
+  /**
+   * If set to true, this means we won't add new properties from the template
+   */
+  ignoreNewProperties?: boolean;
+  /**
+   * If set to true, overwrite will apply undefined values as deleted for the jsonpaths
+   * or for values that are supposed to be merged on top of other values
+   */
+  missingIsDelete?: boolean;
+  /**
+   * Note, if multiple json paths match a rule, we pick the first one in the list that matches
+   */
+  paths: /**
+   * We only override jsonpaths.  Anything not specified is kept the same.
+   */
+  [jsonPath: `$.${string}`, options: BaseJsonMergeOptions][];
+}
+```
+
+#### Example
+
+This
+
+```json
+{
+  "merge": {
+    ".json": {
+      "rules": [
+        {
+          "glob": "metadata.json",
+          "options": {
+            "ignoreNewProperties": false,
+            "missingIsDelete:": true,
+            "paths": [
+              ["$.path", "template-merge"] // if we delete path in the template, it will delete the path
+            ]
+          }
+        },
+        {
+          "glob": "template-lock.json",
+          "options": "overwrite"
+        },
+        {
+          "glob": "package.json",
+          "options": {
+            "ignoreNewProperties": false,
+            "paths": [
+              ["$.scripts.*", "template-merge"],
+              ["$.scripts.specific-script", "template-currrent"] // We end up keeping the current template
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### About Comments
+
+There are numerous json files that support comments now; tsconfig.json is a prime example of this. In order to support this,
+this library makes use of comment-json for parsing and stringifying. This means that, minimally, you will not run into errors
+when merging commented json files (and that you can comment on your templatesync config files).
+
+One thing to note however, is that only include comments from the template if they are inside of an object that is being merged.
+This is because this plugin has not yet defined a good configuration for merging comments. If you run into a pertinent need for this,
+please feel free to open an issue and potentially contribute a fix in a PR.
+
+Example of comment merging:
+
+```json
+// In template
+
+{
+  // I have a comment here
+  "newField": 44,
+  "overridingField": {
+    // Comment in here
+    "value": "v",
+  }
+}
+
+// In the extending repo
+{
+  // My custom comment
+  "newField": 88,
+  "overridingField": 66
+}
+
+// After merging with template-merge
+{
+  // My custom comment
+  "newField": 44,
+  "overridingField": {
+    // Comment in here
+    "value": "v",
+  }
+}
+```
+
+From the above example, you can see that we specifically keep as many of the comments as possible from the extending repo
+and only the comment that was fully nested inside new value that we were adding was kept.

package/docs/merge-plugins/DEVELOPMENT.md

@@ -0,0 +1,129 @@
+# Plugin Development
+
+- [Plugin Development](#plugin-development)
+  - [Example: hello \<world\>](#example-hello-world) - [Example one: local file](#example-one-local-file) - [Example two: npm package](#example-two-npm-package)
+  <!-- Created with Markdown All In One VsCode Extension -->
+
+The templatesync.json and templatesync.local.config files make use of a `"merge"` property where you can
+customize the baseline behavior of ignore or overwrite from template.
+
+This library provides a set of typescript types for you to create additional plugins.
+
+At the core of it, you need to have a file or npm package that exposes the interface:
+
+```typescript
+export interface MergePlugin<PluginOptions> {
+  /**
+   * This method will be called when a file from the template and it's analog in the downstream repo
+   * have some differences.  The plugin must perform the merge and return the appropriate file contents
+   * as a string
+   *
+   * TODO: we may create a V2 plugin that could deal with large files and not pass around strings in memory,
+   * but for now, this is the current implementation
+   *
+   * @param current - The downstream repo's current file contents
+   * @param fromTemplateRepo - the current
+   * @param context - an object defining the context around the file and the specific options
+   */
+  merge(
+    current: string,
+    fromTemplateRepo: string,
+    context: MergeContext<PluginOptions>,
+  ): Promise<string>;
+  /**
+   * Given an options object for the merge, this validates the options object and returns error messages if there is anything wrong.
+   * @param options any json value that the user provided - must be validated against the expected options
+   */
+  validate(options: unknown): string[] | undefined;
+}
+```
+
+## Example: hello \<world>
+
+So let's say that we want to have a plugin that will take every file assigned to it, and just write hello {world} instead.
+(Not much of a merge, but so it goes)
+
+We will define and export an options object, validate function, and merge function
+
+```typescript
+// src/hello-plugin.ts
+import { MergePlugin, MergeContext } from '@hanseltime/template-repo-sync'
+
+export interface HelloOptions: {
+  /** the name of the world we're greeting */
+  world: string
+}
+
+export const validate: MergePlugin<HelloOptions>['validate'] = (options: unknown) => {
+  const errors: string[] = [];
+  // In our case, we have decided you HAVE to use an object
+  if (typeof options !== 'object') {
+    errors.push('must provide an object');
+    return errors;
+  }
+
+  // make sure there aren't extra keys
+  const { world, ...rest } = options;
+  const unknownKeys = Object.keys(rest);
+  if (unknownKeys.length > 0 ) {
+    errors.push(`Unexpected options keys: ${unknownKeys.join(' ')}`);
+  }
+
+  if (!world) {
+    errors.push(`Must provide a valid world value`);
+  }
+
+  return errors
+}
+
+export const merge:  MergePlugin<HelloOptions>['merge'] = async (current, fromTemplateRepo, options: HelloOptions) => {
+  // Note, we don't use the current with this simple plugin, but we would use the first 2 args normally
+  return `Hello ${options.world}`
+}
+
+```
+
+With all of that set up, as long as we have the package available to the pacakge manager running our script, we can use it:
+
+### Example one: local file
+
+```json
+{
+  "merge": {
+    ".txt": {
+      "plugin": "dist/hello-plugin.js", // Note, we make it point to the compiled .js so you will need to build and commit the file
+      "rules": [
+        {
+          "glob": "**/*",
+          "options": {
+            "world": "chad"
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### Example two: npm package
+
+Let's assume that you published this as an npm package to @myscope/hello-merge. Once you have install the pacakge to the project, you can
+simply reference the package (assuming that it exposes the required functions as it's index file).
+
+```json
+{
+  "merge": {
+    ".txt": {
+      "plugin": "@myscope/hello-merge",
+      "rules": [
+        {
+          "glob": "**/*",
+          "options": {
+            "world": "chad"
+          }
+        }
+      ]
+    }
+  }
+}
+```

package/docs/merge-plugins/README.md

@@ -0,0 +1,85 @@
+# Merge Plugins
+
+The templatesync.json and templatesync.local.config files make use of a `"merge"` property where you can
+customize the baseline behavior of just ignoring or overwriting from the template.
+
+# Example Use Case
+
+One example of this behavior is around an npm package.json. If you were making a template for a particular
+set of boilerplate for an npm package, you would probably provide an example package.json like:
+
+```json
+{
+  "name": "<fill in your package name",
+  "description": "<fill in your package description",
+  "license": "MIT",
+  "scripts": {
+    "build": "tsc",
+    "publish": "our-artifact-script"
+  },
+  "devDependencies": {
+    "typescript": "^5.0.0",
+    "our-artifact-package": "^1.0.0"
+  }
+}
+```
+
+In this scenario, we expect the template user to declare their own `name` and `description`, and add their own
+`scripts` and `devDependencies`. However, we are hoping to make sure that the publish method and and its
+our-artifact-package are kept up-to-date on syncs. Because of this, we would make use of the default json merge
+plugin that is built-in with this library:
+
+```json
+{
+  "merge": {
+    ".json": {
+      // This uses the default plugin for json merges
+      "rules": [
+        {
+          "glob": "package.json",
+          "options": {
+            "paths": [
+              ["$.scripts.publish", "merge-template"], // Any changes to publish are so critical that we want them to sync
+              ["$.devDependencies", "merge-template"] // Always ensure dev dependency versions for our scripts are updated
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+As a repo maintainer, we can be sure that when people sync from our repo in its current state, the publish script
+and devDependency should be synced from package.json, without anything else!
+
+## What if the repo extender is annoyed?
+
+If the repo extender already made the decision to update the publish method for a good reason, they may find it
+tedious to constantly get their publish script overwritten and then have to undo it. Due to the local
+config file, they can specify their own merge configuration for the package.json.
+
+```json
+{
+  "merge": {
+    ".json": {
+      // This uses the default plugin for json merges
+      "rules": [
+        {
+          "glob": "package.json",
+          "options": {
+            "paths": [
+              ["$.devDependencies", "merge-template"] // Always ensure dev dependency versions for our scripts are updated
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+This configuration will now override the template repo's merge and only allow devDependencies to be updated. We report
+this override as part of the output of the sync call, and for things like our
+[github action](https://github.com/HanseltimeIndustries/template-repo-sync-action), we format that output into the PR
+that is opened up when performing a sync.

package/lib/cjs/plugins/json-merge.js

@@ -1,13 +1,36 @@
 "use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || function (mod) {
+    if (mod && mod.__esModule) return mod;
+    var result = {};
+    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
+    __setModuleDefault(result, mod);
+    return result;
+};
 var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.merge = exports.validate = void 0;
 const lodash_merge_1 = __importDefault(require("lodash.merge"));
-const lodash_clonedeep_1 = __importDefault(require("lodash.clonedeep"));
 const jsonpath_1 = __importDefault(require("jsonpath"));
 const infer_json_indent_1 = require("../formatting/infer-json-indent");
+const commentJSON = __importStar(require("comment-json"));
 function stringOptionError(value) {
     if (value === "overwrite" ||
         value === "merge-template" ||
@@ -78,18 +101,18 @@ async function merge(current, fromTemplateRepo, context) {
     if (context.mergeArguments === "overwrite") {
         return fromTemplateRepo;
     }
-    const currentJson = JSON.parse(current);
-    const fromTemplateJson = JSON.parse(fromTemplateRepo);
+    const currentJson = commentJSON.parse(current);
+    const fromTemplateJson = commentJSON.parse(fromTemplateRepo);
     if (context.mergeArguments === "merge-current") {
         // Performs Lodash Merge with current as the override
-        return JSON.stringify((0, lodash_merge_1.default)(fromTemplateJson, currentJson), null, 4);
+        return commentJSON.stringify((0, lodash_merge_1.default)(fromTemplateJson, currentJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
     }
     if (context.mergeArguments === "merge-template") {
         // Performs Lodash Merge with current as the override
-        return JSON.stringify((0, lodash_merge_1.default)(currentJson, fromTemplateJson), null, 4);
+        return commentJSON.stringify((0, lodash_merge_1.default)(currentJson, fromTemplateJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
     }
     const { missingIsDelete, ignoreNewProperties, paths } = context.mergeArguments;
-    const returnJson = (0, lodash_clonedeep_1.default)(currentJson);
+    const returnJson = commentJSON.parse(current);
     paths.forEach((p) => {
         const [jPath, overrideType] = p;
         const fromTemplatePaths = new Map();
@@ -143,7 +166,7 @@ async function merge(current, fromTemplateRepo, context) {
             }
         });
     }
-    return JSON.stringify(returnJson, null, (0, infer_json_indent_1.inferJSONIndent)(current));
+    return commentJSON.stringify(returnJson, null, (0, infer_json_indent_1.inferJSONIndent)(current));
 }
 exports.merge = merge;
 /**

package/lib/cjs/ref-drivers/git-current-ref.d.ts

@@ -0,0 +1,3 @@
+export declare function gitCurrentRef(options: {
+    rootDir: string;
+}): Promise<string>;

package/lib/cjs/ref-drivers/git-current-ref.js

@@ -0,0 +1,12 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.gitCurrentRef = void 0;
+const child_process_1 = require("child_process");
+async function gitCurrentRef(options) {
+    return (0, child_process_1.execSync)(`git rev-parse HEAD`, {
+        cwd: options.rootDir,
+    })
+        .toString()
+        .trim();
+}
+exports.gitCurrentRef = gitCurrentRef;

package/lib/cjs/ref-drivers/index.d.ts

@@ -0,0 +1 @@
+export * from "./git-current-ref";

package/lib/cjs/ref-drivers/index.js

@@ -0,0 +1,17 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./git-current-ref"), exports);

package/lib/cjs/ref-drivers/types.d.ts

@@ -0,0 +1,10 @@
+/**
+ * A function that operates within the root dir and returns the
+ * current ref for its state.  This abstracts the idea of getting
+ * the current commit sha, so that things like other custom templating
+ * frameworks can provide their own ref
+ */
+export type TemplateRefDriverFn = (options: {
+    /** The root dir where we want to get the "current" ref */
+    rootDir: string;
+}) => Promise<string>;

package/lib/cjs/ref-drivers/types.js

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/lib/cjs/template-sync.d.ts

@@ -1,6 +1,7 @@
 import { Change } from "diff";
 import { TemplateCloneDriverFn } from "./clone-drivers";
 import { TemplateDiffDriverFn } from "./diff-drivers";
+import { TemplateRefDriverFn } from "./ref-drivers/types";
 export interface TemplateSyncOptions {
     repoUrl: string;
     /**
@@ -11,6 +12,11 @@ export interface TemplateSyncOptions {
      * The repo directory path that we are going to merge toward
      */
     repoDir: string;
+    /**
+     * If set to true, template sync will apply the current ref
+     * of the template repo to afterRef
+     */
+    updateAfterRef?: boolean;
     /**
      * Defaults to using git clone
      */
@@ -19,6 +25,10 @@ export interface TemplateSyncOptions {
      * Defaults to using git diff
      */
     diffDriver?: TemplateDiffDriverFn;
+    /**
+     * Defaults to using git current ref
+     */
+    currentRefDriver?: TemplateRefDriverFn;
 }
 export interface TemplateSyncReturn {
     /**