@hanseltime/template-repo-sync 1.1.0 → 1.3.0

package/CHANGELOG.md

@@ -1,3 +1,17 @@
+# [1.3.0](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.2.0...v1.3.0) (2024-06-09)
+
+
+### Features
+
+* support specifying the template branch ([54bad24](https://github.com/HanseltimeIndustries/template-repo-sync/commit/54bad24935bb8609617b8d30409aa8a47fc64139))
+
+# [1.2.0](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.1.0...v1.2.0) (2024-03-11)
+
+
+### Features
+
+* adding comment-json support for merges ([c4393aa](https://github.com/HanseltimeIndustries/template-repo-sync/commit/c4393aaa6bf3bb985670f4377f2a370338450618))
+
 # [1.1.0](https://github.com/HanseltimeIndustries/template-repo-sync/compare/v1.0.1...v1.1.0) (2024-03-04)
 
 

package/README.md

@@ -13,6 +13,16 @@ best practice development patterns, then we naturally want to have a way to allo
 changes, while also having control over things that may be specifically changed due to their need to support something beyond the
 orgnaization standard.
 
+- [Template Sync](#template-sync)
+- [How to use this](#how-to-use-this)
+  - [Config file](#config-file)
+    - [File format](#file-format)
+    - [Example 1 - Using a custom plugin](#example-1---using-a-custom-plugin)
+    - [Example 2 - Using a custom plugin for some paths](#example-2---using-a-custom-plugin-for-some-paths)
+  - [From SHA/Tag directive](#from-shatag-directive)
+  - [Programmatic API](#programmatic-api)
+  <!-- Created with Markdown All In One VsCode Entension, rerun to update -->
+
 # How to use this
 
 This repository publishes a github action that can be used for ease of use in github. It also provides itself as an npm package
@@ -32,6 +42,10 @@ This library will always respect the overrides of the local template sync file i
 templates and their repos, will also provide a list of all files whose template sync behavior was either ignored or overridden by the local
 file. In this way, teams should be able to track (with a little extra CI/CD wiring) or at the very least, explicitly acknowledge a deviation.
 
+All config files have the ability to write custom merge plugins either in repo or published as packages for larger use.
+
+Please see the [plugins](./docs/merge-plugins/) documentation for more information beyond the simple examples in this readme.
+
 ### File format
 
 ```typescript

package/docs/merge-plugins/CURRENT_PLUGINS.md

@@ -0,0 +1,168 @@
+# Current Merge Plugins
+
+This document lists all merge plugins that are provided provided as defaults for certain file extensions.
+These should be found in the [plugins folder](src/plugins)
+
+- [Current Merge Plugins](#current-merge-plugins)
+  - [Json Merge Plugin](#json-merge-plugin)
+  - [Configuration Options:](#configuration-options)
+    - [Simple merge spec](#simple-merge-spec)
+    - [JsonPath config](#jsonpath-config)
+      - [Example](#example)
+    - [About Comments](#about-comments)
+
+<!-- Created with Markdown All In One VsCode Extension -->
+
+## Json Merge Plugin
+
+The json merge plugin allows you to configure jsonpath based merges on any .json file.
+
+## Configuration Options:
+
+### Simple merge spec
+
+At it's simplest, you can take advantage of lodash merge behavior by just specifying one of:
+
+- overwrite - the template completely overwrites the file
+- merge-template - keys are merged together with the template overwriting any matching keys on local file
+- merge-current - keys are merged together with the local file keeping any keys that match in the template
+
+Example config:
+
+```json
+{
+  "merge": {
+    ".json": {
+      "rules": [
+        {
+          "glob": "metadata.json",
+          "options": "merge-template"
+        },
+        {
+          "glob": "template-lock.json",
+          "options": "overwrite"
+        },
+        {
+          "glob": "package.json",
+          "options": "merge-current"
+        }
+      ]
+    }
+  }
+}
+```
+
+### JsonPath config
+
+If you would like further control over what merges within a .json file, you can actually specify, via way of jsonpath operators,
+the level of merge per field.
+
+A few rules:
+
+- Once you have provided jsonpath options, only the json path options (or new fields if the option is enabled) will be merged
+- jsonpaths are run from first to last. This means you can layer merges.
+
+```typescript
+interface Options {
+  /**
+   * If set to true, this means we won't add new properties from the template
+   */
+  ignoreNewProperties?: boolean;
+  /**
+   * If set to true, overwrite will apply undefined values as deleted for the jsonpaths
+   * or for values that are supposed to be merged on top of other values
+   */
+  missingIsDelete?: boolean;
+  /**
+   * Note, if multiple json paths match a rule, we pick the first one in the list that matches
+   */
+  paths: /**
+   * We only override jsonpaths.  Anything not specified is kept the same.
+   */
+  [jsonPath: `$.${string}`, options: BaseJsonMergeOptions][];
+}
+```
+
+#### Example
+
+This
+
+```json
+{
+  "merge": {
+    ".json": {
+      "rules": [
+        {
+          "glob": "metadata.json",
+          "options": {
+            "ignoreNewProperties": false,
+            "missingIsDelete:": true,
+            "paths": [
+              ["$.path", "template-merge"] // if we delete path in the template, it will delete the path
+            ]
+          }
+        },
+        {
+          "glob": "template-lock.json",
+          "options": "overwrite"
+        },
+        {
+          "glob": "package.json",
+          "options": {
+            "ignoreNewProperties": false,
+            "paths": [
+              ["$.scripts.*", "template-merge"],
+              ["$.scripts.specific-script", "template-currrent"] // We end up keeping the current template
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### About Comments
+
+There are numerous json files that support comments now; tsconfig.json is a prime example of this. In order to support this,
+this library makes use of comment-json for parsing and stringifying. This means that, minimally, you will not run into errors
+when merging commented json files (and that you can comment on your templatesync config files).
+
+One thing to note however, is that only include comments from the template if they are inside of an object that is being merged.
+This is because this plugin has not yet defined a good configuration for merging comments. If you run into a pertinent need for this,
+please feel free to open an issue and potentially contribute a fix in a PR.
+
+Example of comment merging:
+
+```json
+// In template
+
+{
+  // I have a comment here
+  "newField": 44,
+  "overridingField": {
+    // Comment in here
+    "value": "v",
+  }
+}
+
+// In the extending repo
+{
+  // My custom comment
+  "newField": 88,
+  "overridingField": 66
+}
+
+// After merging with template-merge
+{
+  // My custom comment
+  "newField": 44,
+  "overridingField": {
+    // Comment in here
+    "value": "v",
+  }
+}
+```
+
+From the above example, you can see that we specifically keep as many of the comments as possible from the extending repo
+and only the comment that was fully nested inside new value that we were adding was kept.

package/docs/merge-plugins/DEVELOPMENT.md

@@ -0,0 +1,129 @@
+# Plugin Development
+
+- [Plugin Development](#plugin-development)
+  - [Example: hello \<world\>](#example-hello-world) - [Example one: local file](#example-one-local-file) - [Example two: npm package](#example-two-npm-package)
+  <!-- Created with Markdown All In One VsCode Extension -->
+
+The templatesync.json and templatesync.local.config files make use of a `"merge"` property where you can
+customize the baseline behavior of ignore or overwrite from template.
+
+This library provides a set of typescript types for you to create additional plugins.
+
+At the core of it, you need to have a file or npm package that exposes the interface:
+
+```typescript
+export interface MergePlugin<PluginOptions> {
+  /**
+   * This method will be called when a file from the template and it's analog in the downstream repo
+   * have some differences.  The plugin must perform the merge and return the appropriate file contents
+   * as a string
+   *
+   * TODO: we may create a V2 plugin that could deal with large files and not pass around strings in memory,
+   * but for now, this is the current implementation
+   *
+   * @param current - The downstream repo's current file contents
+   * @param fromTemplateRepo - the current
+   * @param context - an object defining the context around the file and the specific options
+   */
+  merge(
+    current: string,
+    fromTemplateRepo: string,
+    context: MergeContext<PluginOptions>,
+  ): Promise<string>;
+  /**
+   * Given an options object for the merge, this validates the options object and returns error messages if there is anything wrong.
+   * @param options any json value that the user provided - must be validated against the expected options
+   */
+  validate(options: unknown): string[] | undefined;
+}
+```
+
+## Example: hello \<world>
+
+So let's say that we want to have a plugin that will take every file assigned to it, and just write hello {world} instead.
+(Not much of a merge, but so it goes)
+
+We will define and export an options object, validate function, and merge function
+
+```typescript
+// src/hello-plugin.ts
+import { MergePlugin, MergeContext } from '@hanseltime/template-repo-sync'
+
+export interface HelloOptions: {
+  /** the name of the world we're greeting */
+  world: string
+}
+
+export const validate: MergePlugin<HelloOptions>['validate'] = (options: unknown) => {
+  const errors: string[] = [];
+  // In our case, we have decided you HAVE to use an object
+  if (typeof options !== 'object') {
+    errors.push('must provide an object');
+    return errors;
+  }
+
+  // make sure there aren't extra keys
+  const { world, ...rest } = options;
+  const unknownKeys = Object.keys(rest);
+  if (unknownKeys.length > 0 ) {
+    errors.push(`Unexpected options keys: ${unknownKeys.join(' ')}`);
+  }
+
+  if (!world) {
+    errors.push(`Must provide a valid world value`);
+  }
+
+  return errors
+}
+
+export const merge:  MergePlugin<HelloOptions>['merge'] = async (current, fromTemplateRepo, options: HelloOptions) => {
+  // Note, we don't use the current with this simple plugin, but we would use the first 2 args normally
+  return `Hello ${options.world}`
+}
+
+```
+
+With all of that set up, as long as we have the package available to the pacakge manager running our script, we can use it:
+
+### Example one: local file
+
+```json
+{
+  "merge": {
+    ".txt": {
+      "plugin": "dist/hello-plugin.js", // Note, we make it point to the compiled .js so you will need to build and commit the file
+      "rules": [
+        {
+          "glob": "**/*",
+          "options": {
+            "world": "chad"
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+### Example two: npm package
+
+Let's assume that you published this as an npm package to @myscope/hello-merge. Once you have install the pacakge to the project, you can
+simply reference the package (assuming that it exposes the required functions as it's index file).
+
+```json
+{
+  "merge": {
+    ".txt": {
+      "plugin": "@myscope/hello-merge",
+      "rules": [
+        {
+          "glob": "**/*",
+          "options": {
+            "world": "chad"
+          }
+        }
+      ]
+    }
+  }
+}
+```

package/docs/merge-plugins/README.md

@@ -0,0 +1,85 @@
+# Merge Plugins
+
+The templatesync.json and templatesync.local.config files make use of a `"merge"` property where you can
+customize the baseline behavior of just ignoring or overwriting from the template.
+
+# Example Use Case
+
+One example of this behavior is around an npm package.json. If you were making a template for a particular
+set of boilerplate for an npm package, you would probably provide an example package.json like:
+
+```json
+{
+  "name": "<fill in your package name",
+  "description": "<fill in your package description",
+  "license": "MIT",
+  "scripts": {
+    "build": "tsc",
+    "publish": "our-artifact-script"
+  },
+  "devDependencies": {
+    "typescript": "^5.0.0",
+    "our-artifact-package": "^1.0.0"
+  }
+}
+```
+
+In this scenario, we expect the template user to declare their own `name` and `description`, and add their own
+`scripts` and `devDependencies`. However, we are hoping to make sure that the publish method and and its
+our-artifact-package are kept up-to-date on syncs. Because of this, we would make use of the default json merge
+plugin that is built-in with this library:
+
+```json
+{
+  "merge": {
+    ".json": {
+      // This uses the default plugin for json merges
+      "rules": [
+        {
+          "glob": "package.json",
+          "options": {
+            "paths": [
+              ["$.scripts.publish", "merge-template"], // Any changes to publish are so critical that we want them to sync
+              ["$.devDependencies", "merge-template"] // Always ensure dev dependency versions for our scripts are updated
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+As a repo maintainer, we can be sure that when people sync from our repo in its current state, the publish script
+and devDependency should be synced from package.json, without anything else!
+
+## What if the repo extender is annoyed?
+
+If the repo extender already made the decision to update the publish method for a good reason, they may find it
+tedious to constantly get their publish script overwritten and then have to undo it. Due to the local
+config file, they can specify their own merge configuration for the package.json.
+
+```json
+{
+  "merge": {
+    ".json": {
+      // This uses the default plugin for json merges
+      "rules": [
+        {
+          "glob": "package.json",
+          "options": {
+            "paths": [
+              ["$.devDependencies", "merge-template"] // Always ensure dev dependency versions for our scripts are updated
+            ]
+          }
+        }
+      ]
+    }
+  }
+}
+```
+
+This configuration will now override the template repo's merge and only allow devDependencies to be updated. We report
+this override as part of the output of the sync call, and for things like our
+[github action](https://github.com/HanseltimeIndustries/template-repo-sync-action), we format that output into the PR
+that is opened up when performing a sync.

package/lib/cjs/checkout-drivers/git-checkout.d.ts

@@ -0,0 +1,8 @@
+export declare function gitCheckout(options: {
+    /** The directory where we cloned to */
+    tmpDir: string;
+    /** The name of the remote that git checks out against */
+    remoteName: string;
+    /** The branch to checkout against */
+    branch: string;
+}): Promise<boolean>;

package/lib/cjs/checkout-drivers/git-checkout.js

@@ -0,0 +1,17 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.gitCheckout = void 0;
+const child_process_1 = require("child_process");
+async function gitCheckout(options) {
+    const { branch, remoteName, tmpDir } = options;
+    (0, child_process_1.execSync)(`git fetch ${remoteName} ${branch}`, {
+        cwd: tmpDir,
+        env: process.env,
+    });
+    (0, child_process_1.execSync)(`git checkout -b ${branch} --track ${remoteName}/${branch}`, {
+        cwd: tmpDir,
+        env: process.env,
+    });
+    return true;
+}
+exports.gitCheckout = gitCheckout;

package/lib/cjs/checkout-drivers/index.d.ts

@@ -0,0 +1,2 @@
+export * from "./git-checkout";
+export * from "./types";

package/lib/cjs/checkout-drivers/index.js

@@ -0,0 +1,18 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __exportStar = (this && this.__exportStar) || function(m, exports) {
+    for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+__exportStar(require("./git-checkout"), exports);
+__exportStar(require("./types"), exports);

package/lib/cjs/checkout-drivers/types.d.ts

@@ -0,0 +1,14 @@
+/**
+ * A function that will checkout a given "branch" after the repo has been
+ * "cloned" by a clone drvier.
+ *
+ * @returns true if the checkout succeeded
+ */
+export type TemplateCheckoutDriverFn = (options: {
+    /** The directory where we cloned to */
+    tmpDir: string;
+    /** The name of the remote that git checks out against */
+    remoteName: string;
+    /** The branch to checkout against */
+    branch: string;
+}) => Promise<boolean>;

package/lib/cjs/checkout-drivers/types.js

@@ -0,0 +1,2 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });

package/lib/cjs/clone-drivers/git-clone.d.ts

@@ -1 +1,2 @@
-export declare function gitClone(tmpDir: string, repoUrl: string): Promise<string>;
+import { CloneReturn } from "./types";
+export declare function gitClone(tmpDir: string, repoUrl: string): Promise<CloneReturn>;

package/lib/cjs/clone-drivers/git-clone.js

@@ -9,6 +9,9 @@ async function gitClone(tmpDir, repoUrl) {
         cwd: tmpDir,
         env: process.env,
     });
-    return (0, path_1.resolve)(tmpDir, CLONE_DIR);
+    return {
+        dir: (0, path_1.resolve)(tmpDir, CLONE_DIR),
+        remoteName: "origin",
+    };
 }
 exports.gitClone = gitClone;

package/lib/cjs/clone-drivers/types.d.ts

@@ -1,5 +1,16 @@
+/**
+ * @deprecated - return the remote name in the CloneReturn
+ */
+type CloneDir = string;
+export interface CloneReturn {
+    /** The directory where the clone occurred - absolute path to avoid working dir issues */
+    dir: string;
+    /** The name of the remote for the particular technology that you used - passed to checkout drivers */
+    remoteName: string;
+}
 /**
  * A function that clones the template repo into the provided tmpDir
  * and then returns the relative path within that directory to the template root
  */
-export type TemplateCloneDriverFn = (tmpDir: string, repoUrl: string) => Promise<string>;
+export type TemplateCloneDriverFn = (tmpDir: string, repoUrl: string) => Promise<CloneReturn | CloneDir>;
+export {};

package/lib/cjs/plugins/json-merge.js

@@ -1,13 +1,36 @@
 "use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || function (mod) {
+    if (mod && mod.__esModule) return mod;
+    var result = {};
+    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
+    __setModuleDefault(result, mod);
+    return result;
+};
 var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.merge = exports.validate = void 0;
 const lodash_merge_1 = __importDefault(require("lodash.merge"));
-const lodash_clonedeep_1 = __importDefault(require("lodash.clonedeep"));
 const jsonpath_1 = __importDefault(require("jsonpath"));
 const infer_json_indent_1 = require("../formatting/infer-json-indent");
+const commentJSON = __importStar(require("comment-json"));
 function stringOptionError(value) {
     if (value === "overwrite" ||
         value === "merge-template" ||
@@ -78,18 +101,18 @@ async function merge(current, fromTemplateRepo, context) {
     if (context.mergeArguments === "overwrite") {
         return fromTemplateRepo;
     }
-    const currentJson = JSON.parse(current);
-    const fromTemplateJson = JSON.parse(fromTemplateRepo);
+    const currentJson = commentJSON.parse(current);
+    const fromTemplateJson = commentJSON.parse(fromTemplateRepo);
     if (context.mergeArguments === "merge-current") {
         // Performs Lodash Merge with current as the override
-        return JSON.stringify((0, lodash_merge_1.default)(fromTemplateJson, currentJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
+        return commentJSON.stringify((0, lodash_merge_1.default)(fromTemplateJson, currentJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
     }
     if (context.mergeArguments === "merge-template") {
         // Performs Lodash Merge with current as the override
-        return JSON.stringify((0, lodash_merge_1.default)(currentJson, fromTemplateJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
+        return commentJSON.stringify((0, lodash_merge_1.default)(currentJson, fromTemplateJson), null, (0, infer_json_indent_1.inferJSONIndent)(current));
     }
     const { missingIsDelete, ignoreNewProperties, paths } = context.mergeArguments;
-    const returnJson = (0, lodash_clonedeep_1.default)(currentJson);
+    const returnJson = commentJSON.parse(current);
     paths.forEach((p) => {
         const [jPath, overrideType] = p;
         const fromTemplatePaths = new Map();
@@ -143,7 +166,7 @@ async function merge(current, fromTemplateRepo, context) {
             }
         });
     }
-    return JSON.stringify(returnJson, null, (0, infer_json_indent_1.inferJSONIndent)(current));
+    return commentJSON.stringify(returnJson, null, (0, infer_json_indent_1.inferJSONIndent)(current));
 }
 exports.merge = merge;
 /**

package/lib/cjs/template-sync.d.ts

@@ -2,8 +2,17 @@ import { Change } from "diff";
 import { TemplateCloneDriverFn } from "./clone-drivers";
 import { TemplateDiffDriverFn } from "./diff-drivers";
 import { TemplateRefDriverFn } from "./ref-drivers/types";
+import { TemplateCheckoutDriverFn } from "./checkout-drivers";
 export interface TemplateSyncOptions {
+    /**
+     * This is the url of the template repo
+     */
     repoUrl: string;
+    /**
+     * Optional Branch to check out - if not specified, this checks out the
+     * default branch of the template repo
+     */
+    branch?: string;
     /**
      * The directory for cloning our template repo into via the cloneDriver
      */
@@ -29,6 +38,10 @@ export interface TemplateSyncOptions {
      * Defaults to using git current ref
      */
     currentRefDriver?: TemplateRefDriverFn;
+    /**
+     * Defaults to using git checkout driver
+     */
+    checkoutDriver?: TemplateCheckoutDriverFn;
 }
 export interface TemplateSyncReturn {
     /**