@flisk-sh/flisk-import 0.13.0 → 0.15.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -17,8 +17,10 @@ flisk import . --yes # write service.json files and project setup
17
17
  - **Keeps:** `stop_grace_period` (`container.stopGracePeriod`, such as `30s`), the `aliases` of every network (`container.networkAliases`), `extra_hosts` in list or map form (`container.extraHosts`, such as `host.docker.internal:host-gateway`) and `ulimits` (`container.ulimits`). A service renamed to kebab-case (`lychee_api` becomes `lychee-api`) keeps its Compose name as a network alias, so `http://lychee_api:8000` in another service still resolves. `container_name` is dropped with a note: Flisk names containers itself. These fields, an image with variables and `on-failure:N` need the Flisk core release that reads them; an older core does not apply them.
18
18
  - **Keeps:** the service names (`db` stays `db`, so it stays the hostname), the build context and Dockerfile (or the image), the container port as written (3000, 80, 5432), `depends_on`, and an HTTP health path. Databases and brokers are private, with no proxy route.
19
19
  - **Runtime keys:** named volumes and bind mounts inside the project (`volumes`), `cap_add`, `cap_drop`, `init`, `read_only`, `user`, `security_opt` and `shm_size` (`container`), extra published ports (`extraPorts`, on the same host address as in Compose: `"80:80"` stays on every interface, `"127.0.0.1:5432:5432"` on loopback, and another address or a `${VAR:-default}` or `${VAR-default}` host side as `publish`), `network_mode: service:<name>` (`sidecarOf`) and `restart`.
20
- - **Environment:** values that cannot be secrets go into the committed `service.json` `env`: numbers, `true`/`false`, another imported service's address without a user, password or query (`http://invidious:3000`, `redis:6379`), and pure `${VAR}` references. Every other name, and any name that looks like a credential (`PASSWORD`, `TOKEN`, `KEY`, ...), goes into `secrets.required`, with its literal value copied to the project `.env`, which is added to `.gitignore`. Values in `.env` already are kept.
21
- - **Keeps:** a `command` or `entrypoint` on an image as the `CMD`/`ENTRYPOINT` of a small `Dockerfile` next to `service.json` (Flisk sets no container command; a string is split into exec-form arguments as Compose splits it, with no shell, so `command: server /export` on `minio/minio` stays the arguments of its `ENTRYPOINT`; only a string with `$` variables or unbalanced quotes runs through `sh -c`, and the plan says so), a `healthcheck` as `healthCheck.command`, a published database port as `extraPorts`, and `restart: on-failure:N` with its retry limit (N from 1 to 100).
20
+ - **Environment:** the committed `service.json` `env` gets every pure Compose expression as written, whatever its name (`${DB_PASSWORD:-password}`, `${PORT-8080}`, `${A:-${B:-file}}`), so a fresh clone keeps the defaults, and every literal (`sqlite`, `worker`, `lychee_db`, `http://invidious:3000`) unless its name looks like a credential (`PASSWORD`, `TOKEN`, `KEY`, ...) or the value holds one (a URL with a user or password, a token query parameter, a long random string, a PEM block). Those go into `secrets.required`, with its literal value copied to the project `.env`, which is added to `.gitignore`. Values in `.env` already are kept.
21
+ - **Round trip:** a routed port that Compose also publishes (`"${APP_PORT:-8000}:8000"`) stays in `extraPorts` too, and a `healthcheck` stays as `healthCheck.command` (with `interval`, `timeout`, `retries` and `start_period` as `startPeriod`) even when an HTTP path was read, so `flisk-export compose` writes a file that `docker compose` starts the same way (LycheeOrg/Lychee round-trips with no difference but names and aliases). A service with no `restart` keeps none, so Flisk runs it with its default `unless-stopped` (unlike Compose, Flisk then restarts it after a crash or a reboot; the plan says so). `restart: "no"` is written only when Compose sets it or another service waits for this one with `condition: service_completed_successfully`, because Flisk runs a `restart: "no"` service as a one-shot job and waits for it to exit, `healthcheck: { disable: true }` or `test: ["NONE"]` becomes `healthCheck: false`, a value that mixes `${VAR}` with literal text (`${SERVER_NAME:-localhost}, php:80`) stays in `env` unless it could hold a credential, `*_FILE` paths (`POSTGRES_PASSWORD_FILE=/run/secrets/pg`) stay in `env`, and the Compose project name (`name:` or the folder's, such as `pl_mail`) is recorded as `composeProjectNames.<stack>` in `.flisk/project.json`, so `flisk-export compose` keeps the container names. A generated Dockerfile for an image with variables declares them as `ARG NAME=default` before `FROM`.
22
+ - **A tracked `.env`:** a repo that commits its `.env` (Symfony) keeps it as it is, and it is not added to `.gitignore`; values that must stay local go to `.env.local` when git already ignores it, else to `.flisk/.env.import` (added to `.gitignore`), and each service that needs them reads that file as `envFile`.
23
+ - **Keeps:** a `command` or `entrypoint` on an image as the `CMD`/`ENTRYPOINT` of a small `Dockerfile` next to `service.json` (Flisk sets no container command; a string is split into exec-form arguments as Compose splits it, with no shell, so `command: server /export` on `minio/minio` stays the arguments of its `ENTRYPOINT`; `$$` becomes `$` as Compose passes it to the container; only a string with Compose variables (`${VAR}`, filled from `.env` by Compose, by the container's shell here) or unbalanced quotes runs through `sh -c`, and the plan says so), a `healthcheck` as `healthCheck.command`, a published database port as `extraPorts`, and `restart: on-failure:N` with its retry limit (N from 1 to 100).
22
24
  - **Drops, and says so:** `command` on a service with a `build`, `env_file` and every other untranslated key; anonymous volumes and paths outside the project; `network_mode` other than `service:<name>`; services with invalid profile names, `extends`, an out-of-tree build, or an image with characters or variable forms other than `${NAME}`, `${NAME:-default}` and `${NAME-default}` (`postgres:${POSTGRES_VERSION:-18}-alpine` is written as it is).
23
25
  - **Does not start anything.** Run `flisk up` with Flisk core **1.4.26 or later** (`publish` needs it). The importer does not check the core version.
24
26
  - **Exit codes:** `0` when the plan succeeds (`--dry-run`, files written, or all `service.json` files already exist), `1` when no root `docker-compose.yml` or mappable service is found, and `2` when the write prompt is declined or closed without an answer (`Nothing written.`).
package/dist/cli.js CHANGED
@@ -1,6 +1,6 @@
1
1
  #!/usr/bin/env node
2
2
  import { readFileSync } from "node:fs";
3
- import { join, relative } from "node:path";
3
+ import { relative } from "node:path";
4
4
  import { createInterface } from "node:readline/promises";
5
5
  import { Command } from "commander";
6
6
  import "./detectors/index.js";
@@ -59,11 +59,13 @@ const program = new Command("flisk-import")
59
59
  console.log(` wrote ${relative(process.cwd(), item.path) || manifestRelPath(plan, item)}`);
60
60
  }
61
61
  console.log(`\nWrote ${written.length} service.json file(s)${writes.projectConfig ? " and created .flisk/project.json" : ""}.`);
62
- const envFile = relative(process.cwd(), join(writes.projectRoot, ".env")) || ".env";
62
+ const envFile = relative(process.cwd(), writes.env.path) || ".env";
63
63
  console.log("\nNext:");
64
64
  const copied = Object.keys(writes.env.add);
65
65
  if (copied.length)
66
66
  console.log(` - Copied ${copied.length} environment value(s) from docker-compose.yml to ${envFile} (gitignored).`);
67
+ if (writes.env.tracked && copied.length)
68
+ console.log(` - ${writes.env.tracked} is tracked by git, so it was left as it is and not added to .gitignore; the values went to ${envFile}, which each service that needs them reads as envFile.`);
67
69
  if (writes.items.some((item) => item.notes.some((note) => note.startsWith("no literal value"))))
68
70
  console.log(` - Add the values listed above as missing to ${envFile}.`);
69
71
  console.log(` - Run \`flisk up\` with Flisk core ${MIN_CORE_VERSION} or later. The import did not start anything.`);
@@ -271,6 +271,12 @@ function hostSideOf(port, hostIp, hostPort) {
271
271
  const publish = `${hostIp}:${hostNumber ?? port}`;
272
272
  return /^\d{1,3}(\.\d{1,3}){3}:\d+$/.test(publish) ? { publish } : undefined;
273
273
  }
274
+ /** Whether a `ports:` entry names a host port ("8000:8000", "127.0.0.1:8000:8000", `published:`), not only a container port. */
275
+ function hasHostSide(entry) {
276
+ if (isObj(entry))
277
+ return entry.published !== undefined;
278
+ return typeof entry === "string" && (entry.split("/")[0] ?? "").includes(":");
279
+ }
274
280
  function publishedPort(entry) {
275
281
  const port = containerPort(entry);
276
282
  if (port === undefined)
@@ -352,7 +358,7 @@ function volumeOf(entry, composeDir) {
352
358
  return { source: resolved, target, ...(readOnly ? { readOnly } : {}), bind };
353
359
  }
354
360
  /** Compose's project name: top-level `name`, else the directory name, lowercased with other characters dropped. */
355
- function projectNameOf(root, absPath) {
361
+ export function projectNameOf(root, absPath) {
356
362
  const raw = isObj(root) && typeof root.name === "string" && !root.name.includes("$")
357
363
  ? root.name
358
364
  : basename(dirname(absPath));
@@ -399,9 +405,76 @@ function envValues(environment) {
399
405
  }
400
406
  return out;
401
407
  }
402
- /** A value that is nothing but Compose interpolation, such as `${URL}` or `${PORT:-80}`. Holds no secret itself. */
403
- const ONLY_REFERENCES = /^(\$\{[A-Za-z_][A-Za-z0-9_]*(?::?[-?+][^}\n]*)?\})+$/;
404
- /** Values that are only `${VAR}` references, which service.json `env` can keep as they are (Flisk 1.4.24). */
408
+ /** The end of the `${...}` expression that starts at `start`, nested defaults included; undefined when it is not one. */
409
+ function expressionEnd(text, start) {
410
+ if (!text.startsWith("${", start))
411
+ return undefined;
412
+ const name = /^[A-Za-z_][A-Za-z0-9_]*/.exec(text.slice(start + 2));
413
+ if (!name)
414
+ return undefined;
415
+ let i = start + 2 + name[0].length;
416
+ if (text[i] === "}")
417
+ return i + 1;
418
+ const op = /^:?[-?+]/.exec(text.slice(i));
419
+ if (!op)
420
+ return undefined;
421
+ i += op[0].length;
422
+ while (i < text.length) {
423
+ if (text.startsWith("${", i)) {
424
+ const end = expressionEnd(text, i);
425
+ if (end === undefined)
426
+ return undefined;
427
+ i = end;
428
+ }
429
+ else if (text[i] === "}")
430
+ return i + 1;
431
+ else if (text[i] === "\n" || text[i] === "$")
432
+ return undefined;
433
+ else
434
+ i++;
435
+ }
436
+ return undefined;
437
+ }
438
+ /**
439
+ * A value that is nothing but Compose interpolation: `${URL}`, `${PORT:-80}`, `${PORT-80}`, nested `${A:-${B:-file}}`.
440
+ * It holds no secret itself (a default is what the project's Compose file already commits), so it stays in `env`.
441
+ */
442
+ export function isComposeExpression(value) {
443
+ let i = 0;
444
+ while (i < value.length) {
445
+ const end = expressionEnd(value, i);
446
+ if (end === undefined)
447
+ return false;
448
+ i = end;
449
+ }
450
+ return value.length > 0;
451
+ }
452
+ /**
453
+ * A value that mixes Compose expressions with literal text (`${SERVER_NAME:-localhost, :443}, php:80`), every `${`
454
+ * starting a valid expression. Undefined when it is not one; else the literal text with the expressions taken out.
455
+ */
456
+ export function composeTemplateText(value) {
457
+ if (!value.includes("${") || value.includes("\n"))
458
+ return undefined;
459
+ let literal = "";
460
+ let i = 0;
461
+ while (i < value.length) {
462
+ if (value.startsWith("${", i)) {
463
+ const end = expressionEnd(value, i);
464
+ if (end === undefined)
465
+ return undefined;
466
+ i = end;
467
+ }
468
+ else {
469
+ if (value[i] === "$" && value[i + 1] !== "$")
470
+ return undefined;
471
+ literal += value[i];
472
+ i++;
473
+ }
474
+ }
475
+ return literal;
476
+ }
477
+ /** Values with `${VAR}` references (alone or in literal text), which service.json `env` can keep as they are (Flisk 1.4.24). */
405
478
  function envReferences(environment) {
406
479
  const pairs = Array.isArray(environment)
407
480
  ? environment
@@ -412,7 +485,7 @@ function envReferences(environment) {
412
485
  : [];
413
486
  const out = {};
414
487
  for (const [key, raw] of pairs) {
415
- if (typeof raw === "string" && ONLY_REFERENCES.test(raw))
488
+ if (typeof raw === "string" && composeTemplateText(raw) !== undefined)
416
489
  out[key] = raw;
417
490
  }
418
491
  return out;
@@ -491,25 +564,68 @@ export function shellWords(text) {
491
564
  */
492
565
  function containerCommandOf(command) {
493
566
  if (typeof command === "string" && command.trim()) {
494
- if (command.includes("$"))
495
- return { argv: ["sh", "-c", command], shell: "uses $ variables" };
496
- const words = shellWords(command);
567
+ const { text, interpolates } = unescapeDollars(command);
568
+ if (interpolates)
569
+ return {
570
+ argv: ["sh", "-c", text],
571
+ note: "uses Compose variables, which Compose filled from .env; in the generated Dockerfile it runs through sh -c, so the container's shell fills them from its own environment",
572
+ };
573
+ const words = shellWords(text);
497
574
  return words?.length
498
575
  ? { argv: words }
499
- : { argv: ["sh", "-c", command], shell: "has unbalanced quotes" };
576
+ : {
577
+ argv: ["sh", "-c", text],
578
+ note: "has unbalanced quotes; it runs through sh -c, which Compose would not do",
579
+ };
580
+ }
581
+ if (Array.isArray(command) && command.length > 0 && command.every((c) => typeof c === "string")) {
582
+ const parts = command.map(unescapeDollars);
583
+ return {
584
+ argv: parts.map((p) => p.text),
585
+ ...(parts.some((p) => p.interpolates)
586
+ ? {
587
+ note: "uses Compose variables, which Compose filled from .env; the generated Dockerfile passes them as written (a sh -c argument fills them from the container's environment)",
588
+ }
589
+ : {}),
590
+ };
500
591
  }
501
- if (Array.isArray(command) && command.length > 0 && command.every((c) => typeof c === "string"))
502
- return { argv: command };
503
592
  return undefined;
504
593
  }
505
- /** Seconds from a Compose duration such as `30s` or `2m`; undefined for anything it cannot read exactly. */
594
+ /**
595
+ * A Compose command as the container sees it: `$$` is Compose's escape for a literal `$` (so `$$PASSWORD` reaches the
596
+ * container's shell as `$PASSWORD`). `interpolates` says whether a single `$` (`${VAR}`, `$VAR`) asks Compose to fill
597
+ * a value from .env, which a Dockerfile cannot do.
598
+ */
599
+ function unescapeDollars(value) {
600
+ let text = "";
601
+ let interpolates = false;
602
+ for (let i = 0; i < value.length; i++) {
603
+ if (value[i] === "$" && value[i + 1] === "$") {
604
+ text += "$";
605
+ i++;
606
+ continue;
607
+ }
608
+ if (value[i] === "$" && /[{A-Za-z_]/.test(value[i + 1] ?? ""))
609
+ interpolates = true;
610
+ text += value[i];
611
+ }
612
+ return { text, interpolates };
613
+ }
614
+ const DURATION_UNIT = { h: 3600, m: 60, s: 1, ms: 0.001, us: 0.000001 };
615
+ /** Seconds from a Compose duration such as `30s`, `2m` or `1m30s`; undefined for anything it cannot read exactly. */
506
616
  function secondsOf(value) {
507
617
  if (typeof value === "number" && value > 0)
508
618
  return value;
509
- const match = typeof value === "string" ? value.match(/^(\d+)(s|m)?$/) : null;
510
- if (!match)
619
+ if (typeof value !== "string")
620
+ return undefined;
621
+ if (/^\d+$/.test(value))
622
+ return Number(value) > 0 ? Number(value) : undefined;
623
+ if (!DURATION.test(value))
511
624
  return undefined;
512
- const seconds = Number(match[1]) * (match[2] === "m" ? 60 : 1);
625
+ let seconds = 0;
626
+ for (const [, amount, unit] of value.matchAll(/([0-9]+(?:\.[0-9]+)?)(h|ms|us|m|s)/g))
627
+ seconds += Number(amount) * DURATION_UNIT[unit];
628
+ seconds = Math.round(seconds * 1000) / 1000;
513
629
  return seconds > 0 ? seconds : undefined;
514
630
  }
515
631
  /** Compose `healthcheck` as a probe command (exit 0 is ready). `NONE` and `disable` give no probe. */
@@ -531,6 +647,7 @@ function healthProbeOf(healthcheck) {
531
647
  return undefined;
532
648
  const interval = secondsOf(healthcheck.interval);
533
649
  const timeout = secondsOf(healthcheck.timeout);
650
+ const startPeriod = secondsOf(healthcheck.start_period);
534
651
  const retries = typeof healthcheck.retries === "number" &&
535
652
  Number.isInteger(healthcheck.retries) &&
536
653
  healthcheck.retries > 0
@@ -541,6 +658,7 @@ function healthProbeOf(healthcheck) {
541
658
  ...(interval ? { interval } : {}),
542
659
  ...(timeout ? { timeout } : {}),
543
660
  ...(retries ? { retries } : {}),
661
+ ...(startPeriod ? { startPeriod } : {}),
544
662
  };
545
663
  }
546
664
  /** The models a service uses, joined to their top-level definitions. A model without a literal `model` is skipped. */
@@ -672,6 +790,13 @@ registerDetector({
672
790
  return { candidates, skips: [...skips, { ...base, reason: "no `services:` map" }] };
673
791
  }
674
792
  const project = projectNameOf(root, absPath);
793
+ // Services another one waits for with `condition: service_completed_successfully`: one-shot jobs.
794
+ const awaitedJobs = new Set(Object.values(services).flatMap((svc) => isObj(svc) && isObj(svc.depends_on)
795
+ ? Object.entries(svc.depends_on)
796
+ .filter(([, dep]) => isObj(dep) && dep.condition === "service_completed_successfully")
797
+ .map(([dep]) => dep)
798
+ : []));
799
+ const restartedByFlisk = [];
675
800
  for (const [name, raw] of Object.entries(services)) {
676
801
  if (!isObj(raw)) {
677
802
  skips.push({ ...base, reason: `service "${name}" is not a map` });
@@ -756,13 +881,17 @@ registerDetector({
756
881
  ["command", commandArgs],
757
882
  ["entrypoint", entrypointArgs],
758
883
  ]) {
759
- if (args?.shell)
884
+ if (args?.note)
760
885
  skips.push({
761
886
  ...base,
762
- reason: `service "${name}" ${key} ${JSON.stringify(raw[key])} ${args.shell}; it runs through sh -c, which Compose would not do`,
887
+ reason: `service "${name}" ${key} ${JSON.stringify(raw[key])} ${args.note}`,
763
888
  });
764
889
  }
765
- const healthProbe = healthProbeOf(raw.healthcheck);
890
+ // `disable: true` or `test: ["NONE"]` turns off the image's own HEALTHCHECK too, so it is kept as healthCheck: false.
891
+ const healthOff = isObj(raw.healthcheck) &&
892
+ (raw.healthcheck.disable === true ||
893
+ (Array.isArray(raw.healthcheck.test) && raw.healthcheck.test[0] === "NONE"));
894
+ const healthProbe = healthOff ? false : healthProbeOf(raw.healthcheck);
766
895
  const envKeys = envNames(raw.environment);
767
896
  const env = envValues(raw.environment);
768
897
  const envRefs = envReferences(raw.environment);
@@ -807,11 +936,14 @@ registerDetector({
807
936
  reason: `service "${name}" ${key} ${JSON.stringify(raw[key])} has values Flisk does not accept; not used`,
808
937
  });
809
938
  }
810
- // Every published port Flisk does not route stays published on the same host address (service.json extraPorts).
939
+ // Every published port stays published on the same host address (service.json extraPorts). The routed port too,
940
+ // when Compose gives it a host side ("${APP_PORT:-8000}:8000"), so the app is still reachable without Traefik
941
+ // (and in an exported docker-compose.yml); a bare routed "8000" is reached through Traefik only.
811
942
  const proxied = selectedPort !== undefined && !NON_HTTP_PORTS.has(selectedPort);
812
943
  const extraPorts = (Array.isArray(raw.ports) ? raw.ports : [])
944
+ .filter((entry) => containerPort(entry) !== selectedPort || !proxied || hasHostSide(entry))
813
945
  .map(publishedPort)
814
- .filter((p) => p !== undefined && (p.port !== selectedPort || !proxied));
946
+ .filter((p) => p !== undefined);
815
947
  let sidecarOf;
816
948
  if (raw.network_mode !== undefined) {
817
949
  const mode = typeof raw.network_mode === "string" ? raw.network_mode : "";
@@ -873,7 +1005,7 @@ registerDetector({
873
1005
  ...(models ? { models } : {}),
874
1006
  ...(profiles ? { profiles } : {}),
875
1007
  ...(envFile ? { envFile } : {}),
876
- ...(healthProbe ? { healthCheck: healthProbe } : {}),
1008
+ ...(healthProbe !== undefined ? { healthCheck: healthProbe } : {}),
877
1009
  ...(selectedPort !== undefined ? { port: selectedPort } : {}),
878
1010
  ...(healthCheckPath ? { healthCheckPath } : {}),
879
1011
  exposeViaProxy,
@@ -885,11 +1017,25 @@ registerDetector({
885
1017
  ...(extraPorts.length ? { extraPorts } : {}),
886
1018
  ...(sidecarOf ? { sidecarOf } : {}),
887
1019
  ...(volumes.length ? { volumes } : {}),
888
- ...(restart ? { restart } : {}),
1020
+ // Flisk runs a bring-your-own service with restart "no" as a one-shot job and waits for it to exit, so "no" is
1021
+ // written only when Compose says so or another service waits for this one to complete. Otherwise restart is
1022
+ // left out (Flisk's unless-stopped).
1023
+ ...(restart
1024
+ ? { restart }
1025
+ : raw.restart === undefined && awaitedJobs.has(name)
1026
+ ? { restart: "no" }
1027
+ : {}),
889
1028
  confidence: "high",
890
1029
  sources: [base],
891
1030
  });
1031
+ if (raw.restart === undefined && !awaitedJobs.has(name))
1032
+ restartedByFlisk.push(name);
892
1033
  }
1034
+ if (restartedByFlisk.length)
1035
+ skips.push({
1036
+ ...base,
1037
+ reason: `restart not set for ${restartedByFlisk.join(", ")}: Flisk restarts them (unless-stopped), where Compose would not restart them after a crash or a reboot`,
1038
+ });
893
1039
  return { candidates, skips };
894
1040
  },
895
1041
  });
package/dist/flisk.js CHANGED
@@ -26,7 +26,7 @@ export const HEALTH_CHECK_PATH_PATTERN = /^\/[a-z0-9/-]*$/;
26
26
  * An image reference, which may hold Compose variables `${NAME}`, `${NAME:-default}` and `${NAME-default}` (postgres:${PG_VERSION:-18}-alpine);
27
27
  * Compose resolves them from the project .env. Nothing else: no quotes, spaces or newlines that could change generated YAML.
28
28
  */
29
- export const IMAGE_PATTERN = /^(?:[A-Za-z0-9._/:@-]|\$\{[A-Za-z_][A-Za-z0-9_]*(?::?-[A-Za-z0-9._/:@-]*)?\})+$/;
29
+ export const IMAGE_PATTERN = /^(?:[A-Za-z0-9._/:@-]|\$\{[A-Za-z_][A-Za-z0-9_]*(?::?-[A-Za-z0-9._/:@-]+)?\})+$/;
30
30
  /** A DNS name for a network alias: letters, digits, '-', '_' and '.'. */
31
31
  export const NETWORK_ALIAS = /^[A-Za-z0-9][A-Za-z0-9_.-]{0,252}$/;
32
32
  const KEBAB = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
package/dist/format.js CHANGED
@@ -11,6 +11,8 @@ export function formatPlan(plan, writes) {
11
11
  out.push(`Found ${plan.services.length - jobs} service(s) and ${jobs} job(s) in ${plan.root}`, `Read only: ${plan.files.join(", ")}. Lines below name what was not translated.`, "");
12
12
  if (writes.projectConfig)
13
13
  out.push(` .flisk/project.json [create Flisk project]`, "");
14
+ if (writes.projectConfigUpdate)
15
+ out.push(` .flisk/project.json [add composeProjectNames.${writes.stack}: "${writes.composeProjectName}", so an export keeps the Compose project name]`, "");
14
16
  for (const service of plan.services) {
15
17
  const item = writes.items.find((i) => i.service === service);
16
18
  out.push(` ${service.name} (${service.kind}${service.dir ? `, ${service.dir}/` : ""})`);
package/dist/write.js CHANGED
@@ -1,5 +1,8 @@
1
+ import { execFileSync } from "node:child_process";
1
2
  import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
2
3
  import { basename, dirname, join, relative, resolve } from "node:path";
4
+ import { parse as parseYaml } from "yaml";
5
+ import { composeTemplateText, isComposeExpression, projectNameOf } from "./detectors/compose.js";
3
6
  import { ImportError } from "./errors.js";
4
7
  import { BYO_PORT_RANGE, defaultProjectConfig, findProjectRoot, IMAGE_PATTERN, NETWORK_ALIAS, SERVICE_MANIFEST_SCHEMA_URL, SERVICE_MANIFEST_SCHEMA_VERSION, validateName, } from "./flisk.js";
5
8
  import { toResourceName } from "./merge.js";
@@ -9,28 +12,47 @@ const ENV_NAME = /^[A-Z_][A-Z0-9_]*$/;
9
12
  /** Names that may hold a credential whatever their value looks like; these always stay secrets. */
10
13
  const CREDENTIAL_NAME = /PASS|SECRET|TOKEN|KEY|PRIVATE|CREDENTIAL|AUTH|HMAC|SALT|CERT|SESSION|COOKIE/i;
11
14
  /**
12
- * Whether a literal Compose value cannot be a secret, so it may be committed in service.json `env`: a number, true or false,
13
- * or the address of another service in this import (a URL or host:port with no user, password, query or fragment).
15
+ * Whether a literal value may hold a credential although its name does not say so: a URL or `user:pass@host` with a
16
+ * user or password, a query parameter named like a credential, a PEM block, a long random-looking token, or a value
17
+ * that says it is one (SECRET, TOKEN, PASSWORD).
14
18
  */
15
- function isPlainValue(value, serviceHosts) {
16
- if (/^-?\d+(\.\d+)?$/.test(value) || /^(true|false)$/i.test(value))
19
+ /** Query parameters that carry a credential (`?token=`, `?api_key=`, `&sig=`); `authSource` and the like do not. */
20
+ const CREDENTIAL_QUERY = /token|secret|pass(word|wd)?$|api[_-]?key|^key$|^sig(nature)?$|access[_-]?key/i;
21
+ function looksSecret(value) {
22
+ if (/^[^\s:@/]+:[^\s@/]+@/.test(value) || value.includes("-----BEGIN"))
17
23
  return true;
18
- const hostPort = value.match(/^([a-z0-9][a-z0-9._-]*):(\d{1,5})$/);
19
- if (hostPort)
20
- return serviceHosts.has(hostPort[1]);
24
+ if (/SECRET|TOKEN|PASSWORD|PASSWD/i.test(value))
25
+ return true;
26
+ if (/^[A-Za-z0-9+/_=.-]{32,}$/.test(value) && /[0-9]/.test(value) && /[A-Za-z]/.test(value))
27
+ return true;
28
+ if (!value.includes("://"))
29
+ return false;
21
30
  try {
22
31
  const url = new URL(value);
23
- return (serviceHosts.has(url.hostname) &&
24
- !url.username &&
25
- !url.password &&
26
- !url.search &&
27
- !url.hash &&
28
- /^[A-Za-z0-9._~/-]*$/.test(url.pathname));
32
+ return (Boolean(url.username || url.password) ||
33
+ [...url.searchParams.keys()].some((key) => CREDENTIAL_QUERY.test(key)));
29
34
  }
30
35
  catch {
31
- return false;
36
+ return /:\/\/[^/\s]*@/.test(value);
32
37
  }
33
38
  }
39
+ /**
40
+ * `ARG` lines for the Compose variables in an image, so `FROM postgres:${POSTGRES_VERSION:-18}-alpine` can use them:
41
+ * Docker substitutes only build arguments declared before FROM. The default is the expression's.
42
+ */
43
+ function buildArgLines(image) {
44
+ const args = new Map();
45
+ for (const m of image.matchAll(/\$\{([A-Za-z_][A-Za-z0-9_]*)(?::?-([^}]*))?\}/g))
46
+ if (!args.has(m[1]))
47
+ args.set(m[1], m[2]);
48
+ return [...args]
49
+ .map(([name, value]) => (value === undefined ? `ARG ${name}\n` : `ARG ${name}=${value}\n`))
50
+ .join("");
51
+ }
52
+ /** `POSTGRES_PASSWORD_FILE=/run/secrets/pg`: the path of a mounted secret file, which is not a secret itself. */
53
+ function isSecretFilePath(key, value) {
54
+ return /_FILE$/.test(key) && /^\/[A-Za-z0-9_./-]+$/.test(value);
55
+ }
34
56
  function readEnvKeys(path) {
35
57
  if (!existsSync(path))
36
58
  return new Set();
@@ -71,16 +93,27 @@ export function planWrites(plan, force) {
71
93
  }
72
94
  const servicesDir = join(projectRoot, "services");
73
95
  const projectConfigPath = join(projectRoot, ".flisk", "project.json");
96
+ const composeProjectName = composeProjectNameOf(plan);
97
+ const composeNames = composeProjectName
98
+ ? { composeProjectNames: { [stack]: composeProjectName } }
99
+ : {};
74
100
  const projectConfig = existsSync(projectConfigPath)
75
101
  ? undefined
76
102
  : {
77
103
  path: projectConfigPath,
78
- content: `${JSON.stringify(defaultProjectConfig(projectRoot, {
79
- description: "Flisk proxy and imported Compose services",
80
- enabledStacks: ["proxy", stack],
81
- }), null, 2)}\n`,
104
+ content: `${JSON.stringify({
105
+ ...defaultProjectConfig(projectRoot, {
106
+ description: "Flisk proxy and imported Compose services",
107
+ enabledStacks: ["proxy", stack],
108
+ }),
109
+ ...composeNames,
110
+ }, null, 2)}\n`,
82
111
  };
83
- const envPath = join(projectRoot, ".env");
112
+ const projectConfigUpdate = projectConfig
113
+ ? undefined
114
+ : withComposeProjectName(projectConfigPath, stack, composeProjectName);
115
+ const envTarget = localEnvFile(projectRoot);
116
+ const envPath = envTarget.path;
84
117
  const envKept = readEnvKeys(envPath);
85
118
  const envAdd = {};
86
119
  const used = usedManifestPorts(servicesDir);
@@ -103,8 +136,6 @@ export function planWrites(plan, force) {
103
136
  return !reason;
104
137
  });
105
138
  const names = new Set(accepted.map((s) => s.name));
106
- // Hostnames other services use for these: the resource name, which stays the container hostname, and the Compose name.
107
- const serviceHosts = new Set(plan.services.flatMap((s) => (s.originalName ? [s.name, s.originalName] : [s.name])));
108
139
  for (const service of accepted) {
109
140
  const path = resolve(servicesDir, stack, service.name, "service.json");
110
141
  const exists = existsSync(path);
@@ -155,28 +186,39 @@ export function planWrites(plan, force) {
155
186
  // Infrastructure images and one-shot jobs do not serve HTTP, so an assumed "/" would only
156
187
  // fail the probe. Leave the path out; the runtime then skips the HTTP probe.
157
188
  const nonHttp = service.kind === "job" || (service.image !== undefined && NON_HTTP_IMAGE.test(service.image));
158
- const healthCheckPath = isPrivate || nonHttp ? undefined : (service.healthCheckPath ?? "/");
159
- if (nonHttp && !isPrivate)
189
+ const healthOff = service.healthCheck === false;
190
+ const healthCheckPath = isPrivate || nonHttp || healthOff ? undefined : (service.healthCheckPath ?? "/");
191
+ if (healthOff)
192
+ notes.push("healthCheck: false: Compose turns the healthcheck off");
193
+ else if (nonHttp && !isPrivate)
160
194
  notes.push("no health path: the service does not serve HTTP (non-HTTP image or job)");
161
- if (!isPrivate && !nonHttp && !service.healthCheckPath)
195
+ if (!isPrivate && !nonHttp && !healthOff && !service.healthCheckPath)
162
196
  notes.push("health path / is assumed: no HTTP health path could be read from Compose. Traefik routes only while it answers 2xx or 3xx");
163
- // A probe command is the only way to describe a non-HTTP check; a routed service keeps its HTTP path.
164
- const healthCheck = service.healthCheck && !(healthCheckPath && service.healthCheckPath)
165
- ? service.healthCheck
166
- : undefined;
167
- // Values that cannot be secrets (numbers, booleans, another service's address, pure ${VAR} references) go into the
168
- // committed service.json `env` (Flisk 1.4.24). Every other name is a secret: Flisk injects it from the gitignored .env.
197
+ // The Compose healthcheck stays as healthCheck.command (with its interval, timeout, retries and start period) even
198
+ // when an HTTP path was read too, so an exported docker-compose.yml keeps it and `condition: service_healthy` holds.
199
+ const healthCheck = service.healthCheck;
200
+ if (healthCheck && healthCheckPath && service.healthCheckPath)
201
+ notes.push(`healthCheck.command ${JSON.stringify(healthCheck.command)} is kept as Compose wrote it; Flisk probes with it instead of ${healthCheckPath}`);
202
+ // What goes into the committed service.json `env` (Flisk 1.4.24): a pure Compose expression (`${X}`, `${X:-default}`,
203
+ // `${A:-${B:-file}}`) as it is, whatever its name, since its default is already committed in the Compose file; and
204
+ // a literal whose name does not look like a credential and whose value does not hold one. Every other name is a
205
+ // secret: Flisk injects it from the gitignored .env, which gets the literal value.
169
206
  const validNames = service.envKeys.filter((key) => ENV_NAME.test(key));
170
207
  const env = {};
171
208
  for (const key of validNames) {
172
- if (CREDENTIAL_NAME.test(key))
173
- continue;
174
209
  const literal = service.env[key];
175
210
  const reference = service.envRefs[key];
176
- if (literal !== undefined && isPlainValue(literal, serviceHosts))
211
+ if (literal === undefined && reference !== undefined) {
212
+ // A pure expression is committed whatever its name; one mixed with literal text only when that text, with
213
+ // the expressions taken out, could not be a credential (`mongodb://${USER}:${PASSWORD:-x}@db/app` can).
214
+ const text = composeTemplateText(reference) ?? "";
215
+ if (isComposeExpression(reference) || (!CREDENTIAL_NAME.test(key) && !looksSecret(text)))
216
+ env[key] = reference;
217
+ }
218
+ else if (literal !== undefined && isSecretFilePath(key, literal))
219
+ env[key] = literal;
220
+ else if (literal !== undefined && !CREDENTIAL_NAME.test(key) && !looksSecret(literal))
177
221
  env[key] = literal;
178
- else if (literal === undefined && reference !== undefined)
179
- env[key] = reference;
180
222
  }
181
223
  const referenced = [
182
224
  ...new Set(Object.values(env).flatMap((value) => [...value.matchAll(/\$\{([A-Za-z_][A-Za-z0-9_]*)(:?-)?/g)]
@@ -213,7 +255,7 @@ export function planWrites(plan, force) {
213
255
  if (wrapWithCommand) {
214
256
  files.push({
215
257
  path: resolve(dirname(path), "Dockerfile"),
216
- content: `# From the Compose \`command\` of this service (flisk import). Flisk sets no container command, so it lives here.\nFROM ${service.image}\n${service.containerEntrypoint ? `ENTRYPOINT ${JSON.stringify(service.containerEntrypoint)}\n` : ""}${service.containerCommand ? `CMD ${JSON.stringify(service.containerCommand)}\n` : ""}`,
258
+ content: `# From the Compose \`command\` of this service (flisk import). Flisk sets no container command, so it lives here.\n${buildArgLines(service.image)}FROM ${service.image}\n${service.containerEntrypoint ? `ENTRYPOINT ${JSON.stringify(service.containerEntrypoint)}\n` : ""}${service.containerCommand ? `CMD ${JSON.stringify(service.containerCommand)}\n` : ""}`,
217
259
  });
218
260
  notes.push(`${service.containerEntrypoint ? "entrypoint" : "command"} ${JSON.stringify(service.containerEntrypoint ?? service.containerCommand)} is written to a Dockerfile on ${service.image}`);
219
261
  }
@@ -287,6 +329,10 @@ export function planWrites(plan, force) {
287
329
  return [{ ...volume, source: rel.startsWith(".") ? rel : `./${rel}` }];
288
330
  });
289
331
  // env_file paths are relative to the service directory in service.json, and must stay inside the project.
332
+ // With the project .env tracked by git, secrets live in another local file, which Flisk then reads as envFile.
333
+ const localEnv = envTarget.git.tracked && secretNames.length
334
+ ? [relative(dirname(path), envPath).split("\\").join("/")]
335
+ : [];
290
336
  const envFile = (service.envFile ?? []).flatMap((file) => {
291
337
  const absolute = resolve(plan.root, file);
292
338
  const fromProject = relative(projectRoot, absolute).split("\\").join("/");
@@ -304,7 +350,7 @@ export function planWrites(plan, force) {
304
350
  stack,
305
351
  port,
306
352
  ...(healthCheckPath ? { healthCheckPath } : {}),
307
- ...(healthCheck ? { healthCheck } : {}),
353
+ ...(healthCheck !== undefined ? { healthCheck } : {}),
308
354
  ...source,
309
355
  ...(isPrivate ? { exposeViaProxy: false } : {}),
310
356
  ...(dependsOn.length ? { dependsOn } : {}),
@@ -316,7 +362,9 @@ export function planWrites(plan, force) {
316
362
  ...(container ? { container } : {}),
317
363
  ...(service.models ? { models: service.models } : {}),
318
364
  ...(service.profiles ? { profiles: service.profiles } : {}),
319
- ...(envFile.length ? { envFile } : {}),
365
+ ...(envFile.length || localEnv.length
366
+ ? { envFile: [...new Set([...envFile, ...localEnv])] }
367
+ : {}),
320
368
  ...(extraPorts.length ? { extraPorts } : {}),
321
369
  ...(sidecarOf ? { sidecarOf } : {}),
322
370
  ...(volumes.length
@@ -338,8 +386,10 @@ export function planWrites(plan, force) {
338
386
  stack,
339
387
  items,
340
388
  projectConfig,
389
+ ...(projectConfigUpdate ? { projectConfigUpdate } : {}),
390
+ ...(composeProjectName ? { composeProjectName } : {}),
341
391
  rejected,
342
- env: { path: envPath, add: envAdd },
392
+ env: { path: envPath, add: envAdd, ...envTarget.git },
343
393
  };
344
394
  }
345
395
  export function applyWrites(writes) {
@@ -348,6 +398,8 @@ export function applyWrites(writes) {
348
398
  mkdirSync(dirname(writes.projectConfig.path), { recursive: true });
349
399
  writeFileSync(writes.projectConfig.path, writes.projectConfig.content, { flag: "wx" });
350
400
  }
401
+ if (writes.projectConfigUpdate)
402
+ writeFileSync(writes.projectConfigUpdate.path, writes.projectConfigUpdate.content);
351
403
  for (const item of writes.items) {
352
404
  if (item.status === "exists")
353
405
  continue;
@@ -363,14 +415,88 @@ export function applyWrites(writes) {
363
415
  const lines = keys.map((key) => envLine(key, writes.env.add[key]));
364
416
  const block = `${existing && !existing.endsWith("\n") ? "\n" : ""}# From docker-compose.yml (flisk import)\n${lines.join("\n")}\n`;
365
417
  writeFileSync(writes.env.path, existing + block);
366
- ensureGitignored(writes.projectRoot, ".env");
418
+ mkdirSync(dirname(writes.env.path), { recursive: true });
419
+ if (writes.env.gitignore)
420
+ ensureGitignored(writes.projectRoot, writes.env.gitignore);
367
421
  }
368
422
  return written;
369
423
  }
370
424
  export function manifestRelPath(plan, item) {
371
425
  return relative(plan.root, item.path).split("\\").join("/");
372
426
  }
373
- /** Keeps the project .env out of git: it now holds values copied from docker-compose.yml. */
427
+ /** The project name `docker compose` gives the imported Compose file: its `name:`, else its folder's name. */
428
+ function composeProjectNameOf(plan) {
429
+ const file = plan.files.find((f) => /^(docker-)?compose\.ya?ml$/.test(f));
430
+ if (!file)
431
+ return undefined;
432
+ const absPath = join(plan.root, file);
433
+ try {
434
+ return projectNameOf(parseYaml(readFileSync(absPath, "utf8"), { merge: true }), absPath);
435
+ }
436
+ catch {
437
+ return undefined;
438
+ }
439
+ }
440
+ /** An existing project.json with `composeProjectNames.<stack>` set, or undefined when it has it already or is unreadable. */
441
+ function withComposeProjectName(path, stack, name) {
442
+ if (!name)
443
+ return undefined;
444
+ try {
445
+ const config = JSON.parse(readFileSync(path, "utf8"));
446
+ if (typeof config !== "object" || config === null || Array.isArray(config))
447
+ return undefined;
448
+ const names = typeof config.composeProjectNames === "object" && config.composeProjectNames !== null
449
+ ? config.composeProjectNames
450
+ : {};
451
+ if (names[stack] === name)
452
+ return undefined;
453
+ return {
454
+ path,
455
+ content: `${JSON.stringify({ ...config, composeProjectNames: { ...names, [stack]: name } }, null, 2)}\n`,
456
+ };
457
+ }
458
+ catch {
459
+ return undefined;
460
+ }
461
+ }
462
+ /** Whether git in `root` tracks `file` (false outside a git repo, or without git). */
463
+ function gitTracks(root, file) {
464
+ try {
465
+ execFileSync("git", ["-C", root, "ls-files", "--error-unmatch", file], { stdio: "ignore" });
466
+ return true;
467
+ }
468
+ catch {
469
+ return false;
470
+ }
471
+ }
472
+ /** Whether git in `root` ignores `file` (false outside a git repo, or without git). */
473
+ function gitIgnores(root, file) {
474
+ try {
475
+ execFileSync("git", ["-C", root, "check-ignore", "-q", file], { stdio: "ignore" });
476
+ return true;
477
+ }
478
+ catch {
479
+ return false;
480
+ }
481
+ }
482
+ /**
483
+ * The local file that gets values copied from docker-compose.yml: the project .env, never one git tracks. A tracked
484
+ * .env is left as it is, and not added to .gitignore.
485
+ */
486
+ function localEnvFile(projectRoot) {
487
+ if (!gitTracks(projectRoot, ".env"))
488
+ return { path: join(projectRoot, ".env"), git: { gitignore: ".env" } };
489
+ if (gitIgnores(projectRoot, ".env.local") && !gitTracks(projectRoot, ".env.local"))
490
+ return { path: join(projectRoot, ".env.local"), git: { tracked: ".env" } };
491
+ return {
492
+ path: join(projectRoot, ".flisk", ".env.import"),
493
+ git: {
494
+ tracked: ".env",
495
+ ...(gitIgnores(projectRoot, ".flisk/.env.import") ? {} : { gitignore: ".flisk/.env.import" }),
496
+ },
497
+ };
498
+ }
499
+ /** Keeps the local env file out of git: it now holds values copied from docker-compose.yml. */
374
500
  function ensureGitignored(projectRoot, entry) {
375
501
  const path = join(projectRoot, ".gitignore");
376
502
  const content = existsSync(path) ? readFileSync(path, "utf8") : "";
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@flisk-sh/flisk-import",
3
- "version": "0.13.0",
3
+ "version": "0.15.0",
4
4
  "description": "Import a repository's root docker-compose.yml into Flisk service.json files, or register its Tiltfile.",
5
5
  "type": "module",
6
6
  "license": "MIT",