conlink 2.0.2 → 2.0.3

package/.github/workflows/push.yml

@@ -0,0 +1,17 @@
+name: Push (compose tests)
+
+on:
+  push: {}
+  pull_request:
+    branches: [ master ]
+  workflow_dispatch: {}
+
+jobs:
+  compose-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: stub step
+        run: "echo stub stub"

package/Dockerfile

@@ -24,7 +24,7 @@ FROM node:16-slim as run
 RUN apt-get -y update
 # Runtime deps and utilities
 RUN apt-get -y install libpcap-dev tcpdump iproute2 iputils-ping curl \
-                       iptables \
+                       iptables bridge-utils \
                        openvswitch-switch openvswitch-testcontroller
 
 COPY --from=build /app/ /app/

package/README.md

@@ -1,14 +1,24 @@
 # conlink: Declarative Low-Level Networking for Containers
 
+
 Create (layer 2 and layer 3) networking between containers using
 a declarative configuration.
 
 ## Prerequisites
 
+General:
+* docker
 * docker-compose version 1.25.4 or later.
-* `openvswitch` kernel module loaded on the host
-* `geneve` (and/or `vxlan`) kernel module loaded on the host (only
-  needed for `test5-geneve-compose` example)
+
+Other:
+* For Open vSwtich (OVS) bridging, the `openvswitch` kernel module
+  must loaded on the host system (where docker engine is running).
+* For podman usage (e.g. second part of `test3`), podman is required.
+* For remote connections/links (e.g. `test5`), the `geneve` (and/or
+  `vxlan`) kernel module must be loaded on the host system (where
+  docker engine is running)
+* For CloudFormation deployment (e.g. `test6`), the AWS CLI is
+  required.
 
 ## Usage Notes
 
@@ -34,6 +44,18 @@ will also be required for the conlink container. In particular, if the
 container uses systemd, then it will likely use `SYS_NICE` and
 `NET_BROADCAST` and conlink will likewise need those capabilities.
 
+### Bridging: Open vSwtich/OVS or Linux bridge
+
+Conlink creates bridges/switches and connects veth container links to
+those bridges (specified by `bridge:` in the link specification).
+By default, conlink will attempt to create Open vSwitch/OVS bridges
+for these connections, however, if the kernel does not provide support
+(`openvswitch` kernel module loaded), then conlink will fallback to
+using standard Linux bridges. The fallback behavior can be changed by
+setting the `--bridge-mode` option to either "ovs" or "linux". If the
+bridge mode is set to "ovs" then conlink will fail to start if the
+`openvswitch` kernel module is not detected.
+
 ## Network Configuration Syntax
 
 Network configuration can either be loaded directly from configuration
@@ -71,8 +93,8 @@ The following table describes the link properties:
 | route     | *          | string     |         | ip route add args        |
 | nat       | *          | IP         |         | DNAT/SNAT to IP          |
 | netem     | *          | string     |         | tc qdisc NetEm options   |
-| mode      | 5          | IP         |         | virt intf mode           |
-| vlanid    | vlan       | IP         |         | VLAN ID                  |
+| mode      | 5          | string     |         | virt intf mode           |
+| vlanid    | vlan       | number     |         | VLAN ID                  |
 
 - 1 - veth, dummy, vlan, ipvlan, macvlan, ipvtap, macvtap
 - 2 - defaults to outer compose service
@@ -186,11 +208,11 @@ From the second node ping an address in the internet service:
 docker-compose -f examples/test2-compose.yaml exec --index 2 node ping 8.8.8.8
 ```
 
-Scale the nodes from 2 to 5 and then ping from first node from the fifth:
+Scale the nodes from 2 to 5 and then ping the fifth node from the second:
 
 ```
 docker-compose -f examples/test2-compose.yaml up -d --scale node=5
-docker-compose -f examples/test2-compose.yaml exec --index 5 node ping 10.0.1.1
+docker-compose -f examples/test2-compose.yaml exec --index 2 node ping 10.0.1.5
 ```
 
 
@@ -261,7 +283,7 @@ defined in the first compose file.
 MODES_DIR=./examples/test4-multiple/modes ./mdc node1 up --build --force-recreate
 ```
 
-Ping the router host from `node`:
+Ping the router host from `node1`:
 
 ```
 docker-compose exec node1 ping 10.0.0.100
@@ -282,6 +304,14 @@ docker-compose exec --index 1 node2 ping 10.1.0.1
 docker-compose exec --index 2 node2 ping 10.1.0.1
 ```
 
+From `node1`, ping both `node2` replicas across the switches and `r0` router:
+
+```
+docker-compose exec node1 ping 10.2.0.1
+docker-compose exec node1 ping 10.2.0.2
+```
+
+
 Restart the compose instance and add another compose file that starts
 conlink using an addition network file `web-network.yaml`. The network
 file starts up a simple web server on the router.
@@ -395,8 +425,8 @@ Show the links in both node containers to see that the MAC addresses
 are `00:0a:0b:0c:0d:0*` and the MTUs are set to `4111`.
 
 ```
-docker-compose -f examples/test7-compose.yaml exec --index 1 ip link
-docker-compose -f examples/test7-compose.yaml exec --index 2 ip link
+docker-compose -f examples/test7-compose.yaml exec --index 1 node ip link
+docker-compose -f examples/test7-compose.yaml exec --index 2 node ip link
 ```
 
 Ping the second node from the first to show the the NetEm setting is

package/mdc

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 
 set -e
 shopt -s dotglob  # recursive copy of dot files too

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "conlink",
-  "version": "2.0.2",
+  "version": "2.0.3",
   "description": "conlink -  Declarative Low-Level Networking for Containers",
   "repository": "https://github.com/LonoCloud/conlink",
   "license": "SEE LICENSE IN LICENSE",

package/scripts/copy.sh

@@ -26,9 +26,11 @@ dst_dir="${1}"; shift || die 2 "Usage: ${0} [-T|--template] SRC_DIR DST_DIR"
   cp -a "${src}" "${dst}" || die 1 "Failed to copy file"
   # TODO: make this configurable
   chown root.root "${dst}" || die 1 "Unable to set ownership"
+  chmod +w "${dst}" || die 1 "Unable to make writable"
 
   [ -z "${TEMPLATE}" ] && continue
 
+  tmpfile="$(mktemp)"
   # match all {{FOO}} style variables and replace from environment
   for v in $(cat "${dst}" | grep -o '{{[^ }{]*}}' | sed 's/[}{]//g' | sort -u); do
     if set | grep -qs "^${v}="; then
@@ -36,9 +38,11 @@ dst_dir="${1}"; shift || die 2 "Usage: ${0} [-T|--template] SRC_DIR DST_DIR"
           | sed "s/^['\"]\(.*\)['\"]$/\1/" \
           | sed 's/[\/&]/\\&/g')
       echo "Replacing '{{${v}}}' with '${val}' in '${dst}'"
-      sed -i "s/{{${v}}}/${val}/g" "${dst}"
+      sed "s/{{${v}}}/${val}/g" "${dst}" > "${tmpfile}"
+      cp "${tmpfile}" "${dst}"
     fi
   done
+  rm -f "${tmpfile}"
 done
 
 if [ "${*}" ]; then

package/src/conlink/core.cljs

@@ -27,9 +27,9 @@ General Options:
   -v, --verbose                     Show verbose output (stderr)
                                     [env: VERBOSE]
   --show-config                     Print loaded network config JSON and exit
-  --bridge-mode BRIDGE-MODE         Bridge mode (ovs or linux) to use for
-                                    bridge/switch connections
-                                    [default: ovs]
+  --bridge-mode BRIDGE-MODE         Bridge mode (ovs, linux, or auto)
+                                    to use for bridge/switch connections
+                                    [default: auto] [env: CONLINK_BRIDGE_MODE]
   --network-file NETWORK-FILE...    Network config file
   --compose-file COMPOSE-FILE...    Docker compose file with network config
   --compose-project NAME            Docker compose project name for resolving
@@ -651,17 +651,38 @@ General Options:
     (fatal 2 "Could not find config-schema" orig-config-schema)))
 
 (defn startup-checks
-  "Check startup state and exit if openvswitch kernel module is not
-  loaded or if no docker or podman connection could be established."
-  [bridge-mode docker podman]
+  "Check startup state and return map of :bridge-mode, :docker, and
+  :podman. If bridge-mode is :auto then return :ovs if the
+  'openvswitch' kernel module is loaded otherwise fall back to :linux.
+  Exit with an error if bridge-mode is :ovs and the 'openvswitch'
+  kernel module is not loaded or if neither a docker or podman
+  connection could be established."
+  [{:keys [bridge-mode docker-socket podman-socket]}]
   (P/let
-    [kmod-okay? (if (= :ovs bridge-mode)
-                  (kmod-loaded? "openvswitch")
-                  true)]
-    (when (not kmod-okay?)
-      (fatal 1 "bridge-mode is 'ovs', but no 'openvswitch' module loaded"))
+    [{:keys [info warn]} @ctx
+     ovs? (kmod-loaded? "openvswitch")
+     bridge-mode (condp = [bridge-mode ovs?]
+                   [:auto true]
+                   :ovs
+
+                   [:auto false]
+                   (do
+                     (warn (str "bridge-mode is 'auto' but no 'openvswitch' "
+                                "kernel module loaded, so using 'linux'"))
+                    :linux)
+
+                   [:ovs false]
+                   (fatal 1 (str "bridge-mode is 'ovs', but no 'openvswitch' "
+                                 "kernel module loaded"))
+
+                   bridge-mode)
+     docker (docker-client docker-socket)
+     podman (docker-client podman-socket)]
     (when (and (not docker) (not podman))
-      (fatal 1 "Failed to start either docker or podman client/listener"))))
+      (fatal 1 "Failed to start either docker or podman client/listener"))
+    {:bridge-mode bridge-mode
+     :docker docker
+     :podman podman}))
 
 (defn server
   "Process:
@@ -691,7 +712,7 @@ General Options:
      _ (arg-checks opts)
      _ (info (str "User options:\n" (indent-pprint-str opts "  ")))
 
-     {:keys [network-file compose-file compose-project bridge-mode]} opts
+     {:keys [network-file compose-file compose-project]} opts
      env (js->clj (js/Object.assign #js {} js/process.env))
      self-pid js/process.pid
      schema (load-config (:config-schema opts))
@@ -703,9 +724,7 @@ General Options:
          (println (js/JSON.stringify (->js network-config)))
          (js/process.exit 0))
 
-     docker (docker-client (:docker-socket opts))
-     podman (docker-client (:podman-socket opts))
-     _ (startup-checks bridge-mode docker podman)
+     {:keys [bridge-mode docker podman]} (startup-checks opts)
      self-cid (get-container-id)
      self-container-obj (when self-cid
                           (get-container (or docker podman) self-cid))