@graphql-hive/federation-gateway-audit 0.0.0

package/.changeset/@graphql-hive_federation-gateway-audit-293-dependencies.md

@@ -0,0 +1,5 @@
+---
+"@graphql-hive/federation-gateway-audit": patch
+---
+dependencies updates:
+  - Removed dependency [`dotenv@17.3.1` ↗︎](https://www.npmjs.com/package/dotenv/v/17.3.1) (from `dependencies`)

package/.changeset/README.md

@@ -0,0 +1,9 @@
+# Changesets
+
+Hello and welcome! This folder has been automatically generated by `@changesets/cli`, a build tool
+that works with multi-package repos, or single-package repos to help you version and publish your
+code. You can find the full documentation for it
+[in our repository](https://github.com/changesets/changesets)
+
+We have a quick list of common questions to get you started engaging with this project in
+[our documentation](https://github.com/changesets/changesets/blob/master/docs/common-questions.md)

package/.changeset/config.json

@@ -0,0 +1,18 @@
+{
+  "$schema": "https://unpkg.com/@changesets/config@2.1.0/schema.json",
+  "changelog": [
+    "@changesets/changelog-github",
+    { "repo": "graphql-hive/federation-gateway-audit" }
+  ],
+  "commit": false,
+  "fixed": [],
+  "linked": [],
+  "access": "public",
+  "baseBranch": "main",
+  "updateInternalDependencies": "patch",
+  "ignore": [],
+  "snapshot": {
+    "useCalculatedVersion": true,
+    "prereleaseTemplate": "{tag}-{datetime}-{commit}"
+  }
+}

package/.changeset/funky-times-cry.md

@@ -0,0 +1,5 @@
+---
+"@graphql-hive/federation-gateway-audit": patch
+---
+
+Remove \`dotenv\` dependency, and use the native \`process.loadEnvFile\`

package/.github/CONTRIBUTING.md

@@ -0,0 +1,9 @@
+# Contributing guide
+
+1. Create a new directory in the `gateways` folder with the name of the new gateway.
+2. Prepare the `install.sh` script that will install the necessary dependencies, the gateway itself.
+3. Create the `run.sh` script that will start the gateway. See an example using [supergraph.graphql](../gateways/grafbase-gateway/run.sh) and [subgraphs information](../gateways/cosmo-router/run.sh).
+4. Add the `gateway.json` file with the necessary information about the gateway. See an example [here](../gateways/hive-gateway/gateway.json).
+5. The `.gitignore` should be created as well to avoid committing unnecessary files.
+6. Execute `make test-[name of the gateway]` to test your gateway.
+7. Run `make summary` to include your gateway in the results.

package/.github/workflows/ci.yaml

@@ -0,0 +1,97 @@
+name: ci
+on:
+  pull_request: {}
+  push:
+    branches: ["main"]
+
+permissions: write-all
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - uses: the-guild-org/shared-config/setup@v1
+        name: set up env
+        with:
+          node-version-file: .node-version
+
+      - name: Install
+        run: make install
+
+      - name: Test
+        run: npm start -- test --cwd ./gateways/hive-router --run-script ./run.sh  --reporter tap --graphql http://127.0.0.1:4000/graphql --healthcheck http://127.0.0.1:4000/health --exit-on-fail
+
+  gateways:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          repository: ${{ github.event.pull_request.head.repo.full_name }}
+          ref: ${{ github.event.pull_request.head.ref }}
+
+      - uses: the-guild-org/shared-config/setup@v1
+        name: set up env
+        with:
+          node-version-file: .node-version
+
+      - name: Prepare
+        run: make install
+
+      - name: Test Gateways
+        run: make test-all
+
+      - name: Upload Logs
+        uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
+        with:
+          name: gateways-logs
+          path: gateways/*/logs/*.log
+
+      - name: Summary
+        run: |
+          EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
+          echo "summary<<$EOF" >> "$GITHUB_ENV"
+          cat ./REPORT.md >> "$GITHUB_ENV"
+          echo "$EOF" >> "$GITHUB_ENV"
+
+      - run: |
+          git config user.name github-actions
+          git config user.email github-actions@github.com
+
+      - name: publish report
+        uses: EndBug/add-and-commit@a94899bca583c204427a224a7af87c02f9b325d5 # v9
+        if: github.event_name == 'pull_request'
+        with:
+          add: ./REPORT.md ./README.md ./website/index.html ./website/data.json
+          message: "Update results"
+          pull: "--rebase --autostash"
+          author_email: github-actions@github.com
+          author_name: github-actions
+          github_token: ${{ secrets.BOT_GITHUB_TOKEN }}
+
+      - name: Publish Summary
+        if: github.event_name == 'pull_request'
+        uses: marocchino/sticky-pull-request-comment@773744901bac0e8cbb5a0dc842800d45e9b2b405 # v2
+        with:
+          header: "Test Results"
+          message: ${{ env.summary }}
+
+      - name: Publish to Cloudflare Pages
+        uses: cloudflare/pages-action@f0a1cd58cd66095dee69bfa18fa5efd1dde93bca # v1
+        with:
+          apiToken: ${{ secrets.CF_API_TOKEN }}
+          accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
+          projectName: federation-gateway-compatibility
+          directory: ./website
+          gitHubToken: ${{ secrets.GITHUB_TOKEN }}
+          branch: ${{ github.event.pull_request.number != '' && format('pr-{0}', github.event.pull_request.number) || 'main' }}

package/.github/workflows/release.yml

@@ -0,0 +1,58 @@
+name: Release
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+env:
+  NODE_NO_WARNINGS: 1
+  CI: true
+  NPM_CONFIG_PROVENANCE: true
+
+jobs:
+  dependencies:
+    if: github.event_name == 'pull_request' && github.event.pull_request.title != 'Upcoming Release Changes'
+    name: Dependencies
+    concurrency:
+      group: ${{ github.workflow }}-${{ github.ref }}-dependencies
+      cancel-in-progress: true
+    uses: the-guild-org/shared-config/.github/workflows/changesets-dependencies.yml@v1
+    with:
+      node-version-file: .node-version
+    secrets:
+      githubToken: ${{ secrets.GITHUB_TOKEN }}
+  snapshot:
+    name: Snapshot
+    concurrency:
+      group: ${{ github.workflow }}-${{ github.ref }}-release-snapshot
+      cancel-in-progress: true
+    permissions:
+      contents: read
+      id-token: write
+      pull-requests: write
+    uses: the-guild-org/shared-config/.github/workflows/release-snapshot.yml@v1
+    with:
+      npmTag: alpha
+      buildScript: build
+      node-version-file: .node-version
+    secrets:
+      githubToken: ${{ secrets.GITHUB_TOKEN }}
+      npmToken: ${{ secrets.NPM_TOKEN }}
+  stable:
+    if: github.ref == 'refs/heads/main'
+    concurrency: ${{ github.workflow }}-${{ github.ref }}-release-stable
+    name: Stable
+    permissions:
+      contents: read
+      id-token: write
+      pull-requests: write
+    uses: the-guild-org/shared-config/.github/workflows/release-stable.yml@v1
+    with:
+      node-version-file: .node-version
+      releaseScript: run release
+      versionScript: run version
+    secrets:
+      githubToken: ${{ secrets.GITHUB_TOKEN }}
+      npmToken: ${{ secrets.NPM_TOKEN }}

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 - The Guild
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,113 @@
+# Federation-Compatible Gateway Implementations
+
+This repository contains a set of tests to evaluate and compare the compatibility of different GraphQL gateways with Apollo Federation.
+
+[🌐 See the results of our audit](https://the-guild.dev/graphql/hive/federation-gateway-audit)
+
+[📖 Read more about our methodology and why we created this audit](https://the-guild.dev/graphql/hive/federation-gateway-audit#manifesto)
+
+<!-- gateways:start -->
+
+|                                                   Gateway                                                   | Compatibility |  Test Cases  | Test Suites |
+| :---------------------------------------------------------------------------------------------------------: | :-----------: | :----------: | :---------: |
+| [Hive Gateway (Rust QP)](https://the-guild.dev/graphql/hive/docs/gateway/other-features/rust-query-planner) |    100.00%    |    🟢 192    |    🟢 44    |
+|                            [Hive Router](https://github.com/graphql-hive/router)                            |    100.00%    |    🟢 192    |    🟢 44    |
+|                         [Hive Gateway](https://the-guild.dev/graphql/hive/gateway)                          |    99.48%     | 🟢 191 ❌ 1  | 🟢 43 ❌ 1  |
+|                               [Apollo Router](https://www.apollographql.com/)                               |    97.40%     | 🟢 187 ❌ 5  | 🟢 41 ❌ 3  |
+|                              [Apollo Gateway](https://www.apollographql.com/)                               |    96.88%     | 🟢 186 ❌ 6  | 🟢 40 ❌ 4  |
+|                                   [Cosmo Router](https://wundergraph.com)                                   |    94.27%     | 🟢 181 ❌ 11 | 🟢 37 ❌ 7  |
+|                                  [Grafbase Gateway](https://grafbase.com)                                   |    91.67%     | 🟢 176 ❌ 16 | 🟢 38 ❌ 6  |
+|                                      [Inigo Gateway](https://inigo.io)                                      |    47.92%     | 🟢 92 ❌ 100 | 🟢 12 ❌ 32 |
+
+<!-- gateways:end -->
+
+[See the full report](./REPORT.md)
+
+## Apollo Federation Coverage
+
+The tests are based on the Apollo Federation specification and cover the following directives:
+
+- `@interfaceObject`
+- `@key`
+- `@external`
+- `@provides`
+- `@requires`
+- `@extends`
+- `@inaccessible`
+- `@shareable`
+- `@skip`
+- `@include`
+- `@composeDirective`
+- `@override`
+
+**Out of scope (limited by Enterprise license of Apollo Router):**
+
+We are not able to test the following directives on Apollo Router due to the limitations of the Enterprise license:
+
+- `@authenticated`
+- `@policy`
+- `@requiresScopes`
+- `@override(label:)`
+
+We plan to test these directives as soon as we have access to the Enterprise license.
+
+---
+
+## CLI
+
+> TODO: we're working on a CLI to make it easier to run the tests, stay tuned!
+
+---
+
+## Instructions
+
+First of all, you need to install and prepare the gateways. You can do this by running the following command:
+
+```bash
+make install
+```
+
+> [!IMPORTANT]
+> Be aware that `Node` and `npm` are required to run the whole setup.
+
+### Testing all gateways
+
+You can run the tests for each gateway by running the following command:
+
+```bash
+make test-all
+```
+
+### Testing a specific gateway
+
+You can run the tests for a specific gateway by running the following command:
+
+```bash
+make test-[name of the gateway]
+
+make test-grafbase-gateway
+make test-cosmo-router
+make test-hive-gateway
+make test-hive-gateway-router-runtime
+make test-apollo-router
+```
+
+### Running a gateway for a single test suite
+
+In case you want to run only a limited set of tests, you can do so by running the following command:
+
+```bash
+make test-suite-[name of the gateway] TEST_SUITE=[id of the test suite]
+```
+
+### Running a gateway for a specific supergraph
+
+There's also the possibility to start a gateway for a selected supergraph, in case you want to run the queries yourself.
+
+```bash
+make run-[name of the gateway] TEST_SUITE=[id of the test suite]
+```
+
+## Contributing or adding a new gateway
+
+[See the contributing guide](./.github/CONTRIBUTING.md)