cartography 0.97.1__tar.gz → 0.98.0__tar.gz

cartography-0.98.0/.cache/.gitignore

@@ -0,0 +1,5 @@
+# This .gitignore is a placeholder so that we can store .cache/ in github.
+# We are including a pre-created .cache in Github so that when
+# we run docker-compose in WSL2 during dev linting, WSL2 does not attempt
+# to create .cache/ with root as the owner. The contents of .cache
+# should never be pushed back upstream to the main cartography repo.

cartography-0.98.0/.dockerignore

@@ -0,0 +1,6 @@
+build/
+*.egg-info/
+venv/
+.compose
+.cache
+.local

cartography-0.98.0/.env

@@ -0,0 +1,4 @@
+# This file is for docker-compose dev use so that mounting
+# Neo4j volumes doesn't fail with permissions errors.
+GID=1000
+UID=1000

cartography-0.98.0/.github/ISSUE_TEMPLATE/issue-report-feature-request.md

@@ -0,0 +1,47 @@
+---
+name: Issue report/feature request
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Feature request template**
+
+*Title*: *One line description*
+
+*Description*:
+> Describe your idea.  Please be detailed.  If a feature request, please
+describe the desired behavior, what scenario it enables, and how it
+would be used.
+
+[optional *Relevant Links*:]
+> Any extra documentation required to understand the issue.
+
+
+
+**Bug Template**
+
+*Title*: *One line description*
+
+*Description*:
+> What issue is being seen?  Describe what should be happening instead of the bug, for example: Cartography should not crash, the expected value isn't returned, the data schema is wrong, etc.
+
+*To Reproduce*:
+> Steps to reproduce the behavior.  Provide all data and inputs required to reproduce the issue.
+
+*Logs*:
+> If applicable, copy and paste your console log with the failing stack trace.
+
+*Screenshots*:
+> If applicable, add screenshots to help explain your problem.
+
+*Please complete the following information:*:
+>  - Cartography release version or commit hash [e.g. 0.12.0 or 95e8e11913e2a44a4d4682506d8364a638ceac69]
+>  - Python version: [e.g. 3.10.0]
+>  - OS (feel free to omit this if you don't think it's relevant to your issue): [e.g. Ubuntu bla bla, OSX bla bla]
+
+
+*Additional context*:
+> Add any other context about the problem here.

cartography-0.98.0/.github/pull_request_template.md

@@ -0,0 +1,23 @@
+### Summary
+> Describe your changes.
+
+
+
+### Related issues or links
+> Include links to relevant issues or other pages.
+
+- https://github.com/lyft/cartography/issues/...
+
+
+### Checklist
+
+Provide proof that this works (this makes reviews move faster). Please perform one or more of the following:
+- [ ] Update/add unit or integration tests.
+- [ ] Include a screenshot showing what the graph looked like before and after your changes.
+- [ ] Include console log trace showing what happened before and after your changes.
+
+If you are changing a node or relationship:
+- [ ] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md).
+
+If you are implementing a new intel module:
+- [ ] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).

cartography-0.98.0/.github/stale.yml

@@ -0,0 +1,17 @@
+# Number of days of inactivity before an issue becomes stale
+daysUntilStale: 14
+# Number of days of inactivity before a stale issue is closed
+daysUntilClose: 21
+# Issues with these labels will never be considered stale
+exemptLabels:
+  - keep fresh
+# Label to use when marking an issue as stale
+staleLabel: stale
+# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs.
+# Comment to post when closing a stale issue. Set to `false` to disable
+closeComment: >
+  This issue has been automatically closed for inactivity. If you still wish to
+  make these changes, please open a new change or reopen this one.

cartography-0.98.0/.github/workflows/ossf-scorecard.yml

@@ -0,0 +1,64 @@
+---
+# https://github.com/marketplace/actions/ossf-scorecard-action
+name: Scorecards supply-chain security
+on:
+  # Only the default branch is supported.
+  branch_protection_rule:
+  schedule:
+    # Weekly on Saturdays.
+    - cron: '30 1 * * 6'
+  push:
+    branches: [ main, master ]
+
+# Declare default permissions as read only.
+permissions: read-all
+
+jobs:
+  analysis:
+    name: Scorecards analysis
+    runs-on: ubuntu-latest
+    permissions:
+      # Needed to upload the results to code-scanning dashboard.
+      security-events: write
+      # Used to receive a badge. (Upcoming feature)
+      id-token: write
+      actions: read
+      contents: read
+
+    steps:
+      - name: "Checkout code"
+        uses: actions/checkout@a12a3943b4bdde767164f792f33f40b04645d846 # tag=v3.0.0
+        with:
+          persist-credentials: false
+
+      - name: "Run analysis"
+        uses: ossf/scorecard-action@3e15ea8318eee9b333819ec77a36aca8d39df13e # tag=v1.1.1
+        with:
+          results_file: results.sarif
+          results_format: sarif
+          # (Optional) Read-only PAT token. Uncomment the `repo_token` line below if:
+          # - you want to enable the Branch-Protection check on a *public* repository, or
+          # - you are installing Scorecards on a *private* repository
+          # To create the PAT, follow the steps in https://github.com/ossf/scorecard-action#authentication-with-pat.
+          # repo_token: ${{ secrets.SCORECARD_READ_TOKEN }}
+
+          # Publish the results for public repositories to enable scorecard badges. For more details, see
+          # https://github.com/ossf/scorecard-action#publishing-results.
+          # For private repositories, `publish_results` will automatically be set to `false`, regardless
+          # of the value entered here.
+          publish_results: true
+
+      # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
+      # format to the repository Actions tab.
+      - name: "Upload artifact"
+        uses: actions/upload-artifact@6673cd052c4cd6fcf4b4e6e60ea986c889389535 # tag=v3.0.0
+        with:
+          name: SARIF file
+          path: results.sarif
+          retention-days: 5
+
+      # Upload the results to GitHub's code scanning dashboard.
+      - name: "Upload to code-scanning"
+        uses: github/codeql-action/upload-sarif@5f532563584d71fdef14ee64d17bafb34f751ce5 # tag=v1.0.26
+        with:
+          sarif_file: results.sarif

cartography-0.98.0/.github/workflows/publish-to-ghcr-and-pypi.yml

@@ -0,0 +1,92 @@
+name: Publish Docker 🐳 images 📦 to GitHub Container Registry
+
+on:
+  release:
+    types: [created]
+  workflow_dispatch: {}
+
+jobs:
+
+  build-and-publish-to-pypi-and-ghcr:
+    # Explicitly grant the `secrets.GITHUB_TOKEN` permissions.
+    permissions:
+      # Grant the ability to write to GitHub Packages (push Docker images to
+      # GitHub Container Registry).
+      packages: write
+    name: Build and publish Python 🐍 distributions 📦 to PyPI, and Docker 🐳 images 📦 to GitHub Container Registry
+    runs-on: ubuntu-latest
+    steps:
+      # 1. Publish to PyPI
+      - uses: actions/checkout@v2
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v1
+        with:
+          python-version: "3.10"
+
+      - name: Install pypa/build
+        run: >-
+          python -m
+          pip install
+          build
+          --user
+
+      - name: Build a binary wheel and a source tarball
+        run: >-
+          python -m
+          build
+          --sdist
+          --wheel
+          --outdir dist/
+          .
+
+      - name: Publish distribution 📦 to PyPI
+        if: startsWith(github.ref, 'refs/tags')
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}
+          skip_existing: true
+
+      # 2. Publish to GHCR
+      - uses: actions/checkout@v2
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ghcr.io/${{ github.repository }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          # This is the user that triggered the Workflow. In this case, it will
+          # either be the user whom created the Release or manually triggered
+          # the workflow_dispatch.
+          username: ${{ github.actor }}
+          # `secrets.GITHUB_TOKEN` is a secret that's automatically generated by
+          # GitHub Actions at the start of a workflow run to identify the job.
+          # This is used to authenticate against GitHub Container Registry.
+          # See https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret
+          # for more detailed information.
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      # Get the cartography version from the GitHub release event metadata and sleep to wait for PyPI to finish publishing
+      - name: Extract version and sleep
+        id: version
+        run: |
+          echo "VERSION=${{ github.event.release.tag_name }}" >> $GITHUB_ENV
+          sleep 10
+
+      - name: Build and push
+        uses: docker/build-push-action@v3
+        with:
+          file: Dockerfile
+          context: .
+          push: true # push the image to ghcr
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          # e.g. '==0.98.0'
+          build-args: VERSION_SPECIFIER===${{ env.VERSION }}

cartography-0.98.0/.github/workflows/sphinx.yml

@@ -0,0 +1,109 @@
+# Based on: https://github.com/rkdarst/sphinx-actions-test/blob/master/.github/workflows/sphinx-build.yml
+
+name: sphinx
+on:
+  push:
+    branches:
+      - 'master'
+
+env:
+  DEFAULT_BRANCH: "master"
+
+jobs:
+  build-and-deploy:
+    name: Build and gh-pages
+    runs-on: ubuntu-latest
+    steps:
+      # https://github.com/marketplace/actions/checkout
+      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+          lfs: true
+      # https://github.com/marketplace/actions/setup-python
+      # ^-- This gives info on matrix testing.
+      - name: Install Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: "3.10"
+      # https://docs.github.com/en/actions/guides/building-and-testing-python#caching-dependencies
+      # ^-- How to set up caching for pip on Ubuntu
+      - name: Cache pip
+        uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-
+            ${{ runner.os }}-
+
+      # Build
+      - name: Build Sphinx docs
+        run: |
+         ./docs/build.sh
+
+      # Clone and set up the old gh-pages branch
+      - name: Clone old gh-pages
+        if: ${{ github.event_name == 'push' }}
+        run: |
+          set -x
+          git fetch
+          ( git branch gh-pages remotes/origin/gh-pages && git clone . --branch=gh-pages _gh-pages/ ) || mkdir _gh-pages
+          rm -rf _gh-pages/.git/
+          mkdir -p _gh-pages/branch/
+      # If a push and default branch, copy build to _gh-pages/ as the "main"
+      # deployment.
+      - name: Copy new build (default branch)
+        if: |
+          contains(github.event_name, 'push') &&
+          contains(github.ref, env.DEFAULT_BRANCH)
+        run: |
+          set -x
+          # Delete everything under _gh-pages/ that is from the
+          # primary branch deployment.  Eicludes the other branches
+          # _gh-pages/branch-* paths, and not including
+          # _gh-pages itself.
+          find _gh-pages/ -mindepth 1 ! -path '_gh-pages/branch*' -delete
+          rsync -a generated/docs/ _gh-pages/
+      # If a push and not on default branch, then copy the build to
+      # _gh-pages/branch/$brname (transforming '/' into '--')
+      - name: Copy new build (branch)
+        if: |
+          contains(github.event_name, 'push') &&
+          !contains(github.ref, env.DEFAULT_BRANCH)
+        run: |
+          set -x
+          brname="${{github.ref}}"
+          brname="${brname##refs/heads/}"
+          brdir=${brname//\//--}   # replace '/' with '--'
+          rm -rf   _gh-pages/branch/${brdir}
+          rsync -a generated/docs/ _gh-pages/branch/${brdir}
+      # Go through each branch in _gh-pages/branch/, if it's not a
+      # ref, then delete it.
+      - name: Delete old feature branches
+        if: ${{ github.event_name == 'push' }}
+        run: |
+          set -x
+          for brdir in `ls _gh-pages/branch/` ; do
+              brname=${brdir//--/\/}   # replace '--' with '/'
+              if ! git show-ref remotes/origin/$brname ; then
+                  echo "Removing $brdir"
+                  rm -r _gh-pages/branch/$brdir/
+              fi
+          done
+
+      # Add the .nojekyll file
+      - name: nojekyll
+        if: ${{ github.event_name == 'push' }}
+        run: |
+          touch _gh-pages/.nojekyll
+
+      # Deploy
+      # https://github.com/peaceiris/actions-gh-pages
+      - name: Deploy
+        uses: peaceiris/actions-gh-pages@v3.8.0
+        if: ${{ github.event_name == 'push' }}
+        with:
+          publish_branch: gh-pages
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          publish_dir: _gh-pages/
+          force_orphan: true

cartography-0.98.0/.github/workflows/test_suite.yml

@@ -0,0 +1,114 @@
+name: Test Suite
+on:
+  pull_request:
+    branches:
+      - master
+  push:
+    branches:
+      - master
+
+# Explicitly grant the `secrets.GITHUB_TOKEN` no permissions.
+permissions: {}
+
+jobs:
+  linter:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v3
+        with:
+          python-version: "3.10"
+      - uses: pre-commit/action@v3.0.0
+        with:
+          extra_args: --all-files --show-diff-on-failure
+
+  unit-and-integration-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      # https://stackoverflow.com/a/64592785
+      - name: neo4j 4 instance setup
+        run: |
+          docker run --detach \
+            --name neo4j-4 \
+            --env NEO4J_AUTH=none \
+            --publish 7474:7474 \
+            --publish 7473:7473 \
+            --publish 7687:7687 \
+            neo4j:4.4-community
+      - uses: actions/setup-python@v2
+        with:
+          python-version: "3.10"
+      # Cache our pip dir for efficiency; see https://medium.com/ai2-blog/python-caching-in-github-actions-e9452698e98d.
+      - uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: ${{ hashFiles('setup.py') }}-${{ hashFiles('pyproject.toml') }}
+      - name: Install cartography
+        run: |
+          pip install -e .
+          pip install .[dev]
+      - name: Wait for neo4j 4 to be ready
+        timeout-minutes: 1
+        run: (docker logs -f neo4j-4 & ) | grep -q Started
+      - name: make test_unit
+        run: make test_unit
+      - name: make test_integration
+        run: make test_integration
+
+  unit-and-integration-tests-neo4j5:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      # https://stackoverflow.com/a/64592785
+      - name: neo4j 5 setup
+        run: |
+          docker run --detach \
+            --name neo4j-5 \
+            --env NEO4J_AUTH=none \
+            --publish 7474:7474 \
+            --publish 7473:7473 \
+            --publish 7687:7687 \
+            neo4j:5
+      - uses: actions/setup-python@v2
+        with:
+          python-version: "3.10"
+      # Cache our pip dir for efficiency; see https://medium.com/ai2-blog/python-caching-in-github-actions-e9452698e98d.
+      - uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: ${{ hashFiles('setup.py') }}-${{ hashFiles('pyproject.toml') }}
+      - name: Install cartography
+        run: |
+          pip install -e .
+          pip install .[dev]
+      - name: Wait for neo4j 5 to be ready
+        timeout-minutes: 1
+        run: (docker logs -f neo4j-5 & ) | grep -q Started
+      - name: make test_unit
+        run: make test_unit
+      - name: make test_integration
+        run: make test_integration
+
+  # Test that the docker image builds successfully on each commit
+  build-docker-image:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ghcr.io/${{ github.repository }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+
+      - name: Build
+        uses: docker/build-push-action@v3
+        with:
+          file: Dockerfile
+          push: false # only build the image, don't push it anywhere
+          context: .
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

cartography-0.98.0/.gitignore

@@ -0,0 +1,21 @@
+*~
+*.pyc
+*.pyo
+*.pyt
+*.pytc
+*.egg-info
+.*.swp
+.DS_Store
+/venv*
+.venv/
+.cache/
+build/
+.idea/
+.coverage
+.mypy_cache
+*.vscode
+.compose
+generated
+dist/
+.local
+cartography/_version.py

cartography-0.98.0/.pre-commit-config.yaml

@@ -0,0 +1,48 @@
+default_language_version:
+    # force all unspecified python hooks to run python3
+    python: python3
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v5.0.0
+    hooks:
+    -   id: check-docstring-first
+    -   id: check-executables-have-shebangs
+    -   id: check-merge-conflict
+    -   id: check-vcs-permalinks
+    -   id: check-yaml
+    -   id: debug-statements
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+-   repo: https://github.com/pycqa/flake8
+    rev: 7.1.1
+    hooks:
+    -   id: flake8
+-   repo: https://github.com/hhatto/autopep8
+    rev: v2.3.1
+    hooks:
+    -   id: autopep8
+        # disable a few rewrites which will cause autopep8 to reflow
+        args: [--in-place, '--ignore=E265,E501,W504']
+-   repo: https://github.com/asottile/pyupgrade
+    rev: v3.19.0
+    hooks:
+    -   id: pyupgrade
+        args: [--py36-plus]
+-   repo: https://github.com/asottile/add-trailing-comma
+    rev: v3.1.0
+    hooks:
+    -   id: add-trailing-comma
+        args: [--py36-plus]
+-   repo: https://github.com/asottile/reorder_python_imports
+    rev: v3.14.0
+    hooks:
+    -   id: reorder-python-imports
+        args: [--py3-plus]
+-   repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.13.0
+    hooks:
+    -   id: mypy
+        additional_dependencies:
+        -   types-requests
+        -   types-PyYAML
+        -   types-python-dateutil

cartography-0.98.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,2 @@
+This project is governed by [Lyft's code of conduct](https://github.com/lyft/code-of-conduct).
+All contributors and participants agree to abide by its terms.

cartography-0.98.0/Dockerfile

@@ -0,0 +1,25 @@
+# This is a thin distribution of the cartography software.
+# It is published at ghcr.io.
+FROM python:3.10-slim
+
+# Default to ''. Overridden with a specific version specifier e.g. '==0.98.0' by build args or from GitHub actions.
+ARG VERSION_SPECIFIER
+
+# the UID and GID to run cartography as
+# (https://github.com/hexops/dockerfile#do-not-use-a-uid-below-10000).
+ARG uid=10001
+ARG gid=10001
+
+WORKDIR /var/cartography
+ENV HOME=/var/cartography
+
+# Install cartography at the given version specifier. Can be ''.
+RUN pip install cartography${VERSION_SPECIFIER}
+
+USER ${uid}:${gid}
+
+# verify that the binary at least runs
+RUN cartography -h
+
+ENTRYPOINT ["cartography"]
+CMD ["-h"]

cartography-0.98.0/GOVERNANCE.md

@@ -0,0 +1,16 @@
+# Process for becoming a maintainer
+Generally, express interest. This usually includes things like:
+- Contributing 2+ PRs to Cartography.
+- Regularly participating in discussion on GitHub, on Slack, or in our live video meetings.
+
+# Maintainer responsibilities
+- Hang out on Slack; delayed response is perfectly acceptable
+- Triage GitHub issues and perform pull request reviews for other maintainers and the community.
+- Deeply understand the project's best practices and development patterns.
+
+# When does a maintainer lose maintainer status
+- If a maintainer is no longer interested or cannot perform the maintainer duties listed above, they should volunteer to be moved to emeritus status.
+- If we do not hear from you in over a year, we may remove you from maintainer status without notice.
+
+# Conflict resolution
+We handle conflicts on a case-by-case basis. In general, let's talk it out (Slack, video call, etc). For larger issues we may need to involve more people to come to a decision.

cartography-0.98.0/MAINTAINERS.md

@@ -0,0 +1,32 @@
+See https://github.com/lyft/cartography/#contributing for general contribution guidelines.
+
+## Maintainers
+
+List of community members who have shown they are committed to Cartography's continued development and are active in pull request reviews and community conversations.
+
+- Alex Chantavy (https://github.com/achantavy): General catch-all
+- Ramon Petgrave (https://github.com/ramonpetgrave64): General catch-all
+- Jérémy Chapeau (https://github.com/resilience-jychp): Plugin development
+- Ryan Lane (https://github.com/ryan-lane): Plugin development, general engineering best practices
+- Marco Lancini (https://github.com/marco-lancini): Deployment and use-cases
+- Kedar Ghule (https://github.com/kedarghule): Plugin development
+- Purusottam Mupunu (https://https://github.com/mpurusottamc): Deployment and use-cases
+- Chandan Chowdhury (https://github.com/chandanchowdhury): Plugin development
+
+
+## Friends of Cartography
+
+Not maintainers but are continually active in discussions and offer subject matter expertise or community advocacy (e.g. writing blog posts)
+
+- Daniel D'Agostino (https://github.com/danielsaporo): Deployment and use-cases
+- Ashley Lowde (https://github.com/alowde-ps): Deployment and use-cases
+
+
+## Emeritus maintainers
+
+Not actively involved in reviews, but Cartography would not exist without the contributions of these key community members.
+
+- Sacha Faust (https://github.com/sachafaust)
+- Andrew Johnson (https://github.com/SecPrez)
+- Taya Steere (https://github.com/tmsteere)
+- Evan Davis (https://github.com/ecdavis)

cartography-0.98.0/Makefile

@@ -0,0 +1,10 @@
+test: test_lint test_unit test_integration
+
+test_lint:
+	pre-commit run --all-files --show-diff-on-failure
+
+test_unit:
+	pytest -vvv --cov-report term-missing --cov=cartography tests/unit
+
+test_integration:
+	pytest -vvv --cov-report term-missing --cov=cartography tests/integration