rails-mermaid_erd_markdown 0.2.0 → 0.3.1

data/README.md

@@ -1,6 +1,37 @@
 # rails-mermaid_erd_markdown
 
-This is a rails gem that extends the rails-mermaid_erd gem to generate a mermaid ERD for Rails Models in markdown directly in source code. This avoids the need to download the ERD through a html page, which can be difficult for scenarios where one would like to programmatically generate an ERD.
+A Ruby on Rails gem that extends rails-mermaid_erd to generate mermaid Entity-Relationship Diagrams (ERD) for ActiveRecord Models. When combined with Continuous Integration (CI) pipelines, it enables one to generate living, self-updating documentation of their data. 
+
+## Example ERD
+
+```mermaid
+erDiagram
+    %% --------------------------------------------------------
+    %% Entity-Relationship Diagram
+    %% --------------------------------------------------------
+
+    %% table name: articles
+    Article{
+        integer id PK 
+        string title  
+        text content  
+        datetime created_at  
+        datetime updated_at  
+    }
+
+    %% table name: comments
+    Comment{
+        integer id PK 
+        string commenter  
+        text body  
+        integer article_id FK 
+        datetime created_at  
+        datetime updated_at  
+    }
+
+    Comment }o--|| Article : "BT:article"
+```
+
 
 ## Installation
 
@@ -10,30 +41,93 @@ Add this line to your application's Gemfile:
 gem 'rails-mermaid_erd_markdown'
 ```
 
-And then execute:
-
-    $ bundle install
-
 Or install it yourself as:
 
     $ gem install rails-mermaid_erd_markdown
 
 ## Usage
 
-To generate a mermaid ERD in markdown, run `rails generate_erd` or `rake generate_erd` from the terminal
+### Generating ERDs
+To generate the comprehensive mermaid ERD in markdown, run one of the following:
+   ```
+   rails generate_erd
+   ``` 
+   OR
+   ```
+   rake generate_erd
+   ``` 
+**If all of your models do not appear, be sure to eager load them first.** 
+
+### Configuration 
+You can configure the ERD generation by creating a `erd.yml` file in your root, and modifying the following fields. 
+
+``` 
+# erd.yml
+erd:
+    output_path: 'app/ERD.md' # Set output path of ERDs, default: 'app/ERD.md'
+    split_output: false, # Generates individual ERDs for each model with a specified depth, default: false
+    relationship_depth: 1 # Configured depth of individual ERD model generation, default: 1
 
-The default path for the generated ERD is `app/models/ERD.md`. You can modify this by creating an `erd.yml` file in the root directory
-and modifying the `result_path` as seen in the example at `docs/examples/erd.yml`. Make sure to include the markdown file name you 
-wish to generate in the path. 
-
-If an ERD already exists at the path specified, it will be parsed to determine if it is up to date. If it is, nothing happens. If it 
-is not, the ERD is ed. 
-
-One can create self-updating, living documentation by integrating this rake task into their CI. This ensures that the ERD is always up 
-to date and accurately describes the latest state of the models and their relationships.
+```
 
-You can view the ERD by navigating to the file in Github, which supports rendering mermaid diagrams from code. If you are a Visual 
-Studio Code user, you can use the [Markdown Preview Enhanced](https://marketplace.visualstudio.com/items?itemName=shd101wyy.markdown-preview-enhanced) extension to view the ERD directly in your IDE. 
+If your entity diagram is too large to be displayed you can set a `split_output` configuration to `true` to generate multiple ERD files based on each model in your project. 
+
+You can also set a `relationship_depth` configuration to include more than 1 level (the default) of associations in each document.
+
+
+### Viewing Diagrams
+
+You can view the ERD by navigating to the file in Github, which supports rendering mermaid diagrams from code. If you are a Visual Studio Code user, you can use any markdown preview extension such as [Markdown Preview Enhanced](https://marketplace.visualstudio.com/items?itemName=shd101wyy.markdown-preview-enhanced) view the ERD directly in your IDE.
+
+
+### Integration with CI Pipeline
+The most value of this gem comes with the integration of it's diagram generation into your CI. The amount of effort required to update documentation has always been the greatest deterrant to creating new documentation. By integrating this rake task into your CI, you can have the CI update the model for you anytime that it changes. 
+
+The following is a Github Actions job example for how one might do this for a Rails app that uses PostgreSQL: 
+
+```yml
+permissions:
+  contents: write
+  packages: read
+
+jobs:
+  update-erd:
+      runs-on: ubuntu-latest
+      services:
+          postgres:
+              image: postgres
+              env:
+                  POSTGRES_USER: postgres
+                  POSTGRES_PASSWORD: postgres
+              ports:
+              - 5432:5432
+              options:
+              - --health-cmd="pg_is_ready" --health-interval=10s --health-timeout=5s --health-retries=3
+      steps:
+      - name: Install packages
+        run:
+          sudo apt-get update && sudo apt-get install --no-install-recommends -y google-chrome-stable curl libvips postgresql-client libpq-dev
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with: 
+          ref: ${{ github.event.pull_request.head.ref }}
+      - name: Setup database
+        env:
+          RAILS_ENV: test
+          DATABASE_URL: postgres://postgres:postgres@localhost:5432
+        run: bin/rails db:setup
+      - name: Setup database
+        env:
+          RAILS_ENV: test
+          DATABASE_URL: postgres://postgres:postgres@localhost:5432
+        run: bin/rails generate_erd
+      - name: Commit and Push Updated ERD
+        run: |
+          git config --global user.name 'Github Actions Bot'
+          git config --global user.email 'actions/@users.noreply.github.com'
+          git add app/models/ERD.md 
+          git commit -m "Update ERD" || echo "No changes to commit"
+```
 
 ## Development
 

data/Rakefile

@@ -14,4 +14,4 @@ require "rubocop/rake_task"
 
 RuboCop::RakeTask.new
 
-task default: %i[test rubocop]
+task default: %i[test]

data/bin/console

@@ -2,7 +2,7 @@
 # frozen_string_literal: true
 
 require "bundler/setup"
-require "rails/mermaid_erd_markdown"
+require "rails-mermaid_erd_markdown"
 
 # You can add fixtures and/or initialization code here to make experimenting
 # with your gem easier. You can also use a different console, if you like.

data/bin/rubocop

@@ -0,0 +1,29 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'rubocop' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require "pathname"
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
+                                           Pathname.new(__FILE__).realpath)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300) =~ /This file was generated by Bundler/
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("rubocop", "rubocop")

data/docs/examples/erd.yml

@@ -1,4 +1,4 @@
-# This is the path and filename of the generated ERD. You can modify it to change where the ERD is stored/updated.
-# Default output path: 'app/ERD.md'
 erd:
-    output_path: 'app/ERD.md'
+    output_path: 'app/ERD.md' # Set output path of ERDs, default: 'app/ERD.md'
+    split_output: false, # Generates individual ERDs for each model with a specified depth, default: false
+    relationship_depth: 1 # Configured depth of individual ERD model generation, default: 1

data/lib/rails-mermaid_erd_markdown/configuration.rb

@@ -1,20 +1,28 @@
+# frozen_string_literal: true
+
 require "yaml"
 
 module MermaidErdMarkdown
   class Configuration
-    attr_accessor :output_path
+    attr_accessor :output_path, :split_output, :relationship_depth
 
     def initialize
       config = {
-        output_path: "app/models/ERD.md"
+        output_path: "app/models/ERD.md",
+        split_output: false,
+        relationship_depth: 1
       }
       erd_config_path = "erd.yml"
 
       begin
         erd_yml = YAML.safe_load File.open(erd_config_path)
-        @output_path = erd_yml["erd"]["output_path"]
+        @output_path = erd_yml["erd"]["output_path"] || config[:output_path]
+        @split_output = erd_yml["erd"]["split_output"] || config[:split_output]
+        @relationship_depth = erd_yml["erd"]["relationship_depth"] || config[:relationship_depth]
       rescue StandardError
         @output_path = config[:output_path]
+        @split_output = config[:split_output]
+        @relationship_depth = config[:relationship_depth]
       end
     end
   end

data/lib/rails-mermaid_erd_markdown/generator.rb

@@ -0,0 +1,169 @@
+# frozen_string_literal: true
+
+require_relative "configuration"
+require_relative "markdown_document"
+require_relative "source_data"
+require "digest/md5"
+require "logger"
+require "pathname"
+
+module MermaidErdMarkdown
+  class Generator
+    attr_writer :logger
+
+    def self.generate
+      new.generate
+    end
+
+    def generate
+      return unless find_or_create_output
+      return unless erd_changed?
+
+      update_erd
+
+      update_split_erds if configuration.split_output
+    end
+
+    def index_markdown(files)
+      MermaidErdMarkdown::MarkdownDocument.create do |doc|
+        doc.add(doc.header("Entity Relationship Diagrams"))
+        doc.add(
+          "Each model has its own ERD diagram. The diagram shows the " \
+          "selected model, plus #{configuration.relationship_depth} " \
+          "associated model(s) deep. Click on the model name to view " \
+          "the diagram.\n"
+        )
+        doc.add(doc.subheader("Models"))
+        files.each do |model, path|
+          doc.add(doc.list_item(doc.link(model, "../#{path}")))
+        end
+        doc.add("")
+      end
+    end
+
+    def model_markdown(output)
+      MermaidErdMarkdown::MarkdownDocument.create do |doc|
+        model_name = output[:Models].first[:ModelName]
+        doc.add(doc.header("#{model_name} Entity-Relationship Diagram"))
+        doc.add(doc.subheader("Associated Models"))
+        output[:Models].each do |model|
+          next if model[:ModelName] == model_name
+
+          model_path = "../#{output_path(model[:ModelName])}"
+
+          doc.add(doc.list_item(doc.link(model[:ModelName], model_path)))
+        end
+        doc.add("")
+        doc.add(doc.subheader("Entity-Relationship Diagram"))
+        doc.add(mermaid_markdown(output))
+      end
+    end
+
+    def mermaid_markdown(source)
+      MermaidErdMarkdown::MarkdownDocument.create do
+        erd do
+          add(
+            source[:Models].map do |model|
+              erd_table(model[:TableName], model[:ModelName]) do
+                model[:Columns].map do |column|
+                  erd_table_column(column)
+                end
+              end
+            end
+          )
+          add(
+            source[:Relations].map do |relation|
+              erd_relation(relation)
+            end
+          )
+        end
+      end
+    end
+
+    private
+
+    def comprehensive_erd
+      @comprehensive_erd ||= mermaid_markdown(source_data.data)
+    end
+
+    def configuration
+      @configuration ||= MermaidErdMarkdown::Configuration.new
+    end
+
+    def erd_changed?
+      # check if two diagrams are the same by comparing their MD5 hashes
+      existing_erd_hash = Digest::MD5.hexdigest(File.read(output_path))
+      new_erd_hash = Digest::MD5.hexdigest(comprehensive_erd)
+
+      return true if existing_erd_hash != new_erd_hash
+
+      logger.info("ERD already exists and is up to date. Skipping...")
+
+      false
+    end
+
+    def find_or_create_output
+      return true if output_path.exist?
+
+      logger.info("ERD does not currently exist at result path. Creating...")
+      begin
+        write_file("", output_path)
+        logger.info("ERD successfully created at #{output_path}")
+      rescue StandardError
+        logger.info("Could not create ERD. Output path is invalid.")
+
+        false
+      end
+
+      true
+    end
+
+    def logger
+      @logger ||= Logger.new($stdout).tap do |log|
+        log.progname = self.class.name.split("::").first
+      end
+    end
+
+    def output_path(extension = nil)
+      return Pathname.new(configuration.output_path) unless extension
+
+      Pathname.new(configuration.output_path).sub_ext("_#{extension}.md")
+    end
+
+    def source_data
+      @source_data ||= MermaidErdMarkdown::SourceData.new
+    end
+
+    def update_erd
+      logger.info("ERD writing to #{output_path}...")
+      write_file(comprehensive_erd, output_path)
+      logger.info("ERD successfully written")
+    end
+
+    def update_split_erds
+      # first remove all existing model ERD files
+      Dir.glob(output_path.sub_ext("_*.md")).each do |file|
+        File.delete(file)
+      end
+
+      files = {}
+
+      depth = configuration.relationship_depth
+
+      source_data.split_output(depth).each do |output|
+        model_name = output[:Models].first[:ModelName]
+        model_path = output_path(model_name)
+        write_file(model_markdown(output), model_path)
+        logger.info("ERD successfully written to #{model_path}")
+        files[model_name] = model_path
+      end
+
+      index_path = output_path("index")
+      write_file(index_markdown(files), index_path)
+    end
+
+    def write_file(new_erd, path)
+      File.write(path, new_erd)
+    end
+  end
+end

data/lib/rails-mermaid_erd_markdown/markdown_document.rb

@@ -0,0 +1,104 @@
+# frozen_string_literal: true
+
+module MermaidErdMarkdown
+  class MarkdownDocument
+    attr_accessor :is_show_key, :is_show_comment
+
+    def self.create(&block)
+      new.generate(&block)
+    end
+
+    def initialize
+      @is_show_key = true
+      @is_show_comment = true
+    end
+
+    def add(element)
+      document << element
+    end
+
+    def document
+      @document ||= []
+    end
+
+    def erd(&block)
+      add(erd_header)
+      block.arity == 1 ? block[self] : instance_eval(&block)
+      add(erd_footer)
+    end
+
+    def erd_footer
+      "```"
+    end
+
+    def erd_header
+      [
+        "```mermaid",
+        "erDiagram",
+        "    %% --------------------------------------------------------",
+        "    %% Entity-Relationship Diagram",
+        "    %% --------------------------------------------------------",
+        ""
+      ].join("\n")
+    end
+
+    def erd_relation(relation)
+      left_model_name = relation[:LeftModelName].tr(":", "-")
+      right_model_name = relation[:RightModelName].tr(":", "-")
+      comment = is_show_comment ? ": \"#{relation[:Comment]}\"" : ": \"\""
+
+      "    #{left_model_name} #{relation[:LeftValue]}" \
+        "#{relation[:Line]}#{relation[:RightValue]} " \
+        "#{right_model_name} #{comment}"
+    end
+
+    def erd_table(table_name, model_name)
+      lines = [erd_table_header(table_name, model_name)]
+      lines << yield
+      lines << erd_table_footer
+      lines.join("\n")
+    end
+
+    def erd_table_column(column)
+      key = is_show_key ? column[:key] : ""
+      line = "        #{column[:type]} #{column[:name]}"
+      line << " #{key}" unless key.empty?
+      line
+    end
+
+    def erd_table_header(table_name, model_name)
+      [
+        "    %% table name: #{table_name}",
+        "    #{model_name.tr(":", "-")}{"
+      ].join("\n")
+    end
+
+    def erd_table_footer
+      [
+        "    }",
+        ""
+      ].join("\n")
+    end
+
+    def header(text)
+      "# #{text}\n"
+    end
+
+    def generate(&block)
+      block.arity == 1 ? block[self] : instance_eval(&block)
+      document.join("\n")
+    end
+
+    def link(text, url)
+      "[#{text}](#{url})"
+    end
+
+    def list_item(text)
+      "- #{text}"
+    end
+
+    def subheader(text)
+      "## #{text}\n"
+    end
+  end
+end

data/lib/rails-mermaid_erd_markdown/source_data.rb

@@ -0,0 +1,63 @@
+# frozen_string_literal: true
+
+require "rails-mermaid_erd"
+
+module MermaidErdMarkdown
+  class SourceData
+    def data
+      @data ||= RailsMermaidErd::Builder.model_data
+    end
+
+    def split_output(depth = 1)
+      source_models = data[:Models]
+      source_relations = data[:Relations]
+      output = []
+
+      source_models.each do |model|
+        model_names = [model[:ModelName]]
+        search_models = model_names
+        relations = []
+
+        depth.times do
+          found_relations = []
+          next_search_models = []
+          search_models.each do |search_model|
+            found_relations += related_models(search_model, source_relations)
+            next_search_models += related_model_names(search_model, found_relations)
+          end
+          search_models = next_search_models
+          model_names += search_models
+          relations += found_relations
+        end
+
+        output << {
+          Models: models(model_names.uniq, source_models),
+          Relations: relations.uniq
+        }
+      end
+
+      output
+    end
+
+    private
+
+    def models(model_names, source_models)
+      model_names.map do |model_name|
+        source_models.find { |m| m[:ModelName] == model_name }
+      end.compact
+    end
+
+    def related_model_names(model_name, relations)
+      relations.map do |r|
+        r[:LeftModelName] == model_name ? r[:RightModelName] : r[:LeftModelName]
+      end
+    end
+
+    def related_models(model_name, relations)
+      relations.select do |relation|
+        relation[:LeftModelName] == model_name ||
+          relation[:RightModelName] == model_name
+      end
+    end
+  end
+end

data/lib/rails-mermaid_erd_markdown/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module MermaidErdMarkdown
-  VERSION = "0.2.0"
+  VERSION = "0.3.1"
 end

data/lib/rails-mermaid_erd_markdown.rb

@@ -1,109 +1,19 @@
 # frozen_string_literal: true
 
-require_relative "rails-mermaid_erd_markdown/configuration"
-require "rails-mermaid_erd"
-require "digest/md5"
+require_relative "rails-mermaid_erd_markdown/generator"
 require "rake"
 require "rake/dsl_definition"
-require "pathname"
-require "logger"
 
 module MermaidErdMarkdown
   extend Rake::DSL
 
   class << self
-    attr_writer :logger
-
-    def logger
-      @logger ||= Logger.new($stdout).tap do |log|
-        log.progname = name
-      end
-    end
-
-    def configuration
-      @configuration ||= MermaidErdMarkdown::Configuration.new
-    end
-
     def perform
-      new_erd = generate_mermaid_erd
-      existing_erd_path = Pathname.new(configuration.output_path)
-
-      unless existing_erd_path.exist?
-        logger.info("ERD does not currently exist at result path. Creating...")
-        begin
-          create_erd_file(new_erd)
-          logger.info("ERD successfully created at #{configuration.output_path}")
-        rescue StandardError
-          logger.info("Could not create ERD. Output path is invalid.")
-        end
-
-        return
-      end
-
-      existing_erd = File.read(existing_erd_path)
-      update_erd_file(existing_erd, new_erd)
-    end
-
-    def update_erd_file(current_erd, new_erd)
-      # check if two diagrams are the same by comparing their MD5 hashes
-      if Digest::MD5.hexdigest(current_erd) == Digest::MD5.hexdigest(new_erd)
-        logger.info("ERD already exists and is up to date. Skipping...")
-        return
-      end
-
-      logger.info("ERD already exists but is out of date. Overwriting...")
-      File.write(configuration.output_path, new_erd)
-      logger.info("ERD successfully updated")
-
-      nil
-    end
-
-    def create_erd_file(erd)
-      File.write(configuration.output_path, erd)
-
-      nil
-    end
-
-    def generate_mermaid_erd
-      data = RailsMermaidErd::Builder.model_data
-
-      is_show_key = true
-      is_show_relation_comment = true
-
-      lines = ["```mermaid"]
-      lines << "erDiagram"
-      lines << "    %% --------------------------------------------------------"
-      lines << "    %% Entity-Relationship Diagram"
-      lines << "    %% --------------------------------------------------------"
-      lines << ""
-
-      data[:Models].each do |model|
-        lines << "    %% table name: #{model[:TableName]}"
-        lines << "    #{model[:ModelName].tr(":", "-")}{"
-
-        model[:Columns].each do |column|
-          key = is_show_key ? column[:key] : ""
-          lines << "        #{column[:type]} #{column[:name]} #{key} "
-        end
-
-        lines << "    }"
-        lines << ""
-      end
-
-      data[:Relations].each do |relation|
-        left_model_name = relation[:LeftModelName].tr(":", "-")
-        right_model_name = relation[:RightModelName].tr(":", "-")
-        comment = is_show_relation_comment ? ": \"#{relation[:Comment]}\"" : ": \"\""
-
-        lines << "    #{left_model_name} #{relation[:LeftValue]}#{relation[:Line]}#{relation[:RightValue]} #{right_model_name} #{comment}"
-      end
-
-      lines << "```"
-      lines.join("\n")
+      MermaidErdMarkdown::Generator.generate
     end
   end
 
-  desc "Generate/update mermaid ERD diagram for database Models."
+  desc "Generate/update mermaid ERD diagram(s) for database Models."
   task generate_erd: :environment do
     perform
   end