platformos-check 0.0.1

data/docs/language_server/how_to_correct_code_with_code_actions_and_execute_command.md

@@ -0,0 +1,197 @@
+# How to correct code with Code Actions, Commands and Workspace Edits
+
+The [Language Server Protocol (LSP)][lsp] empowers Language Server developers to offer refactorings, quick fixes and the execution of commands to their users. In PlatformOS Check, we take advantage of this to offer quick fixes and autocorrection to users.
+
+This document exists to give you an overview of how that works.
+
+## Overview
+
+It goes like this:
+
+1. The Client (VS Code, vim, etc.) asks the Server for [Code Actions](#code-actions) at a character position, and the server responds with available `quickfix` and `source.fixAll` code actions.
+2. The User _may_ select one of the code actions by interacting with the UI (see [visual examples](#code-actions)). The selection triggers the Client to asks the Server to execute a `correction` [Command](#Commands) for certain [Diagnostics](#diagnostic) on behalf of the user.
+3. The Server asks the Client to apply a [Workspace Edit](#workspace-edits) to fix the diagnostics.
+
+## Sequence Diagram
+
+<img src="code-action-flow.png" width="700">
+
+<details>
+<summary>Sequence Diagram Explanation</summary>
+
+1. The Client asks the Server for [code actions](#code-actions) for the current file and character range.
+2. The Server responds with a list of code actions that can be applied for this location and character range. (`quickfix` and `source.fixAll`)
+3. ...Wait for user input...
+4. The User selects one of the code actions
+5. The Client sends a `workspace/executeCommand` request to the Server for this code action.
+6. The Server figures out the [workspace edit](#workspace-edits) for the command and arguments.
+7. The Server sends a `workspace/applyEdit` request for the file modifications that would fix the diagnostics.
+8. The Client responds with the status of the `applyEdit`.
+9. The Server cleans up its internal representation of the diagnostics and updates the client with the latest diagnostics.
+11. The Server responds to the `workspace/executeCommand` request.
+
+</details>
+
+## What it's like in the code
+
+We handle two Client->Server LSP requests:
+
+1. On `textDocument/codeAction`, our `CodeActionEngine` returns the results obtained from all `CodeActionProvider`.
+2. On `workspace/executeCommand`, our `ExecuteCommandEngine` dispatches the command and arguments to appropriate `ExecuteCommandProvider`.
+
+We define providers:
+
+- Two `CodeActionProvider`:
+  1. [`QuickFixCodeActionProvider`](/lib/platformos_check/language_server/code_action_providers/quickfix_code_action_provider.rb) - This one provides code actions that fix _one_ diagnostic.
+  2. [`SourceFixAllCodeActionProvider`](/lib/platformos_check/language_server/code_action_providers/source_fix_all_code_action_provider.rb) - This one provides code actions that fix _all diagnostics in the current file_.
+- One `ExecuteCommandProvider`:
+  1. [`CorrectionExecuteCommandProvider`](/lib/platformos_check/language_server/execute_command_providers/correction_execute_command_provider.rb) - This one takes a list of diagnostics as arguments, turns them into a [WorkspaceEdit](#workspace-edit) and tries to apply them with the server->client `workspace/applyEdit` request.
+
+We define a [`DocumentChangeCorrector`](/lib/platformos_check/language_server/document_change_corrector.rb) (an LSP analog to our [`Corrector`](/lib/platformos_check/corrector.rb) class). This class turns corrector calls into document changes supported by the LSP. For more details, see the [LSP reference on resource changes][lspresourcechange].
+
+## Definitions
+
+### Code Actions
+
+A [CodeAction][lspcodeaction] is the Language Server Protocol construct for "stuff you might want to do on the code."
+
+Think refactoring, running tests, fixing lint errors, etc.
+
+The client figures out which one it can run by executing the client->server `textDocument/codeAction` request.
+
+<details>
+<summary>Visual Examples</summary>
+
+In VS Code, code actions of different kinds have special meanings and are mapped to multiple places in the UI.
+
+* `Quick Fix...` button on diagnostic hover
+
+  <img src="code-action-quickfix.png" width=500>
+
+* Diagnostic right click in the problems tab
+
+  <img src="code-action-problem.png" width=500>
+
+* Command palette `Quick Fix...` and `Fix All`
+
+  <img src="code-action-command-palette.png" width=500>
+
+* Keyboard shortcuts
+
+  <img src="code-action-keyboard.png" width=700>
+</details>
+
+<details>
+  <summary>TypeScript Interface</summary>
+
+  ```ts
+  interface CodeAction {
+    title: string; // UI string, human readable for the action
+    kind?: CodeActionKind; // OPTIONAL, for filtering
+    diagnostics?: Diagnostic[]; // The diagnostics that the action SOLVES.
+    isPreferred?: boolean; // Are used by auto fix and can be targetted by keybindings.
+    // Shown as faded out in the code action menu when the user request a more specific type of code action
+    disabled?: {
+      reason: string;
+    },
+
+    // if both edit and command are present, edit is run first then command.
+    // I think edit is used so the client performs the change, wheras the command
+    // would be done by the server
+    edit?: WorkspaceEdit; // what this action does ??!!
+    command?: Command; // the command that it executes
+    data?: any; // sent from the CodeAction to the codeAction/resolve.
+  }
+
+  interface Command {
+    title: string; // Title of the command, like `save`
+    command: string; // id
+    arguments?: any[]
+  }
+  ```
+</details>
+
+---
+
+### Commands
+
+A [Command][lspcommand] is the Language Server Protocol representation of a command that can be executed by the Server.
+
+Think of them as data representation of function calls that can be made by the Client.
+
+Typically, a command is associated with a Code Action. If the Client wants to perform this Code Action, it will make a Client->Server `workspace/executeCommand` request.
+
+<details>
+  <summary>TypeScript Interface</summary>
+
+  ```ts
+  interface Command {
+    title: string; // Title of the command, like `save`
+    command: string; // id
+    arguments?: any[]
+  }
+  ```
+</details>
+
+---
+
+### Workspace Edits
+
+A [WorkspaceEdit][lspworkspaceedit] is the Language Server Protocol construct that abstracts code changes as data.
+
+Think edit file, create file, delete file but as data.
+
+<details>
+  <summary>TypeScript Interface</summary>
+
+  ```ts
+  interface WorkspaceEdit {
+    changes?: { uri: TextEdit[] ];
+    documentChanges: (TextDocumentEdit | CreateFile | RenameFile | DeleteFile|)[];
+    changeAnnotations?: { [id: string]: ChangeAnnotation }
+  }
+
+  // see https://microsoft.github.io/language-server-protocol/specifications/specification-current/#resourceChanges
+  interface CreateFile {
+    kind: 'create';
+    uri: DocumentUri;
+    options?: CreateFileOptions;
+    annotationId?: ChangeAnnotationIdentifier;
+  }
+
+  interface TextEdit {
+    range: Range;
+    newText: string; // can be empty to delete
+  }
+
+  interface TextDocumentEdit {
+    textDocument: OptionalVersionedTextDocumentIdentifier;
+    edits: (TextEdit | AnnotatedTextEdit)[]
+  }
+
+  interface OptionalVersionedTextDocumentIdentifier {
+    uri: TextDocumentURI;
+    // null is for when the file wasn't open from the client
+    // integer is for when you know what it was.
+    version: integer | null;
+  }
+  ```
+</details>
+
+---
+
+### Diagnostic
+
+A [Diagnostic][lspdiagnostic] is the Language Server Protocol construct for errors, warnings and information bubbles.
+
+In our case, diagnostics are PlatformOS Check offenses.
+
+They appear in the Problems tab and in the gutter of the editor.
+
+[lsp]: https://microsoft.github.io/language-server-protocol/specification
+[lspcodeaction]: https://microsoft.github.io/language-server-protocol/specification#textDocument_codeAction
+[lspcommand]: https://microsoft.github.io/language-server-protocol/specification#command
+[lspexecutecommand]: https://microsoft.github.io/language-server-protocol/specification#workspace_executeCommand
+[lspworkspaceedit]: https://microsoft.github.io/language-server-protocol/specification#workspaceEdit
+[lspdiagnostic]: https://microsoft.github.io/language-server-protocol/specifications/specification-3-17/#diagnostic
+[lspresourcechange]: https://microsoft.github.io/language-server-protocol/specifications/specification-current/#resourceChanges

data/exe/platformos-check

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "platformos_check"
+
+PlatformosCheck::Cli.parse_and_run(ARGV)

data/exe/platformos-check-language-server

@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'platformos_check'
+
+status_code = PlatformosCheck::LanguageServer.start
+exit! status_code

data/lib/platformos_check/analyzer.rb

@@ -0,0 +1,178 @@
+# frozen_string_literal: true
+
+module PlatformosCheck
+  class Analyzer
+    def initialize(platformos_app, checks = Check.all.map(&:new), auto_correct = false)
+      @platformos_app = platformos_app
+      @auto_correct = auto_correct
+
+      @liquid_checks = Checks.new
+      @yaml_checks = Checks.new
+      @html_checks = Checks.new
+
+      checks.each do |check|
+        check.platformos_app = @platformos_app
+
+        case check
+        when LiquidCheck
+          @liquid_checks << check
+        when YamlCheck
+          @yaml_checks << check
+        when HtmlCheck
+          @html_checks << check
+        end
+      end
+    end
+
+    def offenses
+      @liquid_checks.flat_map(&:offenses) +
+        @yaml_checks.flat_map(&:offenses) +
+        @html_checks.flat_map(&:offenses)
+    end
+
+    def yaml_file_count
+      @yaml_file_count ||= @platformos_app.yaml.size
+    end
+
+    def liquid_file_count
+      @liquid_file_count ||= @platformos_app.liquid.size
+    end
+
+    def total_file_count
+      yaml_file_count + liquid_file_count
+    end
+
+    # Returns all offenses for all files in platformos_app
+    def analyze_platformos_app
+      reset
+
+      liquid_visitor = LiquidVisitor.new(@liquid_checks, @disabled_checks)
+      html_visitor = HtmlVisitor.new(@html_checks)
+
+      PlatformosCheck.with_liquid_c_disabled do
+        @platformos_app.liquid.each_with_index do |liquid_file, i|
+          yield(liquid_file.relative_path.to_s, i, total_file_count) if block_given?
+          liquid_visitor.visit_liquid_file(liquid_file)
+          html_visitor.visit_liquid_file(liquid_file)
+        end
+      end
+
+      @platformos_app.yaml.each_with_index do |yaml_file, i|
+        yield(yaml_file.relative_path.to_s, liquid_file_count + i, total_file_count) if block_given?
+        @yaml_checks.call(:on_file, yaml_file)
+      end
+
+      finish(false)
+
+      offenses
+    end
+
+    # When only_single_file is false:
+    #   Runs single file checks for each file in `files`
+    #   Runs whole platformos_app checks
+    #   Returns single file checks offenses for file in `files` + whole platformos_app checks
+    # When only_single_file is true:
+    #   Runs single file checks for each file in `files`
+    #   Does not run whole platformos_app checks
+    #   Returns single file checks offenses for file in `files`
+    # When files is empty and only_single_file is false:
+    #   Only returns whole platformos_app checks
+    # When files is empty and only_single_file is true:
+    #   Returns empty array
+    def analyze_files(files, only_single_file: false)
+      reset
+
+      PlatformosCheck.with_liquid_c_disabled do
+        total = files.size
+        offset = 0
+
+        unless only_single_file
+          # Call all checks that run on the whole platformos_app
+          liquid_visitor = LiquidVisitor.new(@liquid_checks.whole_platformos_app, @disabled_checks)
+          html_visitor = HtmlVisitor.new(@html_checks.whole_platformos_app)
+          total += total_file_count
+          offset = total_file_count
+          @platformos_app.liquid.each_with_index do |liquid_file, i|
+            yield(liquid_file.relative_path.to_s, i, total) if block_given?
+            liquid_visitor.visit_liquid_file(liquid_file)
+            html_visitor.visit_liquid_file(liquid_file)
+          end
+
+          @platformos_app.yaml.each_with_index do |yaml_file, i|
+            yield(yaml_file.relative_path.to_s, liquid_file_count + i, total) if block_given?
+            @yaml_checks.whole_platformos_app.call(:on_file, yaml_file)
+          end
+        end
+
+        # Call checks that run on a single files, only on specified file
+        liquid_visitor = LiquidVisitor.new(@liquid_checks.single_file, @disabled_checks)
+        html_visitor = HtmlVisitor.new(@html_checks.single_file)
+        files.each_with_index do |app_file, i|
+          yield(app_file.relative_path.to_s, offset + i, total) if block_given?
+          if app_file.liquid?
+            liquid_visitor.visit_liquid_file(app_file)
+            html_visitor.visit_liquid_file(app_file)
+          elsif app_file.yaml?
+            @yaml_checks.single_file.call(:on_file, app_file)
+          end
+        end
+      end
+
+      finish(only_single_file)
+
+      offenses
+    end
+
+    def uncorrectable_offenses
+      return offenses unless @auto_correct
+
+      offenses.select { |offense| !offense.correctable? }
+    end
+
+    def correct_offenses
+      return unless @auto_correct
+
+      offenses.each(&:correct)
+    end
+
+    def write_corrections
+      return unless @auto_correct
+
+      @platformos_app.liquid.each(&:write)
+    end
+
+    private
+
+    def reset
+      @disabled_checks = DisabledChecks.new
+
+      @liquid_checks.each do |check|
+        check.offenses.clear
+      end
+
+      @html_checks.each do |check|
+        check.offenses.clear
+      end
+
+      @yaml_checks.each do |check|
+        check.offenses.clear
+      end
+    end
+
+    def finish(only_single_file = false)
+      if only_single_file
+        @liquid_checks.single_file.call(:on_end)
+        @html_checks.single_file.call(:on_end)
+        @yaml_checks.single_file.call(:on_end)
+      else
+        @liquid_checks.call(:on_end)
+        @html_checks.call(:on_end)
+        @yaml_checks.call(:on_end)
+      end
+
+      @disabled_checks.remove_disabled_offenses(@liquid_checks)
+      @disabled_checks.remove_disabled_offenses(@yaml_checks)
+      @disabled_checks.remove_disabled_offenses(@html_checks)
+    end
+  end
+end

data/lib/platformos_check/api_call_file.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+module PlatformosCheck
+  class ApiCallFile < LiquidFile
+    def notification?
+      true
+    end
+  end
+end

data/lib/platformos_check/app.rb

@@ -0,0 +1,138 @@
+# frozen_string_literal: true
+
+# frozen_string_lite/al: true
+
+require "pathname"
+
+module PlatformosCheck
+  class App
+    API_CALLS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(notifications/api_call_notifications|api_calls)/(.+)\.liquid\z}
+    ASSETS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)assets/}
+    EMAILS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(notifications/email_notifications|emails)/(.+)\.liquid\z}
+    GRAPHQL_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(graph_queries|graphql)s?/(.+)\.graphql\z}
+
+    MIGRATIONS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)migrations/(.+)\.liquid\z}
+    PAGES_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(pages|views/pages)/(.+)}
+    PARTIALS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(views/partials|lib)/(.+)}
+    LAYOUTS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(views/layouts)/(.+)}
+    SCHEMA_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(custom_model_types|model_schemas|schema)/(.+)\.yml\z}
+    SMSES_REGEX =  %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)(notifications/sms_notifications|smses)/(.+)\.liquid\z}
+    USER_SCHEMA_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/)?)user.yml}
+    TRANSLATIONS_REGEX = %r{\A(?-mix:^/?((marketplace_builder|app)/|modules/(.+)(private|public|marketplace_builder|app)/)?)translations.+.yml}
+    CONFIG_REGEX = %r{(?-mix:^\\/?((marketplace_builder|app)\\/)?)config.yml}
+
+    REGEXP_MAP = {
+      API_CALLS_REGEX => ApiCallFile,
+      ASSETS_REGEX => AssetFile,
+      EMAILS_REGEX => EmailFile,
+      GRAPHQL_REGEX => GraphqlFile,
+      MIGRATIONS_REGEX => MigrationFile,
+      PAGES_REGEX => PageFile,
+      PARTIALS_REGEX => PartialFile,
+      LAYOUTS_REGEX => LayoutFile,
+      SCHEMA_REGEX => SchemaFile,
+      SMSES_REGEX => SmsFile,
+      USER_SCHEMA_REGEX => UserSchemaFile,
+      TRANSLATIONS_REGEX => TranslationFile,
+      CONFIG_REGEX => ConfigFile
+    }
+
+    attr_reader :storage
+
+    def initialize(storage)
+      @storage = storage
+    end
+
+    def grouped_files
+      @grouped_files ||= begin
+        hash = {}
+        REGEXP_MAP.each_value { |v| hash[v] = {} }
+        storage.files.each do |path|
+          regexp, klass = REGEXP_MAP.detect { |k, _v| k.match?(path) }
+          if regexp
+            f = klass.new(path, storage)
+            hash[klass][f.name] = f
+          elsif /\.liquid$/i.match?(path)
+            hash[LiquidFile] ||= {}
+            f = LiquidFile.new(path, storage)
+            hash[LiquidFile][f.name] = f
+          end
+        end
+        hash
+      end
+    end
+
+    def assets
+      grouped_files[AssetFile]&.values
+    end
+
+    def liquid
+      layouts + partials + pages + notifications
+    end
+
+    def yaml
+      schema + translations
+    end
+
+    def schema
+      grouped_files[SchemaFile]&.values || []
+    end
+
+    def translations
+      grouped_files[TranslationFile]&.values || []
+    end
+
+    def partials
+      grouped_files[PartialFile]&.values || []
+    end
+
+    def layouts
+      grouped_files[LayoutFile]&.values || []
+    end
+
+    def notifications
+      emails + smses + api_calls
+    end
+
+    def emails
+      grouped_files[EmailFile]&.values || []
+    end
+
+    def smses
+      grouped_files[SmsFile]&.values || []
+    end
+
+    def api_calls
+      grouped_files[ApiCallFile]&.values || []
+    end
+
+    def pages
+      grouped_files[PageFile]&.values || []
+    end
+
+    def directories
+      storage.directories
+    end
+
+    def all
+      @all ||= grouped_files.values.map(&:values).flatten
+    end
+
+    def [](name_or_relative_path)
+      case name_or_relative_path
+      when Pathname
+        all.find { |t| t.relative_path == name_or_relative_path }
+      else
+        all.find { |t| t.name == name_or_relative_path }
+      end
+    end
+
+    def sections
+      liquid.select(&:section?)
+    end
+
+    def snippets
+      liquid.select(&:snippet?)
+    end
+  end
+end

data/lib/platformos_check/app_file.rb

@@ -0,0 +1,89 @@
+# frozen_string_literal: true
+
+require "pathname"
+
+module PlatformosCheck
+  class AppFile
+    attr_reader :version, :storage
+
+    def initialize(relative_path, storage)
+      @relative_path = relative_path
+      @storage = storage
+      @source = nil
+      @version = nil
+      @eol = "\n"
+    end
+
+    def path
+      @storage.path(@relative_path)
+    end
+
+    def relative_path
+      @relative_pathname ||= Pathname.new(@relative_path)
+    end
+
+    def name
+      @name ||= dir_prefix.nil? ? relative_path.sub_ext('').to_s : build_name
+    end
+
+    def build_name
+      n = relative_path.sub(dir_prefix, '').sub_ext('').to_s
+      return n if module_name.nil?
+
+      prefix = "modules#{File::SEPARATOR}#{module_name}#{File::SEPARATOR}"
+      return n if n.start_with?(prefix)
+
+      "#{prefix}#{n}"
+    end
+
+    def dir_prefix
+      nil
+    end
+
+    def module_name
+      @module_name ||= begin
+        dir_names = @relative_path.split(File::SEPARATOR).reject(&:empty?)
+        dir_names.first == 'modules' ? dir_names[1] : nil
+      end
+    end
+
+    # For the corrector to work properly, we should have a
+    # simple mental model of the internal representation of eol
+    # characters (Windows uses \r\n, Linux uses \n).
+    #
+    # Parser::Source::Buffer strips the \r from the source file, so if
+    # you are autocorrecting the file you might lose that info and
+    # cause a git diff. It also makes the node.start_index/end_index
+    # calculation break. That's not cool.
+    #
+    # So in here we track whether the source file has \r\n in it and
+    # we'll make sure that the file we write has the same eol as the
+    # source file.
+    def source
+      return @source if @source
+
+      if @storage.versioned?
+        @source, @version = @storage.read_version(@relative_path)
+      else
+        @source = @storage.read(@relative_path)
+      end
+      @eol = @source.include?("\r\n") ? "\r\n" : "\n"
+      @source = @source
+                .gsub(/\r(?!\n)/, "\r\n") # fix rogue \r without followup \n with \r\n
+                .gsub("\r\n", "\n")
+    end
+
+    def yaml?
+      false
+    end
+
+    def liquid?
+      false
+    end
+
+    def ==(other)
+      other.is_a?(self.class) && relative_path == other.relative_path
+    end
+    alias eql? ==
+  end
+end