lookbooklet 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: '0068d04263ac310030a31b1269136504f3149292365ecf2b230ff5a0dcb50f80'
+  data.tar.gz: 4b9d16e8e6b9a340c52f61ea95fa7e4b260e2d927e60c3955cdefb9363f1ddb6
+SHA512:
+  metadata.gz: 7ddb7794ef885c0c23991692bf096b79fad733a494737a20d3db02656154741b45f1bf41bf2d799a1b66fffad08a6967db24bb1ed60b20c326624af4ed2c1fd3
+  data.tar.gz: dc2286fe4440f0d391fb9be637eb0d655d8c92e4b86c2ee7c60375940f22275f9303eaa32c8479e4730e40167012a0bb3e0f689ef30973aa1d6abfbe743fa1bb

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Mark Perkins
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,234 @@
+# Booklet
+
+An experimental new standalone, extendable **parser-analyzer engine** for [Lookbook](https://lookbook.build).
+
+The aim is for Booklet to eventually replace the existing file parsing/analyzing code in Lookbook and thus provide a more robust foundation for future releases to build upon.
+
+Additionally, Booklet is being developed as a standalone gem in part so that it can also be used as a springboard for building additional tools (CLIs, MCP servers etc) to help Lookbook integrate more seamlessly into a variety of Rails frontend development workflows.
+
+> [!WARNING]
+> Booklet is in a very early stage of development and is **not ready for public use.**<br>It is currently incomplete and will likely see many breaking changes before a stable release candidate is available.
+
+
+### Aims and objectives
+
+* Use common, tried-and-tested parser/transformer implementation patterns over custom workflows where possible.
+* Implement the analyzing process as a series of small, incremental steps to aid comprehension and testabilty.
+* Be backwards compatable with existing Lookbook parser behaviour as surfaced in the app (although underlying APIs and conceptual definitions may differ).
+* Make it straightforward to customise and extend the processing pipeline using plugins/middleware.
+* Minimise any 'special-casing' of core functionality over that provided by third party extensions.
+* Do not have any dependency on Rails (outside of conveniences provided by the `ActiveSupport` gem).
+
+### Development status
+
+Booklet is a brand new project and is not yet ready for public use. 
+
+The [issues list](https://github.com/lookbook-hq/booklet/issues) contains an (incomplete) set of work that is planned before an initial beta release will be made available. 
+
+Issues specifically related to achieving compatability with Lookbook's current parser output are tagged with the `compatability` label and the [Lookbook compatability](https://github.com/orgs/lookbook-hq/projects/3) project board has been set up to track progress on this metric.
+
+## Implementation details
+
+Booklet generates a **traversable tree of entity node objects** from an input directory of files, via a number of intermediate steps.
+
+Trees can contain a number of different node types. These include **folder nodes**, **file-based entity nodes** and entity **child nodes** that represent the contents of key resource types.
+
+The hierarchy of the nodes in the tree broadly reflects the grouping of input files into folders and subfolders within the root directory as well as parent-child entity-content relationships where present.
+
+All tree mutations and transformations are performed by ['double dispatch'-style](https://www.bigbinary.com/blog/visitor-pattern-and-double-dispatch) **node visitors**.
+
+### File processing pipeline
+
+Booklet breaks up the processing of files into four steps:
+
+1. File tree creation
+2. File tree mutation
+3. File tree &rarr; entity tree transformation
+4. Entity tree mutation
+  
+#### 1. File tree creation
+
+In the first step a tree of generic file and directory nodes is constructed by recursively scanning the root directory and adding a node for every file and folder found. This is handled by the [`FilesystemLoader` visitor](./lib/booklet/visitors/filesystem_loader.rb).
+
+```ruby
+file_tree = DirectoryNode.from("test/fixtures/demo").accept(FilesystemLoader.new)
+```
+
+<details>
+<summary>Resulting file tree</summary>
+
+> _ASCII tree visualisation generated using the [`AsciiTreeRenderer` visitor](./lib/booklet/visitors/ascii_tree_renderer.rb)_
+
+```
+ [DirectoryNode] demo
+ ├── [DirectoryNode] docs
+ │   ├── [FileNode] _tmp_notes.txt
+ │   ├── [FileNode] banner.png
+ │   ├── [FileNode] overview.md
+ │   ├── [FileNode] resources.md
+ │   └── [DirectoryNode] usage
+ │       ├── [FileNode] getting_started.md
+ │       ├── [FileNode] installation.md
+ │       └── [FileNode] screenshot.svg
+ └── [DirectoryNode] view_specs
+     ├── [DirectoryNode] elements
+     │   ├── [FileNode] button_component_spec.rb
+     │   └── [FileNode] card_component_spec.rb
+     ├── [FileNode] helpers_spec.rb
+     └── [DirectoryNode] layouts
+         ├── [FileNode] article_spec.rb
+         └── [FileNode] landing_page_spec.rb
+```
+</details>
+
+#### 2. File tree mutation
+
+This is an optional step where additional node visitors can be provided to mutate the file tree before it is handed off to the entity transformer.
+
+For example you might want to create a custom Visitor class that removes all files with names beginning with an underscore (such as `_tmp_notes.txt` in the example above) to clean up the tree before the entity transformer is run.
+
+```ruby
+file_tree.accept(SomeCustomFileTreeProcessor.new)
+```
+
+#### 3. File tree &rarr; Entity tree transformation
+
+In this step the [`EntityTransformer`](./lib/booklet/visitors/entity_transformer.rb) visitor is applied to the raw file tree. The transformer visits each of the generic file/directory nodes in the file tree and converts all 'recognized' file types to their corresponding Lookbook entity node type.
+
+For example, files with `.md` extensions are transformed into `DocumentNode` instances, whilst component preview class files (names ending in `_preview.rb`) are transformed into `SpecNode` instances.
+
+```ruby
+entity_tree = file_tree.accept(EntityTransformer.new)
+```
+
+<details>
+<summary>Resulting entity tree</summary>
+
+```
+ [FolderNode] Demo
+ ├── [FolderNode] Docs
+ │   ├── [AnonNode] _tmp_notes.txt
+ │   ├── [AssetNode] banner.png
+ │   ├── [DocumentNode] Overview
+ │   ├── [DocumentNode] Resources
+ │   └── [FolderNode] Usage
+ │       ├── [DocumentNode] Getting Started
+ │       ├── [DocumentNode] Installation
+ │       └── [AssetNode] screenshot.svg
+ └── [FolderNode] Previews
+     ├── [FolderNode] Elements
+     │   ├── [SpecNode] Button Component Preview
+     │   └── [SpecNode] Card Component Preview
+     ├── [SpecNode] Helpers Preview
+     └── [FolderNode] Layouts
+         ├── [SpecNode] Article Preview
+         └── [SpecNode] Landing Page Preview
+```
+</details>
+
+
+#### 4. Entity tree mutation
+
+This final step is where enity node visitors can be applied to perform tasks such as parsing file contents and generally 'building out' the skeleton entity node objects created in the previous step. 
+
+By default Booklet will apply the [`PreviewClassParser`](./lib/booklet/visitors/preview_class_parser.rb) and [`FrontmatterExtractor`](./lib/booklet/visitors/frontmatter_extractor.rb) visitors at this stage.
+
+```ruby
+entity_tree
+  .accept(PreviewClassParser.new)
+  .accept(FrontMatterExtractor.new)
+```
+
+* The `PreviewClassParser` visitor uses the [YARD parser](https://yardoc.org/) to extract annotations data from preview class files and creates and appends corresponding `ScenarioNode` and `ProseNode` children to the appropriate `SpecNode` instance.
+* The `FrontmatterExtractor` visitor _(not yet implemented!)_ extracts YAML-formatted 'frontmatter' from the contents of markdown files and updates the related `DocumentNode` instances with the parsed data.
+
+Additional entity node vistors can be applied here as needed to make changes to the entity tree nodes before the finalised entity tree is returned for use by the calling code. 
+
+<details>
+<summary>Final entity tree</summary>
+
+_Note that the `docs` branch has been omitted for brevity._
+
+```
+...
+└── [FolderNode] Previews
+    ├── [FolderNode] Elements
+    │   ├── [SpecNode] Button Component Preview
+    │   │   ├── [ScenarioNode] Default
+    │   │   ├── [ScenarioNode] Secondary
+    │   │   └── [ScenarioNode] Danger
+    │   └── [SpecNode] Card Component Preview
+    │       ├── [ScenarioNode] No Title
+    │       └── [ScenarioNode] With Title
+    ├── [SpecNode] Helpers Preview
+    │   ├── [ScenarioNode] Blah Generator
+    │   └── [ScenarioNode] Char Tag Wrapper
+    └── [FolderNode] Layouts
+        ├── [SpecNode] Article Preview
+        │   └── [ScenarioNode] Default
+        └── [SpecNode] Landing Page Preview
+            └── [ScenarioNode] Default
+```
+
+</details>
+
+## Installation
+
+> [!IMPORTANT]
+> Booklet is not yet ready for public use - these instructions are for illustrative purposes only at this point.
+> See the [Development status](#development-status) section for more info.
+
+Booklet is both a **command line tool** and a **library**.
+
+### Using as a dependency
+
+Add Booklet to your Gemfile:
+
+```ruby
+gem "lookbooklet"
+```
+
+After running `bundle install` you can then make use of the Booklet API in your codebase:
+
+```ruby
+require "lookbooklet"
+
+result = Booklet.analyze("path/to/root/directory")
+```
+
+### CLI interface
+
+To use the booklet CLI you can install the gem globally:
+
+```sh
+gem install lookbooklet
+```
+
+You can then view the available booklet CLI commands using:
+
+```sh
+booklet -h
+```
+
+## API
+
+> _Details coming soon._
+
+
+## Testing
+
+Booklet uses Minitest for its test framework.
+
+Run the tests: 
+
+```sh
+bin/test
+```
+
+## Acknowlegments
+
+[Marco Roth](https://marcoroth.dev/)'s fantastic work on [Herb](https://herb-tools.dev/) (and my subsequent deep-dive into the world of ASTs) was been instrumental in sparking the initial idea for Booklet and for shaping its approach.
+
+Booklet's double-dispatch style node visitor base class is based on the very nice [BasicVisitor](https://github.com/yippee-fun/refract/blob/main/lib/refract/basic_visitor.rb) class from the [Refract gem](https://github.com/yippee-fun/refract). 
+
+In addition much of the original incarnation of Booklet's `Node` class was based on code adapted from the excellent [RubyTree](https://github.com/evolve75/RubyTree) gem.

data/bin/booklet

@@ -0,0 +1,6 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "booklet"
+
+Booklet::CLI.call

data/lib/booklet/analyzer.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+module Booklet
+  class Analyzer < Booklet::Object
+    DEFAULT_FILE_VISITORS = []
+    DEFAULT_ENTITY_VISITORS = [
+      PreviewClassParser,
+      FrontmatterExtractor
+    ]
+
+    prop :loader, Visitor, default: -> { FilesystemLoader.new }
+    prop :after_load, _Array(Visitor), reader: :public, default: -> { DEFAULT_FILE_VISITORS.map(&:new) }
+
+    prop :transformer, Visitor, default: -> { EntityTransformer.new }
+    prop :after_transform, _Array(Visitor), reader: :public, default: -> { DEFAULT_ENTITY_VISITORS.map(&:new) }
+
+    def analyze(path)
+      path = Pathname(path.to_s).expand_path unless path.nil?
+      files = DirectoryNode.from(path).accept(@loader)
+      @after_load.each { files.accept(_1) }
+
+      entities = files.accept(@transformer)
+      @after_transform.each { entities.accept(_1) }
+
+      ParserResult.new(path:, files:, entities:)
+    end
+  end
+end

data/lib/booklet/cli/cli.rb

@@ -0,0 +1,14 @@
+require "dry/cli"
+
+module Booklet
+  module CLI
+    extend Dry::CLI::Registry
+
+    register "version", VersionCommand, aliases: ["v", "-v", "--version"]
+    register "analyze", AnalyzeCommand
+
+    def self.call
+      Dry::CLI.new(self).call
+    end
+  end
+end

data/lib/booklet/cli/commands/analyze_command.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+module Booklet
+  class AnalyzeCommand < Dry::CLI::Command
+    include Colorful
+
+    desc "Analyze a directory of files and display a summary of the results"
+
+    argument :path, required: true, desc: "Root directory path"
+
+    def call(path:, **)
+      result = Booklet.analyze(path)
+
+      files = result.files.count
+      entities = result.entities.count
+      warnings = result.warnings.count
+      errors = result.errors.count
+
+      hr = "⎯" * 20
+
+      puts <<~RESULT
+
+        #{hr}
+
+        #{cyan("#{files} #{"file".pluralize(files)}...")}
+
+        #{result.files.accept(AsciiTreeRenderer.new(indent: 1))}
+
+        #{cyan("...converted into #{entities} #{"entity".pluralize(entities)}:")}
+
+        #{result.entities.accept(AsciiTreeRenderer.new(indent: 1))}
+
+        #{hr}
+
+        #{green("Booklet analysis complete", :underline)}
+
+        #{cyan("Root:".ljust(9))} #{path}
+        #{cyan(("File".pluralize(files) + ":").ljust(9))} #{files}
+        #{cyan(("Entity".pluralize(entities) + ":").ljust(9))} #{entities}
+        #{yellow(("Warning".pluralize(warnings) + ":").ljust(9))} #{warnings}
+        #{red(("Error".pluralize(errors) + ":").ljust(9))} #{errors}
+      RESULT
+    end
+  end
+end

data/lib/booklet/cli/commands/version_command.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Booklet
+  class VersionCommand < Dry::CLI::Command
+    include Colorful
+
+    desc "Print version"
+
+    def call(*)
+      puts "#{pink("booklet")} v#{VERSION}"
+    end
+  end
+end

data/lib/booklet/cli/concerns/colorful.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+require "paint"
+
+module Booklet
+  module Colorful
+    extend ActiveSupport::Concern
+
+    included do
+      def pink(str, *)
+        paint(str, "#FF1493", *)
+      end
+
+      def cyan(str, *)
+        paint(str, :cyan, *)
+      end
+
+      def magenta(str, *)
+        paint(str, :cyan, *)
+      end
+
+      def gray(str, *)
+        paint(str, :gray, *)
+      end
+
+      def green(str, *)
+        paint(str, :green, *)
+      end
+
+      def orange(str, *)
+        paint(str, "#FFA500", *)
+      end
+
+      def yellow(str, *)
+        paint(str, "#DAA520", *)
+      end
+
+      def gold(str, *)
+        paint(str, "#DAA520", *)
+      end
+
+      def red(str, *)
+        paint(str, :red, *)
+      end
+
+      def paint(*)
+        Paint[*]
+      end
+    end
+  end
+end

data/lib/booklet/concerns/locatable.rb

@@ -0,0 +1,22 @@
+module Booklet
+  module Locatable
+    extend ActiveSupport::Concern
+
+    included do
+      prop :file, File, reader: :public, writer: false
+
+      delegate :name, :path, to: :file
+
+      def label
+        name.titleize
+      end
+    end
+
+    class_methods do
+      def from(file_or_path)
+        file = file_or_path.is_a?(File) ? file_or_path : File.new(file_or_path)
+        new(file.path, file:)
+      end
+    end
+  end
+end

data/lib/booklet/file.rb

@@ -0,0 +1,90 @@
+# frozen_string_literal: true
+
+require "marcel"
+
+module Booklet
+  class File < Booklet::Object
+    prop :path, Pathname, :positional do |value|
+      Pathname(value.to_s) unless value.nil?
+    end
+
+    prop :contents, _Nilable(String), :positional
+
+    after_initialize do
+      raise ArgumentError, "File paths must be absolute" unless @path.absolute?
+    end
+
+    def path(strip_extension: false)
+      strip_extension ? Pathname(@path.to_s.delete_suffix(ext)) : @path
+    end
+
+    def relative_path(root = Dir.pwd, strip_extension: false)
+      path(strip_extension).relative_path_from(root)
+    end
+
+    def relative_path_segments
+      relative_path.to_s.split("/")
+    end
+
+    def ext
+      basename.to_s.gsub(/^([^.]+)/, "") unless directory?
+    end
+
+    def ext?(*extensions)
+      basename.end_with?(*extensions)
+    end
+
+    def name
+      (directory? ? basename : basename(ext)).to_s
+    end
+
+    def basename(*)
+      path.basename(*).to_s
+    end
+
+    def mime_type
+      Marcel::MimeType.for(path, name: basename) unless directory?
+    end
+
+    def mtime
+      ::File.mtime(path)
+    end
+
+    def contents
+      raise "Cannot read contents of a directory" if directory?
+
+      @contents || ::File.read(path)
+    end
+
+    def parent_directory_of?(child_path)
+      return false unless directory?
+
+      self.class.new(child_path).dirname == path
+    end
+
+    def anscestor_directory_of?(descendant_path)
+      return false unless directory?
+
+      self.class.new(descendant_path).path.to_s.start_with?("#{path}/")
+    end
+
+    def to_h
+      {path:, basename:, name:, ext:, directory: directory?}.compact
+    end
+
+    def to_s
+      path.to_s
+    end
+
+    alias_method :value, :path
+    alias_method :to_pathname, :path
+
+    delegate_missing_to :path
+
+    class << self
+      def file?(file)
+        file.is_a?(File)
+      end
+    end
+  end
+end

data/lib/booklet/helpers.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module Booklet
+  module Helpers
+    def strip_indent(value)
+      value.to_s.gsub(/^#{value.scan(/^[ \t]*(?=\S)/).min}/, "").strip
+    end
+
+    def strip_whitespace(value)
+      value.to_s.strip
+    end
+
+    def hexdigest(str)
+      Digest::MD5.hexdigest(str.to_s)[0..6] if str.present?
+    end
+
+    extend self
+  end
+end