markly 0.15.0 → 0.15.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d9b96430051a815282d6f83d487fc0c0cdc636b174a1f1f775f0574950e8bab3
-  data.tar.gz: d16b1c808df1030e6bcc9c67c99f88a0eeae8d8e604b68745e0d8f4c16ac171d
+  metadata.gz: 712a0a7ad856598eb83b20c370e7613896596af5c76cc2379a368382802f09db
+  data.tar.gz: f775598d6fc6e4d5e340bf68518816d3bd23bc44facd209fcd59bd4f0d429639
 SHA512:
-  metadata.gz: 9e7e24631dbd95cb90ee889919be9bb6a267c6e2cf31e7355dd21f5e0c903a83a288178bffcf4430eed4ed58b3d2c014e8e3617629a9c703f063069a03cfeaf7
-  data.tar.gz: 2c51b26c4ce3d2467079ce968f0d2bae307090b49b59b0acfb7396ad00c140e276f76d9291ac4bc65e24e368005b6929990521de4749fecd678f1b6b0ed6d77d
+  metadata.gz: f13cbaaba186b5716efb05567721ea024633f0c353f7296721406dd7d4f7918625e5518ae4836371a784808ae99de8d4831bf938df6595ed26da462398c6e39f
+  data.tar.gz: a92d4ee382baaf2bea98defccfd318142f0fdccf42ff5fca5293c17bf611e432d21e178f3d69a272f0c021309d3725454e93ea401d7473495ff4620490071d5d

data/context/abstract-syntax-tree.md

@@ -0,0 +1,95 @@
+# Abstract Syntax Tree
+
+This guide explains how to use Markly's abstract syntax tree (AST) to parse and manipulate Markdown documents.
+
+## Parsing
+
+You can parse Markdown to a `Document` node using `Markly.parse`:
+
+~~~ ruby
+require 'markly'
+
+document = Markly.parse('*Hello* world')
+
+pp document 
+~~~
+
+This will print out the following:
+
+~~~
+#<Markly::Node(document):
+	source_position={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>13}
+	children=[#<Markly::Node(paragraph):
+			 source_position={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>13}
+			 children=[#<Markly::Node(emph):
+						source_position={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>7}
+						children=[#<Markly::Node(text): source_position={:start_line=>1, :start_column=>2, :end_line=>1, :end_column=>6}, string_content="Hello">]>,
+					#<Markly::Node(text): source_position={:start_line=>1, :start_column=>8, :end_line=>1, :end_column=>13}, string_content=" world">]>]>
+~~~
+
+As you can see, a document consists of a root node, which contains several children, they themselves containing children, and so on. We refer to this as the abstract syntax tree (AST).
+
+## Example: Walking the AST
+
+You can use `walk` or `each` to iterate over nodes:
+
+	- `walk` will iterate on a node and recursively iterate on a node's children.
+	- `each` will iterate on a node and its children, but no further.
+
+<!-- end list -->
+
+``` ruby
+require 'markly'
+
+document = Markly.parse("# The site\n\n [GitHub](https://www.github.com)")
+
+# Walk tree and print out URLs for links:
+document.walk do |node|
+	if node.type == :link
+		puts "URL = #{node.url}"
+	end
+end
+
+# Capitalize all regular text in headers:
+document.walk do |node|
+	if node.type == :header
+		node.each do |subnode|
+			if subnode.type == :text
+				subnode.string_content = subnode.string_content.upcase
+			end
+		end
+	end
+end
+
+# Transform links to regular text:
+document.walk do |node|
+	if node.type == :link
+		node.insert_before(node.first_child)
+		node.delete
+	end
+end
+```
+
+### Creating a Custom Renderer
+
+You can also derive a class from {ruby Markly::Renderer::HTML} class. Using a pure Ruby renderer is slower, but allows you to customize the output. For example:
+
+``` ruby
+class MyHtmlRenderer < Markly::Renderer::HTML
+	def initialize
+		super
+		@header_id = 1
+	end
+
+	def header(node)
+		block do
+			out("<h", node.header_level, " id=\"", @header_id, "\">",
+							 :children, "</h", node.header_level, ">")
+			@header_id += 1
+		end
+	end
+end
+
+my_renderer = MyHtmlRenderer.new
+puts my_renderer.render(document)
+```

data/context/getting-started.md

@@ -0,0 +1,101 @@
+# Getting Started
+
+This guide explains now to install and use Markly.
+
+## Installation 
+
+Add the gem to your project:
+
+	$ bundle add markly
+
+## Usage
+
+Markly's most basic usage is to convert Markdown to HTML. You can do this in a few ways:
+
+~~~ ruby
+require 'markly'
+
+Markly.render_html('Hi *there*')
+# <p>Hi <em>there</em></p>\n
+~~~
+
+You can also parse a string to receive a `Document` node. You can then print that node to HTML, iterate over the children, and other fun node stuff. For example:
+
+~~~ ruby
+require 'markly'
+
+document = Markly.parse('*Hello* world')
+puts(document.to_html) # <p>Hi <em>there</em></p>\n
+
+document.walk do |node|
+	puts node.type # [:document, :paragraph, :text, :emph, :text]
+end
+~~~
+
+## Options
+
+Markly accepts integer flags which control how the Markdown is parsed and rendered.
+
+### Parse Options
+
+| Name                                 | Description
+| ------------------------------------ | -----------
+| `Markly::DEFAULT`                    | The default parsing system.
+| `Markly::UNSAFE`                     | Allow raw/custom HTML and unsafe links.
+| `Markly::FOOTNOTES`                  | Parse footnotes.
+| `Markly::LIBERAL_HTML_TAG`           | Support liberal parsing of inline HTML tags.
+| `Markly::SMART`                      | Use smart punctuation (curly quotes, etc.).
+| `Markly::STRIKETHROUGH_DOUBLE_TILDE` | Parse strikethroughs by double tildes (compatibility with [redcarpet](https://github.com/vmg/redcarpet))
+| `Markly::VALIDATE_UTF8`              | Replace illegal sequences with the replacement character `U+FFFD`.
+
+### Render Options
+
+| Name                                    | Description                                                     |
+| --------------------------------------- | --------------------------------------------------------------- |
+| `Markly::DEFAULT`                       | The default rendering system.                                   |
+| `Markly::UNSAFE`                        | Allow raw/custom HTML and unsafe links.                         |
+| `Markly::GITHUB_PRE_LANG`               | Use GitHub-style `<pre lang>` for fenced code blocks.           |
+| `Markly::HARD_BREAKS`                   | Treat `\n` as hardbreaks (by adding `<br/>`).                   |
+| `Markly::NO_BREAKS`                     | Translate `\n` in the source to a single whitespace.            |
+| `Markly::SOURCE_POSITION`               | Include source position in rendered HTML.                       |
+| `Markly::TABLE_PREFER_STYLE_ATTRIBUTES` | Use `style` insted of `align` for table cells.                  |
+| `Markly::FULL_INFO_STRING`              | Include full info strings of code blocks in separate attribute. |
+
+### Passing Options
+
+To apply a single option, pass it in as a flags option:
+
+``` ruby
+Markly.parse("\"Hello,\" said the spider.", flags: Markly::SMART)
+# <p>“Hello,” said the spider.</p>\n
+```
+
+To have multiple options applied, `|` (or) the flags together:
+
+``` ruby
+Markly.render_html("\"'Shelob' is my name.\"", flags: Markly::HARD_BREAKS|Markly::SOURCE_POSITION)
+```
+
+## Extensions
+
+Both `render_html` and `parse` take an optional `extensions:` argument defining the extensions you want enabled as your CommonMark document is being processed:
+
+``` ruby
+Markly.render_html("<script>hi</script>", flags: Markly::UNSAFE, extensions: [:tagfilter])
+```
+
+The documentation for these extensions are [defined in this spec](https://github.github.com/gfm/), and the rationale is provided [in this blog post](https://githubengineering.com/a-formal-spec-for-github-markdown/).
+
+The available extensions are:
+
+  - `:table` - This provides support for tables.
+  - `:tasklist` - This provides support for task list items.
+  - `:strikethrough` - This provides support for strikethroughs.
+  - `:autolink` - This provides support for automatically converting URLs to anchor tags.
+  - `:tagfilter` - This escapes [several "unsafe" HTML tags](https://github.github.com/gfm/#disallowed-raw-html-extension-), causing them to not have any effect.
+
+## Developing Locally
+
+After cloning the repo:
+
+	$ bake build test

data/context/headings.md

@@ -0,0 +1,116 @@
+# Headings
+
+This guide explains how to work with headings in Markly, including extracting them for navigation and handling duplicate heading text.
+
+## Unique ID Generation
+
+When rendering HTML with `ids: true`, duplicate heading text automatically gets unique IDs to avoid collisions. This is particularly useful when multiple sections have the same title (e.g., multiple "Deployment" sections under different parent headings).
+
+``` ruby
+markdown = <<~MARKDOWN
+  ## Kubernetes
+  
+  ### Deployment
+  
+  ## Systemd
+  
+  ### Deployment
+MARKDOWN
+
+renderer = Markly::Renderer::HTML.new(ids: true)
+html = renderer.render(Markly.parse(markdown))
+
+# Generates:
+# <section id="kubernetes">...</section>
+# <section id="deployment">...</section>
+# <section id="systemd">...</section>
+# <section id="deployment-2">...</section>
+```
+
+The first occurrence gets the clean ID, subsequent duplicates get numbered suffixes (`-2`, `-3`, etc.).
+
+## Extracting Headings for Table of Contents
+
+The `Headings` class can extract headings for building navigation or table of contents:
+
+``` ruby
+document = Markly.parse(markdown)
+headings = Markly::Renderer::Headings.extract(document, min_level: 2, max_level: 3)
+
+headings.each do |heading|
+  puts "#{heading.level}: #{heading.text} (#{heading.anchor})"
+end
+
+# Output:
+# 2: Kubernetes (kubernetes)
+# 3: Deployment (deployment)
+# 2: Systemd (systemd)
+# 3: Deployment (deployment-2)
+```
+
+Each `Heading` object has:
+- `level` - The heading level (1-6)
+- `text` - The plain text content
+- `anchor` - The unique ID/anchor
+- `node` - The original Markly AST node
+
+### Level Filtering
+
+Use `min_level` and `max_level` to filter which heading levels to extract:
+
+``` ruby
+# Only extract h2 and h3 headings
+headings = Markly::Renderer::Headings.extract(document, min_level: 2, max_level: 3)
+
+# Only h1 headings
+headings = Markly::Renderer::Headings.extract(document, min_level: 1, max_level: 1)
+```
+
+## Custom Heading Strategies
+
+For advanced use cases, you can provide a custom `Headings` instance to the HTML renderer:
+
+### Sharing State Across Documents
+
+To ensure IDs remain unique across multiple documents:
+
+``` ruby
+# Share heading state across multiple documents
+headings = Markly::Renderer::Headings.new
+renderer = Markly::Renderer::HTML.new(headings: headings)
+
+doc1_html = renderer.render(Markly.parse(doc1_markdown))
+doc2_html = renderer.render(Markly.parse(doc2_markdown))
+# IDs remain unique across both documents
+```
+
+### Custom ID Generation
+
+Subclass `Headings` to implement alternative ID generation strategies:
+
+``` ruby
+class HierarchicalHeadings < Markly::Renderer::Headings
+  def initialize
+    super
+    @parent_context = []
+  end
+  
+  def anchor_for(node)
+    base = base_anchor_for(node)
+    
+    # Custom logic: could incorporate parent heading context
+    # to generate IDs like "kubernetes-deployment" instead of "deployment-2"
+    
+    if @ids.key?(base)
+      @ids[base] += 1
+      "#{base}-#{@ids[base]}"
+    else
+      @ids[base] = 1
+      base
+    end
+  end
+end
+
+renderer = Markly::Renderer::HTML.new(headings: HierarchicalHeadings.new)
+```
+

data/context/index.yaml

@@ -0,0 +1,20 @@
+# Automatically generated context index for Utopia::Project guides.
+# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
+---
+description: CommonMark parser and renderer. Written in C, wrapped in Ruby.
+metadata:
+  documentation_uri: https://ioquatix.github.io/markly/
+  funding_uri: https://github.com/sponsors/ioquatix/
+  source_code_uri: https://github.com/ioquatix/markly.git
+files:
+- path: getting-started.md
+  title: Getting Started
+  description: This guide explains now to install and use Markly.
+- path: abstract-syntax-tree.md
+  title: Abstract Syntax Tree
+  description: This guide explains how to use Markly's abstract syntax tree (AST)
+    to parse and manipulate Markdown documents.
+- path: headings.md
+  title: Headings
+  description: This guide explains how to work with headings in Markly, including
+    extracting them for navigation and handling duplicate heading text.

data/lib/markly/version.rb

@@ -7,5 +7,5 @@
 # Copyright, 2020-2025, by Samuel Williams.
 
 module Markly
-	VERSION = "0.15.0"
+	VERSION = "0.15.1"
 end

data/readme.md

@@ -28,6 +28,10 @@ Please see the [project documentation](https://ioquatix.github.io/markly/) for m
 
 Please see the [project releases](https://ioquatix.github.io/markly/releases/index) for all releases.
 
+### v0.15.1
+
+  - Add agent context.
+
 ### v0.15.0
 
   - Introduced `Markly::Renderer::Headings` class for extracting headings from markdown documents with automatic duplicate ID resolution. When rendering HTML with `ids: true`, duplicate heading text now automatically gets unique IDs (`deployment`, `deployment-2`, `deployment-3`). The `Headings` class can also be used to extract headings for building navigation or table of contents.

data/releases.md

@@ -1,5 +1,9 @@
 # Releases
 
+## v0.15.1
+
+  - Add agent context.
+
 ## v0.15.0
 
   - Introduced `Markly::Renderer::Headings` class for extracting headings from markdown documents with automatic duplicate ID resolution. When rendering HTML with `ids: true`, duplicate heading text now automatically gets unique IDs (`deployment`, `deployment-2`, `deployment-3`). The `Headings` class can also be used to extract headings for building navigation or table of contents.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: markly
 version: !ruby/object:Gem::Version
-  version: 0.15.0
+  version: 0.15.1
 platform: ruby
 authors:
 - Garen Torikian
@@ -62,6 +62,10 @@ extensions:
 - ext/markly/extconf.rb
 extra_rdoc_files: []
 files:
+- context/abstract-syntax-tree.md
+- context/getting-started.md
+- context/headings.md
+- context/index.yaml
 - ext/markly/arena.c
 - ext/markly/autolink.c
 - ext/markly/autolink.h