kramdown-syntax_tree_sitter 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 02e58047b1cac1778ad13d25f88a64a42f0714faeba928322851a78afeb4ce66
4
- data.tar.gz: 87cc78c75fb83a9a56cca8bb663f72b1d3abbd03f3920e37bad6ac629c699d69
3
+ metadata.gz: '09858b648cc381cb603b21a3ed091b1d96a25b55264f4dd2c4e8fe7436d68288'
4
+ data.tar.gz: 9ceeb8b78ac8cea098cbb329abbdc9e650202e676b042b647b3f0ca55665c3ed
5
5
  SHA512:
6
- metadata.gz: 3b609ade16479c2e09504248970b1f95eb496b982cc641e766881dbfc74bedd580a45bc5f0c24f9b859535a3967a74e51126e6dfdf8df7448a5284da259e3105
7
- data.tar.gz: ae2cdab47a180b7435fc9c01bff81aefb377a353af341dc98f3f5bd8685105f6b976d35c25a900fcc7caa8744e3c34ef3fdbb63fb09dc27d52ee63f67644466e
6
+ metadata.gz: a3a7cf735cab71d863433b3be34e8b1dd537cca412bfddb611b56d4ff8292b74f0f45283da701510d928d432ad4aa39224316a012eb07cbd524cc1977bfb3a2e
7
+ data.tar.gz: 23197c154c12ca2e97534829179dd2785c9907760f89205a7a7be3e80f3cca7f8f6dd2c6314897a887e1c45b55eaddbd05a2937321017378caa0d03e33e450a5
data/README.md CHANGED
@@ -1,5 +1,8 @@
1
1
  # Kramdown Tree-sitter Highlighter
2
2
 
3
+ [![Build status](https://img.shields.io/github/actions/workflow/status/andrewtbiehl/kramdown-syntax_tree_sitter/quality-control.yml?branch=main&style=flat-square&logo=github)](https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter/actions/workflows/quality-control.yml)
4
+ [![RubyGems page](https://img.shields.io/gem/v/kramdown-syntax_tree_sitter?style=flat-square&color=blue&logo=rubygems&logoColor=white)](https://rubygems.org/gems/kramdown-syntax_tree_sitter)
5
+
3
6
  ***Syntax highlight code with [Tree-sitter](https://tree-sitter.github.io/tree-sitter)
4
7
  via [Kramdown](https://kramdown.gettalong.org).***
5
8
 
@@ -19,8 +22,8 @@ essentially an adapter for the
19
22
  hence also requires a compatible [Rust](https://www.rust-lang.org) installation to
20
23
  function. It is officially compatible with the following environments:
21
24
 
22
- - **Ruby**: 2.7, 3.0, 3.1
23
- - **Rust**: 1.61, 1.62, 1.63, 1.64, 1.65
25
+ - **Ruby**: 2.7, 3.0, 3.1, 3.2
26
+ - **Rust**: 1.62, 1.63, 1.64, 1.65, 1.66
24
27
  - **Platforms**: MacOS, Linux
25
28
 
26
29
  ### Installation
@@ -29,14 +32,13 @@ For projects using [Bundler](https://bundler.io) for dependency management, run
29
32
  following command to both install the gem and add it to the Gemfile:
30
33
 
31
34
  ```shell
32
- bundle add kramdown-syntax_tree_sitter --github andrewtbiehl/kramdown-syntax_tree_sitter
35
+ bundle add kramdown-syntax_tree_sitter
33
36
  ```
34
37
 
35
- Otherwise, download this project's repository and then run the following command from
36
- within it to build and install the gem:
38
+ Otherwise, install the gem via the following command:
37
39
 
38
40
  ```shell
39
- gem build && gem install kramdown-syntax_tree_sitter
41
+ gem install kramdown-syntax_tree_sitter
40
42
  ```
41
43
 
42
44
  ## Usage
@@ -54,7 +56,7 @@ require 'kramdown'
54
56
  require 'kramdown/syntax_tree_sitter'
55
57
 
56
58
  text = <<~MARKDOWN
57
- ~~~source.python
59
+ ~~~python
58
60
  print('Hello, World!')
59
61
  ~~~
60
62
  MARKDOWN
@@ -62,6 +64,13 @@ MARKDOWN
62
64
  Kramdown::Document.new(text, syntax_highlighter: :'tree-sitter').to_html
63
65
  ```
64
66
 
67
+ ### General usage
68
+
69
+ To successfully highlight input text via Kramdown with this plugin enabled, make sure
70
+ that every language referenced has a corresponding Tree-sitter parser library installed.
71
+ See the subsequent ['Language identifiers'](#language-identifiers) and
72
+ ['Tree-sitter parsers'](#tree-sitter-parsers) sections for more information.
73
+
65
74
  ### Usage with Jekyll
66
75
 
67
76
  This plugin can be used with the popular static site generator
@@ -80,11 +89,9 @@ kramdown:
80
89
  # Other Kramdown options...
81
90
  ```
82
91
 
83
- To highlight every code block in a Jekyll project via the plugin, make sure that every
84
- language identifier is expressed with the correct Tree-sitter scope and that every
85
- language referenced has a corresponding Tree-sitter parser library installed. See the
86
- subsequent ['Language identifiers'](#language-identifiers) and
87
- ['Tree-sitter parsers'](#tree-sitter-parsers) sections for more information.
92
+ Note that this plugin's general usage prerequisites (described in the previous
93
+ ['General usage'](#general-usage) section) still apply when using this plugin with
94
+ Jekyll.
88
95
 
89
96
  Also, there are multiple ways to render highlighted code blocks with Jekyll, as
90
97
  illustrated in the following table:
@@ -101,13 +108,13 @@ illustrated in the following table:
101
108
 
102
109
  ---
103
110
  ````
104
- ```source.python
111
+ ```python
105
112
  print('Hello, World!')
106
113
  ```
107
114
 
108
115
  or
109
116
 
110
- ~~~source.python
117
+ ~~~python
111
118
  print('Hello, World!')
112
119
  ~~~
113
120
  ````
@@ -123,7 +130,7 @@ print('Hello, World!')
123
130
  ---
124
131
  ```
125
132
  print('Hello, World!')
126
- {: class="language-source.python" }
133
+ {: class="language-python" }
127
134
  ```
128
135
  ---
129
136
 
@@ -136,7 +143,7 @@ print('Hello, World!')
136
143
 
137
144
  ---
138
145
  ```
139
- {% highlight source.python %}
146
+ {% highlight python %}
140
147
  print('Hello, World!')
141
148
  {% endhighlight %}
142
149
  ```
@@ -169,21 +176,71 @@ can be found on
169
176
 
170
177
  ### Language identifiers
171
178
 
172
- Tree-sitter uses a string-based identifier called a 'scope' to identify each language.
173
- For example, the scope string for Python is 'source.python', whereas for HTML it is
174
- 'text.html.basic'. Currently, this plugin follows this same convention, so a given code
175
- block will only be correctly highlighted if the language identifier provided for that
176
- code block is its language's corresponding Tree-sitter scope string. This is illustrated
177
- by the code block used in the [Quickstart](#quickstart) example.
179
+ For most popular languages, this plugin recognizes the same language identifiers as used
180
+ by the popular highlighter [Rouge](https://github.com/rouge-ruby/rouge). For example,
181
+ both 'python' and 'py' may be used to identify a code block written in Python. For the
182
+ complete list of Rouge language identifiers recognized by this plugin, see
183
+ [`lib/kramdown/syntax_tree_sitter/languages.rb`](https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter/blob/main/lib/kramdown/syntax_tree_sitter/languages.rb).
184
+
185
+ This plugin also automatically supports the language identifier format used by
186
+ Tree-sitter, known as as the 'scope' name. For example, the scope name for Python is
187
+ 'source.python', whereas for HTML it is 'text.html.basic'.
188
+
189
+ For any identifiers currently supported by Rouge but not by this plugin, feel free to
190
+ open an issue or pull request on GitHub to have them included.
191
+
192
+ ### CSS styling
193
+
194
+ Code highlights can be rendered either via inline CSS styles or via CSS classes, which
195
+ are applied to each token in the parsed code.
196
+
197
+ To use CSS classes for highlighting, set the `css_classes` Kramdown syntax-highlighting
198
+ option to `true`. Otherwise, the plugin will apply highlights via inline CSS styles.
199
+
200
+ The inline CSS styles are derived from Tree-sitter's built-in default highlighting
201
+ theme, repeated here for convenience:
202
+
203
+ | Token name | CSS style |
204
+ | :-- | :-- |
205
+ | attribute | `color: #af0000; font-style: italic;` |
206
+ | comment | `color: #8a8a8a; font-style: italic;` |
207
+ | constant | `color: #875f00;` |
208
+ | function | `color: o#005fd7;` |
209
+ | function.builtin | `color: #005fd7; font-weight: bold;` |
210
+ | keyword | `color: #5f00d7;` |
211
+ | operator | `color: #4e4e4e; font-weight: bold;` |
212
+ | property | `color: #af0000;` |
213
+ | string | `color: #008700;` |
214
+ | string.special | `color: #008787;` |
215
+ | tag | `color: #000087;` |
216
+ | type | `color: #005f5f;` |
217
+ | type.builtin | `color: #005f5f; font-weight: bold;` |
218
+ | variable.builtin | `font-weight: bold;` |
219
+ | variable.parameter | `text-decoration: underline;` |
220
+ | constant.builtin, number | `color: #875f00; font-weight: bold;` |
221
+ | constructor, module | `color: #af8700;` |
222
+ | punctuation.bracket, punctuation.delimiter | `color: #4e4e4e;` |
223
+
224
+ Any and all token types not represented in this default theme are consequently not
225
+ highlighted when using the inline CSS styles option.
226
+
227
+ The CSS class names are derived directly from Tree-sitter token names by replacing all
228
+ full stops ('.') with dashes ('-') and adding the prefix 'ts-'. For example, the CSS
229
+ class for 'function.builtin' tokens is 'ts-function-builtin'. The use of CSS classes
230
+ allows for customization of highlighting styles, including the ability to highlight more
231
+ token types than with the default inline CSS method. Of course, this also requires that
232
+ an externally created CSS stylesheet defining the style for each token type is provided
233
+ whenever the Kramdown-generated HTML is rendered.
178
234
 
179
235
  ### Configuration
180
236
 
181
237
  This Kramdown plugin currently supports the following options when provided as sub-keys
182
238
  of the Kramdown option `syntax_highlighter_opts`:
183
239
 
184
- | Key | Description | Default value |
185
- | :-- | :-- | :-- |
186
- | `tree_sitter_parsers_dir` | The path to the Tree-sitter language parsers directory. | `~/tree_sitter_parsers` |
240
+ | Key | Description | Type | Default value |
241
+ | :-- | :-- | :-- | :-- |
242
+ | `tree_sitter_parsers_dir` | The path to the Tree-sitter language parsers directory. | String | `~/tree_sitter_parsers` |
243
+ | `css_classes` | Whether to use CSS classes for highlights. | Boolean | `false` |
187
244
 
188
245
  ## Contributing
189
246
 
@@ -31,9 +31,9 @@ dependencies = [
31
31
 
32
32
  [[package]]
33
33
  name = "anyhow"
34
- version = "1.0.66"
34
+ version = "1.0.68"
35
35
  source = "registry+https://github.com/rust-lang/crates.io-index"
36
- checksum = "216261ddc8289130e551ddcd5ce8a064710c0d064a4d2895c67151c92b5443f6"
36
+ checksum = "2cb2f989d18dd141ab8ae82f64d1a8cdd37e0840f73a406896cf5e99502fab61"
37
37
 
38
38
  [[package]]
39
39
  name = "ascii"
@@ -1,21 +1,28 @@
1
1
  #[macro_use]
2
2
  extern crate rutie;
3
3
 
4
- use rutie::{AnyException, Class, Exception, Object, RString, VM};
4
+ use rutie::{AnyException, Boolean, Class, Exception, Object, RString, VM};
5
5
 
6
6
  mod tree_sitter_adapter;
7
7
 
8
8
  class!(TreeSitterAdapter);
9
9
 
10
+ #[rustfmt::skip]
10
11
  methods!(
11
12
  TreeSitterAdapter,
12
13
  _rtself,
13
- fn pub_highlight(raw_code: RString, raw_parsers_dir: RString, raw_scope: RString) -> RString {
14
+ fn pub_highlight(
15
+ raw_code: RString,
16
+ raw_parsers_dir: RString,
17
+ raw_scope: RString,
18
+ css_classes: Boolean
19
+ ) -> RString {
14
20
  VM::unwrap_or_raise_ex(
15
21
  tree_sitter_adapter::highlight(
16
22
  &raw_code.unwrap().to_string(),
17
23
  &raw_parsers_dir.unwrap().to_string(),
18
24
  &raw_scope.unwrap().to_string(),
25
+ css_classes.unwrap().to_bool(),
19
26
  )
20
27
  .as_ref()
21
28
  .map(String::as_str)
@@ -1,7 +1,9 @@
1
- use anyhow::{Context, Error, Result};
1
+ use anyhow::{Context, Result};
2
+ use std::collections::HashMap;
2
3
  use std::convert;
3
4
  use std::path::PathBuf;
4
5
  use tree_sitter::Language;
6
+ use tree_sitter_cli::highlight::Style;
5
7
  use tree_sitter_cli::highlight::Theme;
6
8
  use tree_sitter_highlight::{Error as TSError, HighlightEvent};
7
9
  use tree_sitter_highlight::{Highlight, HighlightConfiguration, Highlighter, HtmlRenderer};
@@ -9,126 +11,93 @@ use tree_sitter_loader::{Config, LanguageConfiguration, Loader};
9
11
 
10
12
  const LOADER_ERROR_MSG: &str = "Error loading Tree-sitter parsers from directory";
11
13
  const NO_LANGUAGE_ERROR_MSG: &str = "Error retrieving language configuration for scope";
14
+ const NO_HIGHLIGHT_ERROR_MSG: &str = "Error retrieving highlight configuration for scope";
12
15
 
13
- pub fn highlight(code: &str, parsers_dir: &str, scope: &str) -> Result<String, String> {
14
- let parsers_dir = PathBuf::from(parsers_dir);
15
- let theme = Theme::default();
16
- let mut loader = Loader::new_from_dir(parsers_dir).map_err(Error::to_formatted_string)?;
17
- loader.configure_highlights(&theme.highlight_names);
18
- loader
19
- .language_configuration_from_scope(scope)
20
- .and_then(LanguageConfigurationAdapter::highlight_config)
21
- .and_then(|config| {
22
- let default_css_style = String::new();
23
- let css_attribute_callback = get_css_styles(&theme, &default_css_style);
24
- HighlighterAdapter::new(&loader, config)
25
- .highlight(code)
26
- .and_then(|highlights| render_html(highlights, &css_attribute_callback))
27
- })
28
- .map_err(Error::to_formatted_string)
29
- }
30
-
31
- trait LoaderExt {
32
- fn new_from_dir(parser_directory: PathBuf) -> Result<Loader>;
33
-
34
- fn language_configuration_from_scope<'a>(
35
- &'a self,
36
- scope: &'a str,
37
- ) -> Result<LanguageConfigurationAdapter<'a>>;
16
+ trait ResultExt<T, E> {
17
+ fn flatten_(self) -> Result<T, E>;
38
18
  }
39
19
 
40
- impl LoaderExt for Loader {
41
- fn new_from_dir(parser_directory: PathBuf) -> Result<Loader> {
42
- Loader::new()
43
- .and_then(|mut loader| {
44
- let config = {
45
- let parser_directory = parser_directory.clone();
46
- let parser_directories = vec![parser_directory];
47
- Config { parser_directories }
48
- };
49
- loader.find_all_languages(&config)?;
50
- Ok(loader)
51
- })
52
- .with_context(|| {
53
- let parser_directory_string = parser_directory.display();
54
- format!("{LOADER_ERROR_MSG} '{parser_directory_string}'")
55
- })
56
- }
57
-
58
- fn language_configuration_from_scope<'a>(
59
- &'a self,
60
- scope: &'a str,
61
- ) -> Result<LanguageConfigurationAdapter<'a>> {
62
- self.language_configuration_for_scope(scope)
63
- .transpose()
64
- .context("Language not found")
65
- .flatten_()
66
- .map(|(language, config)| LanguageConfigurationAdapter { language, config })
67
- .with_context(|| format!("{NO_LANGUAGE_ERROR_MSG} '{scope}'"))
20
+ impl<T, E> ResultExt<T, E> for Result<Result<T, E>, E> {
21
+ fn flatten_(self) -> Result<T, E> {
22
+ self.and_then(convert::identity)
68
23
  }
69
24
  }
70
25
 
71
- struct LanguageConfigurationAdapter<'a> {
72
- language: Language,
73
- config: &'a LanguageConfiguration<'a>,
26
+ fn loader(parser_directory: PathBuf) -> Result<Loader> {
27
+ Loader::new()
28
+ .and_then(|mut loader| {
29
+ let config = {
30
+ let parser_directory = parser_directory.clone();
31
+ let parser_directories = vec![parser_directory];
32
+ Config { parser_directories }
33
+ };
34
+ loader.find_all_languages(&config)?;
35
+ Ok(loader)
36
+ })
37
+ .with_context(|| {
38
+ let parser_directory_str = parser_directory.display();
39
+ format!("{LOADER_ERROR_MSG} '{parser_directory_str}'")
40
+ })
74
41
  }
75
42
 
76
- impl<'a> LanguageConfigurationAdapter<'a> {
77
- fn highlight_config(self) -> Result<&'a HighlightConfiguration> {
78
- self.config
79
- .highlight_config(self.language)
80
- .transpose()
81
- .context("Another issue")
82
- .flatten_()
83
- }
43
+ fn language_and_configuration<'a>(
44
+ loader: &'a Loader,
45
+ scope: &'a str,
46
+ ) -> Result<(Language, &'a LanguageConfiguration<'a>)> {
47
+ loader
48
+ .language_configuration_for_scope(scope)
49
+ .transpose()
50
+ .context("Language not found")
51
+ .flatten_()
52
+ .with_context(|| format!("{NO_LANGUAGE_ERROR_MSG} '{scope}'"))
84
53
  }
85
54
 
86
- struct HighlightsAdapter<'a, T: Iterator<Item = Result<HighlightEvent, TSError>>> {
87
- code: &'a str,
88
- highlights: T,
55
+ fn highlight_configuration<'a>(
56
+ language: Language,
57
+ config: &'a LanguageConfiguration<'a>,
58
+ scope: &'a str,
59
+ ) -> Result<&'a HighlightConfiguration> {
60
+ config
61
+ .highlight_config(language)
62
+ .transpose()
63
+ .with_context(|| format!("{NO_HIGHLIGHT_ERROR_MSG} '{scope}'"))
64
+ .flatten_()
89
65
  }
90
66
 
91
- struct HighlighterAdapter<'a> {
92
- loader: &'a Loader,
93
- config: &'a HighlightConfiguration,
94
- highlighter: Highlighter,
67
+ fn highlights(
68
+ code: &str,
69
+ config: &HighlightConfiguration,
70
+ loader: &Loader,
71
+ ) -> Result<impl Iterator<Item = Result<HighlightEvent, TSError>>> {
72
+ Highlighter::new()
73
+ .highlight(config, code.as_bytes(), None, |s| {
74
+ loader.highlight_config_for_injection_string(s)
75
+ })
76
+ .map(Iterator::collect)
77
+ .map(Vec::into_iter)
78
+ .map_err(Into::into)
95
79
  }
96
80
 
97
- impl<'a> HighlighterAdapter<'a> {
98
- fn new(loader: &'a Loader, config: &'a HighlightConfiguration) -> Self {
99
- Self {
100
- loader,
101
- config,
102
- highlighter: Highlighter::new(),
103
- }
104
- }
105
-
106
- fn highlight(
107
- &'a mut self,
108
- code: &'a str,
109
- ) -> Result<HighlightsAdapter<'a, impl Iterator<Item = Result<HighlightEvent, TSError>> + 'a>>
110
- {
111
- let Self {
112
- loader,
113
- config,
114
- highlighter,
115
- } = self;
116
- highlighter
117
- .highlight(config, code.as_bytes(), None, |s| {
118
- loader.highlight_config_for_injection_string(s)
119
- })
120
- .map(|highlights| HighlightsAdapter { code, highlights })
121
- .map_err(Into::into)
81
+ fn create_html_attribute_callback<'a>(
82
+ html_attributes: &'a [String],
83
+ ) -> impl Fn(Highlight) -> &'a [u8] {
84
+ |highlight| {
85
+ html_attributes
86
+ .get(highlight.0)
87
+ .map(String::as_str)
88
+ .unwrap_or_default()
89
+ .as_bytes()
122
90
  }
123
91
  }
124
92
 
125
- fn render_html<'a, F: Fn(Highlight) -> &'a [u8]>(
126
- highlights: HighlightsAdapter<'a, impl Iterator<Item = Result<HighlightEvent, TSError>> + 'a>,
127
- css_attribute_callback: &F,
93
+ fn render_html(
94
+ code: &str,
95
+ highlights: impl Iterator<Item = Result<HighlightEvent, TSError>>,
96
+ html_attributes: &[String],
128
97
  ) -> Result<String> {
129
- let HighlightsAdapter { code, highlights } = highlights;
98
+ let html_attribute_callback = create_html_attribute_callback(html_attributes);
130
99
  let mut renderer = HtmlRenderer::new();
131
- renderer.render(highlights, code.as_bytes(), css_attribute_callback)?;
100
+ renderer.render(highlights, code.as_bytes(), &html_attribute_callback)?;
132
101
  // Remove erroneously appended newline
133
102
  if renderer.html.ends_with(&[b'\n']) && !code.ends_with('\n') {
134
103
  renderer.html.pop();
@@ -136,36 +105,65 @@ fn render_html<'a, F: Fn(Highlight) -> &'a [u8]>(
136
105
  Ok(renderer.lines().collect())
137
106
  }
138
107
 
139
- fn get_css_styles<'a>(
140
- theme: &'a Theme,
141
- default_css_style: &'a String,
142
- ) -> impl Fn(Highlight) -> &'a [u8] {
143
- |highlight| {
144
- theme
145
- .styles
146
- .get(highlight.0)
147
- .and_then(|style| style.css.as_ref())
148
- .unwrap_or(default_css_style)
149
- .as_bytes()
150
- }
108
+ fn highlight_names(scope: &str, loader: &Loader) -> Result<Vec<String>> {
109
+ let (language, config) = language_and_configuration(loader, scope)?;
110
+ let highlight_config = highlight_configuration(language, config, scope)?;
111
+ Ok(highlight_config.names().iter().map(String::from).collect())
151
112
  }
152
113
 
153
- trait ResultExt<T, E> {
154
- fn flatten_(self) -> Result<T, E>;
114
+ fn highlight_name_styles() -> HashMap<String, Style> {
115
+ let theme = Theme::default();
116
+ theme
117
+ .highlight_names
118
+ .into_iter()
119
+ .zip(theme.styles.into_iter())
120
+ .collect()
155
121
  }
156
122
 
157
- impl<T, E> ResultExt<T, E> for Result<Result<T, E>, E> {
158
- fn flatten_(self) -> Result<T, E> {
159
- self.and_then(convert::identity)
160
- }
123
+ fn inline_css_attributes(highlight_names: &[String]) -> Vec<String> {
124
+ let highlight_name_styles = highlight_name_styles();
125
+ highlight_names
126
+ .iter()
127
+ .map(|n| highlight_name_styles.get(n))
128
+ .map(|o| o.and_then(|style| style.css.as_ref()))
129
+ .map(|o| o.map(String::from))
130
+ .map(Option::unwrap_or_default)
131
+ .collect()
161
132
  }
162
133
 
163
- trait ErrorExt {
164
- fn to_formatted_string(self) -> String;
134
+ fn css_class_attributes(highlight_names: &[String]) -> Vec<String> {
135
+ highlight_names
136
+ .iter()
137
+ .map(|s| s.replace('.', "-"))
138
+ .map(|s| format!("class='ts-{s}'"))
139
+ .collect()
165
140
  }
166
141
 
167
- impl ErrorExt for Error {
168
- fn to_formatted_string(self) -> String {
169
- format!("{self:#}")
170
- }
142
+ fn highlight_adapter(
143
+ code: &str,
144
+ parsers_dir: &str,
145
+ scope: &str,
146
+ css_classes: bool,
147
+ ) -> Result<String> {
148
+ let parsers_dir = PathBuf::from(parsers_dir);
149
+ let mut loader = loader(parsers_dir)?;
150
+ let highlight_names = highlight_names(scope, &loader)?;
151
+ loader.configure_highlights(&highlight_names);
152
+ let (language, config) = language_and_configuration(&loader, scope)?;
153
+ let highlight_config = highlight_configuration(language, config, scope)?;
154
+ let highlights = highlights(code, highlight_config, &loader)?;
155
+ let html_attributes = match css_classes {
156
+ true => css_class_attributes,
157
+ false => inline_css_attributes,
158
+ }(&highlight_names);
159
+ render_html(code, highlights, &html_attributes)
160
+ }
161
+
162
+ pub fn highlight(
163
+ code: &str,
164
+ parsers_dir: &str,
165
+ scope: &str,
166
+ css_classes: bool,
167
+ ) -> Result<String, String> {
168
+ highlight_adapter(code, parsers_dir, scope, css_classes).map_err(|e| format!("{e:#}"))
171
169
  }
@@ -3,6 +3,8 @@
3
3
  require 'kramdown'
4
4
  require 'tree_sitter_adapter'
5
5
 
6
+ require_relative '../../syntax_tree_sitter/languages'
7
+
6
8
  module Kramdown
7
9
  module Converter # rubocop:disable Style/Documentation
8
10
  module SyntaxHighlighter
@@ -13,17 +15,30 @@ module Kramdown
13
15
  module TreeSitter
14
16
  DEFAULT_PARSERS_DIR = '~/tree_sitter_parsers'
15
17
 
16
- def self.call(converter, raw_text, language, type, _)
17
- return nil unless language
18
+ def self.call(converter, raw_text, language_alias, type, _)
19
+ return nil unless language_alias
18
20
 
19
- parsers_dir = get_option(converter, :tree_sitter_parsers_dir)
20
- .then { _1 || DEFAULT_PARSERS_DIR }
21
- .then { File.expand_path _1 }
22
- rendered_text = TreeSitterAdapter.highlight raw_text, parsers_dir, language
21
+ language_scope = LANGUAGE_SCOPES.fetch(language_alias, language_alias)
22
+ rendered_text = TreeSitterAdapter.highlight(
23
+ raw_text,
24
+ get_parsers_dir(converter),
25
+ language_scope,
26
+ get_use_css_classes(converter)
27
+ )
23
28
  # Code blocks are additionally wrapped in HTML code tags
24
29
  type == :block ? "<pre><code>#{rendered_text}</code></pre>" : rendered_text
25
30
  end
26
31
 
32
+ def self.get_parsers_dir(converter)
33
+ File.expand_path(
34
+ get_option(converter, :tree_sitter_parsers_dir) || DEFAULT_PARSERS_DIR
35
+ )
36
+ end
37
+
38
+ def self.get_use_css_classes(converter)
39
+ get_option(converter, :css_classes) || false
40
+ end
41
+
27
42
  def self.get_option(converter, name)
28
43
  converter.options[:syntax_highlighter_opts][name]
29
44
  end
@@ -0,0 +1,58 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Kramdown
4
+ module Converter
5
+ module SyntaxHighlighter
6
+ module TreeSitter
7
+ # Maps each recognized language alias to its corresponding Tree-sitter scope
8
+ LANGUAGE_SCOPES =
9
+ {
10
+ 'source.R' => %w[R S r s],
11
+ 'source.bash' => %w[bash ksh sh shell zsh],
12
+ 'source.c' => %w[c],
13
+ 'source.cpp' => %w[c++ cpp],
14
+ 'source.cs' => %w[c# cs csharp],
15
+ 'source.css' => %w[css],
16
+ 'source.d' => %w[d dlang],
17
+ 'source.dot' => %w[dot],
18
+ 'source.elixir' => %w[elixir exs],
19
+ 'source.elm' => %w[elm],
20
+ 'source.emacs.lisp' => %w[elisp emacs-lisp],
21
+ 'source.go' => %w[go golang],
22
+ 'source.hack' => %w[hack hh],
23
+ 'source.haskell' => %w[haskell hs],
24
+ 'source.hcl' => %w[hcl],
25
+ 'source.java' => %w[java],
26
+ 'source.js' => %w[javascript js],
27
+ 'source.json' => %w[json],
28
+ 'source.julia' => %w[jl julia],
29
+ 'source.lua' => %w[lua],
30
+ 'source.mk' => %w[bsdmake gnumake make makefile mf],
31
+ 'source.nix' => %w[nix nixos],
32
+ 'source.objc' => %w[obj-c obj_c objc objective_c objectivec],
33
+ 'source.ocaml' => %w[ocaml],
34
+ 'source.perl' => %w[perl pl],
35
+ 'source.php' => %w[php php3 php4 php5],
36
+ 'source.proto' => %w[proto protobuf],
37
+ 'source.python' => %w[py python],
38
+ 'source.racket' => %w[racket],
39
+ 'source.ruby' => %w[rb ruby],
40
+ 'source.rust' => %w[rs rust],
41
+ 'source.scala' => %w[scala],
42
+ 'source.sparql' => %w[sparql],
43
+ 'source.sql' => %w[sql],
44
+ 'source.swift' => %w[swift],
45
+ 'source.toml' => %w[toml],
46
+ 'source.ts' => %w[ts typescript],
47
+ 'source.vhd' => %w[vhdl],
48
+ 'text.html.basic' => %w[html],
49
+ 'text.html.erb' => %w[erb eruby rhtml]
50
+ }
51
+ .map { |scope, aliases| aliases.map { |alias_| [alias_, scope] } }
52
+ .flatten(1)
53
+ .to_h
54
+ .freeze
55
+ end
56
+ end
57
+ end
58
+ end
@@ -5,7 +5,7 @@ module Kramdown
5
5
  module SyntaxHighlighter
6
6
  module TreeSitter
7
7
  # Version of kramdown-syntax_tree_sitter gem
8
- VERSION = '0.1.0'
8
+ VERSION = '0.3.0'
9
9
  end
10
10
  end
11
11
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kramdown-syntax_tree_sitter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew T. Biehl
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-11-11 00:00:00.000000000 Z
11
+ date: 2023-01-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: kramdown
@@ -139,6 +139,7 @@ files:
139
139
  - ext/tree_sitter_adapter/src/tree_sitter_adapter.rs
140
140
  - lib/kramdown/converter/syntax_highlighter/tree_sitter.rb
141
141
  - lib/kramdown/syntax_tree_sitter.rb
142
+ - lib/kramdown/syntax_tree_sitter/languages.rb
142
143
  - lib/kramdown/syntax_tree_sitter/version.rb
143
144
  - lib/tree_sitter_adapter.rb
144
145
  homepage: https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter
@@ -161,7 +162,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
161
162
  - !ruby/object:Gem::Version
162
163
  version: '0'
163
164
  requirements: []
164
- rubygems_version: 3.3.7
165
+ rubygems_version: 3.4.1
165
166
  signing_key:
166
167
  specification_version: 4
167
168
  summary: Syntax highlight code with Tree-sitter via Kramdown.