kramdown-syntax_tree_sitter 0.1.0 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 02e58047b1cac1778ad13d25f88a64a42f0714faeba928322851a78afeb4ce66
4
- data.tar.gz: 87cc78c75fb83a9a56cca8bb663f72b1d3abbd03f3920e37bad6ac629c699d69
3
+ metadata.gz: '09858b648cc381cb603b21a3ed091b1d96a25b55264f4dd2c4e8fe7436d68288'
4
+ data.tar.gz: 9ceeb8b78ac8cea098cbb329abbdc9e650202e676b042b647b3f0ca55665c3ed
5
5
  SHA512:
6
- metadata.gz: 3b609ade16479c2e09504248970b1f95eb496b982cc641e766881dbfc74bedd580a45bc5f0c24f9b859535a3967a74e51126e6dfdf8df7448a5284da259e3105
7
- data.tar.gz: ae2cdab47a180b7435fc9c01bff81aefb377a353af341dc98f3f5bd8685105f6b976d35c25a900fcc7caa8744e3c34ef3fdbb63fb09dc27d52ee63f67644466e
6
+ metadata.gz: a3a7cf735cab71d863433b3be34e8b1dd537cca412bfddb611b56d4ff8292b74f0f45283da701510d928d432ad4aa39224316a012eb07cbd524cc1977bfb3a2e
7
+ data.tar.gz: 23197c154c12ca2e97534829179dd2785c9907760f89205a7a7be3e80f3cca7f8f6dd2c6314897a887e1c45b55eaddbd05a2937321017378caa0d03e33e450a5
data/README.md CHANGED
@@ -1,5 +1,8 @@
1
1
  # Kramdown Tree-sitter Highlighter
2
2
 
3
+ [![Build status](https://img.shields.io/github/actions/workflow/status/andrewtbiehl/kramdown-syntax_tree_sitter/quality-control.yml?branch=main&style=flat-square&logo=github)](https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter/actions/workflows/quality-control.yml)
4
+ [![RubyGems page](https://img.shields.io/gem/v/kramdown-syntax_tree_sitter?style=flat-square&color=blue&logo=rubygems&logoColor=white)](https://rubygems.org/gems/kramdown-syntax_tree_sitter)
5
+
3
6
  ***Syntax highlight code with [Tree-sitter](https://tree-sitter.github.io/tree-sitter)
4
7
  via [Kramdown](https://kramdown.gettalong.org).***
5
8
 
@@ -19,8 +22,8 @@ essentially an adapter for the
19
22
  hence also requires a compatible [Rust](https://www.rust-lang.org) installation to
20
23
  function. It is officially compatible with the following environments:
21
24
 
22
- - **Ruby**: 2.7, 3.0, 3.1
23
- - **Rust**: 1.61, 1.62, 1.63, 1.64, 1.65
25
+ - **Ruby**: 2.7, 3.0, 3.1, 3.2
26
+ - **Rust**: 1.62, 1.63, 1.64, 1.65, 1.66
24
27
  - **Platforms**: MacOS, Linux
25
28
 
26
29
  ### Installation
@@ -29,14 +32,13 @@ For projects using [Bundler](https://bundler.io) for dependency management, run
29
32
  following command to both install the gem and add it to the Gemfile:
30
33
 
31
34
  ```shell
32
- bundle add kramdown-syntax_tree_sitter --github andrewtbiehl/kramdown-syntax_tree_sitter
35
+ bundle add kramdown-syntax_tree_sitter
33
36
  ```
34
37
 
35
- Otherwise, download this project's repository and then run the following command from
36
- within it to build and install the gem:
38
+ Otherwise, install the gem via the following command:
37
39
 
38
40
  ```shell
39
- gem build && gem install kramdown-syntax_tree_sitter
41
+ gem install kramdown-syntax_tree_sitter
40
42
  ```
41
43
 
42
44
  ## Usage
@@ -54,7 +56,7 @@ require 'kramdown'
54
56
  require 'kramdown/syntax_tree_sitter'
55
57
 
56
58
  text = <<~MARKDOWN
57
- ~~~source.python
59
+ ~~~python
58
60
  print('Hello, World!')
59
61
  ~~~
60
62
  MARKDOWN
@@ -62,6 +64,13 @@ MARKDOWN
62
64
  Kramdown::Document.new(text, syntax_highlighter: :'tree-sitter').to_html
63
65
  ```
64
66
 
67
+ ### General usage
68
+
69
+ To successfully highlight input text via Kramdown with this plugin enabled, make sure
70
+ that every language referenced has a corresponding Tree-sitter parser library installed.
71
+ See the subsequent ['Language identifiers'](#language-identifiers) and
72
+ ['Tree-sitter parsers'](#tree-sitter-parsers) sections for more information.
73
+
65
74
  ### Usage with Jekyll
66
75
 
67
76
  This plugin can be used with the popular static site generator
@@ -80,11 +89,9 @@ kramdown:
80
89
  # Other Kramdown options...
81
90
  ```
82
91
 
83
- To highlight every code block in a Jekyll project via the plugin, make sure that every
84
- language identifier is expressed with the correct Tree-sitter scope and that every
85
- language referenced has a corresponding Tree-sitter parser library installed. See the
86
- subsequent ['Language identifiers'](#language-identifiers) and
87
- ['Tree-sitter parsers'](#tree-sitter-parsers) sections for more information.
92
+ Note that this plugin's general usage prerequisites (described in the previous
93
+ ['General usage'](#general-usage) section) still apply when using this plugin with
94
+ Jekyll.
88
95
 
89
96
  Also, there are multiple ways to render highlighted code blocks with Jekyll, as
90
97
  illustrated in the following table:
@@ -101,13 +108,13 @@ illustrated in the following table:
101
108
 
102
109
  ---
103
110
  ````
104
- ```source.python
111
+ ```python
105
112
  print('Hello, World!')
106
113
  ```
107
114
 
108
115
  or
109
116
 
110
- ~~~source.python
117
+ ~~~python
111
118
  print('Hello, World!')
112
119
  ~~~
113
120
  ````
@@ -123,7 +130,7 @@ print('Hello, World!')
123
130
  ---
124
131
  ```
125
132
  print('Hello, World!')
126
- {: class="language-source.python" }
133
+ {: class="language-python" }
127
134
  ```
128
135
  ---
129
136
 
@@ -136,7 +143,7 @@ print('Hello, World!')
136
143
 
137
144
  ---
138
145
  ```
139
- {% highlight source.python %}
146
+ {% highlight python %}
140
147
  print('Hello, World!')
141
148
  {% endhighlight %}
142
149
  ```
@@ -169,21 +176,71 @@ can be found on
169
176
 
170
177
  ### Language identifiers
171
178
 
172
- Tree-sitter uses a string-based identifier called a 'scope' to identify each language.
173
- For example, the scope string for Python is 'source.python', whereas for HTML it is
174
- 'text.html.basic'. Currently, this plugin follows this same convention, so a given code
175
- block will only be correctly highlighted if the language identifier provided for that
176
- code block is its language's corresponding Tree-sitter scope string. This is illustrated
177
- by the code block used in the [Quickstart](#quickstart) example.
179
+ For most popular languages, this plugin recognizes the same language identifiers as used
180
+ by the popular highlighter [Rouge](https://github.com/rouge-ruby/rouge). For example,
181
+ both 'python' and 'py' may be used to identify a code block written in Python. For the
182
+ complete list of Rouge language identifiers recognized by this plugin, see
183
+ [`lib/kramdown/syntax_tree_sitter/languages.rb`](https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter/blob/main/lib/kramdown/syntax_tree_sitter/languages.rb).
184
+
185
+ This plugin also automatically supports the language identifier format used by
186
+ Tree-sitter, known as as the 'scope' name. For example, the scope name for Python is
187
+ 'source.python', whereas for HTML it is 'text.html.basic'.
188
+
189
+ For any identifiers currently supported by Rouge but not by this plugin, feel free to
190
+ open an issue or pull request on GitHub to have them included.
191
+
192
+ ### CSS styling
193
+
194
+ Code highlights can be rendered either via inline CSS styles or via CSS classes, which
195
+ are applied to each token in the parsed code.
196
+
197
+ To use CSS classes for highlighting, set the `css_classes` Kramdown syntax-highlighting
198
+ option to `true`. Otherwise, the plugin will apply highlights via inline CSS styles.
199
+
200
+ The inline CSS styles are derived from Tree-sitter's built-in default highlighting
201
+ theme, repeated here for convenience:
202
+
203
+ | Token name | CSS style |
204
+ | :-- | :-- |
205
+ | attribute | `color: #af0000; font-style: italic;` |
206
+ | comment | `color: #8a8a8a; font-style: italic;` |
207
+ | constant | `color: #875f00;` |
208
+ | function | `color: o#005fd7;` |
209
+ | function.builtin | `color: #005fd7; font-weight: bold;` |
210
+ | keyword | `color: #5f00d7;` |
211
+ | operator | `color: #4e4e4e; font-weight: bold;` |
212
+ | property | `color: #af0000;` |
213
+ | string | `color: #008700;` |
214
+ | string.special | `color: #008787;` |
215
+ | tag | `color: #000087;` |
216
+ | type | `color: #005f5f;` |
217
+ | type.builtin | `color: #005f5f; font-weight: bold;` |
218
+ | variable.builtin | `font-weight: bold;` |
219
+ | variable.parameter | `text-decoration: underline;` |
220
+ | constant.builtin, number | `color: #875f00; font-weight: bold;` |
221
+ | constructor, module | `color: #af8700;` |
222
+ | punctuation.bracket, punctuation.delimiter | `color: #4e4e4e;` |
223
+
224
+ Any and all token types not represented in this default theme are consequently not
225
+ highlighted when using the inline CSS styles option.
226
+
227
+ The CSS class names are derived directly from Tree-sitter token names by replacing all
228
+ full stops ('.') with dashes ('-') and adding the prefix 'ts-'. For example, the CSS
229
+ class for 'function.builtin' tokens is 'ts-function-builtin'. The use of CSS classes
230
+ allows for customization of highlighting styles, including the ability to highlight more
231
+ token types than with the default inline CSS method. Of course, this also requires that
232
+ an externally created CSS stylesheet defining the style for each token type is provided
233
+ whenever the Kramdown-generated HTML is rendered.
178
234
 
179
235
  ### Configuration
180
236
 
181
237
  This Kramdown plugin currently supports the following options when provided as sub-keys
182
238
  of the Kramdown option `syntax_highlighter_opts`:
183
239
 
184
- | Key | Description | Default value |
185
- | :-- | :-- | :-- |
186
- | `tree_sitter_parsers_dir` | The path to the Tree-sitter language parsers directory. | `~/tree_sitter_parsers` |
240
+ | Key | Description | Type | Default value |
241
+ | :-- | :-- | :-- | :-- |
242
+ | `tree_sitter_parsers_dir` | The path to the Tree-sitter language parsers directory. | String | `~/tree_sitter_parsers` |
243
+ | `css_classes` | Whether to use CSS classes for highlights. | Boolean | `false` |
187
244
 
188
245
  ## Contributing
189
246
 
@@ -31,9 +31,9 @@ dependencies = [
31
31
 
32
32
  [[package]]
33
33
  name = "anyhow"
34
- version = "1.0.66"
34
+ version = "1.0.68"
35
35
  source = "registry+https://github.com/rust-lang/crates.io-index"
36
- checksum = "216261ddc8289130e551ddcd5ce8a064710c0d064a4d2895c67151c92b5443f6"
36
+ checksum = "2cb2f989d18dd141ab8ae82f64d1a8cdd37e0840f73a406896cf5e99502fab61"
37
37
 
38
38
  [[package]]
39
39
  name = "ascii"
@@ -1,21 +1,28 @@
1
1
  #[macro_use]
2
2
  extern crate rutie;
3
3
 
4
- use rutie::{AnyException, Class, Exception, Object, RString, VM};
4
+ use rutie::{AnyException, Boolean, Class, Exception, Object, RString, VM};
5
5
 
6
6
  mod tree_sitter_adapter;
7
7
 
8
8
  class!(TreeSitterAdapter);
9
9
 
10
+ #[rustfmt::skip]
10
11
  methods!(
11
12
  TreeSitterAdapter,
12
13
  _rtself,
13
- fn pub_highlight(raw_code: RString, raw_parsers_dir: RString, raw_scope: RString) -> RString {
14
+ fn pub_highlight(
15
+ raw_code: RString,
16
+ raw_parsers_dir: RString,
17
+ raw_scope: RString,
18
+ css_classes: Boolean
19
+ ) -> RString {
14
20
  VM::unwrap_or_raise_ex(
15
21
  tree_sitter_adapter::highlight(
16
22
  &raw_code.unwrap().to_string(),
17
23
  &raw_parsers_dir.unwrap().to_string(),
18
24
  &raw_scope.unwrap().to_string(),
25
+ css_classes.unwrap().to_bool(),
19
26
  )
20
27
  .as_ref()
21
28
  .map(String::as_str)
@@ -1,7 +1,9 @@
1
- use anyhow::{Context, Error, Result};
1
+ use anyhow::{Context, Result};
2
+ use std::collections::HashMap;
2
3
  use std::convert;
3
4
  use std::path::PathBuf;
4
5
  use tree_sitter::Language;
6
+ use tree_sitter_cli::highlight::Style;
5
7
  use tree_sitter_cli::highlight::Theme;
6
8
  use tree_sitter_highlight::{Error as TSError, HighlightEvent};
7
9
  use tree_sitter_highlight::{Highlight, HighlightConfiguration, Highlighter, HtmlRenderer};
@@ -9,126 +11,93 @@ use tree_sitter_loader::{Config, LanguageConfiguration, Loader};
9
11
 
10
12
  const LOADER_ERROR_MSG: &str = "Error loading Tree-sitter parsers from directory";
11
13
  const NO_LANGUAGE_ERROR_MSG: &str = "Error retrieving language configuration for scope";
14
+ const NO_HIGHLIGHT_ERROR_MSG: &str = "Error retrieving highlight configuration for scope";
12
15
 
13
- pub fn highlight(code: &str, parsers_dir: &str, scope: &str) -> Result<String, String> {
14
- let parsers_dir = PathBuf::from(parsers_dir);
15
- let theme = Theme::default();
16
- let mut loader = Loader::new_from_dir(parsers_dir).map_err(Error::to_formatted_string)?;
17
- loader.configure_highlights(&theme.highlight_names);
18
- loader
19
- .language_configuration_from_scope(scope)
20
- .and_then(LanguageConfigurationAdapter::highlight_config)
21
- .and_then(|config| {
22
- let default_css_style = String::new();
23
- let css_attribute_callback = get_css_styles(&theme, &default_css_style);
24
- HighlighterAdapter::new(&loader, config)
25
- .highlight(code)
26
- .and_then(|highlights| render_html(highlights, &css_attribute_callback))
27
- })
28
- .map_err(Error::to_formatted_string)
29
- }
30
-
31
- trait LoaderExt {
32
- fn new_from_dir(parser_directory: PathBuf) -> Result<Loader>;
33
-
34
- fn language_configuration_from_scope<'a>(
35
- &'a self,
36
- scope: &'a str,
37
- ) -> Result<LanguageConfigurationAdapter<'a>>;
16
+ trait ResultExt<T, E> {
17
+ fn flatten_(self) -> Result<T, E>;
38
18
  }
39
19
 
40
- impl LoaderExt for Loader {
41
- fn new_from_dir(parser_directory: PathBuf) -> Result<Loader> {
42
- Loader::new()
43
- .and_then(|mut loader| {
44
- let config = {
45
- let parser_directory = parser_directory.clone();
46
- let parser_directories = vec![parser_directory];
47
- Config { parser_directories }
48
- };
49
- loader.find_all_languages(&config)?;
50
- Ok(loader)
51
- })
52
- .with_context(|| {
53
- let parser_directory_string = parser_directory.display();
54
- format!("{LOADER_ERROR_MSG} '{parser_directory_string}'")
55
- })
56
- }
57
-
58
- fn language_configuration_from_scope<'a>(
59
- &'a self,
60
- scope: &'a str,
61
- ) -> Result<LanguageConfigurationAdapter<'a>> {
62
- self.language_configuration_for_scope(scope)
63
- .transpose()
64
- .context("Language not found")
65
- .flatten_()
66
- .map(|(language, config)| LanguageConfigurationAdapter { language, config })
67
- .with_context(|| format!("{NO_LANGUAGE_ERROR_MSG} '{scope}'"))
20
+ impl<T, E> ResultExt<T, E> for Result<Result<T, E>, E> {
21
+ fn flatten_(self) -> Result<T, E> {
22
+ self.and_then(convert::identity)
68
23
  }
69
24
  }
70
25
 
71
- struct LanguageConfigurationAdapter<'a> {
72
- language: Language,
73
- config: &'a LanguageConfiguration<'a>,
26
+ fn loader(parser_directory: PathBuf) -> Result<Loader> {
27
+ Loader::new()
28
+ .and_then(|mut loader| {
29
+ let config = {
30
+ let parser_directory = parser_directory.clone();
31
+ let parser_directories = vec![parser_directory];
32
+ Config { parser_directories }
33
+ };
34
+ loader.find_all_languages(&config)?;
35
+ Ok(loader)
36
+ })
37
+ .with_context(|| {
38
+ let parser_directory_str = parser_directory.display();
39
+ format!("{LOADER_ERROR_MSG} '{parser_directory_str}'")
40
+ })
74
41
  }
75
42
 
76
- impl<'a> LanguageConfigurationAdapter<'a> {
77
- fn highlight_config(self) -> Result<&'a HighlightConfiguration> {
78
- self.config
79
- .highlight_config(self.language)
80
- .transpose()
81
- .context("Another issue")
82
- .flatten_()
83
- }
43
+ fn language_and_configuration<'a>(
44
+ loader: &'a Loader,
45
+ scope: &'a str,
46
+ ) -> Result<(Language, &'a LanguageConfiguration<'a>)> {
47
+ loader
48
+ .language_configuration_for_scope(scope)
49
+ .transpose()
50
+ .context("Language not found")
51
+ .flatten_()
52
+ .with_context(|| format!("{NO_LANGUAGE_ERROR_MSG} '{scope}'"))
84
53
  }
85
54
 
86
- struct HighlightsAdapter<'a, T: Iterator<Item = Result<HighlightEvent, TSError>>> {
87
- code: &'a str,
88
- highlights: T,
55
+ fn highlight_configuration<'a>(
56
+ language: Language,
57
+ config: &'a LanguageConfiguration<'a>,
58
+ scope: &'a str,
59
+ ) -> Result<&'a HighlightConfiguration> {
60
+ config
61
+ .highlight_config(language)
62
+ .transpose()
63
+ .with_context(|| format!("{NO_HIGHLIGHT_ERROR_MSG} '{scope}'"))
64
+ .flatten_()
89
65
  }
90
66
 
91
- struct HighlighterAdapter<'a> {
92
- loader: &'a Loader,
93
- config: &'a HighlightConfiguration,
94
- highlighter: Highlighter,
67
+ fn highlights(
68
+ code: &str,
69
+ config: &HighlightConfiguration,
70
+ loader: &Loader,
71
+ ) -> Result<impl Iterator<Item = Result<HighlightEvent, TSError>>> {
72
+ Highlighter::new()
73
+ .highlight(config, code.as_bytes(), None, |s| {
74
+ loader.highlight_config_for_injection_string(s)
75
+ })
76
+ .map(Iterator::collect)
77
+ .map(Vec::into_iter)
78
+ .map_err(Into::into)
95
79
  }
96
80
 
97
- impl<'a> HighlighterAdapter<'a> {
98
- fn new(loader: &'a Loader, config: &'a HighlightConfiguration) -> Self {
99
- Self {
100
- loader,
101
- config,
102
- highlighter: Highlighter::new(),
103
- }
104
- }
105
-
106
- fn highlight(
107
- &'a mut self,
108
- code: &'a str,
109
- ) -> Result<HighlightsAdapter<'a, impl Iterator<Item = Result<HighlightEvent, TSError>> + 'a>>
110
- {
111
- let Self {
112
- loader,
113
- config,
114
- highlighter,
115
- } = self;
116
- highlighter
117
- .highlight(config, code.as_bytes(), None, |s| {
118
- loader.highlight_config_for_injection_string(s)
119
- })
120
- .map(|highlights| HighlightsAdapter { code, highlights })
121
- .map_err(Into::into)
81
+ fn create_html_attribute_callback<'a>(
82
+ html_attributes: &'a [String],
83
+ ) -> impl Fn(Highlight) -> &'a [u8] {
84
+ |highlight| {
85
+ html_attributes
86
+ .get(highlight.0)
87
+ .map(String::as_str)
88
+ .unwrap_or_default()
89
+ .as_bytes()
122
90
  }
123
91
  }
124
92
 
125
- fn render_html<'a, F: Fn(Highlight) -> &'a [u8]>(
126
- highlights: HighlightsAdapter<'a, impl Iterator<Item = Result<HighlightEvent, TSError>> + 'a>,
127
- css_attribute_callback: &F,
93
+ fn render_html(
94
+ code: &str,
95
+ highlights: impl Iterator<Item = Result<HighlightEvent, TSError>>,
96
+ html_attributes: &[String],
128
97
  ) -> Result<String> {
129
- let HighlightsAdapter { code, highlights } = highlights;
98
+ let html_attribute_callback = create_html_attribute_callback(html_attributes);
130
99
  let mut renderer = HtmlRenderer::new();
131
- renderer.render(highlights, code.as_bytes(), css_attribute_callback)?;
100
+ renderer.render(highlights, code.as_bytes(), &html_attribute_callback)?;
132
101
  // Remove erroneously appended newline
133
102
  if renderer.html.ends_with(&[b'\n']) && !code.ends_with('\n') {
134
103
  renderer.html.pop();
@@ -136,36 +105,65 @@ fn render_html<'a, F: Fn(Highlight) -> &'a [u8]>(
136
105
  Ok(renderer.lines().collect())
137
106
  }
138
107
 
139
- fn get_css_styles<'a>(
140
- theme: &'a Theme,
141
- default_css_style: &'a String,
142
- ) -> impl Fn(Highlight) -> &'a [u8] {
143
- |highlight| {
144
- theme
145
- .styles
146
- .get(highlight.0)
147
- .and_then(|style| style.css.as_ref())
148
- .unwrap_or(default_css_style)
149
- .as_bytes()
150
- }
108
+ fn highlight_names(scope: &str, loader: &Loader) -> Result<Vec<String>> {
109
+ let (language, config) = language_and_configuration(loader, scope)?;
110
+ let highlight_config = highlight_configuration(language, config, scope)?;
111
+ Ok(highlight_config.names().iter().map(String::from).collect())
151
112
  }
152
113
 
153
- trait ResultExt<T, E> {
154
- fn flatten_(self) -> Result<T, E>;
114
+ fn highlight_name_styles() -> HashMap<String, Style> {
115
+ let theme = Theme::default();
116
+ theme
117
+ .highlight_names
118
+ .into_iter()
119
+ .zip(theme.styles.into_iter())
120
+ .collect()
155
121
  }
156
122
 
157
- impl<T, E> ResultExt<T, E> for Result<Result<T, E>, E> {
158
- fn flatten_(self) -> Result<T, E> {
159
- self.and_then(convert::identity)
160
- }
123
+ fn inline_css_attributes(highlight_names: &[String]) -> Vec<String> {
124
+ let highlight_name_styles = highlight_name_styles();
125
+ highlight_names
126
+ .iter()
127
+ .map(|n| highlight_name_styles.get(n))
128
+ .map(|o| o.and_then(|style| style.css.as_ref()))
129
+ .map(|o| o.map(String::from))
130
+ .map(Option::unwrap_or_default)
131
+ .collect()
161
132
  }
162
133
 
163
- trait ErrorExt {
164
- fn to_formatted_string(self) -> String;
134
+ fn css_class_attributes(highlight_names: &[String]) -> Vec<String> {
135
+ highlight_names
136
+ .iter()
137
+ .map(|s| s.replace('.', "-"))
138
+ .map(|s| format!("class='ts-{s}'"))
139
+ .collect()
165
140
  }
166
141
 
167
- impl ErrorExt for Error {
168
- fn to_formatted_string(self) -> String {
169
- format!("{self:#}")
170
- }
142
+ fn highlight_adapter(
143
+ code: &str,
144
+ parsers_dir: &str,
145
+ scope: &str,
146
+ css_classes: bool,
147
+ ) -> Result<String> {
148
+ let parsers_dir = PathBuf::from(parsers_dir);
149
+ let mut loader = loader(parsers_dir)?;
150
+ let highlight_names = highlight_names(scope, &loader)?;
151
+ loader.configure_highlights(&highlight_names);
152
+ let (language, config) = language_and_configuration(&loader, scope)?;
153
+ let highlight_config = highlight_configuration(language, config, scope)?;
154
+ let highlights = highlights(code, highlight_config, &loader)?;
155
+ let html_attributes = match css_classes {
156
+ true => css_class_attributes,
157
+ false => inline_css_attributes,
158
+ }(&highlight_names);
159
+ render_html(code, highlights, &html_attributes)
160
+ }
161
+
162
+ pub fn highlight(
163
+ code: &str,
164
+ parsers_dir: &str,
165
+ scope: &str,
166
+ css_classes: bool,
167
+ ) -> Result<String, String> {
168
+ highlight_adapter(code, parsers_dir, scope, css_classes).map_err(|e| format!("{e:#}"))
171
169
  }
@@ -3,6 +3,8 @@
3
3
  require 'kramdown'
4
4
  require 'tree_sitter_adapter'
5
5
 
6
+ require_relative '../../syntax_tree_sitter/languages'
7
+
6
8
  module Kramdown
7
9
  module Converter # rubocop:disable Style/Documentation
8
10
  module SyntaxHighlighter
@@ -13,17 +15,30 @@ module Kramdown
13
15
  module TreeSitter
14
16
  DEFAULT_PARSERS_DIR = '~/tree_sitter_parsers'
15
17
 
16
- def self.call(converter, raw_text, language, type, _)
17
- return nil unless language
18
+ def self.call(converter, raw_text, language_alias, type, _)
19
+ return nil unless language_alias
18
20
 
19
- parsers_dir = get_option(converter, :tree_sitter_parsers_dir)
20
- .then { _1 || DEFAULT_PARSERS_DIR }
21
- .then { File.expand_path _1 }
22
- rendered_text = TreeSitterAdapter.highlight raw_text, parsers_dir, language
21
+ language_scope = LANGUAGE_SCOPES.fetch(language_alias, language_alias)
22
+ rendered_text = TreeSitterAdapter.highlight(
23
+ raw_text,
24
+ get_parsers_dir(converter),
25
+ language_scope,
26
+ get_use_css_classes(converter)
27
+ )
23
28
  # Code blocks are additionally wrapped in HTML code tags
24
29
  type == :block ? "<pre><code>#{rendered_text}</code></pre>" : rendered_text
25
30
  end
26
31
 
32
+ def self.get_parsers_dir(converter)
33
+ File.expand_path(
34
+ get_option(converter, :tree_sitter_parsers_dir) || DEFAULT_PARSERS_DIR
35
+ )
36
+ end
37
+
38
+ def self.get_use_css_classes(converter)
39
+ get_option(converter, :css_classes) || false
40
+ end
41
+
27
42
  def self.get_option(converter, name)
28
43
  converter.options[:syntax_highlighter_opts][name]
29
44
  end
@@ -0,0 +1,58 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Kramdown
4
+ module Converter
5
+ module SyntaxHighlighter
6
+ module TreeSitter
7
+ # Maps each recognized language alias to its corresponding Tree-sitter scope
8
+ LANGUAGE_SCOPES =
9
+ {
10
+ 'source.R' => %w[R S r s],
11
+ 'source.bash' => %w[bash ksh sh shell zsh],
12
+ 'source.c' => %w[c],
13
+ 'source.cpp' => %w[c++ cpp],
14
+ 'source.cs' => %w[c# cs csharp],
15
+ 'source.css' => %w[css],
16
+ 'source.d' => %w[d dlang],
17
+ 'source.dot' => %w[dot],
18
+ 'source.elixir' => %w[elixir exs],
19
+ 'source.elm' => %w[elm],
20
+ 'source.emacs.lisp' => %w[elisp emacs-lisp],
21
+ 'source.go' => %w[go golang],
22
+ 'source.hack' => %w[hack hh],
23
+ 'source.haskell' => %w[haskell hs],
24
+ 'source.hcl' => %w[hcl],
25
+ 'source.java' => %w[java],
26
+ 'source.js' => %w[javascript js],
27
+ 'source.json' => %w[json],
28
+ 'source.julia' => %w[jl julia],
29
+ 'source.lua' => %w[lua],
30
+ 'source.mk' => %w[bsdmake gnumake make makefile mf],
31
+ 'source.nix' => %w[nix nixos],
32
+ 'source.objc' => %w[obj-c obj_c objc objective_c objectivec],
33
+ 'source.ocaml' => %w[ocaml],
34
+ 'source.perl' => %w[perl pl],
35
+ 'source.php' => %w[php php3 php4 php5],
36
+ 'source.proto' => %w[proto protobuf],
37
+ 'source.python' => %w[py python],
38
+ 'source.racket' => %w[racket],
39
+ 'source.ruby' => %w[rb ruby],
40
+ 'source.rust' => %w[rs rust],
41
+ 'source.scala' => %w[scala],
42
+ 'source.sparql' => %w[sparql],
43
+ 'source.sql' => %w[sql],
44
+ 'source.swift' => %w[swift],
45
+ 'source.toml' => %w[toml],
46
+ 'source.ts' => %w[ts typescript],
47
+ 'source.vhd' => %w[vhdl],
48
+ 'text.html.basic' => %w[html],
49
+ 'text.html.erb' => %w[erb eruby rhtml]
50
+ }
51
+ .map { |scope, aliases| aliases.map { |alias_| [alias_, scope] } }
52
+ .flatten(1)
53
+ .to_h
54
+ .freeze
55
+ end
56
+ end
57
+ end
58
+ end
@@ -5,7 +5,7 @@ module Kramdown
5
5
  module SyntaxHighlighter
6
6
  module TreeSitter
7
7
  # Version of kramdown-syntax_tree_sitter gem
8
- VERSION = '0.1.0'
8
+ VERSION = '0.3.0'
9
9
  end
10
10
  end
11
11
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kramdown-syntax_tree_sitter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.3.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrew T. Biehl
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-11-11 00:00:00.000000000 Z
11
+ date: 2023-01-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: kramdown
@@ -139,6 +139,7 @@ files:
139
139
  - ext/tree_sitter_adapter/src/tree_sitter_adapter.rs
140
140
  - lib/kramdown/converter/syntax_highlighter/tree_sitter.rb
141
141
  - lib/kramdown/syntax_tree_sitter.rb
142
+ - lib/kramdown/syntax_tree_sitter/languages.rb
142
143
  - lib/kramdown/syntax_tree_sitter/version.rb
143
144
  - lib/tree_sitter_adapter.rb
144
145
  homepage: https://github.com/andrewtbiehl/kramdown-syntax_tree_sitter
@@ -161,7 +162,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
161
162
  - !ruby/object:Gem::Version
162
163
  version: '0'
163
164
  requirements: []
164
- rubygems_version: 3.3.7
165
+ rubygems_version: 3.4.1
165
166
  signing_key:
166
167
  specification_version: 4
167
168
  summary: Syntax highlight code with Tree-sitter via Kramdown.