sas-lexer 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (14) hide show
  1. checksums.yaml +7 -0
  2. data/LICENSE +661 -0
  3. data/README.md +111 -0
  4. data/Rakefile +208 -0
  5. data/ffi-wrapper/Cargo.lock +336 -0
  6. data/ffi-wrapper/Cargo.toml +12 -0
  7. data/ffi-wrapper/src/lib.rs +281 -0
  8. data/lib/native/arm64-darwin/libsas_lexer_ffi.dylib +0 -0
  9. data/lib/native/x86_64-linux/libsas_lexer_ffi.so +0 -0
  10. data/lib/sas_lexer/error.rb +5 -0
  11. data/lib/sas_lexer/lexer.rb +489 -0
  12. data/lib/sas_lexer/version.rb +5 -0
  13. data/lib/sas_lexer.rb +5 -0
  14. metadata +73 -0
data/README.md ADDED
@@ -0,0 +1,111 @@
1
+ # sas-lexer
2
+
3
+ A Ruby gem that wraps the [`sas-lexer`](https://github.com/mishamsk/sas-lexer) Rust crate through the [FFI](https://github.com/ffi/ffi) interface. Tokenizes SAS source code into a stream of typed tokens with full position metadata.
4
+
5
+ This gem is a thin Ruby binding only — all lexing logic lives in the upstream Rust crate. The gem ships prebuilt native shims for supported platforms; a runtime loader picks the matching one for the host.
6
+
7
+ ## Installation
8
+
9
+ Add to your Gemfile:
10
+
11
+ ```ruby
12
+ gem "sas-lexer"
13
+ ```
14
+
15
+ Or install directly:
16
+
17
+ ```sh
18
+ gem install sas-lexer
19
+ ```
20
+
21
+ ## Usage
22
+
23
+ ```ruby
24
+ require "sas_lexer"
25
+
26
+ lexer = SasLexer::Lexer.new
27
+ tokens = lexer.tokenize("data test; set input; run;")
28
+ lexer.free # release the underlying Rust buffers
29
+
30
+ tokens.each do |t|
31
+ puts "#{t[:start_line]}:#{t[:start_column]} type=#{t[:type]} text=#{t[:text].inspect}"
32
+ end
33
+ ```
34
+
35
+ Each token is a hash with the following keys:
36
+
37
+ | key | description |
38
+ | --------------- | ----------------------------------------------------- |
39
+ | `:index` | 0-based position in the token stream |
40
+ | `:text` | raw source text the token spans |
41
+ | `:type` | token type integer (see `SasLexer::Lexer::TokenType`) |
42
+ | `:channel` | `0` default, `1` hidden (whitespace), `2` comment |
43
+ | `:start` | byte offset of token start in the source |
44
+ | `:end` | byte offset of token end |
45
+ | `:start_line` | 1-based line number of token start |
46
+ | `:end_line` | 1-based line number of token end |
47
+ | `:start_column` | 0-based column number of token start |
48
+ | `:end_column` | 0-based column number of token end |
49
+
50
+ Constants for token types and channels are exposed under `SasLexer::Lexer::TokenType` and `SasLexer::Lexer::TokenChannel`. They mirror the enums in `crates/sas-lexer/src/lexer/token_type.rs` of the upstream Rust crate.
51
+
52
+ ## Native library loading
53
+
54
+ `SasLexer::Lexer` probes for a shared library in this order:
55
+
56
+ 1. `lib/native/<platform>/libsas_lexer_ffi.{so,dylib,dll}` — the prebuilt shipped inside the published universal gem for the host's platform.
57
+ 2. `lib/native/libsas_lexer_ffi.{so,dylib,dll}` — the flat path produced by `bundle exec rake sas_lexer:install` for local development.
58
+
59
+ If neither is found, `require "sas_lexer"` raises `SasLexer::Error` immediately. `SasLexer::Lexer::LIBRARY_PATH` exposes the resolved path.
60
+
61
+ ## Building from source
62
+
63
+ You only need to build from source when contributing a prebuilt for a new platform — the published gem already ships every committed platform's artifact.
64
+
65
+ Prerequisites:
66
+
67
+ - Rust toolchain (https://rustup.rs/)
68
+ - Ruby 3.4+
69
+
70
+ ```sh
71
+ bundle install
72
+ bundle exec rake sas_lexer:install
73
+ ```
74
+
75
+ This:
76
+
77
+ 1. Clones the upstream `sas-lexer` repo into `vendor/sas-lexer/`.
78
+ 2. Compiles the FFI shim under `ffi-wrapper/` against it.
79
+ 3. Installs the resulting shared library at `lib/native/libsas_lexer_ffi.<ext>`.
80
+
81
+ To ship the artifact in the universal gem, move it under `lib/native/<platform>/` and commit it:
82
+
83
+ ```sh
84
+ mkdir -p lib/native/$(ruby -e 'puts RUBY_PLATFORM')
85
+ mv lib/native/libsas_lexer_ffi.* lib/native/$(ruby -e 'puts RUBY_PLATFORM')/
86
+ git add lib/native/$(ruby -e 'puts RUBY_PLATFORM')/
87
+ ```
88
+
89
+ To cross-build an `x86_64-linux` `.so` from any host (typically macOS), use the Docker helper:
90
+
91
+ ```sh
92
+ bin/build_linux_extension
93
+ ```
94
+
95
+ ## Testing
96
+
97
+ ```sh
98
+ bundle exec rake spec
99
+ ```
100
+
101
+ ## Why a universal gem (not platform-tagged)?
102
+
103
+ The published gem is `platform: ruby` and ships every committed `lib/native/<platform>/` together. The loader globs at runtime to pick the matching one. Empirically, platform-tagged gems served from some private registries can be served under the generic `<gem>-<version>.gem` URL too, which causes Bundler to lock the version without a platform suffix and then fail to materialize at install time. A single universal gem sidesteps that failure mode at the cost of ~1–2 MB extra per platform shipped.
104
+
105
+ There is no source-build fallback at install time — a host without a committed prebuilt for its platform fails fast at FFI load.
106
+
107
+ ## License
108
+
109
+ `sas-lexer` (the gem) is licensed under the [GNU Affero General Public License v3.0 or later](LICENSE), matching the upstream Rust crate. See `LICENSE` for the full text.
110
+
111
+ The upstream Rust crate is © Misha Perlov. The FFI shim and Ruby binding in this repository are © Mon Ami, Inc.
data/Rakefile ADDED
@@ -0,0 +1,208 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "bundler/setup"
4
+ require "bundler/gem_tasks"
5
+ require "rspec/core/rake_task"
6
+ require "rubocop/rake_task"
7
+ require "fileutils"
8
+
9
+ RSpec::Core::RakeTask.new(:spec)
10
+ RuboCop::RakeTask.new
11
+
12
+ # SAS Lexer build tasks. The published gem ships prebuilt native
13
+ # libraries under `lib/native/<platform>/`; these tasks let
14
+ # contributors build a fresh artifact for their host (e.g. when
15
+ # adding a new platform) by cloning the upstream Rust crate and
16
+ # compiling the FFI shim under `ffi-wrapper/`.
17
+ namespace :sas_lexer do
18
+ SAS_LEXER_REPO = "https://github.com/craigmcnamara/sas-lexer.git"
19
+ SAS_LEXER_BRANCH = "main"
20
+ SAS_LEXER_DIR = "vendor/sas-lexer"
21
+ LIB_DIR = "lib/native"
22
+
23
+ desc "Clone (or update) the upstream sas-lexer repository"
24
+ task :clone do
25
+ if Dir.exist?(SAS_LEXER_DIR)
26
+ puts "SAS Lexer repository already exists at #{SAS_LEXER_DIR}"
27
+ puts "Updating repository..."
28
+ Dir.chdir(SAS_LEXER_DIR) do
29
+ sh "git fetch origin"
30
+ sh "git reset --hard origin/#{SAS_LEXER_BRANCH}"
31
+ end
32
+ else
33
+ puts "Cloning sas-lexer repository..."
34
+ FileUtils.mkdir_p("vendor")
35
+ sh "git clone --branch #{SAS_LEXER_BRANCH} #{SAS_LEXER_REPO} #{SAS_LEXER_DIR}"
36
+ end
37
+ end
38
+
39
+ desc "Check that Rust and Cargo are installed"
40
+ task :check_rust do
41
+ sh "rustc --version", verbose: false
42
+ sh "cargo --version", verbose: false
43
+ puts "✓ Rust and Cargo are installed"
44
+ rescue StandardError
45
+ puts "✗ Rust is not installed or not in PATH"
46
+ puts "Please install Rust from https://rustup.rs/"
47
+ exit 1
48
+ end
49
+
50
+ desc "Build the sas-lexer FFI wrapper library"
51
+ task build: %i[check_rust clone] do
52
+ puts "Building sas-lexer FFI wrapper library..."
53
+
54
+ Dir.chdir("ffi-wrapper") do
55
+ target = case RbConfig::CONFIG["host_os"]
56
+ when /darwin/
57
+ case RbConfig::CONFIG["host_cpu"]
58
+ when /arm64|aarch64/ then "aarch64-apple-darwin"
59
+ when /x86_64/ then "x86_64-apple-darwin"
60
+ end
61
+ when /linux/
62
+ case RbConfig::CONFIG["host_cpu"]
63
+ when /aarch64/ then "aarch64-unknown-linux-gnu"
64
+ when /x86_64/ then "x86_64-unknown-linux-gnu"
65
+ end
66
+ end
67
+
68
+ if target && RbConfig::CONFIG["host_os"] =~ /darwin/
69
+ # Pin the target on macOS so universal-binary hosts produce a
70
+ # single-arch artifact that matches the running Ruby.
71
+ puts "Building FFI wrapper for target: #{target}"
72
+ sh "cargo build --release --target #{target}"
73
+ else
74
+ puts "Building FFI wrapper for default target"
75
+ sh "cargo build --release"
76
+ end
77
+ end
78
+
79
+ puts "✓ SAS Lexer FFI wrapper built successfully"
80
+ end
81
+
82
+ desc "Install the built library into lib/native/"
83
+ task install: [:build] do
84
+ puts "Installing sas-lexer library locally..."
85
+
86
+ FileUtils.mkdir_p(LIB_DIR)
87
+
88
+ case RbConfig::CONFIG["host_os"]
89
+ when /darwin/
90
+ lib_extension = "dylib"
91
+ lib_name = "libsas_lexer_ffi.dylib"
92
+ when /linux/
93
+ lib_extension = "so"
94
+ lib_name = "libsas_lexer_ffi.so"
95
+ when /mswin|mingw|cygwin/
96
+ lib_extension = "dll"
97
+ lib_name = "sas_lexer_ffi.dll"
98
+ else
99
+ puts "Unsupported platform: #{RbConfig::CONFIG['host_os']}"
100
+ exit 1
101
+ end
102
+
103
+ target_arch = case RbConfig::CONFIG["host_os"]
104
+ when /darwin/
105
+ case RbConfig::CONFIG["host_cpu"]
106
+ when /arm64|aarch64/ then "aarch64-apple-darwin"
107
+ when /x86_64/ then "x86_64-apple-darwin"
108
+ end
109
+ end
110
+
111
+ target_dir = if target_arch && RbConfig::CONFIG["host_os"] =~ /darwin/
112
+ File.join("ffi-wrapper", "target", target_arch, "release")
113
+ else
114
+ File.join("ffi-wrapper", "target", "release")
115
+ end
116
+
117
+ possible_names = [
118
+ "libsas_lexer_ffi.#{lib_extension}",
119
+ "sas_lexer_ffi.#{lib_extension}"
120
+ ]
121
+
122
+ built_lib = possible_names
123
+ .map { |name| File.join(target_dir, name) }
124
+ .find { |path| File.exist?(path) }
125
+
126
+ if built_lib.nil?
127
+ puts "Could not find built library in #{target_dir}"
128
+ puts "Available files:"
129
+ Dir.glob(File.join(target_dir, "*")).each { |f| puts " #{File.basename(f)}" }
130
+ exit 1
131
+ end
132
+
133
+ dest_path = File.join(LIB_DIR, lib_name)
134
+ FileUtils.cp(built_lib, dest_path)
135
+
136
+ puts "✓ Library installed to #{dest_path}"
137
+ puts ""
138
+ puts "To ship this artifact in the universal gem, move it under"
139
+ puts "lib/native/<platform>/ and commit it. For example:"
140
+ puts ""
141
+ puts " mkdir -p lib/native/$(ruby -e 'puts RUBY_PLATFORM')"
142
+ puts " mv #{dest_path} lib/native/$(ruby -e 'puts RUBY_PLATFORM')/"
143
+ puts " git add lib/native/$(ruby -e 'puts RUBY_PLATFORM')/#{lib_name}"
144
+ end
145
+
146
+ desc "Clean Rust build artifacts"
147
+ task :clean do
148
+ puts "Cleaning sas-lexer build artifacts..."
149
+
150
+ if Dir.exist?(SAS_LEXER_DIR)
151
+ Dir.chdir(SAS_LEXER_DIR) do
152
+ sh "cargo clean" if File.exist?("Cargo.toml")
153
+ end
154
+ end
155
+
156
+ if Dir.exist?("ffi-wrapper/target")
157
+ Dir.chdir("ffi-wrapper") { sh "cargo clean" }
158
+ end
159
+
160
+ [
161
+ File.join(LIB_DIR, "libsas_lexer_ffi.dylib"),
162
+ File.join(LIB_DIR, "libsas_lexer_ffi.so"),
163
+ File.join(LIB_DIR, "sas_lexer_ffi.dll")
164
+ ].each do |path|
165
+ FileUtils.rm_f(path)
166
+ end
167
+
168
+ puts "✓ Clean completed"
169
+ end
170
+
171
+ desc "Remove the cloned upstream repository and any flat-path libs"
172
+ task :distclean do
173
+ puts "Removing sas-lexer repository..."
174
+ FileUtils.rm_rf(SAS_LEXER_DIR) if Dir.exist?(SAS_LEXER_DIR)
175
+ FileUtils.rm_rf("ffi-wrapper/target") if Dir.exist?("ffi-wrapper/target")
176
+ [
177
+ File.join(LIB_DIR, "libsas_lexer_ffi.dylib"),
178
+ File.join(LIB_DIR, "libsas_lexer_ffi.so"),
179
+ File.join(LIB_DIR, "sas_lexer_ffi.dll")
180
+ ].each { |path| FileUtils.rm_f(path) }
181
+ puts "✓ Repository and flat-path libs removed"
182
+ end
183
+
184
+ desc "Check whether a sas-lexer library is available"
185
+ task :check do
186
+ flat_paths = [
187
+ File.join(LIB_DIR, "libsas_lexer_ffi.so"),
188
+ File.join(LIB_DIR, "libsas_lexer_ffi.dylib"),
189
+ File.join(LIB_DIR, "sas_lexer_ffi.dll")
190
+ ]
191
+ platform_paths = Dir.glob(File.join(LIB_DIR, "*", "libsas_lexer_ffi.{so,dylib,dll}"))
192
+
193
+ if (found = flat_paths.find { |p| File.exist?(p) })
194
+ puts "✓ SAS Lexer library is available (dev-build): #{found}"
195
+ elsif platform_paths.any?
196
+ puts "✓ SAS Lexer libraries are available (committed):"
197
+ platform_paths.each { |p| puts " #{p}" }
198
+ else
199
+ puts "✗ SAS Lexer library is not available"
200
+ puts "Run 'rake sas_lexer:install' to build and install it"
201
+ end
202
+ end
203
+ end
204
+
205
+ desc "Build and install the sas-lexer dependency"
206
+ task build_deps: ["sas_lexer:install"]
207
+
208
+ task default: %i[spec]
data/ffi-wrapper/Cargo.lock ADDED
@@ -0,0 +1,336 @@
1
+ # This file is automatically @generated by Cargo.
2
+ # It is not intended for manual editing.
3
+ version = 3
4
+
5
+ [[package]]
6
+ name = "bit-vec"
7
+ version = "0.8.0"
8
+ source = "registry+https://github.com/rust-lang/crates.io-index"
9
+ checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
10
+
11
+ [[package]]
12
+ name = "encoding"
13
+ version = "0.2.33"
14
+ source = "registry+https://github.com/rust-lang/crates.io-index"
15
+ checksum = "6b0d943856b990d12d3b55b359144ff341533e516d94098b1d3fc1ac666d36ec"
16
+ dependencies = [
17
+ "encoding-index-japanese",
18
+ "encoding-index-korean",
19
+ "encoding-index-simpchinese",
20
+ "encoding-index-singlebyte",
21
+ "encoding-index-tradchinese",
22
+ ]
23
+
24
+ [[package]]
25
+ name = "encoding-index-japanese"
26
+ version = "1.20141219.5"
27
+ source = "registry+https://github.com/rust-lang/crates.io-index"
28
+ checksum = "04e8b2ff42e9a05335dbf8b5c6f7567e5591d0d916ccef4e0b1710d32a0d0c91"
29
+ dependencies = [
30
+ "encoding_index_tests",
31
+ ]
32
+
33
+ [[package]]
34
+ name = "encoding-index-korean"
35
+ version = "1.20141219.5"
36
+ source = "registry+https://github.com/rust-lang/crates.io-index"
37
+ checksum = "4dc33fb8e6bcba213fe2f14275f0963fd16f0a02c878e3095ecfdf5bee529d81"
38
+ dependencies = [
39
+ "encoding_index_tests",
40
+ ]
41
+
42
+ [[package]]
43
+ name = "encoding-index-simpchinese"
44
+ version = "1.20141219.5"
45
+ source = "registry+https://github.com/rust-lang/crates.io-index"
46
+ checksum = "d87a7194909b9118fc707194baa434a4e3b0fb6a5a757c73c3adb07aa25031f7"
47
+ dependencies = [
48
+ "encoding_index_tests",
49
+ ]
50
+
51
+ [[package]]
52
+ name = "encoding-index-singlebyte"
53
+ version = "1.20141219.5"
54
+ source = "registry+https://github.com/rust-lang/crates.io-index"
55
+ checksum = "3351d5acffb224af9ca265f435b859c7c01537c0849754d3db3fdf2bfe2ae84a"
56
+ dependencies = [
57
+ "encoding_index_tests",
58
+ ]
59
+
60
+ [[package]]
61
+ name = "encoding-index-tradchinese"
62
+ version = "1.20141219.5"
63
+ source = "registry+https://github.com/rust-lang/crates.io-index"
64
+ checksum = "fd0e20d5688ce3cab59eb3ef3a2083a5c77bf496cb798dc6fcdb75f323890c18"
65
+ dependencies = [
66
+ "encoding_index_tests",
67
+ ]
68
+
69
+ [[package]]
70
+ name = "encoding_index_tests"
71
+ version = "0.1.4"
72
+ source = "registry+https://github.com/rust-lang/crates.io-index"
73
+ checksum = "a246d82be1c9d791c5dfde9a2bd045fc3cbba3fa2b11ad558f27d01712f00569"
74
+
75
+ [[package]]
76
+ name = "heck"
77
+ version = "0.5.0"
78
+ source = "registry+https://github.com/rust-lang/crates.io-index"
79
+ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
80
+
81
+ [[package]]
82
+ name = "lexical"
83
+ version = "7.0.4"
84
+ source = "registry+https://github.com/rust-lang/crates.io-index"
85
+ checksum = "70ed980ff02623721dc334b9105150b66d0e1f246a92ab5a2eca0335d54c48f6"
86
+ dependencies = [
87
+ "lexical-core",
88
+ ]
89
+
90
+ [[package]]
91
+ name = "lexical-core"
92
+ version = "1.0.5"
93
+ source = "registry+https://github.com/rust-lang/crates.io-index"
94
+ checksum = "b765c31809609075565a70b4b71402281283aeda7ecaf4818ac14a7b2ade8958"
95
+ dependencies = [
96
+ "lexical-parse-float",
97
+ "lexical-parse-integer",
98
+ "lexical-util",
99
+ "lexical-write-float",
100
+ "lexical-write-integer",
101
+ ]
102
+
103
+ [[package]]
104
+ name = "lexical-parse-float"
105
+ version = "1.0.5"
106
+ source = "registry+https://github.com/rust-lang/crates.io-index"
107
+ checksum = "de6f9cb01fb0b08060209a057c048fcbab8717b4c1ecd2eac66ebfe39a65b0f2"
108
+ dependencies = [
109
+ "lexical-parse-integer",
110
+ "lexical-util",
111
+ "static_assertions",
112
+ ]
113
+
114
+ [[package]]
115
+ name = "lexical-parse-integer"
116
+ version = "1.0.5"
117
+ source = "registry+https://github.com/rust-lang/crates.io-index"
118
+ checksum = "72207aae22fc0a121ba7b6d479e42cbfea549af1479c3f3a4f12c70dd66df12e"
119
+ dependencies = [
120
+ "lexical-util",
121
+ "static_assertions",
122
+ ]
123
+
124
+ [[package]]
125
+ name = "lexical-util"
126
+ version = "1.0.6"
127
+ source = "registry+https://github.com/rust-lang/crates.io-index"
128
+ checksum = "5a82e24bf537fd24c177ffbbdc6ebcc8d54732c35b50a3f28cc3f4e4c949a0b3"
129
+ dependencies = [
130
+ "static_assertions",
131
+ ]
132
+
133
+ [[package]]
134
+ name = "lexical-write-float"
135
+ version = "1.0.5"
136
+ source = "registry+https://github.com/rust-lang/crates.io-index"
137
+ checksum = "c5afc668a27f460fb45a81a757b6bf2f43c2d7e30cb5a2dcd3abf294c78d62bd"
138
+ dependencies = [
139
+ "lexical-util",
140
+ "lexical-write-integer",
141
+ "static_assertions",
142
+ ]
143
+
144
+ [[package]]
145
+ name = "lexical-write-integer"
146
+ version = "1.0.5"
147
+ source = "registry+https://github.com/rust-lang/crates.io-index"
148
+ checksum = "629ddff1a914a836fb245616a7888b62903aae58fa771e1d83943035efa0f978"
149
+ dependencies = [
150
+ "lexical-util",
151
+ "static_assertions",
152
+ ]
153
+
154
+ [[package]]
155
+ name = "libc"
156
+ version = "0.2.175"
157
+ source = "registry+https://github.com/rust-lang/crates.io-index"
158
+ checksum = "6a82ae493e598baaea5209805c49bbf2ea7de956d50d7da0da1164f9c6d28543"
159
+
160
+ [[package]]
161
+ name = "phf"
162
+ version = "0.11.3"
163
+ source = "registry+https://github.com/rust-lang/crates.io-index"
164
+ checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
165
+ dependencies = [
166
+ "phf_macros",
167
+ "phf_shared",
168
+ ]
169
+
170
+ [[package]]
171
+ name = "phf_generator"
172
+ version = "0.11.3"
173
+ source = "registry+https://github.com/rust-lang/crates.io-index"
174
+ checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
175
+ dependencies = [
176
+ "phf_shared",
177
+ "rand",
178
+ ]
179
+
180
+ [[package]]
181
+ name = "phf_macros"
182
+ version = "0.11.3"
183
+ source = "registry+https://github.com/rust-lang/crates.io-index"
184
+ checksum = "f84ac04429c13a7ff43785d75ad27569f2951ce0ffd30a3321230db2fc727216"
185
+ dependencies = [
186
+ "phf_generator",
187
+ "phf_shared",
188
+ "proc-macro2",
189
+ "quote",
190
+ "syn",
191
+ ]
192
+
193
+ [[package]]
194
+ name = "phf_shared"
195
+ version = "0.11.3"
196
+ source = "registry+https://github.com/rust-lang/crates.io-index"
197
+ checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5"
198
+ dependencies = [
199
+ "siphasher",
200
+ ]
201
+
202
+ [[package]]
203
+ name = "proc-macro2"
204
+ version = "1.0.101"
205
+ source = "registry+https://github.com/rust-lang/crates.io-index"
206
+ checksum = "89ae43fd86e4158d6db51ad8e2b80f313af9cc74f5c0e03ccb87de09998732de"
207
+ dependencies = [
208
+ "unicode-ident",
209
+ ]
210
+
211
+ [[package]]
212
+ name = "quote"
213
+ version = "1.0.40"
214
+ source = "registry+https://github.com/rust-lang/crates.io-index"
215
+ checksum = "1885c039570dc00dcb4ff087a89e185fd56bae234ddc7f056a945bf36467248d"
216
+ dependencies = [
217
+ "proc-macro2",
218
+ ]
219
+
220
+ [[package]]
221
+ name = "rand"
222
+ version = "0.8.5"
223
+ source = "registry+https://github.com/rust-lang/crates.io-index"
224
+ checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
225
+ dependencies = [
226
+ "rand_core",
227
+ ]
228
+
229
+ [[package]]
230
+ name = "rand_core"
231
+ version = "0.6.4"
232
+ source = "registry+https://github.com/rust-lang/crates.io-index"
233
+ checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
234
+
235
+ [[package]]
236
+ name = "rustc_version"
237
+ version = "0.4.1"
238
+ source = "registry+https://github.com/rust-lang/crates.io-index"
239
+ checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
240
+ dependencies = [
241
+ "semver",
242
+ ]
243
+
244
+ [[package]]
245
+ name = "rustversion"
246
+ version = "1.0.22"
247
+ source = "registry+https://github.com/rust-lang/crates.io-index"
248
+ checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
249
+
250
+ [[package]]
251
+ name = "sas-lexer"
252
+ version = "1.0.0-beta.4"
253
+ dependencies = [
254
+ "bit-vec",
255
+ "encoding",
256
+ "lexical",
257
+ "phf",
258
+ "rustc_version",
259
+ "sas-lexer-macro",
260
+ "strum",
261
+ "unicode-ident",
262
+ ]
263
+
264
+ [[package]]
265
+ name = "sas-lexer-ffi"
266
+ version = "0.1.0"
267
+ dependencies = [
268
+ "libc",
269
+ "sas-lexer",
270
+ ]
271
+
272
+ [[package]]
273
+ name = "sas-lexer-macro"
274
+ version = "0.1.1"
275
+ dependencies = [
276
+ "proc-macro2",
277
+ "quote",
278
+ "syn",
279
+ ]
280
+
281
+ [[package]]
282
+ name = "semver"
283
+ version = "1.0.27"
284
+ source = "registry+https://github.com/rust-lang/crates.io-index"
285
+ checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
286
+
287
+ [[package]]
288
+ name = "siphasher"
289
+ version = "1.0.1"
290
+ source = "registry+https://github.com/rust-lang/crates.io-index"
291
+ checksum = "56199f7ddabf13fe5074ce809e7d3f42b42ae711800501b5b16ea82ad029c39d"
292
+
293
+ [[package]]
294
+ name = "static_assertions"
295
+ version = "1.1.0"
296
+ source = "registry+https://github.com/rust-lang/crates.io-index"
297
+ checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
298
+
299
+ [[package]]
300
+ name = "strum"
301
+ version = "0.26.3"
302
+ source = "registry+https://github.com/rust-lang/crates.io-index"
303
+ checksum = "8fec0f0aef304996cf250b31b5a10dee7980c85da9d759361292b8bca5a18f06"
304
+ dependencies = [
305
+ "strum_macros",
306
+ ]
307
+
308
+ [[package]]
309
+ name = "strum_macros"
310
+ version = "0.26.4"
311
+ source = "registry+https://github.com/rust-lang/crates.io-index"
312
+ checksum = "4c6bee85a5a24955dc440386795aa378cd9cf82acd5f764469152d2270e581be"
313
+ dependencies = [
314
+ "heck",
315
+ "proc-macro2",
316
+ "quote",
317
+ "rustversion",
318
+ "syn",
319
+ ]
320
+
321
+ [[package]]
322
+ name = "syn"
323
+ version = "2.0.106"
324
+ source = "registry+https://github.com/rust-lang/crates.io-index"
325
+ checksum = "ede7c438028d4436d71104916910f5bb611972c5cfd7f89b8300a8186e6fada6"
326
+ dependencies = [
327
+ "proc-macro2",
328
+ "quote",
329
+ "unicode-ident",
330
+ ]
331
+
332
+ [[package]]
333
+ name = "unicode-ident"
334
+ version = "1.0.19"
335
+ source = "registry+https://github.com/rust-lang/crates.io-index"
336
+ checksum = "f63a545481291138910575129486daeaf8ac54aee4387fe7906919f7830c7d9d"
data/ffi-wrapper/Cargo.toml ADDED
@@ -0,0 +1,12 @@
1
+ [package]
2
+ name = "sas-lexer-ffi"
3
+ version = "0.1.0"
4
+ edition = "2021"
5
+
6
+ [lib]
7
+ name = "sas_lexer_ffi"
8
+ crate-type = ["cdylib"]
9
+
10
+ [dependencies]
11
+ sas-lexer = { path = "../vendor/sas-lexer/crates/sas-lexer" }
12
+ libc = "0.2"