magical-py 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. magical_py-0.1.0/CHANGELOG.md +336 -0
  2. magical_py-0.1.0/Cargo.lock +7 -0
  3. magical_py-0.1.0/Cargo.toml +24 -0
  4. magical_py-0.1.0/LICENSE +21 -0
  5. magical_py-0.1.0/PKG-INFO +169 -0
  6. magical_py-0.1.0/README.md +140 -0
  7. magical_py-0.1.0/bindings/python/Cargo.lock +137 -0
  8. magical_py-0.1.0/bindings/python/Cargo.toml +38 -0
  9. magical_py-0.1.0/bindings/python/README.md +140 -0
  10. magical_py-0.1.0/bindings/python/gates.sh +34 -0
  11. magical_py-0.1.0/bindings/python/gen_kinds.ps1 +279 -0
  12. magical_py-0.1.0/bindings/python/pyrightconfig.json +25 -0
  13. magical_py-0.1.0/bindings/python/src/lib.rs +229 -0
  14. magical_py-0.1.0/bindings/python/tests/test_detect.py +147 -0
  15. magical_py-0.1.0/bindings/python/tests/test_drift.py +108 -0
  16. magical_py-0.1.0/bindings/python/tests/test_metadata.py +110 -0
  17. magical_py-0.1.0/bindings/python/tests/test_readme.py +160 -0
  18. magical_py-0.1.0/bindings/python/tests/test_stub.py +68 -0
  19. magical_py-0.1.0/bindings/python/uv.lock +325 -0
  20. magical_py-0.1.0/pyproject.toml +47 -0
  21. magical_py-0.1.0/python/magical_py/__init__.py +98 -0
  22. magical_py-0.1.0/python/magical_py/_kinds.py +525 -0
  23. magical_py-0.1.0/python/magical_py/_magical_rs.pyi +25 -0
  24. magical_py-0.1.0/python/magical_py/py.typed +1 -0
  25. magical_py-0.1.0/readme.md +519 -0
  26. magical_py-0.1.0/src/lib.rs +24 -0
  27. magical_py-0.1.0/src/magical/async_dyn_magic.rs +121 -0
  28. magical_py-0.1.0/src/magical/bytes_read.rs +177 -0
  29. magical_py-0.1.0/src/magical/dyn_magic.rs +145 -0
  30. magical_py-0.1.0/src/magical/ext_fn/shebang.rs +62 -0
  31. magical_py-0.1.0/src/magical/ext_fn/webp.rs +24 -0
  32. magical_py-0.1.0/src/magical/magic.rs +376 -0
  33. magical_py-0.1.0/src/magical/magic_custom.rs +802 -0
  34. magical_py-0.1.0/src/magical/match_rules.rs +4 -0
  35. magical_py-0.1.0/src/magical/signatures.rs +575 -0
  36. magical_py-0.1.0/src/magical/signatures_ext.rs +175 -0
@@ -0,0 +1,336 @@
1
+ # CHANGELOG
2
+ - [CHANGELOG](#changelog)
3
+ - [Version: 0.1.3](#version-013)
4
+ - [Version: 0.2.0](#version-020)
5
+ - [Version: 0.2.1:](#version-021)
6
+ - [Version: 0.3.0:](#version-030)
7
+ - [Version: 0.3.1, `Minor edits`](#version-031-minor-edits)
8
+ - [Version: 0.4.0 `Major API Update`](#version-040-major-api-update)
9
+ - [Version: 0.4.5 `Major API Update`](#version-045-major-api-update)
10
+ - [Version: 0.6.0 `Signature Tightening`](#version-060-signature-tightening)
11
+ - [Version: 0.6.1 `Documentation and Test Coverage`](#version-061-documentation-and-test-coverage)
12
+
13
+
14
+ ## Version: 0.1.3
15
+ **What has been changed:**
16
+ * Added examples of how to use `magical_rs`.
17
+ * Specifically as follows:
18
+
19
+ | Use case | Can be found at |
20
+ | ------------------ | -------------------------------------- |
21
+ | Basic usage | [normal_usage](examples/normal_usage) |
22
+ | Magic Custom usage | [magic_custom](examples/magic_custom/) |
23
+
24
+ * Added category `"no-std"` to [`[Cargo.toml]`](Cargo.toml)
25
+ * Changed repository URL of `magical_rs` in [`Cargo.toml`](Cargo.toml)
26
+ * Added examples to [`Cargo metadata`](Cargo.toml)
27
+
28
+ ## Version: 0.2.0
29
+ **What has been changed:**
30
+ * Added methods to normalize and extends file matching in `CustomRulesMatches`
31
+ * Added documentation and test cases.
32
+ * Still retains backward compatibility for `no_std`.
33
+ * Added some macro to standardize the syntax sugar to make the API more friendly.
34
+ * Some examples of using the macros have also been added. Can be found at: [examples](examples).
35
+ * Some examples of using `DynMagic` have also been added. Can be found at: [examples](examples).
36
+
37
+ **Bellow is a list of macros that have been added:**
38
+
39
+ ---
40
+ | Macro name | Support `no_std`, backward compatibility? |
41
+ | ------------------ | ----------------------------------------- |
42
+ | `match_custom!` | Yes |
43
+ | `magic_custom!` | Yes |
44
+ | `with_fn_matches!` | Yes |
45
+ | `any_matches!` | Yes |
46
+ | `all_matches!` | Yes |
47
+
48
+ ---
49
+ * The list of supported file in `readme.md` will also synchronized.
50
+
51
+ **Bellow is a list of the signature files have been added:**
52
+
53
+ | Name | Signature | Offset |
54
+ | ----------------------- | ------------------------ | ------ |
55
+ | VMDK File | `0x4B, 0x44, 0x4D` | `0` |
56
+ | Google Chrome Extension | `0x43, 0x72, 0x32, 0x34` | `0` |
57
+
58
+ * Bellow is the development roadmap for version `0.2.0`:
59
+
60
+ | Name | Description | Status |
61
+ | ----------------- | ----------------------------------------------------------------------- | ------ |
62
+ | `Macro Supported` | Allows the use of macros to sugar-syntaxize the API | [x] |
63
+ | `MultipeFn` | Support for multiple `OR`, `AND` type pointer function in `CustomMagic` | [x] |
64
+
65
+ ## Version: 0.2.1:
66
+ **What has been changed:**
67
+
68
+ * Fixed the documentation and added use for each module in [readme.md](readme.md)
69
+ * Fixed blank signatures & offsets blank in [magic_custom example](examples/magic_custom/src/v_2_0_0/magic_custom_macro.rs)
70
+
71
+ ## Version: 0.3.0:
72
+ **What has been changed:**
73
+
74
+ * Added feature only avalable in version `0.3.0` of `magical_rs`: `AsyncDynMagic`
75
+ * Added documentation and usage warnings to [`lib.rs`](src/lib.rs) and [`readme.md`](readme.md)
76
+ * From this version onwards, `AsyncDynMagic` becomes an optional module. Cargo and flags are required to enable it:
77
+
78
+ ```bash
79
+ cargo add magical_rs --features magical_async_dyn
80
+ ```
81
+ * Of course, flag `magical_async_dyn` has also been added to [`Cargo.toml`](Cargo.toml)
82
+ * Instructions on how to use have also added at [`AsyncDynMagic Examples`](examples/async_dyn_magic)
83
+ * Current flags in version `0.3.0` can be used:
84
+
85
+ | Name | Description | Cargo flag |
86
+ | ------------------- | -------------------------------------------------------------- | ----------------------- |
87
+ | `magical_dyn` | Unlock lvl 3 with file dection with infinite rules at run time | `magical_dyn` |
88
+ | `magical_async_dyn` | Has all the features of level 3 but supports asynchronous | `magical_async_dyn` |
89
+ | `no_std` | Used in non-std environments like kernel, emebedded | `--no-default-features` |
90
+
91
+ ## Version: 0.3.1, `Minor edits`
92
+ **What has been changed:**
93
+ * Minor edit in [`Cargo.toml`](Cargo.toml), added category slug `asynchronous`
94
+ * Edited some keywords related to the framework in [`Cargo.toml`](Cargo.toml)
95
+ * Changed the description of the framework to better identify it's purpose
96
+
97
+
98
+ ## Version: 0.4.0 `Major API Update`
99
+ **What has been changed:**
100
+ * Added feature flag `unsafe_context` to [`Cargo.toml`](Cargo.toml)
101
+ * Release new features included in the module `magic_custom` is `WithUnsafeFn`
102
+ - Test can be found at: [`here`](tests/unsafe.rs).
103
+ - Documentation and instructions, security warnings have also added for `magic_custom` module.
104
+ - This unsafe feature is only compiled and used when the `unsafe_context` flag is explicitly enabled via `Cargo`:
105
+ ```bash
106
+ cargo add magical_rs --features unsafe_context
107
+ ```
108
+ - This version also adds more documentation and warnings for features like `Default`, `WithFn`.
109
+ - However, `no_std` support is still absolutely guaranteed.
110
+ - Edited [`Makefile`](Makefile) rules, allowing testing with `unsafe_context` feature
111
+ - Added example for using `unsafe_context` [`here`](examples/unsafe_context) and [`readme.md`](readme.md)
112
+ - We do a plan to add bindings to Python. However, we can't show them yet. So, the `bindings` folder will be ignored by Git for now. [`.gitignore`](.gitignore)
113
+
114
+ ## Version: 0.4.5 `Major API Update`
115
+ **What has been changed:**
116
+ * Added support for multiple unsafe function pointers. It will be disabled by default.
117
+ - Only usable if feature flag is explicitly used by Cargo:
118
+ - ```bash
119
+ cargo add magical_rs --features unsafe_context
120
+ ```
121
+ - These new features will not affect `no_std`, and will still be supported.
122
+ - Added testing for the above features. Can be found at [`test`](tests/unsafe.rs)
123
+ - Samples for the above features:
124
+ - `AllMatchesUnsafe`:
125
+ - ```rust
126
+ use core::slice;
127
+ use magical_rs::magical::magic_custom::match_types_custom;
128
+ use magical_rs::magical::magic_custom::{CustomMatchRules, MagicCustom};
129
+
130
+ #[derive(Clone, Copy, PartialEq, Eq, Debug)]
131
+ enum MagicKind {
132
+ MoeMoe,
133
+ UnknownFallback,
134
+ }
135
+
136
+ fn is_shoujo_girl(data: *const ()) -> bool {
137
+ unsafe {
138
+ let slice_ptr = data.cast::<u8>();
139
+ let slice = slice::from_raw_parts(slice_ptr, 100);
140
+
141
+ slice.starts_with(b"MagicalGirl")
142
+ }
143
+ }
144
+
145
+ fn is_not_shoujo_girl(data: *const ()) -> bool {
146
+ unsafe {
147
+ let slice_ptr = data.cast::<u8>();
148
+ let slice = slice::from_raw_parts(slice_ptr, 100);
149
+
150
+ !slice.starts_with(b"MagicalGirl")
151
+ }
152
+ }
153
+
154
+ let rules: &[MagicCustom<MagicKind>] = &[MagicCustom {
155
+ signatures: &[],
156
+ offsets: &[],
157
+ max_bytes_read: 200,
158
+ kind: MagicKind::MoeMoe,
159
+ rules: CustomMatchRules::AllMatchesUnsafe(&[is_shoujo_girl, is_not_shoujo_girl]),
160
+ }];
161
+
162
+ let result = match_types_custom(b"MagicalGirl", rules, MagicKind::UnknownFallback);
163
+
164
+ assert_ne!(result, MagicKind::MoeMoe);
165
+ assert_eq!(result, MagicKind::UnknownFallback);
166
+ ```
167
+ - `AnyMatchesUnsafe`:
168
+ - ```rust
169
+ use core::slice;
170
+ use magical_rs::magical::magic_custom::match_types_custom;
171
+ use magical_rs::magical::magic_custom::{CustomMatchRules, MagicCustom};
172
+
173
+ #[derive(Clone, Copy, PartialEq, Eq, Debug)]
174
+ enum MagicKind {
175
+ MoeMoe,
176
+ UnknownFallback,
177
+ }
178
+
179
+ fn is_shoujo_girl(data: *const ()) -> bool {
180
+ unsafe {
181
+ let slice_ptr = data.cast::<u8>();
182
+ let slice = slice::from_raw_parts(slice_ptr, 100);
183
+
184
+ slice.starts_with(b"MagicalGirl")
185
+ }
186
+ }
187
+
188
+ fn is_not_shoujo_girl(data: *const ()) -> bool {
189
+ unsafe {
190
+ let slice_ptr = data.cast::<u8>();
191
+ let slice = slice::from_raw_parts(slice_ptr, 100);
192
+
193
+ !slice.starts_with(b"MagicalGirl")
194
+ }
195
+ }
196
+
197
+ let rules: &[MagicCustom<MagicKind>] = &[MagicCustom {
198
+ signatures: &[],
199
+ offsets: &[],
200
+ max_bytes_read: 200,
201
+ kind: MagicKind::MoeMoe,
202
+ rules: CustomMatchRules::AnyMatchesUnsafe(&[is_shoujo_girl, is_not_shoujo_girl]),
203
+ }];
204
+
205
+ let result = match_types_custom(b"MagicalGirl", rules, MagicKind::UnknownFallback);
206
+
207
+ assert_eq!(result, MagicKind::MoeMoe);
208
+ assert_ne!(result, MagicKind::UnknownFallback);
209
+ ```
210
+
211
+
212
+ ## Version: 0.6.1 `Documentation and Test Coverage`
213
+
214
+ **No API change, and no format added or removed.** The signature table is
215
+ byte-for-byte the one in `0.6.0`, and the twelve public items are the same
216
+ twelve. This release exists because `0.6.0` shipped a crate that nothing in
217
+ the repository was checking, and a readme whose claims no test agreed with.
218
+
219
+ **What this fixes:**
220
+
221
+ * The readme is now the single source of the crate's documentation. `src/lib.rs`
222
+ carries `#![doc = include_str!("../readme.md")]` instead of 242 lines of
223
+ duplicated doc comment, so the two can no longer disagree. Nothing but
224
+ documentation was removed from the crate.
225
+ * The format table is checked rather than asserted. `tests/table_size.rs`,
226
+ `tests/signature_coverage.rs` and `tests/readme_coverage.rs` fail if the
227
+ built-in table and the readme stop matching, and the nine two-byte-only
228
+ signatures are listed and checked individually.
229
+ * The `no_std` claim is verified instead of asserted. `make test-nostd` runs
230
+ the suite without `std` and cross compiles to `thumbv7em-none-eabi`. It
231
+ excludes doctests deliberately, since the readme's quick start calls a
232
+ `std`-gated function; the cross compile is the real gate.
233
+ * `crate_dev.yml` now runs on pull requests into `master`, not only into
234
+ `dev`. It had not run at all on the code that shipped as `0.6.0`, because
235
+ `dev` had not moved since the commit before it.
236
+
237
+ **Not part of this release:** the Python bindings are a separate package,
238
+ `magical-py`, and are not reachable through this crate.
239
+
240
+ ## Version: 0.6.0 `Signature Tightening` and `Format Table Expansion`
241
+
242
+ **Breaking: three signatures changed.**
243
+ Detection results for the same bytes can differ from `0.5.x`. Review before upgrading.
244
+
245
+ | Format | `0.5.x` | `0.6.0` | Why |
246
+ | --- | --- | --- | --- |
247
+ | `FileKind::Bzip` | `BZ` | `BZh` | `BZ` is only a prefix. The bzip2 block header is `BZh`, so the old rule reported any file starting with those two letters as bzip2. |
248
+ | `FileKind::ScriptExecute` | `#!` | `#!` plus `/` on the same line | `#!` claimed every file starting with those bytes. A real shebang must name an interpreter by path. |
249
+ | `FileKind::Ply` | `ply` | `ply` plus a line break | `ply` claimed any text file starting with that word. The PLY spec puts a line ending straight after the keyword. |
250
+
251
+ **What this fixes:**
252
+ * An AMR audio file is now reported as [`FileKind::Amr`] instead of
253
+ `FileKind::ScriptExecute`. The AMR header `#!AMR` has no path separator, so
254
+ it no longer matches the narrowed shebang rule.
255
+ * Bzip2 detection no longer produces false positives on files that merely
256
+ begin with the letters `BZ`.
257
+ * A `#` comment or `#include` line in a source file is no longer reported as a
258
+ script.
259
+
260
+ **New public module:** `magical::ext_fn::shebang`, exposing `is_shebang`.
261
+ Entries that rely on a predicate rather than a byte signature are now
262
+ `ScriptExecute` and `WEBP`.
263
+
264
+ **Known limitation of the new shebang rule:** a relative interpreter name such
265
+ as `#!python` is not detected, because it contains no path separator. Such a
266
+ script is non-portable in practice. Previously it was also undetected for a
267
+ different reason, so no realistic script is lost.
268
+
269
+ **What was changed:**
270
+ * `magical_rs` is now licensed under the MIT License instead of the GNU General
271
+ Public License v3.0.
272
+ * The built-in format table grew from 48 to 114 formats (142 distinct magic
273
+ signatures), defined in the new `src/magical/signatures_ext.rs`.
274
+ * The 65 new rules are **appended** to `SIGNATURE_KIND`, never interleaved.
275
+ Because `match_types` returns the first match, appending guarantees no
276
+ pre-existing rule can be shadowed. Behaviour of the original 48 formats is
277
+ bit-for-bit unchanged.
278
+ * `readme.md` was rewritten and is now the single source of crate
279
+ documentation, pulled in by `#![doc = include_str!("../readme.md")]`.
280
+ Previously the README and the crate docs were two separate copies that had
281
+ drifted apart; the crate docs contained a doctest referencing `async_std`,
282
+ which is not a dependency, so `cargo test --doc` was already failing on
283
+ `master`.
284
+ * The 65 new rules are **appended** to `SIGNATURE_KIND`, never interleaved.
285
+ Because `match_types` returns the first match, appending guarantees no
286
+ pre-existing rule can be shadowed, apart from the three deliberate changes
287
+ listed above.
288
+
289
+ **Bugs found and fixed in the documentation:**
290
+ * The old format table misdescribed 9 signatures, including XML (the docs
291
+ claimed `<!DOCTYPE` was accepted; the code only matches `<?xml ` with a
292
+ trailing space) and the environment module format (docs said
293
+ `MODULE\0\0\0`; the code checks `#%Module`).
294
+ * The level 5 example in the old README did not compile. It referenced
295
+ `CustomMatchRules::WithFnUnsafe`, which does not exist; the available
296
+ variants are `AllMatchesUnsafe` and `AnyMatchesUnsafe`.
297
+ * `with_bytes_read()` returns 36,870 bytes, not 2,048, because ISO 9660 stores
298
+ its magic at offset 36,865. This is now documented and covered by a test.
299
+
300
+ **New tests:**
301
+ * `tests/signature_coverage.rs` walks `SIGNATURE_KIND` and asserts every entry
302
+ detects itself, that no signature matches at an undeclared offset, that
303
+ padding is never misdetected, and that truncated input never panics. Any
304
+ format added in the future is covered automatically.
305
+ * `tests/signature_tightening.rs` pins the `0.6.0` behaviour change. Each of the
306
+ three tightened formats is tested both ways: real files are still detected,
307
+ and the specific over-match the change was made to fix no longer happens.
308
+ * `tests/readme_coverage.rs` checks the README against the code in both
309
+ directions. Every format the table can return must be named in the README,
310
+ every `FileKind` the README advertises must be one the table can actually
311
+ return, and the stated table size must match `SIGNATURE_KIND.len()`. A guard
312
+ assertion fails the test if the README table layout changes and the parser
313
+ silently stops matching anything.
314
+ * `tests/readme_examples.rs` compiles and runs every code example in the
315
+ README.
316
+ * `tests/table_size.rs` reports the live table size.
317
+
318
+ **Formats evaluated and deliberately excluded:**
319
+ * 3D Studio Max, whose magic is byte-for-byte identical to BigTIFF.
320
+ * Text-based formats, which have no magic bytes.
321
+
322
+ **Why:**
323
+ * The previous GPL-3.0 license capped adoption. Many organizations block GPL-licensed
324
+ dependencies in their build and security policy, which ruled out both corporate
325
+ adoption and the use of `magical_rs` inside permissively licensed tooling.
326
+ * MIT removes that ceiling without changing a single line of library code.
327
+
328
+ **What this does not change:**
329
+ * `magical_rs` is a single-author project. No other contributor holds copyright,
330
+ so no third-party consent was required for this relicense.
331
+ * Versions `0.4.5` and earlier were already distributed under the GPL. Those grants
332
+ are permanent and cannot be revoked, for anyone who already received those
333
+ versions. Anyone depending on `0.4.5` may continue to use it under GPL terms.
334
+ Only versions from `0.5.0` onward are MIT-licensed.
335
+ * No API, behavior, feature flag, or `no_std` support was modified by this change.
336
+
@@ -0,0 +1,7 @@
1
+ # This file is automatically @generated by Cargo.
2
+ # It is not intended for manual editing.
3
+ version = 4
4
+
5
+ [[package]]
6
+ name = "magical_rs"
7
+ version = "0.6.1"
@@ -0,0 +1,24 @@
1
+ [package]
2
+ name = "magical_rs"
3
+ description = """Rust framework for file recognition, aiming for high extensibility and customization."""
4
+ version = "0.6.1"
5
+ edition = "2024"
6
+ license = "MIT"
7
+ keywords = ["detect-file", "no-dependency", "no-std", "magic", "rlib"]
8
+ categories = ["filesystem", "no-std", "asynchronous"]
9
+ include = ["LICENSE", "src/**/*.rs", "CHANGELOG.md"]
10
+ authors = ["Reim-developer", "contact.kaxtr@gmail.com"]
11
+ readme = "readme.md"
12
+ repository = "https://github.com/reim-developer/magical_rs"
13
+
14
+ [lib]
15
+ crate-type = ["lib"]
16
+
17
+ [features]
18
+ default = ["std"]
19
+ std = []
20
+ magical_dyn = []
21
+ magical_async_dyn = []
22
+ unsafe_context = []
23
+
24
+ [dependencies]
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Reim-developer
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,169 @@
1
+ Metadata-Version: 2.4
2
+ Name: magical-py
3
+ Version: 0.1.0
4
+ Classifier: Development Status :: 3 - Alpha
5
+ Classifier: Intended Audience :: Developers
6
+ Classifier: License :: OSI Approved :: MIT License
7
+ Classifier: Programming Language :: Python :: 3
8
+ Classifier: Programming Language :: Python :: 3.8
9
+ Classifier: Programming Language :: Python :: 3.9
10
+ Classifier: Programming Language :: Python :: 3.10
11
+ Classifier: Programming Language :: Python :: 3.11
12
+ Classifier: Programming Language :: Python :: 3.12
13
+ Classifier: Programming Language :: Python :: 3.13
14
+ Classifier: Programming Language :: Python :: 3.14
15
+ Classifier: Programming Language :: Rust
16
+ Classifier: Topic :: Software Development :: Libraries
17
+ Classifier: Topic :: Utilities
18
+ Classifier: Typing :: Typed
19
+ Summary: Native Python bindings for magical_rs, a zero-dependency file type detection library.
20
+ Keywords: file-type,magic,mime,detection,no-dependency
21
+ Home-Page: https://github.com/Reim-developer/magical_rs/blob/master/bindings/python
22
+ License: MIT
23
+ Requires-Python: >=3.8
24
+ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
25
+ Project-URL: Changelog, https://github.com/Reim-developer/magical_rs/blob/master/CHANGELOG.md
26
+ Project-URL: Homepage, https://github.com/Reim-developer/magical_rs
27
+ Project-URL: Repository, https://github.com/Reim-developer/magical_rs
28
+
29
+ # `magical_py`
30
+
31
+ Native Python bindings for
32
+ [`magical_rs`](https://github.com/Reim-developer/magical_rs), a zero-dependency
33
+ Rust library that identifies files by their magic bytes.
34
+
35
+ This is **not** a drop-in replacement for `python-magic`, and it is not trying to
36
+ be one. `python-magic` returns a string like `"image/png"` and leaves you to parse
37
+ it. This returns a typed value that carries its media type and conventional
38
+ extension with it.
39
+
40
+ ## Install
41
+
42
+ Not released to PyPI yet. From a checkout of this repository:
43
+
44
+ ```bash
45
+ pip install ./bindings/python
46
+ ```
47
+
48
+ Once published this becomes `pip install magical-py`, and there is a binary
49
+ wheel for every platform we publish for. One `abi3` wheel covers CPython 3.8
50
+ and later, so you will not have to build from source.
51
+
52
+ `FileKind` covers all 114 formats the Rust crate detects, and the enum is
53
+ declared in Python rather than generated, so it is a real `enum.Enum` and your
54
+ editor completes `FileKind.` with every member and its docstring.
55
+
56
+ ## Use
57
+
58
+ ```python
59
+ from magical_py import FileKind, detect
60
+
61
+ kind = detect("photo.jpg")
62
+
63
+ kind is FileKind.Jpg # True, a real enum member your editor can complete
64
+ kind.value # 'jpg', a stable id for serialising
65
+ kind.mime # 'image/jpeg'
66
+ kind.extension # 'jpg'
67
+ kind.description # 'JPEG'
68
+ ```
69
+
70
+ For data you already have in memory:
71
+
72
+ ```python
73
+ from magical_py import detect_bytes
74
+
75
+ detect_bytes(header) # FileKind.Png | None
76
+ ```
77
+
78
+ Both return `None` when nothing matches, and raise the usual `OSError`
79
+ subclasses when a file cannot be read:
80
+
81
+ ```python
82
+ try:
83
+ kind = detect(path)
84
+ except FileNotFoundError:
85
+ kind = None
86
+ ```
87
+
88
+ ## What it looks like
89
+
90
+ Each member carries its display name as its `__doc__`, which is the attribute
91
+ the REPL and `help()` read. Python 3.9 and later render it for all 114 members;
92
+ Python 3.8 omits it from `help()`, so on 3.8 read `__doc__` directly as below.
93
+
94
+ ```python
95
+ >>> FileKind.Png.__doc__
96
+ 'PNG'
97
+ >>> FileKind.ISO.__doc__
98
+ 'ISO 9660'
99
+ >>> FileKind.PkgZip.description
100
+ 'Zip / JAR / APK'
101
+ ```
102
+
103
+ ## Things worth knowing
104
+
105
+ **Detection reads magic bytes only.** The file name is never consulted, so a
106
+ `.jpg` that actually contains PNG data is reported as `FileKind.Png`. That is
107
+ the behaviour you want from a detection library, and it is also why the
108
+ extension is advisory rather than something to switch on.
109
+
110
+ **Some media types are `None`.** `FileKind.mime` and `FileKind.extension`
111
+ return `None` when no media type is registered or verified for that format.
112
+ `None` means "there isn't one", never "we guessed". A wrong media type would be
113
+ a documentation bug of the same kind as a wrong magic byte, and this project
114
+ does not ship those.
115
+
116
+ **`detect` reads 36,870 bytes.** ISO 9660 stores its magic at offset 36,865, so
117
+ detecting it means reading that far into the file. `detect` does that for you;
118
+ `magical_py.bytes_read()` reports the figure if you are reading headers
119
+ yourself. Do not hardcode 2048 and assume you are done.
120
+
121
+ **Some signatures are only two bytes.** 9 formats match on nothing but a
122
+ two-byte prefix: `Arj`, `Bitmap`, `Gzip`, `MP3`, `MSDOS`, `Pcx`, `Pickle`,
123
+ `SerializedJavaData` and `Zlib`. Those are the values the formats themselves
124
+ specify, so a file beginning with those bytes is reported as that format even
125
+ if it is not really one. Each one is listed with its bytes in the
126
+ [`magical_rs` README](https://github.com/Reim-developer/magical_rs#two-things-to-know-before-you-rely-on-it).
127
+
128
+ **Two rules look at more than a fixed pattern.** `FileKind.ScriptExecute`
129
+ requires a `/` later on the first line, so a bare `#!` does not count;
130
+ `FileKind.WEBP` requires the full `RIFF....WEBP` layout. The shebang rule
131
+ exists because `#!AMR` is the literal magic of AMR audio: without the `/`
132
+ check, the script rule claimed those files and they became undetectable.
133
+
134
+ ## Type checking
135
+
136
+ The package ships a `py.typed` marker and stubs for the compiled module, and it
137
+ is checked with `pyright` in strict mode. `pythonVersion` is pinned to 3.8 so the
138
+ checker enforces the same floor as the wheel.
139
+
140
+ ```python
141
+ from magical_py import FileKind
142
+
143
+ # Narrowed to FileKind after the None check.
144
+ kind: FileKind | None = detect("photo.jpg")
145
+ if kind is not None:
146
+ reveal_type(kind) # FileKind
147
+ reveal_type(kind.mime) # str | None
148
+ ```
149
+
150
+ ## Development
151
+
152
+ Managed with [uv](https://docs.astral.sh/uv/).
153
+
154
+ ```bash
155
+ uv sync # create the venv and install maturin, pytest, pyright
156
+ uv run maturin develop # build the extension into the venv
157
+ uv run pytest # run the tests
158
+ uv run pyright # type check in strict mode
159
+ uv build # build the wheel and the sdist
160
+ ```
161
+
162
+ `python/magical_py/_kinds.py` is generated. Edit the metadata table in
163
+ `gen_kinds.ps1` and re-run it; `tests/test_drift.py` fails if the enum and the
164
+ Rust detection table disagree.
165
+
166
+ ## License
167
+
168
+ MIT, the same as `magical_rs`.
169
+