bkmr 2.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. bkmr-2.0.1/PKG-INFO +167 -0
  2. bkmr-2.0.1/README.md +151 -0
  3. bkmr-2.0.1/bkmr/Cargo.lock +4003 -0
  4. bkmr-2.0.1/bkmr/Cargo.toml +62 -0
  5. bkmr-2.0.1/bkmr/diesel.toml +8 -0
  6. bkmr-2.0.1/bkmr/migrations/.keep +0 -0
  7. bkmr-2.0.1/bkmr/migrations/2022-12-29-110455_create_bookmarks/down.sql +3 -0
  8. bkmr-2.0.1/bkmr/migrations/2022-12-29-110455_create_bookmarks/up.sql +104 -0
  9. bkmr-2.0.1/bkmr/migrations/2023-12-17-200409_add_embedding_and_content_hash_to_bookmarks/down.sql +3 -0
  10. bkmr-2.0.1/bkmr/migrations/2023-12-17-200409_add_embedding_and_content_hash_to_bookmarks/up.sql +3 -0
  11. bkmr-2.0.1/bkmr/migrations/README.md +5 -0
  12. bkmr-2.0.1/bkmr/src/adapter/dal.rs +361 -0
  13. bkmr-2.0.1/bkmr/src/adapter/embeddings.rs +279 -0
  14. bkmr-2.0.1/bkmr/src/adapter/json.rs +207 -0
  15. bkmr-2.0.1/bkmr/src/adapter/schema.rs +27 -0
  16. bkmr-2.0.1/bkmr/src/environment.rs +88 -0
  17. bkmr-2.0.1/bkmr/src/helper.rs +252 -0
  18. bkmr-2.0.1/bkmr/src/lib.rs +175 -0
  19. bkmr-2.0.1/bkmr/src/macros.rs +35 -0
  20. bkmr-2.0.1/bkmr/src/main.rs +1052 -0
  21. bkmr-2.0.1/bkmr/src/model/bms.rs +133 -0
  22. bkmr-2.0.1/bkmr/src/model/bookmark.rs +371 -0
  23. bkmr-2.0.1/bkmr/src/model/tag.rs +202 -0
  24. bkmr-2.0.1/bkmr/src/service/embeddings.rs +62 -0
  25. bkmr-2.0.1/bkmr/src/service/fzf.rs +250 -0
  26. bkmr-2.0.1/bkmr/src/service/process.rs +670 -0
  27. bkmr-2.0.1/bkmr/tests/adapter/test_json.rs +32 -0
  28. bkmr-2.0.1/bkmr/tests/resources/bkmr.pptx +0 -0
  29. bkmr-2.0.1/bkmr/tests/resources/bkmr.v1.db +0 -0
  30. bkmr-2.0.1/bkmr/tests/resources/bkmr.v2.db +0 -0
  31. bkmr-2.0.1/bkmr/tests/resources/bkmr.v2.noembed.db +0 -0
  32. bkmr-2.0.1/bkmr/tests/resources/data.ndjson +3 -0
  33. bkmr-2.0.1/bkmr/tests/resources/invalid_data.ndjson +3 -0
  34. bkmr-2.0.1/bkmr/tests/resources/sample_docu.md +9 -0
  35. bkmr-2.0.1/bkmr/tests/service/test_embeddings.rs +66 -0
  36. bkmr-2.0.1/bkmr/tests/test_bms.rs +81 -0
  37. bkmr-2.0.1/bkmr/tests/test_dal.rs +265 -0
  38. bkmr-2.0.1/bkmr/tests/test_embeddings.rs +54 -0
  39. bkmr-2.0.1/bkmr/tests/test_fzf.rs +48 -0
  40. bkmr-2.0.1/bkmr/tests/test_lib.rs +122 -0
  41. bkmr-2.0.1/bkmr/tests/test_main.rs +37 -0
  42. bkmr-2.0.1/bkmr/tests/test_models.rs +87 -0
  43. bkmr-2.0.1/bkmr/tests/test_process.rs +66 -0
  44. bkmr-2.0.1/pyproject.toml +72 -0
bkmr-2.0.1/PKG-INFO ADDED
@@ -0,0 +1,167 @@
1
+ Metadata-Version: 2.3
2
+ Name: bkmr
3
+ Version: 2.0.1
4
+ Classifier: Programming Language :: Rust
5
+ Classifier: Programming Language :: Python :: Implementation :: CPython
6
+ License-File: LICENSE
7
+ License-File: AUTHORS
8
+ Summary: Super fast bookmark manager with semantic full text search'
9
+ Keywords: bookmark,launcher
10
+ Author-email: sysid <sysid@gmx.de>
11
+ License: BSD-3-Clause
12
+ Requires-Python: >=3.10
13
+ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
14
+ Project-URL: Source Code, https://github.com/sysid/bkmr
15
+
16
+ # bkmr
17
+
18
+ ![Crates.io](https://img.shields.io/crates/v/bkmr)
19
+ ![Crates.io](https://img.shields.io/crates/d/bkmr)
20
+ [![Docs.rs](https://docs.rs/bkmr/badge.svg)](https://docs.rs/bkmr)
21
+
22
+ [![PyPI Version][pypi-image]][pypi-url]
23
+ [![Downloads](https://static.pepy.tech/badge/bkmr/month)](https://pepy.tech/project/bkmr)
24
+ [![Build Status][build-image]][build-url]
25
+
26
+
27
+ ### [Generalized Semantic Search](https://github.com/sysid/bkmr/wiki/Semantic-Search)
28
+
29
+ # Ultrafast Bookmark Manager and Launcher
30
+
31
+ > New Feature: Semantic Search (AI Embeddings)
32
+
33
+ [Elevating Bookmark Management with AI-Driven Semantic Search](https://sysid.github.io/elevating-bookmark-management-with-ai-driven-semantic-search/)
34
+
35
+ Features:
36
+ - semantic search using OpenAI embeddings (requires OpenAI API key)
37
+ - full-text search with semantic ranking (FTS5)
38
+ - fuzzy search `--fzf` (CTRL-O: copy to clipboard, CTRL-E: edit, CTRL-D: delete, Enter: open)
39
+ - tags for classification
40
+ - can handle HTTP URLs, directories, files (e.g. Office, Images, ....)
41
+ - can execute URI strings as shell commands via protocol prefix: 'shell::'
42
+ URI-Example: `shell::vim +/"## SqlAlchemy" $HOME/document.md`
43
+ - automatically enriches URLs with title and description from Web
44
+ - manages statistics about bookmark usage
45
+
46
+ **`bkmr search --fzf` is a great way to open bookmarks very fast.**
47
+
48
+ ## Usage
49
+ ```bash
50
+ bkmr --help
51
+
52
+ A Bookmark Manager and Launcher for the Terminal
53
+
54
+ Usage: bkmr [OPTIONS] [NAME] [COMMAND]
55
+
56
+ Commands:
57
+ search Searches Bookmarks
58
+ sem-search Semantic Search with OpenAI
59
+ open Open/launch bookmarks
60
+ add Add a bookmark
61
+ delete Delete bookmarks
62
+ update Update bookmarks
63
+ edit Edit bookmarks
64
+ show Show Bookmarks (list of ids, separated by comma, no blanks)
65
+ surprise Opens n random URLs
66
+ tags Tag for which related tags should be shown. No input: all tags are printed
67
+ create-db Initialize bookmark database
68
+ backfill Backfill embeddings for bookmarks
69
+ load-texts Load texts for semantic similarity search
70
+ help Print this message or the help of the given subcommand(s)
71
+
72
+ Arguments:
73
+ [NAME] Optional name to operate on
74
+ ```
75
+
76
+ <a href="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7?autoplay=1&speed=2"><img src="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7.png" width="836"/></a>
77
+
78
+ ### Examples
79
+ ```bash
80
+ # FTS examples (https://www.sqlite.org/fts5.htm)
81
+ bkmr search '"https://securit" *'
82
+ bkmr search 'security NOT keycloak'
83
+
84
+ # FTS combined with tag filtering
85
+ bkmr search -t tag1,tag2 -n notag1 <searchquery>
86
+
87
+ # Search by any tag and sort by bookmark age ascending
88
+ bkmr search -T tag1,tag2 -O
89
+
90
+ # Give me the 10 oldest bookmarks
91
+ bkmr search -O --limit 10
92
+
93
+ # Adding URI to local files
94
+ bkmr add /home/user/presentation.pptx tag1,tag2 --title 'My super Presentation'
95
+
96
+ # Adding shell commands as URI
97
+ bkmr add "shell::vim +/'# SqlAlchemy' sql.md" shell,sql,doc --title 'sqlalchemy snippets'
98
+
99
+ # JSON dump of entire database
100
+ bkmr search --json
101
+
102
+ # Semantic Search based on OpenAI Embeddings
103
+ bkmr --openai sem-search "python security" # requires OPENAI_API_KEY
104
+ ```
105
+ Tags must be separated by comma without blanks.
106
+
107
+ ## Installation
108
+ 1. `cargo install bkmr`
109
+ 2. initialize the database: `bkmr create-db db_path`
110
+ 3. `export "BKMR_DB_URL=db-path"`, location of created sqlite database must be known
111
+ 4. add URLs
112
+
113
+ More configuration options can be found at [documentation page](https://github.com/sysid/bkmr/wiki/configuration).
114
+
115
+ ### Upgrade to 1.x.x
116
+ A database migration will be performed on the first run of the new version.
117
+ This will add two columns to the bookmarks table for the OpenAI embeddings.
118
+ No destructive changes are made to the database.
119
+
120
+ ## Semantic Search
121
+ `bkmr` provides now full semantic search of generalized bookmarks using OpenAI's Embeddings.
122
+
123
+ You can find more information on the [documentation page](https://github.com/sysid/bkmr/wiki/semantic-search).
124
+
125
+ ## Benchmarking
126
+ - ca. 20x faster than the Python original [twbm](https://github.com/sysid/twbm) after warming up Python.
127
+ ```bash
128
+ time twbm search 'zzz*' --np
129
+ 0. zzzeek : Asynchronous Python and Databases [343]
130
+ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
131
+ async, knowhow, py
132
+
133
+
134
+ Found: 1
135
+ 343
136
+
137
+ real 0m0.501s
138
+ user 0m0.268s
139
+ sys 0m0.070s
140
+
141
+
142
+
143
+ time bkmr search 'zzz*' --np
144
+ 1. zzzeek : Asynchronous Python and Databases [343]
145
+ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
146
+ async knowhow py
147
+
148
+
149
+ real 0m0.027s
150
+ user 0m0.008s
151
+ sys 0m0.016s
152
+ ```
153
+ [sysid blog: bkmr](https://sysid.github.io/bkmr/)
154
+
155
+
156
+ <!-- Badges -->
157
+ [pypi-url]: https://pypi.org/project/bkmr/
158
+
159
+ [build-image]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml/badge.svg
160
+ [build-url]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml
161
+
162
+ [quality-image]: https://api.codeclimate.com/v1/badges/3130fa0ba3b7993fbf0a/maintainability
163
+ [quality-url]: https://codeclimate.com/github/nalgeon/podsearch-py
164
+
165
+ [pypi-image]: https://badge.fury.io/py/bkmr.svg
166
+ [pypi-url]: https://pypi.org/project/bkmr/
167
+
bkmr-2.0.1/README.md ADDED
@@ -0,0 +1,151 @@
1
+ # bkmr
2
+
3
+ ![Crates.io](https://img.shields.io/crates/v/bkmr)
4
+ ![Crates.io](https://img.shields.io/crates/d/bkmr)
5
+ [![Docs.rs](https://docs.rs/bkmr/badge.svg)](https://docs.rs/bkmr)
6
+
7
+ [![PyPI Version][pypi-image]][pypi-url]
8
+ [![Downloads](https://static.pepy.tech/badge/bkmr/month)](https://pepy.tech/project/bkmr)
9
+ [![Build Status][build-image]][build-url]
10
+
11
+
12
+ ### [Generalized Semantic Search](https://github.com/sysid/bkmr/wiki/Semantic-Search)
13
+
14
+ # Ultrafast Bookmark Manager and Launcher
15
+
16
+ > New Feature: Semantic Search (AI Embeddings)
17
+
18
+ [Elevating Bookmark Management with AI-Driven Semantic Search](https://sysid.github.io/elevating-bookmark-management-with-ai-driven-semantic-search/)
19
+
20
+ Features:
21
+ - semantic search using OpenAI embeddings (requires OpenAI API key)
22
+ - full-text search with semantic ranking (FTS5)
23
+ - fuzzy search `--fzf` (CTRL-O: copy to clipboard, CTRL-E: edit, CTRL-D: delete, Enter: open)
24
+ - tags for classification
25
+ - can handle HTTP URLs, directories, files (e.g. Office, Images, ....)
26
+ - can execute URI strings as shell commands via protocol prefix: 'shell::'
27
+ URI-Example: `shell::vim +/"## SqlAlchemy" $HOME/document.md`
28
+ - automatically enriches URLs with title and description from Web
29
+ - manages statistics about bookmark usage
30
+
31
+ **`bkmr search --fzf` is a great way to open bookmarks very fast.**
32
+
33
+ ## Usage
34
+ ```bash
35
+ bkmr --help
36
+
37
+ A Bookmark Manager and Launcher for the Terminal
38
+
39
+ Usage: bkmr [OPTIONS] [NAME] [COMMAND]
40
+
41
+ Commands:
42
+ search Searches Bookmarks
43
+ sem-search Semantic Search with OpenAI
44
+ open Open/launch bookmarks
45
+ add Add a bookmark
46
+ delete Delete bookmarks
47
+ update Update bookmarks
48
+ edit Edit bookmarks
49
+ show Show Bookmarks (list of ids, separated by comma, no blanks)
50
+ surprise Opens n random URLs
51
+ tags Tag for which related tags should be shown. No input: all tags are printed
52
+ create-db Initialize bookmark database
53
+ backfill Backfill embeddings for bookmarks
54
+ load-texts Load texts for semantic similarity search
55
+ help Print this message or the help of the given subcommand(s)
56
+
57
+ Arguments:
58
+ [NAME] Optional name to operate on
59
+ ```
60
+
61
+ <a href="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7?autoplay=1&speed=2"><img src="https://asciinema.org/a/ULCDIrw4pG9diaVJb17AjIAa7.png" width="836"/></a>
62
+
63
+ ### Examples
64
+ ```bash
65
+ # FTS examples (https://www.sqlite.org/fts5.htm)
66
+ bkmr search '"https://securit" *'
67
+ bkmr search 'security NOT keycloak'
68
+
69
+ # FTS combined with tag filtering
70
+ bkmr search -t tag1,tag2 -n notag1 <searchquery>
71
+
72
+ # Search by any tag and sort by bookmark age ascending
73
+ bkmr search -T tag1,tag2 -O
74
+
75
+ # Give me the 10 oldest bookmarks
76
+ bkmr search -O --limit 10
77
+
78
+ # Adding URI to local files
79
+ bkmr add /home/user/presentation.pptx tag1,tag2 --title 'My super Presentation'
80
+
81
+ # Adding shell commands as URI
82
+ bkmr add "shell::vim +/'# SqlAlchemy' sql.md" shell,sql,doc --title 'sqlalchemy snippets'
83
+
84
+ # JSON dump of entire database
85
+ bkmr search --json
86
+
87
+ # Semantic Search based on OpenAI Embeddings
88
+ bkmr --openai sem-search "python security" # requires OPENAI_API_KEY
89
+ ```
90
+ Tags must be separated by comma without blanks.
91
+
92
+ ## Installation
93
+ 1. `cargo install bkmr`
94
+ 2. initialize the database: `bkmr create-db db_path`
95
+ 3. `export "BKMR_DB_URL=db-path"`, location of created sqlite database must be known
96
+ 4. add URLs
97
+
98
+ More configuration options can be found at [documentation page](https://github.com/sysid/bkmr/wiki/configuration).
99
+
100
+ ### Upgrade to 1.x.x
101
+ A database migration will be performed on the first run of the new version.
102
+ This will add two columns to the bookmarks table for the OpenAI embeddings.
103
+ No destructive changes are made to the database.
104
+
105
+ ## Semantic Search
106
+ `bkmr` provides now full semantic search of generalized bookmarks using OpenAI's Embeddings.
107
+
108
+ You can find more information on the [documentation page](https://github.com/sysid/bkmr/wiki/semantic-search).
109
+
110
+ ## Benchmarking
111
+ - ca. 20x faster than the Python original [twbm](https://github.com/sysid/twbm) after warming up Python.
112
+ ```bash
113
+ time twbm search 'zzz*' --np
114
+ 0. zzzeek : Asynchronous Python and Databases [343]
115
+ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
116
+ async, knowhow, py
117
+
118
+
119
+ Found: 1
120
+ 343
121
+
122
+ real 0m0.501s
123
+ user 0m0.268s
124
+ sys 0m0.070s
125
+
126
+
127
+
128
+ time bkmr search 'zzz*' --np
129
+ 1. zzzeek : Asynchronous Python and Databases [343]
130
+ https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
131
+ async knowhow py
132
+
133
+
134
+ real 0m0.027s
135
+ user 0m0.008s
136
+ sys 0m0.016s
137
+ ```
138
+ [sysid blog: bkmr](https://sysid.github.io/bkmr/)
139
+
140
+
141
+ <!-- Badges -->
142
+ [pypi-url]: https://pypi.org/project/bkmr/
143
+
144
+ [build-image]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml/badge.svg
145
+ [build-url]: https://github.com/sysid/bkmr/actions/workflows/release_wheels.yml
146
+
147
+ [quality-image]: https://api.codeclimate.com/v1/badges/3130fa0ba3b7993fbf0a/maintainability
148
+ [quality-url]: https://codeclimate.com/github/nalgeon/podsearch-py
149
+
150
+ [pypi-image]: https://badge.fury.io/py/bkmr.svg
151
+ [pypi-url]: https://pypi.org/project/bkmr/