strprob 1.0.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- strprob-1.0.2/LICENSE +9 -0
- strprob-1.0.2/PKG-INFO +179 -0
- strprob-1.0.2/README.md +161 -0
- strprob-1.0.2/pyproject.toml +23 -0
- strprob-1.0.2/setup.cfg +21 -0
- strprob-1.0.2/src/pstr/__init__.py +0 -0
- strprob-1.0.2/src/pstr/strprob.py +204 -0
- strprob-1.0.2/src/strprob.egg-info/PKG-INFO +179 -0
- strprob-1.0.2/src/strprob.egg-info/SOURCES.txt +11 -0
- strprob-1.0.2/src/strprob.egg-info/dependency_links.txt +1 -0
- strprob-1.0.2/src/strprob.egg-info/requires.txt +2 -0
- strprob-1.0.2/src/strprob.egg-info/top_level.txt +1 -0
strprob-1.0.2/LICENSE
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright © 2026 Stephen Mather, spmather, smather
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
|
6
|
+
|
|
7
|
+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
|
8
|
+
|
|
9
|
+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
strprob-1.0.2/PKG-INFO
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: strprob
|
|
3
|
+
Version: 1.0.2
|
|
4
|
+
Summary: A string comparison tool
|
|
5
|
+
Home-page: https://github.com/spmather/pstr
|
|
6
|
+
Author-email: spmather <spm-pypi-issues@pm.me>
|
|
7
|
+
License-Expression: MIT
|
|
8
|
+
Project-URL: Homepage, https://github.com/spmather/pstr
|
|
9
|
+
Project-URL: Issues, https://github.com/spmather/pstr/issues
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: string
|
|
16
|
+
Requires-Dist: argparse
|
|
17
|
+
Dynamic: license-file
|
|
18
|
+
|
|
19
|
+
# Pstr "strprob"
|
|
20
|
+
|
|
21
|
+
Pstr or strprob is a small fuzzy string comparison tool in python for comparing two strings and how closely they match using the equation: `( (common words/total words) + (common characters/total characters) ) / (the mean of the input)`. The output is an number between 0 and 1, including float values.
|
|
22
|
+
|
|
23
|
+
# When to use
|
|
24
|
+
|
|
25
|
+
There are a few times when a regex is not powerful enough without significant modification to compare strings for matching. I created this fuzzy string to accomodate specific needs I had.
|
|
26
|
+
|
|
27
|
+
# Math
|
|
28
|
+
|
|
29
|
+
The equation that is calculated is a probability range from 0 to 1 based on how closely the strings match characters and words.
|
|
30
|
+
|
|
31
|
+
##### Working through the math
|
|
32
|
+
|
|
33
|
+
A single word comparison: `Car` and `Cat`
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
# Comparing Characters
|
|
37
|
+
C = C ... 100%
|
|
38
|
+
A = A ... 100%
|
|
39
|
+
R != T ... 0%
|
|
40
|
+
# Matches/Total = 4/6
|
|
41
|
+
# Result: 0.66 match
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
# Comparing words
|
|
45
|
+
Cat != Car ... 0%
|
|
46
|
+
# Matches/Total = 0/2
|
|
47
|
+
# Result: 0.00 match
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
# Mean of characters and words
|
|
51
|
+
0.66 + 0.00 / 2 = 0.33 or 33% match
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
A multiple word comparison: `Hello my new friend` and `Hello my old friend`
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
# Comparing Characters
|
|
58
|
+
H = H ... 100%
|
|
59
|
+
E = E ... 100%
|
|
60
|
+
L = L ... 100%
|
|
61
|
+
L = L ... 100%
|
|
62
|
+
O = O ... 100%
|
|
63
|
+
M = M ... 100%
|
|
64
|
+
Y = Y ... 100%
|
|
65
|
+
N != O ... 0%
|
|
66
|
+
E != L ... 0%
|
|
67
|
+
W != D ... 0%
|
|
68
|
+
F = F ... 100%
|
|
69
|
+
R = R ... 100%
|
|
70
|
+
I = I ... 100%
|
|
71
|
+
E = E ... 100%
|
|
72
|
+
N = N ... 100%
|
|
73
|
+
D = D ... 100%
|
|
74
|
+
# Matches/Total = 26/32
|
|
75
|
+
# Result: 0.8125 match
|
|
76
|
+
|
|
77
|
+
|
|
78
|
+
# Comparing words
|
|
79
|
+
Hello = Hello ... 100%
|
|
80
|
+
My = My ... 100%
|
|
81
|
+
New != Old ... 0%
|
|
82
|
+
Friend = Friend ... 100%
|
|
83
|
+
# Matches/Total = 6/8
|
|
84
|
+
# Result: 0.75 match
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
# Mean of characters and words
|
|
88
|
+
(0.8125 + 0.75) / 2 = 0.78125 or 78.125% match
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
##### Accuracy
|
|
94
|
+
|
|
95
|
+
I have used this over several workloads and typically, I will put the accuracy threshhold at `0.859` for important work. I.E.
|
|
96
|
+
|
|
97
|
+
```powershell
|
|
98
|
+
if ($pstrValue -gt 0.859) {
|
|
99
|
+
# do your command
|
|
100
|
+
}
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
I also find it to have excellent use at the end of any `if` chains/statements involving matching a string to regex or any string-related validation:
|
|
104
|
+
|
|
105
|
+
```powershell
|
|
106
|
+
if ($something -match $regex) {
|
|
107
|
+
# do your command
|
|
108
|
+
}
|
|
109
|
+
elseif ($pstrValue -gt 0.859) {
|
|
110
|
+
# do your command
|
|
111
|
+
}
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
##### In summary
|
|
115
|
+
|
|
116
|
+
1. Above `0.949` = Too high for most uses
|
|
117
|
+
|
|
118
|
+
2. `0.859` = Highly accurate
|
|
119
|
+
|
|
120
|
+
3. `0.799` = Pretty good accurate
|
|
121
|
+
|
|
122
|
+
4. `0.649` = Absolute minimum for most uses
|
|
123
|
+
|
|
124
|
+
5. Below `0.549` = Patently inaccurate
|
|
125
|
+
|
|
126
|
+
##### Windows
|
|
127
|
+
|
|
128
|
+
```powershell
|
|
129
|
+
pip install pstr
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
##### Linux
|
|
135
|
+
|
|
136
|
+
If the package is not added to your distro's repo, then you need to create a venv like this:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
# Makes a directory at a path. Change
|
|
140
|
+
# "/path/to/new/virtual/environment"
|
|
141
|
+
# to an actual path in your file system
|
|
142
|
+
mkdir /path/to/new/virtual/environment
|
|
143
|
+
|
|
144
|
+
# Create the virtual environment
|
|
145
|
+
# Some distros do not come packaged with venv
|
|
146
|
+
# follow the install prompts to add python3-venv
|
|
147
|
+
# if applicable
|
|
148
|
+
python -m venv /path/to/new/virtual/environment
|
|
149
|
+
|
|
150
|
+
# Use the new venv
|
|
151
|
+
source /path/to/new/virtual/environment/bin/activate
|
|
152
|
+
|
|
153
|
+
# Install the pstr package
|
|
154
|
+
pip install pstr
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
|
|
159
|
+
##### Next use the following syntax
|
|
160
|
+
|
|
161
|
+
From PowerShell or Bash
|
|
162
|
+
|
|
163
|
+
1. Example 1: Comparing the similarity of `"string 1 here"` and `"string 2 here"`
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
./pstr.py pstr "string 1 here" "string 2 here"
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
2. Example 2: Comparing the similarity of `"string 1 here"` and `"string 2 here"` but with debug
|
|
170
|
+
|
|
171
|
+
|
|
172
|
+
|
|
173
|
+
```
|
|
174
|
+
./pstr.py pstrd "string 1 here" "string 2 here"
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
### Future plans
|
|
178
|
+
1. add case insensitive options
|
|
179
|
+
2. fix the brute force recursion in some cases where 1,000 iterations doesn't remove items from the character lists
|
strprob-1.0.2/README.md
ADDED
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Pstr "strprob"
|
|
2
|
+
|
|
3
|
+
Pstr or strprob is a small fuzzy string comparison tool in python for comparing two strings and how closely they match using the equation: `( (common words/total words) + (common characters/total characters) ) / (the mean of the input)`. The output is an number between 0 and 1, including float values.
|
|
4
|
+
|
|
5
|
+
# When to use
|
|
6
|
+
|
|
7
|
+
There are a few times when a regex is not powerful enough without significant modification to compare strings for matching. I created this fuzzy string to accomodate specific needs I had.
|
|
8
|
+
|
|
9
|
+
# Math
|
|
10
|
+
|
|
11
|
+
The equation that is calculated is a probability range from 0 to 1 based on how closely the strings match characters and words.
|
|
12
|
+
|
|
13
|
+
##### Working through the math
|
|
14
|
+
|
|
15
|
+
A single word comparison: `Car` and `Cat`
|
|
16
|
+
|
|
17
|
+
```
|
|
18
|
+
# Comparing Characters
|
|
19
|
+
C = C ... 100%
|
|
20
|
+
A = A ... 100%
|
|
21
|
+
R != T ... 0%
|
|
22
|
+
# Matches/Total = 4/6
|
|
23
|
+
# Result: 0.66 match
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
# Comparing words
|
|
27
|
+
Cat != Car ... 0%
|
|
28
|
+
# Matches/Total = 0/2
|
|
29
|
+
# Result: 0.00 match
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
# Mean of characters and words
|
|
33
|
+
0.66 + 0.00 / 2 = 0.33 or 33% match
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
A multiple word comparison: `Hello my new friend` and `Hello my old friend`
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
# Comparing Characters
|
|
40
|
+
H = H ... 100%
|
|
41
|
+
E = E ... 100%
|
|
42
|
+
L = L ... 100%
|
|
43
|
+
L = L ... 100%
|
|
44
|
+
O = O ... 100%
|
|
45
|
+
M = M ... 100%
|
|
46
|
+
Y = Y ... 100%
|
|
47
|
+
N != O ... 0%
|
|
48
|
+
E != L ... 0%
|
|
49
|
+
W != D ... 0%
|
|
50
|
+
F = F ... 100%
|
|
51
|
+
R = R ... 100%
|
|
52
|
+
I = I ... 100%
|
|
53
|
+
E = E ... 100%
|
|
54
|
+
N = N ... 100%
|
|
55
|
+
D = D ... 100%
|
|
56
|
+
# Matches/Total = 26/32
|
|
57
|
+
# Result: 0.8125 match
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
# Comparing words
|
|
61
|
+
Hello = Hello ... 100%
|
|
62
|
+
My = My ... 100%
|
|
63
|
+
New != Old ... 0%
|
|
64
|
+
Friend = Friend ... 100%
|
|
65
|
+
# Matches/Total = 6/8
|
|
66
|
+
# Result: 0.75 match
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
# Mean of characters and words
|
|
70
|
+
(0.8125 + 0.75) / 2 = 0.78125 or 78.125% match
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
##### Accuracy
|
|
76
|
+
|
|
77
|
+
I have used this over several workloads and typically, I will put the accuracy threshhold at `0.859` for important work. I.E.
|
|
78
|
+
|
|
79
|
+
```powershell
|
|
80
|
+
if ($pstrValue -gt 0.859) {
|
|
81
|
+
# do your command
|
|
82
|
+
}
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
I also find it to have excellent use at the end of any `if` chains/statements involving matching a string to regex or any string-related validation:
|
|
86
|
+
|
|
87
|
+
```powershell
|
|
88
|
+
if ($something -match $regex) {
|
|
89
|
+
# do your command
|
|
90
|
+
}
|
|
91
|
+
elseif ($pstrValue -gt 0.859) {
|
|
92
|
+
# do your command
|
|
93
|
+
}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
##### In summary
|
|
97
|
+
|
|
98
|
+
1. Above `0.949` = Too high for most uses
|
|
99
|
+
|
|
100
|
+
2. `0.859` = Highly accurate
|
|
101
|
+
|
|
102
|
+
3. `0.799` = Pretty good accurate
|
|
103
|
+
|
|
104
|
+
4. `0.649` = Absolute minimum for most uses
|
|
105
|
+
|
|
106
|
+
5. Below `0.549` = Patently inaccurate
|
|
107
|
+
|
|
108
|
+
##### Windows
|
|
109
|
+
|
|
110
|
+
```powershell
|
|
111
|
+
pip install pstr
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
##### Linux
|
|
117
|
+
|
|
118
|
+
If the package is not added to your distro's repo, then you need to create a venv like this:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
# Makes a directory at a path. Change
|
|
122
|
+
# "/path/to/new/virtual/environment"
|
|
123
|
+
# to an actual path in your file system
|
|
124
|
+
mkdir /path/to/new/virtual/environment
|
|
125
|
+
|
|
126
|
+
# Create the virtual environment
|
|
127
|
+
# Some distros do not come packaged with venv
|
|
128
|
+
# follow the install prompts to add python3-venv
|
|
129
|
+
# if applicable
|
|
130
|
+
python -m venv /path/to/new/virtual/environment
|
|
131
|
+
|
|
132
|
+
# Use the new venv
|
|
133
|
+
source /path/to/new/virtual/environment/bin/activate
|
|
134
|
+
|
|
135
|
+
# Install the pstr package
|
|
136
|
+
pip install pstr
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
##### Next use the following syntax
|
|
142
|
+
|
|
143
|
+
From PowerShell or Bash
|
|
144
|
+
|
|
145
|
+
1. Example 1: Comparing the similarity of `"string 1 here"` and `"string 2 here"`
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
./pstr.py pstr "string 1 here" "string 2 here"
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
2. Example 2: Comparing the similarity of `"string 1 here"` and `"string 2 here"` but with debug
|
|
152
|
+
|
|
153
|
+
|
|
154
|
+
|
|
155
|
+
```
|
|
156
|
+
./pstr.py pstrd "string 1 here" "string 2 here"
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
### Future plans
|
|
160
|
+
1. add case insensitive options
|
|
161
|
+
2. fix the brute force recursion in some cases where 1,000 iterations doesn't remove items from the character lists
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
[project]
|
|
2
|
+
name = "strprob"
|
|
3
|
+
version = "1.0.2"
|
|
4
|
+
authors = [
|
|
5
|
+
{ name="spmather", email="spm-pypi-issues@pm.me" },
|
|
6
|
+
]
|
|
7
|
+
description = "A string comparison tool"
|
|
8
|
+
readme = "README.md"
|
|
9
|
+
requires-python = ">=3.9"
|
|
10
|
+
classifiers = [
|
|
11
|
+
"Programming Language :: Python :: 3",
|
|
12
|
+
"Operating System :: OS Independent"
|
|
13
|
+
]
|
|
14
|
+
license = "MIT"
|
|
15
|
+
license-files = ["LICEN[CS]E*"]
|
|
16
|
+
dependencies = [
|
|
17
|
+
"string",
|
|
18
|
+
"argparse"
|
|
19
|
+
]
|
|
20
|
+
|
|
21
|
+
[project.urls]
|
|
22
|
+
Homepage = "https://github.com/spmather/pstr"
|
|
23
|
+
Issues = "https://github.com/spmather/pstr/issues"
|
strprob-1.0.2/setup.cfg
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
[metadata]
|
|
2
|
+
name = strprob
|
|
3
|
+
version = 1.0.2
|
|
4
|
+
authors = spmather
|
|
5
|
+
author_email = spm-pypi-issues@pm.me
|
|
6
|
+
description = A string comparison tool
|
|
7
|
+
long_description = file: README.md
|
|
8
|
+
long_description_content_type = text/markdown
|
|
9
|
+
license = MIT
|
|
10
|
+
url = https://github.com/spmather/pstr
|
|
11
|
+
classifiers =
|
|
12
|
+
Programming Language :: Python :: 3
|
|
13
|
+
Operating System :: OS Independent
|
|
14
|
+
|
|
15
|
+
[options]
|
|
16
|
+
python_requires = >=3.9
|
|
17
|
+
|
|
18
|
+
[egg_info]
|
|
19
|
+
tag_build =
|
|
20
|
+
tag_date = 0
|
|
21
|
+
|
|
File without changes
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
#!/bin/python3
|
|
2
|
+
|
|
3
|
+
# spmather
|
|
4
|
+
# created 2026-07-15
|
|
5
|
+
# updated 2026-08-04
|
|
6
|
+
# packaged for pypi 2026-10-17
|
|
7
|
+
# 1.0.2
|
|
8
|
+
|
|
9
|
+
import string
|
|
10
|
+
import argparse
|
|
11
|
+
|
|
12
|
+
def pstrd(reference,difference):
|
|
13
|
+
"""
|
|
14
|
+
Probability of two STRings matching (with Debug)... P STR D
|
|
15
|
+
using the equation
|
|
16
|
+
((common words/total words)+(common characters/total characters))/2 or the mean of the input
|
|
17
|
+
|
|
18
|
+
"""
|
|
19
|
+
|
|
20
|
+
# Formats words in a string as a list
|
|
21
|
+
refword_l = []
|
|
22
|
+
difword_l = []
|
|
23
|
+
for sym in string.punctuation:
|
|
24
|
+
reference = reference.replace(sym,' ')
|
|
25
|
+
ref_replace = reference
|
|
26
|
+
print(f"Reference list with replacement {ref_replace}")
|
|
27
|
+
refword_l = ref_replace.split()
|
|
28
|
+
print(f"Reference word list {refword_l}")
|
|
29
|
+
for sym in string.punctuation:
|
|
30
|
+
difference = difference.replace(sym,' ')
|
|
31
|
+
dif_replace = difference
|
|
32
|
+
print(f"Difference list with replacement {dif_replace}")
|
|
33
|
+
difword_l = dif_replace.split()
|
|
34
|
+
print(f"Difference word list {difword_l}")
|
|
35
|
+
|
|
36
|
+
# Formats characters in a string as a list
|
|
37
|
+
refchar_l = list(ref_replace.lower().replace(' ',''))
|
|
38
|
+
print(f"Reference character list is {refchar_l}")
|
|
39
|
+
difchar_l = list(dif_replace.lower().replace(' ',''))
|
|
40
|
+
print(f"Difference character list is {difchar_l}")
|
|
41
|
+
|
|
42
|
+
# Find the total number of characters
|
|
43
|
+
charlen = len(refchar_l) + len(difchar_l)
|
|
44
|
+
print(f"Total number of characters {charlen}")
|
|
45
|
+
|
|
46
|
+
# Find the total number of words
|
|
47
|
+
wordlen = len(refword_l) + len(difword_l)
|
|
48
|
+
print(f"Total number of words {wordlen}")
|
|
49
|
+
|
|
50
|
+
# Compare the values in character lists
|
|
51
|
+
sortrefchar_l = sorted(refchar_l)
|
|
52
|
+
sortdifchar_l = sorted(difchar_l)
|
|
53
|
+
commonchar_l = []
|
|
54
|
+
|
|
55
|
+
# Brute force iterations
|
|
56
|
+
# I had issues with enumerate(zip()) missing letters
|
|
57
|
+
# In: pstrd("Hello my old friend","Hello my new friend"), the letters m and r were excluded
|
|
58
|
+
|
|
59
|
+
upperlimit = charlen ** 2
|
|
60
|
+
errorcount = 0
|
|
61
|
+
|
|
62
|
+
for i in range(charlen):
|
|
63
|
+
for refletter in sortrefchar_l:
|
|
64
|
+
for difletter in sortdifchar_l:
|
|
65
|
+
print(i)
|
|
66
|
+
print(f"~~~~ iteration count is {i} ~~~~")
|
|
67
|
+
if refletter == difletter:
|
|
68
|
+
print(f"reference letter is {refletter}")
|
|
69
|
+
print(f"difference letter is {difletter}")
|
|
70
|
+
commonchar_l.append(refletter)
|
|
71
|
+
commonchar_l.append(difletter)
|
|
72
|
+
print(f"common character list is now {commonchar_l}")
|
|
73
|
+
sortrefchar_l.remove(refletter)
|
|
74
|
+
sortdifchar_l.remove(difletter)
|
|
75
|
+
print(f"sorted reference character list is now {sortrefchar_l}")
|
|
76
|
+
print(f"sorted difference character list is now {sortdifchar_l}")
|
|
77
|
+
else:
|
|
78
|
+
errorcount += 1
|
|
79
|
+
print(f"errorcount is {errorcount}")
|
|
80
|
+
if i > upperlimit:
|
|
81
|
+
print(f"iteration {i} is over exponential limits {upperlimit}")
|
|
82
|
+
if errorcount == upperlimit:
|
|
83
|
+
print(f"errorcount {errorcount} is at exponential limits {upperlimit}")
|
|
84
|
+
elif errorcount > upperlimit:
|
|
85
|
+
print(f"errorcount {errorcount} is over exponential limits {upperlimit}")
|
|
86
|
+
print(f"Common characters are {commonchar_l}")
|
|
87
|
+
commoncharlen = len(commonchar_l)
|
|
88
|
+
print(f"Number of common characters {commoncharlen}")
|
|
89
|
+
|
|
90
|
+
# Divide Common characters by total characters
|
|
91
|
+
charpercent = commoncharlen / charlen
|
|
92
|
+
print(f"Percent of characters that are common {charpercent}")
|
|
93
|
+
|
|
94
|
+
# Compare the values in word lists
|
|
95
|
+
#commonword_l = list(set(refword_l) & set(difword_l))
|
|
96
|
+
sordrefword_l = sorted(refword_l)
|
|
97
|
+
sortdifword_l = sorted(difword_l)
|
|
98
|
+
commonword_l = []
|
|
99
|
+
for i, (ref,dif) in enumerate(zip(sordrefword_l,sortdifword_l)):
|
|
100
|
+
if ref == dif:
|
|
101
|
+
print(f"refword is {ref}")
|
|
102
|
+
print(f"difword is {dif}")
|
|
103
|
+
commonword_l.append(ref)
|
|
104
|
+
commonword_l.append(dif)
|
|
105
|
+
print(f"Common words are {commonword_l}")
|
|
106
|
+
commonwordlen = len(commonword_l)
|
|
107
|
+
print(f"Total common words {commonwordlen}")
|
|
108
|
+
|
|
109
|
+
# Divide Common words by total words
|
|
110
|
+
wordpercent = commonwordlen / wordlen
|
|
111
|
+
print(f"Percent of words that are common {wordpercent}")
|
|
112
|
+
|
|
113
|
+
# Get the mean of common words and common characters
|
|
114
|
+
prob = (charpercent + wordpercent) / 2
|
|
115
|
+
|
|
116
|
+
# End
|
|
117
|
+
return prob
|
|
118
|
+
|
|
119
|
+
|
|
120
|
+
def pstr(reference,difference):
|
|
121
|
+
"""
|
|
122
|
+
Probability of two STRings matching (with no debug)... P STR
|
|
123
|
+
using the equation
|
|
124
|
+
((common words/total words)+(common characters/total characters))/2 or the mean of the input
|
|
125
|
+
|
|
126
|
+
"""
|
|
127
|
+
refword_l = []
|
|
128
|
+
difword_l = []
|
|
129
|
+
for sym in string.punctuation:
|
|
130
|
+
reference = reference.replace(sym,' ')
|
|
131
|
+
ref_replace = reference
|
|
132
|
+
refword_l = ref_replace.split()
|
|
133
|
+
for sym in string.punctuation:
|
|
134
|
+
difference = difference.replace(sym,' ')
|
|
135
|
+
dif_replace = difference
|
|
136
|
+
difword_l = dif_replace.split()
|
|
137
|
+
refchar_l = list(ref_replace.lower().replace(' ',''))
|
|
138
|
+
difchar_l = list(dif_replace.lower().replace(' ',''))
|
|
139
|
+
charlen = len(refchar_l) + len(difchar_l)
|
|
140
|
+
wordlen = len(refword_l) + len(difword_l)
|
|
141
|
+
sortrefchar_l = sorted(refchar_l)
|
|
142
|
+
sortdifchar_l = sorted(difchar_l)
|
|
143
|
+
commonchar_l = []
|
|
144
|
+
errorcount = 0
|
|
145
|
+
for i in range(charlen):
|
|
146
|
+
for refletter in sortrefchar_l:
|
|
147
|
+
for difletter in sortdifchar_l:
|
|
148
|
+
if refletter == difletter:
|
|
149
|
+
commonchar_l.append(refletter)
|
|
150
|
+
commonchar_l.append(difletter)
|
|
151
|
+
sortrefchar_l.remove(refletter)
|
|
152
|
+
sortdifchar_l.remove(difletter)
|
|
153
|
+
else:
|
|
154
|
+
errorcount += 1
|
|
155
|
+
commoncharlen = len(commonchar_l)
|
|
156
|
+
charpercent = commoncharlen / charlen
|
|
157
|
+
sordrefword_l = sorted(refword_l)
|
|
158
|
+
sortdifword_l = sorted(difword_l)
|
|
159
|
+
commonword_l = []
|
|
160
|
+
for i, (ref,dif) in enumerate(zip(sordrefword_l,sortdifword_l)):
|
|
161
|
+
if ref == dif:
|
|
162
|
+
commonword_l.append(ref)
|
|
163
|
+
commonword_l.append(dif)
|
|
164
|
+
commonwordlen = len(commonword_l)
|
|
165
|
+
wordpercent = commonwordlen / wordlen
|
|
166
|
+
prob = (charpercent + wordpercent) / 2
|
|
167
|
+
return prob
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+
def main():
|
|
171
|
+
"""
|
|
172
|
+
Argparse added for functions to allow running from the terminal easily
|
|
173
|
+
Import note if running from terminal:
|
|
174
|
+
I added argparse stuff for both functions:
|
|
175
|
+
pstr "probability of strings" and pstrd "probability of strings with debug."
|
|
176
|
+
To execute (in case I forget later), use /path/to/file.py functionname param1 param2
|
|
177
|
+
i.e. ~/Documents/pythonscripts/pstr.py pstrd "A car" "A cat"
|
|
178
|
+
Hypothetically, one can add the file to /bin/pstr without the file extension too. As long as
|
|
179
|
+
it has +x permissions.
|
|
180
|
+
"""
|
|
181
|
+
parser = argparse.ArgumentParser(description="probability of string match")
|
|
182
|
+
subparsers = parser.add_subparsers(dest="command", required=True)
|
|
183
|
+
|
|
184
|
+
# pstr parameters
|
|
185
|
+
pstr_parser = subparsers.add_parser("pstr", help="probability of string match")
|
|
186
|
+
pstr_parser.add_argument("ref", type=str, help="reference string")
|
|
187
|
+
pstr_parser.add_argument("dif", type=str, help="difference string")
|
|
188
|
+
|
|
189
|
+
# pstrd parameters
|
|
190
|
+
pstrd_parser = subparsers.add_parser("pstrd", help="probability of string match with debug")
|
|
191
|
+
pstrd_parser.add_argument("ref", type=str, help="reference string")
|
|
192
|
+
pstrd_parser.add_argument("dif", type=str, help="difference string")
|
|
193
|
+
|
|
194
|
+
# Do the thing
|
|
195
|
+
doit = parser.parse_args()
|
|
196
|
+
if doit.command == "pstr":
|
|
197
|
+
print(pstr(doit.ref, doit.dif))
|
|
198
|
+
if doit.command == "pstrd":
|
|
199
|
+
print(pstrd(doit.ref, doit.dif))
|
|
200
|
+
|
|
201
|
+
if __name__ == "__main__":
|
|
202
|
+
main()
|
|
203
|
+
|
|
204
|
+
# fin
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: strprob
|
|
3
|
+
Version: 1.0.2
|
|
4
|
+
Summary: A string comparison tool
|
|
5
|
+
Home-page: https://github.com/spmather/pstr
|
|
6
|
+
Author-email: spmather <spm-pypi-issues@pm.me>
|
|
7
|
+
License-Expression: MIT
|
|
8
|
+
Project-URL: Homepage, https://github.com/spmather/pstr
|
|
9
|
+
Project-URL: Issues, https://github.com/spmather/pstr/issues
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: string
|
|
16
|
+
Requires-Dist: argparse
|
|
17
|
+
Dynamic: license-file
|
|
18
|
+
|
|
19
|
+
# Pstr "strprob"
|
|
20
|
+
|
|
21
|
+
Pstr or strprob is a small fuzzy string comparison tool in python for comparing two strings and how closely they match using the equation: `( (common words/total words) + (common characters/total characters) ) / (the mean of the input)`. The output is an number between 0 and 1, including float values.
|
|
22
|
+
|
|
23
|
+
# When to use
|
|
24
|
+
|
|
25
|
+
There are a few times when a regex is not powerful enough without significant modification to compare strings for matching. I created this fuzzy string to accomodate specific needs I had.
|
|
26
|
+
|
|
27
|
+
# Math
|
|
28
|
+
|
|
29
|
+
The equation that is calculated is a probability range from 0 to 1 based on how closely the strings match characters and words.
|
|
30
|
+
|
|
31
|
+
##### Working through the math
|
|
32
|
+
|
|
33
|
+
A single word comparison: `Car` and `Cat`
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
# Comparing Characters
|
|
37
|
+
C = C ... 100%
|
|
38
|
+
A = A ... 100%
|
|
39
|
+
R != T ... 0%
|
|
40
|
+
# Matches/Total = 4/6
|
|
41
|
+
# Result: 0.66 match
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
# Comparing words
|
|
45
|
+
Cat != Car ... 0%
|
|
46
|
+
# Matches/Total = 0/2
|
|
47
|
+
# Result: 0.00 match
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
# Mean of characters and words
|
|
51
|
+
0.66 + 0.00 / 2 = 0.33 or 33% match
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
A multiple word comparison: `Hello my new friend` and `Hello my old friend`
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
# Comparing Characters
|
|
58
|
+
H = H ... 100%
|
|
59
|
+
E = E ... 100%
|
|
60
|
+
L = L ... 100%
|
|
61
|
+
L = L ... 100%
|
|
62
|
+
O = O ... 100%
|
|
63
|
+
M = M ... 100%
|
|
64
|
+
Y = Y ... 100%
|
|
65
|
+
N != O ... 0%
|
|
66
|
+
E != L ... 0%
|
|
67
|
+
W != D ... 0%
|
|
68
|
+
F = F ... 100%
|
|
69
|
+
R = R ... 100%
|
|
70
|
+
I = I ... 100%
|
|
71
|
+
E = E ... 100%
|
|
72
|
+
N = N ... 100%
|
|
73
|
+
D = D ... 100%
|
|
74
|
+
# Matches/Total = 26/32
|
|
75
|
+
# Result: 0.8125 match
|
|
76
|
+
|
|
77
|
+
|
|
78
|
+
# Comparing words
|
|
79
|
+
Hello = Hello ... 100%
|
|
80
|
+
My = My ... 100%
|
|
81
|
+
New != Old ... 0%
|
|
82
|
+
Friend = Friend ... 100%
|
|
83
|
+
# Matches/Total = 6/8
|
|
84
|
+
# Result: 0.75 match
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
# Mean of characters and words
|
|
88
|
+
(0.8125 + 0.75) / 2 = 0.78125 or 78.125% match
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
##### Accuracy
|
|
94
|
+
|
|
95
|
+
I have used this over several workloads and typically, I will put the accuracy threshhold at `0.859` for important work. I.E.
|
|
96
|
+
|
|
97
|
+
```powershell
|
|
98
|
+
if ($pstrValue -gt 0.859) {
|
|
99
|
+
# do your command
|
|
100
|
+
}
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
I also find it to have excellent use at the end of any `if` chains/statements involving matching a string to regex or any string-related validation:
|
|
104
|
+
|
|
105
|
+
```powershell
|
|
106
|
+
if ($something -match $regex) {
|
|
107
|
+
# do your command
|
|
108
|
+
}
|
|
109
|
+
elseif ($pstrValue -gt 0.859) {
|
|
110
|
+
# do your command
|
|
111
|
+
}
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
##### In summary
|
|
115
|
+
|
|
116
|
+
1. Above `0.949` = Too high for most uses
|
|
117
|
+
|
|
118
|
+
2. `0.859` = Highly accurate
|
|
119
|
+
|
|
120
|
+
3. `0.799` = Pretty good accurate
|
|
121
|
+
|
|
122
|
+
4. `0.649` = Absolute minimum for most uses
|
|
123
|
+
|
|
124
|
+
5. Below `0.549` = Patently inaccurate
|
|
125
|
+
|
|
126
|
+
##### Windows
|
|
127
|
+
|
|
128
|
+
```powershell
|
|
129
|
+
pip install pstr
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
##### Linux
|
|
135
|
+
|
|
136
|
+
If the package is not added to your distro's repo, then you need to create a venv like this:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
# Makes a directory at a path. Change
|
|
140
|
+
# "/path/to/new/virtual/environment"
|
|
141
|
+
# to an actual path in your file system
|
|
142
|
+
mkdir /path/to/new/virtual/environment
|
|
143
|
+
|
|
144
|
+
# Create the virtual environment
|
|
145
|
+
# Some distros do not come packaged with venv
|
|
146
|
+
# follow the install prompts to add python3-venv
|
|
147
|
+
# if applicable
|
|
148
|
+
python -m venv /path/to/new/virtual/environment
|
|
149
|
+
|
|
150
|
+
# Use the new venv
|
|
151
|
+
source /path/to/new/virtual/environment/bin/activate
|
|
152
|
+
|
|
153
|
+
# Install the pstr package
|
|
154
|
+
pip install pstr
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
|
|
159
|
+
##### Next use the following syntax
|
|
160
|
+
|
|
161
|
+
From PowerShell or Bash
|
|
162
|
+
|
|
163
|
+
1. Example 1: Comparing the similarity of `"string 1 here"` and `"string 2 here"`
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
./pstr.py pstr "string 1 here" "string 2 here"
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
2. Example 2: Comparing the similarity of `"string 1 here"` and `"string 2 here"` but with debug
|
|
170
|
+
|
|
171
|
+
|
|
172
|
+
|
|
173
|
+
```
|
|
174
|
+
./pstr.py pstrd "string 1 here" "string 2 here"
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
### Future plans
|
|
178
|
+
1. add case insensitive options
|
|
179
|
+
2. fix the brute force recursion in some cases where 1,000 iterations doesn't remove items from the character lists
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
setup.cfg
|
|
5
|
+
src/pstr/__init__.py
|
|
6
|
+
src/pstr/strprob.py
|
|
7
|
+
src/strprob.egg-info/PKG-INFO
|
|
8
|
+
src/strprob.egg-info/SOURCES.txt
|
|
9
|
+
src/strprob.egg-info/dependency_links.txt
|
|
10
|
+
src/strprob.egg-info/requires.txt
|
|
11
|
+
src/strprob.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
pstr
|