waymore 7.1__tar.gz → 7.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: waymore
3
- Version: 7.1
3
+ Version: 7.4
4
4
  Summary: Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal & Intelligence X!
5
5
  Home-page: https://github.com/xnl-h4ck3r/waymore
6
6
  Author: xnl-h4ck3r
@@ -21,7 +21,7 @@ Dynamic: license-file
21
21
 
22
22
  <center><img src="https://github.com/xnl-h4ck3r/waymore/blob/main/waymore/images/title.png"></center>
23
23
 
24
- ## About - v7.1
24
+ ## About - v7.4
25
25
 
26
26
  The idea behind **waymore** is to find even more links from the Wayback Machine than other existing tools.
27
27
 
@@ -120,6 +120,7 @@ pipx install git+https://github.com/xnl-h4ck3r/waymore.git
120
120
  | -urlr | --urlscan-rate-limit-retry | The number of minutes the user wants to wait for a rate limit pause on URLScan.io instead of stopping with a `429` error (default: 1). |
121
121
  | -co | --check-only | This will make a few minimal requests to show you how many requests, and roughly how long it could take, to get URLs from the sources and downloaded responses from Wayback Machine (unfortunately it isn't possible to check how long it will take to download responses from URLScan). |
122
122
  | -nd | --notify-discord | Whether to send a notification to Discord when waymore completes. It requires `WEBHOOK_DISCORD` to be provided in the `config.yml` file. |
123
+ | -nt | --notify-telegram | Whether to send a notification to Telegram when waymore completes. It requires `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` to be provided in the `config.yml` file. |
123
124
  | -oijs | --output-inline-js | Whether to save combined inline javascript of all relevant files in the response directory when `-mode R` (or `-mode B`) has been used. The files are saved with the name `combinedInline{}.js` where `{}` is the number of the file, saving 1000 unique scripts per file. The file `combinedInlineSrc.txt` will also be created, containing the `src` value of all external scripts referenced in the files. |
124
125
  | -v | --verbose | Verbose output |
125
126
  | | --version | Show current version number. |
@@ -170,7 +171,9 @@ The `config.yml` file (typically in `~/.config/waymore/`) have values that can b
170
171
  - `FILTER_KEYWORDS` - Only links and responses will be returned that contain the specified keywords if the `-ko`/`--keywords-only` argument is passed (without providing an explicit value on the command line), e.g. `admin,portal`
171
172
  - `URLSCAN_API_KEY` - You can sign up to [urlscan.io](https://urlscan.io/user/signup) to get a **FREE** API key (there are also paid subscriptions available). It is recommended you get a key and put it into the config file so that you can get more back (and quicker) from their API. NOTE: You will get rate limited unless you have a full paid subscription.
172
173
  - `CONTINUE_RESPONSES_IF_PIPED` - If retrieving archive responses doesn't complete, you will be prompted next time whether you want to continue with the previous run. However, if `stdout` is piped to another process it is assumed you don't want to have an interactive prompt. A value of `True` (default) will determine assure the previous run will be continued. if you want a fresh run every time then set to `False`.
173
- - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook when a successful XSS is found.
174
+ - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook.
175
+ - `TELEGRAM_BOT_TOKEN` - If the `--notify-telegram` argument is passed, `waymore` will use this token to send a notification to Telegram.
176
+ - `TELEGRAM_CHAT_ID` - If the `--notify-telegram` argument is passed, `waymore` will send the notification to this chat ID.
174
177
  - `DEFAULT_OUTPUT_DIR` - This is the default location of any output files written if the `-oU` and `-oR` arguments are not used. If the value of this key is blank, then it will default to the location of the `config.yml` file.
175
178
  - `INTELX_API_KEY` - You can sign up to [intelx.io here](https://intelx.io/product). It requires a paid API key to do the `/phonebook/search` through their API (as of 2024-09-01, the Phonebook service has been restricted to paid users due to constant abuse by spam accounts).
176
179
 
@@ -1,6 +1,6 @@
1
1
  <center><img src="https://github.com/xnl-h4ck3r/waymore/blob/main/waymore/images/title.png"></center>
2
2
 
3
- ## About - v7.1
3
+ ## About - v7.4
4
4
 
5
5
  The idea behind **waymore** is to find even more links from the Wayback Machine than other existing tools.
6
6
 
@@ -99,6 +99,7 @@ pipx install git+https://github.com/xnl-h4ck3r/waymore.git
99
99
  | -urlr | --urlscan-rate-limit-retry | The number of minutes the user wants to wait for a rate limit pause on URLScan.io instead of stopping with a `429` error (default: 1). |
100
100
  | -co | --check-only | This will make a few minimal requests to show you how many requests, and roughly how long it could take, to get URLs from the sources and downloaded responses from Wayback Machine (unfortunately it isn't possible to check how long it will take to download responses from URLScan). |
101
101
  | -nd | --notify-discord | Whether to send a notification to Discord when waymore completes. It requires `WEBHOOK_DISCORD` to be provided in the `config.yml` file. |
102
+ | -nt | --notify-telegram | Whether to send a notification to Telegram when waymore completes. It requires `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` to be provided in the `config.yml` file. |
102
103
  | -oijs | --output-inline-js | Whether to save combined inline javascript of all relevant files in the response directory when `-mode R` (or `-mode B`) has been used. The files are saved with the name `combinedInline{}.js` where `{}` is the number of the file, saving 1000 unique scripts per file. The file `combinedInlineSrc.txt` will also be created, containing the `src` value of all external scripts referenced in the files. |
103
104
  | -v | --verbose | Verbose output |
104
105
  | | --version | Show current version number. |
@@ -149,7 +150,9 @@ The `config.yml` file (typically in `~/.config/waymore/`) have values that can b
149
150
  - `FILTER_KEYWORDS` - Only links and responses will be returned that contain the specified keywords if the `-ko`/`--keywords-only` argument is passed (without providing an explicit value on the command line), e.g. `admin,portal`
150
151
  - `URLSCAN_API_KEY` - You can sign up to [urlscan.io](https://urlscan.io/user/signup) to get a **FREE** API key (there are also paid subscriptions available). It is recommended you get a key and put it into the config file so that you can get more back (and quicker) from their API. NOTE: You will get rate limited unless you have a full paid subscription.
151
152
  - `CONTINUE_RESPONSES_IF_PIPED` - If retrieving archive responses doesn't complete, you will be prompted next time whether you want to continue with the previous run. However, if `stdout` is piped to another process it is assumed you don't want to have an interactive prompt. A value of `True` (default) will determine assure the previous run will be continued. if you want a fresh run every time then set to `False`.
152
- - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook when a successful XSS is found.
153
+ - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook.
154
+ - `TELEGRAM_BOT_TOKEN` - If the `--notify-telegram` argument is passed, `waymore` will use this token to send a notification to Telegram.
155
+ - `TELEGRAM_CHAT_ID` - If the `--notify-telegram` argument is passed, `waymore` will send the notification to this chat ID.
153
156
  - `DEFAULT_OUTPUT_DIR` - This is the default location of any output files written if the `-oU` and `-oR` arguments are not used. If the value of this key is blank, then it will default to the location of the `config.yml` file.
154
157
  - `INTELX_API_KEY` - You can sign up to [intelx.io here](https://intelx.io/product). It requires a paid API key to do the `/phonebook/search` through their API (as of 2024-09-01, the Phonebook service has been restricted to paid users due to constant abuse by spam accounts).
155
158
 
@@ -0,0 +1 @@
1
+ __version__ = "7.4"
@@ -109,6 +109,11 @@ linkCountAlienVault = 0
109
109
  linkCountURLScan = 0
110
110
  linkCountVirusTotal = 0
111
111
  linkCountIntelx = 0
112
+ linksFoundCommonCrawl = set()
113
+ linksFoundAlienVault = set()
114
+ linksFoundURLScan = set()
115
+ linksFoundVirusTotal = set()
116
+ linksFoundIntelx = set()
112
117
 
113
118
  # Thread lock for protecting shared state during concurrent operations
114
119
  links_lock = threading.Lock()
@@ -182,6 +187,8 @@ FILTER_KEYWORDS = ""
182
187
  URLSCAN_API_KEY = ""
183
188
  CONTINUE_RESPONSES_IF_PIPED = True
184
189
  WEBHOOK_DISCORD = ""
190
+ TELEGRAM_BOT_TOKEN = ""
191
+ TELEGRAM_CHAT_ID = ""
185
192
  DEFAULT_OUTPUT_DIR = ""
186
193
  INTELX_API_KEY = ""
187
194
 
@@ -792,6 +799,24 @@ def showOptions():
792
799
  else:
793
800
  write(colored("Discord Webhook: ", "magenta") + colored(WEBHOOK_DISCORD))
794
801
 
802
+ if args.notify_telegram:
803
+ if (
804
+ TELEGRAM_BOT_TOKEN == ""
805
+ or TELEGRAM_BOT_TOKEN == "YOUR_TOKEN"
806
+ or TELEGRAM_CHAT_ID == ""
807
+ or TELEGRAM_CHAT_ID == "YOUR_CHAT_ID"
808
+ ):
809
+ write(
810
+ colored("Telegram: ", "magenta")
811
+ + colored(
812
+ "It looks like Telegram Bot Token or Chat ID has not been set in config.yml file.",
813
+ "red",
814
+ )
815
+ )
816
+ else:
817
+ write(colored("Telegram Bot Token: ", "magenta") + colored(TELEGRAM_BOT_TOKEN))
818
+ write(colored("Telegram Chat ID: ", "magenta") + colored(TELEGRAM_CHAT_ID))
819
+
795
820
  write(colored("Default Output Directory: ", "magenta") + colored(str(DEFAULT_OUTPUT_DIR)))
796
821
 
797
822
  if args.regex_after is not None:
@@ -850,7 +875,7 @@ def getConfig():
850
875
  """
851
876
  Try to get the values from the config file, otherwise use the defaults
852
877
  """
853
- global FILTER_CODE, FILTER_MIME, FILTER_URL, FILTER_KEYWORDS, URLSCAN_API_KEY, VIRUSTOTAL_API_KEY, CONTINUE_RESPONSES_IF_PIPED, subs, path, waymorePath, inputIsDomainANDPath, HTTP_ADAPTER, HTTP_ADAPTER_CC, argsInput, terminalWidth, MATCH_CODE, WEBHOOK_DISCORD, DEFAULT_OUTPUT_DIR, MATCH_MIME, INTELX_API_KEY
878
+ global FILTER_CODE, FILTER_MIME, FILTER_URL, FILTER_KEYWORDS, URLSCAN_API_KEY, VIRUSTOTAL_API_KEY, CONTINUE_RESPONSES_IF_PIPED, subs, path, waymorePath, inputIsDomainANDPath, HTTP_ADAPTER, HTTP_ADAPTER_CC, argsInput, terminalWidth, MATCH_CODE, WEBHOOK_DISCORD, TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, DEFAULT_OUTPUT_DIR, MATCH_MIME, INTELX_API_KEY
854
879
  try:
855
880
 
856
881
  # Set terminal width
@@ -1115,6 +1140,45 @@ def getConfig():
1115
1140
  )
1116
1141
  WEBHOOK_DISCORD = ""
1117
1142
 
1143
+ if args.notify_telegram:
1144
+ try:
1145
+ TELEGRAM_BOT_TOKEN = config.get("TELEGRAM_BOT_TOKEN")
1146
+ if str(TELEGRAM_BOT_TOKEN) == "None" or str(TELEGRAM_BOT_TOKEN) == "YOUR_TOKEN":
1147
+ writerr(
1148
+ colored(
1149
+ 'No value for "TELEGRAM_BOT_TOKEN" in config.yml - default set',
1150
+ "yellow",
1151
+ )
1152
+ )
1153
+ TELEGRAM_BOT_TOKEN = ""
1154
+ except Exception:
1155
+ writerr(
1156
+ colored(
1157
+ 'Unable to read "TELEGRAM_BOT_TOKEN" from config.yml - default set',
1158
+ "red",
1159
+ )
1160
+ )
1161
+ TELEGRAM_BOT_TOKEN = ""
1162
+
1163
+ try:
1164
+ TELEGRAM_CHAT_ID = config.get("TELEGRAM_CHAT_ID")
1165
+ if str(TELEGRAM_CHAT_ID) == "None" or str(TELEGRAM_CHAT_ID) == "YOUR_CHAT_ID":
1166
+ writerr(
1167
+ colored(
1168
+ 'No value for "TELEGRAM_CHAT_ID" in config.yml - default set',
1169
+ "yellow",
1170
+ )
1171
+ )
1172
+ TELEGRAM_CHAT_ID = ""
1173
+ except Exception:
1174
+ writerr(
1175
+ colored(
1176
+ 'Unable to read "TELEGRAM_CHAT_ID" from config.yml - default set',
1177
+ "red",
1178
+ )
1179
+ )
1180
+ TELEGRAM_CHAT_ID = ""
1181
+
1118
1182
  try:
1119
1183
  DEFAULT_OUTPUT_DIR = config.get("DEFAULT_OUTPUT_DIR")
1120
1184
  if str(DEFAULT_OUTPUT_DIR) == "None" or str(DEFAULT_OUTPUT_DIR) == "":
@@ -1214,6 +1278,8 @@ def getConfig():
1214
1278
  FILTER_KEYWORDS = ""
1215
1279
  CONTINUE_RESPONSES_IF_PIPED = True
1216
1280
  WEBHOOK_DISCORD = ""
1281
+ TELEGRAM_BOT_TOKEN = ""
1282
+ TELEGRAM_CHAT_ID = ""
1217
1283
  DEFAULT_OUTPUT_DIR = os.path.expanduser("~/.config/waymore")
1218
1284
 
1219
1285
  except Exception as e:
@@ -4732,7 +4798,7 @@ def getIntelxUrls():
4732
4798
  """
4733
4799
  Get URLs from the Intelligence X Phonebook search
4734
4800
  """
4735
- global INTELX_API_KEY, linksFound, waymorePath, subs, stopProgram, stopSourceIntelx, argsInput, checkIntelx, argsInputHostname, intelxAPIIssue, linkCountIntelx
4801
+ global INTELX_API_KEY, linksFound, waymorePath, subs, stopProgram, stopSourceIntelx, argsInput, checkIntelx, argsInputHostname, intelxAPIIssue, linkCountIntelx, linksFoundIntelx
4736
4802
 
4737
4803
  # Write the file of URL's for the passed domain/URL
4738
4804
  try:
@@ -5763,6 +5829,36 @@ def notifyDiscord():
5763
5829
  writerr(colored("ERROR notifyDiscord 1: " + str(e), "red"))
5764
5830
 
5765
5831
 
5832
+ def notifyTelegram():
5833
+ global TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, args
5834
+ try:
5835
+ url = "https://api.telegram.org/bot" + TELEGRAM_BOT_TOKEN + "/sendMessage"
5836
+ data = {
5837
+ "chat_id": TELEGRAM_CHAT_ID,
5838
+ "text": "waymore has finished for `-i " + args.input + " -mode " + args.mode + "` ! 🤘",
5839
+ }
5840
+ try:
5841
+ result = requests.post(url, json=data)
5842
+ if 300 <= result.status_code < 200:
5843
+ writerr(
5844
+ colored(
5845
+ getSPACER(
5846
+ "WARNING: Failed to send notification to Telegram - " + result.json()
5847
+ ),
5848
+ "yellow",
5849
+ )
5850
+ )
5851
+ except Exception as e:
5852
+ writerr(
5853
+ colored(
5854
+ getSPACER("WARNING: Failed to send notification to Telegram - " + str(e)),
5855
+ "yellow",
5856
+ )
5857
+ )
5858
+ except Exception as e:
5859
+ writerr(colored("ERROR notifyTelegram 1: " + str(e), "red"))
5860
+
5861
+
5766
5862
  def checkScript(script):
5767
5863
  try:
5768
5864
  if script.replace("\n", "").strip() != "":
@@ -6307,6 +6403,12 @@ def main():
6307
6403
  action="store_true",
6308
6404
  help="Whether to send a notification to Discord when waymore completes. It requires WEBHOOK_DISCORD to be provided in the config.yml file.",
6309
6405
  )
6406
+ parser.add_argument(
6407
+ "-nt",
6408
+ "--notify-telegram",
6409
+ action="store_true",
6410
+ help="Whether to send a notification to Telegram when waymore completes. It requires TELEGRAM_BOT_TOKEN and TELEGRAM_CHAT_ID to be provided in the config.yml file.",
6411
+ )
6310
6412
  parser.add_argument(
6311
6413
  "-oijs",
6312
6414
  "--output-inline-js",
@@ -6526,12 +6628,17 @@ def main():
6526
6628
  writerr(colored("ERROR main 1: " + str(e), "red"))
6527
6629
 
6528
6630
  finally:
6529
- # Send a notification to discord if requested
6631
+ # Send a notification to discord or telegram if requested
6530
6632
  try:
6531
6633
  if args.notify_discord and WEBHOOK_DISCORD != "":
6532
6634
  notifyDiscord()
6533
6635
  except Exception:
6534
6636
  pass
6637
+ try:
6638
+ if args.notify_telegram and TELEGRAM_BOT_TOKEN != "" and TELEGRAM_CHAT_ID != "":
6639
+ notifyTelegram()
6640
+ except Exception:
6641
+ pass
6535
6642
  try:
6536
6643
  if sys.stdout.isatty():
6537
6644
  writerr(
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: waymore
3
- Version: 7.1
3
+ Version: 7.4
4
4
  Summary: Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal & Intelligence X!
5
5
  Home-page: https://github.com/xnl-h4ck3r/waymore
6
6
  Author: xnl-h4ck3r
@@ -21,7 +21,7 @@ Dynamic: license-file
21
21
 
22
22
  <center><img src="https://github.com/xnl-h4ck3r/waymore/blob/main/waymore/images/title.png"></center>
23
23
 
24
- ## About - v7.1
24
+ ## About - v7.4
25
25
 
26
26
  The idea behind **waymore** is to find even more links from the Wayback Machine than other existing tools.
27
27
 
@@ -120,6 +120,7 @@ pipx install git+https://github.com/xnl-h4ck3r/waymore.git
120
120
  | -urlr | --urlscan-rate-limit-retry | The number of minutes the user wants to wait for a rate limit pause on URLScan.io instead of stopping with a `429` error (default: 1). |
121
121
  | -co | --check-only | This will make a few minimal requests to show you how many requests, and roughly how long it could take, to get URLs from the sources and downloaded responses from Wayback Machine (unfortunately it isn't possible to check how long it will take to download responses from URLScan). |
122
122
  | -nd | --notify-discord | Whether to send a notification to Discord when waymore completes. It requires `WEBHOOK_DISCORD` to be provided in the `config.yml` file. |
123
+ | -nt | --notify-telegram | Whether to send a notification to Telegram when waymore completes. It requires `TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` to be provided in the `config.yml` file. |
123
124
  | -oijs | --output-inline-js | Whether to save combined inline javascript of all relevant files in the response directory when `-mode R` (or `-mode B`) has been used. The files are saved with the name `combinedInline{}.js` where `{}` is the number of the file, saving 1000 unique scripts per file. The file `combinedInlineSrc.txt` will also be created, containing the `src` value of all external scripts referenced in the files. |
124
125
  | -v | --verbose | Verbose output |
125
126
  | | --version | Show current version number. |
@@ -170,7 +171,9 @@ The `config.yml` file (typically in `~/.config/waymore/`) have values that can b
170
171
  - `FILTER_KEYWORDS` - Only links and responses will be returned that contain the specified keywords if the `-ko`/`--keywords-only` argument is passed (without providing an explicit value on the command line), e.g. `admin,portal`
171
172
  - `URLSCAN_API_KEY` - You can sign up to [urlscan.io](https://urlscan.io/user/signup) to get a **FREE** API key (there are also paid subscriptions available). It is recommended you get a key and put it into the config file so that you can get more back (and quicker) from their API. NOTE: You will get rate limited unless you have a full paid subscription.
172
173
  - `CONTINUE_RESPONSES_IF_PIPED` - If retrieving archive responses doesn't complete, you will be prompted next time whether you want to continue with the previous run. However, if `stdout` is piped to another process it is assumed you don't want to have an interactive prompt. A value of `True` (default) will determine assure the previous run will be continued. if you want a fresh run every time then set to `False`.
173
- - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook when a successful XSS is found.
174
+ - `WEBHOOK_DISCORD` - If the `--notify-discord` argument is passed, `waymore` will send a notification to this Discord wehook.
175
+ - `TELEGRAM_BOT_TOKEN` - If the `--notify-telegram` argument is passed, `waymore` will use this token to send a notification to Telegram.
176
+ - `TELEGRAM_CHAT_ID` - If the `--notify-telegram` argument is passed, `waymore` will send the notification to this chat ID.
174
177
  - `DEFAULT_OUTPUT_DIR` - This is the default location of any output files written if the `-oU` and `-oR` arguments are not used. If the value of this key is blank, then it will default to the location of the `config.yml` file.
175
178
  - `INTELX_API_KEY` - You can sign up to [intelx.io here](https://intelx.io/product). It requires a paid API key to do the `/phonebook/search` through their API (as of 2024-09-01, the Phonebook service has been restricted to paid users due to constant abuse by spam accounts).
176
179
 
@@ -1 +0,0 @@
1
- __version__ = "7.1"
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes