w4b-file 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: da5651afceb70d38d2d4eb87e320c251bbe6712400ac53ca83d85bf06078a647
+  data.tar.gz: 308135bd32cef25667211de403f5a180beb3276e61411c22cac32bd7b9c3971f
+SHA512:
+  metadata.gz: 5b4d6a968375942d59534e2bcff56f4a1fa5b017ea52fd147904e1c1b6026968ef5143ccd9889448b4fbda702fbfd7f2aa66b9e19ad7ed72db987048f3a70ddc
+  data.tar.gz: 74ce8f7d1e46055264ee8c4bab03a7f75f1c11dec615eddd4cabacd769623101d37354cc18b7496c53392a4d7d61063feae63f5b6c3229ed46dc04f895fa4873

data/README.md

@@ -0,0 +1,86 @@
+# WebFile Analyzer (w4b-file)
+
+WebFile Analyzer is an advanced command-line utility meticulously crafted for web developers, security professionals, and researchers. This tool provides a comprehensive solution to efficiently scan websites for diverse file types, elevating the process of file discovery, security analysis, and content audits.
+
+## Installation
+
+```bash
+gem install w4b-file
+```
+
+Execute WebFile Analyzer:
+
+```
+w4b-file
+```
+
+
+### Options:
+
+`<website_url>   The URL of the website to scan for files.`
+
+`<website_url> <file_type>   Specify the type of files to check (pdf, image, video, php, zip, doc)`
+
+### Examples:
+
+```
+w4b-file https://example-url.co.vias --check pdf       # Check all PDF files on the website.
+```
+  
+```
+w4b-file https://example-url.co.vias --check image     # Check all image files on the website.
+```
+  
+```
+w4b-file https://example-url.co.vias --check zip       # Check all ZIP files on the website.
+```
+
+```
+w4b-file https://example-url.co.vias --check document  # Check all document files on the website.
+```
+
+```
+w4b-file https://example-url.co.vias --check videos    # Check Video files on the website.
+```
+
+## Key Features
+
+- **File Type Identification**: Swiftly pinpoint various file types, including images, videos, PDFs, documents, and more.
+
+- **Link Verification**: Identify broken links and missing resources, ensuring a seamless user experience for website administrators.
+
+- **Concealed File Detection**: Unearth hidden files that may contain sensitive information or indicate potential security vulnerabilities.
+
+- **Automated Insights**: Seamlessly integrate WebFile Analyzer into automated testing workflows for continual file integrity checks and content audits.
+
+## Advantages
+
+- **Website Maintenance**: Regular scans with WebFile Analyzer contribute to maintaining a polished and user-friendly online presence by identifying missing or outdated files.
+
+- **Content Strategy Optimization**: Evaluate your content strategy by analyzing the distribution of various file types, enabling informed decisions about file prioritization.
+
+- **Security Enhancement**: Identify hidden files that could pose security risks or inadvertently expose sensitive information.
+
+- **Competitor Analysis**: Study file types on competitor websites to gain insights into their content focus and potentially identify gaps in your own strategy.
+
+- **Research and Analysis**: WebFile Analyzer extends beyond your websites. Utilize it for research by analyzing trends in file types across multiple websites.
+
+## Security Details
+
+- **HTTPS Support**: WebFile Analyzer supports scanning websites with secure HTTPS connections for a comprehensive analysis.
+
+- **Security Protocol Compliance**: The tool adheres to security protocols, ensuring reliable scans while respecting website security measures.
+
+- **Robust File Encryption Handling**: WebFile Analyzer appropriately handles encrypted files, maintaining the integrity of the scanning process.
+
+## Disadvantages
+
+- **Limited Scanning on Highly Secure Websites**: Some highly secure websites may restrict access, preventing complete scans of all files and resources.
+
+- **Image and File Encryption Challenges**: Encrypted images and files may pose challenges in terms of analysis and may not be fully accessible.
+
+- **Dependency on Website Permissions**: The effectiveness of the tool is subject to the permissions granted by the website, impacting the extent of the scan.
+
+```
+WebFile Analyzer (w4b-file) is a tool designed for educational purposes, catering to web developers, security professionals, and researchers. It is essential to emphasize responsible use, and the developers are not liable for any misuse or issues that may arise. This tool is intended for educational exploration, promoting a better understanding of website file scanning without endorsing any unauthorized activities. Users are urged to exercise caution, adhere to ethical guidelines, and respect legal boundaries when utilizing this tool. The developers disclaim responsibility for any unintended consequences resulting from the tool's usage.
+```

data/bin/w4b-file

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require_relative '../lib/w4b-file/cli'
+
+W4bFile::CLI.start

data/lib/w4b-file/cli.rb

@@ -0,0 +1,90 @@
+require 'optparse'
+require_relative 'scanner'
+
+module W4bFile
+  class CLI
+    def self.start
+      options = {}
+      OptionParser.new do |opts|
+        opts.banner = "Usage: w4b-file <website_url> [--check <file_type>]"
+
+        opts.on("--check TYPE", "Specify the type of files to check") do |type|
+          options[:check_type] = type
+        end
+
+        opts.on("--check all", "Check and display all files on the website") do
+          options[:check_all] = true
+        end
+
+        opts.on("--check hidden", "Scan for hidden files on the website") do
+          options[:check_hidden] = true
+        end
+      end.parse!
+
+      website_url = ARGV[0]
+      check_type = options[:check_type]
+
+      if website_url.nil?
+        puts "Usage: w4b-file <website_url> [--check <file_type>]"
+        exit 1
+      end
+
+      scanner = Scanner.new(website_url)
+
+      if options[:check_hidden]
+        hidden_files = scanner.scan_hidden_files
+        display_files(hidden_files, "Hidden", website_url) unless hidden_files.empty?
+      elsif options[:check_all]
+        resources = scanner.scan_website
+        display_files(resources, "All", website_url) unless resources.empty?
+      elsif check_type
+        run_checks(check_type, scanner, website_url)
+      else
+        puts "Invalid option. Please specify a valid option."
+        exit 1
+      end
+    end
+
+    def self.run_checks(check_type, scanner, website_url)
+      case check_type
+      when "videos"
+        files_found = check_files("mp4|avi", scanner, website_url)
+        display_files(files_found, "Video", website_url) if !files_found.empty?
+      when "images"
+        files_found = check_files("jpg|jpeg|png", scanner, website_url)
+        display_files(files_found, "Image", website_url) if !files_found.empty?
+      when "zip"
+        files_found = check_files("zip", scanner, website_url)
+        display_files(files_found, "ZIP", website_url) if !files_found.empty?
+      when "pdf"
+        files_found = check_files("pdf", scanner, website_url)
+        display_files(files_found, "PDF", website_url) if !files_found.empty?
+      when "document"
+        files_found = check_files("doc|docx|txt", scanner, website_url)
+        display_files(files_found, "Document", website_url) if !files_found.empty?
+      else
+        puts "Invalid check type. Available options: videos, images, zip, pdf, document, --check hidden"
+        exit 1
+      end
+    end
+
+    def self.check_files(type, scanner, url)
+      resources = scanner.scan_website
+      files = resources.grep(/\.(#{type})$/i)
+      files
+    end
+
+    def self.display_files(files, type, base_url)
+      return if files.empty?
+
+      puts "[+] #{type} files :"
+      files.each do |file|
+        display_url = URI.join(base_url, file).to_s
+        puts "    Link : #{display_url}"
+      end
+      puts
+    end
+  end
+end
+
+W4bFile::CLI.start if __FILE__ == $PROGRAM_NAME

data/lib/w4b-file/scanner.rb

@@ -0,0 +1,40 @@
+require 'uri'
+require 'net/http'
+require 'openssl'
+
+module W4bFile
+  class Scanner
+    def initialize(url)
+      @url = url
+    end
+
+    def scan_website
+      uri = URI.parse(@url)
+      response = Net::HTTP.get_response(uri)
+
+      return [] unless response.is_a?(Net::HTTPSuccess)
+
+      response.body.scan(/href="([^"#]*)"|\ssrc="([^"#]*)"/).flatten.compact
+    end
+
+    def scan_hidden_files
+      uri = URI.parse(@url)
+      http = Net::HTTP.new(uri.host, uri.port)
+      http.use_ssl = (uri.scheme == 'https')
+      http.verify_mode = OpenSSL::SSL::VERIFY_PEER
+
+      request = Net::HTTP::Get.new(uri)
+      request['User-Agent'] = 'Googlebot'
+      response = http.request(request)
+
+      hidden_files = []
+      if response.is_a?(Net::HTTPSuccess)
+        response.body.scan(/href="([^"#]*)"/).flatten.compact.each do |file|
+          hidden_files << file if file.start_with?(".") || file.include?("/.")
+        end
+      end
+
+      hidden_files
+    end
+  end
+end

data/lib/w4b-file/version.rb

@@ -0,0 +1,3 @@
+module W4bFile
+  VERSION = "1.0.0"
+end

data/w4b-file.gemspec

@@ -0,0 +1,20 @@
+# coding: utf-8
+lib = File.expand_path('lib', __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+Gem::Specification.new do |spec|
+  spec.name          = 'w4b-file'
+  spec.version       = '1.0.0'
+  spec.authors       = ['MrFidal']
+  spec.license       = 'MIT'  # 
+  spec.add_runtime_dependency 'openssl', '~> 2.3', '>= 2.3.0'
+  spec.required_ruby_version = '>= 3.0'
+  spec.email         = ['mrfidal@proton.me']
+  spec.summary       = 'Scans websites for files.'
+  spec.description   = 'A Ruby gem to scan websites for specific file types and hidden files.'
+  spec.homepage      = 'https://github.com/Tony-Linux/w4b-file'
+  spec.files         = Dir["{bin,lib}/**/*", "README.md", "w4b-file.gemspec"]
+  spec.executables   = ['w4b-file']
+  spec.require_paths = ['lib']
+  spec.add_runtime_dependency 'net-http', '~> 0.2.1'
+  spec.add_development_dependency 'rspec', '~> 3.0'
+end

metadata

@@ -0,0 +1,98 @@
+--- !ruby/object:Gem::Specification
+name: w4b-file
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- MrFidal
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2024-07-10 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: openssl
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.3'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.3.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.3'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 2.3.0
+- !ruby/object:Gem::Dependency
+  name: net-http
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.2.1
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+description: A Ruby gem to scan websites for specific file types and hidden files.
+email:
+- mrfidal@proton.me
+executables:
+- w4b-file
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- bin/w4b-file
+- lib/w4b-file/cli.rb
+- lib/w4b-file/scanner.rb
+- lib/w4b-file/version.rb
+- w4b-file.gemspec
+homepage: https://github.com/Tony-Linux/w4b-file
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.9
+signing_key:
+specification_version: 4
+summary: Scans websites for files.
+test_files: []