legal_summariser 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 2a32e0da3e5422be003d79a333a6f3ea9417fadcc362164e3cef9cae0d84dafb
+  data.tar.gz: 3219d6167c936a2f056f43b5e2491bc4c67a697ef9d72169c7741be03f5a2726
+SHA512:
+  metadata.gz: 9481e9eb32d6770586b21f8c56ced7f37d99afe8c9ba162fd284cc086b8f02f71b042bef0200bd61104446c0309763da7c362a3e5abae202ccf295c04ef63281
+  data.tar.gz: c41d771b2ef842b185ebf0114de4921060ad6e55a17377acfc47412790237428fcab8a5ceff3efe2112b3341a9380a30160244c21b0b078eecc108181e9d4ce8

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/CHANGELOG.md

@@ -0,0 +1,46 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2024-09-09
+
+### Added
+- Initial release of Legal Summariser
+- Text extraction from PDF, DOCX, and TXT files
+- Basic legal document summarisation with plain English conversion
+- Clause detection for 8 key legal areas:
+  - Data Processing & Privacy
+  - Liability & Indemnification  
+  - Confidentiality & Non-disclosure
+  - Termination & Cancellation
+  - Payment & Fees
+  - Intellectual Property
+  - Dispute Resolution
+  - Governing Law
+- Risk analysis system with high/medium risk detection
+- Compliance gap identification for GDPR and KVKK
+- Unfair terms detection
+- Multiple output formats (JSON, Markdown, Plain Text)
+- Command-line interface with Thor
+- Comprehensive test suite with RSpec
+- Document type auto-detection
+- Offline processing capabilities
+
+### Features
+- Rule-based clause extraction using regex patterns
+- Smart sentence scoring for summarisation
+- Legal language simplification
+- Risk scoring algorithm
+- Compliance checking framework
+- Multi-format document support
+- CLI demo mode with sample NDA
+
+### Technical
+- Ruby gem structure with proper gemspec
+- Modular architecture with separate classes for each function
+- Error handling for unsupported formats and missing files
+- Text cleaning and normalization
+- Comprehensive documentation and examples

data/Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in legal_summariser.gemspec
+gemspec

data/README.md

@@ -0,0 +1,281 @@
+# Legal Summariser 📋⚖️
+
+> A Ruby-based AI-powered toolkit for legal document analysis that summarises contracts, extracts key clauses, flags risks, and translates legal jargon into plain English while preserving legal accuracy.
+
+**Created by [Ahmet KAHRAMAN](https://ahmetxhero.web.app)** 👨‍💻
+
+[![Ruby](https://img.shields.io/badge/Ruby-2.6+-red.svg)](https://ruby-lang.org)
+[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Tests](https://img.shields.io/badge/Tests-26%20passing-green.svg)](#testing)
+
+---
+
+## 👋 About the Author
+
+**Ahmet KAHRAMAN** - Mobile Developer & Cyber Security Expert
+
+- 🌐 **Portfolio**: [ahmetxhero.web.app](https://ahmetxhero.web.app)
+- 🎥 **YouTube**: [@ahmetxhero](https://youtube.com/@ahmetxhero)
+- 💼 **LinkedIn**: [linkedin.com/in/ahmetxhero](https://linkedin.com/in/ahmetxhero)
+- 🐤 **Twitter**: [@ahmetxhero](https://x.com/ahmetxhero)
+- 📧 **Email**: ahmetxhero@gmail.com
+- 🏠 **Location**: Ankara, Turkey 🇹🇷
+
+*"Security first, innovation always"* - Building secure, innovative solutions for a better digital future 🚀
+
+---
+
+## Features
+
+- **Document Processing**: Supports PDF, DOCX, and plain text files
+- **Smart Summarisation**: Converts legal documents into concise plain English
+- **Clause Detection**: Automatically identifies key legal clauses including:
+  - Data Processing & Privacy (GDPR/KVKK compliance)
+  - Liability & Indemnification
+  - Confidentiality & Non-disclosure
+  - Termination & Cancellation
+  - Payment & Fees
+  - Intellectual Property
+  - Dispute Resolution
+  - Governing Law
+- **Risk Analysis**: Flags potential legal risks and unfair terms
+- **Compliance Checking**: Identifies gaps in GDPR, KVKK, and other regulations
+- **Multiple Output Formats**: JSON, Markdown, and plain text
+- **CLI Interface**: Command-line tool for batch processing
+- **Offline Processing**: Works without internet for sensitive documents
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'legal_summariser'
+```
+
+And then execute:
+
+```bash
+bundle install
+```
+
+Or install it yourself as:
+
+```bash
+gem install legal_summariser
+```
+
+## Usage
+
+### Ruby API
+
+```ruby
+require "legal_summariser"
+
+# Basic usage
+summary = LegalSummariser.summarise("contracts/nda.pdf")
+puts summary[:plain_text]
+# => "This Non-Disclosure Agreement establishes confidentiality, valid for 2 years. The company may terminate at any time..."
+
+# With options
+result = LegalSummariser.summarise("contract.pdf", {
+  format: 'markdown',
+  max_sentences: 3
+})
+
+# Access different parts of the analysis
+puts result[:key_points]        # Key contract points
+puts result[:clauses]           # Detected legal clauses
+puts result[:risks]             # Risk analysis
+puts result[:metadata]          # Document metadata
+```
+
+### Command Line Interface
+
+```bash
+# Analyze a document
+legal_summariser analyze contract.pdf
+
+# Specify output format
+legal_summariser analyze contract.pdf --format markdown
+
+# Save to file
+legal_summariser analyze contract.pdf --output summary.md --format markdown
+
+# Run demo
+legal_summariser demo
+
+# Show supported formats
+legal_summariser supported_formats
+
+# Show version
+legal_summariser version
+```
+
+## Example Output
+
+### Plain Text Summary
+```
+This Non-Disclosure Agreement establishes confidentiality obligations between parties. 
+The agreement will remain valid for 2 years from the date of signing. Either party may 
+terminate with 30 days written notice. The receiving party will be liable for any 
+breach of confidentiality obligations.
+```
+
+### Risk Analysis
+```
+High Risks Found:
+- Unlimited Liability: Agreement may expose party to unlimited financial liability
+- Broad Indemnification: Very broad indemnification obligations that could be costly
+
+Compliance Gaps:
+- Missing GDPR Reference: Document processes personal data but lacks GDPR compliance language
+- Missing Data Subject Rights: No mention of data subject rights under GDPR
+```
+
+### Detected Clauses
+- **Confidentiality**: 3 clauses found
+- **Liability**: 2 clauses found  
+- **Termination**: 1 clause found
+- **Data Processing**: 2 clauses found
+
+## Supported Document Types
+
+The system automatically detects and optimizes analysis for:
+
+- **Non-Disclosure Agreements (NDAs)**
+- **Service Agreements**
+- **Employment Contracts**
+- **Privacy Policies**
+- **License Agreements**
+- **General Contracts**
+
+## Supported File Formats
+
+### Input Formats
+- PDF (.pdf)
+- Microsoft Word (.docx)
+- Plain Text (.txt)
+
+### Output Formats
+- JSON (structured data)
+- Markdown (formatted report)
+- Plain Text (simple summary)
+
+## Target Users
+
+- **Law firms & compliance teams**: Faster contract reviews
+- **Startups & SMEs**: Understanding investor or supplier contracts
+- **Forensics experts**: Extracting critical legal clauses for reports
+- **Academics & NGOs**: Analysing legal policies and regulations
+
+## Technical Architecture
+
+### Hybrid Approach
+- **Rule-based extractors**: For structured clause detection
+- **NLP processing**: For summarisation and risk detection
+- **Pattern matching**: For compliance gap identification
+
+### Key Components
+- **TextExtractor**: Multi-format document parsing
+- **Summariser**: Plain English conversion engine
+- **ClauseDetector**: Legal clause identification
+- **RiskAnalyzer**: Risk assessment and flagging
+- **Formatter**: Multi-format output generation
+
+## Development
+
+After checking out the repo, run `bundle install` to install dependencies. Then, run `rake spec` to run the tests.
+
+```bash
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Run linter
+bundle exec rubocop
+
+# Build gem
+gem build legal_summariser.gemspec
+
+# Install local gem
+gem install ./legal_summariser-*.gem
+```
+
+## Roadmap
+
+- **v0.1** ✅ Text extraction + basic summarisation
+- **v0.2** ✅ Clause detection + risk flagging  
+- **v0.3** 🔄 Plain language generator (fine-tuned models)
+- **v1.0** 📋 Multi-language support + PDF annotation output
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/legal-summariser/legal_summariser.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Disclaimer
+
+This tool is designed to assist with legal document analysis but should not replace professional legal advice. Always consult with qualified legal professionals for important legal matters.
+
+## 🌍 Global Impact
+
+- **Innovation**: Bridges AI with law and compliance in a developer-friendly way
+- **Contribution**: Open-source library for legal NLP in Ruby
+- **Public benefit**: Helps both professionals and citizens better understand their rights and obligations
+- **Global relevance**: Applicable across jurisdictions (GDPR, KVKK, HIPAA, CCPA)
+
+## 🛠️ Tech Stack
+
+This project leverages my expertise in:
+
+- **Ruby Development**: Gem architecture, modular design patterns
+- **AI & NLP**: Rule-based text analysis, pattern recognition
+- **Cybersecurity**: Compliance frameworks (GDPR, KVKK), risk assessment
+- **Digital Forensics**: Legal document analysis, evidence extraction
+- **Software Engineering**: Test-driven development, CLI tools
+
+## 🎓 Professional Background
+
+As a **Mobile Developer & Cyber Security Expert** with 10+ years in Public Sector IT:
+
+- 🎓 **Master's in Forensic Informatics** - Gazi University (2021-2023)
+- 🏢 **Mobile Developer** - Gendarmerie General Command (2024-Present)
+- 🔒 **Certified Ethical Hacker (CEH)**
+- 📱 **iOS & Android Development Expert**
+
+## 🤝 Connect & Collaborate
+
+| Platform | Link | Purpose |
+|----------|------|---------|
+| 🌐 **Portfolio** | [ahmetxhero.web.app](https://ahmetxhero.web.app) | Professional showcase |
+| 🎥 **YouTube** | [@ahmetxhero](https://youtube.com/@ahmetxhero) | Tech tutorials & content |
+| 💼 **LinkedIn** | [ahmetxhero](https://linkedin.com/in/ahmetxhero) | Professional network |
+| 🐤 **Twitter** | [@ahmetxhero](https://x.com/ahmetxhero) | Tech updates & thoughts |
+| 📝 **Medium** | [ahmetxhero.medium.com](https://ahmetxhero.medium.com) | Technical articles |
+| 📷 **Instagram** | [@ahmetxhero](https://instagram.com/ahmetxhero) | Behind the scenes |
+
+## ☕ Support My Work
+
+If this project helps you or your organization, consider:
+
+- ⭐ **Star this repository** on GitHub
+- 🔄 **Share it** with your network
+- ☕ **Buy me a coffee** to support open source development
+- 🗣️ **Invite me to speak** at your tech event about legal tech innovation
+
+## 🎯 Current Focus
+
+- 🔒 **Cybersecurity**: Developing secure applications with privacy-first approach
+- 📱 **Mobile Development**: Creating innovative iOS and Android solutions
+- 🔍 **Digital Forensics**: Advancing forensic investigation techniques
+- 📚 **Knowledge Sharing**: Contributing to the tech community through open source
+- ⚖️ **Legal Tech**: Building tools that make legal processes more accessible
+
+---
+
+*Building secure, innovative solutions for a better digital future* 🚀

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/exe/legal_summariser

@@ -0,0 +1,121 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require_relative '../lib/legal_summariser'
+require 'thor'
+
+module LegalSummariser
+  class CLI < Thor
+    desc "analyze FILE", "Analyze a legal document and generate summary"
+    option :format, aliases: '-f', default: 'text', desc: 'Output format (json, markdown, text)'
+    option :output, aliases: '-o', desc: 'Output file path (optional)'
+    option :max_sentences, type: :numeric, default: 5, desc: 'Maximum sentences in summary'
+    def analyze(file_path)
+      begin
+        puts "Analyzing: #{file_path}"
+        puts "Format: #{options[:format]}"
+        puts "-" * 50
+        
+        # Perform analysis
+        results = LegalSummariser.summarise(file_path, {
+          format: options[:format],
+          max_sentences: options[:max_sentences]
+        })
+        
+        # Output results
+        if options[:output]
+          File.write(options[:output], results)
+          puts "Results saved to: #{options[:output]}"
+        else
+          puts results
+        end
+        
+      rescue LegalSummariser::DocumentNotFoundError => e
+        puts "Error: #{e.message}"
+        exit 1
+      rescue LegalSummariser::UnsupportedFormatError => e
+        puts "Error: #{e.message}"
+        exit 1
+      rescue => e
+        puts "Unexpected error: #{e.message}"
+        puts e.backtrace if ENV['DEBUG']
+        exit 1
+      end
+    end
+
+    desc "version", "Show version information"
+    def version
+      puts "Legal Summariser v#{LegalSummariser::VERSION}"
+      puts "Ruby-based AI-powered legal document analysis toolkit"
+    end
+
+    desc "supported_formats", "List supported document formats"
+    def supported_formats
+      puts "Supported document formats:"
+      puts "- PDF (.pdf)"
+      puts "- Microsoft Word (.docx)"
+      puts "- Plain text (.txt)"
+      puts ""
+      puts "Output formats:"
+      puts "- JSON (json)"
+      puts "- Markdown (markdown, md)"
+      puts "- Plain text (text, txt)"
+    end
+
+    desc "demo", "Run demo analysis on sample documents"
+    def demo
+      puts "Legal Summariser Demo"
+      puts "=" * 50
+      puts ""
+      
+      # Create sample NDA text for demo
+      sample_text = create_sample_nda
+      sample_file = "/tmp/sample_nda.txt"
+      File.write(sample_file, sample_text)
+      
+      puts "Analyzing sample NDA document..."
+      puts ""
+      
+      results = LegalSummariser.summarise(sample_file, { format: 'markdown' })
+      puts results
+      
+      # Clean up
+      File.delete(sample_file) if File.exist?(sample_file)
+    end
+
+    private
+
+    def create_sample_nda
+      <<~NDA
+        NON-DISCLOSURE AGREEMENT
+
+        This Non-Disclosure Agreement ("Agreement") is entered into on [DATE] between Company ABC ("Disclosing Party") and John Doe ("Receiving Party").
+
+        1. CONFIDENTIAL INFORMATION
+        The Disclosing Party may disclose certain confidential and proprietary information to the Receiving Party. Confidential information includes all technical data, trade secrets, know-how, research, product plans, products, services, customers, customer lists, markets, software, developments, inventions, processes, formulas, technology, designs, drawings, engineering, hardware configuration information, marketing, finances, or other business information.
+
+        2. OBLIGATIONS
+        The Receiving Party agrees to hold and maintain the Confidential Information in strict confidence for a period of two (2) years from the date of disclosure. The Receiving Party shall not disclose any Confidential Information to third parties without prior written consent.
+
+        3. LIABILITY
+        The Receiving Party shall be liable for any breach of this Agreement and shall indemnify the Disclosing Party against all claims, damages, and expenses arising from such breach.
+
+        4. TERMINATION
+        This Agreement may be terminated by either party with thirty (30) days written notice. Upon termination, all Confidential Information must be returned or destroyed.
+
+        5. GOVERNING LAW
+        This Agreement shall be governed by the laws of England and Wales. Any disputes shall be resolved through binding arbitration.
+
+        6. DATA PROTECTION
+        Both parties acknowledge their obligations under the General Data Protection Regulation (GDPR) regarding any personal data processed under this Agreement.
+
+        IN WITNESS WHEREOF, the parties have executed this Agreement as of the date first written above.
+      NDA
+    end
+  end
+end
+
+# Run CLI if called directly
+if __FILE__ == $0
+  LegalSummariser::CLI.start(ARGV)
+end

data/lib/legal_summariser/clause_detector.rb

@@ -0,0 +1,206 @@
+# frozen_string_literal: true
+
+module LegalSummariser
+  class ClauseDetector
+    attr_reader :text
+
+    def initialize(text)
+      @text = text.downcase
+    end
+
+    # Detect key legal clauses in the document
+    # @return [Hash] Detected clauses with their content
+    def detect
+      {
+        data_processing: detect_data_processing_clauses,
+        liability: detect_liability_clauses,
+        confidentiality: detect_confidentiality_clauses,
+        termination: detect_termination_clauses,
+        payment: detect_payment_clauses,
+        intellectual_property: detect_ip_clauses,
+        dispute_resolution: detect_dispute_resolution_clauses,
+        governing_law: detect_governing_law_clauses
+      }.compact
+    end
+
+    private
+
+    # Detect data processing and privacy clauses
+    # @return [Array<Hash>] Data processing clauses
+    def detect_data_processing_clauses
+      patterns = [
+        /data\s+processing/,
+        /personal\s+data/,
+        /gdpr/,
+        /kvkk/,
+        /data\s+protection/,
+        /privacy\s+policy/,
+        /data\s+subject/,
+        /data\s+controller/,
+        /data\s+processor/
+      ]
+
+      find_clauses_by_patterns(patterns, "Data Processing")
+    end
+
+    # Detect liability and indemnification clauses
+    # @return [Array<Hash>] Liability clauses
+    def detect_liability_clauses
+      patterns = [
+        /liabilit/,
+        /liable/,
+        /indemnif/,
+        /damages/,
+        /limitation\s+of\s+liability/,
+        /exclude.*liability/,
+        /consequential\s+damages/,
+        /indirect\s+damages/
+      ]
+
+      find_clauses_by_patterns(patterns, "Liability")
+    end
+
+    # Detect confidentiality and non-disclosure clauses
+    # @return [Array<Hash>] Confidentiality clauses
+    def detect_confidentiality_clauses
+      patterns = [
+        /confidential/,
+        /non.?disclosure/,
+        /proprietary\s+information/,
+        /trade\s+secret/,
+        /confidentiality\s+agreement/,
+        /nda/
+      ]
+
+      find_clauses_by_patterns(patterns, "Confidentiality")
+    end
+
+    # Detect termination clauses
+    # @return [Array<Hash>] Termination clauses
+    def detect_termination_clauses
+      patterns = [
+        /terminat/,
+        /end\s+this\s+agreement/,
+        /breach.*agreement/,
+        /notice\s+of\s+termination/,
+        /expir/,
+        /cancel/
+      ]
+
+      find_clauses_by_patterns(patterns, "Termination")
+    end
+
+    # Detect payment and fee clauses
+    # @return [Array<Hash>] Payment clauses
+    def detect_payment_clauses
+      patterns = [
+        /payment/,
+        /fee/,
+        /\$[\d,]+/,
+        /invoice/,
+        /billing/,
+        /compensation/,
+        /remuneration/,
+        /salary/,
+        /wage/
+      ]
+
+      find_clauses_by_patterns(patterns, "Payment")
+    end
+
+    # Detect intellectual property clauses
+    # @return [Array<Hash>] IP clauses
+    def detect_ip_clauses
+      patterns = [
+        /intellectual\s+property/,
+        /copyright/,
+        /trademark/,
+        /patent/,
+        /trade\s+mark/,
+        /proprietary\s+rights/,
+        /ownership/,
+        /license/,
+        /licensing/
+      ]
+
+      find_clauses_by_patterns(patterns, "Intellectual Property")
+    end
+
+    # Detect dispute resolution clauses
+    # @return [Array<Hash>] Dispute resolution clauses
+    def detect_dispute_resolution_clauses
+      patterns = [
+        /dispute/,
+        /arbitration/,
+        /mediation/,
+        /litigation/,
+        /court/,
+        /jurisdiction/,
+        /resolution\s+of\s+disputes/,
+        /legal\s+proceedings/
+      ]
+
+      find_clauses_by_patterns(patterns, "Dispute Resolution")
+    end
+
+    # Detect governing law clauses
+    # @return [Array<Hash>] Governing law clauses
+    def detect_governing_law_clauses
+      patterns = [
+        /governing\s+law/,
+        /applicable\s+law/,
+        /laws?\s+of/,
+        /jurisdiction/,
+        /governed\s+by/,
+        /subject\s+to.*law/
+      ]
+
+      find_clauses_by_patterns(patterns, "Governing Law")
+    end
+
+    # Find clauses matching given patterns
+    # @param patterns [Array<Regexp>] Regex patterns to match
+    # @param clause_type [String] Type of clause being detected
+    # @return [Array<Hash>] Found clauses
+    def find_clauses_by_patterns(patterns, clause_type)
+      clauses = []
+      sentences = extract_sentences
+
+      sentences.each_with_index do |sentence, index|
+        patterns.each do |pattern|
+          if sentence.match?(pattern)
+            clauses << {
+              type: clause_type,
+              content: sentence.strip,
+              position: index + 1,
+              keywords: extract_keywords(sentence, pattern)
+            }
+            break # Don't match multiple patterns for the same sentence
+          end
+        end
+      end
+
+      clauses.uniq { |clause| clause[:content] }
+    end
+
+    # Extract sentences from text
+    # @return [Array<String>] Array of sentences
+    def extract_sentences
+      # Split on sentence boundaries
+      sentences = text.split(/(?<=[.!?])\s+/)
+      
+      # Filter out very short sentences
+      sentences.select { |s| s.length > 20 }
+               .map { |s| s.strip.gsub(/\s+/, ' ') }
+    end
+
+    # Extract relevant keywords from a sentence based on pattern
+    # @param sentence [String] The sentence
+    # @param pattern [Regexp] The matching pattern
+    # @return [Array<String>] Extracted keywords
+    def extract_keywords(sentence, pattern)
+      matches = sentence.scan(pattern).flatten
+      matches.map(&:strip).reject(&:empty?)
+    end
+  end
+end

data/lib/legal_summariser/document_parser.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module LegalSummariser
+  # Legacy compatibility - DocumentParser is now handled by TextExtractor
+  class DocumentParser
+    def self.parse(file_path)
+      TextExtractor.extract(file_path)
+    end
+  end
+end