profanity_filter 0.0.0

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008 Adam Bair and Intridea, Inc.
+ 
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+ 
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,88 @@
+= Fu-fu: The Profanity Filter for Rails
+
+This plugin will allow you to filter profanity using basic replacement or a dictionary term.  
+
+== Disclaimer
+
+This plugin is provided as is - therefore, the creators and contributors of this plugin are not responsible for any damages that may result from it's usage.  Use at your own risk; backup your data.
+
+== Install
+
+  ./script/plugin install git://github.com/adambair/fu-fu.git
+
+== Example
+
+=== You can use it in your models:
+
+Notice -- there are two profanity filters, one is destructive.  Beware the exclamation point (profanity_filter!).
+
+Non-Destructive (filters content when called, original text remains in the database)
+
+  profanity_filter :foo, :bar
+    #=> banned words will be replaced with @#=>$%
+
+  profanity_filter :foo, :bar, :method => 'dictionary'
+    #=> banned words will be replaced by value in config/dictionary.yml
+
+  profanity_filter :foo, :bar, :method => 'vowels'
+    #=> banned words will have their vowels replaced
+
+  profanity_filter :foo, :bar, :method => 'hollow'
+    #=> all letters except the first and last will be replaced
+
+  The non-destructive profanity_filter provides different versions of the filtered attribute:
+    some_model.foo => 'filtered version'
+    some_model.foo_original => 'non-filtered version'
+
+Destructive (saves the filtered content to the database)
+
+  profanity_filter! :foo, :bar
+    #=> banned words will be replaced with @#=>$%
+
+  profanity_filter! :foo, :bar, :method => 'dictionary'
+    #=> banned words will be replaced by value in config/dictionary.yml
+
+  profanity_filter! :foo, :bar, :method => 'vowels'
+    #=> banned words will have their vowels replaced
+
+  profanity_filter! :foo, :bar, :method => 'hollow'
+    #=> all letters except the first and last will be replaced
+
+=== You can also use the filter directly:
+
+  ProfanityFilter::Base.clean(text)
+  ProfanityFilter::Base.clean(text, 'dictionary')
+  ProfanityFilter::Base.clean(text, 'vowels')
+  ProfanityFilter::Base.clean(text, 'hollow')
+
+  ProfanityFilter::Base.profane?(text) #=> true/false
+
+== Benchmarks
+
+Inquiring minds can checkout the simple benchmarks I've included so you can have an idea of what kind of performance to expect.  I've included some quick scenarios including strings of (100, 1000, 5000, 1000) words and dictionaries of (100, 1000, 5000, 25000, 50000, 100000) words.
+
+You can run the benchmarks via:
+  ruby test/benchmark/fu-fu_benchmark.rb
+
+== TODO
+
+* Turn this into a gem
+* May break ProfanityFilter out on it's own
+* Clean up dictionary implementation and substitution (suboptimal and messy)
+* Move benchmarks into a rake task
+* Build out rdocs
+
+== License
+
+Fu-fu: The Profanity Filter for Rails uses the MIT License. Please see the MIT-LICENSE file.
+
+== Resources
+
+Created by Adam Bair (adam@intridea.com) of Intridea (http://www.intridea.com) in the open source room at RailsConf 2008.
+
+== Contributors
+
+* Paul Cortens and Greg Benedict - profane? method and tests.
+* Neil Ang - punctionation delimited profanity
+* Flinn Mueller - dynamic word list from method
+* Scott Stewert - additional dictionary words

data/Rakefile

@@ -0,0 +1,43 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+
+require 'jeweler'
+Jeweler::Tasks.new do |gem|
+  gem.name = "profanity_filter"
+  gem.homepage = "http://github.com/intridea/profanity_filter"
+  gem.license = "MIT"
+  gem.summary = %Q{A Rails plugin gem for filtering out profanity.}
+  gem.description = %Q{Allows you to filter profanity using basic replacement or a dictionary term.}
+  gem.email = "adambair@gmail.com"
+  gem.authors = ["Adam Bair"]
+end
+Jeweler::RubygemsDotOrgTasks.new
+
+desc 'Default: run unit tests.'
+task :default => :test
+
+desc 'Test the Profanity Filter plugin.'
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = true
+end
+
+desc 'Generate documentation for the happy plugin.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Profanity Filter'
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+desc 'Measures test coverage using rcov'
+task :rcov do
+  rm_f "coverage"
+  rm_f "coverage.data"
+  rcov = "rcov --rails --aggregate coverage.data --text-summary -Ilib"
+  system("#{rcov} --html #{Dir.glob('test/**/*_test.rb').join(' ')}")
+  system("open coverage/index.html") if PLATFORM['darwin']
+end

data/VERSION

@@ -0,0 +1 @@
+0.0.0

data/config/dictionary.yml

@@ -0,0 +1,200 @@
+# the filter currently checks for words that are 3 or more characters.
+---
+ass: "*ss"
+asses: "*ss*s"
+asshole: "*ssh*l*"
+assholes: "*ssh*l*s"
+bastard: b*st*rd
+beastial: b**st**l
+beastiality: b**st**l*ty
+beastility: b**st*l*ty
+bestial: b*st**l
+bestiality: b*st**l*ty
+bitch: b*tch
+bitcher: b*tch*r
+bitchers: b*tch*rs
+bitches: b*tch*s
+bitchin: b*tch*n
+bitching: b*tch*ng
+blowjob: bl*wj*b
+blowjobs: bl*wj*bs
+bullshit: b*llsh*t
+clit: cl*t
+cock: c*ck
+cocks: c*cks
+cocksuck: c*cks*ck
+cocksucked: c*cks*ck*d
+cocksucker: c*cks*ck*r
+cocksucking: c*cks*ck*ng
+cocksucks: c*cks*cks
+cum: c*m
+cummer: c*mm*r
+cumming: c*mm*ng
+cums: c*ms
+cumshot: c*msh*t
+cunillingus: c*n*ll*ng*s
+cunnilingus: c*nn*l*ng*s
+cunt: c*nt
+cuntlick: c*ntl*ck
+cuntlicker: c*ntl*ck*r
+cuntlicking: c*ntl*ck*ng
+cunts: c*nts
+cyberfuc: cyb*rf*c
+cyberfuck: cyb*rf*ck
+cyberfucked: cyb*rf*ck*d
+cyberfucker: cyb*rf*ck*r
+cyberfuckers: cyb*rf*ck*rs
+cyberfucking: cyb*rf*ck*ng
+damn: d*mn
+dildo: d*ld*
+dildos: d*ld*s
+dick: d*ck
+dink: d*nk
+dinks: d*nks
+ejaculate: "*j*c*l*t*"
+ejaculated: "*j*c*l*t*d"
+ejaculates: "*j*c*l*t*s"
+ejaculating: "*j*c*l*t*ng"
+ejaculatings: "*j*c*l*t*ngs"
+ejaculation: "*j*c*l*t**n"
+fag: f*g
+fagging: f*gg*ng
+faggot: f*gg*t
+faggs: f*ggs
+fagot: f*g*t
+fagots: f*g*ts
+fags: f*gs
+fart: f*rt
+farted: f*rt*d
+farting: f*rt*ng
+fartings: f*rt*ngs
+farts: f*rts
+farty: f*rty
+felatio: f*l*t**
+fellatio: f*ll*t**
+fingerfuck: f*ng*rf*ck
+fingerfucked: f*ng*rf*ck*d
+fingerfucker: f*ng*rf*ck*r
+fingerfuckers: f*ng*rf*ck*rs
+fingerfucking: f*ng*rf*ck*ng
+fingerfucks: f*ng*rf*cks
+fistfuck: f*stf*ck
+fistfucked: f*stf*ck*d
+fistfucker: f*stf*ck*r
+fistfuckers: f*stf*ck*rs
+fistfucking: f*stf*ck*ng
+fistfuckings: f*stf*ck*ngs
+fistfucks: f*stf*cks
+fuck: f*ck
+fucked: f*ck*d
+fucker: f*ck*r
+fuckers: f*ck*rs
+fuckin: f*ck*n
+fucking: f*ck*ng
+fuckings: f*ck*ngs
+fuckme: f*ckm*
+fucks: f*cks
+fuk: f*k
+fuks: f*ks
+gangbang: g*ngb*ng
+gangbanged: g*ngb*ng*d
+gangbangs: g*ngb*ngs
+gaysex: g*ys*x
+goddamn: g*dd*mn
+hardcoresex: h*rdc*r*s*x
+hell: h*ll
+horniest: h*rn**st
+horny: h*rny
+hotsex: h*ts*x
+jism: j*sm
+jiz: j*z
+jizm: j*zm
+kock: k*ck
+kondum: k*nd*m
+kondums: k*nd*ms
+kum: k*m
+kumer: k*mm*r
+kummer: k*mm*r
+kumming: k*mm*ng
+kums: k*ms
+kunilingus: k*n*l*ng*s
+lust: l*st
+lusting: l*st*ng
+mothafuck: m*th*f*ck
+mothafucka: m*th*f*ck*
+mothafuckas: m*th*f*ck*s
+mothafuckaz: m*th*f*ck*z
+mothafucked: m*th*f*ck*d
+mothafucker: m*th*f*ck*r
+mothafuckers: m*th*f*ck*rs
+mothafuckin: m*th*f*ck*n
+mothafucking: m*th*f*ck*ng
+mothafuckings: m*th*f*ck*ngs
+mothafucks: m*th*f*cks
+motherfuck: m*th*rf*ck
+motherfucked: m*th*rf*ck*d
+motherfucker: m*th*rf*ck*r
+motherfuckers: m*th*rf*ck*rs
+motherfuckin: m*th*rf*ck*n
+motherfucking: m*th*rf*ck*ng
+motherfuckings: m*th*rf*ck*ngs
+motherfucks: m*th*rf*cks
+niger: n*gg*r
+nigger: n*gg*r
+niggers: n*gg*rs
+orgasim: "*rg*s*m"
+orgasims: "*rg*s*ms"
+orgasm: "*rg*sm"
+orgasms: "*rg*sms"
+phonesex: ph*n*s*x
+phuk: ph*k
+phuked: ph*k*d
+phuking: ph*k*ng
+phukked: ph*kk*d
+phukking: ph*kk*ng
+phuks: ph*ks
+phuq: ph*q
+pis: p*ss
+piss: p*ss
+pisser: p*ss*r
+pissed: p*ss*d
+pisser: p*ss*r
+pissers: p*ss*rs
+pises: p*ss*s
+pisses: p*ss*s
+pisin: p*ss*n
+pissin: p*ss*n
+pising: p*ss*ng
+pissing: p*ss*ng
+pisof: p*ss*ff
+pissoff: p*ss*ff
+porn: p*rn
+porno: p*rn*
+pornography: p*rn*gr*phy
+pornos: p*rn*s
+prick: pr*ck
+pricks: pr*cks
+pussies: p*ss**s
+pusies: p*ss**s
+pussy: p*ssy
+pusy: p*ssy
+pussys: p*ssys
+pusys: p*ssys
+shit: sh*t
+shited: sh*t*d
+shitfull: sh*tf*ll
+shiting: sh*t*ng
+shitings: sh*t*ngs
+shits: sh*ts
+shitted: sh*tt*d
+shitter: sh*tt*r
+shitters: sh*tt*rs
+shitting: sh*tt*ng
+shittings: sh*tt*ngs
+shitty: sh*tty
+shity: sh*tty
+slut: sl*t
+sluts: sl*ts
+smut: sm*t
+spunk: sp*nk
+twat: tw*t

data/lib/profanity_filter.rb

@@ -0,0 +1,82 @@
+module ProfanityFilter
+  def self.included(base)
+    # base.send :extend, ClassMethods
+      base.class_eval do
+        extend ClassMethods
+      end
+    
+  end
+
+  module ClassMethods
+    def profanity_filter!(*attr_names)
+      option = attr_names.pop[:method] if attr_names.last.is_a?(Hash)
+      attr_names.each { |attr_name| setup_callbacks_for(attr_name, option) }
+    end
+    
+    def profanity_filter(*attr_names)
+      option = attr_names.pop[:method] if attr_names.last.is_a?(Hash)
+
+      attr_names.each do |attr_name| 
+        instance_eval do
+          define_method "#{attr_name}_clean" do; ProfanityFilter::Base.clean(self[attr_name.to_sym], option); end      
+          define_method "#{attr_name}_original"do; self[attr_name]; end
+          alias_method attr_name.to_sym, "#{attr_name}_clean".to_sym
+        end
+      end
+    end
+    
+    def setup_callbacks_for(attr_name, option)
+      before_validation do |record|
+        record[attr_name.to_sym] = ProfanityFilter::Base.clean(record[attr_name.to_sym], option)
+      end
+    end
+  end
+  
+  class Base
+    cattr_accessor :replacement_text, :dictionary_file, :dictionary
+    @@replacement_text = '@#$%'
+    @@dictionary_file  = File.join(File.dirname(__FILE__), '../config/dictionary.yml')
+    @@dictionary       = YAML.load_file(@@dictionary_file)
+
+    class << self
+      def banned?(word = '')
+        dictionary.include?(word.downcase) if word
+      end
+      alias :profane? :banned? 
+      
+      def clean(text, replace_method = '')
+        return text if text.blank?
+        @replace_method = replace_method
+        text.split(/(\s)/).collect{ |word| clean_word(word) }.join
+      end
+      
+      def clean_word(word)
+         return word unless(word.strip.size > 2)
+         
+         if word.index(/[\W]/)
+           word = word.split(/(\W)/).collect{ |subword| clean_word(subword) }.join
+           concat = word.gsub(/\W/, '')
+           word = concat if banned? concat
+         end
+         
+         banned?(word) ? replacement(word) : word
+       end
+       
+       def replacement(word)
+         case @replace_method
+         when 'dictionary'
+           dictionary[word.downcase] || word
+         when 'vowels'
+           word.gsub(/[aeiou]/i, '*')
+         when 'hollow'
+           word[1..word.size-2] = '*' * (word.size-2) if word.size > 2
+           word
+         else
+           replacement_text
+         end
+       end
+    end
+  end
+end
+
+ActiveRecord::Base.send(:include, ProfanityFilter)

data/test/benchmark/dictionary_test_100.yml

@@ -0,0 +1,100 @@
+---
+A+YLRX: happy
+A-AZIZ: happy
+A-Bomb: happy
+A-Call: happy
+A-Copy: happy
+A-Date: happy
+A-Dope: happy
+A-Graf: happy
+A-Lead: happy
+A-MAIL: happy
+A-Mail: happy
+A-Plus: happy
+A-RING: happy
+A-Spot: happy
+A-TACT: happy
+A-Tack: happy
+A-Team: happy
+A-Tech: happy
+A-Tron: happy
+A-Wing: happy
+A-Xing: happy
+A-Zoid: happy
+A/ROSE: happy
+A/VMIG: happy
+A4TECH: happy
+AA/EOE: happy
+AAA/AA: happy
+AAJJVo: happy
+AARNet: happy
+AARnet: happy
+AAUA-L: happy
+AAnJgZ: happy
+AAplay: happy
+AAtest: happy
+ABB-ZX: happy
+ABC-HU: happy
+ABC-TV: happy
+ABCB's: happy
+ABCL/R: happy
+ABCL/c: happy
+ABSoft: happy
+AC/clv: happy
+ACC-AS: happy
+ACCU's: happy
+ACEnet: happy
+ACK'ed: happy
+ACKing: happy
+ACLU's: happy
+ACM-NS: happy
+ACOnet: happy
+ACP/AR: happy
+ACPS/F: happy
+ACS-AJ: happy
+ACSnet: happy
+ACT-IV: happy
+ACTive: happy
+ACTnet: happy
+ACU-HQ: happy
+ACcura: happy
+AChris: happy
+AD-COL: happy
+ADAM's: happy
+ADD-ON: happy
+ADND's: happy
+ADPqkk: happy
+ADS-AS: happy
+ADV/Ma: happy
+ADtoHT: happy
+ADvice: happy
+AE-PSU: happy
+AE/ETM: happy
+AECL's: happy
+AED-JR: happy
+AEDesc: happy
+AEHS'O: happy
+AERO-A: happy
+AERT's: happy
+AESS's: happy
+AESend: happy
+AEleen: happy
+AF-MIL: happy
+AFA8JM: happy
+AFCO's: happy
+AFFC's: happy
+AFIN-B: happy
+AFlags: happy
+AG-INC: happy
+AGA/CD: happy
+AGS-CO: happy
+AGS-DC: happy
+AGS-IS: happy
+AGS-TX: happy
+AHFC's: happy
+AHearn: happy
+AI-CHI: happy
+AI-GIG: happy
+AI-NET: happy
+AI/Exp: happy
+AICS-L: happy