cchardet 0.2.0

data/ext/uchardet/script/BuildLangModelLogs/LangDanishModel.log

@@ -0,0 +1,158 @@
+= Logs of language model for Danish (da) =
+
+- Generated by BuildLangModel.py
+- Started: 2016-02-19 17:53:58.564190
+- Maximum depth: 4
+- Max number of pages: 100
+
+== Parsed pages ==
+
+Forside (revision 2692411)
+16. februar (revision 6877446)
+17. februar (revision 8454583)
+1878 (revision 8280505)
+19. februar (revision 8206479)
+1922 (revision 8455105)
+1926 (revision 8425271)
+1942 (revision 8443554)
+1945 (revision 8448461)
+1948 (revision 8454392)
+1985 (revision 8409096)
+2. verdenskrig (revision 8433181)
+23. oktober (revision 6877825)
+26. oktober (revision 7849938)
+3C 273 (revision 8443798)
+A-bus (revision 8427319)
+Aktuelle begivenheder (revision 8440596)
+B-52 Stratofortress (revision 8422571)
+Borgerkrigen i Syrien (revision 8447763)
+Boutros Boutros-Ghali (revision 8453935)
+Brasilien (revision 8452750)
+Cusco (region) (revision 7693764)
+Danmark (revision 8451178)
+Danmark i Eurovision Song Contest (revision 8453514)
+Dansk (sprog) (revision 8455750)
+Dansk Melodi Grand Prix 2016 (revision 8452164)
+Dobbeltmordet på Peter Bangs Vej (revision 8334648)
+Encyklopædi (revision 8446641)
+Eritrea-sagen (revision 8452285)
+Eurovision Song Contest 2014 (revision 8445804)
+Eurovision Song Contest 2016 (revision 8453588)
+Flygtningekrisen i Europa 2015 (revision 8452286)
+Fonograf (revision 8177165)
+Formel 1 (revision 8450846)
+Formel 1 2016 (revision 8456463)
+Frederik 6. (revision 8438503)
+Første observation af gravitationsbølger (revision 8451269)
+Grammofon (revision 8375093)
+Guadalcanal (revision 7796248)
+Harper Lee (revision 8456583)
+Hartkorn (revision 8437552)
+IC4 (revision 8446402)
+IC4-sagen (revision 8434463)
+Islamisk Stat (revision 8439228)
+Jonathan Leunbach (revision 8452603)
+Juliane Marie af Braunschweig-Wolfenbüttel (revision 8437957)
+Kaliumklorid (revision 8452216)
+Kejserriget Japan (revision 8044942)
+Kevin Magnussen (revision 8455302)
+København (revision 8427847)
+LIGO (revision 8451266)
+Latinamerika (revision 7692181)
+Leonid Hurwicz (revision 8445727)
+Lighthouse X (revision 8452940)
+Linkoban (revision 8455879)
+Machu Picchu (revision 8406907)
+Matador (tv-serie) (revision 8454648)
+Middelaldercentret (revision 8449194)
+Nobelprisen (revision 8409809)
+Nykøbing Falster (revision 8452825)
+Nyligt afdøde (revision 8456580)
+Overvågning (revision 8455039)
+Panorama (foto) (revision 8448393)
+Peru (revision 8437485)
+Peter Lauritsen (revision 8456097)
+Professor (revision 8415451)
+Renault F1 (revision 8450843)
+S-bus (revision 8455589)
+Salomonøerne (revision 8238961)
+Slaget om Belgien (1940) (revision 8430013)
+Slaget om Guadalcanal (revision 7762887)
+Slaget om Henderson Field (revision 8445480)
+Slaget om Iwo Jima (revision 8145239)
+Soldiers of Love (Lighthouse X-sang) (revision 8452929)
+Solen (revision 8276478)
+Stillehavskrigen (revision 8430649)
+Stockholm (revision 8358042)
+Søslaget ved Guadalcanal (revision 7772812)
+Thomas Edison (revision 8282441)
+Togulykken ved Bad Aibling (revision 8455364)
+Topografi (revision 6886168)
+USA (revision 8448088)
+United States Army (revision 8401635)
+United States Marine Corps (revision 8401667)
+Vestallierede (revision 6961443)
+Wikimedia (revision 8263252)
+Wikipedia (revision 8267051)
+Zikavirus (revision 8454832)
+1. februar (revision 8404985)
+10. februar (revision 6877431)
+11. februar (revision 6877433)
+12. februar (revision 6877437)
+13. februar (revision 6877438)
+14. februar (revision 6877441)
+1497 (revision 7369489)
+15. februar (revision 7329463)
+1560 (revision 7874693)
+1568 (revision 7369703)
+1620 (revision 7423903)
+1688 (revision 7367090)
+18. februar (revision 6877450)
+
+== End of Parsed pages ==
+
+- Wikipedia parsing ended at: 2016-02-19 17:56:42.162636
+
+53 characters appeared 1301488 times.
+
+First 30 characters:
+[ 0] Char e: 15.272749345364689 %
+[ 1] Char r: 8.48482659847805 %
+[ 2] Char n: 7.695652975670924 %
+[ 3] Char t: 6.977014002434137 %
+[ 4] Char a: 6.780469739252302 %
+[ 5] Char i: 6.164636170291236 %
+[ 6] Char s: 6.0942551909814 %
+[ 7] Char d: 5.953493232361728 %
+[ 8] Char l: 5.076650725938311 %
+[ 9] Char o: 4.883026197706011 %
+[10] Char g: 4.012253666572415 %
+[11] Char k: 3.232607599916403 %
+[12] Char m: 3.0863135119186653 %
+[13] Char f: 2.701600014752345 %
+[14] Char v: 2.13970470722742 %
+[15] Char b: 1.982423195603801 %
+[16] Char u: 1.8339777239590376 %
+[17] Char p: 1.5789619266562582 %
+[18] Char h: 1.3433085821767086 %
+[19] Char ø: 0.8730775850411222 %
+[20] Char y: 0.859938777768216 %
+[21] Char å: 0.7699648402443973 %
+[22] Char æ: 0.7208671920140639 %
+[23] Char j: 0.644108896893402 %
+[24] Char c: 0.5698093259407694 %
+[25] Char w: 0.11087309295206717 %
+[26] Char z: 0.05309307500338075 %
+[27] Char x: 0.032424424965885205 %
+[28] Char é: 0.032193919575132464 %
+[29] Char q: 0.012139950579644223 %
+
+The first 30 characters have an accumulated ratio of 0.9997241618823994.
+
+964 sequences found.
+
+First 512 (typical positive ratio): 0.9968082796759031
+Next 512 (512-1024): 7.68351302509128e-07
+Rest: 3.903127820947816e-17
+
+- Processing end: 2016-02-19 17:56:42.304278

data/ext/uchardet/script/BuildLangModelLogs/LangEsperantoModel.log

@@ -0,0 +1,110 @@
+= Logs of language model for Esperanto (eo) =
+
+- Generated by BuildLangModel.py
+- Started: 2015-12-04 01:22:51.466573
+- Maximum depth: 3
+- Max number of pages: 50
+
+== Parsed pages ==
+
+Vikipedio:Ĉefpaĝo (revision 5524911)
+10-a de novembro (revision 5792999)
+12-a de novembro (revision 5793854)
+13-a de novembro (revision 5795088)
+18-a de novembro (revision 5796972)
+2-a de novembro (revision 5772615)
+20-a de novembro (revision 5799664)
+2015 (revision 5791963)
+22-a de novembro (revision 5799355)
+24-a de novembro (revision 5800563)
+4-a de decembro (revision 5806422)
+4-a de novembro (revision 5789811)
+5-a de novembro (revision 5789774)
+6-a de novembro (revision 5790336)
+7-a de novembro (revision 5791066)
+8-a de novembro (revision 5791337)
+9-a de novembro (revision 5791916)
+A Night at the Opera (Queen) (revision 5184272)
+Abdelhamid Abaaoud (revision 5800134)
+André Glucksmann (revision 5792591)
+Anglio (revision 5693468)
+Argentino (revision 5804665)
+Atencoj de novembro 2015 en Parizo (revision 5800135)
+Aung San Suu Kyi (revision 5791362)
+Austin FX4 (revision 5583207)
+Azilo (revision 5751210)
+Aŭstrio (revision 5804014)
+Bahio (revision 5773065)
+Bamako (revision 5798202)
+Bataclan (revision 5795605)
+Bejruto (revision 5774306)
+Birmo (revision 5790386)
+Blonda (revision 5441229)
+Bohemian rhapsody (revision 5654078)
+Cayetano Redondo (revision 5591025)
+Ciro la 2-a (revision 5774667)
+DJ Abdel (revision 5628860)
+Daniela Mercury (revision 5764721)
+Decembro de 2015 (revision 5626904)
+Dilatkoeficiento (revision 5806460)
+Eksproprietigo (revision 5586845)
+Elektroniko (revision 5788966)
+Elle s'appelait Sarah (filmo) (revision 5475154)
+Esperanto (revision 5804190)
+Federaciero (revision 5696168)
+Fondaĵo Vikimedio (revision 5772681)
+Francio (revision 5759775)
+François Hollande (revision 5627721)
+
+== End of Parsed pages ==
+
+- Wikipedia parsing ended at: 2015-12-04 01:27:38.176708
+
+56 characters appeared 342524 times.
+
+First 35 characters:
+[ 0] Char a: 12.557952143499435 %
+[ 1] Char o: 9.84719318938235 %
+[ 2] Char e: 9.10242785906973 %
+[ 3] Char i: 8.362333734278474 %
+[ 4] Char n: 7.6359612757062285 %
+[ 5] Char r: 6.630192336887342 %
+[ 6] Char t: 5.70821314710794 %
+[ 7] Char l: 5.610409781504361 %
+[ 8] Char s: 5.004320865107262 %
+[ 9] Char k: 3.8855671427403626 %
+[10] Char d: 3.7194473963868226 %
+[11] Char j: 3.28531723324497 %
+[12] Char u: 2.8465158645817517 %
+[13] Char m: 2.787833845219605 %
+[14] Char p: 2.6582078920017285 %
+[15] Char g: 1.6825098387266293 %
+[16] Char v: 1.4048650605505015 %
+[17] Char c: 1.3823848839789328 %
+[18] Char b: 1.1406499982482978 %
+[19] Char f: 1.077296773364786 %
+[20] Char z: 0.7342551178895493 %
+[21] Char h: 0.6735294461118053 %
+[22] Char ĝ: 0.53572888323154 %
+[23] Char ŭ: 0.4268314045147202 %
+[24] Char ĉ: 0.33545094650301877 %
+[25] Char y: 0.17079095187490512 %
+[26] Char ŝ: 0.15327393116978666 %
+[27] Char w: 0.1442234704721421 %
+[28] Char ĵ: 0.1039343228503696 %
+[29] Char á: 0.0814541462788009 %
+[30] Char ó: 0.05430276418586727 %
+[31] Char é: 0.053718863495696656 %
+[32] Char q: 0.04350060141771087 %
+[33] Char x: 0.040873048311943105 %
+[34] Char ĥ: 0.03824549520617533 %
+
+The first 35 characters have an accumulated ratio of 0.9991971365510156.
+
+989 sequences found.
+
+First 512 (typical positive ratio): 0.9942980632768038
+Next 512 (512-1024): 0.0015327393116978665
+Rest: -5.0306980803327406e-17
+
+- Processing end: 2015-12-04 01:27:38.307198

data/ext/uchardet/script/BuildLangModelLogs/LangEstonianModel.log

@@ -0,0 +1,159 @@
+= Logs of language model for Estonian (et) =
+
+- Generated by BuildLangModel.py
+- Started: 2016-09-26 23:45:22.351942
+- Maximum depth: 5
+- Max number of pages: 100
+
+== Parsed pages ==
+
+Harilik pohl (revision 4248853)
+A-vitamiin (revision 4330862)
+Aasta keskmine sademete hulk (revision 4266801)
+Aasta keskmine õhutemperatuur (revision 3902142)
+Ahm (revision 4343671)
+Ain Raal (revision 4464651)
+Alalehed (revision 2892741)
+Alamliik (revision 3522810)
+Alaska (revision 4216575)
+Aleksander Heintalu (revision 4445156)
+Aleuudid (revision 4335893)
+Ameerika jänes (revision 4325220)
+Ameerika valgejänes (revision 4355263)
+Anneli Sihvart (revision 4211078)
+Arbutiin (revision 4451788)
+Baribal (revision 4268462)
+Bensoehape (revision 3810308)
+Binaarne nomenklatuur (revision 3970950)
+C-vitamiin (revision 4444353)
+Droog (revision 4352968)
+E-vitamiin (revision 4336726)
+Eesti (revision 4474984)
+Eesti Entsüklopeediakirjastus (revision 4012421)
+Eesti köök (revision 4314947)
+Ellips (revision 4272113)
+Emakakael (botaanika) (revision 3521516)
+Euraasia (revision 3710768)
+Fenoloogia (revision 3512905)
+Folaadid (revision 4266628)
+Fosfor (revision 4270122)
+Fotosüntees (revision 4380600)
+Fruktoos (revision 4285660)
+Glükoos (revision 4047315)
+Gneiss (revision 4333338)
+Graniit (revision 4435351)
+Gröönimaa (revision 4331557)
+Halljänes (revision 4051603)
+Haned (revision 4127680)
+Happeline keskkond (revision 2966453)
+Heilongjiang (revision 4342364)
+Hendrik Relve (revision 4342591)
+Hiina (revision 4448121)
+Holland (revision 4307885)
+Hunt (revision 4427752)
+Hõimkond (revision 3489569)
+Hüdrofiilsus (revision 4309797)
+Ida-Euroopa (revision 4337624)
+Ida-sinilind (revision 4248853)
+Ida-vöötorav (revision 3520679)
+Igihaljus (revision 3536500)
+Ilves (revision 4404632)
+Imetaja (revision 4289188)
+Indiaanlased (revision 4479868)
+Indrek Rohtmets (revision 4218674)
+Itaalia (revision 4404119)
+Jaapan (revision 4465542)
+Jilin (revision 3894473)
+Jood (revision 4025060)
+Juurestik (revision 3341159)
+Jääkaru (revision 4372399)
+Jõhvikas (revision 4391549)
+Kaalium (revision 4486067)
+Kaheidulehelised (revision 4031352)
+Kaheli õiekate (revision 3063362)
+Kahesuguline õis (revision 3383221)
+Kaitsestaatus (revision 3527096)
+Kajakas (revision 4456839)
+Kalorsus (revision 3843290)
+Kaltsium (revision 4339861)
+Kanada (revision 4434682)
+Kanalised (revision 3616579)
+Kanarbikulaadsed (revision 4318215)
+Kanarbikulised (revision 3534760)
+Karboksüülhapped (revision 3659011)
+Karoteen (revision 4347634)
+Kasvuperiood (revision 4231717)
+Katteseemnetaimed (revision 4176294)
+Kaukasus (revision 4476003)
+Kesk-Euroopa (revision 3580746)
+Kimalane (revision 4261145)
+Kiudained (toit) (revision 3538655)
+Klass (bioloogia) (revision 3489567)
+Kliima (revision 4160781)
+Korea (revision 4329396)
+Kroom (revision 4030460)
+Kroonlehed (revision 3543291)
+Kuusepüü (revision 4028988)
+Kvertsetiin (revision 4448461)
+Laanemets (revision 4001157)
+Laanepüü (revision 4475093)
+Laiuskraad (revision 3990366)
+Leesikas (revision 4420533)
+Lehed (revision 4471821)
+Leheroots (revision 3595351)
+Liik (bioloogia) (revision 4320981)
+Liiv (revision 4399494)
+Liivakivi (revision 4330598)
+Linnaeus (revision 4276836)
+Linnud (revision 4479668)
+
+== End of Parsed pages ==
+
+- Wikipedia parsing ended at: 2016-09-26 23:47:54.476445
+
+55 characters appeared 433559 times.
+
+First 33 characters:
+[ 0] Char a: 12.486881831538499 %
+[ 1] Char i: 10.26503889897338 %
+[ 2] Char e: 10.177622884082673 %
+[ 3] Char s: 8.710233209320991 %
+[ 4] Char t: 6.56634967789851 %
+[ 5] Char l: 6.051540851418146 %
+[ 6] Char u: 5.423944607308348 %
+[ 7] Char n: 5.131020230233947 %
+[ 8] Char k: 4.663033174262327 %
+[ 9] Char o: 4.526950195936424 %
+[10] Char d: 4.167368224393911 %
+[11] Char r: 3.6740097656835635 %
+[12] Char m: 3.552688330769284 %
+[13] Char v: 2.4700213811730354 %
+[14] Char p: 1.9229216784797456 %
+[15] Char g: 1.865259399528092 %
+[16] Char h: 1.8043680329551455 %
+[17] Char j: 1.6860450365463524 %
+[18] Char ä: 1.0247740215287884 %
+[19] Char b: 0.9255949017319443 %
+[20] Char õ: 0.9246723052687178 %
+[21] Char ü: 0.6536595941959457 %
+[22] Char f: 0.37342091849090897 %
+[23] Char c: 0.34851081398379463 %
+[24] Char ö: 0.24333481717597835 %
+[25] Char y: 0.1287022066200909 %
+[26] Char x: 0.06781084004714467 %
+[27] Char w: 0.04082489349777078 %
+[28] Char q: 0.020989069538401926 %
+[29] Char š: 0.018913227496142396 %
+[30] Char z: 0.017529332801302706 %
+[31] Char ō: 0.010379210211297655 %
+[32] Char ž: 0.009687262863877812 %
+
+The first 33 characters have an accumulated ratio of 0.9995410082595447.
+
+853 sequences found.
+
+First 512 (typical positive ratio): 0.9972721312183132
+Next 512 (512-1024): 9.687262863877811e-05
+Rest: -5.204170427930421e-18
+
+- Processing end: 2016-09-26 23:47:54.561846

data/ext/uchardet/script/BuildLangModelLogs/LangFinnishModel.log

@@ -0,0 +1,156 @@
+= Logs of language model for Finnish (fi) =
+
+- Generated by BuildLangModel.py
+- Started: 2016-09-21 18:12:24.181917
+- Maximum depth: 5
+- Max number of pages: 100
+
+== Parsed pages ==
+
+Yhdistynyt kuningaskunta (revision 15843357)
+1. toukokuuta (revision 15910178)
+1700-luku (revision 15493702)
+1707 (revision 15106709)
+1800-luku (revision 15708929)
+2014 (revision 15891601)
+409 (revision 12809782)
+5. marraskuuta (revision 15421719)
+927 (revision 12785964)
+Aasia (revision 15948161)
+Abhasia (revision 15730328)
+Adolf Hitler (revision 15951829)
+Afrikka (revision 15934209)
+Agatha Christie (revision 15760740)
+Aikavyöhyke (revision 15800313)
+Ajoneuvon kansallisuustunnus (revision 15897445)
+Akrotiri ja Dhekelia (revision 14625383)
+Alamaat (revision 15913741)
+Alan Turing (revision 15904871)
+Alankomaat (revision 15936643)
+Albania (revision 15767604)
+Alec Guinness (revision 15363805)
+Alexander Fleming (revision 15023225)
+Alfred Hitchcock (revision 15892843)
+Alfred Tennyson (revision 15856114)
+Allen Jones (revision 12871703)
+Andorra (revision 15913862)
+Andrew Lloyd Webber (revision 14978349)
+Anglit (revision 15902350)
+Anguilla (revision 15854041)
+Anne Brontë (revision 14287992)
+Anthony Eden (revision 14391831)
+Antigua ja Barbuda (revision 15196967)
+Arabian Lawrence (revision 15736417)
+Argentiina (revision 15676474)
+Armenia (revision 15634470)
+Arthur Conan Doyle (revision 15402837)
+Arts and Crafts (revision 15806930)
+Aurinko (revision 15934252)
+Australia (revision 15934255)
+Avara luonto (revision 15815943)
+Azerbaidžan (revision 15946891)
+BBC (revision 15866026)
+BKT (revision 15656549)
+Bahama (revision 15516869)
+Bangladesh (revision 15883994)
+Bank of England (revision 14481173)
+Barbados (revision 15839821)
+Barbara Hepworth (revision 15106880)
+Bath (revision 15869900)
+Beatrix Potter (revision 15057380)
+Belfast (revision 15715934)
+Belgia (revision 15932391)
+Belize (revision 15665086)
+Ben Nevis (revision 15610196)
+Bengalin kieli (revision 15551820)
+Benjamin Britten (revision 15081615)
+Bermuda (revision 15632621)
+Bertrand Russell (revision 14631969)
+Bhutan (revision 15377394)
+Big Ben (revision 14897401)
+Big Brother (revision 14641391)
+Birmingham (revision 15855259)
+Black Sabbath (revision 15839917)
+Bosnia ja Hertsegovina (revision 15934266)
+Botswana (revision 15524955)
+Bristol (revision 15891889)
+Bristolin kanaali (revision 15849713)
+Bristolin kansainvälinen lentoasema (revision 14452870)
+Britannia (provinssi) (revision 14557442)
+Britannian avoin golfturnaus (revision 14293265)
+Britannian kuninkaallinen perhe (revision 15522149)
+Britannian talous (revision 15470242)
+Britannian väestö (revision 15661241)
+Brittein saaret (revision 15805422)
+Brittiläinen Antarktiksen alue (revision 15836227)
+Brittiläinen Intia (revision 15593126)
+Brittiläinen Intian valtameren alue (revision 14272903)
+Brittiläinen imperiumi (revision 15906600)
+Brittiläinen kansainyhteisö (revision 15894379)
+Brittiläinen keittiö (revision 13393533)
+Brittiläinen kulttuuri (revision 15951407)
+Brittiläiset Neitsytsaaret (revision 15910520)
+Brittiläiset merentakaiset alueet (revision 15836213)
+Brunei (revision 15580824)
+Bruttokansantuote (revision 15656549)
+Bulgaria (revision 15944101)
+Burma (revision 15627218)
+Cambridge (revision 14641664)
+Cambridgen yliopisto (revision 15493340)
+Canterburyn tarinoita (revision 15232140)
+Cardiff (revision 15840398)
+Caymansaaret (revision 15914575)
+Channel 4 (revision 15882475)
+Charles Babbage (revision 15203616)
+Charles Chaplin (revision 15674652)
+Charles Darwin (revision 15894085)
+Charles Dickens (revision 15699592)
+Charles Dickensin joulutarina (revision 15116247)
+
+== End of Parsed pages ==
+
+- Wikipedia parsing ended at: 2016-09-21 18:15:05.189221
+
+61 characters appeared 940364 times.
+
+First 30 characters:
+[ 0] Char a: 12.508773198463574 %
+[ 1] Char i: 10.969475649854738 %
+[ 2] Char n: 8.815841525196626 %
+[ 3] Char t: 8.80169806585535 %
+[ 4] Char e: 7.8206949649284745 %
+[ 5] Char s: 7.595782058862313 %
+[ 6] Char l: 5.963541777439374 %
+[ 7] Char o: 5.439808414613916 %
+[ 8] Char u: 5.0102938861972595 %
+[ 9] Char k: 4.589712068943515 %
+[10] Char r: 3.1231523112326713 %
+[11] Char ä: 3.041800834570443 %
+[12] Char m: 3.0392486313810396 %
+[13] Char v: 2.156292669647073 %
+[14] Char h: 1.996141919512019 %
+[15] Char j: 1.9248929138078446 %
+[16] Char p: 1.6324529650220552 %
+[17] Char y: 1.6323466232224966 %
+[18] Char d: 1.1981530556252684 %
+[19] Char b: 0.6835650875618378 %
+[20] Char g: 0.5793501239945382 %
+[21] Char c: 0.5056552569005194 %
+[22] Char ö: 0.38931732818355447 %
+[23] Char f: 0.215023118707224 %
+[24] Char w: 0.2106631049253268 %
+[25] Char z: 0.06593191572625068 %
+[26] Char x: 0.024458613898447838 %
+[27] Char š: 0.010421496356729947 %
+[28] Char ž: 0.007869293167326695 %
+[29] Char q: 0.007762951367768225 %
+
+The first 30 characters have an accumulated ratio of 0.9996012182516557.
+
+919 sequences found.
+
+First 512 (typical positive ratio): 0.9985378147555799
+Next 512 (512-1024): 1.0634179955846884e-06
+Rest: 3.881443777498106e-17
+
+- Processing end: 2016-09-21 18:15:05.307164