ruletagger 0.0.1

data/test/docs/doc7.txt

@@ -0,0 +1,101 @@
+What is a stem cell transplant?
+
+Most stem cells are in your bone marrow . You also have some in your blood that circulate from your bone marrow. Bone marrow stem cells turn into red blood cells , white blood cells , or platelets to help your body stay healthy. If your bone marrow is attacked by a disease such as leukemia, it can no longer make normal blood cells. In a stem cell transplant, healthy stem cells are placed in your body through an IV to help your bone marrow start to work right.
+
+When the stem cells come from your own blood or bone marrow, it is called an autologous transplant.
+
+When the stem cells come from another person, it is called an allogeneic transplant . The donor may be a relative or a complete stranger. The important thing is that the donor's blood is closely matched to yours. This is most likely when the donor is your brother or sister.
+
+Stem cells can also be found in your bloodstream and in the blood inside a newborn's umbilical cord .
+
+When is a stem cell transplant needed?
+
+Stem cell transplants are used to:
+
+Treat diseases that damage or destroy the bone marrow. Such diseases include non-Hodgkin's lymphoma and Hodgkin's lymphoma , leukemia , multiple myeloma , and aplastic anemia .
+Restore bone marrow that has been damaged by total body radiation and high doses of chemotherapy used for cancer treatment.
+Other uses for stem cells are being studied, such as the treatment of diabetes , Parkinson's disease , sickle cell disease , and thalassemia .
+
+How do I know if I am a good candidate for a stem cell transplant?
+
+Your doctor will consider your overall health and your age. People who are good candidates usually are younger than 70, do not have other diseases such as heart disease or diabetes, and have a normal kidney and liver. Your doctor will also consider how much your disease has grown and how aggressive your cancer is. People with aggressive cancer that has spread to many areas of the body are not usually thought to be good candidates. Your doctor may also consider whether you have cancer that has come back, such as relapsed non-Hodgkin's lymphoma .
+
+How are stem cells collected?
+
+It depends on where the stem cells come from. If they are taken from the bone marrow, a small amount of the liquid portion of the bone marrow is removed through a needle inserted into the bone. This is done many times to collect enough stem cells for the person getting the donated bone marrow. This is called a bone marrow aspiration . The bone marrow cells are put in a blood bag. They are often frozen for future use.
+
+If they are taken from blood, the growth factor G-CSF may be used to stimulate the growth of new stem cells so they spill over into the blood. G-CSF is a protein that is produced naturally in the body. G-CSF may be used so a transplant can be done as soon as possible and a chemotherapy dose does not have to be lower to allow stem cells to grow. The blood is removed from the vein in one arm and passed through a machine that separates the stem cells. The machine then returns the remaining blood through a needle in the person's other arm. This is called apheresis .
+
+In adults, most autologous transplants use stem cells from blood. In a child, the decision whether to use cells from the bone marrow or the blood depends on the size of the child.
+
+Why are chemotherapy and radiation therapy used before a transplant?
+
+Before you have chemotherapy and radiation, you have blood taken and stored (banked) for later. Then you have chemotherapy and radiation to destroy the diseased cells in your bone marrow. This gets rid of the cancer cells in your bone marrow. Later, when you get your stored blood cells back, those new cells will be able to take over the job of making new blood cells.
+
+How are stem cells transplanted?
+
+An IV is inserted in your neck or chest. The stem cells travel from the blood bag through the IV, into your blood, and to your bone marrow, where they will begin to produce new cells in 1 to 3 weeks. During this time:
+
+You will be in isolation and given antibiotics to prevent or to treat infection. Destroying your bone marrow cells with chemotherapy leaves your body unable to fight infection.
+Your blood will be tested often to check the levels of red blood cells, white blood cells, and platelets in your body.
+You may need to receive several transfusions of blood cells and platelets until your body begins to produce its own.
+You may need more antibiotics or other medicines if you get an infection.
+What To Expect After Treatment
+
+An autologous transplant usually causes fewer problems than an allogeneic transplant. Some people are able to receive part or even all of their treatment in an outpatient clinic. Even if you need to be in a hospital, you will not usually have to stay longer than 3 weeks.
+
+Severe, often life-threatening infection can develop after a stem cell transplant. You will need to take antibiotics for several months to prevent infection.
+
+Your immune system may take 1 to 2 years or longer to recover after a transplant. Bone marrow aspiration or biopsy is used to check your bone marrow. You will need to have many immunizations updated. Check with your doctor to find out which immunizations you will need.
+
+Why It Is Done
+
+Autologous stem cell transplant is used:
+
+After high doses of chemotherapy for killing cancer cells, that have also destroyed your bone marrow. The stem cell transplant gives you back your normal bone marrow.
+To treat diseases that damage the bone marrow, such as Hodgkin's lymphoma , non-Hodgkin's lymphoma , and multiple myeloma .
+Experimentally (and rarely) to treat solid tumors, such as breast cancer or testicular cancer . It is also used to treat tumors that develop from cells producing eggs or sperm (germ cell tumors), such as dysgerminomas and teratomas.
+Experimentally for gene therapy and the treatment of other diseases, such as diabetes .
+How Well It Works
+
+The success of a transplant depends on the type and stage of the disease and your age and general health.
+
+The original disease may come back after the transplant. If relapse occurs after autologous transplant, chemotherapy or other treatments may be used.
+
+Risks
+
+Early complications usually occur within 5 to 10 days and include:
+
+Nausea and vomiting.
+Diarrhea.
+Mouth sores.
+Hair loss.
+Bleeding because of severe reduction in red blood cells, white blood cells, and platelets.
+Infection, such as pneumonia , shingles , or herpes simplex .
+Other possible complications include:
+
+Depression.
+Infertility .
+Cataracts .
+Kidney, lung, and heart complications.
+Recurrence of your cancer.
+Other types of cancer later in life.
+What To Think About
+
+Transplants from your own marrow
+
+Using your own stem cells in a transplant is safer than using someone else's, because your body will not reject your own stem cells. But it also means you are more likely to have a relapse.1 That is because your own marrow or blood may still contain some of the cancer cells you are trying to get rid of. Cells from another donor may work better at attacking any leftover cancer cells still in your body.
+
+Some studies show that treating your marrow or blood with certain drugs before it is put back into your body may increase your chances for getting better. These drugs are given to try to kill any cancer cells that may still be around. Treating your marrow or blood in this way is called purging. Researchers are still studying whether purging is really helpful.1
+
+Specialized hospitals
+
+Not every hospital is able to perform transplants. You may have to travel to a hospital that has special equipment and specially trained doctors and nurses. Transplants are very expensive and are not always covered by insurance.
+
+Complete the special treatment information form (PDF) (What is a PDF document?) to help you understand this treatment.
+
+References
+
+Citations
+
+Greer JP, et al. (2004). Acute myeloid leukemia in adults. In JP Greer et al., eds., Wintrobe's Clinical Hematology, 11th ed., vol. 2, pp. 2097a2142. Baltimore: Williams and Wilkins.

data/test/docs/doc8.txt

@@ -0,0 +1,25 @@
+I seem to be salivating more than usual. I feel like I constantly have to swallow and wipe the corners of my mouth. I am 60 years old and otherwise in good health. What causes excessive saliva?
+
+No name, No state given
+
+Excessive saliva is usually a temporary problem and rarely a cause for concern.
+
+Normally, your salivary glands produce 1 to 2 quarts (0.95 to 1.89 liters) of saliva a day. However, because swallowing occurs continually and unconsciously, you don't generally notice the saliva. If you suddenly seem to have excessive saliva in your mouth, it may be because your salivary glands are making more saliva than usual (sialorrhea) or because you are swallowing less often.
+
+Causes of increased saliva production include:
+
+Pregnancy
+New dentures
+Inflammation of mucous membranes of the mouth (stomatitis)
+A side effect of certain medications, such as clozapine, isoproterenol, pilocarpine and reserpine
+Damage to the nerves that control the salivary glands
+Causes of a decreased ability to swallow or to retain saliva in your mouth include:
+
+Neurological disorders, such as Parkinson's disease or cerebral palsy
+Nasal obstruction, such as due to allergies, sinus infection, nasal polyps and enlarged adenoids
+Gastroesophageal reflux
+Sjogren's syndrome
+Bell's palsy
+If you're concerned about excessive salivation, consult your doctor or dentist. It's important to determine if the problem is due to increased saliva production or a decreased ability to swallow.
+
+Treatment of persistent, excessive salivation is directed at the underlying cause, if possible. Anticholinergic medications, such as transdermal scopolamine, can be used to decrease saliva. Although anticholinergics may help in severe cases of drooling, these medications aren't always effective and can cause side effects.

data/test/docs/doc9.txt

@@ -0,0 +1,84 @@
+If you're an older adult, you may have dismissed trembling hands or a shaking head as a normal part of the aging process. But it's more likely your shakiness is a sign of a movement disorder called essential tremor.
+
+Although essential tremor can affect almost any part of your body, trembling occurs most often in your hands, especially when you try to do simple tasks such as drinking a glass of water, tying your shoelaces, writing or shaving. Sometimes, you may also have trembling of your head, voice or arms.
+
+Essential tremor is the most common of the many movement disorders. It's far more common than Parkinson's disease, with which it may sometimes be confused. Unlike Parkinson's disease, however, essential tremor doesn't lead to serious complications. In fact, the word "essential" in essential tremor means the disorder isn't linked to other diseases.
+
+For some people, essential tremor may be distressing but not debilitating. Others may find that their tremors make it difficult to work, perform everyday tasks that require fine-motor skills or do the things they enjoy. Severe tremors can lead to social withdrawal and isolation. Fortunately, a variety of treatments exist that may help bring your tremors under control.
+
+Signs and symptoms
+
+Essential tremor often begins gradually. Sometimes it appears during adolescence. More often, though, tremors begin in mid- to late life.
+
+The most common sign is a trembling, up-and-down movement of your hands, although your arms, legs, head and even your tongue and voice box (larynx) also may be affected. Most people have tremors in both hands. Some people have tremors in only one hand, though the tremors often progress to include both hands.
+
+Tremors usually occur only when you engage in a voluntary movement, such as drinking a glass of water, writing or threading a needle. Actions requiring fine-motor skills - using utensils or small tools, for example - may be especially difficult. Fatigue, anxiety and temperature extremes make the signs worse, but tremors usually disappear when you're asleep or at rest.
+
+Some people have relatively mild tremors throughout their lives, while others develop more severe tremors and increased disability over time. Effects of worsening tremors may include:
+
+Difficulty holding a cup or glass without spilling
+Difficulty eating normally
+Difficulty putting on makeup or shaving
+Difficulty talking, if your voice box or tongue is affected
+Difficulty writing - handwriting may become increasingly large, shaky and illegible
+The inability to perform actions requiring fine-motor skills, such as playing an instrument or drawing
+Essential tremor vs. Parkinson's disease
+Many people associate tremors with Parkinson's disease, a serious movement disorder. But essential tremor and Parkinson's disease aren't related, and the two conditions differ in key ways:
+
+When tremors occur. Essential tremor of the hands typically occurs when your hands are in use. Tremors from Parkinson's are most prominent when your hands are at your sides or resting in your lap. This type of tremor usually decreases with movement of the hands.
+Associated conditions. Essential tremor doesn't cause other health problems, whereas Parkinson's is associated with a stooped posture, slow movement, a shuffling gait, speech problems other than tremor and sometimes memory loss.
+Parts of body affected. Essential tremor can involve your hands, legs, head and voice. Tremors from Parkinson's typically affect your hands, but not your head or voice.
+Causes
+
+About half of all cases of essential tremor appear to occur because of a genetic mutation. This is referred to as benign familial tremor. Genes are information centers in your cells that control your body's growth, development and function. A mutation in just one gene can greatly alter the way your body works. Researchers have identified two genes that appear to be involved in essential tremor. It's possible that mutations in other genes may also lead to the condition.
+
+Exactly what causes essential tremor in people without a known genetic mutation isn't clear. Doctors do know that the problem occurs in the brain circuits that control your movements. Studies using an imaging technique called positron emission tomography (PET) scanning show that certain parts of the brain - including the thalamus - have increased activity in people with essential tremor. More research is needed to understand the precise mechanism behind the disease.
+
+Risk factors
+
+Benign familial tremors are an autosomal dominant disorder, which means that a defective gene from just one parent is needed to pass on the condition. If you have a parent with a genetic mutation for essential tremor, you have a 50 percent chance of developing the disorder yourself. The only other known risk factor is older age. Although essential tremor can affect people of all ages, it usually appears in late middle age or later.
+
+
+
+When to seek medical advice
+
+See your doctor if tremors make it hard to perform daily activities or prevent you from living your life as fully as you'd like. Your doctor can determine whether essential tremor is causing your problem and may be able to suggest treatments that will ease your symptoms.
+
+Screening and diagnosis
+
+You'll receive a diagnosis of essential tremor only after your doctor has ruled out other possible causes for your symptoms. For that reason, you may undergo blood, urine and neurological tests to check for problems such as thyroid disease, heavy metal poisoning, drug side effects and Parkinson's disease.
+
+In addition, your doctor will take a complete medical history and perform a thorough physical and neurological exam that may include checking your tendon reflexes, your muscle strength and tone, your ability to feel certain sensations, and your posture and coordination.
+
+The tremor itself may be evaluated in several ways, including performance tests in which you're asked to write, drink from a glass or hold a piece of paper.
+
+
+
+Treatment
+
+Most people with essential tremor don't need treatment beyond reassurance that the condition isn't a sign of a more serious disease. Lifestyle changes - which include getting plenty of rest and avoiding stressful situations and stimulants such as caffeine - may help ease the tremors. Most people with essential tremor find that fatigue, anxiety, sleep deprivation and even temperature extremes make their tremors worse.
+
+If lifestyle changes don't help and tremors are keeping you from doing the things you enjoy, your doctor may recommend these options:
+
+Medications
+Medications provide relief from tremors roughly half the time. They include:
+
+Beta blockers. Normally used to treat high blood pressure, beta blockers, such as propranolol (Inderal), help relieve tremors in some people. Because beta blockers are especially likely to cause dizziness, confusion and memory loss in older adults, they may be a better choice for younger people. They may not be an option if you also have asthma, diabetes or certain heart problems.
+Anti-seizure medications. These drugs, especially primidone (Mysoline), may be effective in people who don't respond to beta blockers. The main side effects are drowsiness and flu-like symptoms, which usually disappear within a short time.
+Tranquilizers. Doctors sometimes use drugs such as diazepam (Valium) and alprazolam (Xanax) to treat people whose tremors are made much worse by tension or anxiety. Side effects can include confusion and memory loss. Additionally, these medications should be used with caution because they can be habit-forming.
+Botulinum toxin type A (Botox) injections. You're probably familiar with Botox as a treatment for facial wrinkles, but it can also be useful in treating some types of tremors, especially of the head and voice. Botox injections can improve problems for up to three months at a time. When used to treat hand tremors, Botox can sometimes cause weakness in your fingers.
+Surgery
+Surgery may be an option for people whose tremors are severely disabling and who don't respond to medications. Deep brain stimulation (DBS) is a treatment involving a brain implant device called a thalamic stimulator may be appropriate if you have severe tremors and if medications aren't effective. A pacemaker-like chest unit transmits electrical pulses through a wire to a lead implanted in your thalamus. The pulses, which are painless, may interrupt signals from your thalamus that help cause tremors. You turn the pulse generator on and off by passing a magnet over your chest.
+
+
+
+Self-care
+
+The following suggestions can sometimes help reduce or relieve tremors:
+
+Avoid caffeine. Caffeine can cause your body to produce more adrenaline, which may make your tremors worse. Avoid other stimulants as well.
+Use alcohol sparingly. Some people notice that their tremors improve for up to an hour after they drink alcohol, but drinking isn't a good solution for people with essential tremor. That's because tremors tend to worsen once the effects of alcohol wear off. What's more, larger amounts of alcohol eventually are needed to relieve tremors, which can lead to chronic alcoholism. If you have essential tremor, it's best to drink sparingly or not at all.
+Learn to relax. Stress tends to make tremors worse, and a relaxed state often improves them. Although it's not possible to eliminate all stress from your life, you can change how you react to stressful situations using a range of relaxation techniques. These include deep breathing, progressive muscle relaxation, guided imagery and massage as well as disciplines such as yoga and tai chi. Many people also find that physical exercise - walking, jogging, swimming or biking - is a great stress reliever.
+Rest well. Fatigue can exacerbate tremors. Try to get at least seven hours of sound sleep every night. If you have trouble falling asleep, wake up repeatedly or awaken early and can't go back to sleep, talk to your doctor.
+Strengthen your hands. Your doctor may recommend working with a specialist to learn exercises to promote more stability in your hands and wrists. These usually involve using 1- to 2-pound weights strapped to your wrists. You can continue doing these exercises at home.
+Consider joining a support group. For many people, essential tremor can have serious social and psychological consequences. If the effects of essential tremor are making it hard to live your life as fully as you once did, consider joining a support group for people with the disorder. Support groups aren't for everyone, but you may find it helpful to have the encouragement of people who understand what you're going through. Or you might want to consider seeing a counselor or social worker who can help you meet the challenges of living with essential tremor.

data/test/tagger_test.rb

@@ -0,0 +1,60 @@
+require 'test/unit'
+$:.unshift File.join(File.dirname(__FILE__), "..", "lib")
+$:.unshift File.join(File.dirname(__FILE__), "..", "ext", "rule_tagger")
+
+require 'brill/tagger'
+
+puts "loading tagger..."
+$tagger = Brill::Tagger.new( File.join(File.dirname(__FILE__),"LEXICON"),
+                             File.join(File.dirname(__FILE__),"LEXICALRULEFILE"),
+                             File.join(File.dirname(__FILE__),"CONTEXTUALRULEFILE") )
+puts "tagger loaded!"
+
+class TaggerTest < Test::Unit::TestCase
+SAMPLE_DOC=%q(
+Take an active role in your care
+When it comes to making decisions about the goals and direction of treatment, don't sit back. Work closely and actively with your oncologist and the rest of your medical team.
+Dont overlook clinical trials
+If youre eligible to enroll in clinical trials, select an oncologist who participates in them. Patients who enroll in clinical studies receive closer follow-up, the highest standard-of-care treatment and access to experimental therapies at no extra cost.
+Maximize your nutrition strategy
+Doing your best to eat a healthy, well-balanced diet is vital to prompt healing after surgery and for recovery from radiation or chemotherapy. Many oncology practices employ registered dieticians who can help you optimize your nutrition.
+Steer clear of "natural cures"
+Before trying nutritional supplements or herbal remedies, be sure to discuss your plans with a doctor. Most have not been tested in clinical studies, and some may actually interfere with your treatment.
+Build a stronger body
+Even walking regularly is can help you minimize long-term muscle weakness caused by illness or de-conditioning.
+Focus on overall health
+Patients may be cured of cancer but still face life-threatening medical problems that are underemphasized during cancer treatments, such as diabetes, high blood pressure and heart disease. Continue to monitor your overall health.
+Put the fire out for good
+Smoking impairs healing after surgery and radiation and increases your risk of cardiovascular disease and many types of cancers. Ask your doctor for help identifying and obtaining the most appropriate cessation aids.
+Map a healthy future
+Once youve completed treatment, discuss appropriate follow-up plans with your doctor and keep track of them yourself. Intensified screening over many years is frequently recommended to identify and treat a recurrence early on.
+Share your feelings
+Allow yourself time to discuss the emotional consequences of your illness and treatment with family, friends, your doctor and, if necessary, a professional therapist. Many patients also find antidepressants helpful during treatment.
+Stay connected
+Although many newly diagnosed patients fear they will not be able to keep working during treatment, this is usually not the case. Working, even at a reduced schedule, helps you maintain valuable social connections and weekly structure.
+)
+  def test_simple_tagger
+    pairs = tagger.tag( SAMPLE_DOC )
+    assert_equal [["", ")"], ["", ")"], ["Take", "VB"], ["an", "DT"], ["active", "JJ"], ["role", "NN"], ["in", "IN"], ["your", "PRP$"], ["care", "NN"], ["When", "WRB"], ["it", "PRP"], ["comes", "VBZ"], ["to", "TO"], ["making", "VBG"], ["decisions", "NNS"], ["about", "IN"], ["the", "DT"], ["goals", "NNS"], ["and", "CC"], ["direction", "NN"], ["of", "IN"], ["treatment", "NN"], [",", ","], ["", ")"], ["do", "VBP"], ["", ")"], ["n't", "RB"], ["sit", "VB"], ["back.", "CD"], ["Work", "NN"], ["closely", "RB"], ["and", "CC"], ["actively", "RB"], ["with", "IN"], ["your", "PRP$"], ["oncologist", "NN"], ["and", "CC"], ["the", "DT"], ["rest", "NN"], ["of", "IN"], ["your", "PRP$"], ["medical", "JJ"], ["team.", "NNP"], ["Dont", "NNP"], ["overlook", "VB"], ["clinical", "JJ"], ["trials", "NNS"], ["If", "IN"], ["youre", "NN"], ["eligible", "JJ"], ["to", "TO"], ["enroll", "VB"], ["in", "IN"], ["clinical", "JJ"], ["trials", "NNS"], [",", ","], ["", ")"], ["select", "VB"], ["an", "DT"], ["oncologist", "NN"], ["who", "WP"], ["participates", "VBZ"], ["in", "IN"], ["them.", "JJ"], ["Patients", "NNS"], ["who", "WP"], ["enroll", "VBP"], ["in", "IN"], ["clinical", "JJ"], ["studies", "NNS"], ["receive", "VBP"], ["closer", "JJR"], ["follow-up", "NN"], [",", ","], ["", ")"], ["the", "DT"], ["highest", "JJS"], ["standard-of-care", "JJ"], ["treatment", "NN"], ["and", "CC"], ["access", "NN"], ["to", "TO"], ["experimental", "JJ"], ["therapies", "NNS"], ["at", "IN"], ["no", "DT"], ["extra", "JJ"], ["cost.", "NNP"], ["Maximize", "NNP"], ["your", "PRP$"], ["nutrition", "NN"], ["strategy", "NN"], ["Doing", "NNP"], ["your", "PRP$"], ["best", "JJS"], ["to", "TO"], ["eat", "VB"], ["a", "DT"], ["healthy", "JJ"], [",", ","], ["", ")"], ["well-balanced", "JJ"], ["diet", "NN"], ["is", "VBZ"], ["vital", "JJ"], ["to", "TO"], ["prompt", "VB"], ["healing", "NN"], ["after", "IN"], ["surgery", "NN"], ["and", "CC"], ["for", "IN"], ["recovery", "NN"], ["from", "IN"], ["radiation", "NN"], ["or", "CC"], ["chemotherapy.", "JJ"], ["Many", "JJ"], ["oncology", "NN"], ["practices", "NNS"], ["employ", "VBP"], ["registered", "VBN"], ["dieticians", "NNS"], ["who", "WP"], ["can", "MD"], ["help", "VB"], ["you", "PRP"], ["optimize", "VB"], ["your", "PRP$"], ["nutrition.", "JJ"], ["Steer", "VB"], ["clear", "JJ"], ["of", "IN"], ["", ")"], ["``", "``"], ["natural", "JJ"], ["cures", "NNS"], ["''", "''"], ["", ")"], ["Before", "IN"], ["trying", "VBG"], ["nutritional", "JJ"], ["supplements", "NNS"], ["or", "CC"], ["herbal", "JJ"], ["remedies", "NNS"], [",", ","], ["", ")"], ["be", "VB"], ["sure", "JJ"], ["to", "TO"], ["discuss", "VB"], ["your", "PRP$"], ["plans", "NNS"], ["with", "IN"], ["a", "DT"], ["doctor.", "JJ"], ["Most", "JJS"], ["have", "VBP"], ["not", "RB"], ["been", "VBN"], ["tested", "VBN"], ["in", "IN"], ["clinical", "JJ"], ["studies", "NNS"], [",", ","], ["", ")"], ["and", "CC"], ["some", "DT"], ["may", "MD"], ["actually", "RB"], ["interfere", "VB"], ["with", "IN"], ["your", "PRP$"], ["treatment.", "JJ"], ["Build", "VB"], ["a", "DT"], ["stronger", "JJR"], ["body", "NN"], ["Even", "RB"], ["walking", "VBG"], ["regularly", "RB"], ["is", "VBZ"], ["can", "MD"], ["help", "VB"], ["you", "PRP"], ["minimize", "VB"], ["long-term", "JJ"], ["muscle", "NN"], ["weakness", "NN"], ["caused", "VBN"], ["by", "IN"], ["illness", "NN"], ["or", "CC"], ["de-conditioning.", "NNP"], ["Focus", "NNP"], ["on", "IN"], ["overall", "JJ"], ["health", "NN"], ["Patients", "NNS"], ["may", "MD"], ["be", "VB"], ["cured", "VBN"], ["of", "IN"], ["cancer", "NN"], ["but", "CC"], ["still", "JJ"], ["face", "NN"], ["life-threatening", "JJ"], ["medical", "JJ"], ["problems", "NNS"], ["that", "WDT"], ["are", "VBP"], ["underemphasized", "JJ"], ["during", "IN"], ["cancer", "NN"], ["treatments", "NNS"], [",", ","], ["", ")"], ["such", "JJ"], ["as", "IN"], ["diabetes", "NN"], [",", ","], ["", ")"], ["high", "JJ"], ["blood", "NN"], ["pressure", "NN"], ["and", "CC"], ["heart", "NN"], ["disease.", "JJ"], ["Continue", "VB"], ["to", "TO"], ["monitor", "VB"], ["your", "PRP$"], ["overall", "JJ"], ["health.", "JJ"], ["Put", "NN"], ["the", "DT"], ["fire", "NN"], ["out", "IN"], ["for", "IN"], ["good", "JJ"], ["Smoking", "NNP"], ["impairs", "NNS"], ["healing", "NN"], ["after", "IN"], ["surgery", "NN"], ["and", "CC"], ["radiation", "NN"], ["and", "CC"], ["increases", "NNS"], ["your", "PRP$"], ["risk", "NN"], ["of", "IN"], ["cardiovascular", "JJ"], ["disease", "NN"], ["and", "CC"], ["many", "JJ"], ["types", "NNS"], ["of", "IN"], ["cancers.", "CD"], ["Ask", "VB"], ["your", "PRP$"], ["doctor", "NN"], ["for", "IN"], ["help", "NN"], ["identifying", "VBG"], ["and", "CC"], ["obtaining", "VBG"], ["the", "DT"], ["most", "RBS"], ["appropriate", "JJ"], ["cessation", "NN"], ["aids.", "NNP"], ["Map", "NNP"], ["a", "DT"], ["healthy", "JJ"], ["future", "NN"], ["Once", "RB"], ["youve", "VBP"], ["completed", "VBN"], ["treatment", "NN"], [",", ","], ["", ")"], ["discuss", "VB"], ["appropriate", "JJ"], ["follow-up", "NN"], ["plans", "NNS"], ["with", "IN"], ["your", "PRP$"], ["doctor", "NN"], ["and", "CC"], ["keep", "VB"], ["track", "NN"], ["of", "IN"], ["them", "PRP"], ["yourself.", "CD"], ["Intensified", "JJ"], ["screening", "NN"], ["over", "IN"], ["many", "JJ"], ["years", "NNS"], ["is", "VBZ"], ["frequently", "RB"], ["recommended", "VBN"], ["to", "TO"], ["identify", "VB"], ["and", "CC"], ["treat", "VB"], ["a", "DT"], ["recurrence", "NN"], ["early", "JJ"], ["on.", "CD"], ["Share", "VB"], ["your", "PRP$"], ["feelings", "NNS"], ["Allow", "VB"], ["yourself", "PRP"], ["time", "NN"], ["to", "TO"], ["discuss", "VB"], ["the", "DT"], ["emotional", "JJ"], ["consequences", "NNS"], ["of", "IN"], ["your", "PRP$"], ["illness", "NN"], ["and", "CC"], ["treatment", "NN"], ["with", "IN"], ["family", "NN"], [",", ","], ["", ")"], ["friends", "NNS"], [",", ","], ["", ")"], ["your", "PRP$"], ["doctor", "NN"], ["and", "CC"], [",", ","], ["", ")"], ["if", "IN"], ["necessary", "JJ"], [",", ","], ["", ")"], ["a", "DT"], ["professional", "JJ"], ["therapist.", "JJ"], ["Many", "JJ"], ["patients", "NNS"], ["also", "RB"], ["find", "VBP"], ["antidepressants", "NNS"], ["helpful", "JJ"], ["during", "IN"], ["treatment.", "JJ"], ["Stay", "VB"], ["connected", "VBN"], ["Although", "IN"], ["many", "JJ"], ["newly", "RB"], ["diagnosed", "VBN"], ["patients", "NNS"], ["fear", "VBP"], ["they", "PRP"], ["will", "MD"], ["not", "RB"], ["be", "VB"], ["able", "JJ"], ["to", "TO"], ["keep", "VB"], ["working", "VBG"], ["during", "IN"], ["treatment", "NN"], [",", ","], ["", ")"], ["this", "DT"], ["is", "VBZ"], ["usually", "RB"], ["not", "RB"], ["the", "DT"], ["case.", "CD"], ["Working", "NNP"], [",", ","], ["", ")"], ["even", "RB"], ["at", "IN"], ["a", "DT"], ["reduced", "VBN"], ["schedule", "NN"], [",", ","], ["", ")"], ["helps", "VBZ"], ["you", "PRP"], ["maintain", "VBP"], ["valuable", "JJ"], ["social", "JJ"], ["connections", "NNS"], ["and", "CC"], ["weekly", "JJ"], ["structure", "NN"], [".", "."]], pairs 
+    #puts pairs.inspect
+    # enable these lines for memory leak testing
+    $tagger = nil
+    ObjectSpace.garbage_collect
+  end
+
+  def test_multiple_docs
+    timer = Time.now
+    count = 0
+    Dir["#{File.dirname(__FILE__)}/docs/doc*"].each do|doc|
+      tagger.tag( File.read( doc ) )
+      count += 1
+    end
+    duration = Time.now - timer
+    puts "time: #{duration} sec #{count.to_f/duration} docs/sec"
+  end
+
+private
+  def tagger
+    $tagger
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,2 @@
+require 'test/unit'
+require File.dirname(__FILE__) + '/../lib/rbtagger'

data/tools/rakehelp.rb

@@ -0,0 +1,113 @@
+# This final came directly from mongrel 1.0.1 source
+# with a few modifications to support some of my network tests
+# Also, i have figured out yet if this should remain so much a clone of the mongrel tree
+# or become a plugin, need to review more closely how that works
+
+def make(makedir)
+  Dir.chdir(makedir) do
+    sh(PLATFORM =~ /win32/ ? 'nmake' : 'make')
+  end
+end
+
+def extconf(dir)
+  Dir.chdir(dir) do ruby "extconf.rb" end
+end
+
+def setup_tests
+  Rake::TestTask.new do |t|
+    t.test_files = FileList["test/*_test.rb"]
+    t.verbose = true
+  end
+end
+
+
+def setup_clean otherfiles
+  files = ['build/*', '**/*.o', '**/*.so', '**/*.a', 'lib/*-*', '**/*.log'] + otherfiles
+  CLEAN.include(files)
+end
+
+
+def setup_rdoc files
+  Rake::RDocTask.new do |rdoc|
+    rdoc.rdoc_dir = 'doc/rdoc'
+    rdoc.options << '--line-numbers'
+    rdoc.rdoc_files.add(files)
+  end
+end
+
+
+def setup_extension(dir, extension)
+  ext = "ext/#{dir}"
+  ext_so = "#{ext}/#{extension}.#{Config::CONFIG['DLEXT']}"
+  ext_files = FileList[
+    "#{ext}/*.c",
+    "#{ext}/*.h",
+    "#{ext}/extconf.rb",
+    "#{ext}/Makefile",
+    "lib"
+  ]
+
+  task "lib" do
+    directory "lib"
+  end
+
+  desc "Builds just the #{extension} extension"
+  task extension.to_sym => ["#{ext}/Makefile", ext_so ]
+
+  file "#{ext}/Makefile" => ["#{ext}/extconf.rb"] do
+    extconf "#{ext}"
+  end
+
+  file ext_so => ext_files do
+    make "#{ext}"
+    cp ext_so, "lib"
+  end
+end
+
+
+def base_gem_spec(pkg_name, pkg_version)
+  rm_rf "test/coverage"
+  pkg_version = pkg_version
+  pkg_name    = pkg_name
+  pkg_file_name = "#{pkg_name}-#{pkg_version}"
+  Gem::Specification.new do |s|
+    s.name = pkg_name
+    s.version = pkg_version
+    s.platform = Gem::Platform::RUBY
+    s.has_rdoc = true
+    s.extra_rdoc_files = [ "README" ]
+
+    s.files = %w(COPYING LICENSE README Rakefile) +
+      Dir.glob("{bin,doc/rdoc,test}/**/*") + 
+      Dir.glob("ext/**/*.{h,c,rb,rl}") +
+      Dir.glob("{examples,tools,lib}/**/*.rb")
+
+    s.require_path = "lib"
+    s.extensions = FileList["ext/**/extconf.rb"].to_a
+    s.bindir = "bin"
+  end
+end
+
+def setup_gem(pkg_name, pkg_version)
+  spec = base_gem_spec(pkg_name, pkg_version)
+  yield spec if block_given?
+
+  Rake::GemPackageTask.new(spec) do |p|
+    p.gem_spec = spec
+    p.need_tar = true if RUBY_PLATFORM !~ /mswin/
+  end
+end
+
+# Conditional require rcov/rcovtask if present
+begin
+  require 'rcov/rcovtask'
+  
+  Rcov::RcovTask.new do |t|
+    t.test_files = FileList['test/unit/*_test.rb'] + FileList["test/integration/*_test.rb"]
+    t.rcov_opts << "-x /usr"
+    t.output_dir = "test/coverage"
+		t.verbose = true
+  end
+rescue Object => e
+	puts e.message
+end

data/website/index.html

@@ -0,0 +1,113 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+  <link rel="stylesheet" href="stylesheets/screen.css" type="text/css" media="screen" />
+  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+  <title>
+      rbtagger
+  </title>
+  <script src="javascripts/rounded_corners_lite.inc.js" type="text/javascript"></script>
+<style>
+
+</style>
+  <script type="text/javascript">
+    window.onload = function() {
+      settings = {
+          tl: { radius: 10 },
+          tr: { radius: 10 },
+          bl: { radius: 10 },
+          br: { radius: 10 },
+          antiAlias: true,
+          autoPad: true,
+          validTags: ["div"]
+      }
+      var versionBox = new curvyCorners(settings, document.getElementById("version"));
+      versionBox.applyCornersToAll();
+    }
+  </script>
+</head>
+<body>
+<div id="main">
+
+    <h1>rbtagger</h1>
+    <div id="version" class="clickable" onclick='document.location = "http://rubyforge.org/projects/ruletagger"; return false'>
+      <p>Get Version</p>
+      <a href="http://rubyforge.org/projects/ruletagger" class="numbers">0.0.1</a>
+    </div>
+    <h4 style="float:right;padding-right:10px;"> &#x2192; &#8216;rb-brill-tagger&#8217;</h4>
+
+	<h2>What</h2>
+
+
+	<p>A Simple Ruby Rule-Based Part of Speech Tagger</p>
+
+
+	<p>This work is based on the work of Eric Brill</p>
+
+
+	<h2>Installing</h2>
+
+
+	<p><pre class='syntax'>sudo gem install rbtagger</pre></p>
+
+
+	<h2>The basics</h2>
+
+
+	<p><pre class='syntax'>
+<span class="ident">tagger</span> <span class="punct">=</span> <span class="constant">Brill</span><span class="punct">::</span><span class="constant">Tagger</span><span class="punct">.</span><span class="ident">new</span><span class="punct">(</span> <span class="constant">File</span><span class="punct">.</span><span class="ident">join</span><span class="punct">(</span><span class="constant">File</span><span class="punct">.</span><span class="ident">dirname</span><span class="punct">(</span><span class="constant">__FILE__</span><span class="punct">),&quot;</span><span class="string">LEXICON</span><span class="punct">&quot;),</span>
+                            <span class="constant">File</span><span class="punct">.</span><span class="ident">join</span><span class="punct">(</span><span class="constant">File</span><span class="punct">.</span><span class="ident">dirname</span><span class="punct">(</span><span class="constant">__FILE__</span><span class="punct">),&quot;</span><span class="string">LEXICALRULEFILE</span><span class="punct">&quot;),</span>
+                            <span class="constant">File</span><span class="punct">.</span><span class="ident">join</span><span class="punct">(</span><span class="constant">File</span><span class="punct">.</span><span class="ident">dirname</span><span class="punct">(</span><span class="constant">__FILE__</span><span class="punct">),&quot;</span><span class="string">CONTEXTUALRULEFILE</span><span class="punct">&quot;)</span> <span class="punct">)</span>
+<span class="ident">docs</span><span class="punct">.</span><span class="ident">each</span> <span class="keyword">do</span><span class="punct">|</span><span class="ident">doc</span><span class="punct">|</span>
+  <span class="ident">tagger</span><span class="punct">.</span><span class="ident">tag</span><span class="punct">(</span> <span class="constant">File</span><span class="punct">.</span><span class="ident">read</span><span class="punct">(</span> <span class="ident">doc</span> <span class="punct">)</span> <span class="punct">)</span>
+<span class="keyword">end</span>
+</pre></p>
+
+
+	<h2>Forum</h2>
+
+
+	<p><a href="http://groups.google.com/group/rb-brill-tagger">http://groups.google.com/group/rb-brill-tagger</a></p>
+
+
+	<h2>How to submit patches</h2>
+
+
+	<p>Read the <a href="http://drnicwilliams.com/2007/06/01/8-steps-for-fixing-other-peoples-code/">8 steps for fixing other people&#8217;s code</a> and for section <a href="http://drnicwilliams.com/2007/06/01/8-steps-for-fixing-other-peoples-code/#8b-google-groups">8b: Submit patch to Google Groups</a>, use the Google Group above.</p>
+
+
+	<ul>
+	<li>github: <a href="http://github.com/taf2/rb-brill-tagger/tree/master">http://github.com/taf2/rb-brill-tagger/tree/master</a></li>
+	</ul>
+
+
+<pre>git clone git://github.com/taf2/rb-brill-tagger.git</pre>
+
+	<h3>Build and test instructions</h3>
+
+
+<pre>cd rb-brill-tagger
+rake test
+rake install_gem</pre>
+
+	<h2>License</h2>
+
+
+	<p>This code is free to use under the terms of the <span class="caps">MIT</span> license.</p>
+
+
+	<h2>Contact</h2>
+
+
+	<p>Comments are welcome. Send an email to <a href="mailto:FIXME"><span class="caps">FIXME</span> full name</a> email via the <a href="http://groups.google.com/group/rb-brill-tagger">forum</a></p>
+    <p class="coda">
+      <a href="FIXME email">FIXME full name</a>, 14th May 2008<br>
+      Theme extended from <a href="http://rb2js.rubyforge.org/">Paul Battley</a>
+    </p>
+</div>
+
+<!-- insert site tracking codes here, like Google Urchin -->
+
+</body>
+</html>

data/website/index.txt

@@ -0,0 +1,53 @@
+h1. rbtagger
+
+<h4 style="float:right;padding-right:10px;"> &#x2192; 'rb-brill-tagger'</h4>
+
+
+h2. What
+
+A Simple Ruby Rule-Based Part of Speech Tagger
+
+This work is based on the work of Eric Brill
+
+h2. Installing
+
+<pre syntax="bash">sudo gem install rbtagger</pre>
+
+h2. The basics
+
+<pre syntax="ruby">
+tagger = Brill::Tagger.new( File.join(File.dirname(__FILE__),"LEXICON"),
+                            File.join(File.dirname(__FILE__),"LEXICALRULEFILE"),
+                            File.join(File.dirname(__FILE__),"CONTEXTUALRULEFILE") )
+docs.each do|doc|
+  tagger.tag( File.read( doc ) )
+end
+</pre>
+
+h2. Forum
+
+"http://groups.google.com/group/rb-brill-tagger":http://groups.google.com/group/rb-brill-tagger
+
+h2. How to submit patches
+
+Read the "8 steps for fixing other people's code":http://drnicwilliams.com/2007/06/01/8-steps-for-fixing-other-peoples-code/ and for section "8b: Submit patch to Google Groups":http://drnicwilliams.com/2007/06/01/8-steps-for-fixing-other-peoples-code/#8b-google-groups, use the Google Group above.
+
+* github: "http://github.com/taf2/rb-brill-tagger/tree/master":http://github.com/taf2/rb-brill-tagger/tree/master
+
+<pre>git clone git://github.com/taf2/rb-brill-tagger.git</pre>
+
+h3. Build and test instructions
+
+<pre>cd rb-brill-tagger
+rake test
+rake install_gem</pre>
+
+
+h2. License
+
+This code is free to use under the terms of the MIT license. 
+
+h2. Contact
+
+Comments are welcome. Send an email to "FIXME full name":mailto:FIXME email via the "forum":http://groups.google.com/group/rb-brill-tagger
+