RubyGems - rroonga - Versions diffs - 4.0.6-x86-mingw32 → 4.0.7-x86-mingw32 - Mend

rroonga 4.0.6-x86-mingw32 → 4.0.7-x86-mingw32

Files changed (598) hide show

data/vendor/local/share/doc/groonga/en/html/reference/tables.html CHANGED Viewed

@@ -7,7 +7,7 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>7.5. Tables &mdash; Groonga v4.0.6-211-g8c0bb0b documentation</title>
+    <title>7.5. Tables &mdash; Groonga v4.0.8 documentation</title>
     <link rel="stylesheet" href="../_static/groonga.css" type="text/css" />
     <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -15,7 +15,7 @@
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    '../',
-        VERSION:     '4.0.6-211-g8c0bb0b',
+        VERSION:     '4.0.8',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true
@@ -25,12 +25,12 @@
     <script type="text/javascript" src="../_static/underscore.js"></script>
     <script type="text/javascript" src="../_static/doctools.js"></script>
     <link rel="shortcut icon" href="../_static/favicon.ico"/>
-    <link rel="top" title="Groonga v4.0.6-211-g8c0bb0b documentation" href="../index.html" />
+    <link rel="top" title="Groonga v4.0.8 documentation" href="../index.html" />
     <link rel="up" title="7. Reference manual" href="../reference.html" />
     <link rel="next" title="7.6. Column" href="column.html" />
     <link rel="prev" title="7.4. データ型" href="types.html" />
   </head>
-  <body>
+  <body role="document">
 <div class="header">
   <h1 class="title">
     <a id="top-link" href="../index.html">
@@ -48,7 +48,7 @@
 </div>
-    <div class="related">
+    <div class="related" role="navigation" aria-label="related navigation">
       <h3>Navigation</h3>
       <ul>
         <li class="right" style="margin-right: 10px">
@@ -60,7 +60,7 @@
         <li class="right" >
           <a href="types.html" title="7.4. データ型"
              accesskey="P">previous</a> |</li>
-        <li><a href="../index.html">Groonga v4.0.6-211-g8c0bb0b documentation</a> &raquo;</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
           <li><a href="../reference.html" accesskey="U">7. Reference manual</a> &raquo;</li>
       </ul>
     </div>
@@ -68,23 +68,23 @@
     <div class="document">
       <div class="documentwrapper">
         <div class="bodywrapper">
-          <div class="body">
+          <div class="body" role="main">
   <div class="section" id="tables">
 <h1>7.5. Tables<a class="headerlink" href="#tables" title="Permalink to this headline">¶</a></h1>
 <div class="section" id="summary">
 <h2>7.5.1. Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>
 <p>Table in Groonga manages relation between ID and key. Groonga provides
-four table types. They are <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt>, <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt>,
-<tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> and <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>.</p>
-<p>All tables except <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> provides both fast ID search by key
-and fast key search by ID. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> doesn't support
-key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> only manages ID. So <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> doesn't
+four table types. They are <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code>, <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code>,
+<code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> and <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>.</p>
+<p>All tables except <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> provides both fast ID search by key
+and fast key search by ID. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> doesn't support
+key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> only manages ID. So <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> doesn't
 provides ID search and key search.</p>
 </div>
 <div class="section" id="characteristics">
 <h2>7.5.2. Characteristics<a class="headerlink" href="#characteristics" title="Permalink to this headline">¶</a></h2>
-<p>Here is a chracteristic table of all tables in Groonga. (<tt class="docutils literal"><span class="pre">TABLE_</span></tt>
+<p>Here is a chracteristic table of all tables in Groonga. (<code class="docutils literal"><span class="pre">TABLE_</span></code>
 prefix is omitted in the table.)</p>
 <table border="1" class="docutils">
 <colgroup>
@@ -96,10 +96,10 @@ prefix is omitted in the table.)</p>
 </colgroup>
 <thead valign="bottom">
 <tr class="row-odd"><th class="head">&nbsp;</th>
-<th class="head"><tt class="docutils literal"><span class="pre">NO_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">HASH_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">PAT_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">DAT_KEY</span></tt></th>
+<th class="head"><code class="docutils literal"><span class="pre">NO_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">HASH_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">PAT_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">DAT_KEY</span></code></th>
 </tr>
 </thead>
 <tbody valign="top">
@@ -200,36 +200,36 @@ search</td>
 </tbody>
 </table>
 <div class="section" id="table-no-key">
-<h3>7.5.2.1. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt><a class="headerlink" href="#table-no-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is very fast and very small but it doesn't support
-key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is a only table that doesn't support key.</p>
-<p>You cannot use <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> for lexicon for fulltext search
-because lexicon stores tokens as key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is useful for
+<h3>7.5.2.1. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code><a class="headerlink" href="#table-no-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is very fast and very small but it doesn't support
+key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is a only table that doesn't support key.</p>
+<p>You cannot use <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> for lexicon for fulltext search
+because lexicon stores tokens as key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is useful for
 no key records such as log.</p>
 </div>
 <div class="section" id="table-hash-key">
-<h3>7.5.2.2. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt><a class="headerlink" href="#table-hash-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is fast but it doesn't support advanced search
+<h3>7.5.2.2. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code><a class="headerlink" href="#table-hash-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is fast but it doesn't support advanced search
 functions such as common prefix search and predictive search.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is useful for index for exact search such as tag
+<p><code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is useful for index for exact search such as tag
 search.</p>
 </div>
 <div class="section" id="table-pat-key">
-<h3>7.5.2.3. <tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt><a class="headerlink" href="#table-pat-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> is small and supports advanced search functions.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> is useful for lexicon for fulltext search and
+<h3>7.5.2.3. <code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code><a class="headerlink" href="#table-pat-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> is small and supports advanced search functions.</p>
+<p><code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> is useful for lexicon for fulltext search and
 index for range search.</p>
 </div>
 <div class="section" id="table-dat-key">
-<h3>7.5.2.4. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt><a class="headerlink" href="#table-dat-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is fast and supports key update but it is large. It
-is not suitable for storing many records. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is a only
+<h3>7.5.2.4. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code><a class="headerlink" href="#table-dat-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is fast and supports key update but it is large. It
+is not suitable for storing many records. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is a only
 table that supports key update.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is used in Groonga database. Groonga database needs
-to convert object name such as <tt class="docutils literal"><span class="pre">ShortText</span></tt>, <tt class="docutils literal"><span class="pre">TokenBigram</span></tt> and
+<p><code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is used in Groonga database. Groonga database needs
+to convert object name such as <code class="docutils literal"><span class="pre">ShortText</span></code>, <code class="docutils literal"><span class="pre">TokenBigram</span></code> and
 table names to object ID. And Groonga database needs to rename object
-name. Those features are implemented by <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>. The number
-of objects is small. So large data size demerit of <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>
+name. Those features are implemented by <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>. The number
+of objects is small. So large data size demerit of <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>
 can be ignored.</p>
 </div>
 </div>
@@ -255,8 +255,8 @@ database.</p>
 <h3>7.5.4.2. Temporary table<a class="headerlink" href="#temporary-table" title="Permalink to this headline">¶</a></h3>
 <p>Temporary table is anonymous. Records in temporary table are deleted
 after closing table. Temporary table is used to store search result,
-sort result, group (drilldown) result and so on. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is
-used for search result and group result. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is used for
+sort result, group (drilldown) result and so on. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is
+used for search result and group result. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is used for
 sort result.</p>
 </div>
 </div>
@@ -266,7 +266,7 @@ sort result.</p>
 more records in a table.</p>
 <p>The max number of a key size is 4096byte. You cannot use 4097byte or
 larger key. You can use column instead of key for 4097byte or larger
-size data. <tt class="docutils literal"><span class="pre">Text</span></tt> and <tt class="docutils literal"><span class="pre">LargeText</span></tt> types supports 4097byte or
+size data. <code class="docutils literal"><span class="pre">Text</span></code> and <code class="docutils literal"><span class="pre">LargeText</span></code> types supports 4097byte or
 larger size data.</p>
 <p>The max number of total key size is 4GiB. You need to split a table,
 split a database (sharding) or reduce each key size to handle 4GiB or
@@ -284,17 +284,17 @@ more larger total key size.</p>
           </div>
         </div>
       </div>
-      <div class="sphinxsidebar">
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
         <div class="sphinxsidebarwrapper">
   <h3><a href="../index.html">Table Of Contents</a></h3>
   <ul>
 <li><a class="reference internal" href="#">7.5. Tables</a><ul>
 <li><a class="reference internal" href="#summary">7.5.1. Summary</a></li>
 <li><a class="reference internal" href="#characteristics">7.5.2. Characteristics</a><ul>
-<li><a class="reference internal" href="#table-no-key">7.5.2.1. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-hash-key">7.5.2.2. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-pat-key">7.5.2.3. <tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-dat-key">7.5.2.4. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt></a></li>
+<li><a class="reference internal" href="#table-no-key">7.5.2.1. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-hash-key">7.5.2.2. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-pat-key">7.5.2.3. <code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-dat-key">7.5.2.4. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code></a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#record-id">7.5.3. Record ID</a></li>
@@ -315,12 +315,14 @@ more larger total key size.</p>
   <h4>Next topic</h4>
   <p class="topless"><a href="column.html"
                         title="next chapter">7.6. Column</a></p>
-  <h3>This Page</h3>
-  <ul class="this-page-menu">
-    <li><a href="../_sources/reference/tables.txt"
-           rel="nofollow">Show Source</a></li>
-  </ul>
-<div id="searchbox" style="display: none">
+  <div role="note" aria-label="source link">
+    <h3>This Page</h3>
+    <ul class="this-page-menu">
+      <li><a href="../_sources/reference/tables.txt"
+            rel="nofollow">Show Source</a></li>
+    </ul>
+   </div>
+<div id="searchbox" style="display: none" role="search">
   <h3>Quick search</h3>
     <form class="search" action="../search.html" method="get">
       <input type="text" name="q" />
@@ -337,7 +339,7 @@ more larger total key size.</p>
       </div>
       <div class="clearer"></div>
     </div>
-    <div class="related">
+    <div class="related" role="navigation" aria-label="related navigation">
       <h3>Navigation</h3>
       <ul>
         <li class="right" style="margin-right: 10px">
@@ -349,11 +351,11 @@ more larger total key size.</p>
         <li class="right" >
           <a href="types.html" title="7.4. データ型"
              >previous</a> |</li>
-        <li><a href="../index.html">Groonga v4.0.6-211-g8c0bb0b documentation</a> &raquo;</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
           <li><a href="../reference.html" >7. Reference manual</a> &raquo;</li>
       </ul>
     </div>
-    <div class="footer">
+    <div class="footer" role="contentinfo">
         &copy; Copyright 2009-2014, Brazil, Inc.
     </div>
   </body>

data/vendor/local/share/doc/groonga/en/html/reference/token_filters.html ADDED Viewed

@@ -0,0 +1,323 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+  <head>
+    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+    <title>7.9. Token filters &mdash; Groonga v4.0.8 documentation</title>
+    <link rel="stylesheet" href="../_static/groonga.css" type="text/css" />
+    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+    <script type="text/javascript">
+      var DOCUMENTATION_OPTIONS = {
+        URL_ROOT:    '../',
+        VERSION:     '4.0.8',
+        COLLAPSE_INDEX: false,
+        FILE_SUFFIX: '.html',
+        HAS_SOURCE:  true
+      };
+    </script>
+    <script type="text/javascript" src="../_static/jquery.js"></script>
+    <script type="text/javascript" src="../_static/underscore.js"></script>
+    <script type="text/javascript" src="../_static/doctools.js"></script>
+    <link rel="shortcut icon" href="../_static/favicon.ico"/>
+    <link rel="top" title="Groonga v4.0.8 documentation" href="../index.html" />
+    <link rel="up" title="7. Reference manual" href="../reference.html" />
+    <link rel="next" title="7.10. Query expanders" href="query_expanders.html" />
+    <link rel="prev" title="7.8. Tokenizers" href="tokenizers.html" />
+  </head>
+  <body role="document">
+<div class="header">
+  <h1 class="title">
+    <a id="top-link" href="../index.html">
+      <span class="project">groonga</span>
+      <span class="separator">-</span>
+      <span class="description">An open-source fulltext search engine and column store.</span>
+    </a>
+  </h1>
+  <div class="other-language-links">
+    <ul>
+      <li><a href="../../../ja/html/reference/token_filters.html"><img src="../_static/jp.png" alt="日本語">日本語版はこちら</a></li>
+    </ul>
+  </div>
+</div>
+    <div class="related" role="navigation" aria-label="related navigation">
+      <h3>Navigation</h3>
+      <ul>
+        <li class="right" style="margin-right: 10px">
+          <a href="../genindex.html" title="General Index"
+             accesskey="I">index</a></li>
+        <li class="right" >
+          <a href="query_expanders.html" title="7.10. Query expanders"
+             accesskey="N">next</a> |</li>
+        <li class="right" >
+          <a href="tokenizers.html" title="7.8. Tokenizers"
+             accesskey="P">previous</a> |</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
+          <li><a href="../reference.html" accesskey="U">7. Reference manual</a> &raquo;</li>
+      </ul>
+    </div>
+    <div class="document">
+      <div class="documentwrapper">
+        <div class="bodywrapper">
+          <div class="body" role="main">
+  <div class="section" id="token-filters">
+<h1>7.9. Token filters<a class="headerlink" href="#token-filters" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="summary">
+<h2>7.9.1. Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>
+<p>Groonga has token filter module that some processes tokenized token.</p>
+<p>Token filter module can be added as a plugin.</p>
+<p>You can customize tokenized token by registering your token filters plugins to Groonga.</p>
+<p>A table can have zero or more token filters. You can attach token
+filters to a table by <a class="reference internal" href="commands/table_create.html#table-create-token-filters"><em>token_filters</em></a> option in
+<a class="reference internal" href="commands/table_create.html"><em>table_create</em></a>.</p>
+<p>Here is an example <code class="docutils literal"><span class="pre">table_create</span></code> that uses <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code>
+token filter module:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStopWord
+# [[0,0.0,0.0],true]
+</pre></div>
+</div>
+</div>
+<div class="section" id="available-token-filters">
+<h2>7.9.2. Available token filters<a class="headerlink" href="#available-token-filters" title="Permalink to this headline">¶</a></h2>
+<p>Here is the list of available token filters:</p>
+<ul class="simple">
+<li><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code></li>
+<li><code class="docutils literal"><span class="pre">TokenFilterStem</span></code></li>
+</ul>
+<div class="section" id="tokenfilterstopword">
+<span id="token-filter-stop-word"></span><h3>7.9.2.1. <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code><a class="headerlink" href="#tokenfilterstopword" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> removes stop words from tokenized token
+in searching the documents.</p>
+<p><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> can specify stop word after adding the
+documents because it removes token in searching the documents.</p>
+<p>The stop word is specified <code class="docutils literal"><span class="pre">is_stop_word</span></code> column on lexicon table.</p>
+<p>Here is an example that uses <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> token filter:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>register token_filters/stop_word
+# [[0,0.0,0.0],true]
+table_create Memos TABLE_NO_KEY
+# [[0,0.0,0.0],true]
+column_create Memos content COLUMN_SCALAR ShortText
+# [[0,0.0,0.0],true]
+table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStopWord
+# [[0,0.0,0.0],true]
+column_create Terms memos_content COLUMN_INDEX|WITH_POSITION Memos content
+# [[0,0.0,0.0],true]
+column_create Terms is_stop_word COLUMN_SCALAR Bool
+# [[0,0.0,0.0],true]
+load --table Terms
+[
+{&quot;_key&quot;: &quot;and&quot;, &quot;is_stop_word&quot;: true}
+]
+# [[0,0.0,0.0],1]
+load --table Memos
+[
+{&quot;content&quot;: &quot;Hello&quot;},
+{&quot;content&quot;: &quot;Hello and Good-bye&quot;},
+{&quot;content&quot;: &quot;Good-bye&quot;}
+]
+# [[0,0.0,0.0],3]
+select Memos --match_columns content --query &quot;Hello and&quot;
+# [
+#   [
+#     0,
+#     0.0,
+#     0.0
+#   ],
+#   [
+#     [
+#       [
+#         2
+#       ],
+#       [
+#         [
+#           &quot;_id&quot;,
+#           &quot;UInt32&quot;
+#         ],
+#         [
+#           &quot;content&quot;,
+#           &quot;ShortText&quot;
+#         ]
+#       ],
+#       [
+#         1,
+#         &quot;Hello&quot;
+#       ],
+#       [
+#         2,
+#         &quot;Hello and Good-bye&quot;
+#       ]
+#     ]
+#   ]
+# ]
+</pre></div>
+</div>
+<p><code class="docutils literal"><span class="pre">and</span></code> token is marked as stop word in <code class="docutils literal"><span class="pre">Terms</span></code> table.</p>
+<p><code class="docutils literal"><span class="pre">&quot;Hello&quot;</span></code> that doesn't have <code class="docutils literal"><span class="pre">and</span></code> in content is matched. Because
+<code class="docutils literal"><span class="pre">and</span></code> is a stop word and <code class="docutils literal"><span class="pre">and</span></code> is removed from query.</p>
+</div>
+<div class="section" id="tokenfilterstem">
+<span id="token-filter-stem"></span><h3>7.9.2.2. <code class="docutils literal"><span class="pre">TokenFilterStem</span></code><a class="headerlink" href="#tokenfilterstem" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TokenFilterStem</span></code> stems tokenized token.</p>
+<p>Here is an example that uses <code class="docutils literal"><span class="pre">TokenFilterStem</span></code> token filter:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>register token_filters/stem
+# [[0,0.0,0.0],true]
+table_create Memos TABLE_NO_KEY
+# [[0,0.0,0.0],true]
+column_create Memos content COLUMN_SCALAR ShortText
+# [[0,0.0,0.0],true]
+table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStem
+# [[0,0.0,0.0],true]
+column_create Terms memos_content COLUMN_INDEX|WITH_POSITION Memos content
+# [[0,0.0,0.0],true]
+load --table Memos
+[
+{&quot;content&quot;: &quot;I develop Groonga&quot;},
+{&quot;content&quot;: &quot;I&#39;m developing Groonga&quot;},
+{&quot;content&quot;: &quot;I developed Groonga&quot;}
+]
+# [[0,0.0,0.0],3]
+select Memos --match_columns content --query &quot;develops&quot;
+# [
+#   [
+#     0,
+#     0.0,
+#     0.0
+#   ],
+#   [
+#     [
+#       [
+#         3
+#       ],
+#       [
+#         [
+#           &quot;_id&quot;,
+#           &quot;UInt32&quot;
+#         ],
+#         [
+#           &quot;content&quot;,
+#           &quot;ShortText&quot;
+#         ]
+#       ],
+#       [
+#         1,
+#         &quot;I develop Groonga&quot;
+#       ],
+#       [
+#         2,
+#         &quot;I&#39;m developing Groonga&quot;
+#       ],
+#       [
+#         3,
+#         &quot;I developed Groonga&quot;
+#       ]
+#     ]
+#   ]
+# ]
+</pre></div>
+</div>
+<p>All of <code class="docutils literal"><span class="pre">develop</span></code>, <code class="docutils literal"><span class="pre">developing</span></code>, <code class="docutils literal"><span class="pre">developed</span></code> and <code class="docutils literal"><span class="pre">develops</span></code>
+tokens are stemmed as <code class="docutils literal"><span class="pre">develop</span></code>. So we can find <code class="docutils literal"><span class="pre">develop</span></code>,
+<code class="docutils literal"><span class="pre">developing</span></code> and <code class="docutils literal"><span class="pre">developed</span></code> by <code class="docutils literal"><span class="pre">develops</span></code> query.</p>
+</div>
+</div>
+<div class="section" id="see-also">
+<h2>7.9.3. See also<a class="headerlink" href="#see-also" title="Permalink to this headline">¶</a></h2>
+<ul class="simple">
+<li><a class="reference internal" href="commands/table_create.html"><em>table_create</em></a></li>
+</ul>
+</div>
+</div>
+          </div>
+        </div>
+      </div>
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+        <div class="sphinxsidebarwrapper">
+  <h3><a href="../index.html">Table Of Contents</a></h3>
+  <ul>
+<li><a class="reference internal" href="#">7.9. Token filters</a><ul>
+<li><a class="reference internal" href="#summary">7.9.1. Summary</a></li>
+<li><a class="reference internal" href="#available-token-filters">7.9.2. Available token filters</a><ul>
+<li><a class="reference internal" href="#tokenfilterstopword">7.9.2.1. <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code></a></li>
+<li><a class="reference internal" href="#tokenfilterstem">7.9.2.2. <code class="docutils literal"><span class="pre">TokenFilterStem</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#see-also">7.9.3. See also</a></li>
+</ul>
+</li>
+</ul>
+  <h4>Previous topic</h4>
+  <p class="topless"><a href="tokenizers.html"
+                        title="previous chapter">7.8. Tokenizers</a></p>
+  <h4>Next topic</h4>
+  <p class="topless"><a href="query_expanders.html"
+                        title="next chapter">7.10. Query expanders</a></p>
+  <div role="note" aria-label="source link">
+    <h3>This Page</h3>
+    <ul class="this-page-menu">
+      <li><a href="../_sources/reference/token_filters.txt"
+            rel="nofollow">Show Source</a></li>
+    </ul>
+   </div>
+<div id="searchbox" style="display: none" role="search">
+  <h3>Quick search</h3>
+    <form class="search" action="../search.html" method="get">
+      <input type="text" name="q" />
+      <input type="submit" value="Go" />
+      <input type="hidden" name="check_keywords" value="yes" />
+      <input type="hidden" name="area" value="default" />
+    </form>
+    <p class="searchtip" style="font-size: 90%">
+    Enter search terms or a module, class or function name.
+    </p>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+        </div>
+      </div>
+      <div class="clearer"></div>
+    </div>
+    <div class="related" role="navigation" aria-label="related navigation">
+      <h3>Navigation</h3>
+      <ul>
+        <li class="right" style="margin-right: 10px">
+          <a href="../genindex.html" title="General Index"
+             >index</a></li>
+        <li class="right" >
+          <a href="query_expanders.html" title="7.10. Query expanders"
+             >next</a> |</li>
+        <li class="right" >
+          <a href="tokenizers.html" title="7.8. Tokenizers"
+             >previous</a> |</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
+          <li><a href="../reference.html" >7. Reference manual</a> &raquo;</li>
+      </ul>
+    </div>
+    <div class="footer" role="contentinfo">
+        &copy; Copyright 2009-2014, Brazil, Inc.
+    </div>
+  </body>
+</html>