RubyGems - rroonga - Versions diffs - 4.0.6-x86-mingw32 → 4.0.7-x86-mingw32 - Mend

rroonga 4.0.6-x86-mingw32 → 4.0.7-x86-mingw32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (598) hide show

data/vendor/local/share/doc/groonga/en/html/reference/tables.html CHANGED Viewed

@@ -7,7 +7,7 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>7.5. Tables &mdash; Groonga v4.0.6-211-g8c0bb0b documentation</title>
+    <title>7.5. Tables &mdash; Groonga v4.0.8 documentation</title>
     <link rel="stylesheet" href="../_static/groonga.css" type="text/css" />
     <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
@@ -15,7 +15,7 @@
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    '../',
-        VERSION:     '4.0.6-211-g8c0bb0b',
+        VERSION:     '4.0.8',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true
@@ -25,12 +25,12 @@
     <script type="text/javascript" src="../_static/underscore.js"></script>
     <script type="text/javascript" src="../_static/doctools.js"></script>
     <link rel="shortcut icon" href="../_static/favicon.ico"/>
-    <link rel="top" title="Groonga v4.0.6-211-g8c0bb0b documentation" href="../index.html" />
+    <link rel="top" title="Groonga v4.0.8 documentation" href="../index.html" />
     <link rel="up" title="7. Reference manual" href="../reference.html" />
     <link rel="next" title="7.6. Column" href="column.html" />
     <link rel="prev" title="7.4. データ型" href="types.html" />
   </head>
-  <body>
+  <body role="document">
 <div class="header">
   <h1 class="title">
     <a id="top-link" href="../index.html">
@@ -48,7 +48,7 @@
 </div>
-    <div class="related">
+    <div class="related" role="navigation" aria-label="related navigation">
       <h3>Navigation</h3>
       <ul>
         <li class="right" style="margin-right: 10px">
@@ -60,7 +60,7 @@
         <li class="right" >
           <a href="types.html" title="7.4. データ型"
              accesskey="P">previous</a> |</li>
-        <li><a href="../index.html">Groonga v4.0.6-211-g8c0bb0b documentation</a> &raquo;</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
           <li><a href="../reference.html" accesskey="U">7. Reference manual</a> &raquo;</li>
       </ul>
     </div>
@@ -68,23 +68,23 @@
     <div class="document">
       <div class="documentwrapper">
         <div class="bodywrapper">
-          <div class="body">
+          <div class="body" role="main">
   <div class="section" id="tables">
 <h1>7.5. Tables<a class="headerlink" href="#tables" title="Permalink to this headline">¶</a></h1>
 <div class="section" id="summary">
 <h2>7.5.1. Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>
 <p>Table in Groonga manages relation between ID and key. Groonga provides
-four table types. They are <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt>, <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt>,
-<tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> and <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>.</p>
-<p>All tables except <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> provides both fast ID search by key
-and fast key search by ID. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> doesn't support
-key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> only manages ID. So <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> doesn't
+four table types. They are <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code>, <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code>,
+<code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> and <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>.</p>
+<p>All tables except <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> provides both fast ID search by key
+and fast key search by ID. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> doesn't support
+key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> only manages ID. So <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> doesn't
 provides ID search and key search.</p>
 </div>
 <div class="section" id="characteristics">
 <h2>7.5.2. Characteristics<a class="headerlink" href="#characteristics" title="Permalink to this headline">¶</a></h2>
-<p>Here is a chracteristic table of all tables in Groonga. (<tt class="docutils literal"><span class="pre">TABLE_</span></tt>
+<p>Here is a chracteristic table of all tables in Groonga. (<code class="docutils literal"><span class="pre">TABLE_</span></code>
 prefix is omitted in the table.)</p>
 <table border="1" class="docutils">
 <colgroup>
@@ -96,10 +96,10 @@ prefix is omitted in the table.)</p>
 </colgroup>
 <thead valign="bottom">
 <tr class="row-odd"><th class="head">&nbsp;</th>
-<th class="head"><tt class="docutils literal"><span class="pre">NO_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">HASH_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">PAT_KEY</span></tt></th>
-<th class="head"><tt class="docutils literal"><span class="pre">DAT_KEY</span></tt></th>
+<th class="head"><code class="docutils literal"><span class="pre">NO_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">HASH_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">PAT_KEY</span></code></th>
+<th class="head"><code class="docutils literal"><span class="pre">DAT_KEY</span></code></th>
 </tr>
 </thead>
 <tbody valign="top">
@@ -200,36 +200,36 @@ search</td>
 </tbody>
 </table>
 <div class="section" id="table-no-key">
-<h3>7.5.2.1. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt><a class="headerlink" href="#table-no-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is very fast and very small but it doesn't support
-key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is a only table that doesn't support key.</p>
-<p>You cannot use <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> for lexicon for fulltext search
-because lexicon stores tokens as key. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is useful for
+<h3>7.5.2.1. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code><a class="headerlink" href="#table-no-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is very fast and very small but it doesn't support
+key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is a only table that doesn't support key.</p>
+<p>You cannot use <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> for lexicon for fulltext search
+because lexicon stores tokens as key. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is useful for
 no key records such as log.</p>
 </div>
 <div class="section" id="table-hash-key">
-<h3>7.5.2.2. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt><a class="headerlink" href="#table-hash-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is fast but it doesn't support advanced search
+<h3>7.5.2.2. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code><a class="headerlink" href="#table-hash-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is fast but it doesn't support advanced search
 functions such as common prefix search and predictive search.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is useful for index for exact search such as tag
+<p><code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is useful for index for exact search such as tag
 search.</p>
 </div>
 <div class="section" id="table-pat-key">
-<h3>7.5.2.3. <tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt><a class="headerlink" href="#table-pat-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> is small and supports advanced search functions.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt> is useful for lexicon for fulltext search and
+<h3>7.5.2.3. <code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code><a class="headerlink" href="#table-pat-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> is small and supports advanced search functions.</p>
+<p><code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code> is useful for lexicon for fulltext search and
 index for range search.</p>
 </div>
 <div class="section" id="table-dat-key">
-<h3>7.5.2.4. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt><a class="headerlink" href="#table-dat-key" title="Permalink to this headline">¶</a></h3>
-<p><tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is fast and supports key update but it is large. It
-is not suitable for storing many records. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is a only
+<h3>7.5.2.4. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code><a class="headerlink" href="#table-dat-key" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is fast and supports key update but it is large. It
+is not suitable for storing many records. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is a only
 table that supports key update.</p>
-<p><tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt> is used in Groonga database. Groonga database needs
-to convert object name such as <tt class="docutils literal"><span class="pre">ShortText</span></tt>, <tt class="docutils literal"><span class="pre">TokenBigram</span></tt> and
+<p><code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code> is used in Groonga database. Groonga database needs
+to convert object name such as <code class="docutils literal"><span class="pre">ShortText</span></code>, <code class="docutils literal"><span class="pre">TokenBigram</span></code> and
 table names to object ID. And Groonga database needs to rename object
-name. Those features are implemented by <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>. The number
-of objects is small. So large data size demerit of <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt>
+name. Those features are implemented by <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>. The number
+of objects is small. So large data size demerit of <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code>
 can be ignored.</p>
 </div>
 </div>
@@ -255,8 +255,8 @@ database.</p>
 <h3>7.5.4.2. Temporary table<a class="headerlink" href="#temporary-table" title="Permalink to this headline">¶</a></h3>
 <p>Temporary table is anonymous. Records in temporary table are deleted
 after closing table. Temporary table is used to store search result,
-sort result, group (drilldown) result and so on. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt> is
-used for search result and group result. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt> is used for
+sort result, group (drilldown) result and so on. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code> is
+used for search result and group result. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code> is used for
 sort result.</p>
 </div>
 </div>
@@ -266,7 +266,7 @@ sort result.</p>
 more records in a table.</p>
 <p>The max number of a key size is 4096byte. You cannot use 4097byte or
 larger key. You can use column instead of key for 4097byte or larger
-size data. <tt class="docutils literal"><span class="pre">Text</span></tt> and <tt class="docutils literal"><span class="pre">LargeText</span></tt> types supports 4097byte or
+size data. <code class="docutils literal"><span class="pre">Text</span></code> and <code class="docutils literal"><span class="pre">LargeText</span></code> types supports 4097byte or
 larger size data.</p>
 <p>The max number of total key size is 4GiB. You need to split a table,
 split a database (sharding) or reduce each key size to handle 4GiB or
@@ -284,17 +284,17 @@ more larger total key size.</p>
           </div>
         </div>
       </div>
-      <div class="sphinxsidebar">
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
         <div class="sphinxsidebarwrapper">
   <h3><a href="../index.html">Table Of Contents</a></h3>
   <ul>
 <li><a class="reference internal" href="#">7.5. Tables</a><ul>
 <li><a class="reference internal" href="#summary">7.5.1. Summary</a></li>
 <li><a class="reference internal" href="#characteristics">7.5.2. Characteristics</a><ul>
-<li><a class="reference internal" href="#table-no-key">7.5.2.1. <tt class="docutils literal"><span class="pre">TABLE_NO_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-hash-key">7.5.2.2. <tt class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-pat-key">7.5.2.3. <tt class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></tt></a></li>
-<li><a class="reference internal" href="#table-dat-key">7.5.2.4. <tt class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></tt></a></li>
+<li><a class="reference internal" href="#table-no-key">7.5.2.1. <code class="docutils literal"><span class="pre">TABLE_NO_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-hash-key">7.5.2.2. <code class="docutils literal"><span class="pre">TABLE_HASH_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-pat-key">7.5.2.3. <code class="docutils literal"><span class="pre">TABLE_PAT_KEY</span></code></a></li>
+<li><a class="reference internal" href="#table-dat-key">7.5.2.4. <code class="docutils literal"><span class="pre">TABLE_DAT_KEY</span></code></a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#record-id">7.5.3. Record ID</a></li>
@@ -315,12 +315,14 @@ more larger total key size.</p>
   <h4>Next topic</h4>
   <p class="topless"><a href="column.html"
                         title="next chapter">7.6. Column</a></p>
-  <h3>This Page</h3>
-  <ul class="this-page-menu">
-    <li><a href="../_sources/reference/tables.txt"
-           rel="nofollow">Show Source</a></li>
-  </ul>
-<div id="searchbox" style="display: none">
+  <div role="note" aria-label="source link">
+    <h3>This Page</h3>
+    <ul class="this-page-menu">
+      <li><a href="../_sources/reference/tables.txt"
+            rel="nofollow">Show Source</a></li>
+    </ul>
+   </div>
+<div id="searchbox" style="display: none" role="search">
   <h3>Quick search</h3>
     <form class="search" action="../search.html" method="get">
       <input type="text" name="q" />
@@ -337,7 +339,7 @@ more larger total key size.</p>
       </div>
       <div class="clearer"></div>
     </div>
-    <div class="related">
+    <div class="related" role="navigation" aria-label="related navigation">
       <h3>Navigation</h3>
       <ul>
         <li class="right" style="margin-right: 10px">
@@ -349,11 +351,11 @@ more larger total key size.</p>
         <li class="right" >
           <a href="types.html" title="7.4. データ型"
              >previous</a> |</li>
-        <li><a href="../index.html">Groonga v4.0.6-211-g8c0bb0b documentation</a> &raquo;</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
           <li><a href="../reference.html" >7. Reference manual</a> &raquo;</li>
       </ul>
     </div>
-    <div class="footer">
+    <div class="footer" role="contentinfo">
         &copy; Copyright 2009-2014, Brazil, Inc.
     </div>
   </body>

data/vendor/local/share/doc/groonga/en/html/reference/token_filters.html ADDED Viewed

@@ -0,0 +1,323 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+  <head>
+    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+    <title>7.9. Token filters &mdash; Groonga v4.0.8 documentation</title>
+    <link rel="stylesheet" href="../_static/groonga.css" type="text/css" />
+    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+    <script type="text/javascript">
+      var DOCUMENTATION_OPTIONS = {
+        URL_ROOT:    '../',
+        VERSION:     '4.0.8',
+        COLLAPSE_INDEX: false,
+        FILE_SUFFIX: '.html',
+        HAS_SOURCE:  true
+      };
+    </script>
+    <script type="text/javascript" src="../_static/jquery.js"></script>
+    <script type="text/javascript" src="../_static/underscore.js"></script>
+    <script type="text/javascript" src="../_static/doctools.js"></script>
+    <link rel="shortcut icon" href="../_static/favicon.ico"/>
+    <link rel="top" title="Groonga v4.0.8 documentation" href="../index.html" />
+    <link rel="up" title="7. Reference manual" href="../reference.html" />
+    <link rel="next" title="7.10. Query expanders" href="query_expanders.html" />
+    <link rel="prev" title="7.8. Tokenizers" href="tokenizers.html" />
+  </head>
+  <body role="document">
+<div class="header">
+  <h1 class="title">
+    <a id="top-link" href="../index.html">
+      <span class="project">groonga</span>
+      <span class="separator">-</span>
+      <span class="description">An open-source fulltext search engine and column store.</span>
+    </a>
+  </h1>
+  <div class="other-language-links">
+    <ul>
+      <li><a href="../../../ja/html/reference/token_filters.html"><img src="../_static/jp.png" alt="日本語">日本語版はこちら</a></li>
+    </ul>
+  </div>
+</div>
+    <div class="related" role="navigation" aria-label="related navigation">
+      <h3>Navigation</h3>
+      <ul>
+        <li class="right" style="margin-right: 10px">
+          <a href="../genindex.html" title="General Index"
+             accesskey="I">index</a></li>
+        <li class="right" >
+          <a href="query_expanders.html" title="7.10. Query expanders"
+             accesskey="N">next</a> |</li>
+        <li class="right" >
+          <a href="tokenizers.html" title="7.8. Tokenizers"
+             accesskey="P">previous</a> |</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
+          <li><a href="../reference.html" accesskey="U">7. Reference manual</a> &raquo;</li>
+      </ul>
+    </div>
+    <div class="document">
+      <div class="documentwrapper">
+        <div class="bodywrapper">
+          <div class="body" role="main">
+  <div class="section" id="token-filters">
+<h1>7.9. Token filters<a class="headerlink" href="#token-filters" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="summary">
+<h2>7.9.1. Summary<a class="headerlink" href="#summary" title="Permalink to this headline">¶</a></h2>
+<p>Groonga has token filter module that some processes tokenized token.</p>
+<p>Token filter module can be added as a plugin.</p>
+<p>You can customize tokenized token by registering your token filters plugins to Groonga.</p>
+<p>A table can have zero or more token filters. You can attach token
+filters to a table by <a class="reference internal" href="commands/table_create.html#table-create-token-filters"><em>token_filters</em></a> option in
+<a class="reference internal" href="commands/table_create.html"><em>table_create</em></a>.</p>
+<p>Here is an example <code class="docutils literal"><span class="pre">table_create</span></code> that uses <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code>
+token filter module:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStopWord
+# [[0,0.0,0.0],true]
+</pre></div>
+</div>
+</div>
+<div class="section" id="available-token-filters">
+<h2>7.9.2. Available token filters<a class="headerlink" href="#available-token-filters" title="Permalink to this headline">¶</a></h2>
+<p>Here is the list of available token filters:</p>
+<ul class="simple">
+<li><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code></li>
+<li><code class="docutils literal"><span class="pre">TokenFilterStem</span></code></li>
+</ul>
+<div class="section" id="tokenfilterstopword">
+<span id="token-filter-stop-word"></span><h3>7.9.2.1. <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code><a class="headerlink" href="#tokenfilterstopword" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> removes stop words from tokenized token
+in searching the documents.</p>
+<p><code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> can specify stop word after adding the
+documents because it removes token in searching the documents.</p>
+<p>The stop word is specified <code class="docutils literal"><span class="pre">is_stop_word</span></code> column on lexicon table.</p>
+<p>Here is an example that uses <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code> token filter:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>register token_filters/stop_word
+# [[0,0.0,0.0],true]
+table_create Memos TABLE_NO_KEY
+# [[0,0.0,0.0],true]
+column_create Memos content COLUMN_SCALAR ShortText
+# [[0,0.0,0.0],true]
+table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStopWord
+# [[0,0.0,0.0],true]
+column_create Terms memos_content COLUMN_INDEX|WITH_POSITION Memos content
+# [[0,0.0,0.0],true]
+column_create Terms is_stop_word COLUMN_SCALAR Bool
+# [[0,0.0,0.0],true]
+load --table Terms
+[
+{&quot;_key&quot;: &quot;and&quot;, &quot;is_stop_word&quot;: true}
+]
+# [[0,0.0,0.0],1]
+load --table Memos
+[
+{&quot;content&quot;: &quot;Hello&quot;},
+{&quot;content&quot;: &quot;Hello and Good-bye&quot;},
+{&quot;content&quot;: &quot;Good-bye&quot;}
+]
+# [[0,0.0,0.0],3]
+select Memos --match_columns content --query &quot;Hello and&quot;
+# [
+#   [
+#     0,
+#     0.0,
+#     0.0
+#   ],
+#   [
+#     [
+#       [
+#         2
+#       ],
+#       [
+#         [
+#           &quot;_id&quot;,
+#           &quot;UInt32&quot;
+#         ],
+#         [
+#           &quot;content&quot;,
+#           &quot;ShortText&quot;
+#         ]
+#       ],
+#       [
+#         1,
+#         &quot;Hello&quot;
+#       ],
+#       [
+#         2,
+#         &quot;Hello and Good-bye&quot;
+#       ]
+#     ]
+#   ]
+# ]
+</pre></div>
+</div>
+<p><code class="docutils literal"><span class="pre">and</span></code> token is marked as stop word in <code class="docutils literal"><span class="pre">Terms</span></code> table.</p>
+<p><code class="docutils literal"><span class="pre">&quot;Hello&quot;</span></code> that doesn't have <code class="docutils literal"><span class="pre">and</span></code> in content is matched. Because
+<code class="docutils literal"><span class="pre">and</span></code> is a stop word and <code class="docutils literal"><span class="pre">and</span></code> is removed from query.</p>
+</div>
+<div class="section" id="tokenfilterstem">
+<span id="token-filter-stem"></span><h3>7.9.2.2. <code class="docutils literal"><span class="pre">TokenFilterStem</span></code><a class="headerlink" href="#tokenfilterstem" title="Permalink to this headline">¶</a></h3>
+<p><code class="docutils literal"><span class="pre">TokenFilterStem</span></code> stems tokenized token.</p>
+<p>Here is an example that uses <code class="docutils literal"><span class="pre">TokenFilterStem</span></code> token filter:</p>
+<p>Execution example:</p>
+<div class="highlight-none"><div class="highlight"><pre>register token_filters/stem
+# [[0,0.0,0.0],true]
+table_create Memos TABLE_NO_KEY
+# [[0,0.0,0.0],true]
+column_create Memos content COLUMN_SCALAR ShortText
+# [[0,0.0,0.0],true]
+table_create Terms TABLE_PAT_KEY ShortText \
+  --default_tokenizer TokenBigram \
+  --normalizer NormalizerAuto \
+  --token_filters TokenFilterStem
+# [[0,0.0,0.0],true]
+column_create Terms memos_content COLUMN_INDEX|WITH_POSITION Memos content
+# [[0,0.0,0.0],true]
+load --table Memos
+[
+{&quot;content&quot;: &quot;I develop Groonga&quot;},
+{&quot;content&quot;: &quot;I&#39;m developing Groonga&quot;},
+{&quot;content&quot;: &quot;I developed Groonga&quot;}
+]
+# [[0,0.0,0.0],3]
+select Memos --match_columns content --query &quot;develops&quot;
+# [
+#   [
+#     0,
+#     0.0,
+#     0.0
+#   ],
+#   [
+#     [
+#       [
+#         3
+#       ],
+#       [
+#         [
+#           &quot;_id&quot;,
+#           &quot;UInt32&quot;
+#         ],
+#         [
+#           &quot;content&quot;,
+#           &quot;ShortText&quot;
+#         ]
+#       ],
+#       [
+#         1,
+#         &quot;I develop Groonga&quot;
+#       ],
+#       [
+#         2,
+#         &quot;I&#39;m developing Groonga&quot;
+#       ],
+#       [
+#         3,
+#         &quot;I developed Groonga&quot;
+#       ]
+#     ]
+#   ]
+# ]
+</pre></div>
+</div>
+<p>All of <code class="docutils literal"><span class="pre">develop</span></code>, <code class="docutils literal"><span class="pre">developing</span></code>, <code class="docutils literal"><span class="pre">developed</span></code> and <code class="docutils literal"><span class="pre">develops</span></code>
+tokens are stemmed as <code class="docutils literal"><span class="pre">develop</span></code>. So we can find <code class="docutils literal"><span class="pre">develop</span></code>,
+<code class="docutils literal"><span class="pre">developing</span></code> and <code class="docutils literal"><span class="pre">developed</span></code> by <code class="docutils literal"><span class="pre">develops</span></code> query.</p>
+</div>
+</div>
+<div class="section" id="see-also">
+<h2>7.9.3. See also<a class="headerlink" href="#see-also" title="Permalink to this headline">¶</a></h2>
+<ul class="simple">
+<li><a class="reference internal" href="commands/table_create.html"><em>table_create</em></a></li>
+</ul>
+</div>
+</div>
+          </div>
+        </div>
+      </div>
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+        <div class="sphinxsidebarwrapper">
+  <h3><a href="../index.html">Table Of Contents</a></h3>
+  <ul>
+<li><a class="reference internal" href="#">7.9. Token filters</a><ul>
+<li><a class="reference internal" href="#summary">7.9.1. Summary</a></li>
+<li><a class="reference internal" href="#available-token-filters">7.9.2. Available token filters</a><ul>
+<li><a class="reference internal" href="#tokenfilterstopword">7.9.2.1. <code class="docutils literal"><span class="pre">TokenFilterStopWord</span></code></a></li>
+<li><a class="reference internal" href="#tokenfilterstem">7.9.2.2. <code class="docutils literal"><span class="pre">TokenFilterStem</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#see-also">7.9.3. See also</a></li>
+</ul>
+</li>
+</ul>
+  <h4>Previous topic</h4>
+  <p class="topless"><a href="tokenizers.html"
+                        title="previous chapter">7.8. Tokenizers</a></p>
+  <h4>Next topic</h4>
+  <p class="topless"><a href="query_expanders.html"
+                        title="next chapter">7.10. Query expanders</a></p>
+  <div role="note" aria-label="source link">
+    <h3>This Page</h3>
+    <ul class="this-page-menu">
+      <li><a href="../_sources/reference/token_filters.txt"
+            rel="nofollow">Show Source</a></li>
+    </ul>
+   </div>
+<div id="searchbox" style="display: none" role="search">
+  <h3>Quick search</h3>
+    <form class="search" action="../search.html" method="get">
+      <input type="text" name="q" />
+      <input type="submit" value="Go" />
+      <input type="hidden" name="check_keywords" value="yes" />
+      <input type="hidden" name="area" value="default" />
+    </form>
+    <p class="searchtip" style="font-size: 90%">
+    Enter search terms or a module, class or function name.
+    </p>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+        </div>
+      </div>
+      <div class="clearer"></div>
+    </div>
+    <div class="related" role="navigation" aria-label="related navigation">
+      <h3>Navigation</h3>
+      <ul>
+        <li class="right" style="margin-right: 10px">
+          <a href="../genindex.html" title="General Index"
+             >index</a></li>
+        <li class="right" >
+          <a href="query_expanders.html" title="7.10. Query expanders"
+             >next</a> |</li>
+        <li class="right" >
+          <a href="tokenizers.html" title="7.8. Tokenizers"
+             >previous</a> |</li>
+        <li><a href="../index.html">Groonga v4.0.8 documentation</a> &raquo;</li>
+          <li><a href="../reference.html" >7. Reference manual</a> &raquo;</li>
+      </ul>
+    </div>
+    <div class="footer" role="contentinfo">
+        &copy; Copyright 2009-2014, Brazil, Inc.
+    </div>
+  </body>
+</html>