rroonga 5.0.0-x64-mingw32 → 5.0.1-x64-mingw32

data/vendor/local/share/doc/groonga/en/html/_sources/reference/grn_expr/script_syntax.txt

@@ -91,10 +91,10 @@ Here are a schema definition and sample data to show usage.
 ..  "content": "Welcome! This is my first post!",
 ..  "n_likes": 5},
 .. {"_key":    "Groonga",
-..  "content": "I started to use groonga. It's very fast!",
+..  "content": "I started to use Groonga. It's very fast!",
 ..  "n_likes": 10},
 .. {"_key":    "Mroonga",
-..  "content": "I also started to use mroonga. It's also very fast! Really fast!",
+..  "content": "I also started to use Mroonga. It's also very fast! Really fast!",
 ..  "n_likes": 15},
 .. {"_key":    "Good-bye Senna",
 ..  "content": "I migrated all Senna system!",
@@ -228,7 +228,7 @@ control syntax.
 
 ``A ? B : C`` returns ``B`` if ``A`` is true, ``C`` otherwise.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_control_syntax_ternary_operator.log
@@ -248,7 +248,7 @@ an expression. ``a && b || c`` means that ``a`` and ``b`` are matched
 or ``c`` is matched. ``a && (b || c)`` means that ``a`` and one of
 ``b`` and ``c`` are matched.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_grouping.log
@@ -273,7 +273,7 @@ Its syntax is ``name(arugment1, argument2, ...)``.
 
 See :doc:`/reference/function` for available functin list.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_function.log
@@ -836,7 +836,7 @@ Because tag search is also implemented by inverted index.
 
 :doc:`query_syntax` uses this operator by default.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_match_operator.log
@@ -866,7 +866,7 @@ Prefix search can be used with other table types but it causes all
 records scan. It's not problem for small records but it spends more
 time for large records.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_prefix_search_operator.log
@@ -908,7 +908,7 @@ table without ``KEY_WITH_SIS`` flag but it causes all records
 scan. It's not problem for small records but it spends more time for
 large records.
 
-Here is a simple exmaple. It uses fast suffix search for hiragana in
+Here is a simple example. It uses fast suffix search for hiragana in
 Japanese that is one of non-ASCII characters.
 
 .. groonga-command
@@ -952,7 +952,7 @@ N-gram family tokenizer.)
 Note that an index column for full text search must be defined for
 ``column``.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_near_search_operator.log
@@ -992,7 +992,7 @@ search searches records that have similar content to
 Note that an index column for full text search must be defined for
 ``column``.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_similar_search_operator.log
@@ -1018,7 +1018,7 @@ double array trie (``TABLE_DAT_KEY``). You can't use hash table
 support longest common prefix search. Longest common prefix search is
 used to implement the operator.
 
-Here is a simple exmaple.
+Here is a simple example.
 
 .. groonga-command
 .. include:: ../../example/reference/grn_expr/script_syntax/simple_term_extract_operator.log
@@ -1037,3 +1037,33 @@ the successor project to Senna.``. In this case, ``KEY_NORMALIZE``
 flag is specified to ``Words``. So ``Groonga`` can be extracted even
 if it is loaded as ``groonga`` into ``Words``. All of extracted terms
 are also normalized.
+
+.. _script-regular-expression-operator:
+
+Regular expression operator
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. versionadded:: 5.0.1
+
+Its syntax is ``column @~ "pattern"``.
+
+The operator searches records by the regular expression
+``pattern``. If a record's ``column`` value is matched to ``pattern``,
+the record is matched.
+
+``pattern`` must be valid regular expression syntax. See
+:doc:`/reference/regular_expression` about regular expression syntax
+details.
+
+The following example uses ``.roonga`` as pattern. It matches
+``Groonga``, ``Mroonga`` and so on.
+
+.. groonga-command
+.. include:: ../../example/reference/grn_expr/script_syntax/simple_regular_expression_operator.log
+.. select Entries --filter 'content @~ ".roonga"'
+
+In most cases, regular expression is evaluated sequentially. So it may
+be slow against many records.
+
+In some cases, Groonga evaluates regular expression by index. It's
+very fast. See :doc:`/reference/regular_expression` for details.

data/vendor/local/share/doc/groonga/en/html/_sources/reference/indexing.txt

@@ -11,6 +11,8 @@ Indexing
 Groonga supports both online index construction and offline
 index construction since 2.0.0.
 
+.. _online-index-construction:
+
 Online index construction
 -------------------------
 

data/vendor/local/share/doc/groonga/en/html/_sources/reference/normalizers.txt

@@ -73,6 +73,8 @@ Here is a list of built-in normalizers:
   * ``NormalizerAuto``
   * ``NormalizerNFKC51``
 
+.. _normalizer-auto:
+
 ``NormalizerAuto``
 ^^^^^^^^^^^^^^^^^^
 
@@ -100,6 +102,8 @@ Here is an example that uses ``NormalizerAuto`` normalizer:
 .. include:: ../example/reference/normalizers/normalizer-auto.log
 .. table_create NormalLexicon TABLE_HASH_KEY ShortText --normalizer NormalizerAuto
 
+.. _normalizer-nfkc51:
+
 ``NormalizerNFKC51``
 ^^^^^^^^^^^^^^^^^^^^
 

data/vendor/local/share/doc/groonga/en/html/_sources/reference/operations.txt

@@ -6,7 +6,8 @@
 Operations
 ==========
 
-Groonga has the search feature of multiple. This section describes how to search operations.
+Groonga has the multiple search operations. This section describes
+about search operations.
 
 .. toctree::
    :maxdepth: 2

data/vendor/local/share/doc/groonga/en/html/_sources/reference/regular_expression.txt

@@ -0,0 +1,403 @@
+.. -*- rst -*-
+
+.. highlightlang:: none
+
+.. groonga-command
+.. database: regular_expression
+
+Regular expression
+==================
+
+Summary
+-------
+
+.. note::
+
+   Regular expression support is an experimental feature.
+
+.. versionadded:: 5.0.1
+
+Groonga supports pattern match by regular expression. Regular
+expression is widely used format to describe a pattern. Regular
+expression is useful to represent complex pattern.
+
+In most cases, pattern match by regular expression is evaluated as
+sequential search. It'll be slow for many records and many texts.
+
+In some cases, pattern match by regular expression can be evaluated
+by index. It's very fast rather than sequential search. Patterns
+that can be evaluated by index are described later.
+
+There are many regular expression syntaxes. Groonga uses the same
+syntax in Ruby. Because Groonga uses the same regular expression
+engine as Ruby. The regular expression engine is `Onigmo
+<https://github.com/k-takata/Onigmo/>`_. Characteristic difference
+with other regular expression syntax is ``^`` and ``$``. The regular
+expression syntax in Ruby, ``^`` means the beginning of line and ``$``
+means the end of line. ``^`` means the beginning of text and ``$``
+means the end of text in other most regular expression syntaxes. The regular
+expression syntax in Ruby uses ``\A`` for the beginning of text and
+``\z`` for the end of text.
+
+You can use regular expression in :ref:`select-query` and
+:ref:`select-filter` options of :doc:`/reference/commands/select`
+command.
+
+Usage
+-----
+
+Here are a schema definition and sample data to show usage. There is
+only one table, ``Logs``. ``Logs`` table has only ``message``
+column. Log messages are stored into the ``message`` column.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/usage_setup.log
+.. table_create Logs TABLE_NO_KEY
+.. column_create Logs message COLUMN_SCALAR Text
+..
+.. load --table Logs
+.. [
+.. {"message": "host1:[error]: No memory"},
+.. {"message": "host1:[warning]: Remained disk space is less than 30%"},
+.. {"message": "host1:[error]: Disk full"},
+.. {"message": "host2:[error]: No memory"},
+.. {"message": "host2:[info]: Shutdown"}
+.. ]
+
+Here is an example that uses regular expression in
+:ref:`select-query`. You need to use
+``${COLUMN}:~${REGULAR_EXPRESSION}`` syntax.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/usage_query.log
+.. select Logs --query 'message:~"[Dd]isk"'
+
+Here is an example that uses regular expression in
+:ref:`select-filter`. You need to use
+``${COLUMN} @~ ${REGULAR_EXPRESSION}`` syntax.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/usage_filter.log
+.. select Logs --filter 'message @~ "[Dd]isk"'
+
+Index
+-----
+
+Groonga can search records by regular expression with index. It's very
+fast rather than sequential search.
+
+But it doesn't support all regular expression patterns. It supports
+only the following regular expression patterns. The patterns will be
+increased in the future.
+
+  * Literal only pattern such as ``disk``
+  * The begging of text and literal only pattern such as ``\Adisk``
+  * The end of text and literal only pattern such as ``disk\z``
+
+You need to create an index for fast regular expression search. Here
+are requirements of index:
+
+  * Lexicon must be :ref:`table-pat-key` table.
+  * Lexicon must use :ref:`token-regexp` tokenizer.
+  * Index column must has ``WITH_POSITION`` flag.
+
+Other configurations such as lexicon's normalizer are optional. You
+can choose what you like. If you want to use case-insensitive search,
+use :ref:`normalizer-auto` normalizer.
+
+Here are recommended index definitions. In general, it's reasonable
+index definitions.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/index_definitions.log
+.. table_create RegexpLexicon TABLE_PAT_KEY ShortText \
+..   --default_tokenizer TokenRegexp \
+..   --normalizer NormalizerAuto
+.. column_create RegexpLexicon logs_message_index \
+..   COLUMN_INDEX|WITH_POSITION Logs message
+
+Now, you can use index for regular expression search. The following
+regular expression can be evaluated by index because it uses only "the
+beginning of text" and "literal".
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/search_by_index_query.log
+.. select Logs --query message:~\\\\Ahost1
+
+Here is an example that uses :ref:`select-filter` instead of
+:ref:`select-query`. It uses the same regular expression as the
+previous example.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/search_by_index_filter.log
+.. select Logs --filter 'message @~ "\\\\Ahost1:"'
+
+``\`` escape will confuse you because there are some steps that
+require escape between you and Groonga. Here are steps that require
+``\`` escape:
+
+  * Shell only when you pass Groonga command from command line the
+    following::
+
+      % groonga /tmp/db select Logs --filter '"message @~ \"\\\\Ahost1:"\"'
+
+    ``--filter '"message @~ \"\\\\Ahost1:\""'`` is evaluated as the
+    following two arguments by shell:
+
+      * ``--filter``
+      * ``"message @~ \"\\\\Ahost1:\""``
+
+  * Groonga command parser only when you pass Groonga command by
+    command line style (``COMMAND ARG1_VALUE ARG2_VALUE ...``) not
+    HTTP path style
+    (``/d/COMMAND?ARG1_NAME=ARG1_VALUE&ARG2_NAME=ARG3_VALUE``).
+
+    ``"message @~ \"\\\\Ahost1:\""`` is evaluated as the following
+    value by Groonga command parser:
+
+      * ``message @~ "\\Ahost1:"``
+
+  * :doc:`/reference/grn_expr` parser. ``\`` escape is required in both
+    :doc:`/reference/grn_expr/query_syntax` and
+    :doc:`/reference/grn_expr/script_syntax`.
+
+    ``"\\Ahost1:"`` string literal in script syntax is evaluated as
+    the following value:
+
+      * ``\Ahost1``
+
+    The value is evaluated as regular expression.
+
+Syntax
+------
+
+This section describes about only commonly used syntaxes. See `Onigmo
+syntax documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE>`_ for other
+syntaxes and details.
+
+.. _regular-expression-escape:
+
+Escape
+^^^^^^
+
+In regular expression, there are the following special characters:
+
+  * ``\``
+  * ``|``
+  * ``(``
+  * ``)``
+  * ``[``
+  * ``]``
+  * ``.``
+  * ``*``
+  * ``+``
+  * ``?``
+  * ``{``
+  * ``}``
+  * ``^``
+  * ``$``
+
+If you want to write pattern that matches these special character as
+is, you need to escape them.
+
+You can escape them by putting ``\`` before special character. Here
+are regular expressions that match special character itself:
+
+  * ``\\``
+  * ``\|``
+  * ``\(``
+  * ``\)``
+  * ``\[``
+  * ``\]``
+  * ``\.``
+  * ``\*``
+  * ``\+``
+  * ``\?``
+  * ``\{``
+  * ``\}``
+  * ``\^``
+  * ``\$``
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/choice.log
+.. select Logs --filter 'message @~ "\\\\[error\\\\]"'
+
+If your regular expression doesn't work as you expected, confirm that
+some special characters are used without escaping.
+
+.. _regular-expression-choice:
+
+Choice
+^^^^^^
+
+Choice syntax is ``A|B``. The regular expression matches when either
+``A`` pattern or ``B`` pattern is matched.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/choice.log
+.. select Logs --filter 'message @~ "warning|info"'
+
+.. caution::
+
+   Regular expression that uses this syntax can't be evaluated by
+   index.
+
+.. _regular-expression-group:
+
+Group
+^^^^^
+
+Group syntax is ``(...)``. Group provides the following features:
+
+  * Back reference
+  * Scope reducing
+
+You can refer matched groups by ``\n`` (``n`` is the group number)
+syntax. For example, ``e(r)\1o\1`` matches ``error``. Because ``\1``
+is replaced with match result (``r``) of the first group ``(r)``.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/group_back_reference.log
+.. select Logs --filter 'message @~ "e(r)\\\\1o\\\\1"'
+
+You can also use more powerful back reference features. See `"8. Back
+reference" section in Onigmo documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L302>`_ for
+details.
+
+Group syntax reduces scope. For example, ``\[(warning|info)\]``
+reduces choice syntax scope. The regular expression matches
+``[warning]`` and ``[info]``.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/group_scope_reducing.log
+.. select Logs --filter 'message @~ "\\\\[(warning|info)\\\\]"'
+
+You can also use more powerful group related features. See
+`"7. Extended groups" section in Onigmo documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L225>`_ for
+details.
+
+.. caution::
+
+   Regular expression that uses this syntax can't be evaluated by
+   index.
+
+.. _regular-expression-character-class:
+
+Character class
+^^^^^^^^^^^^^^^
+
+Character class syntax is ``[...]``. Character class is useful to
+specify multiple characters to be matched.
+
+For example, ``[Dd]`` matches ``D`` or ``d``.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/character_class_characters.log
+.. select Logs --filter 'message @~ "[Dd]isk"'
+
+You can specify characters by range. For example, ``[0-9]`` matches
+one digit.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/character_class_range.log
+.. select Logs --filter 'message @~ "[0-9][0-9]%"'
+
+You can also use more powerful character class related features. See
+`"6. Character class" section in Onigmo documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L164>`_ for
+details.
+
+.. caution::
+
+   Regular expression that uses this syntax can't be evaluated by
+   index.
+
+.. _regular-expression-anchor:
+
+Anchor
+^^^^^^
+
+There are the following commonly used anchor syntaxes. Some anchors
+can be evaluated by index.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Anchor
+     - Description
+     - Index ready
+   * - ``^``
+     - The beginning of line
+     - o
+   * - ``$``
+     - The end of line
+     - x
+   * - ``\A``
+     - The beginning of text
+     - o
+   * - ``\z``
+     - The end of text
+     - x
+
+Here is an example that uses ``\z``.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/anchor_z.log
+.. select Logs --filter 'message @~ "%\\\\z"'
+
+You can also use more anchors. See `"5. Anchors" section in Onigmo
+documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L152>`_ for
+details.
+
+.. caution::
+
+   Regular expression that uses this syntax except ``\A`` and ``\z``
+   can't be evaluated by index.
+
+.. _regular-expression-quantifier:
+
+Quantifier
+^^^^^^^^^^
+
+There are the following commonly used quantifier syntaxes.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Quantifier
+     - Description
+   * - ``?``
+     - 0 or 1 time
+   * - ``*``
+     - 0 or more times
+   * - ``+``
+     - 1 or more times
+
+For example, ``er+or`` matches ``error``, ``errror`` and so on.
+
+.. groonga-command
+.. include:: ../example/reference/regular_expression/quantifier_plus.log
+.. select Logs --filter 'message @~ "er+or"'
+
+You can also use more quantifiers. See `"4. Quantifier" section in Onigmo
+documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L119>`_ for
+details.
+
+.. caution::
+
+   Regular expression that uses this syntax can't be evaluated by
+   index.
+
+Others
+^^^^^^
+
+There are more syntaxes. If you're interested in them, see `Onigmo
+documentation
+<https://github.com/k-takata/Onigmo/blob/master/doc/RE>`_ for
+details. You may be interested in "character type" and "character"
+syntaxes.