rroonga 7.0.2-x86-mingw32 → 7.1.1-x86-mingw32

data/vendor/local/share/doc/groonga/en/html/_sources/reference/regular_expression.txt

@@ -1,436 +0,0 @@
-.. -*- rst -*-
-
-.. highlightlang:: none
-
-.. groonga-command
-.. database: regular_expression
-
-Regular expression
-==================
-
-Summary
--------
-
-.. note::
-
-   Regular expression support is an experimental feature.
-
-.. versionadded:: 5.0.1
-
-Groonga supports pattern match by regular expression. Regular
-expression is widely used format to describe a pattern. Regular
-expression is useful to represent complex pattern.
-
-In most cases, pattern match by regular expression is evaluated as
-sequential search. It'll be slow for many records and many texts.
-
-In some cases, pattern match by regular expression can be evaluated
-by index. It's very fast rather than sequential search. Patterns
-that can be evaluated by index are described later.
-
-.. versionadded:: 5.0.7
-
-   Groonga normalizes match target text by :ref:`normalizer-auto`
-   normalizer when Groonga doesn't use index for regular expression
-   search. It means that regular expression that has upper case such
-   as ``Groonga`` never match. Because :ref:`normalizer-auto`
-   normalizer normalizes all alphabets to lower case. ``groonga``
-   matches to both ``Groonga`` and ``groonga``.
-
-   Why is match target text normalizered? It's for increasing index
-   search-able patterns. If Groonga doesn't normalize match target
-   text, you need to write complex regular expression such as
-   ``[Dd][Ii][Ss][Kk]`` and ``(?i)disk`` for case-insensitive match.
-   Groonga can't use index against complex regular expression.
-
-   If you write ``disk`` regular expression for case-insensitive
-   match, Groonga can search the pattern with index. It's fast.
-
-   You may feel the behavior is strange. But fast search based on this
-   behavior will help you.
-
-There are many regular expression syntaxes. Groonga uses the same
-syntax in Ruby. Because Groonga uses the same regular expression
-engine as Ruby. The regular expression engine is `Onigmo
-<https://github.com/k-takata/Onigmo/>`_. Characteristic difference
-with other regular expression syntax is ``^`` and ``$``. The regular
-expression syntax in Ruby, ``^`` means the beginning of line and ``$``
-means the end of line. ``^`` means the beginning of text and ``$``
-means the end of text in other most regular expression syntaxes. The regular
-expression syntax in Ruby uses ``\A`` for the beginning of text and
-``\z`` for the end of text.
-
-.. versionadded:: 5.0.6
-
-   Groonga uses multiline mode since 5.0.6. It means that ``.``
-   matches on ``\n``.
-
-   But it's meaningless. Because ``\n`` is removed by
-   :ref:`normalizer-auto` normalizer.
-
-You can use regular expression in :ref:`select-query` and
-:ref:`select-filter` options of :doc:`/reference/commands/select`
-command.
-
-Usage
------
-
-Here are a schema definition and sample data to show usage. There is
-only one table, ``Logs``. ``Logs`` table has only ``message``
-column. Log messages are stored into the ``message`` column.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/usage_setup.log
-.. table_create Logs TABLE_NO_KEY
-.. column_create Logs message COLUMN_SCALAR Text
-..
-.. load --table Logs
-.. [
-.. {"message": "host1:[error]: No memory"},
-.. {"message": "host1:[warning]: Remained disk space is less than 30%"},
-.. {"message": "host1:[error]: Disk full"},
-.. {"message": "host2:[error]: No memory"},
-.. {"message": "host2:[info]: Shutdown"}
-.. ]
-
-Here is an example that uses regular expression in
-:ref:`select-query`. You need to use
-``${COLUMN}:~${REGULAR_EXPRESSION}`` syntax.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/usage_query.log
-.. select Logs --query 'message:~"disk (space|full)"'
-
-Here is an example that uses regular expression in
-:ref:`select-filter`. You need to use
-``${COLUMN} @~ ${REGULAR_EXPRESSION}`` syntax.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/usage_filter.log
-.. select Logs --filter 'message @~ "disk (space|full)"'
-
-.. _regular-expression-index:
-
-Index
------
-
-Groonga can search records by regular expression with index. It's very
-fast rather than sequential search.
-
-But it doesn't support all regular expression patterns. It supports
-only the following regular expression patterns. The patterns will be
-increased in the future.
-
-  * Literal only pattern such as ``disk``
-  * The begging of text and literal only pattern such as ``\Adisk``
-  * The end of text and literal only pattern such as ``disk\z``
-
-You need to create an index for fast regular expression search. Here
-are requirements of index:
-
-  * Lexicon must be :ref:`table-pat-key` table.
-  * Lexicon must use :ref:`token-regexp` tokenizer.
-  * Index column must has ``WITH_POSITION`` flag.
-
-Other configurations such as lexicon's normalizer are optional. You
-can choose what you like. If you want to use case-insensitive search,
-use :ref:`normalizer-auto` normalizer.
-
-Here are recommended index definitions. In general, it's reasonable
-index definitions.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/index_definitions.log
-.. table_create RegexpLexicon TABLE_PAT_KEY ShortText \
-..   --default_tokenizer TokenRegexp \
-..   --normalizer NormalizerAuto
-.. column_create RegexpLexicon logs_message_index \
-..   COLUMN_INDEX|WITH_POSITION Logs message
-
-Now, you can use index for regular expression search. The following
-regular expression can be evaluated by index because it uses only "the
-beginning of text" and "literal".
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/search_by_index_query.log
-.. select Logs --query message:~\\\\Ahost1
-
-Here is an example that uses :ref:`select-filter` instead of
-:ref:`select-query`. It uses the same regular expression as the
-previous example.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/search_by_index_filter.log
-.. select Logs --filter 'message @~ "\\\\Ahost1:"'
-
-``\`` escape will confuse you because there are some steps that
-require escape between you and Groonga. Here are steps that require
-``\`` escape:
-
-  * Shell only when you pass Groonga command from command line the
-    following::
-
-      % groonga /tmp/db select Logs --filter '"message @~ \"\\\\Ahost1:"\"'
-
-    ``--filter '"message @~ \"\\\\Ahost1:\""'`` is evaluated as the
-    following two arguments by shell:
-
-      * ``--filter``
-      * ``"message @~ \"\\\\Ahost1:\""``
-
-  * Groonga command parser only when you pass Groonga command by
-    command line style (``COMMAND ARG1_VALUE ARG2_VALUE ...``) not
-    HTTP path style
-    (``/d/COMMAND?ARG1_NAME=ARG1_VALUE&ARG2_NAME=ARG3_VALUE``).
-
-    ``"message @~ \"\\\\Ahost1:\""`` is evaluated as the following
-    value by Groonga command parser:
-
-      * ``message @~ "\\Ahost1:"``
-
-  * :doc:`/reference/grn_expr` parser. ``\`` escape is required in both
-    :doc:`/reference/grn_expr/query_syntax` and
-    :doc:`/reference/grn_expr/script_syntax`.
-
-    ``"\\Ahost1:"`` string literal in script syntax is evaluated as
-    the following value:
-
-      * ``\Ahost1``
-
-    The value is evaluated as regular expression.
-
-.. _regular-expression-syntax:
-
-Syntax
-------
-
-This section describes about only commonly used syntaxes. See `Onigmo
-syntax documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE>`_ for other
-syntaxes and details.
-
-.. _regular-expression-escape:
-
-Escape
-^^^^^^
-
-In regular expression, there are the following special characters:
-
-  * ``\``
-  * ``|``
-  * ``(``
-  * ``)``
-  * ``[``
-  * ``]``
-  * ``.``
-  * ``*``
-  * ``+``
-  * ``?``
-  * ``{``
-  * ``}``
-  * ``^``
-  * ``$``
-
-If you want to write pattern that matches these special character as
-is, you need to escape them.
-
-You can escape them by putting ``\`` before special character. Here
-are regular expressions that match special character itself:
-
-  * ``\\``
-  * ``\|``
-  * ``\(``
-  * ``\)``
-  * ``\[``
-  * ``\]``
-  * ``\.``
-  * ``\*``
-  * ``\+``
-  * ``\?``
-  * ``\{``
-  * ``\}``
-  * ``\^``
-  * ``\$``
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/choice.log
-.. select Logs --filter 'message @~ "\\\\[error\\\\]"'
-
-If your regular expression doesn't work as you expected, confirm that
-some special characters are used without escaping.
-
-.. _regular-expression-choice:
-
-Choice
-^^^^^^
-
-Choice syntax is ``A|B``. The regular expression matches when either
-``A`` pattern or ``B`` pattern is matched.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/choice.log
-.. select Logs --filter 'message @~ "warning|info"'
-
-.. caution::
-
-   Regular expression that uses this syntax can't be evaluated by
-   index.
-
-.. _regular-expression-group:
-
-Group
-^^^^^
-
-Group syntax is ``(...)``. Group provides the following features:
-
-  * Back reference
-  * Scope reducing
-
-You can refer matched groups by ``\n`` (``n`` is the group number)
-syntax. For example, ``e(r)\1o\1`` matches ``error``. Because ``\1``
-is replaced with match result (``r``) of the first group ``(r)``.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/group_back_reference.log
-.. select Logs --filter 'message @~ "e(r)\\\\1o\\\\1"'
-
-You can also use more powerful back reference features. See `"8. Back
-reference" section in Onigmo documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L302>`_ for
-details.
-
-Group syntax reduces scope. For example, ``\[(warning|info)\]``
-reduces choice syntax scope. The regular expression matches
-``[warning]`` and ``[info]``.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/group_scope_reducing.log
-.. select Logs --filter 'message @~ "\\\\[(warning|info)\\\\]"'
-
-You can also use more powerful group related features. See
-`"7. Extended groups" section in Onigmo documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L225>`_ for
-details.
-
-.. caution::
-
-   Regular expression that uses this syntax can't be evaluated by
-   index.
-
-.. _regular-expression-character-class:
-
-Character class
-^^^^^^^^^^^^^^^
-
-Character class syntax is ``[...]``. Character class is useful to
-specify multiple characters to be matched.
-
-For example, ``[12]`` matches ``1`` or ``2``.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/character_class_characters.log
-.. select Logs --filter 'message @~ "host[12]"'
-
-You can specify characters by range. For example, ``[0-9]`` matches
-one digit.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/character_class_range.log
-.. select Logs --filter 'message @~ "[0-9][0-9]%"'
-
-You can also use more powerful character class related features. See
-`"6. Character class" section in Onigmo documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L164>`_ for
-details.
-
-.. caution::
-
-   Regular expression that uses this syntax can't be evaluated by
-   index.
-
-.. _regular-expression-anchor:
-
-Anchor
-^^^^^^
-
-There are the following commonly used anchor syntaxes. Some anchors
-can be evaluated by index.
-
-.. list-table::
-   :header-rows: 1
-
-   * - Anchor
-     - Description
-     - Index ready
-   * - ``^``
-     - The beginning of line
-     - o
-   * - ``$``
-     - The end of line
-     - x
-   * - ``\A``
-     - The beginning of text
-     - o
-   * - ``\z``
-     - The end of text
-     - x
-
-Here is an example that uses ``\z``.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/anchor_z.log
-.. select Logs --filter 'message @~ "%\\\\z"'
-
-You can also use more anchors. See `"5. Anchors" section in Onigmo
-documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L152>`_ for
-details.
-
-.. caution::
-
-   Regular expression that uses this syntax except ``\A`` and ``\z``
-   can't be evaluated by index.
-
-.. _regular-expression-quantifier:
-
-Quantifier
-^^^^^^^^^^
-
-There are the following commonly used quantifier syntaxes.
-
-.. list-table::
-   :header-rows: 1
-
-   * - Quantifier
-     - Description
-   * - ``?``
-     - 0 or 1 time
-   * - ``*``
-     - 0 or more times
-   * - ``+``
-     - 1 or more times
-
-For example, ``er+or`` matches ``error``, ``errror`` and so on.
-
-.. groonga-command
-.. include:: ../example/reference/regular_expression/quantifier_plus.log
-.. select Logs --filter 'message @~ "er+or"'
-
-You can also use more quantifiers. See `"4. Quantifier" section in Onigmo
-documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE#L119>`_ for
-details.
-
-.. caution::
-
-   Regular expression that uses this syntax can't be evaluated by
-   index.
-
-Others
-^^^^^^
-
-There are more syntaxes. If you're interested in them, see `Onigmo
-documentation
-<https://github.com/k-takata/Onigmo/blob/master/doc/RE>`_ for
-details. You may be interested in "character type" and "character"
-syntaxes.

data/vendor/local/share/doc/groonga/en/html/_sources/reference/scorer.txt

@@ -1,218 +0,0 @@
-.. -*- rst -*-
-
-.. highlightlang:: none
-
-.. groonga-command
-.. database: scorer
-
-Scorer
-======
-
-Summary
--------
-
-Groonga has scorer module that customizes score function. Score
-function computes score of matched record. The default scorer function
-uses the number of appeared terms. It is also known as TF (term
-frequency).
-
-TF is a fast score function but it's not suitable for the following
-cases:
-
-  * Search query contains one or more frequently-appearing words such
-    as "the" and "a".
-  * Document contains many same keywords such as "They are keyword,
-    keyword, keyword ... and keyword". Search engine spammer may use
-    the technique.
-
-Score function can solve these cases. For example, `TF-IDF
-<https://en.wikipedia.org/wiki/Tf%E2%80%93idf>`_ (term
-frequency-inverse document frequency) can solve the first case.
-`Okapi BM25 <https://en.wikipedia.org/wiki/Okapi_BM25>`_ can solve the
-second case. But their are slower than TF.
-
-Groonga provides TF-IDF based scorer as
-:doc:`/reference/scorers/scorer_tf_idf` but doesn't provide Okapi BM25
-based scorer yet.
-
-.. include:: scoring_note.rst
-
-Usage
------
-
-This section describes how to use scorer.
-
-Here are a schema definition and sample data to show usage.
-
-Sample schema:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_setup_schema.log
-.. table_create Memos TABLE_HASH_KEY ShortText
-.. column_create Memos title COLUMN_SCALAR ShortText
-.. column_create Memos content COLUMN_SCALAR Text
-..
-.. table_create Terms TABLE_PAT_KEY ShortText \
-..   --default_tokenizer TokenBigram \
-..   --normalizer NormalizerAuto
-.. column_create Terms title_index COLUMN_INDEX|WITH_POSITION Memos title
-.. column_create Terms content_index COLUMN_INDEX|WITH_POSITION Memos content
-
-Sample data:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_setup_data.log
-.. load --table Memos
-.. [
-.. {
-..   "_key": "memo1",
-..   "title": "Groonga is easy",
-..   "content": "Groonga is very easy full text search engine!"
-.. },
-.. {
-..   "_key": "memo2",
-..   "title": "Mroonga is easy",
-..   "content": "Mroonga is more easier full text search engine!"
-.. },
-.. {
-..   "_key": "memo3",
-..   "title": "Rroonga is easy",
-..   "content": "Ruby is very helpful."
-.. },
-.. {
-..   "_key": "memo4",
-..   "title": "Groonga is fast",
-..   "content": "Groonga! Groonga! Groonga! Groonga is very fast!"
-.. },
-.. {
-..   "_key": "memo5",
-..   "title": "PGroonga is fast",
-..   "content": "PGroonga is very fast!"
-.. },
-.. {
-..   "_key": "memo6",
-..   "title": "PGroonga is useful",
-..   "content": "SQL is easy because many client libraries exist."
-.. },
-.. {
-..   "_key": "memo7",
-..   "title": "Mroonga is also useful",
-..   "content": "MySQL has replication feature. Mroonga can use it."
-.. }
-.. ]
-
-You can specify custom score function in :ref:`select-match-columns`.
-There are some syntaxes.
-
-For score function that doesn't require any parameter such as
-:doc:`/reference/scorers/scorer_tf_idf`::
-
-  SCORE_FUNCTION(COLUMN)
-
-You can specify weight::
-
-  SCORE_FUNCTION(COLUMN) * WEIGHT
-
-For score function that requires one or more parameters such as
-:doc:`/reference/scorers/scorer_tf_at_most`::
-
-  SCORE_FUNCTION(COLUMN, ARGUMENT1, ARGUMENT2, ...)
-
-You can specify weight::
-
-  SCORE_FUNCTION(COLUMN, ARGUMENT1, ARGUMENT2, ...) * WEIGHT
-
-You can use different score function for each
-:ref:`select-match-columns`::
-
-  SCORE_FUNCTION1(COLUMN1) ||
-    SCORE_FUNCTION2(COLUMN2) * WEIGHT ||
-    SCORE_FUNCTION3(COLUMN3, ARGUMENT1) ||
-    ...
-
-Here is a simplest example:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_one_no_argument_no_weight.log
-.. select Memos \
-..   --match_columns "scorer_tf_idf(content)" \
-..   --query "Groonga" \
-..   --output_columns "content, _score" \
-..   --sortby "-_score"
-
-``Groonga! Groonga! Groonga! Groonga is very fast!`` contains 4
-``Groonga``. If you use TF based scorer that is the default scorer,
-``_score`` is ``4``. But the actual ``_score`` is ``2``. Because the
-``select`` command uses TF-IDF based scorer ``scorer_tf_idf()``.
-
-Here is an example that uses weight:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_one_no_argument_weight.log
-.. select Memos \
-..   --match_columns "scorer_tf_idf(content) * 10" \
-..   --query "Groonga" \
-..   --output_columns "content, _score" \
-..   --sortby "-_score"
-
-``Groonga! Groonga! Groonga! Groonga is very fast!`` has ``22`` as
-``_score``. It had ``2`` as ``_score`` in the previous example that
-doesn't specify weight.
-
-Here is an example that uses scorer that requires one
-argument. :doc:`/reference/scorers/scorer_tf_at_most` scorer requires
-one argument. You can limit TF score by the scorer.
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_one_one_argument_no_weight.log
-.. select Memos \
-..   --match_columns "scorer_tf_at_most(content, 2.0)" \
-..   --query "Groonga" \
-..   --output_columns "content, _score" \
-..   --sortby "-_score"
-
-``Groonga! Groonga! Groonga! Groonga is very fast!`` contains 4
-``Groonga``. If you use normal TF based scorer that is the default
-scorer, ``_score`` is ``4``. But the actual ``_score`` is ``2``.
-Because the scorer used in the ``select`` command limits the maximum
-score value to ``2``.
-
-Here is an example that uses multiple scorers:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_multiple_scorers.log
-.. select Memos \
-..   --match_columns "scorer_tf_idf(title) || scorer_tf_at_most(content, 2.0)" \
-..   --query "Groonga" \
-..   --output_columns "title, content, _score" \
-..   --sortby "-_score"
-
-The ``--match_columns`` uses ``scorer_tf_idf(title)`` and
-``scorer_tf_at_most(content, 2.0)``. ``_score`` value is sum of them.
-
-You can use the default scorer and custom scorer in the same
-``--match_columns``. You can use the default scorer by just specifying
-a match column:
-
-.. groonga-command
-.. include:: ../example/reference/scorer/usage_default_and_custom_scorers.log
-.. select Memos \
-..   --match_columns "title || scorer_tf_at_most(content, 2.0)" \
-..   --query "Groonga" \
-..   --output_columns "title, content, _score" \
-..   --sortby "-_score"
-
-The ``--match_columns`` uses the default scorer (TF) for ``title`` and
-:doc:`/reference/scorers/scorer_tf_at_most` for
-``content``. ``_score`` value is sum of them.
-
-Built-in scorers
-----------------
-
-Here are built-in scores:
-
-.. toctree::
-   :maxdepth: 1
-   :glob:
-
-   scorers/*