esruby 0.0.0

data/resources/mruby/build/mrbgems/mruby-regexp-pcre/pcre/doc/pcrejit.3

@@ -0,0 +1,437 @@
+.TH PCREJIT 3 "31 October 2012" "PCRE 8.32"
+.SH NAME
+PCRE - Perl-compatible regular expressions
+.SH "PCRE JUST-IN-TIME COMPILER SUPPORT"
+.rs
+.sp
+Just-in-time compiling is a heavyweight optimization that can greatly speed up
+pattern matching. However, it comes at the cost of extra processing before the
+match is performed. Therefore, it is of most benefit when the same pattern is
+going to be matched many times. This does not necessarily mean many calls of a
+matching function; if the pattern is not anchored, matching attempts may take
+place many times at various positions in the subject, even for a single call.
+Therefore, if the subject string is very long, it may still pay to use JIT for
+one-off matches.
+.P
+JIT support applies only to the traditional Perl-compatible matching function.
+It does not apply when the DFA matching function is being used. The code for
+this support was written by Zoltan Herczeg.
+.
+.
+.SH "8-BIT, 16-BIT AND 32-BIT SUPPORT"
+.rs
+.sp
+JIT support is available for all of the 8-bit, 16-bit and 32-bit PCRE
+libraries. To keep this documentation simple, only the 8-bit interface is
+described in what follows. If you are using the 16-bit library, substitute the
+16-bit functions and 16-bit structures (for example, \fIpcre16_jit_stack\fP
+instead of \fIpcre_jit_stack\fP). If you are using the 32-bit library,
+substitute the 32-bit functions and 32-bit structures (for example,
+\fIpcre32_jit_stack\fP instead of \fIpcre_jit_stack\fP).
+.
+.
+.SH "AVAILABILITY OF JIT SUPPORT"
+.rs
+.sp
+JIT support is an optional feature of PCRE. The "configure" option --enable-jit
+(or equivalent CMake option) must be set when PCRE is built if you want to use
+JIT. The support is limited to the following hardware platforms:
+.sp
+  ARM v5, v7, and Thumb2
+  Intel x86 32-bit and 64-bit
+  MIPS 32-bit
+  Power PC 32-bit and 64-bit
+  SPARC 32-bit (experimental)
+.sp
+If --enable-jit is set on an unsupported platform, compilation fails.
+.P
+A program that is linked with PCRE 8.20 or later can tell if JIT support is
+available by calling \fBpcre_config()\fP with the PCRE_CONFIG_JIT option. The
+result is 1 when JIT is available, and 0 otherwise. However, a simple program
+does not need to check this in order to use JIT. The normal API is implemented
+in a way that falls back to the interpretive code if JIT is not available. For
+programs that need the best possible performance, there is also a "fast path"
+API that is JIT-specific.
+.P
+If your program may sometimes be linked with versions of PCRE that are older
+than 8.20, but you want to use JIT when it is available, you can test
+the values of PCRE_MAJOR and PCRE_MINOR, or the existence of a JIT macro such
+as PCRE_CONFIG_JIT, for compile-time control of your code.
+.
+.
+.SH "SIMPLE USE OF JIT"
+.rs
+.sp
+You have to do two things to make use of the JIT support in the simplest way:
+.sp
+  (1) Call \fBpcre_study()\fP with the PCRE_STUDY_JIT_COMPILE option for
+      each compiled pattern, and pass the resulting \fBpcre_extra\fP block to
+      \fBpcre_exec()\fP.
+.sp
+  (2) Use \fBpcre_free_study()\fP to free the \fBpcre_extra\fP block when it is
+      no longer needed, instead of just freeing it yourself. This ensures that
+      any JIT data is also freed.
+.sp
+For a program that may be linked with pre-8.20 versions of PCRE, you can insert
+.sp
+  #ifndef PCRE_STUDY_JIT_COMPILE
+  #define PCRE_STUDY_JIT_COMPILE 0
+  #endif
+.sp
+so that no option is passed to \fBpcre_study()\fP, and then use something like
+this to free the study data:
+.sp
+  #ifdef PCRE_CONFIG_JIT
+      pcre_free_study(study_ptr);
+  #else
+      pcre_free(study_ptr);
+  #endif
+.sp
+PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete
+matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or
+PCRE_PARTIAL_SOFT options of \fBpcre_exec()\fP, you should set one or both of
+the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE
+when you call \fBpcre_study()\fP:
+.sp
+  PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
+  PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
+.sp
+The JIT compiler generates different optimized code for each of the three
+modes (normal, soft partial, hard partial). When \fBpcre_exec()\fP is called,
+the appropriate code is run if it is available. Otherwise, the pattern is
+matched using interpretive code.
+.P
+In some circumstances you may need to call additional functions. These are
+described in the section entitled
+.\" HTML <a href="#stackcontrol">
+.\" </a>
+"Controlling the JIT stack"
+.\"
+below.
+.P
+If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and
+no JIT data is created. Otherwise, the compiled pattern is passed to the JIT
+compiler, which turns it into machine code that executes much faster than the
+normal interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP
+block containing a pointer to JIT code of the appropriate mode (normal or
+hard/soft partial), it obeys that code instead of running the interpreter. The
+result is identical, but the compiled JIT code runs much faster.
+.P
+There are some \fBpcre_exec()\fP options that are not supported for JIT
+execution. There are also some pattern items that JIT cannot handle. Details
+are given below. In both cases, execution automatically falls back to the
+interpretive code. If you want to know whether JIT was actually used for a
+particular match, you should arrange for a JIT callback function to be set up
+as described in the section entitled
+.\" HTML <a href="#stackcontrol">
+.\" </a>
+"Controlling the JIT stack"
+.\"
+below, even if you do not need to supply a non-default JIT stack. Such a
+callback function is called whenever JIT code is about to be obeyed. If the
+execution options are not right for JIT execution, the callback function is not
+obeyed.
+.P
+If the JIT compiler finds an unsupported item, no JIT data is generated. You
+can find out if JIT execution is available after studying a pattern by calling
+\fBpcre_fullinfo()\fP with the PCRE_INFO_JIT option. A result of 1 means that
+JIT compilation was successful. A result of 0 means that JIT support is not
+available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or
+the JIT compiler was not able to handle the pattern.
+.P
+Once a pattern has been studied, with or without JIT, it can be used as many
+times as you like for matching different subject strings.
+.
+.
+.SH "UNSUPPORTED OPTIONS AND PATTERN ITEMS"
+.rs
+.sp
+The only \fBpcre_exec()\fP options that are supported for JIT execution are
+PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK, PCRE_NO_UTF32_CHECK, PCRE_NOTBOL,
+PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and
+PCRE_PARTIAL_SOFT.
+.P
+The unsupported pattern items are:
+.sp
+  \eC             match a single byte; not supported in UTF-8 mode
+  (?Cn)          callouts
+  (*PRUNE)       )
+  (*SKIP)        ) backtracking control verbs
+  (*THEN)        )
+.sp
+Support for some of these may be added in future.
+.
+.
+.SH "RETURN VALUES FROM JIT EXECUTION"
+.rs
+.sp
+When a pattern is matched using JIT execution, the return values are the same
+as those given by the interpretive \fBpcre_exec()\fP code, with the addition of
+one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means that the memory used
+for the JIT stack was insufficient. See
+.\" HTML <a href="#stackcontrol">
+.\" </a>
+"Controlling the JIT stack"
+.\"
+below for a discussion of JIT stack usage. For compatibility with the
+interpretive \fBpcre_exec()\fP code, no more than two-thirds of the
+\fIovector\fP argument is used for passing back captured substrings.
+.P
+The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if searching a
+very large pattern tree goes on for too long, as it is in the same circumstance
+when JIT is not used, but the details of exactly what is counted are not the
+same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT
+execution.
+.
+.
+.SH "SAVING AND RESTORING COMPILED PATTERNS"
+.rs
+.sp
+The code that is generated by the JIT compiler is architecture-specific, and is
+also position dependent. For those reasons it cannot be saved (in a file or
+database) and restored later like the bytecode and other data of a compiled
+pattern. Saving and restoring compiled patterns is not something many people
+do. More detail about this facility is given in the
+.\" HREF
+\fBpcreprecompile\fP
+.\"
+documentation. It should be possible to run \fBpcre_study()\fP on a saved and
+restored pattern, and thereby recreate the JIT data, but because JIT
+compilation uses significant resources, it is probably not worth doing this;
+you might as well recompile the original pattern.
+.
+.
+.\" HTML <a name="stackcontrol"></a>
+.SH "CONTROLLING THE JIT STACK"
+.rs
+.sp
+When the compiled JIT code runs, it needs a block of memory to use as a stack.
+By default, it uses 32K on the machine stack. However, some large or
+complicated patterns need more than this. The error PCRE_ERROR_JIT_STACKLIMIT
+is given when there is not enough stack. Three functions are provided for
+managing blocks of memory for use as JIT stacks. There is further discussion
+about the use of JIT stacks in the section entitled
+.\" HTML <a href="#stackcontrol">
+.\" </a>
+"JIT stack FAQ"
+.\"
+below.
+.P
+The \fBpcre_jit_stack_alloc()\fP function creates a JIT stack. Its arguments
+are a starting size and a maximum size, and it returns a pointer to an opaque
+structure of type \fBpcre_jit_stack\fP, or NULL if there is an error. The
+\fBpcre_jit_stack_free()\fP function can be used to free a stack that is no
+longer needed. (For the technically minded: the address space is allocated by
+mmap or VirtualAlloc.)
+.P
+JIT uses far less memory for recursion than the interpretive code,
+and a maximum stack size of 512K to 1M should be more than enough for any
+pattern.
+.P
+The \fBpcre_assign_jit_stack()\fP function specifies which stack JIT code
+should use. Its arguments are as follows:
+.sp
+  pcre_extra         *extra
+  pcre_jit_callback  callback
+  void               *data
+.sp
+The \fIextra\fP argument must be the result of studying a pattern with
+PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other
+two options:
+.sp
+  (1) If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block
+      on the machine stack is used.
+.sp
+  (2) If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must be
+      a valid JIT stack, the result of calling \fBpcre_jit_stack_alloc()\fP.
+.sp
+  (3) If \fIcallback\fP is not NULL, it must point to a function that is
+      called with \fIdata\fP as an argument at the start of matching, in
+      order to set up a JIT stack. If the return from the callback
+      function is NULL, the internal 32K stack is used; otherwise the
+      return value must be a valid JIT stack, the result of calling
+      \fBpcre_jit_stack_alloc()\fP.
+.sp
+A callback function is obeyed whenever JIT code is about to be run; it is not
+obeyed when \fBpcre_exec()\fP is called with options that are incompatible for
+JIT execution. A callback function can therefore be used to determine whether a
+match operation was executed by JIT or by the interpreter.
+.P
+You may safely use the same JIT stack for more than one pattern (either by
+assigning directly or by callback), as long as the patterns are all matched
+sequentially in the same thread. In a multithread application, if you do not
+specify a JIT stack, or if you assign or pass back NULL from a callback, that
+is thread-safe, because each thread has its own machine stack. However, if you
+assign or pass back a non-NULL JIT stack, this must be a different stack for
+each thread so that the application is thread-safe.
+.P
+Strictly speaking, even more is allowed. You can assign the same non-NULL stack
+to any number of patterns as long as they are not used for matching by multiple
+threads at the same time. For example, you can assign the same stack to all
+compiled patterns, and use a global mutex in the callback to wait until the
+stack is available for use. However, this is an inefficient solution, and not
+recommended.
+.P
+This is a suggestion for how a multithreaded program that needs to set up
+non-default JIT stacks might operate:
+.sp
+  During thread initalization
+    thread_local_var = pcre_jit_stack_alloc(...)
+.sp
+  During thread exit
+    pcre_jit_stack_free(thread_local_var)
+.sp
+  Use a one-line callback function
+    return thread_local_var
+.sp
+All the functions described in this section do nothing if JIT is not available,
+and \fBpcre_assign_jit_stack()\fP does nothing unless the \fBextra\fP argument
+is non-NULL and points to a \fBpcre_extra\fP block that is the result of a
+successful study with PCRE_STUDY_JIT_COMPILE etc.
+.
+.
+.\" HTML <a name="stackfaq"></a>
+.SH "JIT STACK FAQ"
+.rs
+.sp
+(1) Why do we need JIT stacks?
+.sp
+PCRE (and JIT) is a recursive, depth-first engine, so it needs a stack where
+the local data of the current node is pushed before checking its child nodes.
+Allocating real machine stack on some platforms is difficult. For example, the
+stack chain needs to be updated every time if we extend the stack on PowerPC.
+Although it is possible, its updating time overhead decreases performance. So
+we do the recursion in memory.
+.P
+(2) Why don't we simply allocate blocks of memory with \fBmalloc()\fP?
+.sp
+Modern operating systems have a nice feature: they can reserve an address space
+instead of allocating memory. We can safely allocate memory pages inside this
+address space, so the stack could grow without moving memory data (this is
+important because of pointers). Thus we can allocate 1M address space, and use
+only a single memory page (usually 4K) if that is enough. However, we can still
+grow up to 1M anytime if needed.
+.P
+(3) Who "owns" a JIT stack?
+.sp
+The owner of the stack is the user program, not the JIT studied pattern or
+anything else. The user program must ensure that if a stack is used by
+\fBpcre_exec()\fP, (that is, it is assigned to the pattern currently running),
+that stack must not be used by any other threads (to avoid overwriting the same
+memory area). The best practice for multithreaded programs is to allocate a
+stack for each thread, and return this stack through the JIT callback function.
+.P
+(4) When should a JIT stack be freed?
+.sp
+You can free a JIT stack at any time, as long as it will not be used by
+\fBpcre_exec()\fP again. When you assign the stack to a pattern, only a pointer
+is set. There is no reference counting or any other magic. You can free the
+patterns and stacks in any order, anytime. Just \fIdo not\fP call
+\fBpcre_exec()\fP with a pattern pointing to an already freed stack, as that
+will cause SEGFAULT. (Also, do not free a stack currently used by
+\fBpcre_exec()\fP in another thread). You can also replace the stack for a
+pattern at any time. You can even free the previous stack before assigning a
+replacement.
+.P
+(5) Should I allocate/free a stack every time before/after calling
+\fBpcre_exec()\fP?
+.sp
+No, because this is too costly in terms of resources. However, you could
+implement some clever idea which release the stack if it is not used in let's
+say two minutes. The JIT callback can help to achieve this without keeping a
+list of the currently JIT studied patterns.
+.P
+(6) OK, the stack is for long term memory allocation. But what happens if a
+pattern causes stack overflow with a stack of 1M? Is that 1M kept until the
+stack is freed?
+.sp
+Especially on embedded sytems, it might be a good idea to release memory
+sometimes without freeing the stack. There is no API for this at the moment.
+Probably a function call which returns with the currently allocated memory for
+any stack and another which allows releasing memory (shrinking the stack) would
+be a good idea if someone needs this.
+.P
+(7) This is too much of a headache. Isn't there any better solution for JIT
+stack handling?
+.sp
+No, thanks to Windows. If POSIX threads were used everywhere, we could throw
+out this complicated API.
+.
+.
+.SH "EXAMPLE CODE"
+.rs
+.sp
+This is a single-threaded example that specifies a JIT stack without using a
+callback.
+.sp
+  int rc;
+  int ovector[30];
+  pcre *re;
+  pcre_extra *extra;
+  pcre_jit_stack *jit_stack;
+.sp
+  re = pcre_compile(pattern, 0, &error, &erroffset, NULL);
+  /* Check for errors */
+  extra = pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error);
+  jit_stack = pcre_jit_stack_alloc(32*1024, 512*1024);
+  /* Check for error (NULL) */
+  pcre_assign_jit_stack(extra, NULL, jit_stack);
+  rc = pcre_exec(re, extra, subject, length, 0, 0, ovector, 30);
+  /* Check results */
+  pcre_free(re);
+  pcre_free_study(extra);
+  pcre_jit_stack_free(jit_stack);
+.sp
+.
+.
+.SH "JIT FAST PATH API"
+.rs
+.sp
+Because the API described above falls back to interpreted execution when JIT is
+not available, it is convenient for programs that are written for general use
+in many environments. However, calling JIT via \fBpcre_exec()\fP does have a
+performance impact. Programs that are written for use where JIT is known to be
+available, and which need the best possible performance, can instead use a
+"fast path" API to call JIT execution directly instead of calling
+\fBpcre_exec()\fP (obviously only for patterns that have been successfully
+studied by JIT).
+.P
+The fast path function is called \fBpcre_jit_exec()\fP, and it takes exactly
+the same arguments as \fBpcre_exec()\fP, plus one additional argument that
+must point to a JIT stack. The JIT stack arrangements described above do not
+apply. The return values are the same as for \fBpcre_exec()\fP.
+.P
+When you call \fBpcre_exec()\fP, as well as testing for invalid options, a
+number of other sanity checks are performed on the arguments. For example, if
+the subject pointer is NULL, or its length is negative, an immediate error is
+given. Also, unless PCRE_NO_UTF[8|16|32] is set, a UTF subject string is tested
+for validity. In the interests of speed, these checks do not happen on the JIT
+fast path, and if invalid data is passed, the result is undefined.
+.P
+Bypassing the sanity checks and the \fBpcre_exec()\fP wrapping can give
+speedups of more than 10%.
+.
+.
+.SH "SEE ALSO"
+.rs
+.sp
+\fBpcreapi\fP(3)
+.
+.
+.SH AUTHOR
+.rs
+.sp
+.nf
+Philip Hazel (FAQ by Zoltan Herczeg)
+University Computing Service
+Cambridge CB2 3QH, England.
+.fi
+.
+.
+.SH REVISION
+.rs
+.sp
+.nf
+Last updated: 31 October 2012
+Copyright (c) 1997-2012 University of Cambridge.
+.fi

data/resources/mruby/build/mrbgems/mruby-regexp-pcre/pcre/doc/pcrelimits.3

@@ -0,0 +1,67 @@
+.TH PCRELIMITS 3 "24 June 2012" "PCRE 8.30"
+.SH NAME
+PCRE - Perl-compatible regular expressions
+.SH "SIZE AND OTHER LIMITATIONS"
+.rs
+.sp
+There are some size limitations in PCRE but it is hoped that they will never in
+practice be relevant.
+.P
+The maximum length of a compiled pattern is approximately 64K data units (bytes
+for the 8-bit library, 32-bit units for the 32-bit library, and 32-bit units for
+the 32-bit library) if PCRE is compiled with the default internal linkage size
+of 2 bytes. If you want to process regular expressions that are truly enormous,
+you can compile PCRE with an internal linkage size of 3 or 4 (when building the
+16-bit or 32-bit library, 3 is rounded up to 4). See the \fBREADME\fP file in
+the source distribution and the
+.\" HREF
+\fBpcrebuild\fP
+.\"
+documentation for details. In these cases the limit is substantially larger.
+However, the speed of execution is slower.
+.P
+All values in repeating quantifiers must be less than 65536.
+.P
+There is no limit to the number of parenthesized subpatterns, but there can be
+no more than 65535 capturing subpatterns.
+.P
+There is a limit to the number of forward references to subsequent subpatterns
+of around 200,000. Repeated forward references with fixed upper limits, for
+example, (?2){0,100} when subpattern number 2 is to the right, are included in
+the count. There is no limit to the number of backward references.
+.P
+The maximum length of name for a named subpattern is 32 characters, and the
+maximum number of named subpatterns is 10000.
+.P
+The maximum length of a name in a (*MARK), (*PRUNE), (*SKIP), or (*THEN) verb
+is 255 for the 8-bit library and 65535 for the 16-bit and 32-bit library.
+.P
+The maximum length of a subject string is the largest positive number that an
+integer variable can hold. However, when using the traditional matching
+function, PCRE uses recursion to handle subpatterns and indefinite repetition.
+This means that the available stack space may limit the size of a subject
+string that can be processed by certain patterns. For a discussion of stack
+issues, see the
+.\" HREF
+\fBpcrestack\fP
+.\"
+documentation.
+.
+.
+.SH AUTHOR
+.rs
+.sp
+.nf
+Philip Hazel
+University Computing Service
+Cambridge CB2 3QH, England.
+.fi
+.
+.
+.SH REVISION
+.rs
+.sp
+.nf
+Last updated: 04 May 2012
+Copyright (c) 1997-2012 University of Cambridge.
+.fi