perftester 0.4.0__tar.gz → 0.6.0__tar.gz

{perftester-0.4.0 → perftester-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: perftester
-Version: 0.4.0
+Version: 0.6.0
 Summary: Lightweight performance testing in Python
 Home-page: https://github.com/nyggus/perftester
 Author: Nyggus
@@ -18,13 +18,12 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         The package has three external dependencies: [`memory_profiler`](https://pypi.org/project/memory-profiler/) ([repo](https://github.com/pythonprofilers/memory_profiler)), [`easycheck`](https://pypi.org/project/easycheck/) ([repo](https://github.com/nyggus/easycheck)), and [`rounder`](https://pypi.org/project/rounder/) ([repo](https://github.com/nyggus/rounder)).
         
-        > `perftester` is still under heavy testing. If you find anything that does not work as intended, please let me know via nyggus <at> gmail.com.
+        > `perftester` is still under heavy testing. If you find anything that does not work as intended, please let me know via nyggus `<at>` gmail.com.
         
         ## Pre-introduction: TL;DR
         
         At the most basic level, using `perftester` is simple. It offers you two functions for benchmarking (one for execution time and one for memory), and two functions for performance testing (likewise). Read below for a very short introduction of them. If you want to learn more, however, do not stop there, but read on.
         
-        
         ### Benchmarking
         
         You have `time_benchmark()` and `memory_benchmark()` functions:
@@ -34,9 +33,10 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         def foo(x, n): return [x] * n
         pt.time_benchmark(foo, x=129, n=100)
         ```
+        
         and this will print the results of the time benchmark, with raw results similar to those that `timeit.repeat()` returns, but unlike it, `pt.time_benchmark()` returns mean raw time per function run, not overall; in additional, you will see some summaries of the results.
         
-        The above call did actually run  `timeit.repeat()` function, with the default configuration of `number=100_000` and `repeat=5`. If you want to change any of these, you can use arguments `Number` and `Repeat`, correspondigly:
+        The above call did actually run  `timeit.repeat()` function, with the default configuration of `Number=100_000` and `Repeat=5`. If you want to change any of these, you can use arguments `Number` and `Repeat`, correspondigly:
         
         ```python
         pt.time_benchmark(foo, x=129, n=100, Number=1000)
@@ -46,7 +46,7 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         These calls do not change the default settings so you use the arguments' values on the fly. Later you will learn how to change the default settings and the settings for a particular function.
         
-        > Some of you may wonder why the `Number` and `Repeat` arguments violate what we can call the Pythonic style, by using a capital first letter for function arguments. The reason is simple: I wanted to minimize a risk of conflicts that would happen when benchmarking (or testing) a function with any of the arguments `number` or `repeat` (or both). A chance that a Python function will have a `Number` or a `Repeat` argument is rather small. If that happens, however, you can use `functools.partial()` to overcome the problem:
+        > Some of you may wonder why the `Number` and `Repeat` arguments violate what we can call the Pythonic style, by using a capital first letter for function arguments. The reason is simple: I wanted to minimize a risk of conflicts that would happen when benchmarking (or testing) a function with any of the arguments `Number` or `Repeat` (or both). A chance that a Python function will have a `Number` or a `Repeat` argument is rather small. If that happens, however, you can use `functools.partial()` to overcome the problem:
         
         ```python
         from functools import partial
@@ -88,7 +88,7 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         >>> pt.memory_usage_test(foo, raw_limit=25, x=129, n=100)
         
         # A relative test
-        >>> pt.memory_usage_test(foo, relative_limit=1.01, x=129, n=100)
+        >>> pt.memory_usage_test(foo, relative_limit=1.2, x=129, n=100)
         
         ```
         
@@ -109,16 +109,17 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         ## Introduction
         
-        
         `perftester` is a lightweight package for simple performance testing in Python. Here, performance refers to execution time and memory usage, so performance testing means testing if a function performs quickly enough and does not use too much RAM. In addition, the module offers you simple functions for straightforward benchmarking, in terms of both execution time and memory.
         
         Under the hood, `perftester` is a wrapper around two functions from other modules:
+        
         * `perftester.time_benchmark()` and `perftester.time_test()` use `timeit.repeat()`
         * `perftester.memory_usage_benchmark()` and `perftester.memory_usage_test()` use `memory_profiler.memory_usage()`
         
         What `perftester` offers is a testing framework with as simple syntax as possible.
         
         You can use `perftester` in three main ways:
+        
         * in an interactive session, for simple benchmarking of functions;
         * as part of another testing framework, like `doctest` or `pytest`s; and
         * as an independent testing framework.
@@ -127,7 +128,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         When it comes to actual testing, it's difficult to say which of the last two ways is better or more convinient: it may depend on how many performance tests you have, and how much time they take. If the tests do not take more than a couple of seconds, then you can combine them with unit tests. But if they take much time, you should likely make them independent of unit tests, and run them from time to time.
         
-        
         ## Using `perftester`
         
         ### Use it as a separate testing framework
@@ -148,10 +148,9 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         > There is no best approach, but remember to choose one that suits your needs.
         
-        
         ### Use `perftester` inside `pytest`
         
-        This is a very simple approach, perhaps the simplest one: When you use `pytest`, you can simply add `perftester` testing functions to `pytest` testing functions, and that way both frameworks will be combined, or rather the `pytest` framework will run `perftester` tests. The amount of additional work is minimal. 
+        This is a very simple approach, perhaps the simplest one: When you use `pytest`, you can simply add `perftester` testing functions to `pytest` testing functions, and that way both frameworks will be combined, or rather the `pytest` framework will run `perftester` tests. The amount of additional work is minimal.
         
         For instance, you can write the following test function:
         
@@ -184,19 +183,16 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         This is the easiest way to use `perftester`. Its only drawback is that if the performance tests take much time, `pytest` will also take much time, something usually to be avoided. You can then do some `pytest` tricks to not run `perftester` tests, and run them only when you want — or you can simply use the above-described command-line `perftester` framework for performance testing.
         
-        
         ### Use `perftester` inside `doctest`
         
-        In the same way, you can use `perftester` in `doctest`. You will find plenty of examples in the documentation here, and in the [tests/ folder](tests/). 
+        In the same way, you can use `perftester` in `doctest`. You will find plenty of examples in the documentation here, and in the [tests/ folder](tests/).
         
-        > A great fan of `doctest`ing, I do **not** recommend using `perftester` in docstrings. For me, `doctest`s in docstrings should clarify things and explain how functions work, and adding a performance test to a function's docstring would decrease readability. 
+        > A great fan of `doctest`ing, I do **not** recommend using `perftester` in docstrings. For me, `doctest`s in docstrings should clarify things and explain how functions work, and adding a performance test to a function's docstring would decrease readability.
         
         The best way, thus, is to write performance tests as separate `doctest` files, dedicated to performance testing. You can collect such files in a shell script that runs performance tests.
         
-        
         ## Basic use of `perftester`
         
-        
         ### Simple benchmarking
         
         To create a performance test for a function, you likely need to know how it behaves. You can run two simple benchmarking functions, `pt.memory_usage_benchmark()` and `pt.time_benchmark()`, which will run time and memory benchmarks, respectively. First, we will decrease `number` (passed to `timeit.repeat`), in order to shorten the benchmarks (which here serve as `doctest`s):
@@ -204,7 +200,7 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         ```python
         >>> import perftester as pt
         >>> def f(n): return sum(map(lambda i: i**0.5, range(n)))
-        >>> pt.config.set(f, "time", number=1000)
+        >>> pt.config.set(f, "time", Number=1000)
         >>> b_100_time = pt.time_benchmark(f, n=100)
         >>> b_100_memory = pt.memory_usage_benchmark(f, n=100)
         >>> b_1000_time = pt.time_benchmark(f, n=1000)
@@ -260,7 +256,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         For time tests, we have the `pt.time_test()` function. First, a raw time test:
         
-        
         ```python
         >>> pt.time_test(f, raw_limit=2e-05, n=100)
         
@@ -289,8 +284,7 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         ```
         
-        You can read about relative testing below, [in section](#raw-and-relative-performance-testing). 
-        
+        You can read about relative testing below, [in section](#raw-and-relative-performance-testing).
         
         ### Memory testing
         
@@ -298,15 +292,15 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         ```python
         >>> pt.memory_usage_test(f, raw_limit=27, n=100) # test on raw memory
-        >>> pt.memory_usage_test(f, relative_limit=1.01, n=100) # relative time test
-        >>> pt.memory_usage_test(f, raw_limit=27, relative_limit=1.01, n=100) # both
+        >>> pt.memory_usage_test(f, relative_limit=1.2, n=100) # relative time test
+        >>> pt.memory_usage_test(f, raw_limit=27, relative_limit=1.2, n=100) # both
         
         ```
         
         In a memory usage test, a function is called only once. You can change that — but do that only if you have solid reasons — using, for example, `pt.config.set(f, "time", "repeat", 2)`, which will set this setting for the function in the configuration (so it will be used for all next calls for function `f()`). You can also do it just once (so, without saving the setting in `pt.config.settings`), using the `Repeat` argument:
         
         ```python
-        >>> pt.memory_usage_test(f, raw_limit=27, relative_limit=1.01, n=100, Repeat=100)
+        >>> pt.memory_usage_test(f, raw_limit=27, relative_limit=1.2, n=100, Repeat=100)
         
         ```
         
@@ -314,14 +308,13 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         Of course, memory tests do not have to be very useful for functions that do not have to allocate too much memory, but as you will see in other documentation files in `perftester`, some function do use a lot of memory, and such tests do make quite a lot sense for them.
         
-        
         ## Configuration: `pt.config`
         
         The whole configuration is stored in the `pt.config` object, which you can easily change. Here's a short example of how you can use it:
         
         ```python
         >>> def f(n): return list(range(n))
-        >>> pt.config.set(f, "time", number=10_000, repeat=1)
+        >>> pt.config.set(f, "time", Number=10_000, Repeat=1)
         
         ```
         
@@ -334,7 +327,7 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         import perftester as pt
         
         # shorten the tests
-        pt.config.set_defaults("time", number=10_000, repeat=3) 
+        pt.config.set_defaults("time", Number=10_000, Repeat=3) 
         
         # log the results to file (they will be printed in the console anyway)
         pt.config.log_to_file = True
@@ -352,7 +345,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         When you use `perftester` in an interactive session, you update `pt.config` in a normal way, in the session. And when you use `perftester` inside `pytest`, you can do it in conftest.py and in each testing function.
         
-        
         ## Output
         
         If a test fails, you will see something like this:
@@ -383,7 +375,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         > Like in `pytest`, a recommended approach is to use one performance test per `perftester_` function. This can save you some time and trouble, but also this will ensure that all tests will be run.
         
-        
         #### Summary output
         
         At the end, you will see a simple summary of the results, something like this:
@@ -404,7 +395,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         perftester_for_testing.perftester_f_2
         ```
         
-        
         ## Relative tests against another function
         
         In the basic use, when you choose a relative benchmark, you compare the performance of your function with that of a built-in (empty) function `pt.config.benchmark_function()`. In most cases, this is what you need. Sometimes, however, you may wish to benchmark against another function. For instance, you may want to build your own function that does the same thing as a Python built-in function, and you want to test (and show) that your function performs better. There are two ways of achieving this:
@@ -412,7 +402,6 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         * you can use a simple trick; [see here](benchmarking_against_another_function.md);
         * you can overwrite the built-in benchmark functions; [see here](change_benchmarking_function.md).
         
-        
         ## Raw and relative performance testing
         
         Surely, any performance tests are strongly environment-dependent, so you need to remember that when writing and conducting any performance tests. `perftester`, however, offers a solution to this: You can define tests based on
@@ -428,6 +417,99 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         > Warning! Relative results can be different between operating systems.
         
+        ## Tracing full memory usage
+        
+        Currently, `perftester` contains a beta version (under heavy testing) of a new feature that can be used to trace full memory usage of a Python program.
+        
+        >  Warning: Backward compatibility of this feature is not guaranteed! It does not affect the main functionality of `perftester`, however, so its backward compatibility should be kept.
+        
+        The feature works in the following way. When you import `perftester` — but you need to do it with `import perftester`, not via importing particular objects — you will be able to see new objects in the global space. One of the is `MEMLOGS`:
+        
+        ```python-repl
+        >>> import perftester
+        >>> MEMLOGS[0].ID
+        'perftester import'
+        
+        ```
+        
+        It's an empty list for the moment. When you start tracing memory using `perftester`, this list will collect the subsequent measurements. You can measure them in two ways. One is via a `MEMPOINT()` function, and another via a  `MEMTRACE` decorator. They, too, are in the global scope, so you can use them in any module inside a session in which `perftester` was already imported.
+        
+        The  `MEMLOGS` list will contain elements being instances of `MemLog`, which is a `functools.namedtuple `data type, with two attributes:`"ID"`and `"memory"`. This data type is imported with `perftester`, so if you want to use it, you can reach it as `perftester.MemLog`. You don't have to use it, though. Since it's a named tuple, you can treat it as a regular tuple.
+        
+        #### What sort of memory is measured?
+        
+        The feature uses `pympler.asizeof.asizeof(all=True)` to measure the size of all current gc objects, including module, global and stack frame objects, minus the size of `MEMLOGS`. The memory is measured in MB.
+        
+        #### Using `MEMPOINT()`
+        
+        `MEMPOINT()` creates a point of full-memory measurement. It will be appended into `MEMLOGS`.
+        
+        ```python-repl
+        3>>> import perftester
+        >>> def foo(n):
+        ...     x = [i for i in range(n)]
+        ...     MEMPOINT()
+        ...     return x
+        >>> _ = foo(100)
+        >>> _ = foo(1_000_000)
+        >>> len(MEMLOGS)
+        3
+        >>> MEMLOGS[2].memory > MEMLOGS[1].memory
+        True
+        
+        ```
+        
+        The last tests checks whether the second measurement — that is, from the function with `n` of a million — uses more memory that the function using `n` of a hundred. Makes sense, and indeed the test passes.
+        
+        When creating a point, you can use an ID, for instance, `MEMPOINT("from sth() function")`.
+        
+        `MEMPOINT()` can be used to create a point anywhere inside the code. Nevertheless, if you want to trace memory for a function, you can use a `MEMTRACE` decorator:
+        
+        ```python-repl
+        >>> @MEMTRACE
+        ... def bar(n):
+        ...     return [i for i in range(n)]
+        >>> _ = bar(1_000_000)
+        >>> MEMLOGS[-2].memory < MEMLOGS[-1].memory
+        True
+        
+        ```
+        
+        The decorator creates two points: one right before running the test and another right after returning.
+        
+        The last line tests whether memory before running the function is smaller than that after running it — and given so big `n`, it should be.
+        
+        Look here:
+        
+        ```python-repl
+        >>> @MEMTRACE
+        ... def bar(n):
+        ...     x = [i for i in range(n)]
+        ...     y = [i/3 for i in x]
+        ...     z = [i/3 for i in y]
+        ...     MEMPOINT("with x, y, z")
+        ...     del x
+        ...     MEMPOINT("without x")
+        ...     del y
+        ...     MEMPOINT("without x and y")
+        ...     del z
+        ...     MEMPOINT("without x and y and z")
+        ...     return 
+        >>> _ = bar(100_000)
+        >>> MEMLOGS[-3].memory > MEMLOGS[-2].memory > MEMLOGS[-1].memory
+        True
+        
+        ```
+        
+        ### Print `MEMLOGS`
+        
+        You can do whatever you want with `MEMLOGS`. However, when you want to see this object nicely printed, use the `MEMPRINT()` function, available from the global scope, too. You will see the results printed in a pretty way, with memory provided in MB.
+        
+        ### Why the global scope?
+        
+        Since this feature of `perftester` is to be used to debug memory use from various modules, it'd be inconvinient to import the required objects in all these modules. That's why for the moment, the required objects are kept in the global scope — but this can change in future versions.
+        
+        If you have any comments about this, please share them via Issues of the package's repository.
         
         ## Other tools
         
@@ -436,9 +518,8 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         * [`cProfile` and `profile`](https://docs.python.org/3/library/profile.html), the built-in powerful tools for deterministic profiling
         * [the built-in `timeit` module](https://docs.python.org/3/library/timeit.html), for benchmarking
         * [`memory_profiler`](https://pypi.org/project/memory-profiler/), a powerful memory profiler (`memory_profiler` is utilized by `perftester`)
-          
-        In fact, `perftester` is just a simple wrapper around `timeit` and `memory_profiler`, since `perftester` itself does not come with its own solutions. It simply uses these functions and offers an easy-to-use API to benchmark and test memory and time performance.
         
+        In fact, `perftester` is just a simple wrapper around `timeit` and `memory_profiler`, since `perftester` itself does not come with its own solutions. It simply uses these functions and offers an easy-to-use API to benchmark and test memory and time performance.
         
         ## Manipulating the traceback
         
@@ -446,19 +527,16 @@ Description: # `perftester`: Lightweight performance testing of Python functions
         
         > This behavior will not affect any other function than the two `perftester` testing functions: `pt.time_test()` and `pt.memory_usage_test()`. If you want to use this behavior for other functions, too, you can use `pt.config.cut_traceback()`; to reverse, use `pt.config.full_traceback()`.
         
-        
         ## Caveats
         
         * `perftester` does not work with multiple threads or processes.
         * `perftester` is still in a beta version and so is still under testing.
         * Watch out when you're running the same test in different operating systems. Even relative tests can differ from OS to OS.
         
-        
         ## Operating systems
         
         The package is developed in Linux (actually, under WSL) and checked in Windows 10, so it works in both these environments.
         
-        
         ## Support
         
         Any contribution will be welcome. You can submit an issue in the [repository](https://github.com/nyggus/perftester). You can also create your own pull requests.