locust 2.28.1.dev54__tar.gz → 2.28.1.dev66__tar.gz

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: locust
-Version: 2.28.1.dev54
+Version: 2.28.1.dev66
 Summary: Developer-friendly load testing framework
 License: MIT
 Project-URL: Homepage, https://github.com/locustio/locust

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/configuration.rst

@@ -191,9 +191,6 @@ With the following file structure:
 The Web UI will display:
 
 .. image:: images/userclass_picker_example.png
-    :width: 200
-
-|
 
 The class picker additionally allows for disabling individual User tasks, changing the weight or fixed count, and configuring the host.
 
@@ -274,3 +271,15 @@ The list of statistics parameters that can be modified is:
 +-------------------------------------------+--------------------------------------------------------------------------------------+
 | PERCENTILES_TO_STATISTICS                 | List of response time percentiles in the screen of statistics for UI                 |
 +-------------------------------------------+--------------------------------------------------------------------------------------+
+
+Customization of additional static variables
+============================================
+
+This table lists the constants that are set within locust and may be overridden.
+
++-------------------------------------------+--------------------------------------------------------------------------------------+
+| Parameter name                            | Purpose                                                                              |
++-------------------------------------------+--------------------------------------------------------------------------------------+
+| locust.runners.WORKER_LOG_REPORT_INTERVAL | Interval for how frequently worker logs are reported to master. Can be disabled      |
+|                                           | by setting to a negative number                                                      |
++-------------------------------------------+--------------------------------------------------------------------------------------+

locust-2.28.1.dev66/docs/extensions.rst

@@ -0,0 +1,41 @@
+.. _extensions:
+
+======================
+Third party extensions
+======================
+
+Support for load testing other protocols, reporting etc
+-------------------------------------------------------
+
+-  `locust-plugins <https://github.com/SvenskaSpel/locust-plugins/>`__
+
+   -  request logging & graphing
+   -  new protocols like websockets, selenium/webdriver, http users that
+      load html page resources
+   -  readers (ways to get test data into your tests)
+   -  wait time (custom wait time functions)
+   -  debug (support for running a single user in the debugger)
+   -  checks (adds command line parameters to set locust exit code based
+      on requests/s, error percentage and average response times)
+
+Automate distributed runs over SSH
+----------------------------------
+
+-  `locust-swarm <https://github.com/SvenskaSpel/locust-swarm/>`__
+
+Automatically translate a browser recording (HAR-file) to a locustfile
+----------------------------------------------------------------------
+
+-  `har2locust <https://github.com/SvenskaSpel/har2locust>`__
+
+Workers written in other languages than Python
+----------------------------------------------
+
+A Locust master and a Locust worker communicate by exchanging
+`msgpack <http://msgpack.org/>`__ messages, which is supported by many
+languages. So, you can write your User tasks in any languages you like.
+For convenience, some libraries do the job as a worker runner. They run
+your User tasks, and report to master regularly.
+
+-  `Boomer <https://github.com/myzhan/boomer/>`__ - Go
+-  `Locust4j <https://github.com/myzhan/locust4j>`__ - Java

locust-2.28.1.dev66/docs/faq.rst

@@ -0,0 +1,107 @@
+.. _faq:
+
+===
+FAQ
+===
+
+How do I…
+
+..
+
+   Note: Hatch rate/ramp up does not change peak load, it only changes
+   how fast you get there (people keep asking about this for some
+   reason).
+
+Resolve errors that occur during load (error 5xx, Connection aborted, Connection reset by peer, etc)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   Check your server logs. If it works at low load then it is almost
+   certainly not a Locust issue, but an issue with the system you are
+   testing.
+
+Clear cookies, to make the next task iteration seem like a new user to my system under test
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   Call ``self.client.cookies.clear()`` at the end of your task.
+
+Control headers or other things about my HTTP requests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   Locust’s client in ``HttpUser`` inherits from
+   `requests <https://requests.readthedocs.io/en/master/>`__ and the
+   vast majority of parameters and methods for requests should just work
+   with Locust. Check out the docs and Stack Overflow answers for
+   requests first and then ask on Stack Overflow for additional Locust
+   specific help if necessary.
+
+Create a Locust file based on a recorded browser session
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   Try using `Transformer <https://transformer.readthedocs.io/>`__
+
+How to run a Docker container of Locust in Windows Subsystem for Linux (Windows 10 users)?
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   If you use WSL on a Windows computer, then you need one extra step
+   than the `“docker run …”
+   command <https://docs.locust.io/en/stable/running-locust-docker.html>`__:
+   copy your locusttest1.py to a folder in a Windows path on your WSL
+   and mount that folder instead of your normal WSL folder:
+
+::
+
+   $ mkdir /c/Users/[YOUR_Windows_USER]/Documents/Locust/
+   $ cp ~/path/to/locusttest1.py /c/Users/[YOUR_Windows_USER]/Documents/Locust/
+   $ docker run -p 8089:8089 -v /c/Users/[YOUR_Windows_USER]/Documents/Locust/:/mnt/locust locustio/locust:1.3.1 -f /mnt/locust/locusttest1.py
+
+How to run locust on custom endpoint
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   Prefix the the endpoint to all ``@app.route`` definitions in
+   ``locust/web.py`` file & also change as follows (where ``/locust`` is
+   new endpoint)
+
+``app = Flask(__name__, static_url_path='/locust')``
+
+   Change the entries of static content location in file
+   ``locust/templates/index.html``.
+
+Eg:
+``<link rel="shortcut icon" href="{{ url_for('static', filename='img/favicon.ico') }}" type="image/x-icon"/>``
+
+Locust web UI doesn’t show my tasks running, says 0 RPS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Locust only knows what you’re doing when you tell it. There are `Event
+Hooks <https://docs.locust.io/en/stable/api.html#events>`__ that you use
+to tell Locust what’s going on in your code. If you use Locust’s
+``HttpUser`` and then use ``self.client`` to make http calls, the proper
+events are normally fired for you automatically, making less work for
+you unless you want to override the default events.
+
+If you use the plain ``User`` or use ``HttpUser`` and you’re not using
+``self.client`` to make http calls, Locust will not fire events for you.
+You will have to fire events yourself. See `the Locust
+docs <https://docs.locust.io/en/stable/testing-other-systems.html>`__
+for examples.
+
+HTML Report is filled up with failed requests for long running tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+https://github.com/locustio/locust/issues/2328
+
+Other questions and issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`Check the list of issues (a lot of questions/misunderstandings are
+filed as
+issues) <https://github.com/locustio/locust/issues?q=is%3Aissue%20>`__
+
+Add things that you ran into and solved here! Anyone with a GitHub
+account can contribute!
+
+If you have questions about Locust that are not answered here, please
+check
+`StackOverflow <https://stackoverflow.com/questions/tagged/locust>`__,
+or post your question there. This wiki is not for asking questions but
+for answering them :)

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/further-reading.rst

@@ -2,13 +2,11 @@
 Further reading / knowledgebase
 ===============================
 
-The `Locust wiki <https://github.com/locustio/locust/wiki/>`_ serves as a community maintained knowledgebase for Locust, augmenting the official documentation.
+- :ref:`extensions`.
 
-- `Extensions <https://github.com/locustio/locust/wiki/Extensions>`_ Third party extensions and tools
+- An unmaintained list of Locust `articles, blog posts, reviews, and walkthroughs <https://github.com/orgs/locustio/discussions/2752>`_.
 
-- `Articles <https://github.com/locustio/locust/wiki/Articles>`_ Blog posts with reviews & walkthroughs of how to run Locust in various use cases
-
-- `Frequently Asked Questions <https://github.com/locustio/locust/wiki/FAQ>`_ Common questions about Locust answered here
+- :ref:`Common questions and answers about Locust <faq>`.
 
 You'll also find a lot of answers on `stackoverflow <https://stackoverflow.com/questions/tagged/locust?sort=MostVotes>`_, for example:
 

locust-2.28.1.dev66/docs/increasing-request-rate.rst

@@ -0,0 +1,28 @@
+.. _increaserr:
+
+===========================
+Increasing the request rate
+===========================
+
+Increase the number of requests per second using a combination of the following steps:
+
+#. Increase the number of users. To fully utilize your target system you may need a lot of simultaneous users, especially if each request takes a long time to complete.
+
+#. If response times are unexpectedly high and/or increasing as the number of users go up, then you have probably saturated the system you are testing and need to dig into why. This is not really a Locust problem, but here are some things you may want to check:
+
+    -  resource utilization (e.g. CPU, memory & network. Check these metrics on the locust side as well)
+    -  configuration (e.g. max threads for your web server)
+    -  back end response times (e.g. DB)
+    -  client side DNS performance/flood protection (Locust will normally make at least one DNS Request per User)
+
+#. If Locust prints a warning about high CPU usage (``WARNING/root: CPU usage above 90%! ...``) try the following:
+
+    -  Run Locust `distributed <https://docs.locust.io/en/stable/running-locust-distributed.html>`__ to utilize multiple cores & multiple machines
+    -  Try switching to `FastHttpUser <https://docs.locust.io/en/stable/increase-performance.html#increase-performance>`__ to reduce CPU usage
+    -  Check to see that there are no strange/infinite loops in your code
+
+#. If you are using a custom client (not HttpUser or FastHttpUser), make sure any client library you are using is gevent-friendly otherwise it will block the entire Python process (essentially limiting you to one user per worker)
+
+.. note::
+
+    Hatch rate/ramp up does not change peak load, it only changes how fast you get there.

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/index.rst

@@ -30,6 +30,7 @@ Running your Locust tests
     :maxdepth: 1
 
     configuration
+    increasing-request-rate
     running-distributed
     running-in-debugger
     running-in-docker
@@ -49,6 +50,7 @@ Other functionalities
     extending-locust
     logging
     use-as-lib
+    extensions
 
 
 Further reading / knowledgebase
@@ -59,6 +61,7 @@ Further reading / knowledgebase
 
     developing-locust
     further-reading
+    faq
 
 
 API

locust-2.28.1.dev66/docs/installation.rst

@@ -0,0 +1,103 @@
+.. _installation:
+
+Installation
+============
+
+0. `Install Python <https://docs.python-guide.org/starting/installation/>`_ (3.9 or later)
+
+1. Install the package (check `Troubleshooting Installation`_ if the installation fails)
+
+.. code-block:: console
+
+    $ pip3 install locust
+
+2. Validate your installation
+
+.. code-block:: console
+    :substitutions:
+
+    $ locust -V
+    locust |version| from /usr/local/lib/python3.10/site-packages/locust (python 3.10.6)
+
+3. Done! Now you can :ref:`create your first test <quickstart>`
+
+
+Pre-release builds
+------------------
+
+If you need the latest and greatest version of Locust and cannot wait for the next release, you can install a dev build like this:
+
+.. code-block:: console
+
+    $ pip3 install -U --pre locust
+
+Pre-release builds are published every time a branch/PR is merged into master.
+
+Install for development
+-----------------------
+
+If you want to modify Locust, or contribute to the project, see :ref:`developing-locust`.
+
+Troubleshooting installation
+----------------------------
+
+
+.. contents:: Some solutions for common installation issues
+    :depth: 1
+    :local:
+    :backlinks: none
+
+
+psutil/\_psutil_common.c:9:10: fatal error: Python.h: No such file or directory
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`Answered in Stackoverflow thread 63440765 <https://stackoverflow.com/questions/63440765/locust-installation-error-using-pip3-error-command-errored-out-with-exit-statu>`_
+
+ERROR: Failed building wheel for xxx
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While Locust itself is a pure Python package, it has some dependencies
+(e.g. gevent and geventhttpclient) that are compiled from C code. Pretty
+much all common platforms have binary packages on PyPi, but sometimes
+there is a new release that doesn't, or you are running on some exotic
+platform. You have two options:
+
+-  (on macos) Install xcode: ``xcode-select --install``
+-  Use ``pip install --prefer-binary locust`` to select a pre-compiled
+   version of packages even if there is a more recent version available
+   as source.
+-  Try googling the error message for the specific package that failed
+   (not Locust), ensure you have the appropriate build tools installed
+   etc.
+
+Windows
+~~~~~~~
+
+`Answered in Stackoverflow thread 61592069 <https://stackoverflow.com/questions/61592069/locust-is-not-installing-on-my-windows-10-for-load-testing>`_
+
+Installation works, but the ``locust`` command is not found
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When running pip, did you get a warning saying ``The script locust is installed in '...' which is not on PATH``?
+
+Add that directory to your PATH environment variable.
+
+Increasing Maximum Number of Open Files Limit
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Every User/HTTP connection from Locust opens a new file (technically
+a file descriptor). Many operating systems by default set a low limit
+for the maximum number of files that can be open at the same time.
+Locust will try to adjust this automatically for you, but in a lot of
+cases your operating system will not allow it (in which case you will
+get a warning in the log). Instead you will have to do it manually.
+
+How to do this depends on your operating system, but you might find
+some useful information here:
+https://www.tecmint.com/increase-set-open-file-limits-in-linux/ and
+practical examples
+https://www.ibm.com/support/knowledgecenter/SS8NLW_11.0.2/com.ibm.discovery.es.in.doc/iiysiulimits.html
+
+For systemd-based systems (e.g. Debian/Ubuntu) different limits are
+used for graphical login sessions. See
+https://unix.stackexchange.com/a/443467 for additional settings.

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/quickstart.rst

@@ -56,7 +56,7 @@ The following screenshots show what it might look like when running this test us
 
     If your response times are *not* increasing then add even more users until you find the service's breaking point, or celebrate that your service is already performant enough for your expected load.
 
-    If you need some help digging into server side problems, or you're having trouble generating enough load to saturate your system, take a look at the `Locust FAQ  <https://github.com/locustio/locust/wiki/FAQ#increase-my-request-raterps>`_.
+    If you're having trouble generating enough load to saturate your system, take a look at :ref:`increaserr`.
 
 Direct command line usage / headless
 ====================================
@@ -89,7 +89,7 @@ To run Locust distributed across multiple Python processes or machines, you star
 with the ``--master`` command line parameter, and then any number of Locust worker processes using the ``--worker``
 command line parameter. See :ref:`running-distributed` for more info.
 
-To see all available options type: ```locust --help`` or check :ref:`configuration`.
+To see all available options type: ``locust --help`` or check :ref:`configuration`.
 
 Next steps
 ==========

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/running-distributed.rst

@@ -18,7 +18,7 @@ To simplify startup, you can use the ``--processes`` flag. It will launch a mast
 .. note::
     There is almost no limit to how many Users you can run per worker. Locust/gevent can run thousands or even tens of thousands of Users per process just fine, as long as their total request rate (RPS) is not too high.
 
-    If Locust *is* getting close to running out of CPU resources, it will log a warning. If there is no warning but you are still unable to generate the expected load, then the problem must be `something else <https://github.com/locustio/locust/wiki/FAQ#increase-my-request-raterps>`_.
+    If Locust *is* getting close to running out of CPU resources, it will log a warning. If there is no warning but you are still unable to generate the expected load, then the problem must be :ref:`increaserr`.
 
 Single machine
 ==============

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/docs/running-in-docker.rst

@@ -38,4 +38,4 @@ official Locust docker image as a base image::
 Running Locust using Kubernetes
 ===============================
 
-See [Extensions](https://github.com/locustio/locust/wiki/Extensions#helm) in the wiki.
+See :ref:`Helm charts for Locust <helm>`

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/locust/_version.py

@@ -12,5 +12,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '2.28.1.dev54'
-__version_tuple__ = version_tuple = (2, 28, 1, 'dev54')
+__version__ = version = '2.28.1.dev66'
+__version_tuple__ = version_tuple = (2, 28, 1, 'dev66')

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/locust/env.py

@@ -99,6 +99,8 @@ class Environment:
         """List of the available Tasks per User Classes to pick from in the Task Picker"""
         self.dispatcher_class = dispatcher_class
         """A user dispatcher class that decides how users are spawned, default :class:`UsersDispatcher <locust.dispatch.UsersDispatcher>`"""
+        self.worker_logs: dict[str, list[str]] = {}
+        """Captured logs from all connected workers"""
 
         self._remove_user_classes_with_weight_zero()
         self._validate_user_class_name_uniqueness()
@@ -209,6 +211,10 @@ class Environment:
             if key == "tasks":
                 user_class.tasks = [task for task in user_tasks if task.__name__ in value]
 
+    def update_worker_logs(self, worker_log_report):
+        if worker_log_report.get("worker_id", None):
+            self.worker_logs[worker_log_report.get("worker_id")] = worker_log_report.get("logs", [])
+
     def _filter_tasks_by_tags(self) -> None:
         """
         Filter the tasks on all the user_classes recursively, according to the tags and

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/locust/log.py

@@ -2,6 +2,7 @@ import logging
 import logging.config
 import re
 import socket
+from collections import deque
 
 HOSTNAME = re.sub(r"\..*", "", socket.gethostname())
 
@@ -13,7 +14,7 @@ unhandled_greenlet_exception = False
 class LogReader(logging.Handler):
     def __init__(self):
         super().__init__()
-        self.logs = []
+        self.logs = deque(maxlen=500)
 
     def emit(self, record):
         self.logs.append(self.format(record))
@@ -75,6 +76,15 @@ def setup_logging(loglevel, logfile=None):
     logging.config.dictConfig(LOGGING_CONFIG)
 
 
+def get_logs():
+    log_reader_handler = [handler for handler in logging.getLogger("root").handlers if handler.name == "log_reader"]
+
+    if log_reader_handler:
+        return list(log_reader_handler[0].logs)
+
+    return []
+
+
 def greenlet_exception_logger(logger, level=logging.CRITICAL):
     """
     Return a function that can be used as argument to Greenlet.link_exception() that will log the

{locust-2.28.1.dev54 → locust-2.28.1.dev66}/locust/runners.py

@@ -39,7 +39,7 @@ from gevent.pool import Group
 from . import argument_parser
 from .dispatch import UsersDispatcher
 from .exception import RPCError, RPCReceiveError, RPCSendError
-from .log import greenlet_exception_logger
+from .log import get_logs, greenlet_exception_logger
 from .rpc import (
     Message,
     rpc,
@@ -66,6 +66,7 @@ STATE_INIT, STATE_SPAWNING, STATE_RUNNING, STATE_CLEANUP, STATE_STOPPING, STATE_
     "missing",
 ]
 WORKER_REPORT_INTERVAL = 3.0
+WORKER_LOG_REPORT_INTERVAL = 10
 CPU_MONITOR_INTERVAL = 5.0
 CPU_WARNING_THRESHOLD = 90
 HEARTBEAT_INTERVAL = 1
@@ -1116,6 +1117,8 @@ class MasterRunner(DistributedRunner):
                 # a worker finished spawning (this happens multiple times during rampup)
                 self.clients[msg.node_id].state = STATE_RUNNING
                 self.clients[msg.node_id].user_classes_count = msg.data["user_classes_count"]
+            elif msg.type == "logs":
+                self.environment.update_worker_logs(msg.data)
             elif msg.type == "quit":
                 if msg.node_id in self.clients:
                     client = self.clients[msg.node_id]
@@ -1212,6 +1215,7 @@ class WorkerRunner(DistributedRunner):
         self.client_id = socket.gethostname() + "_" + uuid4().hex
         self.master_host = master_host
         self.master_port = master_port
+        self.logs: list[str] = []
         self.worker_cpu_warning_emitted = False
         self._users_dispatcher: UsersDispatcher | None = None
         self.client = rpc.Client(master_host, master_port, self.client_id)
@@ -1220,6 +1224,7 @@ class WorkerRunner(DistributedRunner):
         self.greenlet.spawn(self.heartbeat).link_exception(greenlet_exception_handler)
         self.greenlet.spawn(self.heartbeat_timeout_checker).link_exception(greenlet_exception_handler)
         self.greenlet.spawn(self.stats_reporter).link_exception(greenlet_exception_handler)
+        self.greenlet.spawn(self.logs_reporter).link_exception(greenlet_exception_handler)
 
         # register listener that adds the current number of spawned users to the report that is sent to the master node
         def on_report_to_master(client_id: str, data: dict[str, Any]):
@@ -1417,6 +1422,25 @@ class WorkerRunner(DistributedRunner):
                 logger.error(f"Temporary connection lost to master server: {e}, will retry later.")
             gevent.sleep(WORKER_REPORT_INTERVAL)
 
+    def logs_reporter(self) -> None:
+        if WORKER_LOG_REPORT_INTERVAL < 0:
+            return
+
+        while True:
+            current_logs = get_logs()
+
+            if (len(current_logs) - len(self.logs)) > 10:
+                logger.warning(
+                    "The worker attempted to send more than 10 log lines in one interval. Further log sending was disabled for this worker."
+                )
+                self._send_logs(get_logs())
+                break
+            if len(current_logs) > len(self.logs):
+                self._send_logs(current_logs)
+
+            self.logs = current_logs
+            gevent.sleep(WORKER_LOG_REPORT_INTERVAL)
+
     def send_message(self, msg_type: str, data: dict[str, Any] | None = None, client_id: str | None = None) -> None:
         """
         Sends a message to master node
@@ -1433,6 +1457,9 @@ class WorkerRunner(DistributedRunner):
         self.environment.events.report_to_master.fire(client_id=self.client_id, data=data)
         self.client.send(Message("stats", data, self.client_id))
 
+    def _send_logs(self, current_logs) -> None:
+        self.send_message("logs", {"worker_id": self.client_id, "logs": current_logs})
+
     def connect_to_master(self):
         self.retry += 1
         self.client.send(Message("client_ready", __version__, self.client_id))