profeyelar 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- profeyelar-1.0.0/LICENSE +21 -0
- profeyelar-1.0.0/PKG-INFO +73 -0
- profeyelar-1.0.0/README.md +60 -0
- profeyelar-1.0.0/pyproject.toml +19 -0
- profeyelar-1.0.0/setup.cfg +4 -0
- profeyelar-1.0.0/src/profeyelar/__init__.py +0 -0
- profeyelar-1.0.0/src/profeyelar/__main__.py +62 -0
- profeyelar-1.0.0/src/profeyelar/launcher.py +170 -0
- profeyelar-1.0.0/src/profeyelar/parsing.py +36 -0
- profeyelar-1.0.0/src/profeyelar/preferences.py +11 -0
- profeyelar-1.0.0/src/profeyelar/profiling.py +112 -0
- profeyelar-1.0.0/src/profeyelar/ui.py +469 -0
- profeyelar-1.0.0/src/profeyelar.egg-info/PKG-INFO +73 -0
- profeyelar-1.0.0/src/profeyelar.egg-info/SOURCES.txt +15 -0
- profeyelar-1.0.0/src/profeyelar.egg-info/dependency_links.txt +1 -0
- profeyelar-1.0.0/src/profeyelar.egg-info/requires.txt +3 -0
- profeyelar-1.0.0/src/profeyelar.egg-info/top_level.txt +1 -0
profeyelar-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 puzzleYOU
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: profeyelar
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: UI-based tool for project end-to-end profiling
|
|
5
|
+
Author: puzzleYOU GmbH
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Description-Content-Type: text/markdown
|
|
8
|
+
License-File: LICENSE
|
|
9
|
+
Requires-Dist: setuptools
|
|
10
|
+
Requires-Dist: requests
|
|
11
|
+
Requires-Dist: memray
|
|
12
|
+
Dynamic: license-file
|
|
13
|
+
|
|
14
|
+
# Profeyelar
|
|
15
|
+
**Profeyelar** (a.k.a. **Prof. Eyelar** a.k.a. **Professor Eyelar**) is an end-to-end profiling tool for Python projects/services that makes use of a graphical user interface to simplify the profiling process. It is primarily used to profile requests performed against a web server. The interface contains various settings, which The Professor will use to create a container for your service and begin profiling once the Start button is pressed. Prof. Eyelar will then perform requests for the specified URLs and will shut the container down again once finished. The Professor will handle activation and deactivation of the profiling for you once the program runs.
|
|
16
|
+
|
|
17
|
+
## User Interface
|
|
18
|
+
Prof. Eyelar's interface contains various fields, which affect the profiling or to-be-executed requests.
|
|
19
|
+
- In **Request URLs**, you can write any relative request URLs for The Professor to try. Multiple URLs can be separated by a new line. The URLs must be relative–Prof. Eyelar will build the absolute URLs using the project's base URL when the time is right.
|
|
20
|
+
- **Encode URL Parameters** is a small helper that you can utilize to encode URL parameters when necessary. You can then copy the encoded values and use them in the request URLs.
|
|
21
|
+
- **Repetitions** specifies how many times each request should be executed
|
|
22
|
+
- **Headers** can be used to specify any headers that should be passed along with the requests. The headers should be specified without quotes and multiple headers separated with new lines. A colon must be used between a header and its value. If you wish to pass an empty string as a value, then leave the space after the colon empty.
|
|
23
|
+
- Example
|
|
24
|
+
```
|
|
25
|
+
Accept-Fruit: apple
|
|
26
|
+
Accept-Language:
|
|
27
|
+
Accept-Professor: eyelar
|
|
28
|
+
```
|
|
29
|
+
- Profiling with **tracemalloc** and **cProfile** can be activated with the corresponding checkboxes. These profiling options will create `.tracemalloc` and `.cprofile` files, which you can interpret with corresponding tools.
|
|
30
|
+
The last profiling option is self-explanatory and can be used to limit RAM.
|
|
31
|
+
|
|
32
|
+
## Requirements
|
|
33
|
+
Professor Eyelar has high standards and can only work for your service if specific criteria are met.
|
|
34
|
+
- **Tkinter**: Profeyelar's interface is created with Tkinter, which will only work for you if a Python version that has been configured for Tkinter has been installed. In Ubuntu, for example, Tkinter is generally not included in the default Python installation, so you may need to install the `python3-tk` package. You can check if Tkinter is installed by writing `import tkinter` in a Python session. The import should succeed without error.
|
|
35
|
+
- **Configuration File**: A configuration file in JSON format must be given when calling Profeyelar from the terminal. This file contains information that The Professor will need in order to be able to create a container for your service and save the profiling results. Refer to [Structure of the Configuration File](#structure-of-the-configuration-file) for info on what this file should look like.
|
|
36
|
+
- **Server**: Since Prof. Eyelar is primarily used to profile requests to a web server, your project needs to have a server that The Professor can send requests to once the container has started.
|
|
37
|
+
- **Docker**: The Professor uses Docker commands to create containers for services and profile them, so the Docker Engine is required
|
|
38
|
+
- **Entrypoint**: Profeyelar requires you to implement a `serve-with-eyelar` entrypoint, in which you prepend the command used to start your server with `$*`. This will insert all the arguments that The Professor passes when calling that entrypoint. These arguments essentially wrap the server start in a call to memray, which will be used to profile while the server is active.
|
|
39
|
+
|
|
40
|
+
## Structure of the Configuration File
|
|
41
|
+
### Fields
|
|
42
|
+
- **base_url** (`string`, required): Absolute base URL of your service. The Professor will combine the relative request URLs with this base URL to build the absolute request URLs.
|
|
43
|
+
- **name** (`string`, required): The name of your service. The Professor needs it in order to run his commands on the service container he will create.
|
|
44
|
+
- **api_docs_url** (`string` or `null`, required): If your project has API-Docs that can be reached via URL, you can insert the absolute URL here. Prof. Eyelar will then include an `Open API Docs` button in its interface, which will open the corresponding page in your browser. This can be helpful if you need to quickly reference the Docs to formulate API-Calls in the relative request URLs correctly
|
|
45
|
+
- **output_directory** (`string`, required): Where Professor Eyelar should save the profiling results. Important to note is that this refers to a directory in the **profiled container's filesystem**. Therefore, if you want to access the results, you should ensure the output directory path in question is mapped to a specific path on the host system, which you can access after the container has stopped. Otherwise, you would lose the results after the profiling is complete because The Professor will then shut down the container.
|
|
46
|
+
- **retry_count** (`number`, optional, default 30): Specifies how many times The Professor will attempt to reach the built container before giving up and cancelling the profiling session. Changing the value for this setting can make sense for larger projects that may need more time to build and start a container for. Professor Eyelar checks the availability of the service container with a readiness probe, which, if unsuccessful, is repeated as many times as specified by `retry_count`. As soon as one of these probes succeeds, Prof. Eyelar will begin performing the user-specified requests.
|
|
47
|
+
|
|
48
|
+
### Usage of environment variables in strings
|
|
49
|
+
You can reference available environment variables in `string` values of the configuration file. To do so, you simply wrap the variable name in `$ENV{}`. An example of this can be seen in [Example Configuration File](#example-configuration-file).
|
|
50
|
+
|
|
51
|
+
### Example Configuration File
|
|
52
|
+
```json
|
|
53
|
+
{
|
|
54
|
+
"service": {
|
|
55
|
+
"base_url": "http://$ENV{PROJECT_NAME}.com/",
|
|
56
|
+
"name": "$ENV{PROJECT_NAME}",
|
|
57
|
+
"api_docs_url": null
|
|
58
|
+
},
|
|
59
|
+
"profiling": {
|
|
60
|
+
"output_directory": "i/am/a/directory/hehe/",
|
|
61
|
+
"retry_count": 60
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## Output Files
|
|
67
|
+
Profeyelar can generate up to 3 different types of profile result files, which can be analyzed with corresponding tools
|
|
68
|
+
- **Memray (`.bin`) Files**: Profiling results for memray will always be generated. These files end in `.bin` and have _memray_ in their name. You will need memray in order to generate reports from the data in these files. For more info about memray and the different generable reports, you can refer to their documentation [here](https://bloomberg.github.io/memray/getting_started.html).
|
|
69
|
+
- **Tracemalloc (`.tracemalloc`) Files**: Tracemalloc snapshots will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.tracemalloc`, and results can be viewed with corresponding functions of the [tracemalloc](https://docs.python.org/3/library/tracemalloc.html) module
|
|
70
|
+
- **cProfile (`.cprofile`) Files**: cProfile Stats will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.cprofile`, and results can be interpreted with corresponding tools like [pstats](https://docs.python.org/3.15/library/pstats.html) or [snakeviz](https://pypi.org/project/snakeviz/)
|
|
71
|
+
|
|
72
|
+
## Note regarding Gevent
|
|
73
|
+
As of time of writing, running memray on gunicorn based servers with `GeventWorker` workers results in an error when the server is stopped. We discovered this by chance when we checked the container logs before Profeyelar had finished the container shutdown process. The error does not seem to prevent The Professor from saving the profiling results and shutting the container down successfully. However, it is good to be aware of this behavior nonetheless. We did not encounter a similar error when running a gunicorn based server with Gthread (`ThreadWorker` workers) instead.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Profeyelar
|
|
2
|
+
**Profeyelar** (a.k.a. **Prof. Eyelar** a.k.a. **Professor Eyelar**) is an end-to-end profiling tool for Python projects/services that makes use of a graphical user interface to simplify the profiling process. It is primarily used to profile requests performed against a web server. The interface contains various settings, which The Professor will use to create a container for your service and begin profiling once the Start button is pressed. Prof. Eyelar will then perform requests for the specified URLs and will shut the container down again once finished. The Professor will handle activation and deactivation of the profiling for you once the program runs.
|
|
3
|
+
|
|
4
|
+
## User Interface
|
|
5
|
+
Prof. Eyelar's interface contains various fields, which affect the profiling or to-be-executed requests.
|
|
6
|
+
- In **Request URLs**, you can write any relative request URLs for The Professor to try. Multiple URLs can be separated by a new line. The URLs must be relative–Prof. Eyelar will build the absolute URLs using the project's base URL when the time is right.
|
|
7
|
+
- **Encode URL Parameters** is a small helper that you can utilize to encode URL parameters when necessary. You can then copy the encoded values and use them in the request URLs.
|
|
8
|
+
- **Repetitions** specifies how many times each request should be executed
|
|
9
|
+
- **Headers** can be used to specify any headers that should be passed along with the requests. The headers should be specified without quotes and multiple headers separated with new lines. A colon must be used between a header and its value. If you wish to pass an empty string as a value, then leave the space after the colon empty.
|
|
10
|
+
- Example
|
|
11
|
+
```
|
|
12
|
+
Accept-Fruit: apple
|
|
13
|
+
Accept-Language:
|
|
14
|
+
Accept-Professor: eyelar
|
|
15
|
+
```
|
|
16
|
+
- Profiling with **tracemalloc** and **cProfile** can be activated with the corresponding checkboxes. These profiling options will create `.tracemalloc` and `.cprofile` files, which you can interpret with corresponding tools.
|
|
17
|
+
The last profiling option is self-explanatory and can be used to limit RAM.
|
|
18
|
+
|
|
19
|
+
## Requirements
|
|
20
|
+
Professor Eyelar has high standards and can only work for your service if specific criteria are met.
|
|
21
|
+
- **Tkinter**: Profeyelar's interface is created with Tkinter, which will only work for you if a Python version that has been configured for Tkinter has been installed. In Ubuntu, for example, Tkinter is generally not included in the default Python installation, so you may need to install the `python3-tk` package. You can check if Tkinter is installed by writing `import tkinter` in a Python session. The import should succeed without error.
|
|
22
|
+
- **Configuration File**: A configuration file in JSON format must be given when calling Profeyelar from the terminal. This file contains information that The Professor will need in order to be able to create a container for your service and save the profiling results. Refer to [Structure of the Configuration File](#structure-of-the-configuration-file) for info on what this file should look like.
|
|
23
|
+
- **Server**: Since Prof. Eyelar is primarily used to profile requests to a web server, your project needs to have a server that The Professor can send requests to once the container has started.
|
|
24
|
+
- **Docker**: The Professor uses Docker commands to create containers for services and profile them, so the Docker Engine is required
|
|
25
|
+
- **Entrypoint**: Profeyelar requires you to implement a `serve-with-eyelar` entrypoint, in which you prepend the command used to start your server with `$*`. This will insert all the arguments that The Professor passes when calling that entrypoint. These arguments essentially wrap the server start in a call to memray, which will be used to profile while the server is active.
|
|
26
|
+
|
|
27
|
+
## Structure of the Configuration File
|
|
28
|
+
### Fields
|
|
29
|
+
- **base_url** (`string`, required): Absolute base URL of your service. The Professor will combine the relative request URLs with this base URL to build the absolute request URLs.
|
|
30
|
+
- **name** (`string`, required): The name of your service. The Professor needs it in order to run his commands on the service container he will create.
|
|
31
|
+
- **api_docs_url** (`string` or `null`, required): If your project has API-Docs that can be reached via URL, you can insert the absolute URL here. Prof. Eyelar will then include an `Open API Docs` button in its interface, which will open the corresponding page in your browser. This can be helpful if you need to quickly reference the Docs to formulate API-Calls in the relative request URLs correctly
|
|
32
|
+
- **output_directory** (`string`, required): Where Professor Eyelar should save the profiling results. Important to note is that this refers to a directory in the **profiled container's filesystem**. Therefore, if you want to access the results, you should ensure the output directory path in question is mapped to a specific path on the host system, which you can access after the container has stopped. Otherwise, you would lose the results after the profiling is complete because The Professor will then shut down the container.
|
|
33
|
+
- **retry_count** (`number`, optional, default 30): Specifies how many times The Professor will attempt to reach the built container before giving up and cancelling the profiling session. Changing the value for this setting can make sense for larger projects that may need more time to build and start a container for. Professor Eyelar checks the availability of the service container with a readiness probe, which, if unsuccessful, is repeated as many times as specified by `retry_count`. As soon as one of these probes succeeds, Prof. Eyelar will begin performing the user-specified requests.
|
|
34
|
+
|
|
35
|
+
### Usage of environment variables in strings
|
|
36
|
+
You can reference available environment variables in `string` values of the configuration file. To do so, you simply wrap the variable name in `$ENV{}`. An example of this can be seen in [Example Configuration File](#example-configuration-file).
|
|
37
|
+
|
|
38
|
+
### Example Configuration File
|
|
39
|
+
```json
|
|
40
|
+
{
|
|
41
|
+
"service": {
|
|
42
|
+
"base_url": "http://$ENV{PROJECT_NAME}.com/",
|
|
43
|
+
"name": "$ENV{PROJECT_NAME}",
|
|
44
|
+
"api_docs_url": null
|
|
45
|
+
},
|
|
46
|
+
"profiling": {
|
|
47
|
+
"output_directory": "i/am/a/directory/hehe/",
|
|
48
|
+
"retry_count": 60
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Output Files
|
|
54
|
+
Profeyelar can generate up to 3 different types of profile result files, which can be analyzed with corresponding tools
|
|
55
|
+
- **Memray (`.bin`) Files**: Profiling results for memray will always be generated. These files end in `.bin` and have _memray_ in their name. You will need memray in order to generate reports from the data in these files. For more info about memray and the different generable reports, you can refer to their documentation [here](https://bloomberg.github.io/memray/getting_started.html).
|
|
56
|
+
- **Tracemalloc (`.tracemalloc`) Files**: Tracemalloc snapshots will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.tracemalloc`, and results can be viewed with corresponding functions of the [tracemalloc](https://docs.python.org/3/library/tracemalloc.html) module
|
|
57
|
+
- **cProfile (`.cprofile`) Files**: cProfile Stats will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.cprofile`, and results can be interpreted with corresponding tools like [pstats](https://docs.python.org/3.15/library/pstats.html) or [snakeviz](https://pypi.org/project/snakeviz/)
|
|
58
|
+
|
|
59
|
+
## Note regarding Gevent
|
|
60
|
+
As of time of writing, running memray on gunicorn based servers with `GeventWorker` workers results in an error when the server is stopped. We discovered this by chance when we checked the container logs before Profeyelar had finished the container shutdown process. The error does not seem to prevent The Professor from saving the profiling results and shutting the container down successfully. However, it is good to be aware of this behavior nonetheless. We did not encounter a similar error when running a gunicorn based server with Gthread (`ThreadWorker` workers) instead.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "profeyelar"
|
|
7
|
+
version = "1.0.0"
|
|
8
|
+
description = "UI-based tool for project end-to-end profiling"
|
|
9
|
+
dependencies = [
|
|
10
|
+
"setuptools",
|
|
11
|
+
"requests",
|
|
12
|
+
"memray"
|
|
13
|
+
]
|
|
14
|
+
readme = "README.md"
|
|
15
|
+
authors = [
|
|
16
|
+
{name = "puzzleYOU GmbH"}
|
|
17
|
+
]
|
|
18
|
+
license = "MIT"
|
|
19
|
+
classifiers = []
|
|
File without changes
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Profeyelar (a.k.a. Prof. Eyelar a.k.a Professor Eyelar) is an end-to-end
|
|
3
|
+
profiling tool for projects.
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
import argparse
|
|
7
|
+
import json
|
|
8
|
+
from os.path import isfile
|
|
9
|
+
from tkinter import Tk
|
|
10
|
+
|
|
11
|
+
from parsing import (ConfigurationParseError, EnvironmentVariableNotFoundError,
|
|
12
|
+
parse_setting)
|
|
13
|
+
from ui import MainFrame
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def parse_args():
|
|
17
|
+
parser = argparse.ArgumentParser()
|
|
18
|
+
group = parser.add_argument_group("General")
|
|
19
|
+
|
|
20
|
+
group.add_argument(
|
|
21
|
+
"configuration_file",
|
|
22
|
+
metavar="configuration-file",
|
|
23
|
+
type=str,
|
|
24
|
+
help="""
|
|
25
|
+
The configuration file with the information that Profeyelar
|
|
26
|
+
requires in order to profile. This must be a valid JSON file
|
|
27
|
+
with the structure described in the README.md
|
|
28
|
+
""",
|
|
29
|
+
)
|
|
30
|
+
|
|
31
|
+
options = parser.parse_args()
|
|
32
|
+
return options
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def run_professor():
|
|
36
|
+
options = parse_args()
|
|
37
|
+
config_file_path = options.configuration_file
|
|
38
|
+
if not isfile(config_file_path):
|
|
39
|
+
raise ConfigurationParseError(config_file_path)
|
|
40
|
+
|
|
41
|
+
try:
|
|
42
|
+
with open(config_file_path, "r") as settings_file:
|
|
43
|
+
json_settings = json.load(settings_file)
|
|
44
|
+
prefixed_settings = {
|
|
45
|
+
f"{prefix}__{key}": parse_setting(value)
|
|
46
|
+
for prefix in json_settings.keys()
|
|
47
|
+
for key, value in json_settings[prefix].items()
|
|
48
|
+
}
|
|
49
|
+
except KeyError as ke:
|
|
50
|
+
raise EnvironmentVariableNotFoundError(config_file_path, str(ke))
|
|
51
|
+
except Exception:
|
|
52
|
+
raise ConfigurationParseError(config_file_path)
|
|
53
|
+
|
|
54
|
+
print("👓 Professor Eyelar is coming, please wait a few seconds... 👓")
|
|
55
|
+
|
|
56
|
+
root = Tk()
|
|
57
|
+
frame = MainFrame(root, prefixed_settings)
|
|
58
|
+
frame.enter_main_loop()
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
if __name__ == "__main__":
|
|
62
|
+
run_professor()
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
import json
|
|
2
|
+
import traceback
|
|
3
|
+
from contextlib import suppress
|
|
4
|
+
from dataclasses import dataclass
|
|
5
|
+
from functools import reduce
|
|
6
|
+
from http import HTTPStatus
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
from subprocess import PIPE, CalledProcessError, check_output, run
|
|
9
|
+
from time import sleep
|
|
10
|
+
from urllib.parse import urljoin
|
|
11
|
+
from uuid import uuid4
|
|
12
|
+
|
|
13
|
+
import requests
|
|
14
|
+
from preferences import ProfilingPreferences
|
|
15
|
+
from profiling import CProfileContext, TracemallocContext
|
|
16
|
+
from ui import MainFrame
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def is_service_running(name: str) -> bool:
|
|
20
|
+
cmdline = ["docker", "compose", "ps", name, "--format", "json"]
|
|
21
|
+
if (out := check_output(cmdline).decode()) == "":
|
|
22
|
+
return False
|
|
23
|
+
return json.loads(out)["State"] == "running"
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
@dataclass(frozen=True, slots=True)
|
|
27
|
+
class DockerContainer:
|
|
28
|
+
service_name: str
|
|
29
|
+
container_name: str
|
|
30
|
+
command: str
|
|
31
|
+
command_args: list[str]
|
|
32
|
+
env: dict[str, str]
|
|
33
|
+
|
|
34
|
+
def build(self):
|
|
35
|
+
cmdline = ["docker", "compose", "build", self.service_name]
|
|
36
|
+
run(cmdline, stdout=PIPE, stderr=PIPE, check=True)
|
|
37
|
+
|
|
38
|
+
def run(self, detached: bool = False):
|
|
39
|
+
envs = reduce(
|
|
40
|
+
lambda interim, pair: [*interim, "--env", f"{pair[0]}={pair[1]}"],
|
|
41
|
+
self.env.items(),
|
|
42
|
+
[],
|
|
43
|
+
)
|
|
44
|
+
maybe_detached = ["-d"] if detached else []
|
|
45
|
+
cmdline = [
|
|
46
|
+
"docker",
|
|
47
|
+
"compose",
|
|
48
|
+
"run",
|
|
49
|
+
*maybe_detached,
|
|
50
|
+
*envs,
|
|
51
|
+
"--name",
|
|
52
|
+
self.container_name,
|
|
53
|
+
self.service_name,
|
|
54
|
+
self.command,
|
|
55
|
+
*self.command_args,
|
|
56
|
+
]
|
|
57
|
+
|
|
58
|
+
try:
|
|
59
|
+
run(cmdline, stdout=PIPE, stderr=PIPE, check=True)
|
|
60
|
+
except CalledProcessError as exc:
|
|
61
|
+
stdout = exc.stdout.decode("utf-8")
|
|
62
|
+
stderr = exc.stderr.decode("utf-8")
|
|
63
|
+
text = f"""
|
|
64
|
+
Starting container failed.
|
|
65
|
+
cmdline: {cmdline}
|
|
66
|
+
stdout: {stdout}
|
|
67
|
+
stderr: {stderr}
|
|
68
|
+
"""
|
|
69
|
+
raise RuntimeError(text) from exc
|
|
70
|
+
|
|
71
|
+
def stop(self):
|
|
72
|
+
cmdline = ["docker", "container", "stop", self.container_name]
|
|
73
|
+
run(cmdline, stdout=PIPE, stderr=PIPE, check=True)
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
def start_profiling(
|
|
77
|
+
parsed_settings: dict, prefs: ProfilingPreferences, ui: MainFrame
|
|
78
|
+
):
|
|
79
|
+
ui.freeze()
|
|
80
|
+
service_name = parsed_settings["service__name"]
|
|
81
|
+
if is_service_running(service_name):
|
|
82
|
+
ui.notify_err(
|
|
83
|
+
f"Please stop {service_name} first. The Professor takes care"
|
|
84
|
+
" of starting it with the expected environment settings."
|
|
85
|
+
)
|
|
86
|
+
return
|
|
87
|
+
|
|
88
|
+
container = _initialize_container(parsed_settings, prefs)
|
|
89
|
+
try:
|
|
90
|
+
ui.set_status_text("Building container...")
|
|
91
|
+
container.build()
|
|
92
|
+
ui.set_status_text("Starting container...")
|
|
93
|
+
container.run(detached=True)
|
|
94
|
+
ui.set_status_text(f"Waiting until {service_name} wakes up...")
|
|
95
|
+
_wait_until_container_is_up(parsed_settings)
|
|
96
|
+
_fire_requests(parsed_settings, prefs, ui)
|
|
97
|
+
except Exception as e:
|
|
98
|
+
traceback.print_exception(e)
|
|
99
|
+
ui.notify_err(f"{e.__class__.__name__}: {e}")
|
|
100
|
+
finally:
|
|
101
|
+
ui.set_status_text("Stopping container...")
|
|
102
|
+
container.stop()
|
|
103
|
+
ui.unfreeze()
|
|
104
|
+
ui.set_status_text("Ready")
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
def _initialize_container(
|
|
108
|
+
parsed_settings: dict,
|
|
109
|
+
prefs: ProfilingPreferences,
|
|
110
|
+
) -> DockerContainer:
|
|
111
|
+
service_name = parsed_settings["service__name"]
|
|
112
|
+
output_path = str(
|
|
113
|
+
Path(parsed_settings["profiling__output_directory"])
|
|
114
|
+
/ "memray_output.bin"
|
|
115
|
+
)
|
|
116
|
+
return DockerContainer(
|
|
117
|
+
service_name=service_name,
|
|
118
|
+
container_name=f"{service_name}-{uuid4()}",
|
|
119
|
+
command="serve-with-eyelar",
|
|
120
|
+
command_args=[
|
|
121
|
+
"memray",
|
|
122
|
+
"run",
|
|
123
|
+
"-o",
|
|
124
|
+
output_path,
|
|
125
|
+
"-f",
|
|
126
|
+
"--follow-fork",
|
|
127
|
+
"--native",
|
|
128
|
+
"--trace-python-allocators",
|
|
129
|
+
],
|
|
130
|
+
env={
|
|
131
|
+
CProfileContext.associated_env: str(prefs.cprofile_enabled),
|
|
132
|
+
TracemallocContext.associated_env: str(prefs.tracemalloc_enabled),
|
|
133
|
+
},
|
|
134
|
+
)
|
|
135
|
+
|
|
136
|
+
|
|
137
|
+
def _wait_until_container_is_up(parsed_settings: dict):
|
|
138
|
+
max_tries = int(parsed_settings.get("profiling__retry_count", 0)) or 30
|
|
139
|
+
tries = 0
|
|
140
|
+
while tries <= max_tries:
|
|
141
|
+
with suppress(Exception):
|
|
142
|
+
resp = requests.get(
|
|
143
|
+
urljoin(
|
|
144
|
+
parsed_settings["service__base_url"], "/_/kube/readiness/"
|
|
145
|
+
)
|
|
146
|
+
)
|
|
147
|
+
if resp.status_code == HTTPStatus.OK:
|
|
148
|
+
return
|
|
149
|
+
sleep(1)
|
|
150
|
+
tries += 1
|
|
151
|
+
raise RuntimeError("retries exhausted")
|
|
152
|
+
|
|
153
|
+
|
|
154
|
+
def _fire_requests(
|
|
155
|
+
parsed_settings: dict, prefs: ProfilingPreferences, ui: MainFrame
|
|
156
|
+
):
|
|
157
|
+
maximum = prefs.repetitions * len(prefs.relative_urls)
|
|
158
|
+
request_headers = prefs.request_headers or None
|
|
159
|
+
|
|
160
|
+
i = 1
|
|
161
|
+
for _ in range(prefs.repetitions):
|
|
162
|
+
for url_path in prefs.relative_urls:
|
|
163
|
+
ui.set_status_text(f"Performing request {i}/{maximum}...")
|
|
164
|
+
resp = requests.get(
|
|
165
|
+
urljoin(parsed_settings["service__base_url"], url_path),
|
|
166
|
+
headers=request_headers,
|
|
167
|
+
)
|
|
168
|
+
if resp.status_code != HTTPStatus.OK:
|
|
169
|
+
raise RuntimeError(f"ERROR {resp.status_code}: {resp.content}")
|
|
170
|
+
i += 1
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
import os
|
|
2
|
+
import re
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
class ConfigurationParseError(Exception):
|
|
6
|
+
def __init__(self, settings_file: str):
|
|
7
|
+
super().__init__(
|
|
8
|
+
"[ERROR] Profeyelar could not extract the required settings"
|
|
9
|
+
f" from configuration file {settings_file}. Ensure that the"
|
|
10
|
+
" .json File exists, is properly formatted and contains all"
|
|
11
|
+
" required settings."
|
|
12
|
+
)
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
class EnvironmentVariableNotFoundError(Exception):
|
|
16
|
+
def __init__(self, settings_file: str, variable: str):
|
|
17
|
+
super().__init__(
|
|
18
|
+
"[ERROR] Profeyelar failed to parse the configuration"
|
|
19
|
+
f" file {settings_file} due to the undefined"
|
|
20
|
+
f" environment variable {variable}!"
|
|
21
|
+
)
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
CONFIGURATION_FILE_ENV_REGEX = re.compile(r"\$ENV\{[a-zA-Z0-9_\-]+\}")
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def parse_setting(value: object):
|
|
28
|
+
if type(value) is str:
|
|
29
|
+
env_matches = re.findall(CONFIGURATION_FILE_ENV_REGEX, value)
|
|
30
|
+
for match in env_matches:
|
|
31
|
+
value = value.replace(match, get_parsed_env_variable(match))
|
|
32
|
+
return value
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def get_parsed_env_variable(text: str) -> str:
|
|
36
|
+
return os.environ[text.removeprefix("$ENV{").removesuffix("}")]
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
from dataclasses import dataclass
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
@dataclass(frozen=True, slots=True)
|
|
5
|
+
class ProfilingPreferences:
|
|
6
|
+
relative_urls: list[str]
|
|
7
|
+
repetitions: int
|
|
8
|
+
tracemalloc_enabled: bool
|
|
9
|
+
cprofile_enabled: bool
|
|
10
|
+
ram_limit: int | None
|
|
11
|
+
request_headers: dict[str, str]
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
import logging
|
|
2
|
+
import os
|
|
3
|
+
import tracemalloc
|
|
4
|
+
import types
|
|
5
|
+
from abc import abstractmethod
|
|
6
|
+
from contextlib import AbstractContextManager, contextmanager
|
|
7
|
+
from cProfile import Profile
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
from uuid import uuid4
|
|
10
|
+
|
|
11
|
+
logger = logging.getLogger(__name__)
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
class ProfilerContext(AbstractContextManager):
|
|
15
|
+
associated_env: str
|
|
16
|
+
output_directory: Path
|
|
17
|
+
enabled: bool
|
|
18
|
+
|
|
19
|
+
@abstractmethod
|
|
20
|
+
def _start_tracing(self): ...
|
|
21
|
+
|
|
22
|
+
@abstractmethod
|
|
23
|
+
def _stop_tracing(self): ...
|
|
24
|
+
|
|
25
|
+
@abstractmethod
|
|
26
|
+
def _save_snapshots(self): ...
|
|
27
|
+
|
|
28
|
+
def __init__(
|
|
29
|
+
self, session_id: str, output_directory: Path, log: bool = False
|
|
30
|
+
) -> None:
|
|
31
|
+
super().__init__()
|
|
32
|
+
self.session_id = session_id
|
|
33
|
+
self.enabled = os.environ.get(self.associated_env) == "True"
|
|
34
|
+
self.output_directory = output_directory
|
|
35
|
+
if log:
|
|
36
|
+
logger.info(f"→ {self.associated_env}: {self.enabled}")
|
|
37
|
+
|
|
38
|
+
def __enter__(self) -> None:
|
|
39
|
+
if self.enabled:
|
|
40
|
+
self._start_tracing()
|
|
41
|
+
|
|
42
|
+
def __exit__(
|
|
43
|
+
self,
|
|
44
|
+
exc_type: type[BaseException] | None,
|
|
45
|
+
exc_value: BaseException | None,
|
|
46
|
+
traceback: types.TracebackType | None,
|
|
47
|
+
) -> bool | None:
|
|
48
|
+
if self.enabled:
|
|
49
|
+
self._stop_tracing()
|
|
50
|
+
self._save_snapshots()
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
class CProfileContext(ProfilerContext):
|
|
54
|
+
associated_env = "EYELAR_DEBUG_WITH_CPROFILE"
|
|
55
|
+
|
|
56
|
+
def __init__(
|
|
57
|
+
self, session_id: str, output_directory: Path, log: bool = False
|
|
58
|
+
):
|
|
59
|
+
super().__init__(
|
|
60
|
+
session_id=session_id, output_directory=output_directory, log=log
|
|
61
|
+
)
|
|
62
|
+
self.profile = Profile()
|
|
63
|
+
|
|
64
|
+
def _start_tracing(self):
|
|
65
|
+
self.profile.enable()
|
|
66
|
+
|
|
67
|
+
def _stop_tracing(self):
|
|
68
|
+
# Profile.dump_stats() also calls Profile.disable()
|
|
69
|
+
pass
|
|
70
|
+
|
|
71
|
+
def _save_snapshots(self):
|
|
72
|
+
self.profile.dump_stats(
|
|
73
|
+
str(self.output_directory / f"{self.session_id}.cprofile")
|
|
74
|
+
)
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
class TracemallocContext(ProfilerContext):
|
|
78
|
+
associated_env = "EYELAR_DEBUG_WITH_TRACEMALLOC"
|
|
79
|
+
|
|
80
|
+
def __init__(
|
|
81
|
+
self, session_id: str, output_directory: Path, log: bool = False
|
|
82
|
+
):
|
|
83
|
+
super().__init__(
|
|
84
|
+
session_id=session_id, output_directory=output_directory, log=log
|
|
85
|
+
)
|
|
86
|
+
|
|
87
|
+
def _start_tracing(self):
|
|
88
|
+
tracemalloc.start()
|
|
89
|
+
self.snapshot_start = tracemalloc.take_snapshot()
|
|
90
|
+
|
|
91
|
+
def _stop_tracing(self):
|
|
92
|
+
self.snapshot_end = tracemalloc.take_snapshot()
|
|
93
|
+
tracemalloc.stop()
|
|
94
|
+
|
|
95
|
+
def _save_snapshots(self):
|
|
96
|
+
self.snapshot_start.dump(
|
|
97
|
+
str(self.output_directory / f"{self.session_id}.0.tracemalloc")
|
|
98
|
+
)
|
|
99
|
+
self.snapshot_end.dump(
|
|
100
|
+
str(self.output_directory / f"{self.session_id}.1.tracemalloc")
|
|
101
|
+
)
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
@contextmanager
|
|
105
|
+
def profiling_session_if_enabled(output_directory: Path):
|
|
106
|
+
session_id = str(uuid4())
|
|
107
|
+
logger.info(f"profiling session ID: {session_id}")
|
|
108
|
+
with (
|
|
109
|
+
CProfileContext(session_id, output_directory, log=True),
|
|
110
|
+
TracemallocContext(session_id, output_directory, log=True),
|
|
111
|
+
):
|
|
112
|
+
yield
|
|
@@ -0,0 +1,469 @@
|
|
|
1
|
+
import tkinter
|
|
2
|
+
import tkinter.messagebox
|
|
3
|
+
import traceback
|
|
4
|
+
import urllib.parse
|
|
5
|
+
import webbrowser
|
|
6
|
+
from operator import methodcaller
|
|
7
|
+
from threading import Thread
|
|
8
|
+
from tkinter.scrolledtext import ScrolledText
|
|
9
|
+
|
|
10
|
+
from preferences import ProfilingPreferences
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
class InvalidTextInputError(Exception):
|
|
14
|
+
def __init__(self, given_value: str):
|
|
15
|
+
super().__init__(
|
|
16
|
+
"[ERROR] The following value is not a valid value"
|
|
17
|
+
f" for the corresponding input field: {given_value}"
|
|
18
|
+
)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
class MainFrame(tkinter.Frame):
|
|
22
|
+
HEADING_FONT = ("Helvetica", 14)
|
|
23
|
+
PADDING_SIZE = 10
|
|
24
|
+
PROGRAM_NAME = "Prof. Eyelar"
|
|
25
|
+
STANDARD_FONT = ("Helvetica", 10)
|
|
26
|
+
SPINBOX_WIDTH = 40
|
|
27
|
+
|
|
28
|
+
def __init__(self, root: tkinter.Tk, options):
|
|
29
|
+
super().__init__(
|
|
30
|
+
root, padx=self.PADDING_SIZE, pady=self.PADDING_SIZE / 2
|
|
31
|
+
)
|
|
32
|
+
self._root = root
|
|
33
|
+
root.title(self.PROGRAM_NAME)
|
|
34
|
+
self.row_count = 0
|
|
35
|
+
self.parsed_settings = options
|
|
36
|
+
self._add_widgets()
|
|
37
|
+
|
|
38
|
+
def get_selected_preferences(self) -> ProfilingPreferences:
|
|
39
|
+
ram_limit = int(self._ram_limit.get())
|
|
40
|
+
return ProfilingPreferences(
|
|
41
|
+
relative_urls=self._tell_provided_relative_urls(),
|
|
42
|
+
repetitions=int(self._repetitions.get()),
|
|
43
|
+
tracemalloc_enabled=self._tracemalloc_selected.get(),
|
|
44
|
+
cprofile_enabled=self._cprofile_selected.get(),
|
|
45
|
+
ram_limit=ram_limit or None,
|
|
46
|
+
request_headers=self._build_request_headers(),
|
|
47
|
+
)
|
|
48
|
+
|
|
49
|
+
def freeze(self):
|
|
50
|
+
self._start_button.config(state=tkinter.DISABLED)
|
|
51
|
+
|
|
52
|
+
def unfreeze(self):
|
|
53
|
+
self._start_button.config(state=tkinter.NORMAL)
|
|
54
|
+
|
|
55
|
+
def notify_err(self, msg: str):
|
|
56
|
+
tkinter.messagebox.showerror(
|
|
57
|
+
self.PROGRAM_NAME, f"Invalid input: {msg}"
|
|
58
|
+
)
|
|
59
|
+
|
|
60
|
+
def _build_request_headers(self) -> dict[str, str]:
|
|
61
|
+
headers_text = self._get_long_text_content(self._request_headers)
|
|
62
|
+
lines = self._get_stripped_text_lines(headers_text)
|
|
63
|
+
if len(lines) != headers_text.count(":"):
|
|
64
|
+
raise InvalidTextInputError(headers_text)
|
|
65
|
+
return {
|
|
66
|
+
key: value.lstrip()
|
|
67
|
+
for key, value in map(methodcaller("split", ":"), lines)
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
def _tell_provided_relative_urls(self) -> list[str]:
|
|
71
|
+
full_text = self._get_long_text_content(self._requests)
|
|
72
|
+
result = self._get_stripped_text_lines(full_text)
|
|
73
|
+
if not result:
|
|
74
|
+
raise ValueError("providing no URLs does not quite make sense.")
|
|
75
|
+
return result
|
|
76
|
+
|
|
77
|
+
def _get_long_text_content(self, widget: ScrolledText) -> str:
|
|
78
|
+
return widget.get(1.0, tkinter.END)
|
|
79
|
+
|
|
80
|
+
def _get_stripped_text_lines(self, text: str) -> str:
|
|
81
|
+
stripped = (line.strip() for line in text.splitlines())
|
|
82
|
+
return [line for line in stripped if line]
|
|
83
|
+
|
|
84
|
+
def _set_encoded_value(self):
|
|
85
|
+
current = self._unencoded.get()
|
|
86
|
+
encoded = urllib.parse.quote_plus(current)
|
|
87
|
+
self._encoded.set(encoded)
|
|
88
|
+
|
|
89
|
+
def _add_widgets(self):
|
|
90
|
+
self.grid()
|
|
91
|
+
self._set_up_request_url_section()
|
|
92
|
+
self._set_up_fields_filter_utility_section()
|
|
93
|
+
self._set_up_request_options_section()
|
|
94
|
+
self._set_up_profiling_options_section()
|
|
95
|
+
self._set_up_buttons()
|
|
96
|
+
self._set_up_status_bar()
|
|
97
|
+
self.set_status_text("Ready")
|
|
98
|
+
|
|
99
|
+
def _start_profiling_with_selected_preferences(self):
|
|
100
|
+
try:
|
|
101
|
+
prefs = self.get_selected_preferences()
|
|
102
|
+
from launcher import start_profiling
|
|
103
|
+
|
|
104
|
+
th = Thread(
|
|
105
|
+
target=start_profiling,
|
|
106
|
+
args=(self.parsed_settings, prefs, self),
|
|
107
|
+
)
|
|
108
|
+
th.start()
|
|
109
|
+
except ValueError as e:
|
|
110
|
+
tkinter.messagebox.showerror(
|
|
111
|
+
self.PROGRAM_NAME, f"Invalid input: {e}"
|
|
112
|
+
)
|
|
113
|
+
except Exception as e:
|
|
114
|
+
traceback.print_exception(e)
|
|
115
|
+
tkinter.messagebox.showerror(
|
|
116
|
+
self.PROGRAM_NAME, f"{e.__class__.__name__}: {e}"
|
|
117
|
+
)
|
|
118
|
+
|
|
119
|
+
def _set_up_buttons(self):
|
|
120
|
+
self.frame_buttons = tkinter.Frame(self)
|
|
121
|
+
self.frame_buttons.grid(
|
|
122
|
+
column=0,
|
|
123
|
+
row=self.row_count,
|
|
124
|
+
padx=self.PADDING_SIZE,
|
|
125
|
+
pady=self.PADDING_SIZE / 2,
|
|
126
|
+
sticky="ew",
|
|
127
|
+
)
|
|
128
|
+
self.frame_buttons.columnconfigure(0, weight=2)
|
|
129
|
+
self.frame_buttons.columnconfigure(1, weight=1)
|
|
130
|
+
self.frame_buttons.columnconfigure(2, weight=1)
|
|
131
|
+
|
|
132
|
+
self._start_button = tkinter.Button(
|
|
133
|
+
self.frame_buttons,
|
|
134
|
+
text="Start container & run requests",
|
|
135
|
+
command=self._start_profiling_with_selected_preferences,
|
|
136
|
+
)
|
|
137
|
+
self._start_button.grid(
|
|
138
|
+
column=0,
|
|
139
|
+
row=0,
|
|
140
|
+
sticky="w",
|
|
141
|
+
padx=self.PADDING_SIZE,
|
|
142
|
+
pady=self.PADDING_SIZE / 2,
|
|
143
|
+
)
|
|
144
|
+
|
|
145
|
+
docs_url = self.parsed_settings["service__api_docs_url"]
|
|
146
|
+
if docs_url:
|
|
147
|
+
tkinter.Button(
|
|
148
|
+
self.frame_buttons,
|
|
149
|
+
text="Open API docs",
|
|
150
|
+
command=lambda: webbrowser.open_new_tab(docs_url),
|
|
151
|
+
).grid(
|
|
152
|
+
column=1,
|
|
153
|
+
row=0,
|
|
154
|
+
padx=self.PADDING_SIZE,
|
|
155
|
+
pady=self.PADDING_SIZE / 2,
|
|
156
|
+
sticky="w",
|
|
157
|
+
)
|
|
158
|
+
|
|
159
|
+
tkinter.Button(
|
|
160
|
+
self.frame_buttons, text="Quit", command=self._root.destroy
|
|
161
|
+
).grid(
|
|
162
|
+
column=2,
|
|
163
|
+
row=0,
|
|
164
|
+
padx=self.PADDING_SIZE,
|
|
165
|
+
pady=self.PADDING_SIZE / 2,
|
|
166
|
+
sticky="e",
|
|
167
|
+
)
|
|
168
|
+
|
|
169
|
+
self._increment_row_count()
|
|
170
|
+
|
|
171
|
+
def _set_up_profiling_options_section(self):
|
|
172
|
+
self.frame_profiling_options = tkinter.Frame(self)
|
|
173
|
+
self.frame_profiling_options.grid(
|
|
174
|
+
column=0,
|
|
175
|
+
row=self.row_count,
|
|
176
|
+
padx=self.PADDING_SIZE,
|
|
177
|
+
pady=self.PADDING_SIZE / 2,
|
|
178
|
+
sticky="ew",
|
|
179
|
+
)
|
|
180
|
+
self.frame_profiling_options.columnconfigure(0, weight=1)
|
|
181
|
+
self.frame_profiling_options.columnconfigure(1, weight=2)
|
|
182
|
+
|
|
183
|
+
tkinter.Label(
|
|
184
|
+
self.frame_profiling_options,
|
|
185
|
+
text="Profiling options",
|
|
186
|
+
justify="left",
|
|
187
|
+
font=self.HEADING_FONT,
|
|
188
|
+
).grid(
|
|
189
|
+
column=0,
|
|
190
|
+
row=0,
|
|
191
|
+
padx=self.PADDING_SIZE,
|
|
192
|
+
pady=self.PADDING_SIZE / 2,
|
|
193
|
+
sticky="w",
|
|
194
|
+
)
|
|
195
|
+
|
|
196
|
+
tkinter.Label(
|
|
197
|
+
self.frame_profiling_options,
|
|
198
|
+
text="Enable tracemalloc",
|
|
199
|
+
font=self.STANDARD_FONT,
|
|
200
|
+
justify="left",
|
|
201
|
+
).grid(
|
|
202
|
+
column=0,
|
|
203
|
+
row=1,
|
|
204
|
+
padx=self.PADDING_SIZE,
|
|
205
|
+
pady=self.PADDING_SIZE / 2,
|
|
206
|
+
sticky="w",
|
|
207
|
+
)
|
|
208
|
+
self._tracemalloc_selected = tkinter.BooleanVar()
|
|
209
|
+
tkinter.Checkbutton(
|
|
210
|
+
self.frame_profiling_options, variable=self._tracemalloc_selected
|
|
211
|
+
).grid(
|
|
212
|
+
column=1,
|
|
213
|
+
row=1,
|
|
214
|
+
padx=self.PADDING_SIZE,
|
|
215
|
+
pady=self.PADDING_SIZE / 2,
|
|
216
|
+
sticky="e",
|
|
217
|
+
)
|
|
218
|
+
|
|
219
|
+
tkinter.Label(
|
|
220
|
+
self.frame_profiling_options,
|
|
221
|
+
text="Enable cProfile",
|
|
222
|
+
font=self.STANDARD_FONT,
|
|
223
|
+
justify="left",
|
|
224
|
+
).grid(
|
|
225
|
+
column=0,
|
|
226
|
+
row=2,
|
|
227
|
+
padx=self.PADDING_SIZE,
|
|
228
|
+
pady=self.PADDING_SIZE / 2,
|
|
229
|
+
sticky="w",
|
|
230
|
+
)
|
|
231
|
+
self._cprofile_selected = tkinter.BooleanVar()
|
|
232
|
+
tkinter.Checkbutton(
|
|
233
|
+
self.frame_profiling_options, variable=self._cprofile_selected
|
|
234
|
+
).grid(
|
|
235
|
+
column=1,
|
|
236
|
+
row=2,
|
|
237
|
+
padx=self.PADDING_SIZE,
|
|
238
|
+
pady=self.PADDING_SIZE / 2,
|
|
239
|
+
sticky="e",
|
|
240
|
+
)
|
|
241
|
+
|
|
242
|
+
tkinter.Label(
|
|
243
|
+
self.frame_profiling_options,
|
|
244
|
+
text="Limit RAM to given amount of MiB\n(0 = unlimited)",
|
|
245
|
+
font=self.STANDARD_FONT,
|
|
246
|
+
justify="left",
|
|
247
|
+
).grid(
|
|
248
|
+
column=0,
|
|
249
|
+
row=3,
|
|
250
|
+
padx=self.PADDING_SIZE,
|
|
251
|
+
pady=self.PADDING_SIZE / 2,
|
|
252
|
+
sticky="nw",
|
|
253
|
+
)
|
|
254
|
+
self._ram_limit = tkinter.Spinbox(
|
|
255
|
+
self.frame_profiling_options,
|
|
256
|
+
from_=0,
|
|
257
|
+
increment=16,
|
|
258
|
+
to=2048,
|
|
259
|
+
width=self.SPINBOX_WIDTH,
|
|
260
|
+
)
|
|
261
|
+
self._ram_limit.grid(
|
|
262
|
+
column=1,
|
|
263
|
+
row=3,
|
|
264
|
+
padx=self.PADDING_SIZE,
|
|
265
|
+
pady=self.PADDING_SIZE / 2,
|
|
266
|
+
sticky="sne",
|
|
267
|
+
)
|
|
268
|
+
self._increment_row_count()
|
|
269
|
+
|
|
270
|
+
def _set_up_request_options_section(self):
|
|
271
|
+
self.frame_request_options = tkinter.Frame(self)
|
|
272
|
+
self.frame_request_options.grid(
|
|
273
|
+
column=0,
|
|
274
|
+
row=self.row_count,
|
|
275
|
+
padx=self.PADDING_SIZE,
|
|
276
|
+
pady=self.PADDING_SIZE / 2,
|
|
277
|
+
sticky="ew",
|
|
278
|
+
)
|
|
279
|
+
self.frame_request_options.columnconfigure(0, weight=1)
|
|
280
|
+
self.frame_request_options.columnconfigure(1, weight=2)
|
|
281
|
+
|
|
282
|
+
label_options = tkinter.Label(
|
|
283
|
+
self.frame_request_options,
|
|
284
|
+
text="Request options",
|
|
285
|
+
font=self.HEADING_FONT,
|
|
286
|
+
)
|
|
287
|
+
label_options.grid(
|
|
288
|
+
column=0,
|
|
289
|
+
row=0,
|
|
290
|
+
padx=self.PADDING_SIZE,
|
|
291
|
+
pady=self.PADDING_SIZE / 2,
|
|
292
|
+
sticky="w",
|
|
293
|
+
)
|
|
294
|
+
tkinter.Label(
|
|
295
|
+
self.frame_request_options,
|
|
296
|
+
text="Repetitions",
|
|
297
|
+
font=self.STANDARD_FONT,
|
|
298
|
+
).grid(
|
|
299
|
+
column=0,
|
|
300
|
+
row=1,
|
|
301
|
+
padx=self.PADDING_SIZE,
|
|
302
|
+
pady=self.PADDING_SIZE / 2,
|
|
303
|
+
sticky="w",
|
|
304
|
+
)
|
|
305
|
+
self._repetitions = tkinter.Spinbox(
|
|
306
|
+
self.frame_request_options,
|
|
307
|
+
from_=1,
|
|
308
|
+
increment=1,
|
|
309
|
+
to=1000,
|
|
310
|
+
width=self.SPINBOX_WIDTH,
|
|
311
|
+
)
|
|
312
|
+
self._repetitions.grid(
|
|
313
|
+
column=1,
|
|
314
|
+
row=1,
|
|
315
|
+
padx=self.PADDING_SIZE,
|
|
316
|
+
pady=self.PADDING_SIZE / 2,
|
|
317
|
+
sticky="e",
|
|
318
|
+
)
|
|
319
|
+
|
|
320
|
+
tkinter.Label(
|
|
321
|
+
self.frame_request_options, text="Headers", font=self.STANDARD_FONT
|
|
322
|
+
).grid(
|
|
323
|
+
column=0,
|
|
324
|
+
row=2,
|
|
325
|
+
padx=self.PADDING_SIZE,
|
|
326
|
+
pady=self.PADDING_SIZE / 2,
|
|
327
|
+
sticky="nw",
|
|
328
|
+
)
|
|
329
|
+
self._request_headers = ScrolledText(
|
|
330
|
+
self.frame_request_options, height=3, width=46
|
|
331
|
+
)
|
|
332
|
+
self._request_headers.grid(
|
|
333
|
+
column=1,
|
|
334
|
+
row=2,
|
|
335
|
+
sticky="e",
|
|
336
|
+
padx=self.PADDING_SIZE,
|
|
337
|
+
pady=self.PADDING_SIZE / 2,
|
|
338
|
+
)
|
|
339
|
+
self._increment_row_count()
|
|
340
|
+
|
|
341
|
+
def _set_up_fields_filter_utility_section(self):
|
|
342
|
+
self.frame_utilities = tkinter.Frame(self)
|
|
343
|
+
self.frame_utilities.grid(
|
|
344
|
+
column=0,
|
|
345
|
+
row=self.row_count,
|
|
346
|
+
padx=self.PADDING_SIZE,
|
|
347
|
+
pady=self.PADDING_SIZE / 2,
|
|
348
|
+
sticky="ew",
|
|
349
|
+
)
|
|
350
|
+
self.frame_utilities.columnconfigure(0, weight=2)
|
|
351
|
+
self.frame_utilities.columnconfigure(1, weight=1)
|
|
352
|
+
self.frame_utilities.columnconfigure(2, weight=2)
|
|
353
|
+
|
|
354
|
+
tkinter.Label(
|
|
355
|
+
self.frame_utilities,
|
|
356
|
+
text="Helper: Encode URL Parameters",
|
|
357
|
+
font=self.HEADING_FONT,
|
|
358
|
+
).grid(
|
|
359
|
+
column=0,
|
|
360
|
+
columnspan=3,
|
|
361
|
+
row=0,
|
|
362
|
+
padx=self.PADDING_SIZE,
|
|
363
|
+
pady=self.PADDING_SIZE / 2,
|
|
364
|
+
)
|
|
365
|
+
self._unencoded = tkinter.StringVar()
|
|
366
|
+
tkinter.Entry(
|
|
367
|
+
self.frame_utilities, textvariable=self._unencoded, width=25
|
|
368
|
+
).grid(
|
|
369
|
+
column=0,
|
|
370
|
+
row=1,
|
|
371
|
+
padx=self.PADDING_SIZE,
|
|
372
|
+
pady=self.PADDING_SIZE / 2,
|
|
373
|
+
)
|
|
374
|
+
tkinter.Button(
|
|
375
|
+
self.frame_utilities,
|
|
376
|
+
text="→ Encode",
|
|
377
|
+
command=self._set_encoded_value,
|
|
378
|
+
).grid(
|
|
379
|
+
column=1,
|
|
380
|
+
row=1,
|
|
381
|
+
padx=self.PADDING_SIZE,
|
|
382
|
+
pady=self.PADDING_SIZE / 2,
|
|
383
|
+
)
|
|
384
|
+
self._encoded = tkinter.StringVar()
|
|
385
|
+
tkinter.Entry(
|
|
386
|
+
self.frame_utilities, textvariable=self._encoded, width=25
|
|
387
|
+
).grid(
|
|
388
|
+
column=2,
|
|
389
|
+
row=1,
|
|
390
|
+
padx=self.PADDING_SIZE,
|
|
391
|
+
pady=self.PADDING_SIZE / 2,
|
|
392
|
+
)
|
|
393
|
+
self._increment_row_count()
|
|
394
|
+
|
|
395
|
+
def _set_up_request_url_section(self):
|
|
396
|
+
self.frame_request_urls = tkinter.Frame(self)
|
|
397
|
+
self.frame_request_urls.grid(
|
|
398
|
+
column=0,
|
|
399
|
+
row=self.row_count,
|
|
400
|
+
padx=self.PADDING_SIZE,
|
|
401
|
+
pady=self.PADDING_SIZE / 2,
|
|
402
|
+
sticky="ew",
|
|
403
|
+
)
|
|
404
|
+
|
|
405
|
+
tkinter.Label(
|
|
406
|
+
self.frame_request_urls,
|
|
407
|
+
text="Request URLs",
|
|
408
|
+
font=self.HEADING_FONT,
|
|
409
|
+
).grid(
|
|
410
|
+
column=0,
|
|
411
|
+
row=0,
|
|
412
|
+
padx=self.PADDING_SIZE,
|
|
413
|
+
pady=self.PADDING_SIZE / 2,
|
|
414
|
+
sticky="w",
|
|
415
|
+
)
|
|
416
|
+
tkinter.Label(
|
|
417
|
+
self.frame_request_urls,
|
|
418
|
+
text=(
|
|
419
|
+
"Please enter relative request URLs here, "
|
|
420
|
+
"e. g. /stores/?fields=%5Bid%2Cname%5D"
|
|
421
|
+
),
|
|
422
|
+
font=self.STANDARD_FONT,
|
|
423
|
+
).grid(
|
|
424
|
+
column=0,
|
|
425
|
+
row=1,
|
|
426
|
+
padx=self.PADDING_SIZE,
|
|
427
|
+
pady=self.PADDING_SIZE / 2,
|
|
428
|
+
sticky="w",
|
|
429
|
+
)
|
|
430
|
+
self._requests = ScrolledText(self.frame_request_urls)
|
|
431
|
+
self._requests.grid(
|
|
432
|
+
column=0,
|
|
433
|
+
row=2,
|
|
434
|
+
padx=self.PADDING_SIZE,
|
|
435
|
+
pady=self.PADDING_SIZE / 2,
|
|
436
|
+
)
|
|
437
|
+
self._increment_row_count()
|
|
438
|
+
|
|
439
|
+
def _set_up_status_bar(self):
|
|
440
|
+
self.frame_status_bar = tkinter.Frame(self)
|
|
441
|
+
self.frame_status_bar.grid(
|
|
442
|
+
column=0,
|
|
443
|
+
row=self.row_count,
|
|
444
|
+
padx=self.PADDING_SIZE,
|
|
445
|
+
pady=self.PADDING_SIZE / 2,
|
|
446
|
+
sticky="ew",
|
|
447
|
+
)
|
|
448
|
+
|
|
449
|
+
self._status_bar = tkinter.Label(
|
|
450
|
+
self.frame_status_bar, text="", font=self.STANDARD_FONT
|
|
451
|
+
)
|
|
452
|
+
self._status_bar.grid(
|
|
453
|
+
column=0,
|
|
454
|
+
row=0,
|
|
455
|
+
sticky="w",
|
|
456
|
+
padx=self.PADDING_SIZE,
|
|
457
|
+
pady=self.PADDING_SIZE / 2,
|
|
458
|
+
)
|
|
459
|
+
|
|
460
|
+
self._increment_row_count()
|
|
461
|
+
|
|
462
|
+
def set_status_text(self, status: str):
|
|
463
|
+
self._status_bar.config(text=f"STATUS: {status}")
|
|
464
|
+
|
|
465
|
+
def enter_main_loop(self):
|
|
466
|
+
self._root.mainloop()
|
|
467
|
+
|
|
468
|
+
def _increment_row_count(self):
|
|
469
|
+
self.row_count += 1
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: profeyelar
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: UI-based tool for project end-to-end profiling
|
|
5
|
+
Author: puzzleYOU GmbH
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Description-Content-Type: text/markdown
|
|
8
|
+
License-File: LICENSE
|
|
9
|
+
Requires-Dist: setuptools
|
|
10
|
+
Requires-Dist: requests
|
|
11
|
+
Requires-Dist: memray
|
|
12
|
+
Dynamic: license-file
|
|
13
|
+
|
|
14
|
+
# Profeyelar
|
|
15
|
+
**Profeyelar** (a.k.a. **Prof. Eyelar** a.k.a. **Professor Eyelar**) is an end-to-end profiling tool for Python projects/services that makes use of a graphical user interface to simplify the profiling process. It is primarily used to profile requests performed against a web server. The interface contains various settings, which The Professor will use to create a container for your service and begin profiling once the Start button is pressed. Prof. Eyelar will then perform requests for the specified URLs and will shut the container down again once finished. The Professor will handle activation and deactivation of the profiling for you once the program runs.
|
|
16
|
+
|
|
17
|
+
## User Interface
|
|
18
|
+
Prof. Eyelar's interface contains various fields, which affect the profiling or to-be-executed requests.
|
|
19
|
+
- In **Request URLs**, you can write any relative request URLs for The Professor to try. Multiple URLs can be separated by a new line. The URLs must be relative–Prof. Eyelar will build the absolute URLs using the project's base URL when the time is right.
|
|
20
|
+
- **Encode URL Parameters** is a small helper that you can utilize to encode URL parameters when necessary. You can then copy the encoded values and use them in the request URLs.
|
|
21
|
+
- **Repetitions** specifies how many times each request should be executed
|
|
22
|
+
- **Headers** can be used to specify any headers that should be passed along with the requests. The headers should be specified without quotes and multiple headers separated with new lines. A colon must be used between a header and its value. If you wish to pass an empty string as a value, then leave the space after the colon empty.
|
|
23
|
+
- Example
|
|
24
|
+
```
|
|
25
|
+
Accept-Fruit: apple
|
|
26
|
+
Accept-Language:
|
|
27
|
+
Accept-Professor: eyelar
|
|
28
|
+
```
|
|
29
|
+
- Profiling with **tracemalloc** and **cProfile** can be activated with the corresponding checkboxes. These profiling options will create `.tracemalloc` and `.cprofile` files, which you can interpret with corresponding tools.
|
|
30
|
+
The last profiling option is self-explanatory and can be used to limit RAM.
|
|
31
|
+
|
|
32
|
+
## Requirements
|
|
33
|
+
Professor Eyelar has high standards and can only work for your service if specific criteria are met.
|
|
34
|
+
- **Tkinter**: Profeyelar's interface is created with Tkinter, which will only work for you if a Python version that has been configured for Tkinter has been installed. In Ubuntu, for example, Tkinter is generally not included in the default Python installation, so you may need to install the `python3-tk` package. You can check if Tkinter is installed by writing `import tkinter` in a Python session. The import should succeed without error.
|
|
35
|
+
- **Configuration File**: A configuration file in JSON format must be given when calling Profeyelar from the terminal. This file contains information that The Professor will need in order to be able to create a container for your service and save the profiling results. Refer to [Structure of the Configuration File](#structure-of-the-configuration-file) for info on what this file should look like.
|
|
36
|
+
- **Server**: Since Prof. Eyelar is primarily used to profile requests to a web server, your project needs to have a server that The Professor can send requests to once the container has started.
|
|
37
|
+
- **Docker**: The Professor uses Docker commands to create containers for services and profile them, so the Docker Engine is required
|
|
38
|
+
- **Entrypoint**: Profeyelar requires you to implement a `serve-with-eyelar` entrypoint, in which you prepend the command used to start your server with `$*`. This will insert all the arguments that The Professor passes when calling that entrypoint. These arguments essentially wrap the server start in a call to memray, which will be used to profile while the server is active.
|
|
39
|
+
|
|
40
|
+
## Structure of the Configuration File
|
|
41
|
+
### Fields
|
|
42
|
+
- **base_url** (`string`, required): Absolute base URL of your service. The Professor will combine the relative request URLs with this base URL to build the absolute request URLs.
|
|
43
|
+
- **name** (`string`, required): The name of your service. The Professor needs it in order to run his commands on the service container he will create.
|
|
44
|
+
- **api_docs_url** (`string` or `null`, required): If your project has API-Docs that can be reached via URL, you can insert the absolute URL here. Prof. Eyelar will then include an `Open API Docs` button in its interface, which will open the corresponding page in your browser. This can be helpful if you need to quickly reference the Docs to formulate API-Calls in the relative request URLs correctly
|
|
45
|
+
- **output_directory** (`string`, required): Where Professor Eyelar should save the profiling results. Important to note is that this refers to a directory in the **profiled container's filesystem**. Therefore, if you want to access the results, you should ensure the output directory path in question is mapped to a specific path on the host system, which you can access after the container has stopped. Otherwise, you would lose the results after the profiling is complete because The Professor will then shut down the container.
|
|
46
|
+
- **retry_count** (`number`, optional, default 30): Specifies how many times The Professor will attempt to reach the built container before giving up and cancelling the profiling session. Changing the value for this setting can make sense for larger projects that may need more time to build and start a container for. Professor Eyelar checks the availability of the service container with a readiness probe, which, if unsuccessful, is repeated as many times as specified by `retry_count`. As soon as one of these probes succeeds, Prof. Eyelar will begin performing the user-specified requests.
|
|
47
|
+
|
|
48
|
+
### Usage of environment variables in strings
|
|
49
|
+
You can reference available environment variables in `string` values of the configuration file. To do so, you simply wrap the variable name in `$ENV{}`. An example of this can be seen in [Example Configuration File](#example-configuration-file).
|
|
50
|
+
|
|
51
|
+
### Example Configuration File
|
|
52
|
+
```json
|
|
53
|
+
{
|
|
54
|
+
"service": {
|
|
55
|
+
"base_url": "http://$ENV{PROJECT_NAME}.com/",
|
|
56
|
+
"name": "$ENV{PROJECT_NAME}",
|
|
57
|
+
"api_docs_url": null
|
|
58
|
+
},
|
|
59
|
+
"profiling": {
|
|
60
|
+
"output_directory": "i/am/a/directory/hehe/",
|
|
61
|
+
"retry_count": 60
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
## Output Files
|
|
67
|
+
Profeyelar can generate up to 3 different types of profile result files, which can be analyzed with corresponding tools
|
|
68
|
+
- **Memray (`.bin`) Files**: Profiling results for memray will always be generated. These files end in `.bin` and have _memray_ in their name. You will need memray in order to generate reports from the data in these files. For more info about memray and the different generable reports, you can refer to their documentation [here](https://bloomberg.github.io/memray/getting_started.html).
|
|
69
|
+
- **Tracemalloc (`.tracemalloc`) Files**: Tracemalloc snapshots will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.tracemalloc`, and results can be viewed with corresponding functions of the [tracemalloc](https://docs.python.org/3/library/tracemalloc.html) module
|
|
70
|
+
- **cProfile (`.cprofile`) Files**: cProfile Stats will only be generated if the corresponding option is activated in the Profeyelar user interface. These files end in `.cprofile`, and results can be interpreted with corresponding tools like [pstats](https://docs.python.org/3.15/library/pstats.html) or [snakeviz](https://pypi.org/project/snakeviz/)
|
|
71
|
+
|
|
72
|
+
## Note regarding Gevent
|
|
73
|
+
As of time of writing, running memray on gunicorn based servers with `GeventWorker` workers results in an error when the server is stopped. We discovered this by chance when we checked the container logs before Profeyelar had finished the container shutdown process. The error does not seem to prevent The Professor from saving the profiling results and shutting the container down successfully. However, it is good to be aware of this behavior nonetheless. We did not encounter a similar error when running a gunicorn based server with Gthread (`ThreadWorker` workers) instead.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
src/profeyelar/__init__.py
|
|
5
|
+
src/profeyelar/__main__.py
|
|
6
|
+
src/profeyelar/launcher.py
|
|
7
|
+
src/profeyelar/parsing.py
|
|
8
|
+
src/profeyelar/preferences.py
|
|
9
|
+
src/profeyelar/profiling.py
|
|
10
|
+
src/profeyelar/ui.py
|
|
11
|
+
src/profeyelar.egg-info/PKG-INFO
|
|
12
|
+
src/profeyelar.egg-info/SOURCES.txt
|
|
13
|
+
src/profeyelar.egg-info/dependency_links.txt
|
|
14
|
+
src/profeyelar.egg-info/requires.txt
|
|
15
|
+
src/profeyelar.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
profeyelar
|