robotframework-gemini 0.3.1__tar.gz → 0.3.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- robotframework_gemini-0.3.3/PKG-INFO +200 -0
- robotframework_gemini-0.3.3/README.md +166 -0
- robotframework_gemini-0.3.1/README.md → robotframework_gemini-0.3.3/README.pt-BR.md +9 -36
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/pyproject.toml +3 -2
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/__init__.py +1 -1
- robotframework_gemini-0.3.1/PKG-INFO +0 -227
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/.gitignore +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/LICENSE +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/BrowserGeminiLibrary.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/GeminiLibrary.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/browser_helpers.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/client.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/legacy_env.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/library.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/prompt_build.py +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/py.typed +0 -0
- {robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/response_parse.py +0 -0
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: robotframework-gemini
|
|
3
|
+
Version: 0.3.3
|
|
4
|
+
Summary: Google Gemini oracles for Robot Framework: text-only and multimodal (image) assertions.
|
|
5
|
+
Project-URL: Homepage, https://github.com/carlosnizolli/robotframework-gemini
|
|
6
|
+
Project-URL: Documentation, https://github.com/carlosnizolli/robotframework-gemini/blob/main/README.md
|
|
7
|
+
Project-URL: Repository, https://github.com/carlosnizolli/robotframework-gemini
|
|
8
|
+
Project-URL: Issues, https://github.com/carlosnizolli/robotframework-gemini/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/carlosnizolli/robotframework-gemini/releases
|
|
10
|
+
Author: Carlos Nizolli
|
|
11
|
+
License-Expression: MIT
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Keywords: Gemini,LLM,QA,Robot Framework,multimodal,testing
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Framework :: Robot Framework
|
|
16
|
+
Classifier: Intended Audience :: Developers
|
|
17
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
18
|
+
Classifier: Operating System :: OS Independent
|
|
19
|
+
Classifier: Programming Language :: Python :: 3
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
24
|
+
Classifier: Topic :: Software Development :: Testing
|
|
25
|
+
Requires-Python: >=3.10
|
|
26
|
+
Requires-Dist: google-genai>=1.10.0
|
|
27
|
+
Provides-Extra: browser
|
|
28
|
+
Requires-Dist: robotframework-browser>=18.0.0; extra == 'browser'
|
|
29
|
+
Provides-Extra: dev
|
|
30
|
+
Requires-Dist: build>=1.0; extra == 'dev'
|
|
31
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
32
|
+
Requires-Dist: twine>=5.0; extra == 'dev'
|
|
33
|
+
Description-Content-Type: text/markdown
|
|
34
|
+
|
|
35
|
+
# robotframework-gemini
|
|
36
|
+
|
|
37
|
+
**English** | [Português (Brasil)](README.pt-BR.md)
|
|
38
|
+
|
|
39
|
+
Robot Framework **keyword library** for **Google Gemini** oracles: **text-only** evaluation (API payloads, logs, JSON, etc.) or **multimodal** checks with an image (PNG file or a **Robot Framework Browser** screenshot).
|
|
40
|
+
|
|
41
|
+
- **Multimodal**: sends prompt text and the capture as a byte `Part` (`google-genai`).
|
|
42
|
+
- **Recommended flow**: two inputs — **context** (test framing) and **evaluation** (criterion or yes/no question).
|
|
43
|
+
- **Browser optional**: screenshot keywords need `Library Browser`; text keywords work without a browser.
|
|
44
|
+
|
|
45
|
+
## Installation
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
pip install robotframework-gemini
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
With Robot Framework Browser (Playwright) screenshot support:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
pip install "robotframework-gemini[browser]"
|
|
55
|
+
python -m pip install robotframework
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Local development:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
python -m pip install -e ".[dev]"
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Environment variables and Library import
|
|
65
|
+
|
|
66
|
+
| Variable / argument | Purpose |
|
|
67
|
+
|---------------------|---------|
|
|
68
|
+
| `GEMINI_API_KEY` | Gemini API key (required unless you pass `api_key` on the Library) |
|
|
69
|
+
| `GEMINI_MODEL` | Model id (e.g. `gemini-2.5-flash`). If omitted, defaults to `gemini-2.5-flash`. |
|
|
70
|
+
| `api_key` (import) | Overrides `GEMINI_API_KEY` at Library import |
|
|
71
|
+
| `model` (import) | Overrides `GEMINI_MODEL` at Library import |
|
|
72
|
+
|
|
73
|
+
By default the Library reads key and model from the environment. You can also pass them on import (useful in CI or suites that keep credentials in Robot variables):
|
|
74
|
+
|
|
75
|
+
```robot
|
|
76
|
+
*** Variables ***
|
|
77
|
+
${GEMINI_API_KEY} %{GEMINI_API_KEY}
|
|
78
|
+
${GEMINI_MODEL} gemini-2.5-flash
|
|
79
|
+
|
|
80
|
+
*** Settings ***
|
|
81
|
+
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Model only (key still comes from the environment):
|
|
85
|
+
|
|
86
|
+
```robot
|
|
87
|
+
Library GeminiLibrary model=gemini-2.5-flash
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Explicit package import (equivalent):
|
|
91
|
+
|
|
92
|
+
```robot
|
|
93
|
+
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
> Do not commit a literal API key; prefer `%{GEMINI_API_KEY}` or CI secrets.
|
|
97
|
+
|
|
98
|
+
## Keyword documentation
|
|
99
|
+
|
|
100
|
+
- English: [docs/KEYWORDS.en.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.en.md)
|
|
101
|
+
- Português: [docs/KEYWORDS.pt-BR.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.pt-BR.md)
|
|
102
|
+
|
|
103
|
+
## Usage in Robot Framework
|
|
104
|
+
|
|
105
|
+
> **Note:** examples use `Set Variable` and `Catenate` instead of the `VAR` keyword (Robot Framework 7.0+) to stay compatible with older Robot Framework versions.
|
|
106
|
+
|
|
107
|
+
### Text only (no Browser)
|
|
108
|
+
|
|
109
|
+
```robot
|
|
110
|
+
*** Settings ***
|
|
111
|
+
Library GeminiLibrary
|
|
112
|
+
|
|
113
|
+
*** Keywords ***
|
|
114
|
+
Validate API response
|
|
115
|
+
${context}= Set Variable {"status": "ok", "items": 3}
|
|
116
|
+
${evaluation}= Set Variable Does the payload indicate success with items?
|
|
117
|
+
${model_response}= Gemini Evaluate Text ${context} ${evaluation}
|
|
118
|
+
Log ${model_response}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
### With a screenshot (Browser)
|
|
122
|
+
|
|
123
|
+
Import **Browser** before **Gemini**:
|
|
124
|
+
|
|
125
|
+
```robot
|
|
126
|
+
*** Settings ***
|
|
127
|
+
Library Browser
|
|
128
|
+
Library GeminiLibrary
|
|
129
|
+
|
|
130
|
+
*** Keywords ***
|
|
131
|
+
Check screen against a neutral criterion
|
|
132
|
+
${context}= Set Variable List filtered by Active status.
|
|
133
|
+
${evaluation}= Set Variable Do all visible items show Active status?
|
|
134
|
+
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
135
|
+
Log ${model_response}
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
Recommended import (RobotCode, runtime, and PyPI):
|
|
139
|
+
|
|
140
|
+
```robot
|
|
141
|
+
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Explicit package path (IDEs that prefer a full Python path):
|
|
145
|
+
|
|
146
|
+
```robot
|
|
147
|
+
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### IDE and RobotCode
|
|
151
|
+
|
|
152
|
+
| Import form | Resolved by |
|
|
153
|
+
|-------------|-------------|
|
|
154
|
+
| `Library GeminiLibrary` | Top-level `GeminiLibrary.py` module (installed in the wheel or via `src/` in a clone) |
|
|
155
|
+
| `Library robotframework_gemini.library.GeminiLibrary` | Standard Python package path |
|
|
156
|
+
|
|
157
|
+
In a **repository clone**, without installing:
|
|
158
|
+
|
|
159
|
+
- [`robot.toml`](robot.toml) — `python-path = ["src"]` for RobotCode/LSP
|
|
160
|
+
- [`.vscode/settings.json`](.vscode/settings.json) — `extraPaths` for Pylance/Cursor
|
|
161
|
+
|
|
162
|
+
With a local venv: `pip install -e ".[dev]"`, then reload the IDE window after changes.
|
|
163
|
+
|
|
164
|
+
With an image file already saved:
|
|
165
|
+
|
|
166
|
+
```robot
|
|
167
|
+
Browser.Take Screenshot ${OUTPUT_DIR}/screen.png
|
|
168
|
+
${model_response}= Gemini Evaluate With Image File ${context} ${evaluation} ${OUTPUT_DIR}/screen.png
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Verdict via prompt (first line) and 1–5 score:
|
|
172
|
+
|
|
173
|
+
```robot
|
|
174
|
+
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
175
|
+
... extra_instructions=Reply with one word on the first line: Yes or No.
|
|
176
|
+
${verdict}= Get Line ${model_response} 0
|
|
177
|
+
Should Be Equal As Strings ${verdict} Yes
|
|
178
|
+
|
|
179
|
+
# Score 1–5: two steps (log the reason) or one-line shortcut
|
|
180
|
+
${model_response}= Gemini Evaluate Text Rating ${context} ${evaluation}
|
|
181
|
+
${rating_score}= Gemini Parse Rating ${model_response}
|
|
182
|
+
|
|
183
|
+
${rating_score}= Gemini Evaluate Text Rating Score ${context} ${evaluation}
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Details for the three rating keywords: [Scores 1–5 (EN)](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.en.md#scores-15-three-keywords-when-to-use).
|
|
187
|
+
|
|
188
|
+
See also [examples/demo_template.robot](https://github.com/carlosnizolli/robotframework-gemini/blob/main/examples/demo_template.robot).
|
|
189
|
+
|
|
190
|
+
## Tests
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
pytest
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Tests mock `generate_content`; there are no real API calls.
|
|
197
|
+
|
|
198
|
+
## License
|
|
199
|
+
|
|
200
|
+
MIT.
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# robotframework-gemini
|
|
2
|
+
|
|
3
|
+
**English** | [Português (Brasil)](README.pt-BR.md)
|
|
4
|
+
|
|
5
|
+
Robot Framework **keyword library** for **Google Gemini** oracles: **text-only** evaluation (API payloads, logs, JSON, etc.) or **multimodal** checks with an image (PNG file or a **Robot Framework Browser** screenshot).
|
|
6
|
+
|
|
7
|
+
- **Multimodal**: sends prompt text and the capture as a byte `Part` (`google-genai`).
|
|
8
|
+
- **Recommended flow**: two inputs — **context** (test framing) and **evaluation** (criterion or yes/no question).
|
|
9
|
+
- **Browser optional**: screenshot keywords need `Library Browser`; text keywords work without a browser.
|
|
10
|
+
|
|
11
|
+
## Installation
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
pip install robotframework-gemini
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
With Robot Framework Browser (Playwright) screenshot support:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
pip install "robotframework-gemini[browser]"
|
|
21
|
+
python -m pip install robotframework
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Local development:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
python -m pip install -e ".[dev]"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Environment variables and Library import
|
|
31
|
+
|
|
32
|
+
| Variable / argument | Purpose |
|
|
33
|
+
|---------------------|---------|
|
|
34
|
+
| `GEMINI_API_KEY` | Gemini API key (required unless you pass `api_key` on the Library) |
|
|
35
|
+
| `GEMINI_MODEL` | Model id (e.g. `gemini-2.5-flash`). If omitted, defaults to `gemini-2.5-flash`. |
|
|
36
|
+
| `api_key` (import) | Overrides `GEMINI_API_KEY` at Library import |
|
|
37
|
+
| `model` (import) | Overrides `GEMINI_MODEL` at Library import |
|
|
38
|
+
|
|
39
|
+
By default the Library reads key and model from the environment. You can also pass them on import (useful in CI or suites that keep credentials in Robot variables):
|
|
40
|
+
|
|
41
|
+
```robot
|
|
42
|
+
*** Variables ***
|
|
43
|
+
${GEMINI_API_KEY} %{GEMINI_API_KEY}
|
|
44
|
+
${GEMINI_MODEL} gemini-2.5-flash
|
|
45
|
+
|
|
46
|
+
*** Settings ***
|
|
47
|
+
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Model only (key still comes from the environment):
|
|
51
|
+
|
|
52
|
+
```robot
|
|
53
|
+
Library GeminiLibrary model=gemini-2.5-flash
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Explicit package import (equivalent):
|
|
57
|
+
|
|
58
|
+
```robot
|
|
59
|
+
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
> Do not commit a literal API key; prefer `%{GEMINI_API_KEY}` or CI secrets.
|
|
63
|
+
|
|
64
|
+
## Keyword documentation
|
|
65
|
+
|
|
66
|
+
- English: [docs/KEYWORDS.en.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.en.md)
|
|
67
|
+
- Português: [docs/KEYWORDS.pt-BR.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.pt-BR.md)
|
|
68
|
+
|
|
69
|
+
## Usage in Robot Framework
|
|
70
|
+
|
|
71
|
+
> **Note:** examples use `Set Variable` and `Catenate` instead of the `VAR` keyword (Robot Framework 7.0+) to stay compatible with older Robot Framework versions.
|
|
72
|
+
|
|
73
|
+
### Text only (no Browser)
|
|
74
|
+
|
|
75
|
+
```robot
|
|
76
|
+
*** Settings ***
|
|
77
|
+
Library GeminiLibrary
|
|
78
|
+
|
|
79
|
+
*** Keywords ***
|
|
80
|
+
Validate API response
|
|
81
|
+
${context}= Set Variable {"status": "ok", "items": 3}
|
|
82
|
+
${evaluation}= Set Variable Does the payload indicate success with items?
|
|
83
|
+
${model_response}= Gemini Evaluate Text ${context} ${evaluation}
|
|
84
|
+
Log ${model_response}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
### With a screenshot (Browser)
|
|
88
|
+
|
|
89
|
+
Import **Browser** before **Gemini**:
|
|
90
|
+
|
|
91
|
+
```robot
|
|
92
|
+
*** Settings ***
|
|
93
|
+
Library Browser
|
|
94
|
+
Library GeminiLibrary
|
|
95
|
+
|
|
96
|
+
*** Keywords ***
|
|
97
|
+
Check screen against a neutral criterion
|
|
98
|
+
${context}= Set Variable List filtered by Active status.
|
|
99
|
+
${evaluation}= Set Variable Do all visible items show Active status?
|
|
100
|
+
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
101
|
+
Log ${model_response}
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Recommended import (RobotCode, runtime, and PyPI):
|
|
105
|
+
|
|
106
|
+
```robot
|
|
107
|
+
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Explicit package path (IDEs that prefer a full Python path):
|
|
111
|
+
|
|
112
|
+
```robot
|
|
113
|
+
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
### IDE and RobotCode
|
|
117
|
+
|
|
118
|
+
| Import form | Resolved by |
|
|
119
|
+
|-------------|-------------|
|
|
120
|
+
| `Library GeminiLibrary` | Top-level `GeminiLibrary.py` module (installed in the wheel or via `src/` in a clone) |
|
|
121
|
+
| `Library robotframework_gemini.library.GeminiLibrary` | Standard Python package path |
|
|
122
|
+
|
|
123
|
+
In a **repository clone**, without installing:
|
|
124
|
+
|
|
125
|
+
- [`robot.toml`](robot.toml) — `python-path = ["src"]` for RobotCode/LSP
|
|
126
|
+
- [`.vscode/settings.json`](.vscode/settings.json) — `extraPaths` for Pylance/Cursor
|
|
127
|
+
|
|
128
|
+
With a local venv: `pip install -e ".[dev]"`, then reload the IDE window after changes.
|
|
129
|
+
|
|
130
|
+
With an image file already saved:
|
|
131
|
+
|
|
132
|
+
```robot
|
|
133
|
+
Browser.Take Screenshot ${OUTPUT_DIR}/screen.png
|
|
134
|
+
${model_response}= Gemini Evaluate With Image File ${context} ${evaluation} ${OUTPUT_DIR}/screen.png
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Verdict via prompt (first line) and 1–5 score:
|
|
138
|
+
|
|
139
|
+
```robot
|
|
140
|
+
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
141
|
+
... extra_instructions=Reply with one word on the first line: Yes or No.
|
|
142
|
+
${verdict}= Get Line ${model_response} 0
|
|
143
|
+
Should Be Equal As Strings ${verdict} Yes
|
|
144
|
+
|
|
145
|
+
# Score 1–5: two steps (log the reason) or one-line shortcut
|
|
146
|
+
${model_response}= Gemini Evaluate Text Rating ${context} ${evaluation}
|
|
147
|
+
${rating_score}= Gemini Parse Rating ${model_response}
|
|
148
|
+
|
|
149
|
+
${rating_score}= Gemini Evaluate Text Rating Score ${context} ${evaluation}
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Details for the three rating keywords: [Scores 1–5 (EN)](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.en.md#scores-15-three-keywords-when-to-use).
|
|
153
|
+
|
|
154
|
+
See also [examples/demo_template.robot](https://github.com/carlosnizolli/robotframework-gemini/blob/main/examples/demo_template.robot).
|
|
155
|
+
|
|
156
|
+
## Tests
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
pytest
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Tests mock `generate_content`; there are no real API calls.
|
|
163
|
+
|
|
164
|
+
## License
|
|
165
|
+
|
|
166
|
+
MIT.
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
# robotframework-gemini
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[English](README.md) | **Português (Brasil)**
|
|
4
|
+
|
|
5
|
+
Biblioteca de **keywords do Robot Framework** para oráculos com **Google Gemini**: avaliação **só texto** (API, logs, JSON, etc.) ou **multimodal** com imagem (arquivo PNG ou captura do **Robot Framework Browser**).
|
|
4
6
|
|
|
5
7
|
- **Multimodal**: envia o texto do prompt e a captura como `Part` em bytes (`google-genai`).
|
|
6
8
|
- **Fluxo recomendado**: duas entradas — **contexto** (enquadramento do teste) e **avaliação** (critério ou pergunta objetiva).
|
|
@@ -59,29 +61,10 @@ Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_K
|
|
|
59
61
|
|
|
60
62
|
> Evite commitar a chave literal no repositório; prefira `%{GEMINI_API_KEY}` ou secrets do CI.
|
|
61
63
|
|
|
62
|
-
##
|
|
63
|
-
|
|
64
|
-
```python
|
|
65
|
-
from pathlib import Path
|
|
66
|
-
from robotframework_gemini import GeminiOrchestrator
|
|
67
|
-
|
|
68
|
-
orchestrator = GeminiOrchestrator()
|
|
69
|
-
# ou explicitamente:
|
|
70
|
-
# orchestrator = GeminiOrchestrator(api_key="...", model="gemini-2.5-flash")
|
|
71
|
-
context = "Web dashboard with the 'Active' category filter applied."
|
|
72
|
-
evaluation = "Do the visible list items match the selected category?"
|
|
73
|
-
model_response = orchestrator.evaluate_with_image(context, evaluation, Path("screen.png"))
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
Para formato de saída restrito (ex.: Yes/No), use `extra_instructions`:
|
|
64
|
+
## Documentação de Keywords
|
|
77
65
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
context,
|
|
81
|
-
evaluation,
|
|
82
|
-
extra_instructions="Reply with one word only: Yes or No.",
|
|
83
|
-
)
|
|
84
|
-
```
|
|
66
|
+
- Português: [docs/KEYWORDS.pt-BR.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.pt-BR.md)
|
|
67
|
+
- English: [docs/KEYWORDS.en.md](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.en.md)
|
|
85
68
|
|
|
86
69
|
## Uso no Robot Framework
|
|
87
70
|
|
|
@@ -124,7 +107,7 @@ Import recomendado (RobotCode, runtime e PyPI):
|
|
|
124
107
|
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
125
108
|
```
|
|
126
109
|
|
|
127
|
-
Import explícito do pacote (
|
|
110
|
+
Import explícito do pacote (IDEs que preferem o caminho completo):
|
|
128
111
|
|
|
129
112
|
```robot
|
|
130
113
|
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
@@ -166,19 +149,9 @@ ${rating_score}= Gemini Parse Rating ${model_response}
|
|
|
166
149
|
${rating_score}= Gemini Evaluate Text Rating Score ${context} ${evaluation}
|
|
167
150
|
```
|
|
168
151
|
|
|
169
|
-
Detalhes das três keywords de nota: [
|
|
170
|
-
|
|
171
|
-
Consulte também [`examples/demo_template.robot`](examples/demo_template.robot).
|
|
172
|
-
|
|
173
|
-
## Documentação de Keywords
|
|
174
|
-
|
|
175
|
-
- Português: [`docs/KEYWORDS.pt-BR.md`](docs/KEYWORDS.pt-BR.md)
|
|
176
|
-
- English: [`docs/KEYWORDS.en.md`](docs/KEYWORDS.en.md)
|
|
177
|
-
|
|
178
|
-
## Compatibilidade
|
|
152
|
+
Detalhes das três keywords de nota: [Notas 1–5 (pt-BR)](https://github.com/carlosnizolli/robotframework-gemini/blob/main/docs/KEYWORDS.pt-BR.md#notas-15-três-keywords-quando-usar).
|
|
179
153
|
|
|
180
|
-
|
|
181
|
-
- Variáveis legadas (`LLM_API_KEY`, `LLM_API_KEY_LIA`, `LLM_LIA_MODEL`) são mapeadas para `GEMINI_*` ao importar o pacote.
|
|
154
|
+
Consulte também [examples/demo_template.robot](https://github.com/carlosnizolli/robotframework-gemini/blob/main/examples/demo_template.robot).
|
|
182
155
|
|
|
183
156
|
## Testes
|
|
184
157
|
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "robotframework-gemini"
|
|
7
|
-
version = "0.3.
|
|
7
|
+
version = "0.3.3"
|
|
8
8
|
description = "Google Gemini oracles for Robot Framework: text-only and multimodal (image) assertions."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.10"
|
|
@@ -31,7 +31,7 @@ dependencies = [
|
|
|
31
31
|
|
|
32
32
|
[project.urls]
|
|
33
33
|
Homepage = "https://github.com/carlosnizolli/robotframework-gemini"
|
|
34
|
-
Documentation = "https://github.com/carlosnizolli/robotframework-gemini
|
|
34
|
+
Documentation = "https://github.com/carlosnizolli/robotframework-gemini/blob/main/README.md"
|
|
35
35
|
Repository = "https://github.com/carlosnizolli/robotframework-gemini"
|
|
36
36
|
Issues = "https://github.com/carlosnizolli/robotframework-gemini/issues"
|
|
37
37
|
Changelog = "https://github.com/carlosnizolli/robotframework-gemini/releases"
|
|
@@ -67,6 +67,7 @@ include = [
|
|
|
67
67
|
"src/BrowserGeminiLibrary.py",
|
|
68
68
|
"LICENSE",
|
|
69
69
|
"README.md",
|
|
70
|
+
"README.pt-BR.md",
|
|
70
71
|
]
|
|
71
72
|
|
|
72
73
|
[tool.pytest.ini_options]
|
|
@@ -1,227 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: robotframework-gemini
|
|
3
|
-
Version: 0.3.1
|
|
4
|
-
Summary: Google Gemini oracles for Robot Framework: text-only and multimodal (image) assertions.
|
|
5
|
-
Project-URL: Homepage, https://github.com/carlosnizolli/robotframework-gemini
|
|
6
|
-
Project-URL: Documentation, https://github.com/carlosnizolli/robotframework-gemini#readme
|
|
7
|
-
Project-URL: Repository, https://github.com/carlosnizolli/robotframework-gemini
|
|
8
|
-
Project-URL: Issues, https://github.com/carlosnizolli/robotframework-gemini/issues
|
|
9
|
-
Project-URL: Changelog, https://github.com/carlosnizolli/robotframework-gemini/releases
|
|
10
|
-
Author: Carlos Nizolli
|
|
11
|
-
License-Expression: MIT
|
|
12
|
-
License-File: LICENSE
|
|
13
|
-
Keywords: Gemini,LLM,QA,Robot Framework,multimodal,testing
|
|
14
|
-
Classifier: Development Status :: 4 - Beta
|
|
15
|
-
Classifier: Framework :: Robot Framework
|
|
16
|
-
Classifier: Intended Audience :: Developers
|
|
17
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
18
|
-
Classifier: Operating System :: OS Independent
|
|
19
|
-
Classifier: Programming Language :: Python :: 3
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
21
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
24
|
-
Classifier: Topic :: Software Development :: Testing
|
|
25
|
-
Requires-Python: >=3.10
|
|
26
|
-
Requires-Dist: google-genai>=1.10.0
|
|
27
|
-
Provides-Extra: browser
|
|
28
|
-
Requires-Dist: robotframework-browser>=18.0.0; extra == 'browser'
|
|
29
|
-
Provides-Extra: dev
|
|
30
|
-
Requires-Dist: build>=1.0; extra == 'dev'
|
|
31
|
-
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
32
|
-
Requires-Dist: twine>=5.0; extra == 'dev'
|
|
33
|
-
Description-Content-Type: text/markdown
|
|
34
|
-
|
|
35
|
-
# robotframework-gemini
|
|
36
|
-
|
|
37
|
-
Biblioteca **Python** e **keywords do Robot Framework** para oráculos com **Google Gemini**: avaliação **só texto** (API, logs, JSON, etc.) ou **multimodal** com imagem (arquivo PNG ou captura do **Robot Framework Browser**).
|
|
38
|
-
|
|
39
|
-
- **Multimodal**: envia o texto do prompt e a captura como `Part` em bytes (`google-genai`).
|
|
40
|
-
- **Fluxo recomendado**: duas entradas — **contexto** (enquadramento do teste) e **avaliação** (critério ou pergunta objetiva).
|
|
41
|
-
- **Browser opcional**: keywords de screenshot exigem `Library Browser`; keywords de texto funcionam sem navegador.
|
|
42
|
-
|
|
43
|
-
## Instalação
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
pip install robotframework-gemini
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
Com suporte a captura via Robot Framework Browser (Playwright):
|
|
50
|
-
|
|
51
|
-
```bash
|
|
52
|
-
pip install "robotframework-gemini[browser]"
|
|
53
|
-
python -m pip install robotframework
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
Desenvolvimento local:
|
|
57
|
-
|
|
58
|
-
```bash
|
|
59
|
-
python -m pip install -e ".[dev]"
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
## Variáveis de ambiente e importação da Library
|
|
63
|
-
|
|
64
|
-
| Variável / argumento | Função |
|
|
65
|
-
|----------------------|--------|
|
|
66
|
-
| `GEMINI_API_KEY` | Chave da API Gemini (obrigatória se não passar `api_key` na Library) |
|
|
67
|
-
| `GEMINI_MODEL` | Modelo (ex.: `gemini-2.5-flash`). Se omitido, usa `gemini-2.5-flash`. |
|
|
68
|
-
| `api_key` (import) | Sobrescreve `GEMINI_API_KEY` na importação da Library |
|
|
69
|
-
| `model` (import) | Sobrescreve `GEMINI_MODEL` na importação da Library |
|
|
70
|
-
|
|
71
|
-
Por padrão, a Library lê chave e modelo das variáveis de ambiente. Você também pode passá-los na importação (útil em CI ou suítes com credenciais em variáveis Robot):
|
|
72
|
-
|
|
73
|
-
```robot
|
|
74
|
-
*** Variables ***
|
|
75
|
-
${GEMINI_API_KEY} %{GEMINI_API_KEY}
|
|
76
|
-
${GEMINI_MODEL} gemini-2.5-flash
|
|
77
|
-
|
|
78
|
-
*** Settings ***
|
|
79
|
-
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
Só o modelo (chave continua vinda do ambiente):
|
|
83
|
-
|
|
84
|
-
```robot
|
|
85
|
-
Library GeminiLibrary model=gemini-2.5-flash
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
Import explícito (equivalente):
|
|
89
|
-
|
|
90
|
-
```robot
|
|
91
|
-
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
> Evite commitar a chave literal no repositório; prefira `%{GEMINI_API_KEY}` ou secrets do CI.
|
|
95
|
-
|
|
96
|
-
## Uso em Python (`GeminiOrchestrator`)
|
|
97
|
-
|
|
98
|
-
```python
|
|
99
|
-
from pathlib import Path
|
|
100
|
-
from robotframework_gemini import GeminiOrchestrator
|
|
101
|
-
|
|
102
|
-
orchestrator = GeminiOrchestrator()
|
|
103
|
-
# ou explicitamente:
|
|
104
|
-
# orchestrator = GeminiOrchestrator(api_key="...", model="gemini-2.5-flash")
|
|
105
|
-
context = "Web dashboard with the 'Active' category filter applied."
|
|
106
|
-
evaluation = "Do the visible list items match the selected category?"
|
|
107
|
-
model_response = orchestrator.evaluate_with_image(context, evaluation, Path("screen.png"))
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
Para formato de saída restrito (ex.: Yes/No), use `extra_instructions`:
|
|
111
|
-
|
|
112
|
-
```python
|
|
113
|
-
model_response = orchestrator.evaluate_with_text(
|
|
114
|
-
context,
|
|
115
|
-
evaluation,
|
|
116
|
-
extra_instructions="Reply with one word only: Yes or No.",
|
|
117
|
-
)
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
## Uso no Robot Framework
|
|
121
|
-
|
|
122
|
-
> **Nota:** os exemplos usam `Set Variable` e `Catenate` em vez da keyword `VAR` (Robot Framework 7.0+) para manter retrocompatibilidade com versões anteriores do Robot Framework.
|
|
123
|
-
|
|
124
|
-
### Só texto (sem Browser)
|
|
125
|
-
|
|
126
|
-
```robot
|
|
127
|
-
*** Settings ***
|
|
128
|
-
Library GeminiLibrary
|
|
129
|
-
|
|
130
|
-
*** Keywords ***
|
|
131
|
-
Validar resposta da API
|
|
132
|
-
${context}= Set Variable {"status": "ok", "items": 3}
|
|
133
|
-
${evaluation}= Set Variable O payload indica sucesso com itens?
|
|
134
|
-
${model_response}= Gemini Evaluate Text ${context} ${evaluation}
|
|
135
|
-
Log ${model_response}
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
### Com captura de tela (Browser)
|
|
139
|
-
|
|
140
|
-
Declare a biblioteca **Browser** antes de **Gemini**:
|
|
141
|
-
|
|
142
|
-
```robot
|
|
143
|
-
*** Settings ***
|
|
144
|
-
Library Browser
|
|
145
|
-
Library GeminiLibrary
|
|
146
|
-
|
|
147
|
-
*** Keywords ***
|
|
148
|
-
Checar tela por critério neutro
|
|
149
|
-
${context}= Set Variable Lista filtrada por status Ativo.
|
|
150
|
-
${evaluation}= Set Variable Todos os itens visíveis mostram status Ativo?
|
|
151
|
-
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
152
|
-
Log ${model_response}
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
Import recomendado (RobotCode, runtime e PyPI):
|
|
156
|
-
|
|
157
|
-
```robot
|
|
158
|
-
Library GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
Import explícito do pacote (Python / IDEs que preferem o caminho completo):
|
|
162
|
-
|
|
163
|
-
```robot
|
|
164
|
-
Library robotframework_gemini.library.GeminiLibrary api_key=${GEMINI_API_KEY} model=${GEMINI_MODEL}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
### IDE e RobotCode
|
|
168
|
-
|
|
169
|
-
| Forma de import | Quem resolve |
|
|
170
|
-
|-----------------|--------------|
|
|
171
|
-
| `Library GeminiLibrary` | Módulo top-level `GeminiLibrary.py` (instalado no wheel ou via `src/` no clone) |
|
|
172
|
-
| `Library robotframework_gemini.library.GeminiLibrary` | Pacote Python padrão |
|
|
173
|
-
|
|
174
|
-
No **clone do repositório**, sem instalar:
|
|
175
|
-
|
|
176
|
-
- [`robot.toml`](robot.toml) — `python-path = ["src"]` para RobotCode/LSP
|
|
177
|
-
- [`.vscode/settings.json`](.vscode/settings.json) — `extraPaths` para Pylance/Cursor
|
|
178
|
-
|
|
179
|
-
Com venv local: `pip install -e ".[dev]"` e recarregue a janela do IDE após mudanças.
|
|
180
|
-
|
|
181
|
-
Com arquivo já salvo:
|
|
182
|
-
|
|
183
|
-
```robot
|
|
184
|
-
Browser.Take Screenshot ${OUTPUT_DIR}/tela.png
|
|
185
|
-
${model_response}= Gemini Evaluate With Image File ${context} ${evaluation} ${OUTPUT_DIR}/tela.png
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
Veredito via prompt (primeira linha) e nota 1–5:
|
|
189
|
-
|
|
190
|
-
```robot
|
|
191
|
-
${model_response}= Gemini Evaluate With Screen ${context} ${evaluation}
|
|
192
|
-
... extra_instructions=Responda com uma palavra na primeira linha: Sim ou Não.
|
|
193
|
-
${verdict}= Get Line ${model_response} 0
|
|
194
|
-
Should Be Equal As Strings ${verdict} Sim
|
|
195
|
-
|
|
196
|
-
# Nota 1–5: duas etapas (log da justificativa) ou atalho em uma linha
|
|
197
|
-
${model_response}= Gemini Evaluate Text Rating ${context} ${evaluation}
|
|
198
|
-
${rating_score}= Gemini Parse Rating ${model_response}
|
|
199
|
-
|
|
200
|
-
${rating_score}= Gemini Evaluate Text Rating Score ${context} ${evaluation}
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
Detalhes das três keywords de nota: [`docs/KEYWORDS.pt-BR.md#notas-15-três-keywords-quando-usar`](docs/KEYWORDS.pt-BR.md#notas-15-três-keywords-quando-usar).
|
|
204
|
-
|
|
205
|
-
Consulte também [`examples/demo_template.robot`](examples/demo_template.robot).
|
|
206
|
-
|
|
207
|
-
## Documentação de Keywords
|
|
208
|
-
|
|
209
|
-
- Português: [`docs/KEYWORDS.pt-BR.md`](docs/KEYWORDS.pt-BR.md)
|
|
210
|
-
- English: [`docs/KEYWORDS.en.md`](docs/KEYWORDS.en.md)
|
|
211
|
-
|
|
212
|
-
## Compatibilidade
|
|
213
|
-
|
|
214
|
-
- `BrowserGeminiLibrary` permanece como alias de `GeminiLibrary` para suítes existentes.
|
|
215
|
-
- Variáveis legadas (`LLM_API_KEY`, `LLM_API_KEY_LIA`, `LLM_LIA_MODEL`) são mapeadas para `GEMINI_*` ao importar o pacote.
|
|
216
|
-
|
|
217
|
-
## Testes
|
|
218
|
-
|
|
219
|
-
```bash
|
|
220
|
-
pytest
|
|
221
|
-
```
|
|
222
|
-
|
|
223
|
-
Os testes usam mocks de `generate_content`; não há chamadas reais à API.
|
|
224
|
-
|
|
225
|
-
## Licença
|
|
226
|
-
|
|
227
|
-
MIT.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/client.py
RENAMED
|
File without changes
|
{robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/legacy_env.py
RENAMED
|
File without changes
|
{robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/library.py
RENAMED
|
File without changes
|
|
File without changes
|
{robotframework_gemini-0.3.1 → robotframework_gemini-0.3.3}/src/robotframework_gemini/py.typed
RENAMED
|
File without changes
|
|
File without changes
|