wiliot-certificate 4.5.0a1__tar.gz → 4.5.0a2__tar.gz

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: wiliot_certificate
-Version: 4.5.0a1
+Version: 4.5.0a2
 Summary: A library for certifying Wiliot-compliant boards
 Author-email: Wiliot <support@wiliot.com>
 License-Expression: MIT
@@ -34,7 +34,7 @@ Dynamic: license-file
 
 <!-- Description -->
 wiliot-certificate is a Python library that provides tools for testing and certifying boards for compatibility with Wiliot’s ecosystem.
-This python package includes the following CLI utilities:
+This python package includes the following tools:
  - Certification Wizard (`wlt-cert`)
  - Certificate CLI (`wlt-cert-cli`)
  - Tester Upgrade (`wlt-cert-tester-upgrade`)
@@ -55,33 +55,11 @@ In terminal, run:
 ````commandline
 wlt-cert
 ````
-<!-- This tool walks you through... TODO -->
-<!-- You need to have a json validation schema <LINK>, certification tester & custom broker file <LINK> -->
+This tool is the default to test and certify your device.
+It runs a setup wizard that walks you through the initialization steps before running the tests.
+You'll need a [validation schema](https://community.wiliot.com/customers/s/article/Validation-Schema), tester device and custom broker json file ([more info here](https://community.wiliot.com/customers/s/article/Wiliot-Gateway-Certification)).
+Once set up it opens a terminal and tests your device.
 
-#### The following capabilities are not tested in this version
-##### Cloud Connectivity & Misc
- - Board type registered within the Board Type Management system
- - Bridge OTA progress reporting
-##### Power management
-  - Functionality of energize and transmit in sleep mode
-##### Edge management
-  - Timing of heartbeat and interface packets 
-##### Module Energy 2400
-  - Functionality of energy pattern, output power and duty cycle
-##### Module Energy SUB1G
-  - Functionality of energy pattern and duty cycle 
-##### Module Datapath 
-  - RSSI edge cases: -127 and 0 
-  - Functionality of transmission pattern, output power
-  - Pacer interval with channel 10 and 500k modulation
-  - Pacer interval using GEN3 Pixels 
-  - Packet filter: the following configuration - DEBUG, TEMP & DEBUG, TEMPS & DEBUG & ADVANCE
-  - Rx rate feature with extended advertising 
-  - Functionality of adaptive pacer algorithm
-  - Supported Pixels for extended advertising and GEN3
-##### Calibration 
-  - Functionality of output power and interval calibration 
-  - Functionality of calibration transmission patterns for the configuration STANDARD & EU & DISABLE
 
 
 ### Certificate CLI
@@ -107,4 +85,30 @@ wlt-cert-reg
 ````
 Certify the gateway registration process to Wiliot platform.
 The gateway must use Wiliot production MQTT broker, and mustn't be registered to any account on Wiliot platform.
-Use -h for details on the arguments. <!-- More info can be found in TODO link-->
+Use -h for details on the arguments. [More info here](https://community.wiliot.com/customers/s/article/Wiliot-Gateway-Certification)
+
+
+## The following capabilities are not tested in this version
+##### Cloud Connectivity & Misc
+ - Board type registered within the Board Type Management system
+ - Bridge OTA progress reporting
+##### Power management
+  - Functionality of energize and transmit in sleep mode
+##### Edge management
+  - Timing of heartbeat and interface packets 
+##### Module Energy 2400
+  - Functionality of energy pattern, output power and duty cycle
+##### Module Energy SUB1G
+  - Functionality of energy pattern and duty cycle 
+##### Module Datapath 
+  - RSSI edge cases: -127 and 0 
+  - Functionality of transmission pattern, output power
+  - Pacer interval with channel 10 and 500k modulation
+  - Pacer interval using GEN3 Pixels 
+  - Packet filter: the following configuration - DEBUG, TEMP & DEBUG, TEMPS & DEBUG & ADVANCE
+  - Rx rate feature with extended advertising 
+  - Functionality of adaptive pacer algorithm
+  - Supported Pixels for extended advertising and GEN3
+##### Calibration 
+  - Functionality of output power and interval calibration 
+  - Functionality of calibration transmission patterns for the configuration STANDARD & EU & DISABLE

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/README.md

@@ -2,7 +2,7 @@
 
 <!-- Description -->
 wiliot-certificate is a Python library that provides tools for testing and certifying boards for compatibility with Wiliot’s ecosystem.
-This python package includes the following CLI utilities:
+This python package includes the following tools:
  - Certification Wizard (`wlt-cert`)
  - Certificate CLI (`wlt-cert-cli`)
  - Tester Upgrade (`wlt-cert-tester-upgrade`)
@@ -23,33 +23,11 @@ In terminal, run:
 ````commandline
 wlt-cert
 ````
-<!-- This tool walks you through... TODO -->
-<!-- You need to have a json validation schema <LINK>, certification tester & custom broker file <LINK> -->
+This tool is the default to test and certify your device.
+It runs a setup wizard that walks you through the initialization steps before running the tests.
+You'll need a [validation schema](https://community.wiliot.com/customers/s/article/Validation-Schema), tester device and custom broker json file ([more info here](https://community.wiliot.com/customers/s/article/Wiliot-Gateway-Certification)).
+Once set up it opens a terminal and tests your device.
 
-#### The following capabilities are not tested in this version
-##### Cloud Connectivity & Misc
- - Board type registered within the Board Type Management system
- - Bridge OTA progress reporting
-##### Power management
-  - Functionality of energize and transmit in sleep mode
-##### Edge management
-  - Timing of heartbeat and interface packets 
-##### Module Energy 2400
-  - Functionality of energy pattern, output power and duty cycle
-##### Module Energy SUB1G
-  - Functionality of energy pattern and duty cycle 
-##### Module Datapath 
-  - RSSI edge cases: -127 and 0 
-  - Functionality of transmission pattern, output power
-  - Pacer interval with channel 10 and 500k modulation
-  - Pacer interval using GEN3 Pixels 
-  - Packet filter: the following configuration - DEBUG, TEMP & DEBUG, TEMPS & DEBUG & ADVANCE
-  - Rx rate feature with extended advertising 
-  - Functionality of adaptive pacer algorithm
-  - Supported Pixels for extended advertising and GEN3
-##### Calibration 
-  - Functionality of output power and interval calibration 
-  - Functionality of calibration transmission patterns for the configuration STANDARD & EU & DISABLE
 
 
 ### Certificate CLI
@@ -75,4 +53,30 @@ wlt-cert-reg
 ````
 Certify the gateway registration process to Wiliot platform.
 The gateway must use Wiliot production MQTT broker, and mustn't be registered to any account on Wiliot platform.
-Use -h for details on the arguments. <!-- More info can be found in TODO link-->
+Use -h for details on the arguments. [More info here](https://community.wiliot.com/customers/s/article/Wiliot-Gateway-Certification)
+
+
+## The following capabilities are not tested in this version
+##### Cloud Connectivity & Misc
+ - Board type registered within the Board Type Management system
+ - Bridge OTA progress reporting
+##### Power management
+  - Functionality of energize and transmit in sleep mode
+##### Edge management
+  - Timing of heartbeat and interface packets 
+##### Module Energy 2400
+  - Functionality of energy pattern, output power and duty cycle
+##### Module Energy SUB1G
+  - Functionality of energy pattern and duty cycle 
+##### Module Datapath 
+  - RSSI edge cases: -127 and 0 
+  - Functionality of transmission pattern, output power
+  - Pacer interval with channel 10 and 500k modulation
+  - Pacer interval using GEN3 Pixels 
+  - Packet filter: the following configuration - DEBUG, TEMP & DEBUG, TEMPS & DEBUG & ADVANCE
+  - Rx rate feature with extended advertising 
+  - Functionality of adaptive pacer algorithm
+  - Supported Pixels for extended advertising and GEN3
+##### Calibration 
+  - Functionality of output power and interval calibration 
+  - Functionality of calibration transmission patterns for the configuration STANDARD & EU & DISABLE

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "wiliot_certificate"
-version = "4.5.0a1"
+version = "4.5.0a2"
 authors = [
   { name="Wiliot", email="support@wiliot.com" }
 ]

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/cert_common.py

@@ -1273,7 +1273,6 @@ def output_power_supported(board_type, output_power):
     else:
         return output_power in ag.OUTPUT_POWER_2_4_SUPPORTED_VALUES
 
-
 def wiliot_pkts_validation(test, all_messages_in_test, all_data_pkts):
     PHASE_NAME = "Wiliot packets validation"
     phase_run_print(PHASE_NAME)
@@ -1290,7 +1289,7 @@ def wiliot_pkts_validation(test, all_messages_in_test, all_data_pkts):
         wiliot_pkts_validation_phase = seq_id_validation(wiliot_pkts_validation_phase, all_data_pkts)
 
     field_functionality_pass_fail_print(wiliot_pkts_validation_phase, PHASE_NAME)
-    test.phases[-1:-1] = [wiliot_pkts_validation_phase]
+    test.add_phase(wiliot_pkts_validation_phase)
     return test
 
 
@@ -1373,19 +1372,19 @@ def generate_graph_stress_test(test, results, test_pkts_received):
     graph_data = []
     for i in range(0, len(results), 2):
         if i + 1 < len(results):
-            graph_data.append({'pkts_per_sec': results[i], 'percent_received': results[i + 1]})
+            graph_data.append({'pkts_per_sec': results[i], 'received_pps': results[i + 1]})
     
     graph_df = pd.DataFrame(graph_data)
     html_file_path = os.path.join(ARTIFACTS_DIR, test.dir, 'stress_graph.html')
     
     # First graph: percentage received vs packets per second
-    fig1 = px.line(graph_df, x='pkts_per_sec', y='percent_received',
-                   title='Percentage of packets uploaded by packets per second advertised',
-                   labels={'pkts_per_sec': 'Packets Sent Per Second', 'percent_received': 'Percentage of packets received'},
-                   markers=True, text='percent_received')
+    fig1 = px.line(graph_df, x='pkts_per_sec', y='received_pps',
+                   title='Packets Per Second Uploaded vs Packets Per Second Advertised',
+                   labels={'pkts_per_sec': 'Advertised PPS', 'received_pps': 'Uploaded PPS'},
+                   markers=True, text='received_pps')
     
-    # Set y-axis to 1-100 scale
-    fig1.update_yaxes(range=[1, 100])
+    # Set y-axis to [0 - highest_pps] scale
+    fig1.update_yaxes(range=[0, results[-2]])
     
     # Position text labels next to the markers
     fig1.update_traces(textposition="top right")
@@ -1435,16 +1434,16 @@ def stress_analysis(test, pps, sent_pkts, received_pkts):
     print(f'Number of packets sent: {pkts_sent_count}')
     print(f'Number of packets received: {pkts_received_count}')
     percentage_received = round(pkts_received_count * 100 / pkts_sent_count)
+    received_pps = pps * percentage_received / 100
     
     # PASS/FAIL logic
-    stage_pass = pkts_received_count / pkts_sent_count * 100
-    if stage_pass < 1: # If less than 1% of the packets were received, fail the test
+    if percentage_received < 1: # If less than 1% of the packets were received, fail the test
         test.rc = TEST_FAILED
         test.reason = "Insufficient amount of packets were scanned & uploaded by the gateway"
     else:
-        test.reason = f'{percentage_received}% of packets received, average pps: {pps * percentage_received / 100}'
+        test.reason = f'received pps: {received_pps} ({percentage_received}% of packets)'
 
-    return test, percentage_received
+    return test, received_pps
 
 
 def generate_adv_payload(test_indicator, unique_pkt=False):

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/cert_defines.py

@@ -14,6 +14,7 @@ except importlib.metadata.PackageNotFoundError:
     CERT_VERSION = LOCAL_DEV
 CERT_MQTT_LOG_FILE =            "cert_mqtt_log.json"
 DATA_SIM_LOG_FILE =             "data_sim_log.txt"
+RESULT_NOTES_FILE =             "results_notes.txt"
 UT_RESULT_FILE_HTML =           "results.html"
 UT_RESULT_FILE_PDF =            "results.pdf"
 UTILS_BASE_REL_PATH =           "../../../utils"

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/cert_gw_sim.py

@@ -255,6 +255,9 @@ def interference_analysis(ble_serial):
         for key in CNTRS_KEYS:
             diff_dict[key] = handle_wrap_around(end_cntrs[key] - start_cntrs[key])
         bad_crc_percentage = round((diff_dict[BAD_CRC] / (diff_dict[WLT_RX] + diff_dict[NON_WLT_RX])) * 100)
+        notes_file_path = os.path.join(ARTIFACTS_DIR, RESULT_NOTES_FILE)
+        with open(notes_file_path, "a") as f:
+            f.write(f'Channel {channel[0]} ({channel[1]} MHz) Ambient Interference (bad CRC percentage): {bad_crc_percentage}%\n')
         print(color('WARNING', f'Channel {channel[0]} ({channel[1]} MHz) Ambient Interference (bad CRC percentage) is: {bad_crc_percentage}%'))
         print(f'Good CRC packets = {diff_dict[NON_WLT_RX] + diff_dict[WLT_RX] - diff_dict[BAD_CRC]}, bad CRC packets: {diff_dict[BAD_CRC]}')
 

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/cert_results.py

@@ -36,6 +36,7 @@ BLE_MAC_ADDRESS =                  "BLE MAC Address"
 BOARD_TYPE =                       "Board Type"
 BLE_VER =                          "BLE Version"
 SUP_API_VER =                      "Supported API Version"
+ADD_INFO =                         "Additional information"
 # TODO: This is a temporary list of all schema module names - remove this when auto generated
 MODULE_SCHEMA_NAMES_LIST = ["Cloud Connectivity", "Edge Management", "Calibration", "Datapath", "Energizer 2.4GHz",
                             "Energizer Sub-1GHz", "Power Management", "BLE Sensor", "Custom"]
@@ -256,6 +257,13 @@ def generate_results_files(dut=None, tester=None, html=True, pdf=True, failures=
             hdr_page.append(Paragraph(f"{SIM} {BOARD_TYPE}: {ag.BOARD_TYPES_LIST[internal_brg.board_type]}", bold_text_style))
             hdr_page.append(Paragraph(f"{SIM} {BLE_VER}: {internal_brg.version}", bold_text_style))
             hdr_page.append(Paragraph(f"{SIM} {SUP_API_VER}: {internal_brg.api_version}", bold_text_style))
+        notes_file_path = os.path.join(ARTIFACTS_DIR, RESULT_NOTES_FILE)
+        if os.path.exists(notes_file_path):
+            hdr_page.append(Paragraph(f"<u>{ADD_INFO}:</u>", bold_text_style))
+            with open(notes_file_path, 'r') as f:
+                for line in f.readlines():
+                    hdr_page.append(Paragraph(f"{line}", bold_text_style))
+
         hdr_page.append(Spacer(1, 20))
 
         # Analyze results per module and generate results per module for PDF

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/cert_utils.py

@@ -220,6 +220,9 @@ class WltTest:
                 return phase
         return None
 
+    def add_phase(self, phase):
+        self.phases[-1:-1] = [phase]
+
     def update_overall_rc(self):
         if any([phase.rc == TEST_FAILED for phase in self.phases]):
             self.rc = TEST_FAILED

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/certificate_eth_test_list.txt

@@ -71,4 +71,5 @@ cloud_connectivity/stress_test 20 40 60 80 100 120 140 160 180 200 222 250 285 3
 cloud_connectivity/ext_adv_stress_test 20 40 60 80 100 120 140 160 180 200
 cloud_connectivity/channel_scan_behaviour_test
 cloud_connectivity/acl_test deny allow
-cloud_connectivity/acl_ext_adv_test deny allow
+cloud_connectivity/acl_ext_adv_test deny allow
+cloud_connectivity/deduplication_test

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/certificate_test_list.txt

@@ -73,3 +73,4 @@ cloud_connectivity/channel_scan_behaviour_test
 cloud_connectivity/acl_test deny allow
 cloud_connectivity/acl_ext_adv_test deny allow
 cloud_connectivity/brg_ota_test
+cloud_connectivity/deduplication_test

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/acl_ext_adv_test/acl_ext_adv_test.json

@@ -2,7 +2,7 @@
     "name": "Access Control List using Extended Advertising",
     "module": "Cloud Connectivity",
     "purpose": "Test the access control list (ACL) functionality, configure allow list and deny list and test the behavior, while the Bridge is transmitting in extended advertising",
-    "documentation": "<TEST_DOCUMENTATION_LINK>",
+    "documentation": "https://community.wiliot.com/customers/s/article/Wiliot-Network---Gateway-Extended-Advertising-Implementation",
     "initialCondition": "Gateway configured to defaults",
     "procedure": ["Test prolog", "configure the brg to be on the deny list", "configure the brg to be on the allow list", "Test epilog"],
     "expectedOutcome": "For the deny list - no unified packets are found, for the allow list - unified packets are found",

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/acl_test/acl_test.json

@@ -2,7 +2,7 @@
     "name": "Access Control List",
     "module": "Cloud Connectivity",
     "purpose": "Test the access control list (ACL) functionality, configure allow list and deny list and test the behavior",
-    "documentation": "<TEST_DOCUMENTATION_LINK>",
+    "documentation": "https://community.wiliot.com/customers/s/article/Access-Control-List",
     "initialCondition": "Gateway configured to defaults",
     "procedure": ["Test prolog", "configure the brg to be on the deny list", "configure the brg to be on the allow list", "Test epilog"],
     "expectedOutcome": "For the deny list - no unified packets are found, for the allow list - unified packets are found",

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/channel_scan_behaviour_test/channel_scan_behaviour_test.json

@@ -2,7 +2,7 @@
     "name": "Gateway Channel behavior",
     "module": "Cloud Connectivity",
     "purpose": "Test gateway's channel scanning time cycles on all main channels",
-    "documentation": "<TEST_DOCUMENTATION_LINK>",
+    "documentation": "",
     "initialCondition": "Gateway configured with default settings",
     "procedure": ["Test prolog",
                   "Send packets as close as possible to continuously on all 3 channels, with a unique channel identifier",

wiliot_certificate-4.5.0a2/src/certificate/tests/cloud_connectivity/deduplication_test/deduplication_test.json

@@ -0,0 +1,14 @@
+{
+    "name": "Gateway Deduplication",
+    "module": "Cloud Connectivity",
+    "purpose": "Test gateway's packet deduplication capability when receiving identical BLE packets",
+    "documentation": "",
+    "initialCondition": "Gateway configured to defaults",
+    "procedure": ["Test prolog", "Send mixed packet types (pixels, mgmt, sensor, side_info_sensor) via data simulator", "Collect received packets from MQTT", "Analyze duplicate packets and calculate duplicate percentage", "Test epilog"],
+    "expectedOutcome": "Duplicate percentage should be <= 5% of total received packets",
+    "mandatory": 0,
+    "multiBridgeTest": 0,
+    "gwOnlyTest": 1,
+    "dataSimOnlyTest": 1,
+    "allSupportedValues": []
+}

wiliot_certificate-4.5.0a2/src/certificate/tests/cloud_connectivity/deduplication_test/deduplication_test.py

@@ -0,0 +1,85 @@
+# This test meant to show the GW performance when is put in a pkt stress generated from the BLE via the UART
+from certificate.cert_prints import *
+from certificate.cert_defines import *
+from certificate.wlt_types import *
+import certificate.cert_common as cert_common
+import certificate.cert_utils as cert_utils
+import certificate.cert_mqtt as cert_mqtt
+import certificate.cert_data_sim as cert_data_sim
+import pandas as pd
+
+
+# DEFINES
+UPLINK_TEST_INDICATOR = cert_utils.get_random_hex_str(len=6)
+NUM_OF_BRGS = 3
+
+
+# HELPER FUNCTIONS
+def duplication_analysis(test, pkts, received_pkts):
+    if len(received_pkts) == 0:
+        test.rc = TEST_FAILED
+        test.reason = "No packets were received!"
+        return test
+
+    _sent_pkts = []
+    for pkt in pkts:
+        pkt_str = pkt.get_pkt()
+        adva_endianness_change = cert_common.change_endianness(pkt_str[:12]) + pkt_str[12:]  # Switch to big endian adva
+        _sent_pkts.append(adva_endianness_change)
+
+    sent_df = pd.DataFrame(_sent_pkts, columns=[PACKETS])
+    received_df = pd.DataFrame(received_pkts, columns=[PACKETS])
+
+    merged_df = pd.merge(sent_df, received_df, on=PACKETS, how='inner')
+    # Count total received packets (including duplicates) and unique packets
+    pkts_received_total = len(merged_df)
+    merged_df_unique = merged_df.drop_duplicates(subset=[PACKETS])
+    pkts_duplicates = pkts_received_total - len(merged_df_unique)
+
+    # Prints
+    print(f'Number of packets sent: {len(sent_df)}')
+    print(f'Number of packets received in total: {pkts_received_total}, out of them {pkts_duplicates} are duplicates')
+
+    duplicate_percentage = (pkts_duplicates / pkts_received_total) * 100
+    print(f'{duplicate_percentage:.2f}% of the received packets are duplicates')
+
+    if duplicate_percentage > 5.0:
+        test.rc = TEST_FAILED
+        test.reason = f"{duplicate_percentage:.2f}% of the received packets are duplicates"
+        return test
+
+    return test
+
+
+def run(test):
+    test = cert_common.test_prolog(test)
+    if test.rc == TEST_FAILED:
+        return cert_common.test_epilog(test)
+
+    dut_mqttc = test.get_mqttc_by_target(DUT)
+    dut_mqttc.flush_pkts()
+    pkts = []
+
+    # generate pkts and send them using data simulator
+    data_pkts, _ = cert_data_sim.brg_pkt_gen(num_of_pkts_per_brg=10, num_of_brgs=NUM_OF_BRGS,
+                                             pkt_type=PIXELS_PKT, indicator=UPLINK_TEST_INDICATOR)
+
+    mgmt_pkts, _ = cert_data_sim.brg_pkt_gen(num_of_pkts_per_brg=10, num_of_brgs=NUM_OF_BRGS,
+                                             pkt_type=MGMT_PKT, indicator=UPLINK_TEST_INDICATOR)
+    sensor_pkts, _ = cert_data_sim.brg_pkt_gen(num_of_pkts_per_brg=10, num_of_brgs=NUM_OF_BRGS,
+                                               pkt_type=SENSOR_PKT, indicator=UPLINK_TEST_INDICATOR)
+    si_pkts, _ = cert_data_sim.brg_pkt_gen(num_of_pkts_per_brg=10, num_of_brgs=NUM_OF_BRGS,
+                                           pkt_type=SIDE_INFO_SENSOR_PKT, indicator=UPLINK_TEST_INDICATOR)
+    pkts = data_pkts + mgmt_pkts + sensor_pkts + si_pkts
+
+    pixel_sim_thread = cert_data_sim.GenericSimThread(test=test, pkts=pkts, send_single_cycle=True)
+    pixel_sim_thread.start()
+
+    mqtt_scan_wait(test, 10 + test.dut.upload_wait_time)
+    cert_mqtt.dump_pkts(test, log="deduplication_test")
+    recieved_pkts = cert_mqtt.get_all_data_pkts(dut_mqttc, indicator=UPLINK_TEST_INDICATOR)
+    pixel_sim_thread.stop()
+
+    test = duplication_analysis(test, pkts, [p[ALIAS_BRIDGE_ID] + p[PAYLOAD] for p in recieved_pkts])
+
+    return cert_common.test_epilog(test)

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/ext_adv_stress_test/ext_adv_stress_test.json

@@ -12,5 +12,5 @@
     "multiBridgeTest": 0,
     "gwOnlyTest": 1,
     "dataSimOnlyTest": 1,
-    "allSupportedValues": [20, 40, 60, 80, 100, 120, 140, 160, 180, 200]
+    "allSupportedValues": []
 }

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/ext_adv_stress_test/ext_adv_stress_test.py

@@ -51,13 +51,13 @@ def run(test):
 
     cert_config.gw_action(test, f'{cert_gw_sim.BLE_SIM_INIT} 1', TESTER)
 
+    ppses = [20, 40, 60, 80, 100, 120, 140, 160, 180, 200]
     test_pkts_received = []
     data_pkts_received = []
     results = []
-    for param in test.params:
-        phase_run_print(f"PPS = {param}")
+    for pps in ppses:
+        phase_run_print(f"PPS = {pps}")
         test.flush_all_mqtt_packets()
-        pps = param.value
         # adv_duration is increased only for lowest pps to increase packets sample for reliable results
         adv_duration = ADV_DURATION_LOWEST_PPS if pps == LOWEST_PPS else ADV_DURATION_DEFAULT
         delay = math.floor(1000 / pps)  # This should be bigger than 2ms
@@ -74,17 +74,18 @@ def run(test):
         cert_config.gw_action(test, gw_action_cmd, TESTER)
         mqtt_scan_wait(test, adv_duration + upload_wait_time)
 
-        cert_mqtt.dump_pkts(test, log=param.name)
+        cert_mqtt.dump_pkts(test, log=str(pps))
         phase_pkts_received = cert_mqtt.get_all_aggregated_data_pkts(test.get_mqttc_by_target(DUT), indicator=STRESS_TEST_INDICATOR)
 
-        test, percentage_received = cert_common.stress_analysis(
+        test, received_pps = cert_common.stress_analysis(
             test, pps, generated_payloads,
             [(p[ALIAS_BRIDGE_ID] + p[AGGREGATED_PAYLOAD]) for p in phase_pkts_received])
-        results.extend([pps, percentage_received])
-        cert_mqtt.generate_log_file(test, param.name)
+        results.extend([pps, received_pps])
+        cert_mqtt.generate_log_file(test, str(pps))
 
-        test.set_phase_rc(param.name, test.rc)
-        test.add_phase_reason(param.name, test.reason)
+        test.add_phase(cert_utils.Phase(pps))
+        test.set_phase_rc(str(pps), test.rc)
+        test.add_phase_reason(str(pps), test.reason)
         if test.rc == TEST_FAILED and test.exit_on_param_failure:
             break  # break the whole for loop and keep the test as failed
         test.reset_result()  # reset result and continue to next param

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/stress_test/stress_test.json

@@ -12,5 +12,5 @@
     "multiBridgeTest": 0,
     "gwOnlyTest": 1,
     "dataSimOnlyTest": 1,
-    "allSupportedValues": [20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 222, 250, 285, 333]
+    "allSupportedValues": []
 }

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/stress_test/stress_test.py

@@ -50,14 +50,13 @@ def run(test):
         return cert_common.test_epilog(test)
 
     cert_config.gw_action(test, f'{cert_gw_sim.BLE_SIM_INIT} 1', TESTER)
-
+    ppses = [20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 222, 250, 285, 333]
     test_pkts_received = []
     data_pkts_received = []
     results = []
-    for param in test.params:
-        phase_run_print(f"PPS = {param}")
+    for pps in ppses:
+        phase_run_print(f"PPS = {pps}")
         test.flush_all_mqtt_packets()
-        pps = param.value
         # adv_duration is increased only for lowest pps to increase packets sample for reliable results
         adv_duration = ADV_DURATION_LOWEST_PPS if pps == LOWEST_PPS else ADV_DURATION_DEFAULT
         delay = math.floor(1000 / pps)
@@ -73,17 +72,18 @@ def run(test):
                               f'{BLE_SIM_ADV_37_38_39} {delay} {BLE_SIM_RADIO_1MBPS} {adv_duration * pps}', TESTER)
         mqtt_scan_wait(test, adv_duration + upload_wait_time)
 
-        cert_mqtt.dump_pkts(test, log=param.name)
+        cert_mqtt.dump_pkts(test, log=str(pps))
         phase_pkts_received = cert_mqtt.get_all_data_pkts(test.get_mqttc_by_target(DUT), indicator=STRESS_TEST_INDICATOR)
 
-        test, percentage_received = cert_common.stress_analysis(
+        test, received_pps = cert_common.stress_analysis(
             test, pps, generated_payloads,
             [(p[ALIAS_BRIDGE_ID] + p[PAYLOAD]) for p in phase_pkts_received])
-        results.extend([pps, percentage_received])
-        cert_mqtt.generate_log_file(test, param.name)
+        results.extend([pps, received_pps])
+        cert_mqtt.generate_log_file(test, str(pps))
 
-        test.set_phase_rc(param.name, test.rc)
-        test.add_phase_reason(param.name, test.reason)
+        test.add_phase(cert_utils.Phase(pps))
+        test.set_phase_rc(str(pps), test.rc)
+        test.add_phase_reason(str(pps), test.reason)
         if test.rc == TEST_FAILED and test.exit_on_param_failure:
             break  # break the whole for loop and keep the test as failed
         test.reset_result()  # reset result and continue to next param

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/cloud_connectivity/uplink_test/uplink_test.py

@@ -130,6 +130,7 @@ def run(test):
 
         mqtt_scan_wait(test, 10 + test.dut.upload_wait_time)
         cert_mqtt.dump_pkts(test, log=param.name)
+        pixel_sim_thread.stop()
         all_data_messages_in_test.extend(dut_mqttc._userdata[PKTS].data)
         all_data_pkts.extend(cert_mqtt.get_all_data_pkts(dut_mqttc))
         recieved_pkts = cert_mqtt.get_all_data_pkts(dut_mqttc, indicator=UPLINK_TEST_INDICATOR)

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/datapath/num_of_tags_test/num_of_tags_test.json

@@ -2,7 +2,7 @@
     "name": "Supported Pixels",
     "module": "Datapath",
     "purpose": "Test pixels handling in the bridge for different amount of pixels",
-    "documentation": "<TEST_DOCUMENTATION_LINK>",
+    "documentation": "",
     "initialCondition": "Bridge & data simulator configured to defaults",
     "procedure": ["Test prolog",
                   "Generate packets from the given amount of pixels & scan for packets in the bridge",

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/certificate/tests/datapath/rssi_threshold_test/rssi_threshold_test.json

@@ -2,7 +2,7 @@
     "name": "RSSI Threshold",
     "module": "Datapath",
     "purpose": "Test RSSI threshold configuration and functionality in the bridge",
-    "documentation": "<TEST_DOCUMENTATION_LINK>",
+    "documentation": "",
     "initialCondition": "Bridge configured to defaults",
     "procedure": ["Test prolog",
                   "RSSI Threshold configuration",

{wiliot_certificate-4.5.0a1 → wiliot_certificate-4.5.0a2}/src/common/api_if/api_validation.py

@@ -23,7 +23,7 @@ def api_validation(test):
         utPrint("API validation is skipped for protobuf messages", "WARNING")
         api_validation_phase.rc = TEST_SKIPPED
         api_validation_phase.reason = "API validation is skipped for protobuf messages"
-        test.phases[-1:-1] = [api_validation_phase]
+        test.add_phase(api_validation_phase)
         return test
     else:
         def validate_message(message: dict, msg_type=MESSAGE_TYPES.DATA.value) -> tuple[bool, str]:
@@ -81,5 +81,5 @@ def api_validation(test):
             api_validation_phase.rc = TEST_FAILED
             api_validation_phase.reason = "API (JSON strcture) is invalid"
         field_functionality_pass_fail_print(api_validation_phase, API_VALIDATION)
-        test.phases[-1:-1] = [api_validation_phase]
+        test.add_phase(api_validation_phase)
         return test