From 4cf45d797499817b5f639bd6245c3f84e762de14 Mon Sep 17 00:00:00 2001 From: garciadeblas Date: Thu, 8 Apr 2021 13:52:22 +0200 Subject: [PATCH] Update tags in all testsuites All testsuites have been updated with the tags according to what was agreed in OSM-MR10 and subsequent DevOps calls: - A tag per testsuite using its mnemonic - Cluster tag for each of the statistically similar tests: - cluster_main - cluster_ee_config - cluster_relations - cluster_epa - cluster_k8s - cluster_k8s_charms - cluster_sa - cluster_slices - daily: for all tests that will run in the daily job - regression: for all tests that should pass in the current stable (same as daily for this release) - sanity: for all tests that should be passed by each commit in the stage3 to be successfully verified by Jenkins The keywork "Force Tags" has been used to force the tags in all the tests of a testsuite. In addition, the tag "cleanup" was kept to those tests doing any deletion. Change-Id: I3dcb5299a0d0984204ef60cc0450d1b330e6a3d2 Signed-off-by: garciadeblas --- README.md | 107 ++++++++++++------ robot-systest/run_test.sh | 2 +- ...ic_01-crud_operations_on_vim_targets.robot | 8 +- ...stantiation_parameters_in_cloud_init.robot | 13 +-- .../testsuite/basic_06-vnf_with_charm.robot | 16 +-- .../basic_07-secure_key_management.robot | 2 + ...-disable_port_security_network_level.robot | 15 +-- .../basic_09-manual_vdu_scaling.robot | 17 +-- .../testsuite/basic_11-native_charms.robot | 14 +-- .../testsuite/basic_12-ns_primitives.robot | 18 ++- .../testsuite/basic_13-ns_relations.robot | 14 +-- .../testsuite/basic_14-vnf_relations.robot | 11 +- .../basic_15-rbac_configurations.robot | 10 +- ...c_16-advanced_onboarding_and_scaling.robot | 17 +-- .../basic_17-delete_vnf_package.robot | 11 +- .../testsuite/epa_01-epa_sriov.robot | 13 +-- .../epa_02-additional_capabilities.robot | 14 +-- .../epa_03-crud_operations_on_sdnc.robot | 8 +- .../testsuite/epa_04-epa_underlay_sriov.robot | 15 +-- .../epa_05-epa_underlay_passthrough.robot | 15 +-- robot-systest/testsuite/hackfest_basic.robot | 16 ++- .../testsuite/hackfest_cloudinit.robot | 16 ++- .../testsuite/hackfest_multivdu.robot | 16 ++- .../testsuite/k8s_01-create_k8s_cluster.robot | 20 ++-- .../k8s_02-k8scluster_creation.robot | 8 +- .../testsuite/k8s_03-simple_k8s.robot | 19 ++-- .../testsuite/k8s_04-openldap_helm.robot | 21 ++-- .../testsuite/k8s_05-k8s_proxy_charms.robot | 16 +-- .../k8s_06-k8s_secure_key_management.robot | 16 +-- .../testsuite/k8s_07-dummy_helm.robot | 19 ++-- .../quotas_01-quota_enforcement.robot | 21 ++-- .../sa_01-vnf_with_vim_metrics.robot | 12 +- ...vnf_with_vim_metrics_and_autoscaling.robot | 17 +-- .../sa_07-alarms_from_sa-related_vnfs.robot | 24 ++-- .../sa_08-vnf_with_vnf_indicators_snmp.robot | 12 +- .../testsuite/slice_01-network_slicing.robot | 32 ++---- .../slice_02-shared_network_slicing.robot | 44 ++----- 37 files changed, 281 insertions(+), 388 deletions(-) diff --git a/README.md b/README.md index 065b878..2556456 100644 --- a/README.md +++ b/README.md @@ -32,52 +32,52 @@ This repository contains tools and configuration files for testing and automatio This bash script can be used to setup your environment to execute the tests. ```bash - PACKAGES_FOLDER=osm-packages - add-apt-repository -y ppa:rmescandon/yq && apt update && apt install yq git iputils-ping ssh -y - pip install haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary \ - robotframework-sshlibrary - snap install charm - # Download community packages - git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER} +PACKAGES_FOLDER=osm-packages +add-apt-repository -y ppa:rmescandon/yq && apt update && apt install yq git iputils-ping ssh -y +pip install haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary \ + robotframework-sshlibrary +snap install charm +# Download community packages +git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER} ``` envfile.rc ```bash - # VIM Setup - OS_USERNAME= - OS_PASSWORD= - OS_TENANT_NAME= - OS_AUTH_URL= - OS_TENANT_ID= +# VIM Setup +OS_USERNAME= +OS_PASSWORD= +OS_TENANT_NAME= +OS_AUTH_URL= +OS_TENANT_ID= - # OSM Setup - OSM_HOSTNAME= - VIM_TARGET= - VIM_MGMT_NET= +# OSM Setup +OSM_HOSTNAME= +VIM_TARGET= +VIM_MGMT_NET= - # Clouds file datacenter - OS_CLOUD= - # SDNCs file - OS_SDNC= +# Clouds file datacenter +OS_CLOUD= +# SDNCs file +OS_SDNC= - # K8S config file - K8S_CREDENTIALS= +# K8S config file +K8S_CREDENTIALS= - # The following set of environment variables will be used in host - # of the robot framework. Not needed for docker execution +# The following set of environment variables will be used in host +# of the robot framework. Not needed for docker execution - # Folder where Robot tests are stored - ROBOT_DEVOPS_FOLDER=robot-systest +# Folder where Robot tests are stored +ROBOT_DEVOPS_FOLDER=robot-systest - # Folder to save alternative DUT environments (optional) - ENVIRONMENTS_FOLDER=environments +# Folder to save alternative DUT environments (optional) +ENVIRONMENTS_FOLDER=environments - # Folder where all required packages are stored - PACKAGES_FOLDER=osm-packages +# Folder where all required packages are stored +PACKAGES_FOLDER=osm-packages - # Folder where test results should be exported - ROBOT_REPORT_FOLDER=results +# Folder where test results should be exported +ROBOT_REPORT_FOLDER=results ``` ## Deployment @@ -105,7 +105,7 @@ Volumes: * [OPTIONAL]: It is the kubeconfig file to be used for k8s clusters ```bash - docker run --rm=true -t osmtests --env-file \ +docker run --rm=true -t osmtests --env-file \ -v :/reports osmtests \ -v :/robot-systest/clouds.yaml \ -v :/robot-systest/sdncs.yaml \ @@ -120,10 +120,45 @@ Volumes: The way of executing the tests is via the following command: ```bash - source envfile.rc - robot -d reports -i testsuite/ +source envfile.rc +robot -d reports -i testsuite/ ``` +## Test tags + +All tests in the testsuites have tags. Tags allow to run only a set of tests +identified by a tag. Several tags can be specified when running robot in the +following way: + +```bash +robot -i -i testsuite/ +``` + +The following tags exist for each testsuite: + +- A tag per testsuite using its mnemonic (e.g. `basic_01`) +- Cluster tag for each of the statistically similar tests: + - `cluster_main`: `basic_01`, `basic_05`, `basic_08`, `basic_09`, `basic_15`, + `basic_16`, basic_17`, `hackfest_basic`, `hackfest_multivdu`, + `hackfest_cloudinit`, `quotas_01` + - `cluster_ee_config`: `basic_06`, `basic_07`, `basic_11`, `basic_12`, + `basic_13`, `basic_14`, `k8s_05`, `k8s_06` + - `cluster_relations`: `basic_11`, `basic_13`, `basic_14` + - `cluster_epa`: `epa_01`, `epa_02`, `epa_03`, `epa_04`, `epa_05` + - `cluster_k8s`: `k8s_01`, `k8s_02`, `k8s_03`, `k8s_04`, `k8s_07`, `sa_08` + - `cluster_k8s_charms`: `k8s_05`, `k8s_06` + - `cluster_sa`: `sa_01`, `sa_02`, `sa_07` + - `cluster_slices`: `slice_01`, `slice_02` +- daily: for all testsuites that will run in the daily job +- regression: for all testsuites that should pass in the current stable branch +- sanity: for all testsuites that should be passed by each commit in the + stage3 to be successfully verified by Jenkins, currently `basic_07`, + `basic_11`, `k8s_03`, `k8s_04`, `sa_02` + +In addition, the tag "cleanup" exists in those tests that perform +any deletion. In that way, it can be invoked to retry the deletion if +the tests were forcefully stopped. + ## Built With * [Python](www.python.org/) - The language used diff --git a/robot-systest/run_test.sh b/robot-systest/run_test.sh index 1bd4488..46eaee5 100755 --- a/robot-systest/run_test.sh +++ b/robot-systest/run_test.sh @@ -115,7 +115,7 @@ Options: --env-file: It is the environmental file where is described the OSM target and VIM -o [OPTIONAL]: It is used to specify a particular osmclient version. Default: latest -p [OPTIONAL]: OSM packages repository branch. Default: master - -t [OPTIONAL]: Robot tests tags. [sanity, regression, particular_test]. Default: sanity + -t [OPTIONAL]: Robot tests tags. [sanity, daily, regression, particular_test]. Default: sanity -T [OPTIONAL]: Run SOL005 Robot conformance tests -c To create a VIM for the tests diff --git a/robot-systest/testsuite/basic_01-crud_operations_on_vim_targets.robot b/robot-systest/testsuite/basic_01-crud_operations_on_vim_targets.robot index a05359d..a47d61b 100644 --- a/robot-systest/testsuite/basic_01-crud_operations_on_vim_targets.robot +++ b/robot-systest/testsuite/basic_01-crud_operations_on_vim_targets.robot @@ -18,12 +18,13 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/vim_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_01-crud_operations_on_vim_targets_data.py +Force Tags basic_01 cluster_main daily regression + *** Test Cases *** Create VIM Target Basic [Documentation] Create a VIM Target only with the mandatory parameters. ... Checks the status of the VIM in Prometheus after it creation. - [Tags] vim sanity regression ${rand}= Generate Random String 8 [NUMBERS] ${vim_name}= Catenate SEPARATOR=_ ${vim_name_prefix} ${rand} @@ -36,7 +37,7 @@ Create VIM Target Basic Delete VIM Target By Name [Documentation] Delete the VIM Target created in previous test-case by its name. ... Checks whether the VIM Target was created or not before perform the deletion. - [Tags] vim sanity regression cleanup + [Tags] cleanup ${vim_account_id}= Get VIM Target ID ${vim_name} Should Be Equal As Strings ${vim_account_id} ${created_vim_account_id} @@ -46,7 +47,6 @@ Delete VIM Target By Name Create VIM Target With Extra Config [Documentation] Create a VIM Target using the extra parameter 'config'. ... Checks the status of the VIM in Prometheus after it creation. - [Tags] vim sanity regression ${rand}= Generate Random String 8 [NUMBERS] ${vim_name}= Catenate SEPARATOR=_ ${vim_name_prefix} ${rand} @@ -59,7 +59,7 @@ Create VIM Target With Extra Config Delete VIM Target By ID [Documentation] Delete the VIM Target created in previous test-case by its ID. ... Checks whether the VIM Target was created or not before perform the deletion. - [Tags] vim sanity regression cleanup + [Tags] cleanup ${vim_account_id}= Get VIM Target ID ${vim_name} Should Be Equal As Strings ${vim_account_id} ${created_vim_account_id} diff --git a/robot-systest/testsuite/basic_05-instantiation_parameters_in_cloud_init.robot b/robot-systest/testsuite/basic_05-instantiation_parameters_in_cloud_init.robot index 62a4ca8..b0d66d8 100644 --- a/robot-systest/testsuite/basic_05-instantiation_parameters_in_cloud_init.robot +++ b/robot-systest/testsuite/basic_05-instantiation_parameters_in_cloud_init.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_05-instantiation_parameters_in_cloud_init_data.py +Force Tags basic_05 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -35,27 +37,23 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ], add *** Test Cases *** Create Cloudinit VNF Descriptor - [Tags] instantiation_params sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Cloudinit NS Descriptor - [Tags] instantiation_params sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Cloudinit Network Service Using Instantiation Parameters [Documentation] Instantiates the NS using the instantiation parameter 'additionalParamsForVnf' to change the password of the default user. - [Tags] instantiation_params sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${EMPTY} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] instantiation_params sanity regression ${ip_addr} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} @@ -64,7 +62,6 @@ Get Management Ip Addresses Test SSH Access With The New Password [Documentation] Test SSH access with the new password configured via cloud-init. - [Tags] instantiation_params sanity regression Variable Should Exist ${vnf_ip_addr} msg=IP address of the management VNF is not available Sleep 30s Waiting ssh daemon to be up @@ -72,19 +69,19 @@ Test SSH Access With The New Password Delete NS Instance - [Tags] instantiation_params sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] instantiation_params sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] instantiation_params sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_06-vnf_with_charm.robot b/robot-systest/testsuite/basic_06-vnf_with_charm.robot index ae95791..e455e08 100644 --- a/robot-systest/testsuite/basic_06-vnf_with_charm.robot +++ b/robot-systest/testsuite/basic_06-vnf_with_charm.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_06-vnf_with_charm_data.py +Force Tags basic_06 cluster_ee_config daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,26 +43,22 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - [Tags] proxy_charm charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - [Tags] proxy_charm charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] proxy_charm charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] proxy_charm charm sanity regression ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} @@ -71,7 +69,6 @@ Get Management Ip Addresses Test SSH Access - [Tags] proxy_charm charm sanity regression Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available @@ -83,7 +80,6 @@ Test SSH Access Check Remote Files Created Via Day 1 Operations [Documentation] The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}. ... This test checks whether that files have been created or not. - [Tags] proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} @@ -91,7 +87,6 @@ Check Remote Files Created Via Day 1 Operations Execute Day 2 Operations [Documentation] Performs one Day 2 operation per VNF that creates a new file. - [Tags] proxy_charm charm sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id_1}= Execute NS Action ${ns_name} ${action_name} ${vnf_member_index_1} filename=${day_2_file_name_1} @@ -100,26 +95,25 @@ Execute Day 2 Operations Check Remote Files Created Via Day 2 Operations [Documentation] Check whether the files created in the previous test via Day 2 operations exist or not. - [Tags] proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_1} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_2} Delete NS Instance - [Tags] proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_07-secure_key_management.robot b/robot-systest/testsuite/basic_07-secure_key_management.robot index 23698d7..9ef5da3 100644 --- a/robot-systest/testsuite/basic_07-secure_key_management.robot +++ b/robot-systest/testsuite/basic_07-secure_key_management.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_07-secure_key_management_data.py +Force Tags basic_07 cluster_ee_config daily regression sanity + Suite Teardown Run Keyword And Ignore Error Suite Cleanup diff --git a/robot-systest/testsuite/basic_08-disable_port_security_network_level.robot b/robot-systest/testsuite/basic_08-disable_port_security_network_level.robot index 7e65c9e..12c4931 100644 --- a/robot-systest/testsuite/basic_08-disable_port_security_network_level.robot +++ b/robot-systest/testsuite/basic_08-disable_port_security_network_level.robot @@ -25,6 +25,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/prometheus_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_08-disable_port_security_network_level_data.py +Force Tags basic_08 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -34,57 +36,52 @@ ${port_disabled_msg} port_security_enabled: false *** Test Cases *** Create VIM With Port Security Disabled - [Tags] disable_port_security sanity regression ${created_vim_account_id}= Create VIM Target ${vim_name} ${vim_user} ${vim_password} ${vim_auth_url} ${vim_tenant} ${vim_account_type} config=${vim_config} Check for VIM Target Status ${vim_name} ${prometheus_host} ${prometheus_port} Create VNF Descriptor - [Tags] disable_port_security sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] disable_port_security sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] disable_port_security sanity regression ${id}= Create Network Service ${nsd_name} ${vim_name} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Check Port Security Is Disabled - [Tags] disable_port_security sanity regression ${rc} ${disabled_ports}= Run and Return RC and Output osm ns-show ${ns_name} --literal | grep -c '${port_disabled_msg}' Run Keyword Unless ${disabled_ports} > 0 Fail msg=Found '${disabled_ports}' matches for '${port_disabled_msg}' Delete NS Instance - [Tags] disable_port_security sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] disable_port_security sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] disable_port_security sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} Delete VIM - [Tags] disable_port_security sanity regression cleanup + [Tags] cleanup Delete VIM Target ${vim_name} diff --git a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot index e695768..9b0138b 100644 --- a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot +++ b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot @@ -24,6 +24,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_09-manual_vdu_scaling_data.py +Force Tags basic_09 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -35,26 +37,22 @@ ${vnf_member_index} 1 *** Test Cases *** Create Scaling VNF Descriptor - [Tags] manual_scaling sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Scaling NS Descriptor - [Tags] manual_scaling sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Scaling Network Service - [Tags] manual_scaling sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Vnf Id - [Tags] manual_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} @@ -64,7 +62,6 @@ Get Vnf Id Get Vdus Before Scale Out [Documentation] Get the number of VDU records before the manual scaling. - [Tags] manual_scaling sanity regression @{vdur_list}= Get Vnf Vdur Names ${vnf_id} Log List ${vdur_list} @@ -73,7 +70,6 @@ Get Vdus Before Scale Out Perform Manual Vdu Scale Out - [Tags] manual_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_OUT @@ -81,7 +77,6 @@ Perform Manual Vdu Scale Out Check Vdus After Scale Out [Documentation] Check whether there is one more VDU after scaling or not. - [Tags] manual_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -91,7 +86,6 @@ Check Vdus After Scale Out Perform Manual Vdu Scale In - [Tags] manual_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_IN @@ -99,7 +93,6 @@ Perform Manual Vdu Scale In Check Vdus After Scaling In [Documentation] Check whether there is one less VDU after scaling or not. - [Tags] manual_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -109,19 +102,19 @@ Check Vdus After Scaling In Delete NS Instance - [Tags] manual_scaling sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] manual_scaling sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] manual_scaling sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_11-native_charms.robot b/robot-systest/testsuite/basic_11-native_charms.robot index 87fb8fb..89f5e63 100644 --- a/robot-systest/testsuite/basic_11-native_charms.robot +++ b/robot-systest/testsuite/basic_11-native_charms.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_11-native_charms.py +Force Tags basic_11 cluster_ee_config cluster_relations daily regression sanity + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,26 +43,22 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - [Tags] native_charms charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - [Tags] native_charms charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] native_charms charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] native_charms charm sanity regression ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} @@ -71,7 +69,6 @@ Get Management Ip Addresses Test SSH Access - [Tags] native_charms charm sanity regression Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available @@ -83,26 +80,25 @@ Test SSH Access Check Remote Files Created Via Day 1 Operations [Documentation] The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}. ... This test checks whether that files have been created or not. - [Tags] native_charms charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} Delete NS Instance - [Tags] native_charms charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] native_charms charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] native_charms charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_12-ns_primitives.robot b/robot-systest/testsuite/basic_12-ns_primitives.robot index f2df252..e4113e7 100644 --- a/robot-systest/testsuite/basic_12-ns_primitives.robot +++ b/robot-systest/testsuite/basic_12-ns_primitives.robot @@ -12,6 +12,8 @@ # limitations under the License. *** Settings *** +Documentation [BASIC-12] NS Primitives + Library OperatingSystem Library String Library Collections @@ -24,6 +26,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/packages_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_12-ns_primitives_data.py +Force Tags basic_12 cluster_ee_config daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -36,29 +40,21 @@ Change Juju Password [Documentation] NS package needs to be updated with the Juju credentials for your OSM installation - [Tags] nsprimitives charm sanity regression - ${nsd_yaml}= Get File %{PACKAGES_FOLDER}/${nsd_pkg}/${nsd_file} ${changed_nsd_yaml}= Replace String ${nsd_yaml} ${old_juju_password} %{JUJU_PASSWORD} Create File %{PACKAGES_FOLDER}/${nsd_pkg}/${nsd_file} ${changed_nsd_yaml} Upload Vnfds - [Tags] nsprimitives charm sanity regression - Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg1}' Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg2}' Upload Nsd - [Tags] nsprimitives charm sanity regression - Create NSD %{PACKAGES_FOLDER}/${nsd_pkg}/ Instantiate NS - [Tags] nsprimitives charm sanity regression - ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ns_launch_max_wait_time=40min Set Suite Variable ${ns_id} ${id} @@ -66,19 +62,19 @@ Instantiate NS Delete NS - [Tags] nsprimitives charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] nsprimitives charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptors - [Tags] nsprimitives charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name1} Delete VNFD ${vnfd_name2} diff --git a/robot-systest/testsuite/basic_13-ns_relations.robot b/robot-systest/testsuite/basic_13-ns_relations.robot index 6cfba56..50f0851 100644 --- a/robot-systest/testsuite/basic_13-ns_relations.robot +++ b/robot-systest/testsuite/basic_13-ns_relations.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_13-ns_relations.py +Force Tags basic_13 cluster_ee_config cluster_relations daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,25 +43,21 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor Provides - [Tags] ns_relations charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg1}' Create Charm VNF Descriptor Requires - [Tags] ns_relations charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg2}' Create Charm NS Descriptor - [Tags] ns_relations charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] ns_relations charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} @@ -69,25 +67,25 @@ Instantiate Charm Network Service Delete NS Instance - [Tags] ns_relations charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] ns_relations charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Provides - [Tags] ns_relations charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name1} Delete VNF Descriptor Requires - [Tags] ns_relations charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name2} diff --git a/robot-systest/testsuite/basic_14-vnf_relations.robot b/robot-systest/testsuite/basic_14-vnf_relations.robot index d56fe2c..035612b 100644 --- a/robot-systest/testsuite/basic_14-vnf_relations.robot +++ b/robot-systest/testsuite/basic_14-vnf_relations.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_14-vnf_relations.py +Force Tags basic_14 cluster_ee_config cluster_relations daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,19 +43,16 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - [Tags] vnf_relations charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - [Tags] vnf_relations charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] vnf_relations charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} @@ -63,19 +62,19 @@ Instantiate Charm Network Service Delete NS Instance - [Tags] vnf_relations charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] vnf_relations charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Provides - [Tags] vnf_relations charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_15-rbac_configurations.robot b/robot-systest/testsuite/basic_15-rbac_configurations.robot index e47871e..55e0113 100644 --- a/robot-systest/testsuite/basic_15-rbac_configurations.robot +++ b/robot-systest/testsuite/basic_15-rbac_configurations.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/role_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_15-rbac_configurations_data.py +Force Tags basic_15 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -31,14 +33,12 @@ ${success_return_code} 0 *** Test Cases *** Create And Validate User - [Tags] rbac_configurations sanity regression Create User ${user_name} ${user_password} Check If User Exists ${user_name} Assign Role To User - [Tags] rbac_configurations sanity regression Update User Role ${user_name} ${user_project} ${user_role} Check If User Is Assigned To Project ${user_name} ${user_project} @@ -46,35 +46,31 @@ Assign Role To User Run Action As User - [Tags] rbac_configurations sanity regression ${rc} ${stdout}= Run And Return RC And Output OSM_USER=${user_name} OSM_PROJECT=${user_project} OSM_PASSWORD=${user_password} osm ns-list Should Be Equal As Integers ${rc} ${success_return_code} Create And Update Project - [Tags] rbac_configurations sanity regression Create Project ${project_name} Update Project Name ${project_name} ${new_project_name} Create And Validate Role - [Tags] rbac_configurations sanity regression Create Role ${role_name} Check If Role Exists ${role_name} Update Role Information - [Tags] rbac_configurations sanity regression Update Role ${role_name} add='vims: true' Check If Role Exists ${role_name} Delete Allocated Resources - [Tags] rbac_configurations sanity regression cleanup + [Tags] cleanup Delete User ${user_name} Delete Project ${new_project_name} diff --git a/robot-systest/testsuite/basic_16-advanced_onboarding_and_scaling.robot b/robot-systest/testsuite/basic_16-advanced_onboarding_and_scaling.robot index 78a1aa3..04e9b31 100644 --- a/robot-systest/testsuite/basic_16-advanced_onboarding_and_scaling.robot +++ b/robot-systest/testsuite/basic_16-advanced_onboarding_and_scaling.robot @@ -24,6 +24,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_16-advanced_onboarding_and_scaling_data.py +Force Tags basic_16 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -35,26 +37,22 @@ ${vnf_member_index} 1 *** Test Cases *** Create Scaling VNF Descriptor - [Tags] advanced_scaling sanity regression Create VNFD Overriding Fields '%{PACKAGES_FOLDER}/${vnfd_pkg}' df.0.instantiation-level.0.vdu-level.0.number-of-instances=${init_instances};df.0.vdu-profile.0.min-number-of-instances=${min_instances};df.0.vdu-profile.0.max-number-of-instances=${max_instances};df.0.scaling-aspect.0.aspect-delta-details.deltas.0.vdu-delta.0.number-of-instances=${delta_instances} Create Scaling NS Descriptor - [Tags] advanced_scaling sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Scaling Network Service - [Tags] advanced_scaling sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Vnf Id - [Tags] advanced_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} @@ -64,7 +62,6 @@ Get Vnf Id Check Vdus Before Scale Out [Documentation] Check the number of VDUs instances before the manual scaling. - [Tags] advanced_scaling sanity regression @{vdur_list}= Get Vnf Vdur Names ${vnf_id} Log List ${vdur_list} @@ -74,7 +71,6 @@ Check Vdus Before Scale Out Perform Manual Vdu Scale Out - [Tags] advanced_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_OUT @@ -82,7 +78,6 @@ Perform Manual Vdu Scale Out Check Vdus After Scale Out [Documentation] Check whether there is one more VDU after scaling or not. - [Tags] advanced_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -92,7 +87,6 @@ Check Vdus After Scale Out Perform Manual Vdu Scale In - [Tags] advanced_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_IN @@ -100,7 +94,6 @@ Perform Manual Vdu Scale In Check Vdus After Scaling In [Documentation] Check whether there is one less VDU after scaling or not. - [Tags] advanced_scaling sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -110,19 +103,19 @@ Check Vdus After Scaling In Delete NS Instance - [Tags] advanced_scaling sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] advanced_scaling sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] advanced_scaling sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/basic_17-delete_vnf_package.robot b/robot-systest/testsuite/basic_17-delete_vnf_package.robot index 0cd13f8..d61e682 100644 --- a/robot-systest/testsuite/basic_17-delete_vnf_package.robot +++ b/robot-systest/testsuite/basic_17-delete_vnf_package.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [BASIC-17] Delete VNF Package Before NS Package. + Library OperatingSystem Library String Library Collections @@ -22,36 +24,35 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/nsd_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/basic_17-delete_vnf_package_data.py +Force Tags basic_17 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup *** Test Cases *** Create VNF Package - [Tags] hackfest_basic sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Package - [Tags] hackfest_basic sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Cannot Delete VNF Package - [Tags] hackfest_basic sanity regression cleanup Assert Failure Delete VNFD ${vnfd_name} Delete NS Package - [Tags] hackfest_basic sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Package - [Tags] hackfest_basic sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/epa_01-epa_sriov.robot b/robot-systest/testsuite/epa_01-epa_sriov.robot index d38be13..dec84fa 100644 --- a/robot-systest/testsuite/epa_01-epa_sriov.robot +++ b/robot-systest/testsuite/epa_01-epa_sriov.robot @@ -25,6 +25,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/epa_01-epa_sriov_data.py +Force Tags epa_01 cluster_epa daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -39,26 +41,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create VNF Descriptor - [Tags] epa_sriov sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] epa_sriov sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] epa_sriov sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get VNF IP Address - [Tags] epa_sriov sanity regression ${ip_addr}= Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} @@ -66,25 +64,24 @@ Get VNF IP Address Check SR-IOV Interface - [Tags] epa_sriov sanity regression Sleep 30 seconds Waiting for SSH daemon to be up Execute Remote Command Check Rc Return Output ${vnf_ip_addr} ${username} ${password} ${privatekey} lspci | grep "Ethernet controller" | grep -v "Virtio" Delete NS Instance - [Tags] epa_sriov sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] epa_sriov sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] epa_sriov sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/epa_02-additional_capabilities.robot b/robot-systest/testsuite/epa_02-additional_capabilities.robot index c9db674..91b3500 100644 --- a/robot-systest/testsuite/epa_02-additional_capabilities.robot +++ b/robot-systest/testsuite/epa_02-additional_capabilities.robot @@ -26,6 +26,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/openstack_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/epa_02-additional_capabilities_data.py +Force Tags epa_02 cluster_epa daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -39,27 +41,23 @@ ${success_return_code} 0 *** Test Cases *** Create VNF Descriptor - [Tags] epa_capabilities sanity regression Remove Environment Variable OVERRIDES Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] epa_capabilities sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] epa_capabilities sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get VNF Server ID - [Tags] epa_capabilities sanity regression ${vnfs_list}= Get Ns Vnf List ${ns_id} ${vim_id}= Get VNF VIM ID ${vnfs_list}[0] @@ -68,7 +66,6 @@ Get VNF Server ID Get Server Flavor Properties - [Tags] epa_capabilities sanity regression ${flavor_id}= Get Server Flavor ID ${vnf_server_id} ${properties}= Get Flavor Properties ${flavor_id} @@ -77,7 +74,6 @@ Get Server Flavor Properties Check Flavor Quota Properties - [Tags] epa_capabilities sanity regression ${rc}= Run and Return RC echo ${flavor_properties} | grep "cpu_shares_level=custom" Should Be Equal As Integers ${rc} ${success_return_code} @@ -88,19 +84,19 @@ Check Flavor Quota Properties Delete NS Instance - [Tags] epa_capabilities sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] epa_capabilities sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] epa_capabilities sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/epa_03-crud_operations_on_sdnc.robot b/robot-systest/testsuite/epa_03-crud_operations_on_sdnc.robot index 335d619..4b222e4 100644 --- a/robot-systest/testsuite/epa_03-crud_operations_on_sdnc.robot +++ b/robot-systest/testsuite/epa_03-crud_operations_on_sdnc.robot @@ -17,24 +17,24 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/sdnc_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/epa_03-crud_operations_on_sdnc_data.py +Force Tags epa_03 cluster_epa daily regression + Suite Teardown Run Keyword And Ignore Error Delete Basic SDNC *** Test Cases *** Create Basic SDNC - [Tags] sdnc_crud sanity regression ${created_sdnc_id}= Create SDNC ${sdnc_name} ${sdnc_user} ${sdnc_password} ${sdnc_url} ${sdnc_type} Check for SDNC ${sdnc_name} - Check SDNC Status Is Healthy - [Tags] sdnc_crud sanity regression +Check SDNC Status Is Healthy Check for SDNC Status ${sdnc_name} ${prometheus_host} ${prometheus_port} Delete Basic SDNC - [Tags] sdnc_crud sanity regression cleanup + [Tags] cleanup Delete SDNC ${sdnc_name} diff --git a/robot-systest/testsuite/epa_04-epa_underlay_sriov.robot b/robot-systest/testsuite/epa_04-epa_underlay_sriov.robot index f916735..3efeb03 100644 --- a/robot-systest/testsuite/epa_04-epa_underlay_sriov.robot +++ b/robot-systest/testsuite/epa_04-epa_underlay_sriov.robot @@ -27,8 +27,11 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/openstack_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/epa_04-epa_underlay_sriov.py +Force Tags epa_04 cluster_epa daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup + *** Variables *** ${vnf_member_index_1} 1 ${vnf_member_index_2} 2 @@ -40,27 +43,23 @@ ${success_return_code} 0 *** Test Cases *** Create VNF Descriptor - [Tags] epa_underlay_sriov sanity regression Remove Environment Variable OVERRIDES Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] epa_underlay_sriov sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] epa_underlay_sriov sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] epa_underlay_sriov sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ip} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} @@ -73,7 +72,6 @@ Get Management Ip Addresses Ping from Vnf1 to Vnf2 - [Tags] epa_underlay_sriov sanity regression Variable Should Exist ${ip_mgmt_1} msg=IP address of the data VNF '${vnf_member_index_1}' is not available Variable Should Exist ${privatekey} msg=SSH private key not available @@ -83,7 +81,6 @@ Ping from Vnf1 to Vnf2 Ping from Vnf2 to Vnf1 - [Tags] epa_underlay_sriov sanity regression Variable Should Exist ${ip_mgmt_2} msg=IP address of the data VNF '${vnf_member_index_2}' is not available Variable Should Exist ${privatekey} msg=SSH private key not available @@ -92,19 +89,19 @@ Ping from Vnf2 to Vnf1 Delete NS Instance - [Tags] epa_underlay_sriov sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] epa_underlay_sriov sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] epa_underlay_sriov sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/epa_05-epa_underlay_passthrough.robot b/robot-systest/testsuite/epa_05-epa_underlay_passthrough.robot index 5c96d3d..7a579bf 100644 --- a/robot-systest/testsuite/epa_05-epa_underlay_passthrough.robot +++ b/robot-systest/testsuite/epa_05-epa_underlay_passthrough.robot @@ -27,8 +27,11 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/openstack_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/epa_05-epa_underlay_passthrough.py +Force Tags epa_05 cluster_epa daily regresssion + Suite Teardown Run Keyword And Ignore Error Suite Cleanup + *** Variables *** ${vnf_member_index_1} 1 ${vnf_member_index_2} 2 @@ -40,27 +43,23 @@ ${success_return_code} 0 *** Test Cases *** Create VNF Descriptor - [Tags] epa_underlay_passthrough sanity regression Remove Environment Variable OVERRIDES Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] epa_underlay_passthrough sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] epa_underlay_passthrough sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] epa_underlay_passthrough sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ip} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} @@ -73,7 +72,6 @@ Get Management Ip Addresses Ping from Vnf1 to Vnf2 - [Tags] epa_underlay_passthrough sanity regression Variable Should Exist ${ip_mgmt_1} msg=IP address of the data VNF '${vnf_member_index_1}' is not available Variable Should Exist ${privatekey} msg=SSH private key not available @@ -83,7 +81,6 @@ Ping from Vnf1 to Vnf2 Ping from Vnf2 to Vnf1 - [Tags] epa_underlay_passthrough sanity regression Variable Should Exist ${ip_mgmt_2} msg=IP address of the data VNF '${vnf_member_index_2}' is not available Variable Should Exist ${privatekey} msg=SSH private key not available @@ -92,19 +89,19 @@ Ping from Vnf2 to Vnf1 Delete NS Instance - [Tags] epa_underlay_passthrough sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] epa_underlay_passthrough sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] epa_underlay_passthrough sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/hackfest_basic.robot b/robot-systest/testsuite/hackfest_basic.robot index 95d1827..f721ef7 100644 --- a/robot-systest/testsuite/hackfest_basic.robot +++ b/robot-systest/testsuite/hackfest_basic.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [HACKFEST-BASIC] Basic NS with a single-VDU VNF + Library OperatingSystem Library String Library Collections @@ -25,6 +27,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/hackfest_basic_ns_data.py +Force Tags hackfest_basic cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -39,53 +43,47 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create Hackfest Basic VNF Descriptor - [Tags] hackfest_basic sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Hackfest Basic NS Descriptor - [Tags] hackfest_basic sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Network Service Instance Test - [Tags] hackfest_basic sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Vnf Ip Address - [Tags] hackfest_basic sanity regression ${ip_addr} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} Set Suite Variable ${vnf_ip_addr} ${ip_addr} Test Ping - [Tags] hackfest_basic sanity regression Test Connectivity ${vnf_ip_addr} Test SSH Access - [Tags] hackfest_basic sanity regression Sleep 30s Waiting ssh daemon to be up Test SSH Connection ${vnf_ip_addr} ${username} ${password} ${privatekey} Delete NS Instance Test - [Tags] hackfest_basic sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor Test - [Tags] hackfest_basic sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] hackfest_basic sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/hackfest_cloudinit.robot b/robot-systest/testsuite/hackfest_cloudinit.robot index 3ac1189..52d6e4e 100644 --- a/robot-systest/testsuite/hackfest_cloudinit.robot +++ b/robot-systest/testsuite/hackfest_cloudinit.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [HACKFEST-CLOUDINIT] Basic NS with two multi-VDU VNF with cloudinit + Library OperatingSystem Library String Library Collections @@ -25,6 +27,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/hackfest_cloudinit_ns_data.py +Force Tags hackfest_cloudinit cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -39,26 +43,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create Hackfest Cloudinit VNF Descriptor - [Tags] hackfest_cloudinit sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Hackfest Cloudinit NS Descriptor - [Tags] hackfest_cloudinit sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Network Service Instance Test - [Tags] hackfest_cloudinit sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${EMPTY} Set Suite Variable ${ns_id} ${id} Get Vnf Ip Address - [Tags] hackfest_cloudinit sanity regression ${ip_addr} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} @@ -66,33 +66,31 @@ Get Vnf Ip Address Test SSH Access - [Tags] hackfest_cloudinit sanity regression Sleep 30s Waiting ssh daemon to be up Test SSH Connection ${vnf_ip_addr} ${username} ${password} ${privatekey} Check Remote File Injected Via Cloud-init - [Tags] hackfest_cloudinit sanity regression ${stdout}= Execute Remote Command Check Rc Return Output ${vnf_ip_addr} ${username} ${password} ${privatekey} sudo cat /root/helloworld.txt log ${stdout} Delete NS Instance Test - [Tags] hackfest_cloudinit sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor Test - [Tags] hackfest_cloudinit sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] hackfest_cloudinit sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/hackfest_multivdu.robot b/robot-systest/testsuite/hackfest_multivdu.robot index c30ef94..131c5ac 100644 --- a/robot-systest/testsuite/hackfest_multivdu.robot +++ b/robot-systest/testsuite/hackfest_multivdu.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [HACKFEST-MULTIVDU] Basic NS with two multi-VDU VNF + Library OperatingSystem Library String Library Collections @@ -25,6 +27,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/hackfest_multivdu_ns_data.py +Force Tags hackfest_multivdu cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -40,18 +44,15 @@ ${wait_guard_for_vm_boot} 50s *** Test Cases *** Create Hackfest multivdu VNF Descriptor - [Tags] hackfest_multivdu sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Hackfest Multivdu NS Descriptor - [Tags] hackfest_multivdu sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Network Service Instance Test - [Tags] hackfest_multivdu sanity regression ${status} ${message}= Run Keyword And Ignore Error Variable Should Exist ${publickey} Run Keyword If "${status}" == "FAIL" Set Global Variable ${publickey} ${EMPTY} @@ -60,36 +61,33 @@ Network Service Instance Test Sleep ${wait_guard_for_vm_boot} Waiting for VM's daemons to be up and running Get Vnf Ip Address - [Tags] hackfest_multivdu sanity regression ${ip_addr} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} Set Suite Variable ${vnf_ip_addr} ${ip_addr} Test Ping - [Tags] hackfest_multivdu sanity regression Test Connectivity ${vnf_ip_addr} Test SSH Access - [Tags] hackfest_multivdu sanity regression ${status} ${message}= Run Keyword And Ignore Error Variable Should Exist ${privatekey} Run Keyword If "${status}" == "FAIL" Set Global Variable ${privatekey} ${EMPTY} Test SSH Connection ${vnf_ip_addr} ${username} ${password} ${privatekey} Delete NS Instance Test - [Tags] hackfest_multivdu sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor Test - [Tags] hackfest_multivdu sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] hackfest_multivdu sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/k8s_01-create_k8s_cluster.robot b/robot-systest/testsuite/k8s_01-create_k8s_cluster.robot index 06530be..ade1d70 100644 --- a/robot-systest/testsuite/k8s_01-create_k8s_cluster.robot +++ b/robot-systest/testsuite/k8s_01-create_k8s_cluster.robot @@ -13,6 +13,8 @@ # limitations under the License. *** Settings *** +Documentation [K8s-01] Create K8s cluster. + Library OperatingSystem Library String Library Collections @@ -27,6 +29,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_01-create_k8s_cluster_data.py +Force Tags k8s_01 cluster_k8s daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -37,34 +41,28 @@ ${password} ${EMPTY} *** Test Cases *** Render a template - [Tags] newK8sCluster regression ${stdout}= Render template %{ROBOT_DEVOPS_FOLDER}/resources/${template} %{ROBOT_DEVOPS_FOLDER}/resources/${config_file} IP_VM1=%{IP_VM1} IP_VM2=%{IP_VM2} IP_VM3=%{IP_VM3} IP_VM4=%{IP_VM4} IP_JUJU=%{IP_JUJU} NETWORK=%{VIM_MGMT_NET} Log To Console \n${stdout} Create Controller VNF Descriptor - [Tags] newK8sCluster regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg1}' Create Machines VNF Descriptor - [Tags] newK8sCluster regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg2}' Create K8s Cluster NS Descriptor - [Tags] newK8sCluster regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate K8s Cluster Network Service - [Tags] newK8sCluster regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ns_config=${EMPTY} publickey=${publickey} ns_launch_max_wait_time=70min config_file=%{ROBOT_DEVOPS_FOLDER}/resources/${config_file} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] newK8sCluster regression ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} @@ -83,7 +81,6 @@ Get Management Ip Addresses Set Suite Variable ${vnf_5_ip_addr} ${ip_addr_5} Test SSH Access - [Tags] newK8sCluster regression Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available @@ -98,29 +95,28 @@ Test SSH Access Test SSH Connection ${vnf_5_ip_addr} ${username} ${password} ${privatekey} Check kubeconfig file - [Tags] newK8sCluster regression Check If remote File Exists ${vnf_5_ip_addr} ${username} ${password} ${privatekey} ${kubeconfig_file} Delete NS Instance - [Tags] newK8sCluster regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] newK8sCluster regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete Controller VNF Descriptor - [Tags] newK8sCluster regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name1} Delete Machines VNF Descriptor - [Tags] newK8sCluster regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name2} diff --git a/robot-systest/testsuite/k8s_02-k8scluster_creation.robot b/robot-systest/testsuite/k8s_02-k8scluster_creation.robot index 0c19959..f0b5eae 100644 --- a/robot-systest/testsuite/k8s_02-k8scluster_creation.robot +++ b/robot-systest/testsuite/k8s_02-k8scluster_creation.robot @@ -13,6 +13,8 @@ # limitations under the License. *** Settings *** +Documentation [K8s-02] K8s cluster addition. + Library OperatingSystem Library String Library Collections @@ -22,15 +24,17 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/k8scluster_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_02-k8scluster_creation_data.py +Force Tags k8s_02 cluster_k8s daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup + *** Test Cases *** Add K8s Cluster To OSM - [Tags] k8scluster sanity regression Create K8s Cluster %{K8S_CREDENTIALS} ${k8scluster_version} %{VIM_TARGET} %{VIM_MGMT_NET} ${k8scluster_name} Remove K8s Cluster from OSM - [Tags] k8scluster sanity regression + [Tags] cleanup Delete K8s Cluster ${k8scluster_name} *** Keywords *** diff --git a/robot-systest/testsuite/k8s_03-simple_k8s.robot b/robot-systest/testsuite/k8s_03-simple_k8s.robot index 7d082c2..d235f7e 100644 --- a/robot-systest/testsuite/k8s_03-simple_k8s.robot +++ b/robot-systest/testsuite/k8s_03-simple_k8s.robot @@ -13,6 +13,8 @@ # limitations under the License. *** Settings *** +Documentation [K8s-03] Simple K8s. + Library OperatingSystem Library String Library Collections @@ -28,6 +30,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/k8scluster_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_03-simple_k8s_data.py +Force Tags k8s_03 cluster_k8s daily regression sanity + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -43,47 +47,42 @@ ${customtitle} Day 2 Action *** Test Cases *** Create Simple K8s VNF Descriptor - [Tags] simple_k8s charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Simple K8s Descriptor - [Tags] simple_k8s charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Add K8s Cluster To OSM - [Tags] simple_k8s charm sanity regression Create K8s Cluster %{K8S_CREDENTIALS} ${k8scluster_version} %{VIM_TARGET} %{VIM_MGMT_NET} ${k8scluster_name} Network Service K8s Instance Test - [Tags] simple_k8s charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Execute Day 2 Operations [Documentation] Performs one Day 2 operation per VNF that creates a new file. - [Tags] simple_k8s charm sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute NS K8s Action ${ns_name} ${action_name} ${vnf_member_index} ${kdu_name} application-name=${application_name} customtitle=${customtitle} Delete NS K8s Instance Test - [Tags] simple_k8s charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Remove K8s Cluster from OSM - [Tags] simple_k8s charm sanity regression + [Tags] cleanup Delete K8s Cluster ${k8scluster_name} Delete NS Descriptor Test - [Tags] simple_k8s charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] simple_k8s charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} Delete VNF NS Packages - [Tags] simple_k8s charm sanity regression cleanup + [Tags] cleanup Delete Package '%{PACKAGES_FOLDER}/${vnfd_pkg}' Delete Package '%{PACKAGES_FOLDER}/${nsd_pkg}' diff --git a/robot-systest/testsuite/k8s_04-openldap_helm.robot b/robot-systest/testsuite/k8s_04-openldap_helm.robot index 92d8605..d5bac76 100644 --- a/robot-systest/testsuite/k8s_04-openldap_helm.robot +++ b/robot-systest/testsuite/k8s_04-openldap_helm.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [K8s-04] Openldap Helm chart. + Library OperatingSystem Library String Library Collections @@ -25,6 +27,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/k8scluster_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_04-openldap_helm_data.py +Force Tags k8s_04 cluster_k8s daily regression sanity + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,67 +45,58 @@ ${replica_count} 3 *** Test Cases *** Create Package For OpenLDAP CNF - [Tags] openldap sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Package For OpenLDAP NS - [Tags] openldap sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Add K8s Cluster To OSM - [Tags] openldap sanity regression Create K8s Cluster %{K8S_CREDENTIALS} ${k8scluster_version} %{VIM_TARGET} %{VIM_MGMT_NET} ${k8scluster_name} Create Network Service Instance - [Tags] openldap sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Vnf Id - [Tags] openldap sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} Log List ${vnfr_list} Set Suite Variable ${vnf_id} ${vnfr_list}[0] Execute Upgrade Operation - [Tags] openldap sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute NS K8s Action ${ns_name} ${upgrade_action} ${vnf_member_index} ${kdu_name} replicaCount=${replica_count} Check Replicas After Upgrade Operation - [Tags] openldap sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${count}= Get Vnf Kdu Replica Count ${vnf_id} ${kdu_name} log ${count} Should Be Equal As Integers ${count} ${replica_count} Execute Rollback Operation - [Tags] openldap sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute NS K8s Action ${ns_name} ${rollback_action} ${vnf_member_index} ${kdu_name} Check Replicas After Rollback Operation - [Tags] openldap sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${count}= Get Vnf Kdu Replica Count ${vnf_id} ${kdu_name} log ${count} Should Be Empty ${count} Delete Network Service Instance - [Tags] openldap sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Remove K8s Cluster from OSM - [Tags] openldap sanity regression + [Tags] cleanup Delete K8s Cluster ${k8scluster_name} Delete NS Descriptor Test - [Tags] openldap sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] openldap sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/k8s_05-k8s_proxy_charms.robot b/robot-systest/testsuite/k8s_05-k8s_proxy_charms.robot index 8a7c350..a32ad66 100644 --- a/robot-systest/testsuite/k8s_05-k8s_proxy_charms.robot +++ b/robot-systest/testsuite/k8s_05-k8s_proxy_charms.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_05-k8s_proxy_charms.py +Force Tags k8s_05 cluster_ee_config cluster_k8s_charms daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,26 +43,22 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - [Tags] k8s_proxy_charm charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - [Tags] k8s_proxy_charm charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] k8s_proxy_charm charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] k8s_proxy_charm charm sanity regression ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} @@ -71,7 +69,6 @@ Get Management Ip Addresses Test SSH Access - [Tags] k8s_proxy_charm charm sanity regression Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available @@ -83,7 +80,6 @@ Test SSH Access Check Remote Files Created Via Day 1 Operations [Documentation] The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}. ... This test checks whether that files have been created or not. - [Tags] k8s_proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} @@ -91,7 +87,6 @@ Check Remote Files Created Via Day 1 Operations Execute Day 2 Operations [Documentation] Performs one Day 2 operation per VNF that creates a new file. - [Tags] k8s_proxy_charm charm sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id_1}= Execute NS Action ${ns_name} ${action_name} ${vnf_member_index_1} filename=${day_2_file_name_1} @@ -100,26 +95,25 @@ Execute Day 2 Operations Check Remote Files Created Via Day 2 Operations [Documentation] Check whether the files created in the previous test via Day 2 operations exist or not. - [Tags] k8s_proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_1} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_2} Delete NS Instance - [Tags] k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot index bbd9330..d12153d 100644 --- a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot +++ b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot @@ -23,6 +23,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_06-k8s_secure_key_management_data.py +Force Tags k8s_06 cluster_ee_config cluster_k8s_charms daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,26 +43,22 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - [Tags] nopasswd_k8s_proxy_charm charm sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - [Tags] nopasswd_k8s_proxy_charm charm sanity regression ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} @@ -71,7 +69,6 @@ Get Management Ip Addresses Test SSH Access - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available @@ -83,7 +80,6 @@ Test SSH Access Check Remote Files Created Via Day 1 Operations [Documentation] The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}. ... This test checks whether that files have been created or not. - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_1_file_name} @@ -91,7 +87,6 @@ Check Remote Files Created Via Day 1 Operations Execute Day 2 Operations [Documentation] Performs one Day 2 operation per VNF that creates a new file. - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id_1}= Execute NS Action ${ns_name} ${action_name} ${vnf_member_index_1} filename=${day_2_file_name_1} @@ -100,26 +95,25 @@ Execute Day 2 Operations Check Remote Files Created Via Day 2 Operations [Documentation] Check whether the files created in the previous test via Day 2 operations exist or not. - [Tags] nopasswd_k8s_proxy_charm charm sanity regression Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_1} Check If remote File Exists ${vnf_2_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_2} Delete NS Instance - [Tags] nopasswd_k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] nopasswd_k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] nopasswd_k8s_proxy_charm charm sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/k8s_07-dummy_helm.robot b/robot-systest/testsuite/k8s_07-dummy_helm.robot index 1d7f441..811d14d 100644 --- a/robot-systest/testsuite/k8s_07-dummy_helm.robot +++ b/robot-systest/testsuite/k8s_07-dummy_helm.robot @@ -11,6 +11,8 @@ # limitations under the License. *** Settings *** +Documentation [K8s-07] Openldap Helm in isolated cluster with dummy VIM. + Library OperatingSystem Library String Library Collections @@ -26,6 +28,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/k8scluster_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/k8s_07-dummy_helm_data.py +Force Tags k8s_07 cluster_k8s daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -36,44 +40,39 @@ ${publickey} ${EMPTY} *** Test Cases *** Create Package For OpenLDAP CNF - [Tags] dummy_vim sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Package For OpenLDAP NS - [Tags] dummy_vim sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Create Dummy VIM - [Tags] dummy_vim sanity regression ${created_vim_account_id}= Create VIM Target ${vim_name} ${vim_user} ${vim_password} ${vim_auth_url} ${vim_tenant} ${vim_account_type} Add K8s Cluster To OSM - [Tags] dummy_vim sanity regression Create K8s Cluster %{K8S_CREDENTIALS} ${k8scluster_version} ${vim_name} ${k8scluster_net} ${k8scluster_name} Create Network Service Instance - [Tags] dummy_vim sanity regression ${id}= Create Network Service ${nsd_name} ${vim_name} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Delete Network Service Instance - [Tags] dummy_vim sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Remove K8s Cluster from OSM - [Tags] dummy_vim sanity regression + [Tags] cleanup Delete K8s Cluster ${k8scluster_name} Delete VIM - [Tags] dummy_vim sanity regression cleanup + [Tags] cleanup Delete VIM Target ${vim_name} Delete NS Descriptor Test - [Tags] dummy_vim sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor Test - [Tags] dummy_vim sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/quotas_01-quota_enforcement.robot b/robot-systest/testsuite/quotas_01-quota_enforcement.robot index db91be0..339607d 100644 --- a/robot-systest/testsuite/quotas_01-quota_enforcement.robot +++ b/robot-systest/testsuite/quotas_01-quota_enforcement.robot @@ -21,6 +21,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/project_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/quotas_01-quota_enforcement_data.py +Force Tags quotas_01 cluster_main daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -33,7 +35,6 @@ ${vnfd_pkg} %{PACKAGES_FOLDER}/${vnfd_name} *** Test Cases *** Create First Project With Quotas - [Tags] quota_enforcement sanity regression Create Project With Quotas ${project_1_name} ${project_1_quotas} ${project_1_vnfds}= Get Project Quotas ${project_1_name} vnfds @@ -45,7 +46,6 @@ Create First Project With Quotas Create Second Project With Quotas - [Tags] quota_enforcement sanity regression Create Project With Quotas ${project_2_name} ${project_2_quotas} ${project_2_vnfds}= Get Project Quotas ${project_2_name} vnfds @@ -56,7 +56,6 @@ Create Second Project With Quotas Should Be Equal As Integers 1 ${project_2_ns_instances} Create User In Projects - [Tags] quota_enforcement sanity regression ${rc} ${stdout}= Run And Return RC And Output osm user-create ${user_name} --password ${user_password} --project-role-mappings ${project_1_name},project_admin --project-role-mappings ${project_2_name},project_admin Should Be Equal As Integers ${rc} ${success_return_code} @@ -65,7 +64,6 @@ Create User In Projects Change First Project Name to Third Project Name - [Tags] quota_enforcement sanity regression Update Project Name ${project_1_name} ${project_3_name} ${project_3_vnfds}= Get Project Quotas ${project_3_name} vnfds @@ -78,14 +76,12 @@ Change First Project Name to Third Project Name Create VNFDs On Third Project Until Exceed Quota - [Tags] quota_enforcement sanity regression Create VNFD In Project ${project_3_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=v1;product-name=v1' Run Keyword And Expect Error * Create VNFD In Project ${project_3_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=v2;product-name=v2' Increase Third Project Quotas - [Tags] quota_enforcement sanity regression Update Project Quotas ${project_3_name} ${project_3_quotas} ${project_3_vnfds}= Get Project Quotas ${project_3_name} vnfds @@ -93,7 +89,6 @@ Increase Third Project Quotas Create More VNFDs On Third Project Until Exceed Quota - [Tags] quota_enforcement sanity regression Create VNFD In Project ${project_3_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=v2;product-name=v2' Create VNFD In Project ${project_3_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=v3;product-name=v3' @@ -103,14 +98,12 @@ Create More VNFDs On Third Project Until Exceed Quota Create VNFDs On Second Project Until Exceed Quota - [Tags] quota_enforcement sanity regression Create VNFD In Project ${project_2_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=vp2_1;product-name=vp2_1' Run Keyword And Expect Error * Create VNFD In Project ${project_2_name} ${vnfd_pkg} ${user_name} ${user_password} override='id=vp2_2;product-name=vp2_2' Check Project Scopes - [Tags] quota_enforcement sanity regression ${rc} ${stdout}= Run And Return RC And Output osm --project ${project_3_name} --password ${user_password} --user ${user_name} vnfpkg-show vp2_1 Log ${stdout} @@ -121,13 +114,13 @@ Check Project Scopes Delete Second Project VNFD - [Tags] quota_enforcement sanity regression cleanup + [Tags] cleanup Delete VNFD In Project ${project_2_name} vp2_1 ${user_name} ${user_password} Delete Third Project VNFDs - [Tags] quota_enforcement sanity regression cleanup + [Tags] cleanup Delete VNFD In Project ${project_3_name} v1 ${user_name} ${user_password} Delete VNFD In Project ${project_3_name} v2 ${user_name} ${user_password} @@ -137,7 +130,7 @@ Delete Third Project VNFDs Delete Second Project After Removing User From It - [Tags] quota_enforcement sanity regression cleanup + [Tags] cleanup Run Keyword And Expect Error * Delete Project ${project_2_name} Remove User From Project ${user_name} ${project_2_name} @@ -145,13 +138,13 @@ Delete Second Project After Removing User From It Delete Projects User - [Tags] quota_enforcement sanity regression cleanup + [Tags] cleanup ${rc}= Run And Return RC osm user-delete ${user_name} Delete Third Project - [Tags] quota_enforcement sanity regression cleanup + [Tags] cleanup Delete Project ${project_3_name} diff --git a/robot-systest/testsuite/sa_01-vnf_with_vim_metrics.robot b/robot-systest/testsuite/sa_01-vnf_with_vim_metrics.robot index 29e5e8e..49b9273 100644 --- a/robot-systest/testsuite/sa_01-vnf_with_vim_metrics.robot +++ b/robot-systest/testsuite/sa_01-vnf_with_vim_metrics.robot @@ -25,6 +25,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/prometheus_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/sa_01-vnf_with_vim_metrics_data.py +Force Tags sa_01 cluster_sa daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -34,26 +36,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create VNF Descriptor - [Tags] vnf_vim_metrics sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] vnf_vim_metrics sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] vnf_vim_metrics sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get VNF VIM-based Metrics - [Tags] vnf_vim_metrics sanity regression Variable Should Exist ${prometheus_poll_retries} msg=Metric polling retries is not available Variable Should Exist ${prometheus_poll_timeout} msg=Metric polling timeout is not available @@ -68,19 +66,19 @@ Get VNF VIM-based Metrics Delete NS Instance - [Tags] vnf_vim_metrics sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] vnf_vim_metrics sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] vnf_vim_metrics sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/sa_02-vnf_with_vim_metrics_and_autoscaling.robot b/robot-systest/testsuite/sa_02-vnf_with_vim_metrics_and_autoscaling.robot index fb9621c..77b87b2 100644 --- a/robot-systest/testsuite/sa_02-vnf_with_vim_metrics_and_autoscaling.robot +++ b/robot-systest/testsuite/sa_02-vnf_with_vim_metrics_and_autoscaling.robot @@ -26,6 +26,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/prometheus_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/sa_02-vnf_with_vim_metrics_and_autoscaling_data.py +Force Tags sa_02 cluster_sa daily regression sanity + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -41,26 +43,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create VNF Descriptor - [Tags] vnf_vim_metrics_autoscaling sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] vnf_vim_metrics_autoscaling sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] vnf_vim_metrics_autoscaling sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get VNF Id - [Tags] vnf_vim_metrics_autoscaling sanity regression @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} Log List ${vnfr_list} @@ -68,7 +66,6 @@ Get VNF Id Get VNF IP Address - [Tags] vnf_vim_metrics_autoscaling sanity regression ${ip_addr}= Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} @@ -76,7 +73,6 @@ Get VNF IP Address Get VNF VIM-based Metric Before Auto-scaling - [Tags] vnf_vim_metrics_autoscaling sanity regression Variable Should Exist ${prometheus_poll_retries} msg=Metric polling retries is not available Variable Should Exist ${prometheus_poll_timeout} msg=Metric polling timeout is not available @@ -89,14 +85,12 @@ Get VNF VIM-based Metric Before Auto-scaling Increase VIM-based Metric To Force Auto-scaling - [Tags] vnf_vim_metrics_autoscaling sanity regression Variable Should Exist ${privatekey} msg=SSH private key not available Execute Remote Command Check Rc Return Output ${vnf_ip_addr} ${username} ${password} ${privatekey} for i in {1..9}; do yes > /dev/null & done Wait VIM-based Metric To Exceed Threshold - [Tags] vnf_vim_metrics_autoscaling sanity regression Variable Should Exist ${prometheus_poll_retries} msg=Metric polling retries is not available Variable Should Exist ${prometheus_poll_timeout} msg=Metric polling timeout is not available @@ -104,7 +98,6 @@ Wait VIM-based Metric To Exceed Threshold Get VDUs After Auto-scaling - [Tags] vnf_vim_metrics_autoscaling sanity regression Sleep 2 minutes Wait for auto-scale to take place @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -114,19 +107,19 @@ Get VDUs After Auto-scaling Delete NS Instance - [Tags] vnf_vim_metrics_autoscaling sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] vnf_vim_metrics_autoscaling sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] vnf_vim_metrics_autoscaling sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/sa_07-alarms_from_sa-related_vnfs.robot b/robot-systest/testsuite/sa_07-alarms_from_sa-related_vnfs.robot index 1f92e0d..bdf20bd 100644 --- a/robot-systest/testsuite/sa_07-alarms_from_sa-related_vnfs.robot +++ b/robot-systest/testsuite/sa_07-alarms_from_sa-related_vnfs.robot @@ -26,6 +26,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/prometheus_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/sa_07-alarms_from_sa-related_vnfs_data.py +Force Tags sa_07 cluster_sa daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -44,26 +46,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create Webhook Service VNF Descriptor - [Tags] alarms_sa_related_vnfs sanity regression Create VNFD '%{PACKAGES_FOLDER}/${ws_vnfd_pkg}' Create Webhook Service NS Descriptor - [Tags] alarms_sa_related_vnfs sanity regression Create NSD '%{PACKAGES_FOLDER}/${ws_nsd_pkg}' Instantiate Webhook Service Network Service - [Tags] alarms_sa_related_vnfs sanity regression ${id}= Create Network Service ${ws_nsd_name} %{VIM_TARGET} ${ws_ns_name} ${ns_config} ${publickey} Set Suite Variable ${ws_ns_id} ${id} Get Webhook Service VNF IP Address - [Tags] alarms_sa_related_vnfs sanity regression ${ip_addr}= Get Vnf Management Ip Address ${ws_ns_id} ${vnf_member_index} log ${ip_addr} @@ -71,7 +69,6 @@ Get Webhook Service VNF IP Address Start Webhook Service - [Tags] alarms_sa_related_vnfs sanity regression Variable Should Exist ${privatekey} msg=SSH private key not available Sleep 40 seconds Wait for SSH daemon to be up @@ -79,7 +76,6 @@ Start Webhook Service Create VNF Descriptor - [Tags] alarms_sa_related_vnfs sanity regression ${rc} ${stdout}= Run and Return RC and Output mkdir '%{PACKAGES_FOLDER}/${new_vnfd_pkg}' && WEBHOOK_URL="http://${ws_vnf_ip_addr}:${ws_port}" envsubst < '%{PACKAGES_FOLDER}/${vnfd_pkg}'/'${vnfd_file}' > '%{PACKAGES_FOLDER}/${new_vnfd_pkg}'/'${vnfd_file}' log ${stdout} @@ -88,20 +84,17 @@ Create VNF Descriptor Create NS Descriptor - [Tags] alarms_sa_related_vnfs sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] alarms_sa_related_vnfs sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} Set Suite Variable ${ns_id} ${id} Get Alarm Metric - [Tags] alarms_sa_related_vnfs sanity regression Variable Should Exist ${prometheus_poll_retries} msg=Metric polling retries is not available Variable Should Exist ${prometheus_poll_timeout} msg=Metric polling timeout is not available @@ -113,43 +106,42 @@ Get Alarm Metric Check Alarms Were Received - [Tags] alarms_sa_related_vnfs sanity regression Wait Until Keyword Succeeds 6 times 40 seconds Execute Remote Command Check Rc Return Output ${ws_vnf_ip_addr} ${username} ${password} ${privatekey} cat '${ws_log_file}' | grep '${alarm_msg}' | grep '${ns_name}' Delete NS Instance - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} Delete Webhook Service NS Instance - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete NS ${ws_ns_name} Delete Webhook Service NS Descriptor - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete NSD ${ws_nsd_name} Delete Webhook Service VNF Descriptor - [Tags] alarms_sa_related_vnfs sanity regression cleanup + [Tags] cleanup Delete VNFD ${ws_vnfd_name} diff --git a/robot-systest/testsuite/sa_08-vnf_with_vnf_indicators_snmp.robot b/robot-systest/testsuite/sa_08-vnf_with_vnf_indicators_snmp.robot index 33859ef..7cb6f6a 100644 --- a/robot-systest/testsuite/sa_08-vnf_with_vnf_indicators_snmp.robot +++ b/robot-systest/testsuite/sa_08-vnf_with_vnf_indicators_snmp.robot @@ -25,6 +25,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/prometheus_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/sa_08-vnf_with_vnf_indicators_snmp_data.py +Force Tags sa_08 cluster_k8s daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -34,26 +36,22 @@ ${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] } *** Test Cases *** Create VNF Descriptor - [Tags] vnf_indicators_snmp sanity regression Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create NS Descriptor - [Tags] vnf_indicators_snmp sanity regression Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Network Service - [Tags] vnf_indicators_snmp sanity regression ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${EMPTY} Set Suite Variable ${ns_id} ${id} Get VNF SNMP Metrics - [Tags] vnf_indicators_snmp sanity regression Variable Should Exist ${prometheus_host} msg=Prometheus address is not available Variable Should Exist ${prometheus_port} msg=Prometheus port is not available @@ -66,19 +64,19 @@ Get VNF SNMP Metrics Delete NS Instance - [Tags] vnf_indicators_snmp sanity regression cleanup + [Tags] cleanup Delete NS ${ns_name} Delete NS Descriptor - [Tags] vnf_indicators_snmp sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd_name} Delete VNF Descriptor - [Tags] vnf_indicators_snmp sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd_name} diff --git a/robot-systest/testsuite/slice_01-network_slicing.robot b/robot-systest/testsuite/slice_01-network_slicing.robot index 09c7929..5831e36 100644 --- a/robot-systest/testsuite/slice_01-network_slicing.robot +++ b/robot-systest/testsuite/slice_01-network_slicing.robot @@ -13,6 +13,8 @@ # limitations under the License. *** Settings *** +Documentation [SLICE-01] Network Slicing. + Library OperatingSystem Library String Library Collections @@ -30,6 +32,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/slice_01-network_slicing_data.py +Force Tags slice_01 cluster_slices daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -48,8 +52,6 @@ ${nst_config} {netslice-vld: [ {name: slice_vld_mgmt, vim-network-name: %{VIM_ Create Slice VNF Descriptors [Documentation] Onboards all the VNFDs required for the test: vnfd1_pkg and vnfd2_pkg (in the variables file) - [Tags] basic_network_slicing SLICING-01 sanity regression - Create VNFD '%{PACKAGES_FOLDER}/${vnfd1_pkg}' Create VNFD '%{PACKAGES_FOLDER}/${vnfd2_pkg}' @@ -57,23 +59,17 @@ Create Slice VNF Descriptors Create Slice NS Descriptors [Documentation] Onboards all the NSDs required for the test: nsd1_pkg and nsd2_pkg (in the variables file) - [Tags] basic_network_slicing SLICING-01 sanity regression - Create NSD '%{PACKAGES_FOLDER}/${nsd1_pkg}' Create NSD '%{PACKAGES_FOLDER}/${nsd2_pkg}' Create Slice Template [Documentation] Onboards the Network Slice Template: nst (in the variables file) - [Tags] basic_network_slicing SLICING-01 sanity regression - Create NST '%{PACKAGES_FOLDER}/${nst}' Network Slice Instance Test [Documentation] Instantiates the NST recently onboarded and sets the instantiation id as a suite variable (nsi_id) - [Tags] basic_network_slicing SLICING-01 sanity regression - ${id}= Create Network Slice ${nst_name} %{VIM_TARGET} ${slice_name} ${nst_config} ${publickey} Set Suite Variable ${nsi_id} ${id} @@ -81,8 +77,6 @@ Network Slice Instance Test Get Middle Vnf Management Ip [Documentation] Obtains the management IP of the slice middle VNF (name in the reources file) and sets the ip as a suite variable (mgmt_vnf_ip) - [Tags] basic_network_slicing SLICING-01 sanity regression - ${middle_ns_id}= Run and Return RC and Output osm ns-list | grep ${middle_ns_name} | awk '{print $4}' 2>&1 ${vnf_ip} Get Vnf Management Ip Address ${middle_ns_id}[1] ${vnf_member_index} Run Keyword If '${vnf_ip}' == '${EMPTY}' Fatal Error Variable \$\{ vnf_ip\} Empty @@ -92,8 +86,6 @@ Get Middle Vnf Management Ip Get Slice Vnf Ip Addresses [Documentation] Obtains the list of IPs addresses in the slice and sets the list as a suite variable (slice_vnfs_ips) - [Tags] basic_network_slicing SLICING-01 sanity regression - # Get all the ns_id in the slice except the middle one to avoid self ping @{slice_ns_list} Get Slice Ns List Except One ${slice_name} ${middle_ns_name} log many @{slice_ns_list} @@ -113,7 +105,6 @@ Get Slice Vnf Ip Addresses Test Middle Ns Ping [Documentation] Pings the slice middle vnf (mgmt_vnf_ip) - [Tags] basic_network_slicing SLICING-01 sanity regression Sleep 60s Waiting for the network to be up # Ping to the middle VNF log ${mgmt_vnf_ip} @@ -123,7 +114,6 @@ Test Middle Ns Ping Test Middle Vnf SSH Access [Documentation] SSH access to the slice middle vnf (mgmt_vnf_ip) with the credentials provided in the variables file - [Tags] basic_network_slicing SLICING-01 sanity regression Sleep 30s Waiting ssh daemon to be up Test SSH Connection ${mgmt_vnf_ip} ${username} ${password} ${privatekey} @@ -132,31 +122,26 @@ Test Slice Connectivity [Documentation] SSH access to the slice middle vnf (mgmt_vnf_ip) with the credentials provided in the variables file ... and pings all the IP addresses in the list (slice_vnfs_ips) - [Tags] basic_network_slicing SLICING-01 sanity regression - Ping Many ${mgmt_vnf_ip} ${username} ${password} ${privatekey} @{slice_vnfs_ips} Delete Slice Instance [Documentation] Stops the slice instance (slice_name) - - [Tags] basic_network_slicing SLICING-01 sanity regression cleanup + [Tags] cleanup Delete NSI ${slice_name} Delete Slice Template [Documentation] Deletes the NST (nst_name) from OSM - - [Tags] basic_network_slicing SLICING-01 sanity regression cleanup + [Tags] cleanup Delete NST ${nst_name} Delete NS Descriptors [Documentation] Deletes all the NSDs created for the test: nsd1_name, nsd2_name - - [Tags] basic_network_slicing SLICING-01 sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd1_name} Delete NSD ${nsd2_name} @@ -164,8 +149,7 @@ Delete NS Descriptors Delete VNF Descriptors [Documentation] Deletes all the VNFDs created for the test: vnfd1_name, vnfd2_name - - [Tags] basic_network_slicing SLICING-01 sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd1_name} Delete VNFD ${vnfd2_name} diff --git a/robot-systest/testsuite/slice_02-shared_network_slicing.robot b/robot-systest/testsuite/slice_02-shared_network_slicing.robot index 79010ad..7b9408c 100644 --- a/robot-systest/testsuite/slice_02-shared_network_slicing.robot +++ b/robot-systest/testsuite/slice_02-shared_network_slicing.robot @@ -13,6 +13,8 @@ # limitations under the License. *** Settings *** +Documentation [SLICE-02] Shared Network Slicing. + Library OperatingSystem Library String Library Collections @@ -29,6 +31,8 @@ Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot Variables %{ROBOT_DEVOPS_FOLDER}/resources/slice_02-shared_network_slicing_data.py +Force Tags slice_02 cluster_slices daily regression + Suite Teardown Run Keyword And Ignore Error Suite Cleanup @@ -47,8 +51,6 @@ ${nst_config} {netslice-vld: [ {name: slice_vld_mgmt, vim-network-name: %{VIM_ Create Slice VNF Descriptors [Documentation] Onboards all the VNFDs required for the test: vnfd1_pkg and vnfd2_pkg (in the variables file) - [Tags] shared_network_slicing SLICING-02 sanity regression - Create VNFD '%{PACKAGES_FOLDER}/${vnfd1_pkg}' Create VNFD '%{PACKAGES_FOLDER}/${vnfd2_pkg}' @@ -56,16 +58,12 @@ Create Slice VNF Descriptors Create Slice NS Descriptors [Documentation] Onboards all the NSDs required for the test: nsd1_pkg and nsd2_pkg (in the variables file) - [Tags] shared_network_slicing SLICING-02 sanity regression - Create NSD '%{PACKAGES_FOLDER}/${nsd1_pkg}' Create NSD '%{PACKAGES_FOLDER}/${nsd2_pkg}' Create Slice Templates [Documentation] Onboards the Network Slice Templates: nst, nst2 (in the variables file) - [Tags] shared_network_slicing SLICING-02 sanity regression - Create NST '%{PACKAGES_FOLDER}/${nst}' Create NST '%{PACKAGES_FOLDER}/${nst2}' @@ -73,8 +71,6 @@ Network Slice First Instance [Documentation] Instantiates the First NST recently onboarded (nst_name) and sets the instantiation id as a suite variable (nsi_id) ... The slice contains 3 NS (1 shared) - [Tags] shared_network_slicing SLICING-02 sanity regression - ${id}= Create Network Slice ${nst_name} %{VIM_TARGET} ${slice_name} ${nst_config} ${publickey} Set Suite Variable ${nsi_id} ${id} @@ -83,8 +79,6 @@ Network Slice Second Instance [Documentation] Instantiates the Second NST recently onboarded (nst2_name) and sets the instantiation id as a suite variable (nsi2_id) ... The slice contains 2 NS (1 shared) - [Tags] shared_network_slicing SLICING-02 sanity regression - ${id}= Create Network Slice ${nst2_name} %{VIM_TARGET} ${slice2_name} ${nst_config} ${publickey} Set Suite Variable ${nsi2_id} ${id} @@ -92,8 +86,6 @@ Network Slice Second Instance First Network Slice Ns Count [Documentation] Counts the NS in both slice instances and shoul be equal to 4 - [Tags] shared_network_slicing SLICING-02 sanity regression - ${slice1_count}= Get Slice Ns Count ${slice_name} ${slice2_count}= Get Slice Ns Count ${slice2_name} ${together}= Evaluate ${slice1_count} + ${slice2_count} @@ -103,8 +95,6 @@ First Network Slice Ns Count Get Middle Vnf Management Ip [Documentation] Obtains the management IP of the shared NS main (only) VNF and sets it as a suite variable (mgmt_vnf_ip) - [Tags] shared_network_slicing SLICING-02 sanity regression - ${middle_ns_id}= Run and Return RC and Output osm ns-list | grep ${middle_ns_name} | awk '{print $4}' 2>&1 ${vnf_ip} Get Vnf Management Ip Address ${middle_ns_id}[1] ${vnf_member_index} Run Keyword If '${vnf_ip}' == '${EMPTY}' Fatal Error Variable \$\{ vnf_ip\} Empty @@ -114,8 +104,6 @@ Get Middle Vnf Management Ip Get First Slice Vnf IPs [Documentation] Obtains the list of IPs addresses in the first slice and sets the list as a suite variable (slice1_vnfs_ips) - [Tags] shared_network_slicing SLICING-02 sanity regression - # Get all the ns_id in the slice except the middle one @{ip_list} Get Slice Vnf Ip Addresses ${slice_name} Should Be True ${ip_list} is not None @@ -125,8 +113,6 @@ Get First Slice Vnf IPs Test Middle Ns Ping [Documentation] Pings the slice middle vnf (mgmt_vnf_ip) - [Tags] shared_network_slicing SLICING-02 sanity regression - Sleep 60s Waiting for the network to be up # Ping to the middle VNF Test Connectivity ${mgmt_vnf_ip} @@ -135,8 +121,6 @@ Test Middle Ns Ping Test Middle Vnf SSH Access [Documentation] SSH access to the slice middle vnf (mgmt_vnf_ip) with the credentials provided in the variables file - [Tags] shared_network_slicing SLICING-02 sanity regression - Sleep 30s Waiting ssh daemon to be up Test SSH Connection ${mgmt_vnf_ip} ${username} ${password} ${privatekey} @@ -145,15 +129,13 @@ Test First Slice Connectivity [Documentation] SSH access to the slice middle vnf (mgmt_vnf_ip) with the credentials provided in the variables file ... and pings all the IP addresses in the list (slice1_vnfs_ips) - [Tags] shared_network_slicing SLICING-02 sanity regression - Ping Many ${mgmt_vnf_ip} ${username} ${password} ${privatekey} @{slice1_vnfs_ips} Delete Slice One Instance [Documentation] Stops the slice instance (slice_name) - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete NSI ${slice_name} @@ -161,8 +143,6 @@ Delete Slice One Instance Second Network Slice Ns Count [Documentation] Counts the NS in both slice instances and should be equal to 2 - [Tags] shared_network_slicing SLICING-02 sanity regression - ${slice1_count}= Get Slice Ns Count ${slice_name} ${slice2_count}= Get Slice Ns Count ${slice2_name} ${together}= Evaluate ${slice1_count} + ${slice2_count} @@ -171,8 +151,6 @@ Second Network Slice Ns Count Get Second Slice Vnf IPs [Documentation] Obtains the list of IPs addresses in the second slice and sets the list as a suite variable (slice2_vnfs_ips) - [Tags] shared_network_slicing SLICING-02 sanity regression - # Get all the ns_id in the slice @{ip_list} Get Slice Vnf Ip Addresses ${slice2_name} Should Be True ${ip_list} is not None @@ -183,15 +161,13 @@ Test Second Slice Connectivity [Documentation] SSH access to the slice middle vnf (mgmt_vnf_ip) with the credentials provided in the variables file ... and pings all the IP addresses in the list (slice2_vnfs_ips) - [Tags] shared_network_slicing SLICING-02 sanity regression - Ping Many ${mgmt_vnf_ip} ${username} ${password} ${privatekey} @{slice2_vnfs_ips} Delete Slice Two Instance [Documentation] Stops the slice instance (slice2_name) - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete NSI ${slice2_name} @@ -199,7 +175,7 @@ Delete Slice Two Instance Delete Slice One Template [Documentation] Deletes the NST (nst_name) from OSM - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete NST ${nst_name} @@ -207,7 +183,7 @@ Delete Slice One Template Delete Slice Two Template [Documentation] Deletes the NST (nst2_name) from OSM - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete NST ${nst2_name} @@ -215,7 +191,7 @@ Delete Slice Two Template Delete NS Descriptors [Documentation] Deletes all the NSDs created for the test: nsd1_name, nsd2_name - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete NSD ${nsd1_name} Delete NSD ${nsd2_name} @@ -224,7 +200,7 @@ Delete NS Descriptors Delete VNF Descriptors [Documentation] Deletes all the VNFDs created for the test: vnfd1_name, vnfd2_name - [Tags] shared_network_slicing SLICING-02 sanity regression cleanup + [Tags] cleanup Delete VNFD ${vnfd1_name} Delete VNFD ${vnfd2_name} -- 2.17.1