From: rodriguezgar Date: Thu, 24 Feb 2022 19:05:42 +0000 (+0100) Subject: Add tags for 'prepare' and 'verify' stages to help the testing of OSM version migrati... X-Git-Tag: v10.1.1rc2^0 X-Git-Url: https://osm.etsi.org/gitweb/?a=commitdiff_plain;h=refs%2Fchanges%2F58%2F12158%2F1;p=osm%2Ftests.git Add tags for 'prepare' and 'verify' stages to help the testing of OSM version migration. New Keyword to get ns_id from NS name so that the tests with 'verify' tag can be executed in isolation. Change-Id: I3f199146a8dbbe2f5f6aeb26a10b95440ee74470 Signed-off-by: rodriguezgar --- diff --git a/README.md b/README.md index 68741c2..0bb2493 100644 --- a/README.md +++ b/README.md @@ -159,6 +159,19 @@ In addition, the tag "cleanup" exists in those tests that perform any deletion. In that way, it can be invoked to retry the deletion if the tests were forcefully stopped. +- For helping in the migration tests and other scenarios in which you don't want +to destroy the deployments immediately, the following tags are used: + - prepare: for the tests that are used to deploy the network + services under test + - verify: for the tests that perform the actual testing, or changes for + additional verifications (e.g. scaling). + - cleanup: already described above. + + So, for instance, you could first deploy a number of network services executing + the tests with "prepare" tag, migrate to another OSM version, and then + check the behavior executing with the "verify" tag. Finally, use the "cleanup" + tag. + ## Built With * [Python](www.python.org/) - The language used diff --git a/robot-systest/lib/ns_lib.robot b/robot-systest/lib/ns_lib.robot index 4d6b16f..0976241 100644 --- a/robot-systest/lib/ns_lib.robot +++ b/robot-systest/lib/ns_lib.robot @@ -76,6 +76,17 @@ Get Vnf Id [Return] ${stdout} +Get Ns Id + [Arguments] ${ns_name} + + Should Not Be Empty ${ns_name} + ${rc} ${stdout}= Run and Return RC and Output osm ns-list | grep ${ns_name} | awk '{print $4}' 2>&1 + log ${stdout} + Should Be Equal As Integers ${rc} ${success_return_code} + Should Not Be Empty ${stdout} + [Return] ${stdout} + + Get Ns Vnf List [Arguments] ${ns_id} diff --git a/robot-systest/testsuite/basic_07-secure_key_management.robot b/robot-systest/testsuite/basic_07-secure_key_management.robot index f69420b..699b0e7 100644 --- a/robot-systest/testsuite/basic_07-secure_key_management.robot +++ b/robot-systest/testsuite/basic_07-secure_key_management.robot @@ -50,36 +50,40 @@ ${day_2_file_name} /home/ubuntu/mytouch1 *** Test Cases *** Create Nopasswd Charm VNF Descriptor - + [Tags] prepare Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Nopasswd Charm NS Descriptor - + [Tags] prepare Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Nopasswd Charm Network Service - + [Tags] prepare ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${EMPTY} ${ns_timeout} - Set Suite Variable ${ns_id} ${id} +Get Ns Id + [Tags] verify + ${id}= Get Ns Id ${ns_name} + Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - + [Tags] verify ${ip_addr} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index} log ${ip_addr} Set Suite Variable ${vnf_ip_addr} ${ip_addr} Test SSH Access - + [Tags] verify Variable Should Exist ${vnf_ip_addr} msg=IP address of the management VNF is not available Sleep 30s Waiting ssh daemon to be up Test SSH Connection ${vnf_ip_addr} ${username} ${password} ${EMPTY} Check Remote Files Created Via Day 1 Operations + [Tags] verify [Documentation] The Nopasswd VNF has a Day 1 operation that creates a file named ${day_1_file_name} and performs it without password. ... This test checks whether that files have been created or not. @@ -87,6 +91,7 @@ Check Remote Files Created Via Day 1 Operations Execute Day 2 Operations + [Tags] verify [Documentation] Performs one Day 2 operation that creates a new file, this action is executed without password too. Variable Should Exist ${ns_id} msg=Network service instance is not available @@ -94,6 +99,7 @@ Execute Day 2 Operations Check Remote Files Created Via Day 2 Operations + [Tags] verify [Documentation] Check whether the file created in the previous test via Day 2 operation exists or not. Check If remote File Exists ${vnf_ip_addr} ${username} ${password} ${EMPTY} ${day_2_file_name} diff --git a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot index a549591..fe99726 100644 --- a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot +++ b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot @@ -46,23 +46,26 @@ ${publickey} %{HOME}/.ssh/id_rsa.pub *** Test Cases *** Create Scaling VNF Descriptor - + [Tags] prepare Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Scaling NS Descriptor - + [Tags] prepare Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Scaling Network Service - + [Tags] prepare ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} - Set Suite Variable ${ns_id} ${id} +Get Ns Id + [Tags] verify + ${id}= Get Ns Id ${ns_name} + Set Suite Variable ${ns_id} ${id} Get Vnf Id - + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} Log List ${vnfr_list} @@ -70,6 +73,7 @@ Get Vnf Id Get Vdus Before Scale Out + [Tags] verify [Documentation] Get the number of VDU records before the manual scaling. @{vdur_list}= Get Vnf Vdur Names ${vnf_id} @@ -79,12 +83,13 @@ Get Vdus Before Scale Out Perform Manual Vdu Scale Out - + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_OUT Check Vdus After Scale Out + [Tags] verify [Documentation] Check whether there is one more VDU after scaling or not. Variable Should Exist ${ns_id} msg=Network service instance is not available @@ -95,12 +100,13 @@ Check Vdus After Scale Out Perform Manual Vdu Scale In - + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute Manual VNF Scale ${ns_name} ${vnf_member_index} ${scaling_group} SCALE_IN Check Vdus After Scaling In + [Tags] verify [Documentation] Check whether there is one less VDU after scaling or not. Variable Should Exist ${ns_id} msg=Network service instance is not available diff --git a/robot-systest/testsuite/k8s_04-openldap_helm.robot b/robot-systest/testsuite/k8s_04-openldap_helm.robot index 3d87508..8264bf4 100644 --- a/robot-systest/testsuite/k8s_04-openldap_helm.robot +++ b/robot-systest/testsuite/k8s_04-openldap_helm.robot @@ -56,39 +56,52 @@ ${replica_count} 3 *** Test Cases *** Create Package For OpenLDAP CNF + [Tags] prepare Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Package For OpenLDAP NS + [Tags] prepare Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Add K8s Cluster To OSM + [Tags] prepare Create K8s Cluster %{K8S_CREDENTIALS} ${k8scluster_version} %{VIM_TARGET} %{VIM_MGMT_NET} ${k8scluster_name} Create Network Service Instance + [Tags] prepare ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} + +Get Ns Id + [Tags] verify + ${id}= Get Ns Id ${ns_name} Set Suite Variable ${ns_id} ${id} Get Vnf Id + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available @{vnfr_list}= Get Ns Vnfr Ids ${ns_id} Log List ${vnfr_list} Set Suite Variable ${vnf_id} ${vnfr_list}[0] Execute Upgrade Operation + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute NS K8s Action ${ns_name} ${upgrade_action} ${vnf_member_index} ${kdu_name} replicaCount=${replica_count} Check Replicas After Upgrade Operation + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${count}= Get Vnf Kdu Replica Count ${vnf_id} ${kdu_name} log ${count} Should Be Equal As Integers ${count} ${replica_count} Execute Rollback Operation + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${ns_op_id}= Execute NS K8s Action ${ns_name} ${rollback_action} ${vnf_member_index} ${kdu_name} Check Replicas After Rollback Operation + [Tags] verify Variable Should Exist ${ns_id} msg=Network service instance is not available ${count}= Get Vnf Kdu Replica Count ${vnf_id} ${kdu_name} log ${count} diff --git a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot index ed43a50..6b8d3cf 100644 --- a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot +++ b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot @@ -55,23 +55,26 @@ ${ns_timeout} 15min *** Test Cases *** Create Charm VNF Descriptor - + [Tags] prepare Create VNFD '%{PACKAGES_FOLDER}/${vnfd_pkg}' Create Charm NS Descriptor - + [Tags] prepare Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}' Instantiate Charm Network Service - + [Tags] prepare ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey} ${ns_timeout} - Set Suite Variable ${ns_id} ${id} +Get Ns Id + [Tags] verify + ${id}= Get Ns Id ${ns_name} + Set Suite Variable ${ns_id} ${id} Get Management Ip Addresses - + [Tags] verify ${ip_addr_1} Get Vnf Management Ip Address ${ns_id} ${vnf_member_index_1} log ${ip_addr_1} Set Suite Variable ${vnf_1_ip_addr} ${ip_addr_1} @@ -81,7 +84,7 @@ Get Management Ip Addresses Test SSH Access - + [Tags] verify Variable Should Exist ${vnf_1_ip_addr} msg=IP address of the management VNF '${vnf_member_index_1}' is not available Variable Should Exist ${vnf_2_ip_addr} msg=IP address of the management VNF '${vnf_member_index_2}' is not available Sleep 30s Waiting ssh daemon to be up @@ -90,6 +93,7 @@ Test SSH Access Check Remote Files Created Via Day 1 Operations + [Tags] verify [Documentation] The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}. ... This test checks whether that files have been created or not. @@ -98,6 +102,7 @@ Check Remote Files Created Via Day 1 Operations Execute Day 2 Operations + [Tags] verify [Documentation] Performs one Day 2 operation per VNF that creates a new file. Variable Should Exist ${ns_id} msg=Network service instance is not available @@ -106,6 +111,7 @@ Execute Day 2 Operations Check Remote Files Created Via Day 2 Operations + [Tags] verify [Documentation] Check whether the files created in the previous test via Day 2 operations exist or not. Check If remote File Exists ${vnf_1_ip_addr} ${username} ${password} ${privatekey} ${day_2_file_name_1}