Add tags for 'prepare' and 'verify' stages to help the testing of OSM version migrati...

Change-Id: I3f199146a8dbbe2f5f6aeb26a10b95440ee74470
Signed-off-by: rodriguezgar <frodriguezg@indra.es>

diff --git a/README.md b/README.md
index 68741c2..0bb2493 100644 (file)
--- a/README.md
+++ b/README.md
@@ -159,6 +159,19 @@ In addition, the tag "cleanup" exists in those tests that perform
 any deletion. In that way, it can be invoked to retry the deletion if
 the tests were forcefully stopped.
 
 any deletion. In that way, it can be invoked to retry the deletion if
 the tests were forcefully stopped.
 

+- For helping in the migration tests and other scenarios in which you don't want
+to destroy the deployments immediately, the following tags are used:
+  - prepare: for the tests that are used to deploy the network
+  services under test
+  - verify: for the tests that perform the actual testing, or changes for
+  additional verifications (e.g. scaling).
+  - cleanup: already described above.
+
+  So, for instance, you could first deploy a number of network services executing
+  the tests with "prepare" tag, migrate to another OSM version, and then
+  check the behavior executing with the "verify" tag. Finally, use the "cleanup"
+  tag. 
+

 ## Built With
 
 * [Python](www.python.org/) - The language used
 ## Built With
 
 * [Python](www.python.org/) - The language used

diff --git a/robot-systest/lib/ns_lib.robot b/robot-systest/lib/ns_lib.robot
index a38a69e..2d6fdb7 100644 (file)
--- a/robot-systest/lib/ns_lib.robot
+++ b/robot-systest/lib/ns_lib.robot
@@ -76,6 +76,17 @@ Get Vnf Id
     [Return]  ${stdout}
 
 
     [Return]  ${stdout}
 
 

+Get Ns Id
+    [Arguments]   ${ns_name}
+
+    Should Not Be Empty   ${ns_name}
+    ${rc}   ${stdout}=   Run and Return RC and Output   osm ns-list | grep ${ns_name} | awk '{print $4}' 2>&1
+    log   ${stdout}
+    Should Be Equal As Integers   ${rc}   ${success_return_code}
+    Should Not Be Empty   ${stdout}
+    [Return]   ${stdout}
+
+

 Get Ns Vnf List
     [Arguments]   ${ns_id}
 
 Get Ns Vnf List
     [Arguments]   ${ns_id}
 


@@ -286,4 +297,5 @@ Get Application Names
     ${rc}  ${stdout}=  Run and Return RC and Output  osm ns-show ${ns_name} --literal | yq ._admin.deployed.VCA[].application | tr -d \\"
     Should Be Equal As Integers  ${rc}  ${success_return_code}  msg=${stdout}  values=False
     @{app_names} =  Split String  ${stdout}
     ${rc}  ${stdout}=  Run and Return RC and Output  osm ns-show ${ns_name} --literal | yq ._admin.deployed.VCA[].application | tr -d \\"
     Should Be Equal As Integers  ${rc}  ${success_return_code}  msg=${stdout}  values=False
     @{app_names} =  Split String  ${stdout}

-    [Return]  ${app_names}
\ No newline at end of file
+    [Return]  ${app_names}
+

diff --git a/robot-systest/testsuite/basic_07-secure_key_management.robot b/robot-systest/testsuite/basic_07-secure_key_management.robot
index 8d57bb6..1b01f78 100644 (file)
--- a/robot-systest/testsuite/basic_07-secure_key_management.robot
+++ b/robot-systest/testsuite/basic_07-secure_key_management.robot
@@ -50,36 +50,40 @@ ${day_2_file_name}   /home/ubuntu/mytouch1
 
 *** Test Cases ***
 Create Nopasswd Charm VNF Descriptor
 
 *** Test Cases ***
 Create Nopasswd Charm VNF Descriptor

-
+    [Tags]   prepare

     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Nopasswd Charm NS Descriptor
     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Nopasswd Charm NS Descriptor

-
+    [Tags]   prepare

     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Nopasswd Charm Network Service
     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Nopasswd Charm Network Service

-
+    [Tags]   prepare

     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${EMPTY}  ${ns_timeout}
     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${EMPTY}  ${ns_timeout}

-    Set Suite Variable  ${ns_id}  ${id}

 
 

+Get Ns Id
+    [Tags]   verify
+    ${id}=  Get Ns Id   ${ns_name}
+    Set Suite Variable  ${ns_id}   ${id}

 
 Get Management Ip Addresses
 
 Get Management Ip Addresses

-
+    [Tags]   verify

     ${ip_addr}  Get Vnf Management Ip Address  ${ns_id}  ${vnf_member_index}
     log  ${ip_addr}
     Set Suite Variable  ${vnf_ip_addr}  ${ip_addr}
 
 
 Test SSH Access
     ${ip_addr}  Get Vnf Management Ip Address  ${ns_id}  ${vnf_member_index}
     log  ${ip_addr}
     Set Suite Variable  ${vnf_ip_addr}  ${ip_addr}
 
 
 Test SSH Access

-
+    [Tags]   verify

     Variable Should Exist  ${vnf_ip_addr}  msg=IP address of the management VNF is not available
     Sleep  30s  Waiting ssh daemon to be up
     Test SSH Connection  ${vnf_ip_addr}  ${username}  ${password}  ${EMPTY}
 
 
 Check Remote Files Created Via Day 1 Operations
     Variable Should Exist  ${vnf_ip_addr}  msg=IP address of the management VNF is not available
     Sleep  30s  Waiting ssh daemon to be up
     Test SSH Connection  ${vnf_ip_addr}  ${username}  ${password}  ${EMPTY}
 
 
 Check Remote Files Created Via Day 1 Operations

+    [Tags]   verify

     [Documentation]     The Nopasswd VNF has a Day 1 operation that creates a file named ${day_1_file_name} and performs it without password.
     ...                 This test checks whether that files have been created or not.
 
     [Documentation]     The Nopasswd VNF has a Day 1 operation that creates a file named ${day_1_file_name} and performs it without password.
     ...                 This test checks whether that files have been created or not.
 


@@ -87,6 +91,7 @@ Check Remote Files Created Via Day 1 Operations
 
 
 Execute Day 2 Operations
 
 
 Execute Day 2 Operations

+    [Tags]   verify

     [Documentation]     Performs one Day 2 operation that creates a new file, this action is executed without password too.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     [Documentation]     Performs one Day 2 operation that creates a new file, this action is executed without password too.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available


@@ -94,6 +99,7 @@ Execute Day 2 Operations
 
 
 Check Remote Files Created Via Day 2 Operations
 
 
 Check Remote Files Created Via Day 2 Operations

+    [Tags]   verify

     [Documentation]     Check whether the file created in the previous test via Day 2 operation exists or not.
 
     Check If remote File Exists  ${vnf_ip_addr}  ${username}  ${password}  ${EMPTY}  ${day_2_file_name}
     [Documentation]     Check whether the file created in the previous test via Day 2 operation exists or not.
 
     Check If remote File Exists  ${vnf_ip_addr}  ${username}  ${password}  ${EMPTY}  ${day_2_file_name}

diff --git a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot
index 7989d98..edb1559 100644 (file)
--- a/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot
+++ b/robot-systest/testsuite/basic_09-manual_vdu_scaling.robot
@@ -46,23 +46,26 @@ ${publickey}   %{HOME}/.ssh/id_rsa.pub
 
 *** Test Cases ***
 Create Scaling VNF Descriptor
 
 *** Test Cases ***
 Create Scaling VNF Descriptor

-
+    [Tags]   prepare

     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Scaling NS Descriptor
     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Scaling NS Descriptor

-
+    [Tags]   prepare

     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Scaling Network Service
     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Scaling Network Service

-
+    [Tags]   prepare

     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${publickey}
     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${publickey}

-    Set Suite Variable  ${ns_id}  ${id}

 
 

+Get Ns Id
+    [Tags]   verify
+    ${id}=   Get Ns Id   ${ns_name}
+    Set Suite Variable   ${ns_id}   ${id}

 
 Get Vnf Id
 
 Get Vnf Id

-
+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     @{vnfr_list}=  Get Ns Vnfr Ids  ${ns_id}
     Log List  ${vnfr_list}
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     @{vnfr_list}=  Get Ns Vnfr Ids  ${ns_id}
     Log List  ${vnfr_list}


@@ -70,6 +73,7 @@ Get Vnf Id
 
 
 Get Vdus Before Scale Out
 
 
 Get Vdus Before Scale Out

+    [Tags]   verify

     [Documentation]     Get the number of VDU records before the manual scaling.
 
     @{vdur_list}=  Get Vnf Vdur Names  ${vnf_id}
     [Documentation]     Get the number of VDU records before the manual scaling.
 
     @{vdur_list}=  Get Vnf Vdur Names  ${vnf_id}


@@ -79,12 +83,13 @@ Get Vdus Before Scale Out
 
 
 Perform Manual Vdu Scale Out
 
 
 Perform Manual Vdu Scale Out

-
+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute Manual VNF Scale  ${ns_name}  ${vnf_member_index}  ${scaling_group}  SCALE_OUT
 
 
 Check Vdus After Scale Out
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute Manual VNF Scale  ${ns_name}  ${vnf_member_index}  ${scaling_group}  SCALE_OUT
 
 
 Check Vdus After Scale Out

+    [Tags]   verify

     [Documentation]     Check whether there is one more VDU after scaling or not.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     [Documentation]     Check whether there is one more VDU after scaling or not.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available


@@ -95,12 +100,13 @@ Check Vdus After Scale Out
 
 
 Perform Manual Vdu Scale In
 
 
 Perform Manual Vdu Scale In

-
+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute Manual VNF Scale  ${ns_name}  ${vnf_member_index}  ${scaling_group}  SCALE_IN
 
 
 Check Vdus After Scaling In
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute Manual VNF Scale  ${ns_name}  ${vnf_member_index}  ${scaling_group}  SCALE_IN
 
 
 Check Vdus After Scaling In

+    [Tags]   verify

     [Documentation]     Check whether there is one less VDU after scaling or not.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     [Documentation]     Check whether there is one less VDU after scaling or not.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available

diff --git a/robot-systest/testsuite/k8s_04-openldap_helm.robot b/robot-systest/testsuite/k8s_04-openldap_helm.robot
index 3d87508..8264bf4 100644 (file)
--- a/robot-systest/testsuite/k8s_04-openldap_helm.robot
+++ b/robot-systest/testsuite/k8s_04-openldap_helm.robot
@@ -56,39 +56,52 @@ ${replica_count}   3
 
 *** Test Cases ***
 Create Package For OpenLDAP CNF
 
 *** Test Cases ***
 Create Package For OpenLDAP CNF

+    [Tags]  prepare

     Create VNFD   '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 Create Package For OpenLDAP NS
     Create VNFD   '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 Create Package For OpenLDAP NS

+    [Tags]  prepare

     Create NSD   '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 Add K8s Cluster To OSM
     Create NSD   '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 Add K8s Cluster To OSM

+    [Tags]  prepare

     Create K8s Cluster  %{K8S_CREDENTIALS}  ${k8scluster_version}  %{VIM_TARGET}  %{VIM_MGMT_NET}  ${k8scluster_name}
 
 Create Network Service Instance
     Create K8s Cluster  %{K8S_CREDENTIALS}  ${k8scluster_version}  %{VIM_TARGET}  %{VIM_MGMT_NET}  ${k8scluster_name}
 
 Create Network Service Instance

+    [Tags]  prepare

     ${id}=   Create Network Service   ${nsd_name}   %{VIM_TARGET}   ${ns_name}   ${ns_config}  ${publickey}
     ${id}=   Create Network Service   ${nsd_name}   %{VIM_TARGET}   ${ns_name}   ${ns_config}  ${publickey}

+
+Get Ns Id
+    [Tags]   verify
+    ${id}=   Get Ns Id   ${ns_name}

     Set Suite Variable   ${ns_id}   ${id}
 
 Get Vnf Id
     Set Suite Variable   ${ns_id}   ${id}
 
 Get Vnf Id

+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     @{vnfr_list}=  Get Ns Vnfr Ids  ${ns_id}
     Log List  ${vnfr_list}
     Set Suite Variable  ${vnf_id}  ${vnfr_list}[0]
 
 Execute Upgrade Operation
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     @{vnfr_list}=  Get Ns Vnfr Ids  ${ns_id}
     Log List  ${vnfr_list}
     Set Suite Variable  ${vnf_id}  ${vnfr_list}[0]
 
 Execute Upgrade Operation

+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute NS K8s Action   ${ns_name}   ${upgrade_action}   ${vnf_member_index}   ${kdu_name}   replicaCount=${replica_count}
 
 Check Replicas After Upgrade Operation
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute NS K8s Action   ${ns_name}   ${upgrade_action}   ${vnf_member_index}   ${kdu_name}   replicaCount=${replica_count}
 
 Check Replicas After Upgrade Operation

+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${count}=   Get Vnf Kdu Replica Count   ${vnf_id}   ${kdu_name}
     log   ${count}
     Should Be Equal As Integers   ${count}   ${replica_count}
 
 Execute Rollback Operation
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${count}=   Get Vnf Kdu Replica Count   ${vnf_id}   ${kdu_name}
     log   ${count}
     Should Be Equal As Integers   ${count}   ${replica_count}
 
 Execute Rollback Operation

+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute NS K8s Action   ${ns_name}   ${rollback_action}   ${vnf_member_index}   ${kdu_name}
 
 Check Replicas After Rollback Operation
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${ns_op_id}=  Execute NS K8s Action   ${ns_name}   ${rollback_action}   ${vnf_member_index}   ${kdu_name}
 
 Check Replicas After Rollback Operation

+    [Tags]   verify

     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${count}=   Get Vnf Kdu Replica Count   ${vnf_id}   ${kdu_name}
     log   ${count}
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     ${count}=   Get Vnf Kdu Replica Count   ${vnf_id}   ${kdu_name}
     log   ${count}

diff --git a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot
index 3bbe5b0..f1a6d87 100644 (file)
--- a/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot
+++ b/robot-systest/testsuite/k8s_06-k8s_secure_key_management.robot
@@ -55,23 +55,26 @@ ${ns_timeout}   15min
 
 *** Test Cases ***
 Create Charm VNF Descriptor
 
 *** Test Cases ***
 Create Charm VNF Descriptor

-
+    [Tags]   prepare

     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Charm NS Descriptor
     Create VNFD  '%{PACKAGES_FOLDER}/${vnfd_pkg}'
 
 
 Create Charm NS Descriptor

-
+    [Tags]   prepare

     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Charm Network Service
     Create NSD  '%{PACKAGES_FOLDER}/${nsd_pkg}'
 
 
 Instantiate Charm Network Service

-
+    [Tags]   prepare

     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${publickey}  ${ns_timeout}
     ${id}=  Create Network Service  ${nsd_name}  %{VIM_TARGET}  ${ns_name}  ${ns_config}  ${publickey}  ${ns_timeout}

-    Set Suite Variable  ${ns_id}  ${id}

 
 

+Get Ns Id
+    [Tags]   verify
+    ${id}=   Get Ns Id   ${ns_name}
+    Set Suite Variable   ${ns_id}   ${id}

 
 Get Management Ip Addresses
 
 Get Management Ip Addresses

-
+    [Tags]   verify

     ${ip_addr_1}  Get Vnf Management Ip Address  ${ns_id}  ${vnf_member_index_1}
     log  ${ip_addr_1}
     Set Suite Variable  ${vnf_1_ip_addr}  ${ip_addr_1}
     ${ip_addr_1}  Get Vnf Management Ip Address  ${ns_id}  ${vnf_member_index_1}
     log  ${ip_addr_1}
     Set Suite Variable  ${vnf_1_ip_addr}  ${ip_addr_1}


@@ -81,7 +84,7 @@ Get Management Ip Addresses
 
 
 Test SSH Access
 
 
 Test SSH Access

-
+    [Tags]   verify

     Variable Should Exist  ${vnf_1_ip_addr}  msg=IP address of the management VNF '${vnf_member_index_1}' is not available
     Variable Should Exist  ${vnf_2_ip_addr}  msg=IP address of the management VNF '${vnf_member_index_2}' is not available
     Sleep  30s  Waiting ssh daemon to be up
     Variable Should Exist  ${vnf_1_ip_addr}  msg=IP address of the management VNF '${vnf_member_index_1}' is not available
     Variable Should Exist  ${vnf_2_ip_addr}  msg=IP address of the management VNF '${vnf_member_index_2}' is not available
     Sleep  30s  Waiting ssh daemon to be up


@@ -90,6 +93,7 @@ Test SSH Access
 
 
 Check Remote Files Created Via Day 1 Operations
 
 
 Check Remote Files Created Via Day 1 Operations

+    [Tags]   verify

     [Documentation]     The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}.
     ...                 This test checks whether that files have been created or not.
 
     [Documentation]     The Charm VNF has a Day 1 operation that creates a file named ${day_1_file_name}.
     ...                 This test checks whether that files have been created or not.
 


@@ -98,6 +102,7 @@ Check Remote Files Created Via Day 1 Operations
 
 
 Execute Day 2 Operations
 
 
 Execute Day 2 Operations

+    [Tags]   verify

     [Documentation]     Performs one Day 2 operation per VNF that creates a new file.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available
     [Documentation]     Performs one Day 2 operation per VNF that creates a new file.
 
     Variable Should Exist  ${ns_id}  msg=Network service instance is not available


@@ -106,6 +111,7 @@ Execute Day 2 Operations
 
 
 Check Remote Files Created Via Day 2 Operations
 
 
 Check Remote Files Created Via Day 2 Operations

+    [Tags]   verify

     [Documentation]     Check whether the files created in the previous test via Day 2 operations exist or not.
 
     Check If remote File Exists  ${vnf_1_ip_addr}  ${username}  ${password}  ${privatekey}  ${day_2_file_name_1}
     [Documentation]     Check whether the files created in the previous test via Day 2 operations exist or not.
 
     Check If remote File Exists  ${vnf_1_ip_addr}  ${username}  ${password}  ${privatekey}  ${day_2_file_name_1}