+*** Comments ***
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# See the License for the specific language governing permissions and
# limitations under the License.
+
*** Settings ***
Documentation [HEAL-03] Healing of multiple VDUs
Library Process
Library SSHLibrary
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/vnfd_lib.robot
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/vnf_lib.robot
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/nsd_lib.robot
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/ns_lib.robot
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/ssh_lib.robot
-Resource %{ROBOT_DEVOPS_FOLDER}/lib/openstack_lib.robot
-
-Force Tags heal_03 cluster_heal daily regression
+Resource ../lib/vnfd_lib.resource
+Resource ../lib/vnf_lib.resource
+Resource ../lib/nsd_lib.resource
+Resource ../lib/ns_lib.resource
+Resource ../lib/ns_operation_lib.resource
+Resource ../lib/ssh_lib.resource
+Resource ../lib/openstack_lib.resource
+Test Tags heal_03 cluster_heal daily regression
Suite Teardown Run Keyword And Ignore Error Suite Cleanup
*** Variables ***
# NS and VNF descriptor package folder and ids
-${vnfd_volumes_pkg} several_volumes_vnf
-${vnfd_volumes_name} several_volumes-vnf
-${vdu_volumes_name} several_volumes-VM
-${vnf_several_index} several_volumes_vnf
-${vnfd_charm_pkg} charm-packages/native_manual_scale_charm_vnf
-${vnfd_charm_name} native_manual_scale_charm-vnf
-${vdu_charm_name} mgmtVM
-${vnf_charm_index} charm_vnf
-${vnf_charm_scaling_group} manual-scaling_mgmtVM
-${vnf_charm_cloudinit_file} /root/helloworld.txt
-${vnf_charm_day1_file} /home/ubuntu/first-touch
-${nsd_pkg} volumes_nativecharm_ns
-${nsd_name} volumes_nativecharm-ns
+${VNFD_VOLUMES_PKG} several_volumes_vnf
+${VNFD_VOLUMES_NAME} several_volumes-vnf
+${VDU_VOLUMES_NAME} several_volumes-VM
+${VNF_SEVERAL_INDEX} several_volumes_vnf
+${VNFD_MANUALSCALE_PKG} manual_scale_vnf
+${VNFD_MANUALSCALE_NAME} manual_scale-vnf
+${VDU_MANUALSCALE_NAME} mgmtVM
+${VNF_MANUALSCALE_INDEX} manual_scale_vnf
+${VNF_MANUALSCALE_SCALING_GROUP} manual-scaling_mgmtVM
+${VNF_MANUALSCALE_CLOUDINIT_FILE} /root/helloworld.txt
+${VNF_MANUALSCALE_DAY1_FILE} /home/ubuntu/first-touch
+${NSD_PKG} volumes_healing_ns
+${NSD_NAME} volumes_healing-ns
# NS instance name and configuration
-${ns_name} heal_03
-${ns_config} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] }
-${scale_wait_time} 4min
+${NS_NAME} heal_03
+${NS_CONFIG} {vld: [ {name: mgmtnet, vim-network-name: %{VIM_MGMT_NET}} ] }
+${NS_TIMEOUT} 6min
+${SCALE_WAIT_TIME} 5min
# SSH keys and username to be used
-${publickey} %{HOME}/.ssh/id_rsa.pub
-${privatekey} %{HOME}/.ssh/id_rsa
-${username} ubuntu
-${password} ${EMPTY}
+${PUBLICKEY} %{HOME}/.ssh/id_rsa.pub
+${PRIVATEKEY} %{HOME}/.ssh/id_rsa
+${USERNAME} ubuntu
+${PASSWORD} ${EMPTY}
-${success_return_code} 0
+${SUCCESS_RETURN_CODE} 0
+
+@{VIM_VDUS} @{EMPTY}
+@{VIM_VOLUMES} @{EMPTY}
*** Test Cases ***
Create VNF Descriptors
- Create VNFD '%{PACKAGES_FOLDER}/${vnfd_charm_pkg}'
- Create VNFD '%{PACKAGES_FOLDER}/${vnfd_volumes_pkg}'
-
+ [Documentation] Upload VNF packages for the testsuite.
+ Create VNFD '%{PACKAGES_FOLDER}/${VNFD_MANUALSCALE_PKG}'
+ Create VNFD '%{PACKAGES_FOLDER}/${VNFD_VOLUMES_PKG}'
Create NS Descriptor
- Create NSD '%{PACKAGES_FOLDER}/${nsd_pkg}'
-
+ [Documentation] Upload NS package for the testsuite.
+ Create NSD '%{PACKAGES_FOLDER}/${NSD_PKG}'
Network Service Instance Test
- ${id}= Create Network Service ${nsd_name} %{VIM_TARGET} ${ns_name} ${ns_config} ${publickey}
- Set Suite Variable ${ns_id} ${id}
-
+ [Documentation] Instantiate NS for the testsuite.
+ ${id}= Create Network Service ${NSD_NAME} %{VIM_TARGET} ${NS_NAME} ${NS_CONFIG} ${PUBLICKEY} ${NS_TIMEOUT}
+ Set Suite Variable ${NS_ID} ${id}
Get NS Id
+ [Documentation] Get NS identifier and stores as suite variable to be used later on.
[Tags] cleanup
- ${variables} Get Variables
+ ${variables}= Get Variables
IF not "\${ns_id}" in "${variables}"
- ${id}= Get Ns Id ${ns_name}
- Set Suite Variable ${ns_id} ${id}
+ ${id}= Get Ns Id ${NS_NAME}
+ Set Suite Variable ${NS_ID} ${id}
END
-
-Scale Out Charm VNF
- ${vnf_id}= Get Vnf Id ${ns_id} ${vnf_charm_index}
- Set Suite Variable ${vnf_charm_id} ${vnf_id}
- @{vdur_list}= Get Vnf Vdur Names ${vnf_charm_id}
+Scale Out Manual Scale VNF
+ [Documentation] Perform a manual scale-out operation of the manual scale VNF.
+ ${vnf_id}= Get Vnf Id ${NS_ID} ${VNF_MANUALSCALE_INDEX}
+ Set Suite Variable ${VNF_MANUALSCALE_ID} ${vnf_id}
+ @{vdur_list}= Get Vnf Vdur Names ${VNF_MANUALSCALE_ID}
${vdurs}= Get Length ${vdur_list}
- Set Suite Variable ${initial_vdur_count} ${vdurs}
- Execute Manual VNF Scale ${ns_name} ${vnf_charm_index} ${vnf_charm_scaling_group} SCALE_OUT ${scale_wait_time}
- @{vdur_list}= Get Vnf Vdur Names ${vnf_charm_id}
+ Set Suite Variable ${INITIAL_VDUR_COUNT} ${vdurs}
+ Execute Manual VNF Scale ${NS_NAME} ${VNF_MANUALSCALE_INDEX} ${VNF_MANUALSCALE_SCALING_GROUP} SCALE_OUT ${SCALE_WAIT_TIME}
+ @{vdur_list}= Get Vnf Vdur Names ${VNF_MANUALSCALE_ID}
${vdurs}= Get Length ${vdur_list}
- Run Keyword If ${vdurs} != ${initial_vdur_count} + 1 Fail msg=There is no new VDU records in the VNF after Scale Out
-
-
-Get Charm VNF Info
- Variable Should Exist ${ns_id} msg=NS is not available
- ${variables} Get Variables
- IF not "\${vnf_charm_id}" in "${variables}"
- ${vnf_id}= Get Vnf Id ${ns_id} ${vnf_charm_index}
- Set Suite Variable ${vnf_charm_id} ${vnf_id}
+ IF ${vdurs} != ${INITIAL_VDUR_COUNT} + 1 Fail msg=There is no new VDU records in the VNF after Scale Out
+
+Get VIM Objects
+ [Documentation] Retrieve all VMs and volumes from the NS and stores them in VIM_VDUS and VIM_VOLUMES lists.
+ Variable Should Exist ${NS_ID} msg=NS is not available
+ @{vnf_id_list}= Get Ns Vnf List ${NS_ID}
+ Log ${vnf_id_list}
+ FOR ${vnf_id} IN @{vnf_id_list}
+ Log ${vnf_id}
+ ${id}= Get VNF VIM ID ${vnf_id}
+ @{vdu_ids}= Split String ${id}
+ Append To List ${VIM_VDUS} @{vdu_ids}
END
- ${id}= Get VNF VIM ID ${vnf_charm_id}
- @{vdu_charm_ids}= Split String ${id}
- Set Suite Variable @{vdu_charm_ids} @{vdu_charm_ids}
- log ${vdu_charm_ids}[1]
- @{charm_ip_list}= Get Vnf Vdur IPs ${vnf_charm_id}
- Set Suite Variable @{charm_ip_list} @{charm_ip_list}
-
+ FOR ${vdu_id} IN @{VIM_VDUS}
+ ${volumes_attached}= Get Server Property ${vdu_id} volumes_attached
+ ${match}= Get Regexp Matches ${volumes_attached} '([0-9a-f\-]+)' 1
+ IF ${match} != @{EMPTY}
+ IF not "${match}[0]" in "@{VIM_VOLUMES}"
+ Append To List ${VIM_VOLUMES} ${match}[0]
+ END
+ END
+ END
+ Log Many @{VIM_VDUS}
+ Log Many @{VIM_VOLUMES}
+
+Get Manual Scale VNF Info
+ [Documentation] Get VDU ID and IP addresses of the manual scale VNF and stores them in VDU_MANUALSCALE_IDS and MANUALSCALE_IP_LIST.
+ Variable Should Exist ${NS_ID} msg=NS is not available
+ ${variables}= Get Variables
+ IF not "\${VNF_MANUALSCALE_ID}" in "${variables}"
+ ${vnf_id}= Get Vnf Id ${NS_ID} ${VNF_MANUALSCALE_INDEX}
+ Set Suite Variable ${VNF_MANUALSCALE_ID} ${vnf_id}
+ END
+ ${id}= Get VNF VIM ID ${VNF_MANUALSCALE_ID}
+ @{vdu_manualscale_ids}= Split String ${id}
+ Set Suite Variable @{VDU_MANUALSCALE_IDS} @{vdu_manualscale_ids}
+ Log ${VDU_MANUALSCALE_IDS}[1]
+ @{manualscale_ip_list}= Get Vnf Vdur IPs ${VNF_MANUALSCALE_ID}
+ Set Suite Variable @{MANUALSCALE_IP_LIST} @{manualscale_ip_list}
Get Volume VNF Info
- Variable Should Exist ${ns_id} msg=NS is not available
- ${ip_addr}= Get Vnf Management Ip Address ${ns_id} ${vnf_several_index}
- log ${ip_addr}
- Set Suite Variable ${vnf_volumes_ip_addr} ${ip_addr}
-
- ${vnf_id}= Get Vnf Id ${ns_id} ${vnf_several_index}
- Set Suite Variable ${vnf_volumes_id} ${vnf_id}
+ [Documentation] Get info about volumes and VM of the VNF with several volumes and stores them in suite variables.
+ Variable Should Exist ${NS_ID} msg=NS is not available
+ ${ip_addr}= Get Vnf Management Ip Address ${NS_ID} ${VNF_SEVERAL_INDEX}
+ Log ${ip_addr}
+ Set Suite Variable ${VNF_VOLUMES_IP_ADDR} ${ip_addr}
+ ${vnf_id}= Get Vnf Id ${NS_ID} ${VNF_SEVERAL_INDEX}
+ Set Suite Variable ${VNF_VOLUMES_ID} ${vnf_id}
${id}= Get VNF VIM ID ${vnf_id}
- Set Suite Variable ${vdu_volumes_id} ${id}
- log ${vdu_volumes_id}
-
- @{volumes_ip_list}= Get Vnf Vdur IPs ${vnf_volumes_id}
- Set Suite Variable @{volumes_ip_list} @{volumes_ip_list}
- log @{volumes_ip_list}
-
+ Set Suite Variable ${VDU_VOLUMES_ID} ${id}
+ Log ${VDU_VOLUMES_ID}
+ @{volumes_ip_list}= Get Vnf Vdur IPs ${VNF_VOLUMES_ID}
+ Set Suite Variable @{VOLUMES_IP_LIST} @{volumes_ip_list}
+ Log @{VOLUMES_IP_LIST}
Stop VDUs
- Variable Should Exist @{vdu_charm_ids} msg=Charm VDUs are not available
- Variable Should Exist ${vdu_volumes_id} msg=Volume VDU is not available
- Stop Server ${vdu_charm_ids}[1]
- Stop Server ${vdu_charm_ids}[2]
- Stop Server ${vdu_volumes_id}
+ [Documentation] Stop all VMs.
+ Variable Should Exist @{VDU_MANUALSCALE_IDS} msg=Manual Scale VDUs are not available
+ Variable Should Exist ${VDU_VOLUMES_ID} msg=Volume VDU is not available
+ Stop Server ${VDU_MANUALSCALE_IDS}[1]
+ Stop Server ${VDU_MANUALSCALE_IDS}[2]
+ Stop Server ${VDU_VOLUMES_ID}
Sleep 30
-
Heal VDUs
- Variable Should Exist ${vnf_charm_id} msg=Charm VNF is not available
- Variable Should Exist ${vnf_volumes_id} msg=Volume VNF is not available
- Heal Network Service ${ns_id} --vnf ${vnf_volumes_id} --cause "Heal myvdu1 of several_volumes_vnf" --vdu several_volumes-VM --vnf ${vnf_charm_id} --cause "Heal two VMs of native_manual_scale_charm_vnf" --vdu mgmtVM --count-index 1 --run-day1 --vdu mgmtVM --count-index 2
-
+ [Documentation] Heal manually via OSM commands all stopped VMs . They should be started again.
+ Variable Should Exist ${VNF_MANUALSCALE_ID} msg=Manual Scale VNF is not available
+ Variable Should Exist ${VNF_VOLUMES_ID} msg=Volume VNF is not available
+ Heal Network Service ${NS_ID} --vnf ${VNF_VOLUMES_ID} --cause "Heal myvdu1 of several_volumes_vnf" --vdu several_volumes-VM --vnf ${VNF_MANUALSCALE_ID} --cause "Heal two VMs of manual_scale_vnf" --vdu mgmtVM --count-index 1 --run-day1 --vdu mgmtVM --count-index 2
Check VNFs After Healing
- Variable Should Exist ${vnf_charm_id} msg=Charm VNF is not available
- Variable Should Exist ${vnf_volumes_id} msg=Volume VNF is not available
- ${id}= Get VNF VIM ID ${vnf_charm_id}
+ [Documentation] Check that the IDs of the VM and volumes have not changed after healing.
+ Variable Should Exist ${VNF_MANUALSCALE_ID} msg=Manual scale VNF is not available
+ Variable Should Exist ${VNF_VOLUMES_ID} msg=Volume VNF is not available
+ ${id}= Get VNF VIM ID ${VNF_MANUALSCALE_ID}
@{ids}= Split String ${id}
- Should Be Equal ${vdu_charm_ids}[0] ${ids}[0] VDU[0] id has changed after healing
- Should Not Be Equal ${vdu_charm_ids}[1] ${ids}[1] VDU[1] id has not changed after healing
- Should Not Be Equal ${vdu_charm_ids}[2] ${ids}[2] VDU[2] id has not changed after healing
-
- ${id}= Get VNF VIM ID ${vnf_volumes_id}
- Should Not Be Equal ${id} ${vdu_volumes_id} VDU id has not changed after healing
-
- ${ip}= Get Vdu Attribute ${vnf_charm_id} ip-address 1
- ${stdout}= Execute Remote Command Check Rc Return Output ${ip} ${username} ${password} ${privatekey} sudo ls ${vnf_charm_cloudinit_file}
- log ${stdout}
- Check If remote File Exists ${ip} ${username} ${password} ${privatekey} ${vnf_charm_day1_file}
-
+ Should Be Equal ${VDU_MANUALSCALE_IDS}[0] ${ids}[0] VDU[0] id has changed after healing
+ Should Not Be Equal ${VDU_MANUALSCALE_IDS}[1] ${ids}[1] VDU[1] id has not changed after healing
+ Should Not Be Equal ${VDU_MANUALSCALE_IDS}[2] ${ids}[2] VDU[2] id has not changed after healing
+ ${id}= Get VNF VIM ID ${VNF_VOLUMES_ID}
+ Should Not Be Equal ${id} ${VDU_VOLUMES_ID} VDU id has not changed after healing
+ ${ip}= Get Vdu Attribute ${VNF_MANUALSCALE_ID} ip-address 1
+ ${stdout}= Execute Remote Command Check Rc Return Output ${ip} ${USERNAME} ${PASSWORD} ${PRIVATEKEY} sudo ls ${VNF_MANUALSCALE_CLOUDINIT_FILE}
+ Log ${stdout}
+ Check If Remote File Exists ${ip} ${USERNAME} ${PASSWORD} ${PRIVATEKEY} ${VNF_MANUALSCALE_DAY1_FILE}
+
+Update VIM Objects
+ [Documentation] Retrieve all VMs and volumes from the NS and stores them in VIM_VDUS and VIM_VOLUMES lists.
+ ... This is done again to guarantee that all objects are cleaned in the VIM in case the heal operation
+ ... added new objects.
+ Variable Should Exist ${NS_ID} msg=NS is not available
+ @{vdu_updated}= Create List
+ @{vnf_id_list}= Get Ns Vnf List ${NS_ID}
+ FOR ${vnf_id} IN @{vnf_id_list}
+ ${id}= Get VNF VIM ID ${vnf_id}
+ @{vdu_ids}= Split String ${id}
+ Append To List ${vdu_updated} @{vdu_ids}
+ FOR ${id} IN @{vdu_ids}
+ IF not "${id}" in "@{VIM_VDUS}"
+ Append To List ${VIM_VDUS} ${id}
+ END
+ END
+ END
+ FOR ${vdu_id} IN @{vdu_updated}
+ ${volumes_attached}= Get Server Property ${vdu_id} volumes_attached
+ ${match}= Get Regexp Matches ${volumes_attached} '([0-9a-f\-]+)' 1
+ IF ${match} != @{EMPTY}
+ IF not "${match}[0]" in "@{VIM_VOLUMES}"
+ Append To List ${VIM_VOLUMES} ${match}[0]
+ END
+ END
+ END
+ Log Many @{VIM_VDUS}
+ Log Many @{VIM_VOLUMES}
Delete NS Instance
+ [Documentation] Delete NS instance.
[Tags] cleanup
- Delete NS ${ns_name}
-
+ Delete NS ${NS_NAME}
Delete NS Descriptor
+ [Documentation] Delete NS package from OSM.
[Tags] cleanup
- Delete NSD ${nsd_name}
-
+ Delete NSD ${NSD_NAME}
Delete VNF Descriptors
+ [Documentation] Delete VNF packages from OSM.
[Tags] cleanup
- Delete VNFD ${vnfd_volumes_name}
- Delete VNFD ${vnfd_charm_name}
+ Delete VNFD ${VNFD_VOLUMES_NAME}
+ Delete VNFD ${VNFD_MANUALSCALE_NAME}
+
+Delete Objects In VIM
+ [Documentation] Delete any remaining objects (volumes, VMs, etc.) in the VIM.
+ [Tags] cleanup
+ ${error}= Set Variable 0
+ FOR ${vol_id} IN @{VIM_VOLUMES}
+ Log Checking if volume ${vol_id} is still in VIM
+ ${exists}= Check If Volume Exists ${vol_id}
+ IF ${exists}
+ ${error}= Set Variable 1
+ Log Deleting volume ${vol_id}
+ Run Keyword And Ignore Error Delete Volume ${vol_id}
+ END
+ END
+ FOR ${vdu_id} IN @{VIM_VDUS}
+ Log Checking if server ${vdu_id} is still in VIM
+ ${status}= Run Keyword And Ignore Error Get Server Property ${vdu_id} id
+ Log ${status}[0]
+ IF '${status}[0]' == 'PASS'
+ ${error}= Set Variable 1
+ Log Deleting server ${vdu_id}
+ Run Keyword And Ignore Error Delete Server ${vdu_id}
+ END
+ END
+ IF ${error}==1
+ Fail Some objects created by test were not deleted in VIM
+ END
*** Keywords ***
Suite Cleanup
[Documentation] Test Suit Cleanup: Deleting Descriptor, instance and vim
-
- Run Keyword If Any Tests Failed Delete NS ${ns_name}
- Run Keyword If Any Tests Failed Delete NSD ${nsd_name}
- Run Keyword If Any Tests Failed Delete VNFD ${vnfd_volumes_name}
- Run Keyword If Any Tests Failed Delete VNFD ${vnfd_charm_name}
+ Run Keyword If Any Tests Failed Delete NS ${NS_NAME}
+ Run Keyword If Any Tests Failed Delete NSD ${NSD_NAME}
+ Run Keyword If Any Tests Failed Delete VNFD ${VNFD_VOLUMES_NAME}
+ Run Keyword If Any Tests Failed Delete VNFD ${VNFD_MANUALSCALE_NAME}
projects / osm / tests.git / blobdiff