Commit a1f2122b authored by Mark Beierl's avatar Mark Beierl
Browse files

Merge branch 'beierlm-master-patch-93208' into 'master'

Using Robot for Upgrade Testing

See merge request !100
parents 63b4bf6b 33b7b7f4
Loading
Loading
Loading
Loading
+118 −91
Original line number Diff line number Diff line
@@ -473,132 +473,159 @@ osm user-create test_member_1 --projects admin --project-role-mappings 'test_pro
osm user-update test_member_1 --remove-project-role 'admin,project_admin'
```

### Selection of Packages
### Running Robot Test Suite

In order to test that the upgrade does not impact existing operations, we will deploy a series of network services and slices prior to the upgrade and then verify all functionality post upgrade.  What follows is a list of packages and tests to run.  All packages will come from https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages
#### Install Docker (Charmed OSM Only)

Docker does not get installed with charmed OSM, so that needs to be installed first.
```bash
git clone -j4 --recursive https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git
sudo snap install docker
sudo addgroup --system docker
sudo adduser $USER docker
newgrp docker
sudo snap disable docker
sudo snap enable docker
sudo iptables -I DOCKER-USER -j ACCEPT
```

#### Basic OpenStack VNF
#### Prepare to Run Tests

##### Preparation
You should already have OSM_HOSTNAME and OSM_PASSWORD environment variables set, based on the output from the installer.

```bash
osm package-build hackfest_basic_ns
osm package-build hackfest_basic_vnf
osm upload-package hackfest_basic_vnf
osm upload-package hackfest_basic_ns
Source your openstack.rc file to get all your Openstack environment variables set.

##### Set Environment Variables (K8s Installation)

osm ns-create --ns_name hackfest_basic-ns \
    --nsd_name hackfest_basic-ns \
    --vim_account etsi-vim \
    --config \
    '{vld: [ {name: mgmtnet, vim-network-name: osm-ext} ] }'
```
export OSM_HOSTNAME=127.0.0.1
export PROMETHEUS_HOSTNAME=127.0.0.1
export PROMETHEUS_PORT=9091
export JUJU_PASSWORD=`juju gui 2>&1 | grep password | awk '{print $2}'`
export HOSTIP=127.0.1.1
```

#### Basic OpenStack VNF with Monitoring
##### Set Environment Variables (Charmed OSM)

##### Preparation
```bash
export OSM_HOSTNAME=$(juju config -m osm nbi site_url | sed "s/http.*\?:\/\///"):443
export PROMETHEUS_HOSTNAME=$(juju config -m osm prometheus site_url | sed "s/http.*\?:\/\///")
export PROMETHEUS_PORT=80
export JUJU_PASSWORD=`juju gui 2>&1 | grep password | awk '{print $2}'`
export HOSTIP=$(echo $PROMETHEUS_HOSTNAME | sed "s/prometheus.//" | sed "s/.nip.io//")
```

#### Create robot-systest.cfg
```bash
osm package-build hackfest_basic_metrics_vnf
osm package-build hackfest_basic_metrics_ns
osm upload-package ./hackfest_basic_metrics_vnf.tar.gz
osm upload-package ./hackfest_basic_metrics_ns.tar.gz
cat << EOF > robot-systest.cfg
VIM_TARGET=osm
VIM_MGMT_NET=osm-ext
ENVIRONMENTS_FOLDER=environments
PACKAGES_FOLDER=/robot-systest/osm-packages
OS_CLOUD=openstack
LC_ALL=C.UTF-8
LANG=C.UTF-8
EOF
for line in `env | grep "^OS_" | sort` ; do echo $line >> robot-systest.cfg ; done
```

osm ns-create --ns_name hackfest_basic_metrics_ns \
    --nsd_name hackfest_basic-ns-metrics \
    --vim_account etsi-vim \
    --config '{vld: [ {name: mgmtnet, vim-network-name: osm-ext} ] }'
#### Create robot.etc.hosts (Charmed OSM)
```bash
cat << EOF > robot.etc.hosts
127.0.0.1           localhost
${HOSTIP}      prometheus.${HOSTIP}.nip.io nbi.${HOSTIP}.nip.io
EOF
```

#### Basic OpenStack VNF with Proxy Charms
#### Create clouds.yaml

```bash
osm package-build charm-packages/ha_proxy_charm_vnf
osm package-build charm-packages/ha_proxy_charm_ns
osm upload-package charm-packages/ha_proxy_charm_vnf.tar.gz
osm upload-package charm-packages/ha_proxy_charm_ns.tar.gz
cat << EOF > clouds.yaml
clouds:
  openstack:
    auth:
      auth_url: $OS_AUTH_URL
      project_name: $OS_PROJECT_NAME
      username: $OS_USERNAME
      password: $OS_PASSWORD
      user_domain_name: $OS_USER_DOMAIN_NAME
      project_domain_name: $OS_PROJECT_DOMAIN_NAME
EOF
```

osm ns-create --ns_name ha_proxy_charm-ns \
--nsd_name ha_proxy_charm-ns \
--vim_account etsi-vim \
--config '{vld: [ {name: mgmtnet, vim-network-name: osm-ext} ] }'
#### Create VIM

osm ns-action ha_proxy_charm-ns \
    --vnf_name 1 \
    --action_name touch \
    --params '{filename: file-001.dat}'
Create a VIM called `osm`.  Be sure to add your specific configurations, such as floating
ip addresses, or untrusted SSL certificates.

```
osm vim-create --name osm --user "$OS_USERNAME" --password "$OS_PASSWORD" \
               --auth_url "$OS_AUTH_URL" --tenant "$OS_USERNAME" --account_type openstack \
               --config='{management_network_name: osm-ext}'
```

#### Basic OpenStack VNF with Native Charms
Provide a copy of your Kubernetes cluster configuration file to the Robot container.
```
export KUBECONFIG=/path/to/kubeconfig.yaml
```

```bash
osm package-build charm-packages/native_charm_vnf
osm package-build charm-packages/native_charm_ns
osm upload-package charm-packages/native_charm_vnf.tar.gz
osm upload-package charm-packages/native_charm_ns.tar.gz
#### Start Robot Tests Container
To keep a copy of the reports, create a directory for the container to store them.

osm ns-create --ns_name native_charm-ns \
    --nsd_name native_charm-ns \
    --vim_account etsi-vim \
    --config '{vld: [ {name: mgmtnet, vim-network-name: osm-ext} ] }'
```bash
mkdir reports
```

osm ns-action native_charm-ns \
    --vnf_name 1 \
    --vdu_id mgmtVM \
    --action_name touch \
    --params '{filename: file-001.dat}'
```bash
docker run -ti --entrypoint /bin/bash \
        --env OSM_HOSTNAME=${OSM_HOSTNAME} \
        --env PROMETHEUS_HOSTNAME=${PROMETHEUS_HOSTNAME} \
        --env PROMETHEUS_PORT=${PROMETHEUS_PORT} \
        --env JUJU_PASSWORD=${JUJU_PASSWORD} \
        --env HOSTIP=${HOSTIP} \
        --env OSM_PASSWORD=osm4u
        --env-file robot-systest.cfg \
        -v "$(pwd)/robot.etc.hosts":/etc/hosts \
        -v "${KUBECONFIG}":/root/.kube/config \
        -v "$(pwd)/clouds.yaml":/etc/openstack/clouds.yaml \
        -v "$(pwd)/reports":/robot-systest/reports \
        opensourcemano/tests:10
```

#### KNF with Helm
#### Run Tests Pre-Upgrade

From the Robot tests container command line, execute the prepare step.

```bash
cat << EOF > ~/openldap-params.yaml
./run_test.sh -t prepare
```

vld:
- name: mgmtnet
  vim-network-name: osm-ext
additionalParamsForVnf:
- member-vnf-index: openldap
  additionalParamsForKdu:
  - kdu_name: ldap
    additionalParams:
      adminPassword: osm4u
      configPassword: osm4u
      env:
        LDAP_ORGANISATION: "Example Inc."
        LDAP_DOMAIN: "example.org"
        LDAP_BACKEND: "hdb"
        LDAP_TLS: "true"
        LDAP_TLS_ENFORCE: "false"
        LDAP_REMOVE_CONFIG_AFTER_SETUP: "true"
EOF
After the run has completed successfully, there should be 5 network services present
in OSM:

osm package-build openldap_knf
osm package-build openldap_ns
osm upload-package ./openldap_knf.tar.gz
osm upload-package ./openldap_ns.tar.gz
```
+---------------------------------+--------------------------------------+---------------------+----------+-------------------+---------------+
| ns instance name                | id                                   | date                | ns state | current operation | error details |
+---------------------------------+--------------------------------------+---------------------+----------+-------------------+---------------+
| basic_07_secure_key_management  | 1a8621ea-d51d-434c-90e0-e153701729dd | 2022-08-03T17:51:07 | READY    | IDLE (None)       | N/A           |
| basic_09_manual_scaling_test    | 7611ce54-bff2-480e-94fb-5a8b0549a6c4 | 2022-08-03T17:54:32 | READY    | IDLE (None)       | N/A           |
| basic_21                        | 8090754c-f49c-4891-a7c0-1e5750c7980b | 2022-08-03T17:55:30 | READY    | IDLE (None)       | N/A           |
| k8s_06-nopasswd_k8s_proxy_charm | a5eb22d7-4a4f-4615-ad44-9f8957cf243c | 2022-08-03T18:17:01 | READY    | IDLE (None)       | N/A           |
| ldap                            | be0f6e33-e4d9-463d-92c6-cc27f2f1d5eb | 2022-08-03T18:51:51 | READY    | IDLE (None)       | N/A           |
+---------------------------------+--------------------------------------+---------------------+----------+-------------------+---------------+
```

osm ns-create --ns_name openldap_ns \
    --nsd_name openldap_ns \
    --vim_account etsi-vim \
    --config_file ~/openldap-params.yaml
Run the verify step before upgrading:
```bash
./run_test.sh -t verify
```

#### KNF with Juju
#### Run Tests Post-Upgrade

```bash
osm package-build squid_metrics_cnf
osm package-build squid_metrics_cnf_ns
osm upload-package ./squid_metrics_cnf.tar.gz
osm upload-package ./squid_metrics_cnf_ns.tar.gz
After completing the upgrade procedure, execute the verify step again to ensure the upgrade
was successful

osm ns-create --ns_name squid_cnf_ns \
    --nsd_name squid_cnf_ns \
    --vim_account dimension \
    --config '{vld: [ {name: mgmtnet, vim-network-name: osm-ext} ] }'
```bash
./run_test.sh -t verify
```

This will only verify services that were already deployed in the prepare step.