Updated to ease tracking of Rel EIGHT tests:

### \[BASIC-10\] VNF SW upgrade

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Download and import the `hackfest3-mgmt` image

   ```bash

   wget https://osm-download.etsi.org/ftp/osm-3.0-three/1st-hackfest/images/hackfest3-mgmt.qcow2

   openstack image create \

      --disk-format qcow2 \

      --file hackfest3-mgmt.qcow2 \

      hackfest3-mgmt

   ```

2. Download and onboard the descriptors

   ```bash

   osm vnfd-create hackfest_simplecharm_vnf.tar.gz

   osm nsd-create hackfest_simplecharm_ns.tar.gz

   ```

3. Instantiate the network service:

   ```bash

   osm ns-create --ns_name simple --nsd_name hackfest-simplecharm-ns --vim_account    <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID> --config '{vld: [{name: mgmtnet, vim-network-name: <MGMT    NETWORK>}]}'

   ```

4. Wait for the network service to be complete/complete

   ```bash

   osm ns-list

   ```

5. For each of the following day-2 primitives: start, stop, restart, upgrade:

   ```bash

   osm ns-action simple --vnf_name 1 --action_name <primitive> --params '{}' (Returns an operation    ID)

   ```

6. Verify operationState is COMPLETED

   ```bash

   osm ns-op-list simple

   osm ns-op-show <operational ID>

   ```

### \[BASIC-11\] Native charms

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Download and import the following packages:

   - [`Native ns.tar.gz`](https://osm.etsi.org/wiki/index.php/File:Native_ns.tar.gz)

   - [`Native vnf.tar.gz`](https://osm.etsi.org/wiki/index.php/File:Native_vnf.tar.gz)

2. Onboard the packages:

   ```bash

   osm upload-package Native_vnf.tar.gz

   osm upload-package Native_ns.tar.gz

   ```

3. Onboard the Ubuntu image to your VIM. E.g.:

   ```bash

   wget https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img

   openstack image create --file="./xenial-server-cloudimg-amd64-disk1.img" \

      --container-format=bare \

      --disk-format=qcow2 \

      ubuntu-16.04

   ```

4. Instantiate the Network Service:

   ```bash

   osm ns-create --ns_name test_native_charm --nsd_name native-ns --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

   ```

5. Wait for operational to become `running`, config status to be `configured`, and detailed status to be `done`.

6. Switch to the Juju model for this network service (the id of the ns instance via `osm ns-list`).

   ```bash

   juju switch ce024838-5c81-47b6-8976-adcb9d17ef57

   ```

7. Run `juju status`:

   ```text

   Model                                 Controller  Cloud/Region         Version  SLA          Timestamp

   ce024838-5c81-47b6-8976-adcb9d17ef57  osm         localhost/localhost  2.5.8    unsupported  20:38:24Z

   App               Version  Status  Scale  Charm      Store  Rev  OS      Notes

   test-b-mgmtvm-aa           active      1  native-ci  local    0  ubuntu  

   Unit                 Workload  Agent  Machine  Public address  Ports  Message

   test-b-mgmtvm-aa/0*  active    idle   0        172.21.248.95          Ready!

   Machine  State    DNS            Inst id               Series  AZ  Message

   0        started  172.21.248.95  manual:172.21.248.95  xenial      Manually provisioned machine

   ```

8. Verify that the `test-b-mgmtvm-aa/0` unit is active.

9. Verify that machine 0's instance id begins with `manual`.

### \[BASIC-12\] NS primitives

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Download and import the following packages:

   - [`Nscharm ns.tar.gz`](https://osm.etsi.org/wiki/index.php/File:Nscharm_ns.tar.gz)

   - [`Nscharm policy vnf.tar.gz`](https://osm.etsi.org/wiki/index.php/File:Nscharm_policy_vnf.tar.gz)

   - [`Nscharm user vnf.tar.gz`](https://osm.etsi.org/wiki/index.php/File:Nscharm_user_vnf.tar.gz)

2. Update the NS package

   - The ns package needs to be updated with the Juju credentials for your OSM installation.

     ```bash

     tar xf Nscharm_ns.tar.gz

     vi nscharm_ns/nscharm_nsd.yaml

     ```

   - Replace the value of `juju-password` with the one from your `~/.local/share/juju/accounts.yaml` and save:

     ```bash

     tar cfz Nscharm_ns.tar.gz nscharm_ns/

     ```

3. Onboard the packages:

   ```bash

   osm upload-package Nscharm_policy_vnf.tar.gz

   osm upload-package Nscharm_user_vnf.tar.gz

   osm upload-package Nscharm_ns.tar.gz

   ```

4. Instantiate the Network Service. Note: if you change the `ns_name`, you'll need to update the NSD's ns-configuration with the same name.

   ```bash

   osm ns-create --ns_name test_nscharm --nsd_name nscharm-ns --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

   ```

5. Wait for operational to become `running`, config status to be `configured`, and detailed status to be `done`.

6. Switch to the Juju model for this network service (the id of the ns instance via `osm ns-list`)

   ```bash

   juju switch 510086c6-99cb-466f-9214-e36f2f52c7ab

   ```

7. Check the status of primitives run by the initial-config-primitive:

   ```bash

   juju show-action-status

   ```

8. Validate that the actions were successful:

   ```yaml

   actions:

   - action: verify-ssh-credentials

     completed at: "2019-12-11 07:57:15"

     id: "1"

     status: completed

     unit: app-vnf-b24be8a7d707-vdu-uservm-cnt-z0/0

   - action: verify-ssh-credentials

     completed at: "2019-12-11 07:57:15"

     id: "2"

     status: completed

     unit: app-vnf-70e9ebb7ba72-vdu-policyvm-cnt-z0/0

   - action: add-user

     completed at: "2019-12-11 07:57:46"

     id: "3"

     status: completed

     unit: app-z/0

   - action: add-user

     completed at: "2019-12-11 07:57:25"

     id: "4"

     status: completed

     unit: app-vnf-b24be8a7d707-vdu-uservm-cnt-z0/0

   - action: set-policy

     completed at: "2019-12-11 07:57:38"

     id: "5"

     status: completed

     unit: app-vnf-70e9ebb7ba72-vdu-policyvm-cnt-z0/0

   ```

### \[BASIC-13\] NS relations

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Download the following packages:

   ```bash

   wget https://osm-download.etsi.org/ftp/Packages/hackfests/mysql_epa_vnf.tar.gz

   wget https://osm-download.etsi.org/ftp/Packages/hackfests/wordpress_epa_vnf.tar.gz

   wget https://osm-download.etsi.org/ftp/Packages/hackfests/wordpress_mysql_epa_ns.tar.gz

   ```

2. Onboard the packages:

   ```bash

   osm upload-package mysql_epa_vnf.tar.gz

   osm upload-package wordpress_epa_vnf.tar.gz

   osm upload-package wordpress_mysql_epa_ns.tar.gz

   ```

3. Instantiate the Network Service:

   ```bash

   osm ns-create --ns_name nsrelations --nsd_name wordpress_mysql_epa --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

   ```

5. Wait for operational to become `running`, config status to be `configured`, and detailed status to be `done`.

6. Switch to the Juju model for this network service (the id of the ns instance via `osm ns-list`):

   ```bash

   juju switch ce024838-5c81-47b6-8976-adcb9d17ef57

   ```

7. Run `juju status --relations`

8. Verify that the relation is listed.

### \[BASIC-14\] VNF relations

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Download the following packages:

   ```bash

   wget https://osm-download.etsi.org/ftp/Packages/hackfests/native_blog_vnf.tar.gz

   wget https://osm-download.etsi.org/ftp/Packages/hackfests/native_blog_ns.tar.gz

   ```

2. Onboard the packages:

   ```bash

   osm upload-package native_blog_vnf.tar.gz

   osm upload-package native_blog_ns.tar.gz

   ```

3. Instantiate the Network Service:

   ```bash

   osm ns-create --ns_name vnfrelations --nsd_name native_blog_ns --vim_account <VIM_ACCOUNT_NAME>|<VIM_ACCOUNT_ID>

   ```

4. Wait for operational to become `running`, config status to be `configured`, and detailed status to be `done`.

5. Switch to the Juju model for this network service (the id of the ns instance via `osm ns-list`):

   ```bash

   juju switch ce024838-5c81-47b6-8976-adcb9d17ef57

   ```

6. Run `juju status --relations`

7. Verify that the relation is listed in the bottom of the status message.

### \[BASIC-15\] RBAC setup

### \[BASIC-16\] RBAC enforcement

#### 1. Preparation of the environment

### \[BASIC-17\] Quotas per project

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

This test will check project creation edition and isolation. It tests the quotas per project. Finally it tests that projects cannot be deleted with dependencies.

Optional, for enabling Keystone backend (otherwise it will use internal backend): restart NBI docker with the configurations for using Keystone:

1. Create project with quotas. Tests `osmclient` with both comma-separated list and repetition of `--quotas` option:

```bash

   osm project-create p3 --quotas vnfds=1,nsds=1 --quotas ns_instances=1

   osm project-show p3 | grep -q -e '"vnfds": 1' || echo "FAIL"

   osm project-show p3 | grep -q -e '"nsds": 1' || echo "FAIL"

   osm project-show p3 | grep -q -e '"ns_instances": 1' || echo "FAIL"

docker service update osm_nbi --force \

  --env-add OSMNBI_AUTHENTICATION_BACKEND=keystone \

  --env-add OSMNBI_AUTHENTICATION_AUTH_URL=keystone \

  --env-add OSMNBI_AUTHENTICATION_AUTH_PORT=5000 \

  --env-add OSMNBI_AUTHENTICATION_USER_DOMAIN_NAME=default \

  --env-add OSMNBI_AUTHENTICATION_PROJECT_DOMAIN_NAME=default \

  --env-add OSMNBI_AUTHENTICATION_SERVICE_USERNAME=nbi \

  --env-add OSMNBI_AUTHENTICATION_SERVICE_PROJECT=service

```

#### 2. User operations

   osm project-create p2 --quotas vnfds=1,nsds=1,ns_instances=1

   osm project-show p2 | grep -q -e '"vnfds": 1' || echo "FAIL"

   osm project-show p2 | grep -q -e '"nsds": 1' || echo "FAIL"

   osm project-show p2 | grep -q -e '"ns_instances": 1' || echo "FAIL"

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

2. Create user and assign both projecst with `project_admin` role:

1. Create a user

   ```bash

   osm user-create u --password u --project-role-mappings p3,project_admin --project-role-mappings p2,project_admin

   osm user-create test_user --password test_password

   ```

3. Change project name:

2. List users and verify that `test_user` is present:

   ```bash

   osm project-update p3 --name p1

   osm project-show p1 | grep -q -e '"vnfds": 1' || echo "FAIL"

   osm project-show p1 | grep -q -e '"nsds": 1' || echo "FAIL"

   osm project-show p1 | grep -q -e '"ns_instances": 1' || echo "FAIL"

   osm user-show u | grep -q -e '"p3"' && echo "FAIL"   # user uses renamed project 'p1', not 'p3'

   osm user-list

   ```

4. List packages (list must be empty) without any uuid. Using regular expression for grep:

3. Check the information for `test_user` using username:

   ```bash

   osm --project p1 --password u --user u vnfpkg-list | grep -q -E "[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12}" && echo "FAIL"

   osm user-show test_user

   ```

5. Add packages

4. Check the information for `test_user` using user ID:

   ```bash

   osm --project p1 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=v1;name=v1" || echo "FAIL"

   osm user-show <test_user_ID>

   ```

6. Force a failure for for exceeded quota:

5. Update `test_user` and verify the new information:

   ```bash

   osm --project p1 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=v2;name=v2" && echo "FAIL"

   osm user-update --set-project admin,system_admin test_user

   osm user-update --add-project-role 'admin,project_user' test_user

   osm user-update --add-project-role 'system_admin,project_user' test_user

   osm user-show test_user

   ```

7. Enlarge quota:

6. Try to authenticate with the user and run an operation:

   ```bash

   osm project-update p1 --quotas "vnfds=10,nsds=,ns_instances="

   osm project-show p1 | grep -q -e '"vnfds": 10' || echo "FAIL"

   osm project-show p1 | grep -q -e '"nsds":' && echo "FAIL"

   export OSM_USER=test_user

   export OSM_PROJECT=admin

   export OSM_PASSWORD=test_password

   osm ns-list

   ```

8. Add more packages:

7. Delete `test_user` and verify it has been deleted:

   ```bash

   for i in {2..10} ; do

     osm --project p1 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=v${i};name=v${i}" || echo "FAIL"

   done

   osm user-delete test_user

   osm user-list

   ```

9. Another `vnfpkg` created should fail for quota limits:

#### 3. Project operations

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Create `test_project`:

   ```bash

   osm --project p1 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=v11;name=v11" && echo "FAIL"

   osm project-create test_project

   ```

10. Switch to project 2

2. List projects:

   ```bash

    osm --project p2 --password u --user u vnfpkg-list | grep -q -E "[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12}" && echo "FAIL"

   osm project-list

   ```

11. Test quota in project 2. Only one package is allowed

3. Show `test_project` information:

   ```bash

    osm --project p2 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=vp2_1;name=vp2_1"

    osm --project p2 --password u --user u vnfpkg-create hackfest_basic_vnf.tar.gz --override "id=vp2_2;name=vp2_2" && echo "FAIL"

   osm project-show test_project

   ```

12. Check that one project cannot see the scope of other projects:

4. Show `test_project` information using project ID:

   ```bash

    osm --project p2 --password u --user u --all-projects vnfpkg-list && echo "FAIL"  # not authorized

    osm --project p1 --password u --user u vnfpkg-show vp2_1  && echo "FAIL"

    osm --project p2 --password u --user u vnfpkg-show v1  && echo "FAIL"

   osm project-show <test_project_ID>

   ```

13. Delete. Test dependencies:

    1. Remove packages

5. Update `test_project` information:

   ```bash

       for i in {1..10} ; do

         osm --project p1 --password u --user u vnfpkg-delete v${i}

       done

       osm --project p2 --password u --user u vnfpkg-delete vp2_1

   osm project-update --name project_test test_project

   osm project-list

   ```

    2. Delete project will fail because in use by user 'u'

6. Delete `project_test`:

   ```bash

       osm project-delete p2 && echo "FAIL"

   osm project-delete project_test

   osm project-list

   ```

       osm user-update u --remove-project p2  # remove project from user

       osm user-show u | grep -q -e '"p2"' && echo "FAIL"

       osm project-delete p2 || echo "FAIL"  # p2 can be deleted

       osm project-delete p1 && echo "FAIL"  # p1 still in use by user 'u'

#### 4. Role operations

       osm user-delete u

       osm project-delete p1

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

       osm project-list | grep -e p1 -e p2 && echo "FAIL"

1. Create `test_role`:

   ```bash

   osm role-create test_role

   ```

2. List roles:

   ```bash

   osm role-list

   ```

3. Show `test_role` information:

   ```bash

   osm role-show test_role

   ```

4. Show `test_role` information using role ID:

   ```bash

   osm role-show <test_role_ID>

   ```

5. Update `test_project` information:

   ```bash

   osm role-update --add "vims: true" test_role

   osm role-show test_role

   ```

6. Delete `project_test`:

   ```bash

   osm role-delete test_role

   osm role-list

   ```

### \[BASIC-16\] RBAC enforcement

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

#### Visibility of packages and instances

1. Check you have a `test_user` and `test_project` from previous test.

2. Run the following commands to assign administrative roles to the `test_user` over the `test_project`:

   ```bash

   osm user-update --add-project-role 'admin,project_user' test_user

   osm user-update --add-project-role 'test_project,account_manager' test_user

   ```

3. Export the following variables to authenticate with the new user:

   ```bash

   export OSM_USERNAME=test_user

   export OSM_PASSWORD=test_password

   export OSM_PROJECT=test_project

   ```

4. Create a VIM and instantiate any NS/VNF and validate that packages and instances are independent between the test_project and the admin project:

   ```bash

   osm vim-create ...

   wget https://osm-download.etsi.org/ftp/osm-3.0-three/examples/cirros_2vnf_ns/cirros_2vnf_ns.tar.gz

   wget https://osm-download.etsi.org/ftp/osm-3.0-three/examples/cirros_2vnf_ns/cirros_vnf.tar.gz

   osm vnfd-create cirros_vnf.tar.gz

   osm nsd-create cirros_2vnf_ns.tar.gz

   osm ns-create --nsd_name cirros_2vnf_ns --ns_name  test --vim_account [ VIM_NAME ]

   ```

## Service Assurance

sudo ip address list

```

## Platform resiliency and recovery

### [\RESILIENCY-01\] OSM platform resiliency

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

In case of installation based on Docker Swarm:

1. Check that OSM is working:

   ```bash

   osm vnfpkg-list

   osm vim-list

   ```

2. Check that all services are up and running (all replicas should be 1/1):

   ```bash

   docker service ls

   ```

3. Kill abruptly all docker containers:

   ```bash

   for i in {osm_keystone.1,osm_keystone-db.1,osm_lcm.1,osm_light-ui.1,osm_mon.1,osm_mongo.1,osm_nbi.1,osm_pol.1,osm_prometheus.1,osm_ro.1,osm_ro-db.1}; do

     docker rm -f $(docker ps |grep -i $i|awk '{print $1}')

   done

   ```

4. Check that all services recover (all replicas should be `1/1` after enough time ~5 minutes):

   ```bash

   docker service ls

   ```

5. Check that OSM is working again:

   ```bash

   osm vnfpkg-list

   osm vim-list

   ```

In case of installation based on K8s, the command to kill abruptly all containers (step 3) should be:

```bash

for i in {k8s_keystone,k8s_mysql,k8s_lcm,k8s_light-ui,k8s_mon_,k8s_mongo,k8s_nbi,k8s_pol,k8s_prometheus,k8s_ro}; do

  docker rm -f $(docker ps |grep -i $i|awk '{print $1}')

done

```

### [\RESILIENCY-02\] OSM platform recovery

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

In case of installation based on Docker Swarm:

1. Check that OSM is working:

   ```bash

   osm vnfpkg-list

   osm vim-list

   ```

2. Check that all services are up and running (all replicas should be `1/1`):

   ```bash

   docker service ls

   ```

3. Kill abruptly all docker containers:

   ```bash

   for i in {osm_kafka.1,osm_keystone.1,osm_keystone-db.1,osm_lcm.1,osm_light-ui.1,osm_mon.1,osm_mongo.1,osm_nbi.1,osm_pol.1,osm_prometheus.1,osm_ro.1,osm_ro-db.1,osm_zookeeper.1}; do

     docker rm -f $(docker ps |grep -i $i|awk '{print $1}')

   done

   ```

4. Check that all services recover (all replicas should be 1/1 after enough time ~5 minutes):

   ```bash

   docker service ls

   ```

5. Check that OSM is working again:

   ```bash

   osm vnfpkg-list

   osm vim-list

   ```

In case of installation based on K8s, the command to kill abruptly all containers (step 3) should be:

```bash

for i in {k8s_kafka,k8s_keystone,k8s_mysql,k8s_lcm,k8s_light-ui,k8s_mon_,k8s_mongo,k8s_nbi,k8s_pol,k8s_prometheus,k8s_ro,k8s_zookeeper}; do

  docker rm -f $(docker ps |grep -i $i|awk '{print $1}')

done

```

## Testing of optional or environment-dependent features

### \[OPTIONAL-01\] Support of multi-segment VIM-managed networks

   > tcpdump -i <Interface with 10.10.10.X network>

   ```

### \[OPTIONAL-03\] Alternate images depending on VIM

```diff

-This test description is ported as-is from the old test plan and might require further revision.

-In case of finding any issue during test execution, please report to OSM_TECH@LIST.ETSI.ORG.

```

1. Onboard the packages:

   ```bash

   osm vnfd-create hackfest_basic_vnf.tar.gz

   osm nsd-create hackfest_basic_ns.tar.gz

   ```

2. Deploy in any VIM:

   ```bash

   osm ns-create --ns_name hackfestVIM1 --nsd_name hackfest_basic-ns --vim_account    <VIM1_ACCOUNT_NAME>|<VIM1_ACCOUNT_ID> --ssh_keys

   <KEY1_PUBKEY_FILE> --config '{vld: [ {name: mgmtnet, vim-network-name: <VIM1_MGMT_NAME>} ] }'

   osm ns-list

   osm ns-show hackfestVIM1

   ssh -i <KEY1_PRIVKEY_FILE> ubuntu@<MGMT_IP_VIM1>

   ```

3. Deploy in an AWS VIM (the VNF package indicates an alternative image for it):

   ```bash

   osm ns-create --ns_name hackfestVIM2 --nsd_name hackfest_basic-ns --vim_account    <VIM2_ACCOUNT_NAME>|<VIM2_ACCOUNT_ID> --ssh_keys

   <KEY1_PUBKEY_FILE> --config '{vld: [ {name: mgmtnet, vim-network-name: <VIM2_MGMT_NAME>} ] }'

   osm ns-list

   osm ns-show hackfestVIM2

   ssh -i <KEY1_PRIVKEY_FILE> ubuntu@<MGMT_IP_VIM2>

   osm ns-delete hackfestVIM1

   osm ns-delete hackfestVIM2

   ```

## References

- \[1\]: [OSM Information Model](https://osm.etsi.org/wikipub/index.php/OSM_Information_Model)