Update tags in all testsuites

[osm/tests.git] / README.md

diff --git a/README.md b/README.md
index 065b878..2556456 100644 (file)
--- a/README.md
+++ b/README.md
@@ -32,52 +32,52 @@ This repository contains tools and configuration files for testing and automatio
 This bash script can be used to setup your environment to execute the tests.
 
 ```bash
-   PACKAGES_FOLDER=osm-packages
-   add-apt-repository -y ppa:rmescandon/yq && apt update && apt install yq git iputils-ping ssh -y
-   pip install haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary \
-      robotframework-sshlibrary
-   snap install charm
-   # Download community packages
-   git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER}
+PACKAGES_FOLDER=osm-packages
+add-apt-repository -y ppa:rmescandon/yq && apt update && apt install yq git iputils-ping ssh -y
+pip install haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary \
+    robotframework-sshlibrary
+snap install charm
+# Download community packages
+git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER}
 ```
 
 envfile.rc
 
 ```bash
-   # VIM Setup
-   OS_USERNAME=<openstack_username>
-   OS_PASSWORD=<openstack_password>
-   OS_TENANT_NAME=<openstack_tenant_name>
-   OS_AUTH_URL=<openstack_authorization_url>
-   OS_TENANT_ID=<openstack_tenant_id>
+# VIM Setup
+OS_USERNAME=<openstack_username>
+OS_PASSWORD=<openstack_password>
+OS_TENANT_NAME=<openstack_tenant_name>
+OS_AUTH_URL=<openstack_authorization_url>
+OS_TENANT_ID=<openstack_tenant_id>
 
-   # OSM Setup
-   OSM_HOSTNAME=<osm_ip_address>
-   VIM_TARGET=<osm_vim_name>
-   VIM_MGMT_NET=<osm_vim_mgmt_name>
+# OSM Setup
+OSM_HOSTNAME=<osm_ip_address>
+VIM_TARGET=<osm_vim_name>
+VIM_MGMT_NET=<osm_vim_mgmt_name>
 
-   # Clouds file datacenter
-   OS_CLOUD=<datacenter_in_clouds_file>
-   # SDNCs file
-   OS_SDNC=<SDN_controller_in_sdncs_file>
+# Clouds file datacenter
+OS_CLOUD=<datacenter_in_clouds_file>
+# SDNCs file
+OS_SDNC=<SDN_controller_in_sdncs_file>
 
-   # K8S config file
-   K8S_CREDENTIALS=<path_to_kubeconfig>
+# K8S config file
+K8S_CREDENTIALS=<path_to_kubeconfig>
 
-   # The following set of environment variables will be used in host
-   # of the robot framework. Not needed for docker execution
+# The following set of environment variables will be used in host
+# of the robot framework. Not needed for docker execution
 
-   # Folder where Robot tests are stored
-   ROBOT_DEVOPS_FOLDER=robot-systest
+# Folder where Robot tests are stored
+ROBOT_DEVOPS_FOLDER=robot-systest
 
-   # Folder to save alternative DUT environments (optional)
-   ENVIRONMENTS_FOLDER=environments
+# Folder to save alternative DUT environments (optional)
+ENVIRONMENTS_FOLDER=environments
 
-   # Folder where all required packages are stored
-   PACKAGES_FOLDER=osm-packages
+# Folder where all required packages are stored
+PACKAGES_FOLDER=osm-packages
 
-   # Folder where test results should be exported
-   ROBOT_REPORT_FOLDER=results
+# Folder where test results should be exported
+ROBOT_REPORT_FOLDER=results
 ```
 
 ## Deployment
@@ -105,7 +105,7 @@ Volumes:
 * <path_to_kubeconfig> [OPTIONAL]: It is the kubeconfig file to be used for k8s clusters
 
 ```bash
-   docker run --rm=true -t osmtests --env-file <env_file> \
+docker run --rm=true -t osmtests --env-file <env_file> \
        -v <path_to_reports>:/reports osmtests \
        -v <path_to_clouds.yaml>:/robot-systest/clouds.yaml \
        -v <path_to_sdncs.yaml>:/robot-systest/sdncs.yaml \
@@ -120,10 +120,45 @@ Volumes:
 The way of executing the tests is via the following command:
 
 ```bash
-   source envfile.rc
-   robot -d reports -i <testing_tags> testsuite/
+source envfile.rc
+robot -d reports -i <testing_tags> testsuite/
 ```
 
+## Test tags
+
+All tests in the testsuites have tags. Tags allow to run only a set of tests
+identified by a tag. Several tags can be specified when running robot in the
+following way:
+
+```bash
+robot -i <tag_01> -i <tag_02> testsuite/
+```
+
+The following tags exist for each testsuite:
+
+- A tag per testsuite using its mnemonic (e.g. `basic_01`)
+- Cluster tag for each of the statistically similar tests:
+  - `cluster_main`: `basic_01`, `basic_05`, `basic_08`, `basic_09`, `basic_15`,
+    `basic_16`, basic_17`, `hackfest_basic`, `hackfest_multivdu`,
+    `hackfest_cloudinit`, `quotas_01`
+  - `cluster_ee_config`: `basic_06`, `basic_07`, `basic_11`, `basic_12`,
+    `basic_13`, `basic_14`, `k8s_05`, `k8s_06`
+  - `cluster_relations`: `basic_11`, `basic_13`, `basic_14`
+  - `cluster_epa`: `epa_01`, `epa_02`, `epa_03`, `epa_04`, `epa_05`
+  - `cluster_k8s`: `k8s_01`, `k8s_02`, `k8s_03`, `k8s_04`, `k8s_07`, `sa_08`
+  - `cluster_k8s_charms`: `k8s_05`, `k8s_06`
+  - `cluster_sa`: `sa_01`, `sa_02`, `sa_07`
+  - `cluster_slices`: `slice_01`, `slice_02`
+- daily: for all testsuites that will run in the daily job
+- regression: for all testsuites that should pass in the current stable branch
+- sanity: for all testsuites that should be passed by each commit in the
+  stage3 to be successfully verified by Jenkins, currently `basic_07`,
+  `basic_11`, `k8s_03`, `k8s_04`, `sa_02`
+
+In addition, the tag "cleanup" exists in those tests that perform
+any deletion. In that way, it can be invoked to retry the deletion if
+the tests were forcefully stopped.
+
 ## Built With
 
 * [Python](www.python.org/) - The language used