See the License for the specific language governing permissions and
limitations under the License
-->
-# Project Title
-One Paragraph of project description goes here
+# OSM test automation project - osm/tests
-## Getting Started
+This repository contains tools and configuration files for testing and automation needs of OSM projet
-These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
+## Prerequisites
-### Prerequisites
+* **Robot Framework**
+* **Packages**: ssh ping yq git
+* **Python3 packages**: haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary robotframework-sshlibrary
+* Clone **osm-packages** from gitlab
+* Environment config file for your infrastructure [envfile.rc]
-What things you need to install the software and how to install them
+## Installing
-```
-Give examples
+This bash script can be used to setup your environment to execute the tests.
+
+```bash
+PACKAGES_FOLDER=osm-packages
+add-apt-repository -y ppa:rmescandon/yq && apt update && apt install yq git iputils-ping ssh -y
+pip install haikunator requests robotframework robotframework-seleniumlibrary robotframework-requests robotframework-jsonlibrary \
+ robotframework-sshlibrary
+snap install charm
+# Download community packages
+git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER}
```
-### Installing
+envfile.rc
-A step by step series of examples that tell you how to get a development env running
+```bash
+# VIM Setup
+OS_USERNAME=<openstack_username>
+OS_PASSWORD=<openstack_password>
+OS_TENANT_NAME=<openstack_tenant_name>
+OS_AUTH_URL=<openstack_authorization_url>
+OS_TENANT_ID=<openstack_tenant_id>
-Say what the step will be
+# OSM Setup
+OSM_HOSTNAME=<osm_ip_address>
+VIM_TARGET=<osm_vim_name>
+VIM_MGMT_NET=<osm_vim_mgmt_name>
-```
-Give the example
-```
+# Clouds file datacenter
+OS_CLOUD=<datacenter_in_clouds_file>
+# SDNCs file
+OS_SDNC=<SDN_controller_in_sdncs_file>
-And repeat
+# K8S config file
+K8S_CREDENTIALS=<path_to_kubeconfig>
-```
-until finished
+# The following set of environment variables will be used in host
+# of the robot framework. Not needed for docker execution
+
+# Folder where Robot tests are stored
+ROBOT_DEVOPS_FOLDER=robot-systest
+
+# Folder to save alternative DUT environments (optional)
+ENVIRONMENTS_FOLDER=environments
+
+# Folder where all required packages are stored
+PACKAGES_FOLDER=osm-packages
+
+# Folder where test results should be exported
+ROBOT_REPORT_FOLDER=results
```
-End with an example of getting some data out of the system or using it for a little demo
+## Deployment
-## Running the tests
+It is possible to run the tests directly from the repository or using a docker container with the tests
-Explain how to run the automated tests for this system
+1. Docker container creation:
+```bash
+docker build -f docker/Dockerfile -t osmtests .
```
-Give an example
+
+Options:
+
+* --env-file: It is the environmental file where is described the OSM target and VIM
+* -o <osmclient_version> [OPTIONAL]: It is used to specify a particular osmclient version. Default: latest
+* -p <package_branch> [OPTIONAL]: OSM packages repository branch. Default: master
+* -t <testing_tags> [OPTIONAL]: Robot tests tags. [sanity, regression, particular_test]. Default: sanity
+
+Volumes:
+
+* <path_to_reports> [OPTIONAL]: It is the absolute path to reports location in the host
+* <path_to_clouds.yaml> [OPTIONAL]: It is the absolute path to the clouds.yaml file in the host
+* <path_to_sdncs.yaml> [OPTIONAL]: It is the absolute path to the sdncs.yaml file in the host
+* <path_to_kubeconfig> [OPTIONAL]: It is the kubeconfig file to be used for k8s clusters
+
+```bash
+docker run --rm=true -t osmtests --env-file <env_file> \
+ -v <path_to_reports>:/reports osmtests \
+ -v <path_to_clouds.yaml>:/robot-systest/clouds.yaml \
+ -v <path_to_sdncs.yaml>:/robot-systest/sdncs.yaml \
+ -v <path_to_kubeconfig>:/root/.kube/config \
+ -o <osmclient_version> \
+ -p <package_branch> \
+ -t <testing_tags>
```
-## Deployment
+1. Running the tests manually:
+
+The way of executing the tests is via the following command:
+
+```bash
+source envfile.rc
+robot -d reports -i <testing_tags> testsuite/
+```
+
+## Test tags
-Add additional notes about how to deploy this on a live system
+All tests in the testsuites have tags. Tags allow to run only a set of tests
+identified by a tag. Several tags can be specified when running robot in the
+following way:
+
+```bash
+robot -i <tag_01> -i <tag_02> testsuite/
+```
+
+The following tags exist for each testsuite:
+
+- A tag per testsuite using its mnemonic (e.g. `basic_01`)
+- Cluster tag for each of the statistically similar tests:
+ - `cluster_main`: `basic_01`, `basic_05`, `basic_08`, `basic_09`, `basic_15`,
+ `basic_16`, basic_17`, `hackfest_basic`, `hackfest_multivdu`,
+ `hackfest_cloudinit`, `quotas_01`
+ - `cluster_ee_config`: `basic_06`, `basic_07`, `basic_11`, `basic_12`,
+ `basic_13`, `basic_14`, `k8s_05`, `k8s_06`
+ - `cluster_relations`: `basic_11`, `basic_13`, `basic_14`
+ - `cluster_epa`: `epa_01`, `epa_02`, `epa_03`, `epa_04`, `epa_05`
+ - `cluster_k8s`: `k8s_01`, `k8s_02`, `k8s_03`, `k8s_04`, `k8s_07`, `k8s_08`, `sa_08`
+ - `cluster_k8s_charms`: `k8s_05`, `k8s_06`
+ - `cluster_sa`: `sa_01`, `sa_02`, `sa_07`
+ - `cluster_slices`: `slice_01`, `slice_02`
+- daily: for all testsuites that will run in the daily job
+- regression: for all testsuites that should pass in the current stable branch
+- sanity: for all testsuites that should be passed by each commit in the
+ stage3 to be successfully verified by Jenkins, currently `basic_07`,
+ `basic_11`, `k8s_03`, `k8s_04`, `sa_02`
+
+In addition, the tag "cleanup" exists in those tests that perform
+any deletion. In that way, it can be invoked to retry the deletion if
+the tests were forcefully stopped.
+
+- For helping in the migration tests and other scenarios in which you don't want
+to destroy the deployments immediately, the following tags are used:
+ - prepare: for the tests that are used to deploy the network
+ services under test
+ - verify: for the tests that perform the actual testing, or changes for
+ additional verifications (e.g. scaling).
+ - cleanup: already described above.
+
+ So, for instance, you could first deploy a number of network services executing
+ the tests with "prepare" tag, migrate to another OSM version, and then
+ check the behavior executing with the "verify" tag. Finally, use the "cleanup"
+ tag.
## Built With
* [Python](www.python.org/) - The language used
+* [Robot Framework](robotframework.org) - The testing framework
## Contributing
## Versioning
-We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://osm.etsi.org/gitweb/?p=osm/TEMPLATE.git;a=tags).
+We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://osm.etsi.org/gitweb/?p=osm/tests.git;a=tags).
## License
This project is licensed under the Apache2 License - see the [LICENSE.md](LICENSE) file for details
## Acknowledgments
-
-* **Billie Thompson** - *Initial work* - [PurpleBooth](https://github.com/PurpleBooth)
-
projects / osm / tests.git / blobdiff