X-Git-Url: https://osm.etsi.org/gitweb/?a=blobdiff_plain;f=README.md;h=03d60549bfe19db3e7e8d349957392fe559a1cb9;hb=72993200fd640123134b663efbf3656cea75aa52;hp=f7245e695af66aae2190422f3af3026240226219;hpb=cbc851de5e838e08c0f50130067c7c6bd31417ca;p=osm%2Ftests.git diff --git a/README.md b/README.md index f7245e6..03d6054 100644 --- a/README.md +++ b/README.md @@ -39,50 +39,15 @@ PACKAGES_FOLDER=osm-packages git clone https://osm.etsi.org/gitlab/vnf-onboarding/osm-packages.git ${PACKAGES_FOLDER} ``` -Configure a file `envfile.rc` copying from `envconfig-local.rc` and set the required variables +## Configure the environment -```bash -# VIM Setup -OS_USERNAME= -OS_PASSWORD= -OS_TENANT_NAME= -OS_AUTH_URL= -OS_TENANT_ID= - -# OSM Setup -OSM_HOSTNAME= -VIM_TARGET= -VIM_MGMT_NET= - -# Clouds file datacenter -OS_CLOUD= -# SDNCs file -OS_SDNC= - -# K8S config file -K8S_CREDENTIALS= - -# The following set of environment variables will be used in host -# of the robot framework. Not needed for docker execution - -# Folder where Robot tests are stored -ROBOT_DEVOPS_FOLDER=robot-systest - -# Folder to save alternative DUT environments (optional) -ENVIRONMENTS_FOLDER=environments - -# Folder where all required packages are stored -PACKAGES_FOLDER=osm-packages - -# Folder where test results should be exported -ROBOT_REPORT_FOLDER=results -``` +Create a file `envfile.rc` copying from `envconfig-local.rc` and set the required variables. ## Running the tests ### From the host machine -If you have installed all the dependecnies, the way of executing the tests is via the following command: +If you have installed all the dependencies, the way of executing the tests is via the following command: ```bash source envfile.rc @@ -91,7 +56,7 @@ robot -d reports -i testsuite/ ### From docker container -It is possible to run the tests directly from the repository or using a docker container with the tests +It is possible to run the tests directly from the repository or using a docker container with the tests. Create the docker container: @@ -128,7 +93,7 @@ docker run --rm=true -t osmtests --env-file \ ### From an environment identical to OSM CICD -````bash +```bash git clone https://osm.etsi.org/gerrit/osm/devops git clone https://osm.etsi.org/gerrit/osm/IM git clone https://osm.etsi.org/gerrit/osm/osmclient @@ -180,6 +145,8 @@ The following tags exist for each testsuite: - `cluster_k8s_charms`: `k8s_05`, `k8s_06` - `cluster_sa`: `sa_01`, `sa_02`, `sa_07` - `cluster_slices`: `slice_01`, `slice_02` + - `cluster_heal`: `heal_01`, `heal_02`, `heal_03`, `heal_04` + - `cluster_sol003`: `sol003_01` - daily: for all testsuites that will run in the daily job - regression: for all testsuites that should pass in the current stable branch - sanity: for all testsuites that should be passed by each commit in the @@ -192,17 +159,45 @@ the tests were forcefully stopped. - For helping in the migration tests and other scenarios in which you don't want to destroy the deployments immediately, the following tags are used: - - prepare: for the tests that are used to deploy the network + - `prepare`: for the tests that are used to deploy the network services under test - - verify: for the tests that perform the actual testing, or changes for + - `verify`: for the tests that perform the actual testing, or changes for additional verifications (e.g. scaling). - - cleanup: already described above. + - `cleanup`: already described above. So, for instance, you could first deploy a number of network services executing the tests with "prepare" tag, migrate to another OSM version, and then check the behavior executing with the "verify" tag. Finally, use the "cleanup" tag. +## Post-processing Robot output files + +The output files of Robot include tyipically three files: + +- `report.html`: overview of the test execution results in HTML format +- `log.html`: details about the executed test cases in HTML format +- `output.xml`: all the test execution results in machine readable XML format + +More information about these files [here](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#output-file). + +It is possible to use the tool `rebot`, included as part of the Robot Framework, to post-process the output file `output.xml`. + +```bash +# To re-generate log and report from output.xml: +rebot [-d ] output.xml + +# To re-generate log and report (and optionally new output.xml) to include only certain tags: +rebot [-d ] -i -i ... -i [-o ] output.xml + +# To re-generate log and report (and optionally new output.xml) excluding certain tags: +rebot [-d ] -e -e ... -e [-o ] output.xml + +# To merge several test executions: +rebot [-d ] --merge output1.xml output2.xml ... outputN.xml +``` + +More information about post-processing Robot output files [here](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#post-processing-outputs) + ## Built With * [Python](www.python.org/) - The language used