|
|
|
## Revive Initial TST010 Work
|
|
|
|
|
|
|
|
Frank Bryden performed an initial analysis of the TST010 suite and made adjustments as needed for it to run against OSM. This included a method of fetching the suite from ETSI Forge and running it as part of the core OSM robot tests container.
|
|
|
|
|
|
|
|
Reference for TST010: https://forge.etsi.org/rep/nfv/api-tests
|
|
|
|
|
|
|
|
This goal’s purpose is to determine what needs to be done in order to run the suite in the same manner as Frank’s original work.
|
|
|
|
|
|
|
|
As OSM has changed since then, there are expected to be some challenges and modifications needed to get it working again.
|
|
|
|
|
|
|
|
According to the documentation (https://osm.etsi.org/gitlab/osm/tests/-/blob/master/README_tst010_robot_cicd.md) we should be able to specify a test name using `-t “test name”`. This aligns with the OSM suite where tests are also specified using `-t “test name”`. What needs to be investigated is if there is a conflict because the same `-t` flag is used in both cases.
|
|
|
|
|
|
|
|
From https://osm.etsi.org/gitlab/osm/tests/-/blob/master/robot-systest/run_test.sh#L140 we see there is a `-T` flag to switch to running TST010. Does this still work? Not really.
|
|
|
|
|
|
|
|
Note: `-T` takes a parameter which corresponds to either a TAG or a BRANCH in the https://forge.etsi.org/rep/nfv/api-tests git repository. We will start by using 2.7.1
|
|
|
|
|
|
|
|
```
|
|
|
|
./run_test.sh -T 2.7.1
|
|
|
|
```
|
|
|
|
|
|
|
|
We get the following error:
|
|
|
|
```
|
|
|
|
fatal: repository '/robot-systest/conformance-tests/repo' does not exist
|
|
|
|
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/robot-systest/conformance-tests/repo/requirements.txt'
|
|
|
|
python3: can't open file '/robot-systest/conformance-tests/run_conformance_tests.py': [Errno 2] No such file or directory
|
|
|
|
```
|
|
|
|
|
|
|
|
It also looks like the .deb package does not include the conformance-tests files. This means we will need to fix this bug
|
|
|
|
|
|
|
|
In https://osm.etsi.org/gitlab/osm/tests/-/tree/master/conformance-tests we see there is a file called “osm_client.py”. This appears to be a python friendly wrapper to the osmclient command line.
|
|
|
|
|
|
|
|
## Info on local build process
|
|
|
|
Local-build.sh --stage-2 will cause the .deb package for the tests project to be built. As we are missing source code from the tests source git tree in the .deb package, we need to modify the file that says what goes into the .deb and rebuild it.
|
|
|
|
|
|
|
|
### Modify build-debpkg.sh
|
|
|
|
Change:
|
|
|
|
```
|
|
|
|
PKG_DIRECTORIES="robot-systest"
|
|
|
|
```
|
|
|
|
To:
|
|
|
|
```
|
|
|
|
PKG_DIRECTORIES="robot-systest conformance-tests"
|
|
|
|
```
|
|
|
|
### Build Stage 2
|
|
|
|
|
|
|
|
This requires we have osmclient and IM git clones in place.
|
|
|
|
```
|
|
|
|
~/devops/tools/local-build.sh stage-2 --module IM,osmclient,tests
|
|
|
|
```
|
|
|
|
|
|
|
|
When stage 2 completes, it puts the .deb files into ~/snap/qhttpd/common. We can check that the final package contains the conformance tests code:
|
|
|
|
|
|
|
|
```
|
|
|
|
dpkg -c ~/snap/qhttp/common/osm-tests*.deb | grep conformance-tests
|
|
|
|
```
|
|
|
|
|
|
|
|
There should be the following showing:
|
|
|
|
|
|
|
|
![Screenshot_at_2021-11-02_16-09-42](uploads/63e244b5189cc44833f1462eb59df9f3/Screenshot_at_2021-11-02_16-09-42.png)
|
|
|
|
|
|
|
|
### Build Stage 3
|
|
|
|
|
|
|
|
Before we build stage 3, we need to update the docker build file for the OSM Tests container. This file is found in devops/docker/tests/Dockerfile. Here we need to copy the contents the conformance tests to the final position under /robot tests. Find the line (53) with this content:
|
|
|
|
|
|
|
|
```
|
|
|
|
RUN mv /usr/share/osm-tests/robot-systest /robot-systest
|
|
|
|
```
|
|
|
|
|
|
|
|
Add a line after it, so it looks like this:
|
|
|
|
|
|
|
|
```
|
|
|
|
RUN mv /usr/share/osm-tests/robot-systest /robot-systest
|
|
|
|
RUN mv /usr/share/osm-tests/conformance-tests/ /robot-systest/
|
|
|
|
```
|
|
|
|
|
|
|
|
Now build the container:
|
|
|
|
```
|
|
|
|
~/devops/tools/local-build.sh stage-3 --module tests
|
|
|
|
```
|
|
|
|
|
|
|
|
This can take some time. The last line should read as follows:
|
|
|
|
```
|
|
|
|
Successfully tagged opensourcemano/tests:devel
|
|
|
|
```
|
|
|
|
|
|
|
|
Now start the robot test container, using the “devel” tag instead of “testing-daily”
|
|
|
|
```
|
|
|
|
~/devops/tools/local-build.sh start-robot --openstackrc ~/openstack.rc --osm-tests-image-tag devel
|
|
|
|
```
|
|
|
|
|
|
|
|
Run the test, specifying OSM test suite to run as “nothing” so we don’t waste time running OSM suites.
|
|
|
|
```
|
|
|
|
./run_test.sh -T 2.7.1 -t nothing
|
|
|
|
```
|
|
|
|
(The fact that the run_test.sh goes on to execute OSM tests is probably a problem. Future goal to change the script so that it runs either Conformance, or OSM robot, not both)
|
|
|
|
|
|
|
|
# TODOs
|
|
|
|
- Update documentation for the -T flag: it takes a parameter, which is the tag or branch from the ETSI repo that will be used
|
|
|
|
|
|
|
|
# Next Goals
|
|
|
|
- Make change to the tests packaging so the conformance tests is included
|
|
|
|
- Rebuild docker container for tests
|
|
|
|
- Try running the -T test again
|
|
|
|
- Pick another OSM test (like basic_01) to run and get report |