Tox + Integration testing
This commit implements a VNF Descriptor-driven integration
test framework, which will lead to integration tests being able
to run via jenkins, and more robust testing in general.
N2VC:
- Allow the use of an event loop passed when instantiating N2VC
- Refactor the execution of the initial-config-primitive so that it can
be easily re-run, such as the case of when a proxy charm is deployed
before the VNF's VM is created.
- Refactor GetPrimitiveStatus, to return the status (queued, running,
complete, failed) of a primitive.
- Add GetPrimitiveOutput, to return the output of a completed primitive
- Fix model disconnection when executing a primitive (it was happening
in the wrong scope)
- Fix wait_for_application, which was previously unused and broken
- Add support for parameter's 'data-type' field
- Add support for better SSH key management, allowing for a proxy charm
to be deployed before the VNF, so that it's public SSH key can be
injected when the VNF's VM is created.
Integration Tests:
The integration tests are intended to exercise the expected
functionality of a VNF/charm: deploy the charm, configure it as required
(i.e., ssh credentials), and execute the VNF's
initial-config-primitives.
- test_native_charm: deploy a native charm to a juju-managed machine and
verify primitive execution works
- test_proxy_charm: deploy a proxy charm, configured to talk to a remote
machine, and verify primitive execution works
- test_metrics_native: deploy a native charm and collect a metric
- test_metrics_proxy: deploy a proxy charm and collect a metric from the
vnf
- test_no_initial-config-primitive: deploy a vnf without an
initial-config-primitive
- test_non-string_parameter: deploy a vnf with a non-string parameter in
initial-config-primitive
- test_no_parameter: deploy a vnf with a primitive with no parameters
General:
- Add a build target to tox.ini so that a .deb is built via Jenkins
TODO (in a follow-up commit):
- test multi-vdu, multi-charm
- test deploying a native charm to a manually-provisioned machine
- Update inline pydoc
- Add more integration tests
- Add global per-test timeout to catch stalled tests
Signed-off-by: Adam Israel <adam.israel@canonical.com>
Change-Id: Id322b45d65c44714e8051fc5764f8c20b76d846c
diff --git a/tox.ini b/tox.ini
index 502214f..9ef529f 100644
--- a/tox.ini
+++ b/tox.ini
@@ -4,28 +4,54 @@
# and then run "tox" from this directory.
[tox]
-envlist = lint,py35
+envlist = py3,lint,integration
skipsdist=True
+[pytest]
+markers =
+ serial: mark a test that must run by itself
+
[testenv]
basepython=python3
usedevelop=True
# for testing with other python versions
-commands=nosetests
+commands = py.test --ignore modules/ --tb native -ra -v -s -n auto -k 'not integration' -m 'not serial' {posargs}
+passenv =
+ HOME
+ VCA_HOST
+ VCA_PORT
+ VCA_USER
+ VCA_SECRET
+ # These are needed so executing `charm build` succeeds
+ TERM
+ TERMINFO
deps =
- nose
mock
- pyyaml
+ pyyaml
+ pytest
+ pytest-asyncio
+ pytest-xdist
+ paramiko
+ pylxd
+
+[testenv:py3]
+# default tox env, excludes integration and serial tests
+commands =
+ pytest --ignore modules/ --tb native -ra -v -s -n auto -k 'not integration' -m 'not serial' {posargs}
[testenv:lint]
-envdir = {toxworkdir}/py35
+envdir = {toxworkdir}/py3
commands =
- flake8 --ignore E501 {posargs} juju tests
+ flake8 --ignore E501,E402 {posargs} n2vc tests
deps =
flake8
+[testenv:integration]
+envdir = {toxworkdir}/py3
+commands = py.test --ignore modules/ --tb native -ra -v -s -n 1 -k 'integration' -m 'serial' {posargs}
+
[testenv:build]
-basepython = python3
-deps = stdeb
- setuptools-version-command
+deps =
+ stdeb
+ setuptools-version-command
commands = python3 setup.py --command-packages=stdeb.command bdist_deb