Tox + Integration testing 66/6466/11
authorAdam Israel <adam.israel@canonical.com>
Thu, 6 Sep 2018 23:22:47 +0000 (19:22 -0400)
committerAdam Israel <adam.israel@canonical.com>
Thu, 13 Sep 2018 17:01:10 +0000 (13:01 -0400)
commit5e08a0e8fa4fd9d0156d28f8f4e53e5b176c704a
tree9c54be88e745853203f046e6d2a4eb542e6563b1
parent427f57893cb4bf8aebc0b6f56d0cb3f178d23cf1
Tox + Integration testing

This commit implements a VNF Descriptor-driven integration
test framework, which will lead to integration tests being able
to run via jenkins, and more robust testing in general.

N2VC:

- Allow the use of an event loop passed when instantiating N2VC
- Refactor the execution of the initial-config-primitive so that it can
be easily re-run, such as the case of when a proxy charm is deployed
before the VNF's VM is created.
- Refactor GetPrimitiveStatus, to return the status (queued, running,
complete, failed) of a primitive.
- Add GetPrimitiveOutput, to return the output of a completed primitive
- Fix model disconnection when executing a primitive (it was happening
in the wrong scope)
- Fix wait_for_application, which was previously unused and broken
- Add support for parameter's 'data-type' field
- Add support for better SSH key management, allowing for a proxy charm
to be deployed before the VNF, so that it's public SSH key can be
injected when the VNF's VM is created.

Integration Tests:

The integration tests are intended to exercise the expected
functionality of a VNF/charm: deploy the charm, configure it as required
(i.e., ssh credentials), and execute the VNF's
initial-config-primitives.

- test_native_charm: deploy a native charm to a juju-managed machine and
verify primitive execution works
- test_proxy_charm: deploy a proxy charm, configured to talk to a remote
machine, and verify primitive execution works
- test_metrics_native: deploy a native charm and collect a metric
- test_metrics_proxy: deploy a proxy charm and collect a metric from the
vnf
- test_no_initial-config-primitive: deploy a vnf without an
initial-config-primitive
- test_non-string_parameter: deploy a vnf with a non-string parameter in
initial-config-primitive
- test_no_parameter: deploy a vnf with a primitive with no parameters

General:
- Add a build target to tox.ini so that a .deb is built via Jenkins

TODO (in a follow-up commit):
- test multi-vdu, multi-charm
- test deploying a native charm to a manually-provisioned machine
- Update inline pydoc
- Add more integration tests
- Add global per-test timeout to catch stalled tests

Signed-off-by: Adam Israel <adam.israel@canonical.com>
Change-Id: Id322b45d65c44714e8051fc5764f8c20b76d846c
46 files changed:
Makefile [new file with mode: 0644]
n2vc/vnf.py
tests/README.md [new file with mode: 0644]
tests/base.py [new file with mode: 0644]
tests/charms/layers/metrics-ci/deps/layer/basic [deleted submodule]
tests/charms/layers/metrics-ci/deps/layer/metrics [deleted submodule]
tests/charms/layers/metrics-ci/deps/layer/options [deleted submodule]
tests/charms/layers/metrics-ci/icon.svg [deleted file]
tests/charms/layers/metrics-proxy-ci/README.ex [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/config.yaml [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/layer.yaml [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/metadata.yaml [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/metrics.yaml [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/reactive/metrics_ci.py [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/tests/00-setup [new file with mode: 0644]
tests/charms/layers/metrics-proxy-ci/tests/10-deploy [new file with mode: 0644]
tests/charms/layers/native-ci/README.md [new file with mode: 0644]
tests/charms/layers/native-ci/actions.yaml [new file with mode: 0644]
tests/charms/layers/native-ci/actions/test [new file with mode: 0755]
tests/charms/layers/native-ci/actions/testint [new file with mode: 0755]
tests/charms/layers/native-ci/layer.yaml [new file with mode: 0644]
tests/charms/layers/native-ci/metadata.yaml [new file with mode: 0644]
tests/charms/layers/native-ci/reactive/native-ci.py [new file with mode: 0644]
tests/charms/layers/proxy-ci/README.md [new file with mode: 0644]
tests/charms/layers/proxy-ci/actions.yaml [new file with mode: 0644]
tests/charms/layers/proxy-ci/actions/test [new file with mode: 0755]
tests/charms/layers/proxy-ci/layer.yaml [new file with mode: 0644]
tests/charms/layers/proxy-ci/metadata.yaml [new file with mode: 0644]
tests/charms/layers/proxy-ci/reactive/proxy_ci.py [new file with mode: 0644]
tests/integration/test_charm_native.py [new file with mode: 0644]
tests/integration/test_charm_proxy.py [new file with mode: 0644]
tests/integration/test_metrics.py [deleted file]
tests/integration/test_metrics_native.py [new file with mode: 0644]
tests/integration/test_metrics_proxy.py [new file with mode: 0644]
tests/integration/test_no_initial_config_primitive.py [new file with mode: 0644]
tests/integration/test_no_parameter.py [new file with mode: 0644]
tests/integration/test_non_string_parameter.py [new file with mode: 0644]
tests/test_async_task.py [deleted file]
tests/test_libjuju.py [new file with mode: 0644]
tests/test_lxd.py [new file with mode: 0644]
tests/test_primitive_no_parameter.py [deleted file]
tests/test_primitive_non-string_parameter.py [deleted file]
tests/test_python.py [deleted file]
tests/test_single_vdu_proxy_charm.py [deleted file]
tests/utils.py [deleted file]
tox.ini