Tox + Integration testing

This commit implements a VNF Descriptor-driven integration
test framework, which will lead to integration tests being able
to run via jenkins, and more robust testing in general.

N2VC:

- Allow the use of an event loop passed when instantiating N2VC
- Refactor the execution of the initial-config-primitive so that it can
be easily re-run, such as the case of when a proxy charm is deployed
before the VNF's VM is created.
- Refactor GetPrimitiveStatus, to return the status (queued, running,
complete, failed) of a primitive.
- Add GetPrimitiveOutput, to return the output of a completed primitive
- Fix model disconnection when executing a primitive (it was happening
in the wrong scope)
- Fix wait_for_application, which was previously unused and broken
- Add support for parameter's 'data-type' field
- Add support for better SSH key management, allowing for a proxy charm
to be deployed before the VNF, so that it's public SSH key can be
injected when the VNF's VM is created.

Integration Tests:

The integration tests are intended to exercise the expected
functionality of a VNF/charm: deploy the charm, configure it as required
(i.e., ssh credentials), and execute the VNF's
initial-config-primitives.

- test_native_charm: deploy a native charm to a juju-managed machine and
verify primitive execution works
- test_proxy_charm: deploy a proxy charm, configured to talk to a remote
machine, and verify primitive execution works
- test_metrics_native: deploy a native charm and collect a metric
- test_metrics_proxy: deploy a proxy charm and collect a metric from the
vnf
- test_no_initial-config-primitive: deploy a vnf without an
initial-config-primitive
- test_non-string_parameter: deploy a vnf with a non-string parameter in
initial-config-primitive
- test_no_parameter: deploy a vnf with a primitive with no parameters

General:
- Add a build target to tox.ini so that a .deb is built via Jenkins

TODO (in a follow-up commit):
- test multi-vdu, multi-charm
- test deploying a native charm to a manually-provisioned machine
- Update inline pydoc
- Add more integration tests
- Add global per-test timeout to catch stalled tests

Signed-off-by: Adam Israel <adam.israel@canonical.com>
Change-Id: Id322b45d65c44714e8051fc5764f8c20b76d846c
diff --git a/tests/integration/test_charm_native.py b/tests/integration/test_charm_native.py
new file mode 100644
index 0000000..d1b60ff
--- /dev/null
+++ b/tests/integration/test_charm_native.py
@@ -0,0 +1,141 @@
+"""
+Deploy a native charm (to LXD) and execute a primitive
+"""
+
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: charmnative-ns
+            name: charmnative-ns
+            short-name: charmnative-ns
+            description: NS with 1 VNFs charmnative-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: charmnative-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: charmnative-vnf
+            name: charmnative-vnf
+            short-name: charmnative-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+                vdu-configuration:
+                    juju:
+                        charm: native-ci
+                        proxy: false
+                    initial-config-primitive:
+                    -   seq: '1'
+                        name: test
+    """
+
+    @pytest.mark.asyncio
+    async def test_charm_native(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    loop=event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_charm_native stopped")
+
+        return 'ok'
diff --git a/tests/integration/test_charm_proxy.py b/tests/integration/test_charm_proxy.py
new file mode 100644
index 0000000..c1661ac
--- /dev/null
+++ b/tests/integration/test_charm_proxy.py
@@ -0,0 +1,142 @@
+"""
+Deploy a VNF with a proxy charm, executing an initial-config-primitive
+"""
+
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: charmproxy-ns
+            name: charmproxy-ns
+            short-name: charmproxy-ns
+            description: NS with 1 VNF connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: charmproxy-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmproxy-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: charmproxy-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmproxy-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: charmproxy-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: charmproxy-vnf
+            name: charmproxy-vnf
+            short-name: charmproxy-vnf
+            version: '1.0'
+            description: A VNF consisting of 1 VDUs w/proxy charm
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+                vdu-configuration:
+                    juju:
+                        charm: proxy-ci
+                        proxy: true
+                    initial-config-primitive:
+                    -   seq: '1'
+                        name: test
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_charm_proxy(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_charm_native stopped")
+
+        return 'ok'
diff --git a/tests/integration/test_metrics.py b/tests/integration/test_metrics.py
deleted file mode 100644
index 1151d46..0000000
--- a/tests/integration/test_metrics.py
+++ /dev/null
@@ -1,315 +0,0 @@
-"""Test the collection of charm metrics.
-    1. Deploy a charm w/metrics to a unit
-    2. Collect metrics or wait for collection to run
-    3. Execute n2vc.GetMetrics()
-    5. Destroy Juju unit
-"""
-import asyncio
-import functools
-import logging
-import sys
-import time
-import unittest
-from .. import utils
-
-NSD_YAML = """
-nsd:nsd-catalog:
-    nsd:
-    -   id: singlecharmvdu-ns
-        name: singlecharmvdu-ns
-        short-name: singlecharmvdu-ns
-        description: NS with 1 VNFs singlecharmvdu-vnf connected by datanet and mgmtnet VLs
-        version: '1.0'
-        logo: osm.png
-        constituent-vnfd:
-        -   vnfd-id-ref: singlecharmvdu-vnf
-            member-vnf-index: '1'
-        vld:
-        -   id: mgmtnet
-            name: mgmtnet
-            short-name: mgmtnet
-            type: ELAN
-            mgmt-network: 'true'
-            vim-network-name: mgmt
-            vnfd-connection-point-ref:
-            -   vnfd-id-ref: singlecharmvdu-vnf
-                member-vnf-index-ref: '1'
-                vnfd-connection-point-ref: vnf-mgmt
-            -   vnfd-id-ref: singlecharmvdu-vnf
-                member-vnf-index-ref: '2'
-                vnfd-connection-point-ref: vnf-mgmt
-        -   id: datanet
-            name: datanet
-            short-name: datanet
-            type: ELAN
-            vnfd-connection-point-ref:
-            -   vnfd-id-ref: singlecharmvdu-vnf
-                member-vnf-index-ref: '1'
-                vnfd-connection-point-ref: vnf-data
-            -   vnfd-id-ref: singlecharmvdu-vnf
-                member-vnf-index-ref: '2'
-                vnfd-connection-point-ref: vnf-data
-"""
-
-VNFD_YAML = """
-vnfd:vnfd-catalog:
-    vnfd:
-    -   id: singlecharmvdu-vnf
-        name: singlecharmvdu-vnf
-        short-name: singlecharmvdu-vnf
-        version: '1.0'
-        description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
-        logo: osm.png
-        connection-point:
-        -   id: vnf-mgmt
-            name: vnf-mgmt
-            short-name: vnf-mgmt
-            type: VPORT
-        -   id: vnf-data
-            name: vnf-data
-            short-name: vnf-data
-            type: VPORT
-        mgmt-interface:
-            cp: vnf-mgmt
-        internal-vld:
-        -   id: internal
-            name: internal
-            short-name: internal
-            type: ELAN
-            internal-connection-point:
-            -   id-ref: mgmtVM-internal
-            -   id-ref: dataVM-internal
-        vdu:
-        -   id: mgmtVM
-            name: mgmtVM
-            image: xenial
-            count: '1'
-            vm-flavor:
-                vcpu-count: '1'
-                memory-mb: '1024'
-                storage-gb: '10'
-            interface:
-            -   name: mgmtVM-eth0
-                position: '1'
-                type: EXTERNAL
-                virtual-interface:
-                    type: VIRTIO
-                external-connection-point-ref: vnf-mgmt
-            -   name: mgmtVM-eth1
-                position: '2'
-                type: INTERNAL
-                virtual-interface:
-                    type: VIRTIO
-                internal-connection-point-ref: mgmtVM-internal
-            internal-connection-point:
-            -   id: mgmtVM-internal
-                name: mgmtVM-internal
-                short-name: mgmtVM-internal
-                type: VPORT
-            cloud-init-file: cloud-config.txt
-        vnf-configuration:
-            juju:
-                charm: metrics-ci
-            config-primitive:
-            -   name: touch
-                parameter:
-                -   name: filename
-                    data-type: STRING
-                    default-value: '/home/ubuntu/touched'
-"""
-
-
-class PythonTest(unittest.TestCase):
-    n2vc = None
-    charm = None
-
-    def setUp(self):
-        self.log = logging.getLogger()
-        self.log.level = logging.DEBUG
-
-        self.stream_handler = logging.StreamHandler(sys.stdout)
-        self.log.addHandler(self.stream_handler)
-
-        self.loop = asyncio.get_event_loop()
-
-        self.n2vc = utils.get_n2vc()
-
-        # Parse the descriptor
-        self.log.debug("Parsing the descriptor")
-        self.nsd = utils.get_descriptor(NSD_YAML)
-        self.vnfd = utils.get_descriptor(VNFD_YAML)
-
-
-        # Build the charm
-
-        vnf_config = self.vnfd.get("vnf-configuration")
-        if vnf_config:
-            juju = vnf_config['juju']
-            charm = juju['charm']
-
-            self.log.debug("Building charm {}".format(charm))
-            self.charm = utils.build_charm(charm)
-
-    def tearDown(self):
-        self.loop.run_until_complete(self.n2vc.logout())
-        self.log.removeHandler(self.stream_handler)
-
-    def n2vc_callback(self, model_name, application_name, workload_status,\
-                      workload_message, task=None):
-        """We pass the vnfd when setting up the callback, so expect it to be
-        returned as a tuple."""
-        self.log.debug("status: {}; task: {}".format(workload_status, task))
-
-        # if workload_status in ["stop_test"]:
-        #     # Stop the test
-        #     self.log.debug("Stopping the test1")
-        #     self.loop.call_soon_threadsafe(self.loop.stop)
-        #     return
-
-        if workload_status:
-            if workload_status in ["active"] and not task:
-                # Force a run of the metric collector, so we don't have
-                # to wait for it's normal 5 minute interval run.
-                # NOTE: this shouldn't be done outside of CI
-                utils.collect_metrics(application_name)
-
-                # get the current metrics
-                task = asyncio.ensure_future(
-                    self.n2vc.GetMetrics(
-                        model_name,
-                        application_name,
-                    )
-                )
-                task.add_done_callback(
-                    functools.partial(
-                        self.n2vc_callback,
-                        model_name,
-                        application_name,
-                        "collect_metrics",
-                        task,
-                    )
-                )
-
-            elif workload_status in ["collect_metrics"]:
-
-                if task:
-                    # Check if task returned metrics
-                    results = task.result()
-
-                    foo = utils.parse_metrics(application_name, results)
-                    if 'load' in foo:
-                        self.log.debug("Removing charm")
-                        task = asyncio.ensure_future(
-                            self.n2vc.RemoveCharms(model_name, application_name, self.n2vc_callback)
-                        )
-                        task.add_done_callback(
-                            functools.partial(
-                                self.n2vc_callback,
-                                model_name,
-                                application_name,
-                                "stop_test",
-                                task,
-                            )
-                        )
-                        return
-
-                # No metrics are available yet, so try again in a minute.
-                self.log.debug("Sleeping for 60 seconds")
-                time.sleep(60)
-                task = asyncio.ensure_future(
-                    self.n2vc.GetMetrics(
-                        model_name,
-                        application_name,
-                    )
-                )
-                task.add_done_callback(
-                    functools.partial(
-                        self.n2vc_callback,
-                        model_name,
-                        application_name,
-                        "collect_metrics",
-                        task,
-                    )
-                )
-            elif workload_status in ["stop_test"]:
-                # Stop the test
-                self.log.debug("Stopping the test2")
-                self.loop.call_soon_threadsafe(self.loop.stop)
-
-    def test_deploy_application(self):
-        """Deploy proxy charm to a unit."""
-        if self.nsd and self.vnfd:
-            params = {}
-            vnf_index = 0
-
-            def deploy():
-                """An inner function to do the deployment of a charm from
-                either a vdu or vnf.
-                """
-                charm_dir = "{}/builds/{}".format(utils.get_charm_path(), charm)
-
-                # Setting this to an IP that will fail the initial config.
-                # This will be detected in the callback, which will execute
-                # the "config" primitive with the right IP address.
-                # mgmtaddr = self.container.state().network['eth0']['addresses']
-                # params['rw_mgmt_ip'] = mgmtaddr[0]['address']
-
-                # Legacy method is to set the ssh-private-key config
-                # with open(utils.get_juju_private_key(), "r") as f:
-                #     pkey = f.readline()
-                #     params['ssh-private-key'] = pkey
-
-                ns_name = "default"
-
-                vnf_name = self.n2vc.FormatApplicationName(
-                    ns_name,
-                    self.vnfd['name'],
-                    str(vnf_index),
-                )
-
-                self.loop.run_until_complete(
-                    self.n2vc.DeployCharms(
-                        ns_name,
-                        vnf_name,
-                        self.vnfd,
-                        charm_dir,
-                        params,
-                        {},
-                        self.n2vc_callback
-                    )
-                )
-
-            # Check if the VDUs in this VNF have a charm
-            # for vdu in vnfd['vdu']:
-            #     vdu_config = vdu.get('vdu-configuration')
-            #     if vdu_config:
-            #         juju = vdu_config['juju']
-            #         self.assertIsNotNone(juju)
-            #
-            #         charm = juju['charm']
-            #         self.assertIsNotNone(charm)
-            #
-            #         params['initial-config-primitive'] = vdu_config['initial-config-primitive']
-            #
-            #         deploy()
-            #         vnf_index += 1
-            #
-            # # Check if this VNF has a charm
-            vnf_config = self.vnfd.get("vnf-configuration")
-            if vnf_config:
-                juju = vnf_config['juju']
-                self.assertIsNotNone(juju)
-
-                charm = juju['charm']
-                self.assertIsNotNone(charm)
-
-                if 'initial-config-primitive' in vnf_config:
-                    params['initial-config-primitive'] = vnf_config['initial-config-primitive']
-
-                deploy()
-                vnf_index += 1
-
-            self.loop.run_forever()
-            # while self.loop.is_running():
-            #     # await asyncio.sleep(1)
-            #     time.sleep(1)
diff --git a/tests/integration/test_metrics_native.py b/tests/integration/test_metrics_native.py
new file mode 100644
index 0000000..74faebf
--- /dev/null
+++ b/tests/integration/test_metrics_native.py
@@ -0,0 +1,144 @@
+"""
+Deploy a VNF w/native charm that collects metrics
+"""
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: metricsnative-ns
+            name: metricsnative-ns
+            short-name: metricsnative-ns
+            description: NS with 1 VNFs metricsnative-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: metricsnative-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: metricsnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: metricsnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: metricsnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: metricsnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: metricsnative-vnf
+            name: metricsnative-vnf
+            short-name: metricsnative-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+            vnf-configuration:
+                juju:
+                    charm: metrics-ci
+                    proxy: false
+                config-primitive:
+                -   name: touch
+                    parameter:
+                    -   name: filename
+                        data-type: STRING
+                        default-value: '/home/ubuntu/touched'
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_metrics_native(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_metrics_native stopped")
+
+        return 'ok'
diff --git a/tests/integration/test_metrics_proxy.py b/tests/integration/test_metrics_proxy.py
new file mode 100644
index 0000000..98285fd
--- /dev/null
+++ b/tests/integration/test_metrics_proxy.py
@@ -0,0 +1,139 @@
+"""
+Deploy a VNF w/proxy charm that collects metrics
+"""
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: metricsproxy-ns
+            name: metricsproxy-ns
+            short-name: metricsproxy-ns
+            description: NS with 1 VNFs metricsproxy-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: metricsproxy-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: metricsproxy-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: metricsproxy-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: metricsproxy-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: metricsproxy-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: metricsproxy-vnf
+            name: metricsproxy-vnf
+            short-name: metricsproxy-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+            vnf-configuration:
+                juju:
+                    charm: metrics-proxy-ci
+                    proxy: true
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_metrics_proxy(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+
+            logging.debug("test_metrics_proxy stopped")
+
+        return 'ok'
diff --git a/tests/integration/test_no_initial_config_primitive.py b/tests/integration/test_no_initial_config_primitive.py
new file mode 100644
index 0000000..e66a695
--- /dev/null
+++ b/tests/integration/test_no_initial_config_primitive.py
@@ -0,0 +1,141 @@
+"""
+Test N2VC when the VNF descriptor does not contain an initial-config-primitive.
+"""
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: noinitconfig-ns
+            name: noinitconfig-ns
+            short-name: noinitconfig-ns
+            description: NS with 1 VNFs noinitconfig-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: noinitconfig-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: noinitconfig-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: noinitconfig-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: noinitconfig-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: noinitconfig-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: noinitconfig-vnf
+            name: noinitconfig-vnf
+            short-name: noinitconfig-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+                vdu-configuration:
+                    juju:
+                        charm: native-ci
+                        proxy: false
+                    config-primitive:
+                    -   name: test
+
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_charm_no_initial_config_primitive(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_charm_native stopped")
+
+        return 'ok'
diff --git a/tests/integration/test_no_parameter.py b/tests/integration/test_no_parameter.py
new file mode 100644
index 0000000..39c2443
--- /dev/null
+++ b/tests/integration/test_no_parameter.py
@@ -0,0 +1,142 @@
+"""
+Describe what this test is meant to do.
+"""
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: noparam-ns
+            name: noparam-ns
+            short-name: noparam-ns
+            description: NS with 1 VNFs noparam-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: noparam-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: noparam-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: noparam-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: noparam-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: noparam-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: noparam-vnf
+            name: noparam-vnf
+            short-name: noparam-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+                vdu-configuration:
+                    juju:
+                        charm: native-ci
+                        proxy: false
+                    initial-config-primitive:
+                    -   seq: '1'
+                        name: test
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_charm_no_parameter(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+        logging.warning("event_loop: {}".format(event_loop))
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_charm_native stopped")
+            await self.n2vc.logout()
+
+        return 'ok'
diff --git a/tests/integration/test_non_string_parameter.py b/tests/integration/test_non_string_parameter.py
new file mode 100644
index 0000000..ed3dfc7
--- /dev/null
+++ b/tests/integration/test_non_string_parameter.py
@@ -0,0 +1,147 @@
+"""
+Deploy a VNF with a non-string parameter passed to a primitive
+"""
+import asyncio
+import logging
+import pytest
+from .. import base
+
+
+# @pytest.mark.serial
+class TestCharm(base.TestN2VC):
+
+    NSD_YAML = """
+    nsd:nsd-catalog:
+        nsd:
+        -   id: charmnative-ns
+            name: charmnative-ns
+            short-name: charmnative-ns
+            description: NS with 1 VNFs charmnative-vnf connected by datanet and mgmtnet VLs
+            version: '1.0'
+            logo: osm.png
+            constituent-vnfd:
+            -   vnfd-id-ref: charmnative-vnf
+                member-vnf-index: '1'
+            vld:
+            -   id: mgmtnet
+                name: mgmtnet
+                short-name: mgmtnet
+                type: ELAN
+                mgmt-network: 'true'
+                vim-network-name: mgmt
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-mgmt
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-mgmt
+            -   id: datanet
+                name: datanet
+                short-name: datanet
+                type: ELAN
+                vnfd-connection-point-ref:
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '1'
+                    vnfd-connection-point-ref: vnf-data
+                -   vnfd-id-ref: charmnative-vnf
+                    member-vnf-index-ref: '2'
+                    vnfd-connection-point-ref: vnf-data
+    """
+
+    VNFD_YAML = """
+    vnfd:vnfd-catalog:
+        vnfd:
+        -   id: charmnative-vnf
+            name: charmnative-vnf
+            short-name: charmnative-vnf
+            version: '1.0'
+            description: A VNF consisting of 2 VDUs w/charms connected to an internal VL, and one VDU with cloud-init
+            logo: osm.png
+            connection-point:
+            -   id: vnf-mgmt
+                name: vnf-mgmt
+                short-name: vnf-mgmt
+                type: VPORT
+            -   id: vnf-data
+                name: vnf-data
+                short-name: vnf-data
+                type: VPORT
+            mgmt-interface:
+                cp: vnf-mgmt
+            internal-vld:
+            -   id: internal
+                name: internal
+                short-name: internal
+                type: ELAN
+                internal-connection-point:
+                -   id-ref: mgmtVM-internal
+                -   id-ref: dataVM-internal
+            vdu:
+            -   id: mgmtVM
+                name: mgmtVM
+                image: xenial
+                count: '1'
+                vm-flavor:
+                    vcpu-count: '1'
+                    memory-mb: '1024'
+                    storage-gb: '10'
+                interface:
+                -   name: mgmtVM-eth0
+                    position: '1'
+                    type: EXTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    external-connection-point-ref: vnf-mgmt
+                -   name: mgmtVM-eth1
+                    position: '2'
+                    type: INTERNAL
+                    virtual-interface:
+                        type: VIRTIO
+                    internal-connection-point-ref: mgmtVM-internal
+                internal-connection-point:
+                -   id: mgmtVM-internal
+                    name: mgmtVM-internal
+                    short-name: mgmtVM-internal
+                    type: VPORT
+                cloud-init-file: cloud-config.txt
+                vdu-configuration:
+                    juju:
+                        charm: native-ci
+                        proxy: false
+                    initial-config-primitive:
+                    -   seq: '1'
+                        name: test
+                    -   seq: '2'
+                        name: testint
+                        parameter:
+                        -   name: intval
+                            data-type: INTEGER
+                            value: 1
+    """
+
+    # @pytest.mark.serial
+    @pytest.mark.asyncio
+    async def test_charm_non_string_parameter(self, event_loop):
+        """Deploy and execute the initial-config-primitive of a VNF."""
+
+        if self.nsd and self.vnfd:
+            vnf_index = 0
+
+            for config in self.get_config():
+                juju = config['juju']
+                charm = juju['charm']
+
+                await self.deploy(
+                    vnf_index,
+                    charm,
+                    config,
+                    event_loop,
+                )
+
+            while self.running():
+                logging.debug("Waiting for test to finish...")
+                await asyncio.sleep(15)
+            logging.debug("test_charm_native stopped")
+
+        return 'ok'