From: peusterm Date: Tue, 13 Jun 2017 11:41:33 +0000 (+0200) Subject: Merge remote-tracking branch 'upstream/master' into demo-mano-integration X-Git-Tag: v3.1~21^2~1 X-Git-Url: https://osm.etsi.org/gitweb/?a=commitdiff_plain;h=6dcec5542d668c188ffcf491a8abaac4c5735040;hp=42c44b48f62be3739a3c5cffa58829ef907d7d0d;p=osm%2Fvim-emu.git Merge remote-tracking branch 'upstream/master' into demo-mano-integration --- diff --git a/README.md b/README.md index f144818..f9b8c28 100755 --- a/README.md +++ b/README.md @@ -3,14 +3,16 @@ # son-emu This is the repository of [SONATA's](http://sonata-nfv.eu) emulation platform. -This emulation platform was created to support network service developers to locally prototype and test complete network service chains in realistic end-to-end multi-PoP scenarios. It allows the direct execution of real network functions, packaged as Docker containers, in emulated network topologies running locally on the network service developer's machine. +This emulation platform was created to support network service developers to locally prototype and test complete network service chains in realistic end-to-end multi-PoP scenarios. It allows the execution of real network functions, packaged as Docker containers, in emulated network topologies running locally on the network service developer's machine. ### Cite this work If you use son-emu for your research and/or other publications, please cite the following paper to reference our work: -* Manuel Peuster, Holger Karl, and Steven van Rossem. "**MeDICINE: Rapid Prototyping of Production-Ready Network Services in Multi-PoP Environments.**" to appear in IEEE Conference on Network Function Virtualization and Software Defined Network (NFV-SDN), 2016. - * Pre-print online: http://arxiv.org/abs/1606.05995 +* M. Peuster, H. Karl and S. van Rossem, **"MeDICINE: Rapid prototyping of production-ready network services in multi-PoP environments,"** 2016 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Palo Alto, CA, USA, 2016, pp. 148-153. +doi: 10.1109/NFV-SDN.2016.7919490 + * Link: http://ieeexplore.ieee.org/document/7919490/ + * Pre-print: http://arxiv.org/abs/1606.05995 A short demo that showcases son-emu together with its dummy gatekeeper is available [here](https://www.youtube.com/watch?v=ZANz97pV9ao). @@ -25,21 +27,24 @@ To install the emulator package in development mode, do: * `ansible` Install scripts * `misc` Example packages and VNFs * `src` - * `emuvim` Emulator components - * `api` API endpoint implementations - * `rest` REST API for son-emu-cli - * `sonata` Dummy gatekeeper API - * `cli` Command line client to control the emulator - * `dcemulator` Emulator core - * `resourcemodel` Resource limitation models - * `examples` Example topology scripts - * `test` Test scripts + * `emuvim` Emulator components + * `api` API endpoint implementations + * `rest` REST API for son-emu-cli + * `sonata` Dummy gatekeeper API + * `openstack` OpenStack-like APIs for MANO integration + * `cli` Command line client to control the emulator + * `dashboard` A web-based dashboard to display the emulator's state + * `dcemulator` Emulator core + * `resourcemodel` Resource limitation models + * `examples` Example topology scripts + * `test` Test scripts * `utils` Helper scripts for SONATA's CI/CD setup #### Run Unit Tests * `cd ~/son-emu` * `sudo py.test -v src/emuvim/test/unittests` +(To force using Python2: `python2 -m pytest -v src/emuvim/test/unittests`) ### Building @@ -78,7 +83,7 @@ Contributing to the son-emu is really easy. You must: 4. Follow/answer related [issues](https://github.com/sonata-nfv/son-emu/issues) (see Feedback-Chanel, below). ## Installation -There are two ways to install and use son-emu. The simple one is to use Vagrant to create a VirtualBox-based VM on you machine that contains the pre-installed and configured emulator. The more complicated installation requires a freshly installed Ubuntu 14.04 LTS and is done by a ansible playbook. +There are two ways to install and use son-emu. The simple one is to use Vagrant to create a VirtualBox-based VM on your machine that contains the pre-installed and configured emulator. The more complicated installation requires a freshly installed Ubuntu 16.04 LTS and is done by a ansible playbook. ### Vagrant Installation @@ -92,7 +97,7 @@ Follow the MOTD in the VM to run the example topology and the dummy-gatekeeper. ### Ansible Installation -* Requires: Ubuntu 14.04 LTS +* Requires: Ubuntu 16.04 LTS * `sudo apt-get install ansible git aptitude` * `sudo vim /etc/ansible/hosts` * Add: `localhost ansible_connection=local` @@ -115,7 +120,6 @@ Follow the MOTD in the VM to run the example topology and the dummy-gatekeeper. ## Usage ### Examples -#### Manual Usage Example: This simple example shows how to start the emulator with a simple topology (terminal 1) and how to start (terminal 2) some empty VNF containers in the emulated datacenters (PoPs) by using the son-emu-cli. @@ -129,23 +133,11 @@ This simple example shows how to start the emulator with a simple topology (term * `containernet> vnf1 ifconfig` * `containernet> vnf1 ping -c 2 vnf2` -#### Dummy Gatekeeper Example: - -This example shows how to deploy a SONATA example package in the emulator using the dummy gatekeeper. - -* First terminal (start the emulation platform): - * `sudo python src/emuvim/examples/sonata_y1_demo_topology_1.py` -* Second terminal (deploy the example package): - * Upload: `curl -i -X POST -F package=@sonata-demo-docker.son http://127.0.0.1:5000/packages` - * Instantiate: `curl -X POST http://127.0.0.1:5000/instantiations -d "{}"` - * Verify that service runs: `son-emu-cli compute list` - -Note: The [son-push](https://github.com/mpeuster/son-cli) tool can be used instead of CURL. - ### Further Documentation * [Full CLI command documentation](https://github.com/sonata-nfv/son-emu/wiki/CLI-Command-Overview) * [Requirements for Docker containers executed by the emulator](https://github.com/sonata-nfv/son-emu/wiki/Container-Requirements) +* [REST API](https://github.com/sonata-nfv/son-emu/wiki/REST-API-command-overview) ## License @@ -168,6 +160,7 @@ The following lead developers are responsible for this repository and have admin * Hadi Razzaghi Kouchaksaraei (https://github.com/hadik3r) * Wouter Tavernier (https://github.com/wtaverni) * Geoffroy Chollon (https://github.com/cgeoffroy) +* Eduard Maas (https://github.com/edmaas) #### Feedback-Chanel diff --git a/src/emuvim/api/openstack/__init__.py b/src/emuvim/api/openstack/__init__.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/chain_api.py b/src/emuvim/api/openstack/chain_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/compute.py b/src/emuvim/api/openstack/compute.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/docker_util.py b/src/emuvim/api/openstack/docker_util.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/heat_parser.py b/src/emuvim/api/openstack/heat_parser.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/ip_handler.py b/src/emuvim/api/openstack/ip_handler.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/manage.py b/src/emuvim/api/openstack/manage.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_api_endpoint.py b/src/emuvim/api/openstack/openstack_api_endpoint.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/__init__.py b/src/emuvim/api/openstack/openstack_dummies/__init__.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/base_openstack_dummy.py b/src/emuvim/api/openstack/openstack_dummies/base_openstack_dummy.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/glance_dummy_api.py b/src/emuvim/api/openstack/openstack_dummies/glance_dummy_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/heat_dummy_api.py b/src/emuvim/api/openstack/openstack_dummies/heat_dummy_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/keystone_dummy_api.py b/src/emuvim/api/openstack/openstack_dummies/keystone_dummy_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/neutron_dummy_api.py b/src/emuvim/api/openstack/openstack_dummies/neutron_dummy_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/openstack_dummies/nova_dummy_api.py b/src/emuvim/api/openstack/openstack_dummies/nova_dummy_api.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/__init__.py b/src/emuvim/api/openstack/resources/__init__.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/image.py b/src/emuvim/api/openstack/resources/image.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/instance_flavor.py b/src/emuvim/api/openstack/resources/instance_flavor.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/load_balancer.py b/src/emuvim/api/openstack/resources/load_balancer.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/model.py b/src/emuvim/api/openstack/resources/model.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/net.py b/src/emuvim/api/openstack/resources/net.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/port.py b/src/emuvim/api/openstack/resources/port.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/resource.py b/src/emuvim/api/openstack/resources/resource.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/router.py b/src/emuvim/api/openstack/resources/router.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/server.py b/src/emuvim/api/openstack/resources/server.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/stack.py b/src/emuvim/api/openstack/resources/stack.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/openstack/resources/template.py b/src/emuvim/api/openstack/resources/template.py old mode 100644 new mode 100755 diff --git a/src/emuvim/api/rest/monitor.py b/src/emuvim/api/rest/monitor.py index eac10ef..30422c1 100755 --- a/src/emuvim/api/rest/monitor.py +++ b/src/emuvim/api/rest/monitor.py @@ -199,13 +199,16 @@ class MonitorLinkAction(Resource): # no check if vnfs are really connected to this datacenter... try: - # get URL parameters - data = request.args - #then no data + # check json payload + logging.debug("json: {}".format(request.json)) + logging.debug("args: {}".format(request.args)) + + data = request.json + if data is None: + data = request.args if data is None: data = {} - vnf_src_name = data.get("vnf_src_name") vnf_dst_name = data.get("vnf_dst_name") vnf_src_interface = data.get("vnf_src_interface") diff --git a/src/emuvim/api/rest/network.py b/src/emuvim/api/rest/network.py index 84db87e..0479ee6 100755 --- a/src/emuvim/api/rest/network.py +++ b/src/emuvim/api/rest/network.py @@ -38,7 +38,7 @@ from flask import request import json import networkx -logging.basicConfig(level=logging.INFO) +logging.basicConfig(level=logging.DEBUG) CORS_HEADER = {'Access-Control-Allow-Origin': '*'} @@ -80,16 +80,15 @@ class NetworkAction(Resource): # call DCNetwork method, not really datacenter specific API for now... # no check if vnfs are really connected to this datacenter... try: - # check if json data is a dict - data = request.args - # try json payload + # check json payload + logging.debug("json: {}".format(request.json)) + logging.debug("args: {}".format(request.args)) + + data = request.json if data is None: - data = request.json - # then no data + data = request.args if data is None: data = {} - elif type(data) is not dict: - data = json.loads(request.json) vnf_src_name = data.get("vnf_src_name") vnf_dst_name = data.get("vnf_dst_name") diff --git a/src/emuvim/api/rest/rest_api_endpoint.py b/src/emuvim/api/rest/rest_api_endpoint.py index 10b6c26..b2a7b86 100755 --- a/src/emuvim/api/rest/rest_api_endpoint.py +++ b/src/emuvim/api/rest/rest_api_endpoint.py @@ -56,10 +56,13 @@ class RestApiEndpoint(object): default command line client. """ - def __init__(self, listenip, port): + def __init__(self, listenip, port, DCnetwork=None): self.ip = listenip self.port = port + # connect this DC network to the rest api endpoint (needed for the networking and monitoring api) + self.connectDCNetwork(DCnetwork) + # setup Flask # find directory of dashboard files dashboard_file = pkg_resources.resource_filename('emuvim.dashboard', "index.html") diff --git a/src/emuvim/api/sonata/dummygatekeeper.py b/src/emuvim/api/sonata/dummygatekeeper.py index 151e5b3..759ed41 100755 --- a/src/emuvim/api/sonata/dummygatekeeper.py +++ b/src/emuvim/api/sonata/dummygatekeeper.py @@ -48,6 +48,7 @@ from subprocess import Popen from random import randint import ipaddress import copy +import time logging.basicConfig() LOG = logging.getLogger("sonata-dummy-gatekeeper") @@ -101,6 +102,9 @@ ELINE_SUBNETS = generate_subnets('10.30', 0, subnet_size=50, mask=30) # path to the VNFD for the SAP VNF that is deployed as internal SAP point SAP_VNFD=None +# Time in seconds to wait for vnf stop scripts to execute fully +VNF_STOP_WAIT_TIME = 5 + class Gatekeeper(object): def __init__(self): @@ -242,6 +246,10 @@ class Service(object): # instance_uuid = str(self.uuid.uuid4()) vnf_instances = self.instances[instance_uuid]["vnf_instances"] + # trigger stop skripts in vnf instances and wait a few seconds for completion + self._trigger_emulator_stop_scripts_in_vnfis(vnf_instances) + time.sleep(VNF_STOP_WAIT_TIME) + for v in vnf_instances: self._stop_vnfi(v) @@ -432,6 +440,21 @@ class Service(object): t.daemon = True t.start() + def _trigger_emulator_stop_scripts_in_vnfis(self, vnfi_list): + for vnfi in vnfi_list: + config = vnfi.dcinfo.get("Config", dict()) + env = config.get("Env", list()) + for env_var in env: + var, cmd = map(str.strip, map(str, env_var.split('=', 1))) + if var=="SON_EMU_CMD_STOP": + LOG.info("Executing stop script in %r: %r" % (vnfi.name, cmd)) + # execute command in new thread to ensure that GK is not blocked by VNF + t = threading.Thread(target=vnfi.cmdPrint, args=(cmd,)) + t.daemon = True + t.start() + + + def _unpack_service_package(self): """ unzip *.son file and store contents in CATALOG_FOLDER/services// diff --git a/src/emuvim/cli/rest/compute.py b/src/emuvim/cli/rest/compute.py index 9e5e0ab..fdfc11b 100755 --- a/src/emuvim/cli/rest/compute.py +++ b/src/emuvim/cli/rest/compute.py @@ -110,7 +110,7 @@ class RestApiClient(): Popen(['xterm', '-xrm', 'XTerm.vt100.allowTitleOps: false', '-T', vnf_name, '-e', "docker exec -it mn.{0} /bin/bash".format(vnf_name)]) -parser = argparse.ArgumentParser(description="""son-emu compute +parser = argparse.ArgumentParser(description="""son-emu-cli compute Examples: - son-emu-cli compute start -d dc2 -n client -i sonatanfv/sonata-iperf3-vnf @@ -144,7 +144,7 @@ parser.add_argument( parser.add_argument( "--endpoint", "-e", dest="endpoint", default="http://127.0.0.1:5001", - help="UUID of the plugin to be manipulated.") + help="REST API endpoint of son-emu (default:http://127.0.0.1:5001)") def main(argv): diff --git a/src/emuvim/cli/rest/datacenter.py b/src/emuvim/cli/rest/datacenter.py index e8a41ce..651c55c 100755 --- a/src/emuvim/cli/rest/datacenter.py +++ b/src/emuvim/cli/rest/datacenter.py @@ -80,7 +80,7 @@ class RestApiClient(): print (tabulate(table, headers=headers, tablefmt="grid")) -parser = argparse.ArgumentParser(description='son-emu datacenter') +parser = argparse.ArgumentParser(description='son-emu-cli datacenter') parser.add_argument( "command", choices=['list', 'status'], @@ -91,7 +91,7 @@ parser.add_argument( parser.add_argument( "--endpoint", "-e", dest="endpoint", default="http://127.0.0.1:5001", - help="UUID of the plugin to be manipulated.") + help="REST API endpoint of son-emu (default:http://127.0.0.1:5001)") def main(argv): diff --git a/src/emuvim/cli/rest/monitor.py b/src/emuvim/cli/rest/monitor.py index 7af1fbb..8be1e7f 100755 --- a/src/emuvim/cli/rest/monitor.py +++ b/src/emuvim/cli/rest/monitor.py @@ -46,52 +46,47 @@ class RestApiClient(): print("Command not implemented.") def setup_metric(self, args): - vnf_name = self._parse_vnf_name(args.get("vnf_name")) - vnf_interface = self._parse_vnf_interface(args.get("vnf_name")) - response = put("%s/restapi/monitor/interface/%s/%s/%s" % - (args.get("endpoint"), - vnf_name, - vnf_interface, - args.get("metric"))) - pp.pprint(response.json()) + params = self._create_dict( + vnf_name=self._parse_vnf_name(args.get("vnf_name")), + vnf_interface = self._parse_vnf_interface(args.get("vnf_name")), + metric = args.get("metric")) + + url = "{0}/restapi/monitor/interface".format(args.get("endpoint")) + response = put(url, params=params) + pp.pprint(response.text) def stop_metric(self, args): - vnf_name = self._parse_vnf_name(args.get("vnf_name")) - vnf_interface = self._parse_vnf_interface(args.get("vnf_name")) + params = self._create_dict( + vnf_name=self._parse_vnf_name(args.get("vnf_name")), + vnf_interface=self._parse_vnf_interface(args.get("vnf_name")), + metric=args.get("metric")) - response = delete("%s/restapi/monitor/interface/%s/%s/%s" % - (args.get("endpoint"), - vnf_name, - vnf_interface, - args.get("metric"))) - pp.pprint(response.json()) + url = "{0}/restapi/monitor/interface".format(args.get("endpoint")) + response = put(url, params=params) + pp.pprint(response.text) def setup_flow(self, args): - vnf_name = self._parse_vnf_name(args.get("vnf_name")) - vnf_interface = self._parse_vnf_interface(args.get("vnf_name")) + params = self._create_dict( + vnf_name=self._parse_vnf_name(args.get("vnf_name")), + vnf_interface=self._parse_vnf_interface(args.get("vnf_name")), + metric=args.get("metric"), + cookie=args.get("cookie")) - response = put("%s/restapi/monitor/flow/%s/%s/%s/%s" % - (args.get("endpoint"), - vnf_name, - vnf_interface, - args.get("metric"), - args.get("cookie"))) - - pp.pprint(response.json()) + url = "{0}/restapi/monitor/flow".format(args.get("endpoint")) + response = put(url, params=params) + pp.pprint(response.text) def stop_flow(self, args): - vnf_name = self._parse_vnf_name(args.get("vnf_name")) - vnf_interface = self._parse_vnf_interface(args.get("vnf_name")) + params = self._create_dict( + vnf_name=self._parse_vnf_name(args.get("vnf_name")), + vnf_interface=self._parse_vnf_interface(args.get("vnf_name")), + metric=args.get("metric"), + cookie=args.get("cookie")) - response = delete("%s/restapi/monitor/flow/%s/%s/%s/%s" % - (args.get("endpoint"), - vnf_name, - vnf_interface, - args.get("metric"), - args.get("cookie"))) - - pp.pprint(response.json()) + url = "{0}/restapi/monitor/flow".format(args.get("endpoint")) + response = put(url, params=params) + pp.pprint(response.text) def prometheus(self, args): # This functions makes it more user-friendly to create the correct prometheus query @@ -100,6 +95,7 @@ class RestApiClient(): vnf_interface = self._parse_vnf_interface(args.get("vnf_name")) dc_label = args.get("datacenter") query = args.get("query") + vnf_status = get("%s/restapi/compute/%s/%s" % (args.get("endpoint"), args.get("datacenter"), @@ -122,7 +118,10 @@ class RestApiClient(): return vnf_interface -parser = argparse.ArgumentParser(description='son-emu monitor') + def _create_dict(self, **kwargs): + return kwargs + +parser = argparse.ArgumentParser(description='son-emu-cli monitor') parser.add_argument( "command", choices=['setup_metric', 'stop_metric', 'setup_flow', 'stop_flow','prometheus'], @@ -145,7 +144,7 @@ parser.add_argument( parser.add_argument( "--endpoint", "-e", dest="endpoint", default="http://127.0.0.1:5001", - help="UUID of the plugin to be manipulated.") + help="REST API endpoint of son-emu (default:http://127.0.0.1:5001)") def main(argv): args = vars(parser.parse_args(argv)) diff --git a/src/emuvim/cli/rest/network.py b/src/emuvim/cli/rest/network.py index 91051dd..82fe99f 100755 --- a/src/emuvim/cli/rest/network.py +++ b/src/emuvim/cli/rest/network.py @@ -26,10 +26,8 @@ acknowledge the contributions of their colleagues of the SONATA partner consortium (www.sonata-nfv.eu). """ from requests import get,put, delete -import pprint import argparse -pp = pprint.PrettyPrinter(indent=4) class RestApiClient(): @@ -44,10 +42,9 @@ class RestApiClient(): print("Command not implemented.") def add(self, args): - vnf_src_name = self._parse_vnf_name(args.get("source")) - vnf_dst_name = self._parse_vnf_name(args.get("destination")) - params = self._create_dict( + vnf_src_name=self._parse_vnf_name(args.get("source")), + vnf_dst_name = self._parse_vnf_name(args.get("destination")), vnf_src_interface=self._parse_vnf_interface(args.get("source")), vnf_dst_interface=self._parse_vnf_interface(args.get("destination")), weight=args.get("weight"), @@ -56,18 +53,14 @@ class RestApiClient(): cookie=args.get("cookie"), priority=args.get("priority")) - response = put("%s/restapi/network/%s/%s" % - (args.get("endpoint"), - vnf_src_name, - vnf_dst_name), - json=params) - pp.pprint(response.json()) + response = put("{0}/restapi/network".format(args.get("endpoint")), + params=params) + print(self._nice_print(response.text)) def remove(self, args): - vnf_src_name = self._parse_vnf_name(args.get("source")) - vnf_dst_name = self._parse_vnf_name(args.get("destination")) - params = self._create_dict( + vnf_src_name = self._parse_vnf_name(args.get("source")), + vnf_dst_name = self._parse_vnf_name(args.get("destination")), vnf_src_interface=self._parse_vnf_interface(args.get("source")), vnf_dst_interface=self._parse_vnf_interface(args.get("destination")), weight=args.get("weight"), @@ -76,12 +69,9 @@ class RestApiClient(): cookie=args.get("cookie"), priority=args.get("priority")) - response = delete("%s/restapi/network/%s/%s" % - (args.get("endpoint"), - vnf_src_name, - vnf_dst_name), - json=params) - pp.pprint(response.json()) + response = delete("{0}/restapi/network".format(args.get("endpoint")), + params=params) + print(self._nice_print(response.text)) def _parse_vnf_name(self, vnf_name_str): vnf_name = vnf_name_str.split(':')[0] @@ -98,7 +88,13 @@ class RestApiClient(): def _create_dict(self, **kwargs): return kwargs -parser = argparse.ArgumentParser(description='son-emu network') + def _nice_print(self, text): + # some modules seem to return unicode strings where newlines, other special characters are escaped + text = str(text).replace('\\n', '\n') + text = str(text).replace('\\"', '"') + return text + +parser = argparse.ArgumentParser(description='son-emu-cli network') parser.add_argument( "command", choices=['add', 'remove'], @@ -116,7 +112,7 @@ parser.add_argument( "--weight", "-w", dest="weight", help="weight edge attribute to calculate the path") parser.add_argument( - "--priority", "-p", dest="priority", default="0", + "--priority", "-p", dest="priority", default="1000", help="priority of flow rule") parser.add_argument( "--match", "-m", dest="match", @@ -125,12 +121,12 @@ parser.add_argument( "--bidirectional", "-b", dest="bidirectional", action='store_true', help="add/remove the flow entries from src to dst and back") parser.add_argument( - "--cookie", "-c", dest="cookie", + "--cookie", "-c", dest="cookie", default="10", help="cookie for this flow, as easy to use identifier (eg. per tenant/service)") parser.add_argument( "--endpoint", "-e", dest="endpoint", default="http://127.0.0.1:5001", - help="UUID of the plugin to be manipulated.") + help="REST API endpoint of son-emu (default:http://127.0.0.1:5001)") def main(argv): args = vars(parser.parse_args(argv)) diff --git a/src/emuvim/dashboard/js/graph.json b/src/emuvim/dashboard/js/graph.json old mode 100644 new mode 100755 diff --git a/src/emuvim/dcemulator/net.py b/src/emuvim/dcemulator/net.py index 5dfcbcd..7efb3f0 100755 --- a/src/emuvim/dcemulator/net.py +++ b/src/emuvim/dcemulator/net.py @@ -33,6 +33,7 @@ from subprocess import Popen import re import requests import os +import json from mininet.net import Containernet from mininet.node import Controller, DefaultController, OVSSwitch, OVSKernelSwitch, Docker, RemoteController @@ -50,6 +51,11 @@ LOG.setLevel(logging.DEBUG) # default CPU period used for cpu percentage-based cfs values (microseconds) CPU_PERIOD = 1000000 +# default priority setting for added flow-rules +DEFAULT_PRIORITY = 1000 +# default cookie number for new flow-rules +DEFAULT_COOKIE = 10 + class DCNetwork(Containernet): """ Wraps the original Mininet/Containernet class and provides @@ -710,7 +716,15 @@ class DCNetwork(Containernet): switch_inport_nr = self.DCNetwork_graph[current_hop][next_hop][0]['dst_port_nr'] current_hop = next_hop - return "path {2} between {0} and {1}".format(vnf_src_name, vnf_dst_name, cmd) + flow_options = { + 'priority':kwargs.get('priority', DEFAULT_PRIORITY), + 'cookie':kwargs.get('cookie', DEFAULT_COOKIE), + 'vlan':kwargs['vlan'], + 'path':kwargs['path'], + 'match_input':kwargs.get('match') + } + flow_options_str = json.dumps(flow_options, indent=1) + return "success: {2} between {0} and {1} with options: {3}".format(vnf_src_name, vnf_dst_name, cmd, flow_options_str) def _set_flow_entry_ryu_rest(self, node, switch_inport_nr, switch_outport_nr, **kwargs): match = 'in_port=%s' % switch_inport_nr @@ -722,7 +736,7 @@ class DCNetwork(Containernet): index = kwargs.get('pathindex') vlan = kwargs.get('vlan') - priority = kwargs.get('priority') + priority = kwargs.get('priority', DEFAULT_PRIORITY) # flag to not set the ovs port vlan tag skip_vlan_tag = kwargs.get('skip_vlan_tag') # table id to put this flowentry diff --git a/src/emuvim/examples/demo_topo_3pop.py b/src/emuvim/examples/demo_topo_3pop.py index 8020fc8..aee5f91 100755 --- a/src/emuvim/examples/demo_topo_3pop.py +++ b/src/emuvim/examples/demo_topo_3pop.py @@ -47,7 +47,7 @@ logging.basicConfig(level=logging.INFO) def create_topology1(): # create topology - net = DCNetwork(controller=RemoteController, monitor=False, enable_learning=True) + net = DCNetwork(controller=RemoteController, monitor=True, enable_learning=True) dc1 = net.addDatacenter("dc1") dc2 = net.addDatacenter("dc2") dc3 = net.addDatacenter("dc3") diff --git a/src/emuvim/examples/profiling.py b/src/emuvim/examples/profiling.py old mode 100644 new mode 100755 diff --git a/src/emuvim/examples/simple_topology.py b/src/emuvim/examples/simple_topology.py index d63a07c..b43ad78 100755 --- a/src/emuvim/examples/simple_topology.py +++ b/src/emuvim/examples/simple_topology.py @@ -93,7 +93,7 @@ def create_topology1(): this API, e.g., start/stop/list compute instances. """ # create a new instance of a endpoint implementation - rapi1 = RestApiEndpoint("127.0.0.1", 5001) + rapi1 = RestApiEndpoint("127.0.0.1", 5001, net) # connect data centers to this endpoint rapi1.connectDatacenter(dc1) rapi1.connectDatacenter(dc2) diff --git a/src/emuvim/test/api_base.py b/src/emuvim/test/api_base.py index 57d4aa5..3464f45 100755 --- a/src/emuvim/test/api_base.py +++ b/src/emuvim/test/api_base.py @@ -68,7 +68,7 @@ class SimpleTestTopology(unittest.TestCase): specific controller functionality. """ self.net = DCNetwork(controller=controller, **kwargs) - self.api = RestApiEndpoint("127.0.0.1", 5001) + self.api = RestApiEndpoint("127.0.0.1", 5001, self.net) # add some switches # start from s1 because ovs does not like to have dpid = 0 # and switch name-number is being used by mininet to set the dpid diff --git a/src/emuvim/test/test-initial-template.yml b/src/emuvim/test/test-initial-template.yml old mode 100644 new mode 100755 diff --git a/src/emuvim/test/test-scaleout-template.yml b/src/emuvim/test/test-scaleout-template.yml old mode 100644 new mode 100755 diff --git a/src/emuvim/test/unittests/test_emulator.py b/src/emuvim/test/unittests/test_emulator.py index b5612a1..94db55f 100755 --- a/src/emuvim/test/unittests/test_emulator.py +++ b/src/emuvim/test/unittests/test_emulator.py @@ -117,6 +117,57 @@ class testEmulatorTopology( SimpleTestTopology ): class testEmulatorNetworking( SimpleTestTopology ): + def testSDNChainingSingleService_withLearning(self): + """ + Create a two data centers and interconnect them with additional + switches between them. + Uses Ryu SDN controller. + Connect the Docker hosts to different datacenters and setup the links between. + """ + # create network + self.createNet( + nswitches=3, ndatacenter=2, nhosts=0, ndockers=0, + autolinkswitches=True, + controller=RemoteController, + enable_learning=True) + # setup links + self.net.addLink(self.dc[0], self.s[0]) + self.net.addLink(self.s[2], self.dc[1]) + # start Mininet network + self.startNet() + + # add compute resources + vnf1 = self.dc[0].startCompute("vnf1", network=[{'id':'intf1', 'ip':'10.0.10.1/24'}]) + vnf2 = self.dc[1].startCompute("vnf2", network=[{'id':'intf2', 'ip':'10.0.10.2/24'}]) + # check number of running nodes + self.assertTrue(len(self.getContainernetContainers()) == 2) + self.assertTrue(len(self.net.hosts) == 2) + self.assertTrue(len(self.net.switches) == 5) + # check status + # check get status + s1 = self.dc[0].containers.get("vnf1").getStatus() + print s1 + self.assertTrue(s1["name"] == "vnf1") + self.assertTrue(s1["state"]["Running"]) + self.assertTrue(s1["network"][0]['intf_name'] == 'intf1') + self.assertTrue(s1["network"][0]['ip'] == '10.0.10.1/24') + + s2 = self.dc[1].containers.get("vnf2").getStatus() + print s2 + self.assertTrue(s2["name"] == "vnf2") + self.assertTrue(s2["state"]["Running"]) + self.assertTrue(s2["network"][0]['intf_name'] == 'intf2') + self.assertTrue(s2["network"][0]['ip'] == '10.0.10.2/24') + + # should be connected because learning = True + self.assertTrue(self.net.ping([vnf1, vnf2]) <= 0.0) + # setup links + self.net.setChain('vnf1', 'vnf2', 'intf1', 'intf2', bidirectional=True, cmd='add-flow') + # should still be connected + self.assertTrue(self.net.ping([vnf1, vnf2]) <= 0.0) + # stop Mininet network + self.stopNet() + def testSDNChainingSingleService(self): """ Create a two data centers and interconnect them with additional diff --git a/src/emuvim/test/unittests/test_heatapi_keystone_get_token.json b/src/emuvim/test/unittests/test_heatapi_keystone_get_token.json old mode 100644 new mode 100755 diff --git a/src/emuvim/test/unittests/test_heatapi_template_chaining.json b/src/emuvim/test/unittests/test_heatapi_template_chaining.json old mode 100644 new mode 100755 diff --git a/src/emuvim/test/unittests/test_heatapi_template_create_stack.json b/src/emuvim/test/unittests/test_heatapi_template_create_stack.json old mode 100644 new mode 100755 diff --git a/src/emuvim/test/unittests/test_heatapi_template_update_stack.json b/src/emuvim/test/unittests/test_heatapi_template_update_stack.json old mode 100644 new mode 100755 diff --git a/src/emuvim/test/unittests/test_restapi.py b/src/emuvim/test/unittests/test_restapi.py index ea0b693..268b12b 100755 --- a/src/emuvim/test/unittests/test_restapi.py +++ b/src/emuvim/test/unittests/test_restapi.py @@ -92,14 +92,19 @@ class testRestApi(SimpleTestTopology): print('network add vnf1 vnf2->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli network add -src vnf1 -dst vnf2 -b -c 10", shell=True) + output = subprocess.check_output("son-emu-cli network add -src vnf1 -dst vnf2 -b -c 10", shell=True) + self.assertTrue("add-flow" in output) + self.assertTrue("success" in output) + print('network remove vnf1 vnf2->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli network remove -src vnf1 -dst vnf2 -b", shell=True) + output = subprocess.check_output("son-emu-cli network remove -src vnf1 -dst vnf2 -b", shell=True) + self.assertTrue("del-flows" in output) + self.assertTrue("success" in output) print('>>>>> checking --> son-emu-cli compute stop -d datacenter0 -n vnf2 ->>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli compute stop -d datacenter0 -n vnf2", shell=True) + output = subprocess.check_output("son-emu-cli compute stop -d datacenter0 -n vnf2", shell=True) # check number of running nodes self.assertTrue(len(self.getContainernetContainers()) == 2) @@ -111,7 +116,6 @@ class testRestApi(SimpleTestTopology): print('>>>>> checking --> son-emu-cli compute list ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli compute list", shell=True) output = subprocess.check_output("son-emu-cli compute list", shell=True) # check datacenter list result @@ -119,7 +123,6 @@ class testRestApi(SimpleTestTopology): print('>>>>> checking --> son-emu-cli compute status -d datacenter0 -n vnf1 ->>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli compute status -d datacenter0 -n vnf1", shell=True) output = subprocess.check_output("son-emu-cli compute status -d datacenter0 -n vnf1", shell=True) output = ast.literal_eval(output) @@ -129,18 +132,13 @@ class testRestApi(SimpleTestTopology): print('>>>>> checking --> son-emu-cli datacenter list ->>>>>>>>>>>>>>>>>>>>>>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli datacenter list", shell=True) output = subprocess.check_output("son-emu-cli datacenter list", shell=True) - # check datacenter list result - self.assertTrue("datacenter0" in output) print('->>>>> checking --> son-emu-cli datacenter status -d datacenter0 ->>>>>>>>') print('->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>') - subprocess.call("son-emu-cli datacenter status -d datacenter0", shell=True) output = subprocess.check_output("son-emu-cli datacenter status -d datacenter0", shell=True) - # check datacenter status result self.assertTrue("datacenter0" in output)