osm/N2VC.git
3 months agoBug 1995 fixed: possibility of defining the K8s namespace for Juju Bundles 00/11900/14
Pedro Escaleira [Tue, 19 Apr 2022 19:40:09 +0000 (20:40 +0100)]
Bug 1995 fixed: possibility of defining the K8s namespace for Juju Bundles

Now, N2VC will use the namespace passed by argument to the methods
install and get_services. Also added this argument to other functions
where it should be passed. When it is not passed, for now it is obtained
from the nsrs, but it should be always passed to avoid queries to the
database, while mainaining backward compatibility.
Updated the N2VC tests accordingly.

Change-Id: Iace944506ba212034efdbb87c6f2d74f8265ea4e
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
3 months agoBug 2067 fixed 64/12164/2
Pedro Escaleira [Sun, 5 Jun 2022 00:29:57 +0000 (01:29 +0100)]
Bug 2067 fixed

Added the missing resource_name param to the _get_replica_count_instance
method call under the method get_scale_count

Change-Id: Iae2f437b82993dbbf71b80feb5b208f29c8d7083
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
3 months agoBug 2064 fixed 61/12161/3
Pedro Escaleira [Fri, 3 Jun 2022 18:48:46 +0000 (19:48 +0100)]
Bug 2064 fixed

Removed the unnecessary verification for if the repo_url exists in the
method get_scale_count of the Helm Base Connection class; The same
problem also applies for the scale method, so this verification was also
removed there;
Also did some fixes/additions to the description of some methods being called
in this verification block.

Change-Id: I710b8678b4b70c6f24c2150d44bb714e9e4c5b66
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
3 months agoUpdate _split_version to check if the kdu_model is a reference or a file 83/12183/4
garciadeblas [Wed, 8 Jun 2022 13:39:24 +0000 (15:39 +0200)]
Update _split_version to check if the kdu_model is a reference or a file

Change-Id: I80dac9f24f652d2a71d9dcbb88cbc2c6850440db
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
3 months agoFix bug 2060: Ignore "model not found" exception when deleting models 78/12178/1
David Garcia [Wed, 8 Jun 2022 09:16:54 +0000 (11:16 +0200)]
Fix bug 2060: Ignore "model not found" exception when deleting models

Change-Id: Ife06b41bf2bcf32b080b405e607450d9303d19e0
Signed-off-by: David Garcia <david.garcia@canonical.com>
4 months agoFix bug 2060: Add logic to delete models gracefully 59/12159/1
David Garcia [Thu, 2 Jun 2022 14:11:28 +0000 (16:11 +0200)]
Fix bug 2060: Add logic to delete models gracefully

Change-Id: Icfda4b9d7ca67cd75621413ac09c6fc00857d1c2
Signed-off-by: David Garcia <david.garcia@canonical.com>
4 months agoFix security bug: Deserialization of Untrusted Data 46/12146/1
David Garcia [Tue, 31 May 2022 09:01:09 +0000 (11:01 +0200)]
Fix security bug: Deserialization of Untrusted Data

Change-Id: I6228e249bdb0acf6f18924910fbb7105fc519eb4
Signed-off-by: David Garcia <david.garcia@canonical.com>
4 months agoBug 2053 fixed: raising JujuControllerFailedConnecting with error message 42/12142/1
Pedro Escaleira [Mon, 30 May 2022 18:08:41 +0000 (19:08 +0100)]
Bug 2053 fixed: raising JujuControllerFailedConnecting with error message

Change-Id: Ie4f3fa92ffd7e7263733d2364ed794a2262f00a4
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
4 months agoFix bug 2047: Remove fetching of actions for charms on update_vca_status 29/12129/2
David Garcia [Thu, 26 May 2022 10:46:14 +0000 (12:46 +0200)]
Fix bug 2047: Remove fetching of actions for charms on update_vca_status

Change-Id: If599baf949b74d420400fd2752aeac6e4fd117f6
Signed-off-by: David Garcia <david.garcia@canonical.com>
4 months agoAdd logs 07/12007/6
David Garcia [Fri, 6 May 2022 10:26:20 +0000 (12:26 +0200)]
Add logs

Change-Id: If831d758b463ce0a2592cd651277c5ec7dabc26a
Signed-off-by: David Garcia <david.garcia@canonical.com>
4 months agoFeature 10908 pass over upgrade request to Libjuju 95/11895/10
aticig [Sun, 17 Apr 2022 21:31:42 +0000 (00:31 +0300)]
Feature 10908 pass over upgrade request to Libjuju

Change-Id: I7a4ea434f22b939a80f58941a892ed92dae62ee6
Signed-off-by: aticig <gulsum.atici@canonical.com>
4 months agoUpdate the repo for a helm KDU before install and upgrade 72/11872/4
garciadeblas [Fri, 8 Apr 2022 20:53:25 +0000 (22:53 +0200)]
Update the repo for a helm KDU before install and upgrade

Change-Id: Ibe4efcaa00b0340502a9476b07d28860d33f7083
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
4 months agoUpdate only the repo that was added instead of all repos 71/11871/4
garciadeblas [Wed, 4 May 2022 09:07:41 +0000 (11:07 +0200)]
Update only the repo that was added instead of all repos

Change-Id: I70c75ced584b40873566a36f2dfc900abc824767
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
4 months agoUpdate helm repo after adding the repo 70/11870/4
garciadeblas [Wed, 4 May 2022 08:57:36 +0000 (10:57 +0200)]
Update helm repo after adding the repo

Change-Id: I4cd2a073cb862fac4d7646e65f7b3df6068a1c5a
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
4 months agoAdd deleting lock to K8sJujuConnector 02/12002/2
David Garcia [Thu, 5 May 2022 07:46:06 +0000 (09:46 +0200)]
Add deleting lock to K8sJujuConnector

- Minor fix in delete_lock
- bug 2016

Change-Id: Ic2f66eeee7967f0d7789428abb4b288c7bb90f36
Signed-off-by: David Garcia <david.garcia@canonical.com>
5 months agoAdd lock deleting namespaces 99/11999/2
David Garcia [Wed, 4 May 2022 13:33:03 +0000 (15:33 +0200)]
Add lock deleting namespaces

- Fix bug 2016

Change-Id: Ifd4572937376cdeec79e3dfd808174d3f14325b9
Signed-off-by: David Garcia <david.garcia@canonical.com>
5 months agoSkip exception if model doesn't exist after delete failure 96/11996/2
David Garcia [Tue, 3 May 2022 10:23:59 +0000 (12:23 +0200)]
Skip exception if model doesn't exist after delete failure

- bug: 2016
Change-Id: I9d118602c747840a4a32fd670c8724c82c1ee094
Signed-off-by: David Garcia <david.garcia@canonical.com>
5 months agoFeature 10908: Add Upgrade Charm 90/11890/8
beierlm [Thu, 14 Apr 2022 20:33:51 +0000 (16:33 -0400)]
Feature 10908: Add Upgrade Charm

Adds ability to wait for a unit to go idle
Adds new method to upgrade charm in a particular application

Change-Id: I329286e63519c6398fc1f2229f306eec0b15551a
Signed-off-by: beierlm <mark.beierl@canonical.com>
5 months agoFix bug 2016: Skip exception if model has been removed 61/11961/2
David Garcia [Thu, 28 Apr 2022 11:43:36 +0000 (13:43 +0200)]
Fix bug 2016: Skip exception if model has been removed

Change-Id: I1d1d6e0c18c7e6a8c42c03a1e39e4673caa0b637
Signed-off-by: David Garcia <david.garcia@canonical.com>
5 months agoBug 2004 fixed: Helm repos certs now with the corret variable cluster_uuid 24/11924/1
Pedro Escaleira [Fri, 22 Apr 2022 17:51:00 +0000 (18:51 +0100)]
Bug 2004 fixed: Helm repos certs now with the corret variable cluster_uuid

Change-Id: Id4075db6830cc69ed4804b2300003748ec9422c5
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
5 months agoBug 1939 fixed: : added a random suffix to the end of the Juju app name, in order... 91/11791/4
Pedro Escaleira [Mon, 21 Mar 2022 17:54:45 +0000 (17:54 +0000)]
Bug 1939 fixed: : added a random suffix to the end of the Juju app name, in order to allow multiple Juju charms per VDU

I followed the second option of the bug's description. Now, Juju
applications have a random suffix with size=5 (the random suffix
size used by K8s), in order to avoid collisions between applications'
names;

Also fixed unit-test on test_n2vc_juju_conn.py

Change-Id: I5d3eb8282889e58361f7c21214b11071a7530d26
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
5 months agoBug 1980 fixed 56/11856/3
Pedro Escaleira [Tue, 5 Apr 2022 16:32:13 +0000 (17:32 +0100)]
Bug 1980 fixed

Added the `yaml_format` argument to the `status_kdu` abstract method of
the class `K8sConnector`, and changed the corresponding Helm connectors
methods accordingly, while maintaining backward compatibility

Change-Id: Ic78c951825b7654ccf8246c4fe1fc4d488221bfc
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
5 months agofeature: helm charts repos with certs 03/11403/3
bravof [Tue, 23 Nov 2021 22:33:18 +0000 (19:33 -0300)]
feature: helm charts repos with certs

Change-Id: I427d20f6e184c889de91a384d921eb7edaf00e95
Signed-off-by: bravof <fbravo@whitestack.com>
5 months agoImprove logs for K8sJujuConnector init_env 20/10920/5
garciadeblas [Mon, 31 May 2021 13:49:15 +0000 (15:49 +0200)]
Improve logs for K8sJujuConnector init_env

Change-Id: I013b6c062cf29d9a69e39ab8fdc363e9bd052873
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
5 months agoBug 1965 fixed 26/11826/3
Pedro Escaleira [Sun, 3 Apr 2022 12:51:46 +0000 (13:51 +0100)]
Bug 1965 fixed

Parsed the manifest returned by the `helm status` command (in the
`_status_kdu` method) from a raw string to a list of dictionaries (each
entry in the list corresponds to each manifest's document).

Change-Id: I791c8fd59fee8ac67bcbd640a100bf63d572efcf
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
5 months agoBug 1983 fixed: added the possibility to deploy bundles from the Charm Hub 59/11859/3
Pedro Escaleira [Tue, 5 Apr 2022 20:01:37 +0000 (21:01 +0100)]
Bug 1983 fixed: added the possibility to deploy bundles from the Charm Hub

Also upgraded the version of the juju library to 2.9.8 (and made the
necessary changes to the code to work with this version), once the
previous version didn't supported Bundles from the Charm Hub

Change-Id: I34d87dff75b4e77fdbdaef0eaf4da565b0bf213b
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
6 months agoBug 1962 fixed: removed the variable cluster_uuid from init_env method 22/11822/6
Pedro Escaleira [Fri, 1 Apr 2022 23:44:08 +0000 (00:44 +0100)]
Bug 1962 fixed: removed the variable cluster_uuid from init_env method

This was applied to the class K8sHelmBaseConnector, and then returned
the variable cluster_id instead of cluster_uuid;

Also, refactored the overall Helm related code, in order to use the correct cluster uuid (and fixed the corresponding tests);

Also made a small fix in the function _uninstall_sw, to remove the service accounts when using different namespaces

Change-Id: Ibfdc82f325abf5fd27a225e73f949483a4e68fe2
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
6 months agoBug 1914 fixed 65/11765/1
Pedro Escaleira [Thu, 17 Mar 2022 23:59:29 +0000 (23:59 +0000)]
Bug 1914 fixed

Change-Id: I06ef0123481fa26a06e48e808327e82d1b25b0ce
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
list fixes

Change-Id: I3809a0a6da1c2aee6d43a47585b2b02b0b81a593
Signed-off-by: Pedro Escaleira <escaleira@av.it.pt>
8 months agoBug 1843: LTS Support 15/11515/3
beierlm [Thu, 13 Jan 2022 16:43:45 +0000 (11:43 -0500)]
Bug 1843: LTS Support

Updates base to Ubuntu 20.04
Updates python to 3.8
Updates pip dependencies to more recent versions
Adds apt cache to stage 2 dockerfile

Fixes bug 1843

Change-Id: Iec7c73cb99f5ed3f0e07666d8c8c8be8ca91c0ea
Signed-off-by: beierlm <mark.beierl@canonical.com>
8 months agoFeature 10886 manual scaling for KNF (helm chart) deployment 60/11460/6
aktas [Tue, 19 Oct 2021 15:26:13 +0000 (18:26 +0300)]
Feature 10886 manual scaling for KNF (helm chart) deployment

Change-Id: Ibdcabd4e3f077bf86749f49e8aeb75aec2935ea2
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
10 months agobugfix: kubeconfig passed for helm commands. Bug 1789 67/11367/11
bravof [Wed, 17 Nov 2021 14:14:57 +0000 (11:14 -0300)]
bugfix: kubeconfig passed for helm commands. Bug 1789

Change-Id: Idf8e9141e46296d45dbaa0260e8c9a55b445eb54
Signed-off-by: bravof <fbravo@whitestack.com>
10 months agoFeature 10887: Add cross-model relations support 91/11291/7 release-v11.0-start v11.0.0rc1
David Garcia [Tue, 26 Oct 2021 10:30:44 +0000 (12:30 +0200)]
Feature 10887: Add cross-model relations support

Changes:
- Extend `add_relation` method in N2VCJujuConn to include the CMR case
- Add `add_relation` method to K8sJujuConn
- Add n2vc/definitions.py file that includes definition ofjects for
Offer and RelationEndpoint.
- Change `n2vc.libjuju.Libjuju.list_offers` method to be private, and accept a filter `offer_name` parameter.
- Update `n2vc.libjuju.Libjuju.consume` method arguments.
- Add `n2vc.libjuju.Libjuju.offer` method to create an offer.

Unit tests associated to the code changes have been either created or
updated accordingly

Change-Id: Ibf8d574528dee0fa898e0e97578dd3a6aa68316a
Signed-off-by: David Garcia <david.garcia@canonical.com>
11 months agoFix bug 1722: add support for deploying .charm files 93/11293/1
David Garcia [Tue, 26 Oct 2021 15:24:21 +0000 (17:24 +0200)]
Fix bug 1722: add support for deploying .charm files

Change-Id: Id43c3d0714cf341dcb4d5581a32d6a1102a3e7ac
Signed-off-by: David Garcia <david.garcia@canonical.com>
11 months agoFix bug 1715 78/11278/2
aktas [Tue, 19 Oct 2021 17:03:23 +0000 (20:03 +0300)]
Fix bug 1715

In sometimes, it cannot catch that if the application is in the model.
Jenkinks report: https://osm.etsi.org/jenkins/view/Robot%20tests/job/osm-stage_3-merge/job/v10.0/169/robot/report/report.html#tags?basic_20

Change-Id: I8121d57923d93b7af9ad60e07b87b4616060a916
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
11 months agoUpdate requirements.txt and tox.ini 65/11265/6
David Garcia [Thu, 14 Oct 2021 13:05:25 +0000 (15:05 +0200)]
Update requirements.txt and tox.ini

- Update to juju==2.9.4: needed for feature 10884
- Pin pylint version in tox.ini

Change-Id: Ie7115804973f97e61e9bb8946215ce958abe8089
Signed-off-by: David Garcia <david.garcia@canonical.com>
12 months agoBug 1651 fix 56/11156/2
gatici [Wed, 8 Sep 2021 07:35:18 +0000 (07:35 +0000)]
Bug 1651 fix

This fix is required to fix VCA metric collection for SOL006 compatible descriptors.

Related bug: https://osm.etsi.org/bugzilla/show_bug.cgi?id=1651

Change-Id: I5a25c97b48692c23b5520cf72becb7b5bbcf327a
Signed-off-by: gatici <gulsumatici@gmail.com>
12 months agoAllow the deployment of privileged KDUs 51/11151/1
David Garcia [Tue, 7 Sep 2021 09:15:48 +0000 (11:15 +0200)]
Allow the deployment of privileged KDUs

Change-Id: I7fac53669d08119be4f8da49509c6ee418c1df69
Signed-off-by: David Garcia <david.garcia@canonical.com>
13 months agoBug 1643 fix 21/11121/2
aktas [Thu, 26 Aug 2021 05:15:37 +0000 (08:15 +0300)]
Bug 1643 fix

Change-Id: Id3eaeb2f3c2fac33cc2faf0c45d51f07a2392a7e
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
13 months agoFix bug 1636: remove the default 30s timeout in retry 16/11116/2
David Garcia [Wed, 18 Aug 2021 15:35:50 +0000 (17:35 +0200)]
Fix bug 1636: remove the default 30s timeout in retry

Change-Id: Ie6094ebb91d32c1dcb63dc718f4dbeaf2e07adda
Signed-off-by: David Garcia <david.garcia@canonical.com>
13 months agoFix bug 1638: Give warning when removing unexisting chart 14/11114/2
David Garcia [Wed, 18 Aug 2021 12:52:52 +0000 (14:52 +0200)]
Fix bug 1638: Give warning when removing unexisting chart

Change-Id: I4620fe7b2aac1e9c9109ad1e87e5b0b5d585cde6
Signed-off-by: David Garcia <david.garcia@canonical.com>
13 months agoBug 1609 fix 86/11086/12
aktas [Thu, 29 Jul 2021 14:41:40 +0000 (17:41 +0300)]
Bug 1609 fix

This patch changes the behaviour of native charm deployments.
it  won't deploy another application for the same vnf or
vdu charm at initial deployment or scaling process.
It scales the application.

Change-Id: I3fc52a5ddb0cb7cb16937bc12cf343f7d869c9ee
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
13 months agoRemove unsecure and unused function 03/11103/2
David Garcia [Tue, 10 Aug 2021 15:50:21 +0000 (17:50 +0200)]
Remove unsecure and unused function

Fixes this critical vulnerability:
  https://osm.etsi.org/gitlab/osm/n2vc/-/security/vulnerabilities/948

Change-Id: If3ab88575028808855f2e068c816fccd8be6cb88
Signed-off-by: David Garcia <david.garcia@canonical.com>
14 months agoGive a warning when removing a cloud that does not exist 75/11075/1
David Garcia [Tue, 27 Jul 2021 13:07:42 +0000 (15:07 +0200)]
Give a warning when removing a cloud that does not exist

Fixes bug 1608

Change-Id: Ide9040f5e97ba07952062f23c9217e227af375d0
Signed-off-by: David Garcia <david.garcia@canonical.com>
14 months agoFix bug 1589: Improve deletion of models 70/11070/5
David Garcia [Mon, 26 Jul 2021 13:04:37 +0000 (15:04 +0200)]
Fix bug 1589: Improve deletion of models

Change-Id: I9f1010206f140606cf9f82044cf3b199113ddb20
Signed-off-by: David Garcia <david.garcia@canonical.com>
14 months agoUnit tests improvements 74/11074/2
David Garcia [Mon, 26 Jul 2021 14:04:59 +0000 (16:04 +0200)]
Unit tests improvements

- From now on, black must pass. Jenkins will give a -1 if it doesn't
- Ignore E203 because it is not pep8 compliant, and conflicts with black
    https://github.com/psf/black/issues/315#issuecomment-395457972
- Black fixes and minor fix in unit test
- Fix devops-stages/stage-test.sh: jenkins was not giving a -1 when tox
was failing

Change-Id: Ic7318dcb6d4006a271607cd064a1f36d0936b7b9
Signed-off-by: David Garcia <david.garcia@canonical.com>
15 months agoUpgrade to libjuju 2.9.2 40/11040/8
David Garcia [Thu, 24 Jun 2021 16:47:22 +0000 (18:47 +0200)]
Upgrade to libjuju 2.9.2

- The upgrade to libjuju enables Pebble support
- Libjuju requires a newer version of Kubernetes so I updated it to the
latest one
- Additionally, a group of functions were moved from k8s_juju_conn.p to
kubectl.py, because they fit better there.

Related feature number: 10884

Change-Id: I354a2f49e7fc7b87a204bf60131a8d52a4f74cac
Signed-off-by: David Garcia <david.garcia@canonical.com>
15 months agoFix Bug 1575 37/11037/2
aktas [Thu, 24 Jun 2021 08:37:11 +0000 (11:37 +0300)]
Fix Bug 1575

This fixes the race condtion when k8s cluster responding too fast

Change-Id: I3f9e18c7bba942689e4b056ead60349fdb72c795
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
15 months agoFix racing condition decrypting data in store.py 30/11030/1
David Garcia [Mon, 21 Jun 2021 11:20:30 +0000 (13:20 +0200)]
Fix racing condition decrypting data in store.py

Fixes bug 1580

Change-Id: Idbb9962e2f279b4b6f322e0e989a5831e4ae3b98
Signed-off-by: David Garcia <david.garcia@canonical.com>
16 months agoFix 1539: Add --skip-repo option for Helm 16/10916/5
David Garcia [Fri, 28 May 2021 14:21:51 +0000 (16:21 +0200)]
Fix 1539: Add --skip-repo option for Helm

If stablerepourl is not set, Helm will skip adding the stablerepo.

Depends on:
- https://osm.etsi.org/gerrit/#/c/osm/devops/+/10915

Change-Id: I11f10fdfba24ec274d463f12c07d7b0247b37068
Signed-off-by: David Garcia <david.garcia@canonical.com>
16 months agoFix bug 1533: use bionic as default series in proxy charms 10/10910/2
David Garcia [Fri, 28 May 2021 10:23:44 +0000 (12:23 +0200)]
Fix bug 1533: use bionic as default series in proxy charms

Xenial is 2 LTS behind the current one. This patch will force bionic
rather than Xenial from now on. The current charms do not properly work
in Xenial.

The best solution will be to fix this bug:
  https://osm.etsi.org/bugzilla/show_bug.cgi?id=1551

Change-Id: I1a0c466b6151b19a1eaf5d97f0373ab271601f89
Signed-off-by: David Garcia <david.garcia@canonical.com>
16 months agoWait for cloud-init to finish before provisioning 86/10886/3
David Garcia [Wed, 26 May 2021 11:16:51 +0000 (13:16 +0200)]
Wait for cloud-init to finish before provisioning

Before provisioning a native charm, a script is executed to wait until
cloud-init finishes. This will avoid some racing conditions.

Fixes bug 1184

Change-Id: Icc87f90f5941cf7b13bdd7dd01e5544122fb769e
Signed-off-by: David Garcia <david.garcia@canonical.com>
16 months agoFix bug 1542 to allow juju to add Azure AKS 12/10912/1
garciadeblas [Fri, 28 May 2021 12:10:59 +0000 (14:10 +0200)]
Fix bug 1542 to allow juju to add Azure AKS

Storage classes in Azure might have no metadata annotations. This
change takes into account that situation.

Change-Id: I4f7151d2481b41fd8434e0d95ea5d1a156b68309
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
16 months agoReformat N2VC to standardized format 37/10537/5 release-v10.0-start
garciadeblas [Wed, 24 Mar 2021 08:22:13 +0000 (09:22 +0100)]
Reformat N2VC to standardized format

Change-Id: I7bfc2236cd5f059dc1b659e9fb746fef619124b8
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
16 months agoFeature 10509 manual scaling for native k8s charm 87/10487/14
aktas [Mon, 15 Mar 2021 08:05:35 +0000 (11:05 +0300)]
Feature 10509 manual scaling for native k8s charm

Juju version has to be upgraded to 2.8.6 because of a typo error.
You can see bug report: https://github.com/juju/python-libjuju/issues/477
and release note: https://github.com/juju/python-libjuju/commit/facd1f19eae65728b6c7c3b823939bb35ae45ed2

Change-Id: Iae4262e64debdf6b4d36b37778ec29f0dd46bac1
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
17 months agoIncluding upstream requirements 89/10689/3
beierlm [Thu, 22 Apr 2021 16:59:58 +0000 (12:59 -0400)]
Including upstream requirements

Now that internal OSM software does not specify python modules in
setup.py, we need to include the requirements.txt from the module
in the requirements-dev.in to generate the full list properly.

Change-Id: Ie2c401545e97c5fe4260f8b81c8cab0088a87789
Signed-off-by: beierlm <mark.beierl@canonical.com>
17 months agoFeature 10239: Distributed VCA 16/10616/9
David Garcia [Mon, 12 Apr 2021 10:07:37 +0000 (12:07 +0200)]
Feature 10239: Distributed VCA

- Add vca_id in all calls that invoke libjuju. This is for being able to
talk to the default VCA or the VCA associated to the VIM
- Add store.py: Abstraction to talk to the database.
  - DBMongoStore: Use the db from common to talk to the database
  - MotorStore: Use motor, an asynchronous mongodb client to talk to the
database
- Add vca/connection.py: Represents the data needed to connect the VCA
- Add EnvironConfig in config.py: Class to get the environment config,
and avoid LCM from passing that

Change-Id: I28625e0c56ce408114022c83d4b7cacbb649434c
Signed-off-by: David Garcia <david.garcia@canonical.com>
17 months agoAdd kwargs in N2VCConnector constructor 09/10709/2
David Garcia [Mon, 26 Apr 2021 13:36:24 +0000 (15:36 +0200)]
Add kwargs in N2VCConnector constructor

Change-Id: I05c3df80dc8390fd6016ddc19636d97c18583bb8
Signed-off-by: David Garcia <david.garcia@canonical.com>
17 months agoFix 1462 99/10399/9
aktas [Fri, 26 Feb 2021 12:32:39 +0000 (15:32 +0300)]
Fix 1462

Wait until Juju application is destroyed in
n2vc.libjuju.Libjuju.destroy_application()

Change-Id: Ie7992f229872ffffbea87ca3e8e20825bfec7b6d
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
17 months agoAdding GitLab Scanner 66/10666/2 ng-ro-refactor branch-ng-ro-refactor-start branch-sol006v331-start
beierlm [Mon, 19 Apr 2021 19:30:53 +0000 (15:30 -0400)]
Adding GitLab Scanner

Adds gitlab-ci yaml file to enable security scans in GitLab mirror

Change-Id: Ia3e04da976d51bf42e08ff657bb1f87717660ec6
Signed-off-by: beierlm <mark.beierl@canonical.com>
17 months agoFix bug 1505 43/10643/3
David Garcia [Thu, 15 Apr 2021 08:15:29 +0000 (10:15 +0200)]
Fix bug 1505

Delete ssh-key generation in N2VCConnector initialization

Change-Id: I5f33adf683ffeef34de270bf9b458a172dd4059d
Signed-off-by: David Garcia <david.garcia@canonical.com>
17 months agoFeature-9904: Enhancing NG-UI to enable Juju operational view dashboard 96/10496/3
ksaikiranr [Wed, 17 Mar 2021 07:24:15 +0000 (12:54 +0530)]
Feature-9904: Enhancing NG-UI to enable Juju operational view dashboard

Added unit test for update_vca_status method for KNF

Change-Id: Iba6285d0d5deeaa7bef685a22abeba0ae535ab19
Signed-off-by: jayaramans <selvi.j@tataelxsi.co.in>
Signed-off-by: ksaikiranr <saikiran.k@tataelxsi.co.in>
17 months agoFeature-9904: Enhancing NG-UI to enable Juju operational view dashboard 94/10494/4
ksaikiranr [Wed, 17 Mar 2021 07:23:20 +0000 (12:53 +0530)]
Feature-9904: Enhancing NG-UI to enable Juju operational view dashboard

Unit testing added for new functions
    - UpdateVcaStatus in test_n2vc_juju_conn.py
    - GetExecutedActionsTest in test_libjuju.py
    - GetApplicationConfigsTest in test_libjuju.py

Change-Id: I7d750d8c39bc7bc061568c28d29f53d1b87c4c23
Signed-off-by: gspri <priyadharshini.g@tataelxsi.co.in>
Signed-off-by: ksaikiranr <saikiran.k@tataelxsi.co.in>
17 months agoFeature-9904: Enhancing NG-UI to enable Juju operational view dashboard 95/10495/4
ksaikiranr [Wed, 17 Mar 2021 07:20:20 +0000 (12:50 +0530)]
Feature-9904: Enhancing NG-UI to enable Juju operational view dashboard

Added functions to get action list, config list and executed action list for KNF

Change-Id: Ibec764c719da5507168c474cf48fc23efbb9c846
Signed-off-by: jayaramans <selvi.j@tataelxsi.co.in>
Signed-off-by: ksaikiranr <saikiran.k@tataelxsi.co.in>
17 months agoFeature-9904: Enhancing NG-UI to enable Juju operational view dashboard 79/10479/5
ksaikiranr [Wed, 17 Mar 2021 07:20:00 +0000 (12:50 +0530)]
Feature-9904: Enhancing NG-UI to enable Juju operational view dashboard

Added methods in n2vc_juju_conn and libjuju to suppot configs list
and executed actions/history of actions. Added method in n2vc juju conn
to call libjuju methods to get actions, configs and history of actions.

Change-Id: I845789080d148fa42e859dcda185da1e62b4c489
Signed-off-by: ksaikiranr <saikiran.k@tataelxsi.co.in>
17 months agoFeature 10296 Pip Standardization 06/10306/17
David Garcia [Fri, 12 Feb 2021 16:19:13 +0000 (17:19 +0100)]
Feature 10296 Pip Standardization

Creates standard template for tox.ini
Introduces 'safety' for listing upstream CVEs
Pins all versions of all upstream modules
Removes Makefile step and places all build logic into tox.ini

Change-Id: Ic14321bd5bddb322db08ac3e25202cc64cac2dff
Signed-off-by: David Garcia <david.garcia@canonical.com>
18 months agoAdd ModelConfig 75/10475/4
David Garcia [Wed, 10 Mar 2021 19:00:53 +0000 (20:00 +0100)]
Add ModelConfig

This class maps the model-config from the vca_config. That config will
be passed every time a model is added.

Change-Id: I0a1e47ba7d708f3514c64a6f20d410a21fe8ea1d
Signed-off-by: David Garcia <david.garcia@canonical.com>
18 months agoPin chardet to 3.0.4 02/10502/1
beierlm [Tue, 16 Mar 2021 15:28:00 +0000 (11:28 -0400)]
Pin chardet to 3.0.4

Chardet 3.0.4 was last modified in 2017, and the next release of chardet 4.0.0
was released only recently (Dec 2020).  As many other pip packages still depend
on 3.0.4 we need to explicitly pin that here to avoid conflicts later at LCM
docker build time.

Change-Id: I2a1ee0d61e641c0dc3f240345ce47f0d13d8fb22
Signed-off-by: beierlm <mark.beierl@canonical.com>
18 months agoFix bug 1467 93/10493/2
David Garcia [Mon, 15 Mar 2021 17:41:34 +0000 (18:41 +0100)]
Fix bug 1467

Do not raise exceptions in n2vc.libjuju.add_model() or n2vc.libjuju.destroy_model() functions

In general, functions should be as idempotent as possible. In this
particular case, executing the add_model() function several times with
the same arguments should just work. Same thing for destroy_model(). If
the model has already been destroyed, it should return and not give any
errors saying that the model doesn't exist.

Change-Id: I87e11ea0fe1b4063b2f89900fcc2bbf1f915b953
Signed-off-by: David Garcia <david.garcia@canonical.com>
18 months agoMinor improvements to libjuju.py 73/10473/5
David Garcia [Wed, 10 Mar 2021 16:09:44 +0000 (17:09 +0100)]
Minor improvements to libjuju.py

- Increase default timeout to connect to the controller
- Minor fix in destroy_model function
- Fix unit tests. Now they run faster :-)

Change-Id: Ia2cbcb3e052fe91f4e712ea572ecf444b1d5894f
Signed-off-by: David Garcia <david.garcia@canonical.com>
18 months agoFix bug 1448: minor fix in JujuModelWatcher.wait_for_model 70/10470/5
David Garcia [Wed, 10 Mar 2021 10:50:38 +0000 (11:50 +0100)]
Fix bug 1448: minor fix in JujuModelWatcher.wait_for_model

Add an `application_ready` function in the Juju Watcher to check if all the
applications of the model are ready.  This function checks that not only
the application is ready, but also all the units of the application.

Additionally, I have added a stability check. After considering a model
as ready, we will wait for 10 seconds, and check again.

Change-Id: Icd8d88613416b75cfd241e7044dbeffa6cbc6a8e
Signed-off-by: David Garcia <david.garcia@canonical.com>
19 months agofix 1208: add native charm support for rhel and fix centos support 13/9713/7
endika [Wed, 16 Sep 2020 13:41:18 +0000 (15:41 +0200)]
fix 1208: add native charm support for rhel and fix centos support

The commit adds SO discovery support for rhel and adds a script to
add a DNAT rule to rhel and centos images.

There are some things that needs to be taken in acount.
- It uses firewalld, wich is the default firewall for centos and rhel,
  instead of iptables. This may break some thinghs if the image uses
  iptables.
- If firewalld is not actibe it assumes that the VM is not using it,
  so it creates a input rule to accept all incoming conections
  (by default rejects all). If the firewall was not enabled, with
  accept all it should have the same behavior.

Change-Id: I29c9781d354c4e8268e19f64dcc9568d725a0de3
Signed-off-by: endika <endika.aldecoa@ehu.eus>
19 months agoFixes for scaling proxy and native charm 64/10364/5
aktas [Sun, 21 Feb 2021 16:36:20 +0000 (19:36 +0300)]
Fixes for scaling proxy and native charm

With changes in here, it can remove application while scaling in
and it can add application existed k8s proxy charm model.

Change-Id: If4fd378fdafd925fa1750ce4769506b44adc233c
Signed-off-by: aktas <emin.aktas@ulakhaberlesme.com.tr>
19 months agoAdd generate_kdu_instance_name method in K8sConn 77/10377/2
David Garcia [Tue, 23 Feb 2021 10:47:29 +0000 (11:47 +0100)]
Add generate_kdu_instance_name method in K8sConn

The reason for adding this method is for LCM to call it before calling
K8sConn.install(). That way, LCM can record it in the DB before
finishing the instantiation of the KDU.

This will help fixing bug 1412.

Change-Id: Iacf71c4d2040dbdc966cff2bddd697c4ea1c9a06
Signed-off-by: David Garcia <david.garcia@canonical.com>
19 months agoFix flake8 minor issues 05/10305/1
David Garcia [Fri, 12 Feb 2021 16:18:39 +0000 (17:18 +0100)]
Fix flake8 minor issues

Change-Id: I1348e9807b8071a1cbf50b97144069a5b6709def
Signed-off-by: David Garcia <david.garcia@canonical.com>
19 months agoRevert "Revert "Clean up commented or unused code"" 46/10246/1
almagia [Wed, 3 Feb 2021 09:48:43 +0000 (10:48 +0100)]
Revert "Revert "Clean up commented or unused code""

This reverts commit c1cf1b9a33dd1929fb834424626661f9c9dce64d.

Code Clean up approved by TSC on Feb 2nd

Change-Id: I069215c71aaebf3cf7009a3538e4e5112e66103d

20 months agoRevert "Clean up commented or unused code" 44/10244/1
David Garcia [Mon, 1 Feb 2021 09:39:56 +0000 (10:39 +0100)]
Revert "Clean up commented or unused code"

This reverts commit b4e7f5c425df48f7e946d792184d1d1b44879fe9.

Change-Id: I31a65516e65c3eb6528f241e36c1e45f3753f4c0
Signed-off-by: David Garcia <david.garcia@canonical.com>
20 months agoClean up commented or unused code 43/10243/1
David Garcia [Mon, 1 Feb 2021 09:37:23 +0000 (10:37 +0100)]
Clean up commented or unused code

Change-Id: Ib116128c15d727b9124e51d2475556045b79ac0d
Signed-off-by: David Garcia <david.garcia@canonical.com>
20 months agoFix bug 1400: Change repo stable for helm2 31/10131/5
lloretgalleg [Thu, 17 Dec 2020 12:42:11 +0000 (12:42 +0000)]
Fix bug 1400: Change repo stable for helm2

Change-Id: Idc4c6454e539b0935d57b5b0cdcc0895b5d00895
Signed-off-by: lloretgalleg <illoret@indra.es>
21 months agoChanged url stable repository for helm3 29/10129/3 release-v9.0-start v9.0.0 v9.0.0rc1
lloretgalleg [Thu, 17 Dec 2020 09:19:30 +0000 (09:19 +0000)]
Changed url stable repository for helm3

Change-Id: I28c45df018e32a7bc984645a8f058baaa1f7b6c7
Signed-off-by: lloretgalleg <illoret@indra.es>
21 months agoPin pylint version 18/10118/1
beierlm [Wed, 16 Dec 2020 12:07:10 +0000 (07:07 -0500)]
Pin pylint version

Pins the version of pylint to 2.6.0, which corrects the upstream
dependency on SortImports.

Change-Id: I6d3a08ab34e62c03e119f84f66452ec4934c19c4
Signed-off-by: beierlm <mark.beierl@canonical.com>
21 months agoCorrected bugs on support for helm v3 16/10016/3
lloretgalleg [Fri, 20 Nov 2020 11:28:08 +0000 (11:28 +0000)]
Corrected bugs on support for helm v3

Change-Id: I064a8d45490989c75a31384cf16e7ab8755542c3
Signed-off-by: lloretgalleg <illoret@indra.es>
22 months agoFix bug 1298 47/10047/13
David Garcia [Fri, 27 Nov 2020 14:32:02 +0000 (15:32 +0100)]
Fix bug 1298

This bug was caused because a bad parsing of the kubeconfig.
The token should be the secret from k8s for the created service
account.

When adding a k8s cluster, a clusterrole, clusterrolebinding, and
serviceaccounts are created.

Tests are needed for oauth2 and userpass kubeconfigs.

Change-Id: I6a4a2834bd6477f255e8ca48e7f53cd3a0d3fddf
Signed-off-by: David Garcia <david.garcia@canonical.com>
22 months agoSupport for helm v3 03/9903/8
lloretgalleg [Thu, 22 Oct 2020 09:12:35 +0000 (09:12 +0000)]
Support for helm v3

Added support for helm3 and included unit tests for helm and helm v3

Change-Id: I1a4a14be2e0a5c82445b97a9d263fd67139fd25b
Signed-off-by: lloretgalleg <illoret@indra.es>
22 months agoSyncronize with fs before action and reverse sync after action to allow high availabi... 73/9673/6
lloretgalleg [Wed, 2 Sep 2020 09:40:38 +0000 (09:40 +0000)]
Syncronize with fs before action and reverse sync after action to allow high availability with mongofs

Change-Id: I4b8b2411d605de8e1743d835aff994ad788ae997
Signed-off-by: lloretgalleg <illoret@indra.es>
22 months agoFix minor bug in configure application 65/9965/3
David Garcia [Wed, 11 Nov 2020 15:56:06 +0000 (16:56 +0100)]
Fix minor bug in configure application

Variable model was unassigned if the connection to the controller was
failing

Change-Id: I14764079f505c46ff7c9af7adec8079b43eca14e
Signed-off-by: David Garcia <david.garcia@canonical.com>
22 months agoFeature 9952: Distributed Proxy Charms 37/9937/4
David Garcia [Tue, 3 Nov 2020 19:37:06 +0000 (20:37 +0100)]
Feature 9952: Distributed Proxy Charms

Add cloud_name and cloud_credential arguments in these functions to
allow allocating proxy charms in different clouds:
  - create_execution_environment
  - register_execution_environment
  - install_k8s_proxy_charm
Change-Id: I3e57c6e5b9e25f9abfb7a579a188cf98d457e2b2
Signed-off-by: David Garcia <david.garcia@canonical.com>
22 months agoFeature 9751: Centralized VCA for KNFs 33/9733/15
David Garcia [Tue, 22 Sep 2020 12:52:32 +0000 (14:52 +0200)]
Feature 9751: Centralized VCA for KNFs

- Use libjuju.py for the communication with VCA
- Add the k8s_cloud as an external cloud to the VCA
- Add unit tests

Change-Id: Id55bfada3957f35e13cef7b4bfcc7acb72452df0
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoFix bug 1263 02/9902/1
David Garcia [Thu, 22 Oct 2020 08:50:56 +0000 (10:50 +0200)]
Fix bug 1263

Update database endpoints when a new VCA_HOST_IP is provided

Before, we were not considering the that the VCA_HOST_IP could be
updated, in that case, the old ips stored in the database should be
replace by the new one. Of course, this won't do any model migration
from the previous VCA to the new one. It will only start pointing to
the new VCA provided.

Change-Id: I5e259f773f90e4f440d87c4b6342cd82425c0e0c
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoFix bug 1267: Destroy machines that are pending 98/9898/5
David Garcia [Wed, 21 Oct 2020 12:19:45 +0000 (14:19 +0200)]
Fix bug 1267: Destroy machines that are pending

When osm destroys a NS, and a native charm hasn't been able
to finish the provisioning of a machine, it will be in a pending state.
As it was seen in the bug 1267, when a model is destroyed and machines
are pending, the deletion of the model will get stuck.

This patch solves that by checking (before deleting the model) if there
are any manually provisioned machines in the juju model (native charms)
in a pending state. If there are, it will destroy them so that the deletion
of the model won't get stuck.

Change-Id: Ia3f6ae7390ccc2bb5d4874d7d08fd446019b0901
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoFix bug 1216: Force model deletion 89/9889/6
David Garcia [Tue, 29 Sep 2020 17:48:13 +0000 (19:48 +0200)]
Fix bug 1216: Force model deletion

Change-Id: I3f41eead305349eba5d404f5f3d61d33f0536aca
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoFix bug 1236: Retry if leader unit is not obtained 32/9832/4
David Garcia [Thu, 15 Oct 2020 11:16:45 +0000 (13:16 +0200)]
Fix bug 1236: Retry if leader unit is not obtained

There a racing condition when N2VC tries to get the leader unit, but there
leader-elected hook has not been triggered. In that case, day-1 action might fail.

This commit solved this issue by retrying a couple of times.

Change-Id: If79cd243aa9ebdf8ed0e6235481eeb9fd2640612
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoAdd credential_name option for add_k8s() and add_cloud() 61/9861/1
David Garcia [Fri, 16 Oct 2020 13:38:13 +0000 (15:38 +0200)]
Add credential_name option for add_k8s() and add_cloud()

Change-Id: I4adf740231e4c0d8785768631a8f2daedc198947
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoAdd credential_name in Libjuju.add_model() function 54/9854/1
David Garcia [Fri, 16 Oct 2020 12:37:37 +0000 (14:37 +0200)]
Add credential_name in Libjuju.add_model() function

Change-Id: Ic3ca0fce931de4b9f7d200e99feb604f21c1ab83
Signed-off-by: David Garcia <david.garcia@canonical.com>
23 months agoAdd loop and vca_config options to K8sJujuConnector constructor 49/9849/1
David Garcia [Fri, 16 Oct 2020 11:00:18 +0000 (13:00 +0200)]
Add loop and vca_config options to K8sJujuConnector constructor

Change-Id: I65edb055af2a67d85f495812118721f1eefa53a5
Signed-off-by: David Garcia <david.garcia@canonical.com>
2 years agoAdd new kubectl.py functions, modify some libjuju.py functions, add unit tests 23/9723/5
David Garcia [Mon, 21 Sep 2020 14:19:15 +0000 (16:19 +0200)]
Add new kubectl.py functions, modify some libjuju.py functions, add unit tests

- Kubectl.py: two new functions added (get_configuration and get_default_storage_class)
  - get_configuration(): Returns a kubernetes Configuration object.
    It can be used to properly parse the kubeconfig.
  - get_default_storage_class(): Searches for the default storage class of a k8s cluster.
- Libjuju.py: modified add_k8s function and get_k8s_cloud_credential function was added.
  - add_k8s(): Improves the way of generation Cloud and CloudCredential objects for the K8s Cloud
  - get_k8s_cloud_credential(): It parses the kubeconfig to properly determine the authentication
    method type that should be used for that k8s cluster.
- Unit tests: Added unit tests for all the new functions added.
- Exceptions: Make all Juju Exceptions to inherit from N2VC Exception.
  Now Juju exceptions have the message attribute, that is useful for unit testing, to not only check that
  an exception raised, but to check the message too.
- Move get_k8s_cloud_credential() function to n2vc/utils in order to share that code between different connectors.

Change-Id: Ife9027d80663fe95f1f3ad883cb9a3376b047d0b
Signed-off-by: David Garcia <david.garcia@canonical.com>
2 years agoAdd get_metrics command to n2vc_juju_conn and libjuju 26/9726/1
David Garcia [Mon, 21 Sep 2020 17:51:23 +0000 (19:51 +0200)]
Add get_metrics command to n2vc_juju_conn and libjuju

Change-Id: I6cde1528579c8ca7aa72d935c5e570004468bb1a
Signed-off-by: David Garcia <david.garcia@canonical.com>
2 years agoAdd add_k8s, add_cloud, and remove_cloud commands to libjuju.py and unit tests 17/9717/1
David Garcia [Thu, 17 Sep 2020 14:01:48 +0000 (16:01 +0200)]
Add add_k8s, add_cloud, and remove_cloud commands to libjuju.py and unit tests

- add_k8s: Generates Cloud and CloudCredential objects for adding a Kubernetes cloud to the VCA
- add_cloud: Takes Cloud and CloudCredential as arguments for adding the cloud to the VCA
- remove_cloud: Remove cloud from VCA

Change-Id: Ia6b4c0cbd06f38df6fe4c52414f5bcb8ffb9a5a8
Signed-off-by: David Garcia <david.garcia@canonical.com>
2 years agoRemove EntityType from juju watcher and workaround juju bug for retrieving the status 12/9712/4
David Garcia [Wed, 16 Sep 2020 11:31:33 +0000 (13:31 +0200)]
Remove EntityType from juju watcher and workaround juju bug for retrieving the status

- The juju watcher was doing an unnecessary translation with the entity types. The entity already provides an attribute .entity_type
- Juju has a bug in version 2.8.2 that returns a wrong status. Therefore, charms were getting stuck in waiting for ever.
Change-Id: I44709190acc41601e8a67f4c52074fda00c3d495
Signed-off-by: David Garcia <david.garcia@canonical.com>
2 years agoRevert "Revert ""Remove unused lines of code""" 11/9711/1
almagia [Wed, 16 Sep 2020 07:44:40 +0000 (09:44 +0200)]
Revert "Revert ""Remove unused lines of code"""

This reverts commit 8bfcc14713a71f43f155e3cddec168380134d344.

approved by TSC on 15/09/20

Change-Id: I176b1b625cbc2d94013921401b2e6048aed675f8

2 years agoRevert ""Remove unused lines of code"" 08/9708/1
David Garcia [Mon, 14 Sep 2020 13:10:04 +0000 (15:10 +0200)]
Revert ""Remove unused lines of code""

This reverts commit e8102d9e28e5c502fc66ca842d14e1ad29efbfda.

Change-Id: Ic22f292f601f45451d3fdd56ab98bfa4ea9161eb
Signed-off-by: David Garcia <david.garcia@canonical.com>