Update to have temporal in common
Change-Id: I5a144ed22b65ed5337e0e870ab7281b80c5e0b62
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
Add Python logging to DAGs
Change-Id: I2f8dd3b351ceb9a7da8e8b28d392e6fdef73f663
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Change in tox.ini to use allowlist_externals instead of whitelist_externals
Change-Id: Id457e368adffcc81d3e7451015b2c0905b9cb7ea
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Minor change in tox.ini related to flake8 env
Change-Id: I7d83cce754dc73f515af771855ba682783109569
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Fix black and flake errors in setup.py
Change-Id: I9522836f89e660c280c49a75ad5fd853454401e3
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Update ns_topology and multivim_vm_status DAGs to disable overlapped execution
Change-Id: Idbfa27879a3604e45cf4c92270c4c86de48bca93
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: skeleton of osm_webhook_translator package
Change-Id: I80217214941241e1e97dee80b978e0f1e55f4d1a
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: Added main for webhook-translator
Change-Id: Idcc45514261eeb645becc56c0aee5f681b49fb0a
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Feature 10981: added autohealing DAG and updated requirements
Change-Id: Ib1ed56c220969d54480ddd2382beae03e536b72b
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Fix bug in multivim_vm_status DAG when there are no VMs
Change-Id: Idd67bb9f59a61edbe15012ca05df8c83d920d04e
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Add MANIFEST.in to osm_webhook_translator to include README and requirements
Change-Id: I7f47826c5438348894ff525c591cfd93415ace04
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Clean stage-archive.sh
Change-Id: Ib42096edb6bcbb29031749291fe5de410cb93755
Signed-off-by: garciadeblas <gerardo.garciadeblas@telefonica.com>
Feature 10981: use Python library for logging in Webhook
Change-Id: Ib60ef2005618f94da34da1910fb26f95d77bb7a2
Signed-off-by: aguilard <e.dah.tid@telefonica.com>
Signed-off-by: Mark Beierl <mark.beierl@canonical.com>
diff --git a/osm_webhook_translator/MANIFEST.in b/osm_webhook_translator/MANIFEST.in
new file mode 100644
index 0000000..5f44b2f
--- /dev/null
+++ b/osm_webhook_translator/MANIFEST.in
@@ -0,0 +1,20 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+
+include src/osm_webhook_translator/requirements.txt
+include src/osm_webhook_translator/README.rst
+
diff --git a/osm_webhook_translator/README.rst b/osm_webhook_translator/README.rst
new file mode 100644
index 0000000..a84b409
--- /dev/null
+++ b/osm_webhook_translator/README.rst
@@ -0,0 +1,33 @@
+..
+ #######################################################################################
+ # Copyright ETSI Contributors and Others.
+ #
+ # Licensed under the Apache License, Version 2.0 (the "License");
+ # you may not use this file except in compliance with the License.
+ # You may obtain a copy of the License at
+ #
+ # http://www.apache.org/licenses/LICENSE-2.0
+ #
+ # Unless required by applicable law or agreed to in writing, software
+ # distributed under the License is distributed on an "AS IS" BASIS,
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ # implied.
+ # See the License for the specific language governing permissions and
+ # limitations under the License.
+ #######################################################################################
+
+==================
+webhook-translator
+==================
+
+webhook-translator is a component in the Service Assurance architecture for OSM.
+
+Its role is to receive alerts from entities such as Prometheus AlertManager or external systems, and to translate them to a format that can be consumed by Airflow DAGs. It basically receives HTTP POST messages and forwards them to an Airflow webhook
+
+The main characteristics are:
+
+* Lightweight: a very small number of lines of code does the work.
+* Stateless. It only translates HTTP requests. No state for those translations. When running as a Kubernetes deployment, native scaling is achieved by means of Kubernetes services.
+* Simple. Based on `FastAPI <https://fastapi.tiangolo.com/>`
+* Independent from the source of the alert. No maintenance is required to incorporate new alert sources.
+
diff --git a/osm_webhook_translator/requirements-dist.in b/osm_webhook_translator/requirements-dist.in
new file mode 100644
index 0000000..03ff6e9
--- /dev/null
+++ b/osm_webhook_translator/requirements-dist.in
@@ -0,0 +1,18 @@
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+stdeb
+setuptools-scm
+setuptools<60
diff --git a/osm_webhook_translator/requirements-dist.txt b/osm_webhook_translator/requirements-dist.txt
new file mode 100644
index 0000000..6ddded6
--- /dev/null
+++ b/osm_webhook_translator/requirements-dist.txt
@@ -0,0 +1,32 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+packaging==23.0
+ # via setuptools-scm
+setuptools-scm==7.1.0
+ # via -r osm_webhook_translator/requirements-dist.in
+stdeb==0.10.0
+ # via -r osm_webhook_translator/requirements-dist.in
+tomli==2.0.1
+ # via setuptools-scm
+typing-extensions==4.5.0
+ # via setuptools-scm
+
+# The following packages are considered to be unsafe in a requirements file:
+setuptools==59.8.0
+ # via
+ # -r osm_webhook_translator/requirements-dist.in
+ # setuptools-scm
diff --git a/osm_webhook_translator/requirements.in b/osm_webhook_translator/requirements.in
new file mode 100644
index 0000000..33c6214
--- /dev/null
+++ b/osm_webhook_translator/requirements.in
@@ -0,0 +1,20 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+
+fastapi
+requests
+uvicorn
diff --git a/osm_webhook_translator/requirements.txt b/osm_webhook_translator/requirements.txt
new file mode 100644
index 0000000..6cf7f42
--- /dev/null
+++ b/osm_webhook_translator/requirements.txt
@@ -0,0 +1,48 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+anyio==3.6.2
+ # via starlette
+certifi==2022.12.7
+ # via requests
+charset-normalizer==3.1.0
+ # via requests
+click==8.1.3
+ # via uvicorn
+fastapi==0.95.0
+ # via -r osm_webhook_translator/requirements.in
+h11==0.14.0
+ # via uvicorn
+idna==3.4
+ # via
+ # anyio
+ # requests
+pydantic==1.10.7
+ # via fastapi
+requests==2.28.2
+ # via -r osm_webhook_translator/requirements.in
+sniffio==1.3.0
+ # via anyio
+starlette==0.26.1
+ # via fastapi
+typing-extensions==4.5.0
+ # via
+ # pydantic
+ # starlette
+urllib3==1.26.15
+ # via requests
+uvicorn==0.21.1
+ # via -r osm_webhook_translator/requirements.in
diff --git a/osm_webhook_translator/setup.py b/osm_webhook_translator/setup.py
new file mode 100644
index 0000000..3323757
--- /dev/null
+++ b/osm_webhook_translator/setup.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+#
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+from setuptools import find_namespace_packages, setup
+
+exec(open("src/osm_webhook_translator/_version.py").read())
+
+_name = "osm_webhook_translator"
+_description = "OSM Webhook Translator"
+with open(os.path.join(".", "README.rst")) as readme_file:
+ README = readme_file.read()
+
+setup(
+ name=_name,
+ description=_description,
+ long_description=README,
+ version=__version__, # noqa: F821
+ author="ETSI OSM",
+ author_email="osmsupport@etsi.org",
+ maintainer="ETSI OSM",
+ maintainer_email="osmsupport@etsi.org",
+ url="https://osm.etsi.org/gitweb/?p=osm/NG-SA.git;a=summary",
+ license="Apache 2.0",
+ package_dir={"": "src"},
+ packages=find_namespace_packages(where="src"),
+ include_package_data=True,
+)
diff --git a/osm_webhook_translator/src/osm_webhook_translator/__init__.py b/osm_webhook_translator/src/osm_webhook_translator/__init__.py
new file mode 100644
index 0000000..d5daf3e
--- /dev/null
+++ b/osm_webhook_translator/src/osm_webhook_translator/__init__.py
@@ -0,0 +1,17 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+from osm_webhook_translator._version import __version__ # noqa: F401
diff --git a/osm_webhook_translator/src/osm_webhook_translator/main.py b/osm_webhook_translator/src/osm_webhook_translator/main.py
new file mode 100644
index 0000000..6e9e718
--- /dev/null
+++ b/osm_webhook_translator/src/osm_webhook_translator/main.py
@@ -0,0 +1,63 @@
+#######################################################################################
+# Copyright ETSI Contributors and Others.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#######################################################################################
+from datetime import datetime
+import logging
+import os
+from random import randint
+
+from fastapi import FastAPI
+import requests
+
+
+logging.basicConfig(
+ format="%(asctime)s %(levelname)s %(filename)s:%(lineno)s %(message)s",
+ datefmt="%Y/%m/%d %H:%M:%S",
+)
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.INFO)
+app = FastAPI()
+
+
+def send_to_airflow(output_endpoint, content):
+ try:
+ requests.Session()
+ # Airflow params should come from env variables from configmaps and secrets
+ airflow_host = os.environ["AIRFLOW_HOST"]
+ airflow_port = os.environ["AIRFLOW_PORT"]
+ airflow_user = os.environ["AIRFLOW_USER"]
+ airflow_pass = os.environ["AIRFLOW_PASS"]
+ url = f"http://{airflow_host}:{airflow_port}/api/v1/dags/{output_endpoint}/dagRuns"
+ rnd = str(randint(0, 999999)).rjust(6, "0")
+ timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
+ dag_run_id = output_endpoint + "_" + timestamp + "_" + rnd
+ logger.info(f"HTTP POST {url}")
+ req = requests.post(
+ url=url,
+ auth=(airflow_user, airflow_pass),
+ json={"dag_run_id": dag_run_id, "conf": content},
+ )
+ logger.info(f"Response: {req.text}")
+ # timeout and retries
+ except Exception as e:
+ logger.error(f"HTTP error: {repr(e)}")
+ raise requests.HTTPException(status_code=403, detail=repr(e))
+
+
+@app.post("/{input_endpoint}")
+async def webhook(input_endpoint: str, content: dict):
+ send_to_airflow(input_endpoint, content)
+ return {}