Skip to content
GitLab
This project is mirrored from https://:*****@osm.etsi.org/gerrit/osm/RO.git. Pull mirroring updated .
  1. Feb 10, 2019
    • bravalheria's avatar
      Improve race conditions/MySQL reconnection · dfed511c
      bravalheria authored 
      

This commit aims to provide a better synchronization between all the
different threads in RO, specially regarding DB usage and internal state
consistency.

The following improvements were done:

1. Centralize database retry logic into a single function

  This way we can change the procedure and the rules for retrying in a
  single place and this reflects in several functions simultaneously
  avoiding the need for manual copy and paste (and the potential risk of
  forgetting to change somewhere)

2. Minor fixes/improvements related to database connection loss.

  Previously `db_base` was already able to identify when the connection
  to MySQL was lost, but apparently in a few edge cases the automatic
  reconnection was not done.

3. Implement a transaction method

  This method replaces the old context manager API for the connection
  object that was removed from MySQLdb in version 1.4

  In additional it is possible to use a decorator for transactions
  (not only the context manager), which is handy sometimes.

4. Add lock mechanism directly to db_base

  This helps to improve synchronization between threads.
  Some extra synchronization was introduced to functions, as it seemed
  to be the case.
  Moreover, previously, the cursor object was part of the internal state
  of the db_base object, and it was being changed/used without thread
  synchronization (error-prone). Having the locking mechanism around the
  changes in the cursor property of the class, avoids problems.

5. Add option to fork connection

  Useful when independent threading is needed (as long as different
  threads don't access the same database table, having separated
  connections and locks should work fine).

Change-Id: I3ab34df5e8c2857d96ed14a70e7f65bd0b5189a0
Signed-off-by: default avatarAnderson Bravalheri <a.bravalheri@bristol.ac.uk>
      dfed511c
  3. Feb 07, 2019
  5. Feb 05, 2019
  7. Feb 01, 2019
  9. Jan 30, 2019
  11. Jan 28, 2019
  13. Jan 24, 2019
  15. Jan 21, 2019
  17. Jan 18, 2019
  19. Jan 14, 2019
  21. Jan 10, 2019
  23. Jan 08, 2019
  25. Dec 21, 2018
  27. Dec 20, 2018
  29. Dec 18, 2018
  31. Dec 17, 2018
  33. Dec 11, 2018
  35. Dec 10, 2018
  37. Dec 05, 2018
  39. Dec 04, 2018
  41. Dec 03, 2018
  43. Nov 28, 2018
  45. Nov 26, 2018
  47. Nov 23, 2018
  49. Nov 16, 2018
    • bravalheria's avatar
      Implement feature 5949 · 0446cd5d
      bravalheria authored 
      

Enable dynamic connectivity setup in multi-site Network Services

The code required to implement the feature is contained in `osm_ro/wim`
as much as possible.

* `wim/engine.py` works together with `nfvo.py` to implement the
  feature
* `wim/persistence.py` is equivalent to `nfvo_db.py` and try to
  encapsulate most of the SQL-specific code, implementing a persistence
  layer
* `wim/http_handler.py` extends `httpserver.py` adding WIM-related HTTP
  routes
* `wim/wim_thread.py` is similar to `vim_thread.py` and controls the
  execution of WIM-related tasks
* `wim/actions.py` and `wim/wan_link_actions.py` implement the action
  handling specific code, calling instances of the `wim/wimconn.py`
  subclasses

WIM connectors are still a work in progress

Individual change details (newer to older)

- Add errors for inconsistent state

- Delay re-scheduled tasks

- Move lock to inside the persistence object

- Better errors for connector failures

- Try to cache the wan_link information before it is deleted from the database

- Integrate WanLinkDelete to NFVO

- Add WanLinkDelete implementation draft with some tests

- Add basic wim network creation

- Add minimal documentation for actions

- Add checks to the create action

- Improve documentation, rearrange insert_pending and remove unused functions on WimThread

- Integrate Action classes in refresh_tasks

- Add Action classes to avoid intricate conditions

- Adding Proposed License

- Move grouping of actions to persistence

- Change WimThread to use SQL to do the heavy lifting

- Simplify WimThread reload_actions

- Add tests for derive_wan_links

- Implement find_common_wim(s)

- Add tests for create_wim_account

- Add migration scripts for version 33

- Changes to WIM and VIM threads for vim_wim_actions

- Implement wim_account management according to the discussion

- Add WimHandler integration inside httpserver

- Add quick instructions to run the tests

- Add WIM functional tests using real database

- Add DB WIM port mapping

- RO WIM-related console scripts

- Add WIM integration to NFVO

- Improve database support focusing on tests

- RO NBI WIM-related commands in HTTP server

- Adding WIM tables to MANO DB

- Add wim http handler initial implementation

- Move http utility functions to separated files

    This separation allows the code to be reused more easily and avoids
    circular dependencies.

    (The httpserver can import other modules implementing http routes,
    and those modules can then use the utility functions without having
    to import back httpserver)

- Add a HTTP handler class and custom route decorator

    These tools can be used to create independent groups of bottle
    routes/callbacks in a OOP fashion

- Extract http error codes and related logic to separated file

Change-Id: Icd5fc9fa345852b8cf571e48f427dc10bdbd24c5
Signed-off-by: default avatarAnderson Bravalheri <a.bravalheri@bristol.ac.uk>
      0446cd5d
  51. Nov 14, 2018