Improve race conditions/MySQL reconnection

This commit aims to provide a better synchronization between all the
different threads in RO, specially regarding DB usage and internal state
consistency.

The following improvements were done:

1. Centralize database retry logic into a single function

  This way we can change the procedure and the rules for retrying in a
  single place and this reflects in several functions simultaneously
  avoiding the need for manual copy and paste (and the potential risk of
  forgetting to change somewhere)

2. Minor fixes/improvements related to database connection loss.

  Previously `db_base` was already able to identify when the connection
  to MySQL was lost, but apparently in a few edge cases the automatic
  reconnection was not done.

3. Implement a transaction method

  This method replaces the old context manager API for the connection
  object that was removed from MySQLdb in version 1.4

  In additional it is possible to use a decorator for transactions
  (not only the context manager), which is handy sometimes.

4. Add lock mechanism directly to db_base

  This helps to improve synchronization between threads.
  Some extra synchronization was introduced to functions, as it seemed
  to be the case.
  Moreover, previously, the cursor object was part of the internal state
  of the db_base object, and it was being changed/used without thread
  synchronization (error-prone). Having the locking mechanism around the
  changes in the cursor property of the class, avoids problems.

5. Add option to fork connection

  Useful when independent threading is needed (as long as different
  threads don't access the same database table, having separated
  connections and locks should work fine).

Change-Id: I3ab34df5e8c2857d96ed14a70e7f65bd0b5189a0
Signed-off-by: Anderson Bravalheri <a.bravalheri@bristol.ac.uk>
diff --git a/osm_ro/utils.py b/osm_ro/utils.py
index 1e3a8ee..05c9801 100644
--- a/osm_ro/utils.py
+++ b/osm_ro/utils.py
@@ -32,14 +32,20 @@
 import datetime
 import time
 import warnings
-from functools import reduce
+from functools import reduce, partial, wraps
 from itertools import tee
 
+import six
 from six.moves import filter, filterfalse
 
 from jsonschema import exceptions as js_e
 from jsonschema import validate as js_v
 
+if six.PY3:
+    from inspect import getfullargspec as getspec
+else:
+    from inspect import getargspec as getspec
+
 #from bs4 import BeautifulSoup
 
 def read_file(file_to_read):
@@ -347,3 +353,60 @@
     keys = key_path.split('.')
     target = reduce(lambda acc, key: acc.get(key) or {}, keys[:-1], target)
     return target.get(keys[-1], default)
+
+
+class Attempt(object):
+    """Auxiliary class to be used in an attempt to retry executing a failing
+    procedure
+
+    Attributes:
+        count (int): 0-based "retries" counter
+        max_attempts (int): maximum number of "retries" allowed
+        info (dict): extra information about the specific attempt
+            (can be used to produce more meaningful error messages)
+    """
+    __slots__ = ('count', 'max', 'info')
+
+    MAX = 3
+
+    def __init__(self, count=0, max_attempts=MAX, info=None):
+        self.count = count
+        self.max = max_attempts
+        self.info = info or {}
+
+    @property
+    def countdown(self):
+        """Like count, but in the opposite direction"""
+        return self.max - self.count
+
+    @property
+    def number(self):
+        """1-based counter"""
+        return self.count + 1
+
+
+def inject_args(fn=None, **args):
+    """Partially apply keyword arguments in a function, but only if the function
+    define them in the first place
+    """
+    if fn is None:  # Allows calling the decorator directly or with parameters
+        return partial(inject_args, **args)
+
+    spec = getspec(fn)
+    return wraps(fn)(partial(fn, **filter_dict_keys(args, spec.args)))
+
+
+def get_arg(name, fn, args, kwargs):
+    """Find the value of an argument for a function, given its argument list.
+
+    This function can be used to display more meaningful errors for debugging
+    """
+    if name in kwargs:
+        return kwargs[name]
+
+    spec = getspec(fn)
+    if name in spec.args:
+        i = spec.args.index(name)
+        return args[i] if i < len(args) else None
+
+    return None