Improve handling of closed connections (#148)
authorCory Johns <johnsca@gmail.com>
Thu, 22 Jun 2017 20:19:19 +0000 (16:19 -0400)
committerGitHub <noreply@github.com>
Thu, 22 Jun 2017 20:19:19 +0000 (16:19 -0400)
commitcd48f185bcb9279a96d3ee85579d96ac10d12dd9
tree7f82b98fbabd05076995337f6e92efb6e84307b9
parent7dd0c1586fa760fbdf8a3195d67596bd6b26b10d
Improve handling of closed connections (#148)

* Prevent circular reference between Monitor and Connection

* Improve monitor status logic

* Stop watcher task cleanly on disconnect

Instead of the watcher task blowing up with an exception when the
connection is closed, it should exit cleanly and allow the Monitor to
report the connection error.

Fixes #147

* Raise ConnectionClosed exception in Model.block_until

Rather than blocking indefinitely when waiting on a model change that
will never happen if the connection gets closed out from under it, this
makes `Model.block_until` raise a `websockets.ConnectionClosed` error so
that it can be caught and dealt with by the client.

* Automatically reconnect lost websocket connections

The receiver or all_watcher should immediately reconnect a lost
connection, or it will be reconnected automatically upon issuance of the
next RPC call.  However, there is still a small chance that the
disconnect could happen between sending a API call and receiving the
response, in which case a websockets.ConnectionClosed error will still
be raised to the caller.  These should be quite rare, though.

* Gracefully handle add_signal_handler failing in a thread in loop.run

* Restart AllWatcher if controller stops it

Fixes conjure-up/conjure-up#965

* Explicitly let the controller know we're stopping the watcher

* Skip reconnect if close was requested
juju/client/connection.py
juju/loop.py
juju/model.py
tests/base.py
tests/integration/test_connection.py
tests/integration/test_model.py
tests/unit/test_connection.py