Previously, when the refresh-rate timer activated, and the thread from
the previous activation was still running, we would kill it and start
a new one. For low refresh rates, on slower machines, nodes, or network
connections, this could cause the update to never conclude.
Here we add a timeout-time to eth-watcher's config. If the refresh-rate
timer activates, and a thread exists, but hasn't been running for at
least the specified timeout-time yet, we simply take no action, and wait
for the next refresh timer.
Note that we opted for "at least timeout-time", instead of killing &
restarting directly after the specified timeout-time has passed, to
avoid having to handle an extra timer flow.
In the +on-load logic, we configure the timeout-time for existing
watchdogs as six times the refresh-rate. We want to set
azimuth-tracker's timeout-time to ~m30, and don't care much about other,
less-likely-to-be-active use cases of eth-watcher.
These were deprecated in favor of azimuth-tracker in #1320.
(Azimuth-tracker, however, isn't a general-purpose Ethereum log watcher
tool. Commits to transform it into a more broadly useful tool are
forthcoming.)