chia-blockchain/tests/util/test_lock_queue.py
Mariano Sorgente 8a028c3594
Ms.mempool locking (#9050)
* Prority locking to consensus

* Remove pstats

* Linting

* Do some stuff outside of lock

* Fix startup

* Add log timings

* Try some different locking

* Add limit

* catch excp

* CLVM inside lock

* Try using a semaphore instead

* use events for lock queue

* test

* Add logging for message types

* type

* remove seed

* check new peak waiters

* correct FullNodeAPI self.full_node.new_peak._waiters typo

* correct logging string typos

* only warn about new_peak Waiters if there is at least 1

* remove no-longer-accepted parameter to FullNode.peak_post_processing()

* only warn about respond_transaction Waiters if there is at least 1

* lint

* Change some constants

* Small fix and logging changes

* Put message types outside

* Change some log levels so we can test with info

* More logging

* Increase rate limits but decrease paralelism

* tweaks

* Log dropped tx

* Fix pool rpc test

* Test fixes

* Mempool optimization

* Remove from seen if fails

* Increase queue sizes

* Message types info

* More test and logging

* Small changes to networking just in case

* Decrease logging

* Decrease logging even further

* Decrease logging even further even further

* Decrease logging 3

* Transaction queue

* Don't cancel tasks or close connection

* Cancel tasks on disconnect (for shutdown purposed)

* Fix typo

* Catch cancelled

* Do multiple at a time

* More accurate farmer response time

* More efficiently create tasks

* Increase queue size and priority by fee

* Revert priority

* Don't re-request too many times for dropped TX

* Handle cancelled error so we don't go into a bad state

* Catch cancelled in syncing tasks

* Reduce new_peak_sem to improve performance

* Less bytes conversion

* Missing file, and 2 workers for CLVM

* Validate BLS in a new thread

* tests

* Change semaphore constants

* correct a cancellation triggered exception and assertion

* Fix send_transaction, dont use BaseException, fix tests

* Fix more tests

* only log transaction handler cancellation in debug

* typing in log

* move unfinished validation to diff proc

* it is asyncio.CancelledError

* Add a test for bad signature

* Fix more tests, reduce logging, lint

* One more lint

* blockchain tests, pass bytes directly, single call

* Try to fix rl_wallet failures

* Fix mempool test

* catch everything

* Don't test RL wallet

* Fix more tests and return error code

* Improve error handling in multiprocess

* Add pre-validation time

* Add pre-validation time in logs, and revert pytest.ini changes

* Add log correctly

* Ms.bls cache experiment (#9115)

* Logging for cache

* Less logging

* Return to original plan

* Clean up

* Remove coment

* Remove log

* formalize LockQueue shutdown

* Comments

* Fix blockchain test

* Improve cache

* Remove logs

* Fix sign_coin_spends

* Fix pool wallet

Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
2021-11-04 09:29:05 -07:00

82 lines
2.1 KiB
Python

import asyncio
import logging
import time
from asyncio import CancelledError
import pytest
from chia.full_node.lock_queue import LockQueue, LockClient
log = logging.getLogger(__name__)
@pytest.fixture(scope="module")
def event_loop():
loop = asyncio.get_event_loop()
yield loop
class TestLockQueue:
@pytest.mark.asyncio
async def test_lock_queue(self):
lock = asyncio.Lock()
queue = LockQueue(lock)
low_priority_client = LockClient(1, queue)
high_priority_client = LockClient(0, queue)
async def very_slow_func():
await asyncio.sleep(2)
raise CancelledError()
async def slow_func():
for i in range(100):
await asyncio.sleep(0.01)
async def kind_of_slow_func():
for i in range(100):
await asyncio.sleep(0.001)
async def do_high():
nonlocal high_priority_client
for i in range(10):
log.warning("Starting high")
t1 = time.time()
async with high_priority_client:
log.warning(f"Spend {time.time() - t1} waiting for high")
await slow_func()
async def do_low(i: int):
nonlocal low_priority_client
log.warning(f"Starting low {i}")
t1 = time.time()
async with low_priority_client:
log.warning(f"Spend {time.time() - t1} waiting for low {i}")
await kind_of_slow_func()
h = asyncio.create_task(do_high())
l_tasks = []
for i in range(50):
l_tasks.append(asyncio.create_task(do_low(i)))
winner = None
while True:
if h.done():
if winner is None:
winner = "h"
l_finished = True
for t in l_tasks:
if not t.done():
l_finished = False
if l_finished and winner is None:
winner = "l"
if l_finished and h.done():
break
await asyncio.sleep(1)
assert winner == "h"
queue.close()