chia-blockchain/tests/wallet/test_wallet_retry.py
Kyle Altendorf 7e3e18bdfd
Catchup/long lived atari from release 1.4.0 46ebed37d8 (#11648)
* Resend transactions (#11167)

* Resend transactions

* Don't recheck transactions more frequently than timeout

* Add wallet resend parameter to config, move timeout code out of tx store, but close to call site

* Add a test for wallet transaction resend

* Add test for wallet retry

* isort new files for precommit and update workflows

* Use correct fixture name

* LGTM - remove unused import

Co-authored-by: Earle Lowe <e.lowe@chia.net>

* less except -> false (#10219)

* less except -> false

* establish_connection() and .perform_handshake() can just return None

* remove more pointless, and no wrong, assertions

* Write python version error to stderr (#11239)

Co-authored-by: wjblanke <wjb98672@gmail.com>

* simplify SizedBytes and StructStream (#11429)

* simplify SizedBytes and StructStream

* lint

* super()  !!!

* do not pass parameter up to super().__init__()

* Update chia/util/struct_stream.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* parse fixed-width int data from class name

* add int512 and uint128 .from_bytes(), test .parse() failures

* test serialization against struct.pack()

* use typing_extensions for final

* override ignore

* stop using struct for StructStream

oh the irony

* fixup .to_bytes() to accept parameters again for where we use that

* bring back signed parameter

* format

* adjust tests for new exception

* eliminate custom coding for uint128 and int512

* tidy

* remove unused StructStream.PACK attribute

* add direct tests for parse_metadata_from_name()

* stricter hinting

* remove no-longer-needed typeshed work-around

* apply strict type checking to all touched files

* remove StructStream override of .to_bytes()

* tidy

* types touchup

* add unused parameter comments

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: wjblanke <wjb98672@gmail.com>

* Allow services to set a non-default max request body size limit (#11516)

* updated bls to 13 (#11529)

* reduce the redundant computations of coin_ids in block_body_validation (#11530)

* add test for streamable -> json conversion (#11527)

* add test for streamable -> json conversion

* fixup test

* Uses the new `from_bytes_unchecked` method in blspy, to improve perfo… (#11463)

* Uses the new `from_bytes_unchecked` method in blspy, to improve performance

* Update test

* Fix merge conflict

* Use from_bytes_unchecked in post_init and from_json

* More uniform code

* rename test files that are missing test_ prefix (#10712)

* rename test files that are missing test_ prefix

* update mypy and isort excludes

* skip test_get_host_addr6() in GitHub Actions under macOS

* rebuild workflows

* Coin Selection Refactor With CAT Coin Selection Refactor  (#9975)

* add exact match and best exact match algorithms

* optimize algorithm further

this might be good.

* lint

* fix bad logic

* add final algorithms

* delete lint

* oops

* Update coin_selection.py

* simplify and fix knapsack algoritm

* simplify code and correct logic

* make it way better.

* clarify comments and check for edge cases.

* add comments and stuff

* improve coin selection

addressed comments
Thanks!

* add coin_selection rpc tests.

* clean up and add new unit tests

* undo test changes

* add extra test cases

* move coin_selection to its own function and switch to it for cat and main wallet.

* add cat tests

* lint

* make function align with standards

also removed test

* make test better

* add proper types

* Improve code clarity

* wallet: fix coin selection bugs

* wallet: add an assert just in case

* tests: add some sleeps to reduce flakiness

* Isort

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* fix bad merge

* lint

* fix tests

* address aforementioned changes.

* remove wallet test

* isort

* more tests and fixes

* lint

* rename to amount for coin selection rpc

* fix incase we have no smaller coins

* fix tests + lint

* re add asserts

* oops

missed me.

* lint

* fix test

* Squashed commit of the following:

commit 34a2235de5
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Apr 13 10:09:42 2022 -0400

    clarify comment

commit adbf7f4f94
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Tue Apr 12 20:27:05 2022 -0400

    linty lint

commit 5ebc1ac9fd
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Tue Apr 12 20:17:19 2022 -0400

    add failure test and final changes

commit 7e5a21b4c2
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Tue Apr 12 19:35:18 2022 -0400

    add descriptions and slim down code

commit 31c95b916d
Merge: d7b91295b d9b0ef5f3
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Mon Apr 11 10:12:05 2022 -0400

    Merge branch 'jack-cat-coinselection' into jn_coinselection_dust

commit d7b91295b5
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Sun Apr 10 20:31:09 2022 -0400

    lint

commit 30dc7c0ab4
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Sun Apr 10 20:25:52 2022 -0400

    fix tests

commit 6c8c2e4874
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Thu Mar 31 15:06:00 2022 -0400

    remove duplicate code.

commit 9f79b6f304
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Thu Mar 31 15:01:10 2022 -0400

    address more concerns

commit 67c1b3929f
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Thu Mar 31 12:59:05 2022 -0400

    fix logic error

commit 2d19a53245
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Thu Mar 31 11:47:52 2022 -0400

    simplify and de duplicate code

commit 6ab1cc79bb
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 21:34:50 2022 -0400

    add function and select individual coin

commit 582c17aa8d
Merge: ce2165942 618fbaeba
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 21:14:37 2022 -0400

    Merge branch 'jack-cat-coinselection' into jn_coinselection_dust

commit ce21659429
Merge: 16aabb3fd 6daba28db
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 20:53:21 2022 -0400

    Merge branch 'jack-cat-coinselection' into jn_coinselection_dust

commit 16aabb3fd5
Merge: 0b9fc2845 2286fe426
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 20:49:02 2022 -0400

    Merge branch 'jack-cat-coinselection' into jn_coinselection_dust

commit 0b9fc28455
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 20:38:12 2022 -0400

    lint

commit 62e74c72f4
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 20:34:22 2022 -0400

    fix logic and tests

commit e738f44320
Author: Jack Nelson <jack@jacknelson.xyz>
Date:   Wed Mar 30 18:52:05 2022 -0400

    deal with dust and add tests

* make sure that we do not use any dust

* minor change

* address concerns

* adjust comments

* adjust comment

Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>

* rebuild workflows (#11543)

* remove the cache from CoinStore. It appears the cost of maintaining the cache outweighs the gains. (#11540)

* Keep daemon websocket alive during keyring unlock (#11371)

* When prompting to unlock the keyring during daemon launch, it's possible
that the daemon websocket connection will timeout before the user has
entered their passphrase. We can't quickly detect that the connection
has timed-out, so instead we now reconnect after collecting the
passphrase, unlock the keyring, and then close the old connection.

* mypy fix

* Switch to using asyncio.loop.run_in_executor to collect the passphrase on its own thread.

* Use a new ThreadPoolExecutor configured with 1 worker thread.

* Support searching derived addresses on testnet. (#11449)

* Support searching derived addresses on testnet.

* Added tests

* Optimize code to not perform useless subgroup checks (#11546)

* Optimize code to not perform useless subgroup checks

* Revert less important optimizations

* hints and strict type checking for test_wallet (#11541)

* hints and strict type checking for test_wallet

* fixup

* flip flop

* disable pytest-monitor by default (#11507)

* disable pytest-monitor by default

* take 2

* make recurse_jsonify() work directly on types (#11537)

* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols

* add benchmark for Streamable.to_json_dict()

* tests: Split up and improve `test_wallet_rpc.py` (#11552)

This breaks up the one big test into 

```
test_send_transaction
test_create_signed_transaction
test_create_signed_transaction_with_coin_announcement
test_create_signed_transaction_with_puzzle_announcement
test_send_transaction_multi
test_get_transactions
test_get_transaction_count
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```

Note that there is still much more room for improvements. Also the 
following tests can still be split up more:

```
test_get_transactions
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```

* Remove unneeded uint64() intermediate in CAT wallet (#11575)

* Get get_args() and get_origin() from typing_extensions (#11571)

* Get get_args() and get_origin() from typing_extensions

* Update streamable.py

* cleanup

* Fix several flaky tests (#11576)

* Increase some of the timeouts to reduce flakiness

* Experiment with CI for flakiness

* Force more runs

* Add argument name

* Fix flaky mempool test

* Attempt to fix another flaky test

* Fix wallet retry test

* Missing arg

* Lint

* Increase benchmark test time

* Debug

* Debug

* Simplify retry test

* Lint

* More lint

* Return false instead of assert

* No need to check disconnect twice (since we don't break, this was flaky)

* Remove useless changes

* Accidental changes revert

* Accidental changes revert 2

* Accidental changes revert 3

* Revert fixture

* Fix a few additional flaky tests (#11597)

* Fix a few additional flaky tests

* Lint

* Update tests/core/full_node/stores/test_full_node_store.py

Co-authored-by: Jeff <jeff@chia.net>

* Fix merge conflict

* Remove unnecessary function

Co-authored-by: Jeff <jeff@chia.net>

* Bind port 0 to fix race condition when grabbing available ports (#11578)

* port 0 to fix flakiness

* Try fixing setup_full_system

* Try fixing setup_full_system, and lint

* More attempts to fix

* No more calls to get random ports in setup_nodes

* Revert accidental changes

* Timelord extra arg

* Try with port 0

* Fix daemon test, and lint

* Try without 0.0.0.0

* Back to 0.0.0.0

* Try a few timelord changes to get test running

* Increase timeout again

* Use the correct interface to get the port

* INFO logging to debug issue

* Revert "INFO logging to debug issue"

This reverts commit 7c379e5cca.

* Fix advertised port log

* Add extra log

* Logging back

* Rollback the timelord changes

* Try port 0 timelord

* Revert "Try port 0 timelord"

This reverts commit 4997faf3b2.

* Try full green, change ordering

* Remove unused var

* speed up simulation and cleanup

* Now try without the port config

* Fix a flaky call to get_event_loop

* Try getting the port dynamically

* No dynamic port

* Try changing the ordering

* Try adding a sleep

* Back to what works

* Timelord before vdf clients

* Dynamic port for 1st timelord

* Revert "Dynamic port for 1st timelord"

This reverts commit 0f322a15b7.

* Revert "Timelord before vdf clients"

This reverts commit 3286c34696.

* Revert "Back to what works"

This reverts commit 30380dffb7.

* Revert "Try adding a sleep"

This reverts commit 9212b665a6.

* Revert "Try changing the ordering"

This reverts commit a62597d70d.

* Revert "No dynamic port"

This reverts commit 5d2e15749b.

* Revert "Try getting the port dynamically"

This reverts commit ef9cd75679.

* Revert "Fix a flaky call to get_event_loop"

This reverts commit 01a000fdfb.

* Try one to 0

* Just not 0

* Don't get port dynamically

* Cleanup a bit

* Fix

* Some cleanup work

* Some cleanup work

* Fix daemon test

* Cleanup

* Remove arguments

* restore missing hints being stored as None (instead of 0-length bytes) (#11568)

* Coin simplification (#11567)

* factor out as_list() from Coin into free function. Remove unused name_str() from Coin. Minor optimization of hash_coin_ids() for common case.

* extend Coin unit test

* test installer on fedora:36, drop 33

* add tests.util.misc.assert_maximum_duration for benchmark assertions (#11589)

* add tests.util.misc.assert_maximum_duration for benchmark assertions

* fix some imports

* more

* lint

* correct duration

* hint fixup

* rename to caller_file_and_line()

* set default timer to thread_time()

* extract gc manager and default to disabling

* self calibrate overhead

* wrap none-ness behind .results() method

* add messages

* from __future__ import annotations

* yield out self

* final from typing_extensions

* split it

* just use Future

* tweak

* tweak

* correction in loosely hint checked file

* all future indexing in hints

* union for the hints

* rename

* assert_runtime()

* another rename...

* message -> label

* label all results with test names

* oops

* early return from _set_spent function (#11594)

* check for removals before calling function

* move condition to function

* remove redundant condition (#11582)

* fix built-in profiler to work for mac (#11590)

* Bump actions/setup-python from 2 to 3 (#10949)

* Bump actions/setup-python from 2 to 3

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixup templates for python@v3

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* avoid cancelling release/** and long_lived/** (#11519)

* tests: Some adjustments in `test_offer_endpoints` (#11558)

This is a follow up after #11552 which basically used the initial parts 
of `test_cat_endpoints` to make `test_offer_endpoints` working. Now this 
PR drops a lot of the stuff we can assume to be not needed in this test 
because its tested in `test_cat_endpoints` and it adds some general 
improvments to it.

* async sleep and require connection reset error for DoS test (#11612)

* fix jsonify bool

* Remove is not None and length assertion in select_coins() (#11569)

* Remove is not None and length assertion in select_coins()

The `coins is not None` check seems unneeded since `select_coins()` is hinted to return `Set[Coin]`.  The length requirement needlessly restricts from being able to request a zero amount.  Either situation would either trigger an exception in the next assert, or trigger the assert itself for non-zero amounts requested.

* also remove from cat_wallet.py

* new port 0 stuff

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Evan Graham <me@luakt.net>
Co-authored-by: wjblanke <wjb98672@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Jeff <jeff@chia.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
2022-05-26 18:11:21 -07:00

93 lines
3.7 KiB
Python

import asyncio
from typing import Any, List, Optional, Tuple
import pytest
from chia.full_node.full_node_api import FullNodeAPI
from chia.simulator.full_node_simulator import FullNodeSimulator
from chia.types.peer_info import PeerInfo
from chia.types.spend_bundle import SpendBundle
from chia.util.ints import uint16, uint64
from chia.wallet.transaction_record import TransactionRecord
from chia.wallet.wallet_node import WalletNode
from tests.block_tools import BlockTools
from tests.pools.test_pool_rpc import farm_blocks, wallet_is_synced
from tests.time_out_assert import time_out_assert, time_out_assert_custom_interval
def assert_sb_in_pool(node: FullNodeAPI, sb: SpendBundle) -> None:
assert sb == node.full_node.mempool_manager.get_spendbundle(sb.name())
def assert_sb_not_in_pool(node: FullNodeAPI, sb: SpendBundle) -> None:
assert node.full_node.mempool_manager.get_spendbundle(sb.name()) is None
assert not node.full_node.mempool_manager.seen(sb.name())
def evict_from_pool(node: FullNodeAPI, sb: SpendBundle) -> None:
mempool_item = node.full_node.mempool_manager.mempool.spends[sb.name()]
node.full_node.mempool_manager.mempool.remove_from_pool(mempool_item)
node.full_node.mempool_manager.remove_seen(sb.name())
@pytest.mark.asyncio
async def test_wallet_tx_retry(
bt: BlockTools,
setup_two_nodes_and_wallet_fast_retry: Tuple[List[FullNodeSimulator], List[Tuple[Any, Any]]],
self_hostname: str,
) -> None:
wait_secs = 20
nodes, wallets = setup_two_nodes_and_wallet_fast_retry
server_1 = nodes[0].full_node.server
full_node_1: FullNodeSimulator = nodes[0]
wallet_node_1: WalletNode = wallets[0][0]
wallet_node_1.config["tx_resend_timeout_secs"] = 5
wallet_server_1 = wallets[0][1]
assert wallet_node_1.wallet_state_manager is not None
wallet_1 = wallet_node_1.wallet_state_manager.main_wallet
reward_ph = await wallet_1.get_new_puzzlehash()
await wallet_server_1.start_client(PeerInfo(self_hostname, uint16(server_1._port)), None)
await farm_blocks(full_node_1, reward_ph, 2)
await time_out_assert(wait_secs, wallet_is_synced, True, wallet_node_1, full_node_1)
transaction: TransactionRecord = await wallet_1.generate_signed_transaction(uint64(100), reward_ph)
sb1: Optional[SpendBundle] = transaction.spend_bundle
assert sb1 is not None
await wallet_1.push_transaction(transaction)
async def sb_in_mempool() -> bool:
return full_node_1.full_node.mempool_manager.get_spendbundle(transaction.name) == transaction.spend_bundle
# SpendBundle is accepted by peer
await time_out_assert(wait_secs, sb_in_mempool)
# Evict SpendBundle from peer
evict_from_pool(full_node_1, sb1)
assert_sb_not_in_pool(full_node_1, sb1)
# Wait some time so wallet will retry
await asyncio.sleep(2)
our_ph = await wallet_1.get_new_puzzlehash()
await farm_blocks(full_node_1, our_ph, 2)
# Wait for wallet to catch up
await time_out_assert(wait_secs, wallet_is_synced, True, wallet_node_1, full_node_1)
async def check_transaction_in_mempool_or_confirmed(transaction: TransactionRecord) -> bool:
assert wallet_node_1.wallet_state_manager is not None
txn = await wallet_node_1.wallet_state_manager.get_transaction(transaction.name)
assert txn is not None
sb = txn.spend_bundle
assert sb is not None
full_node_sb = full_node_1.full_node.mempool_manager.get_spendbundle(sb.name())
if full_node_sb is None:
return False
in_mempool: bool = full_node_sb.name() == sb.name()
return txn.confirmed or in_mempool
# Check that wallet resent the unconfirmed SpendBundle
await time_out_assert_custom_interval(wait_secs, 1, check_transaction_in_mempool_or_confirmed, True, transaction)