Commit Graph

102 Commits

Author SHA1 Message Date
Jack Nelson
0d36874caa
Refactor Seeder & Crawler code + add tests (#15781)
* Cleanup seeder & mypy all files

holy crap mother of all tech debt this was horrid

* Make UDP Protocol class a dataclass & separate out functions.

* add TCP protocol class to DNS server

* Fix mypy types & other cleanup

also fix a couple bugs

* Add seeder and crawler tests

* change log levels

* re add db lock timeout

oops

* add edns & edns tests

* fix repeated shutdown on close signal

* fix binding to use both ipv6 and ipv4

whyyyyyyy

* add ipv6 and ipv4 tests + add ipv4 if windows
2023-08-11 05:25:19 -05:00
Amine Khaldi
a417a438ad
Merge commit '6a5bf15e9e39e6c0f27ab69fe718ba82ce007354' into checkpoint/main_from_release_2.0.0_6a5bf15e9e39e6c0f27ab69fe718ba82ce007354 2023-08-09 13:00:33 +01:00
Amine Khaldi
95c892dddc
Merge commit '99e4d5b41268073f691c6a745b9f924bf3ab6379' into checkpoint/main_from_release_2.0.0_99e4d5b41268073f691c6a745b9f924bf3ab6379 2023-08-07 14:58:25 +01:00
Arvid Norberg
6a5bf15e9e
bump soft-fork3 activation height (#15938) 2023-08-04 20:36:31 -05:00
Arvid Norberg
8071dc6211
introduce SOFT_FORK4_HEIGHT and tie CHIP-13 to it (#15885)
* introduce SOFT_FORK4_HEIGHT and tie CHIP-13 to it

* fix test_plot_refreshing
2023-08-04 18:41:34 -05:00
Matt Hauff
289260c374
Credential Restricted CATs (#15411)
* Allow create_tandem_xch_tx to take a set of announcements

* Create potential announcement in correct arm of create_tandem_xch_tx

* Make does_coin_belong_to_wallet more robust

* Add match_hinted_coin to wallet protocol

* Add CROuterPuzzle

* Add conditions with fee announce.clsp

* Add CRCATWallet

* Add CLI support for CR-CATs

* Fix bug in CATOuterPuzzle

* Add SOFTFORK_HEIGHTS as an exportable variable from conftest

* Allow specifying num of blocks to farm to wallets_prefarm_services

* Add potential subclasses to cat wallet creation

* Integrate CR-CATs into offers

* Address comments by @geoffwalmsley

* Add CRCAT info to get_wallets

* Changes due to (#15846)
2023-08-03 19:36:21 -05:00
Arvid Norberg
1d8d476056
drop support for database schema v1 (#15895)
* don't run tests with database schema v1

* remove support for database schema v1 from BlockStore

* remove support for database schema v1 from CoinStore

* remove support for database schema v1 from HintStore

* remove support for v1 BlockStore schema from blockchain reorg logic

* remove support for database schema v1 from BlockHeightMap

* run block store tests both with and without the cache

* add test with empty blockchain for BlockHeightMap

* fix typo
2023-08-03 14:18:33 -05:00
Arvid Norberg
f0cb47baef
remove softfork2 logic (#15793) 2023-08-01 11:28:16 -05:00
Felix Brucker
c5ca0978f6
Add support for multi node farmers (#15444)
* Extract `respond_signatures` logic

* Add `call_api_of_specific` method to `ChiaServer`

* Fix typehint for test fixture `farmer_one_harvester`

* Make test helper `add_dummy_connection` work with any node type

* Add `reply_types` to `request_signatures` of harvester

* Fix duplicate SP processing with multiple nodes

* Handle state in `request_signed_values`

* Add set typehint

* Add comment re `_process_respond_signatures()` usage

* Fix imports

* Fix wording to not confuse with SignedValues message
2023-07-28 14:48:32 -05:00
Arvid Norberg
99e4d5b412
bump softfork3 activation by about 2 weeks (#15861) 2023-07-26 20:26:21 -05:00
William Allen
97245c959d
Chip13 (#15688) (#15825)
* Chip13 (#15688)

* Chip13

* Lint.

* Black.

* Lint.

* Fix BlockTools.

* Trigger the soft fork.

* Parametrize bt.

* Syntax.

* Add softfork3 to empty_blockchain too.

* Skip test.

* Skip test.

* Try fix.

* Add persistent blocks.

* Lint.

* Run tests without block cache.

* Try to fix tests.

* Revert "Try to fix tests."

This reverts commit f10c7fc847.

* WIP plot filters.

* pytest.

* Lint.

* Skip cache tests.

* Try to lower the constraints.

* Try to increase timeout.

* Test blockchain.

* Store test.

* Revert to main

* Revert to main

* Revert to main

* Add tests with new interface

* Test.

* Fixtures.

* Fixtures.

* Lint.

* Add first test.

* Improve test.

* Isort.

* merge fix

* fix counter

* Bump test-cache version to soft_fork3.0 DBs

* Enable tests.

* Enable all.

* Syntax.

* Attempt higher timeouts.

* Try to skip big test.

* Revert tests larger timeouts.

* Try to get test to work with 4000.

* Skip 10000 blocks.

* Test.

* Update test_full_node_store.py

* Update test_blockchain.py

* Update test_blockchain.py

* Update tests/conftest.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Update tests/blockchain/test_blockchain.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Address some comments.

* Some more comments.

* Use new passes_plot_filter signature.

* Lint.

* Lint.

* Reduce benchmark timeout.

* Skip test.

* Address review comments.

* Delete leftovers.

* Review comments.

* await.

* Revert async call.

* Validate in body too.

* Fix test.

---------

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Fix daemon test to use client ssl context

* fix daemon test to close client

---------

Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
2023-07-21 15:54:11 -05:00
Florin Chirica
7c6cf2cfe9
Chip13 (#15688)
* Chip13

* Lint.

* Black.

* Lint.

* Fix BlockTools.

* Trigger the soft fork.

* Parametrize bt.

* Syntax.

* Add softfork3 to empty_blockchain too.

* Skip test.

* Skip test.

* Try fix.

* Add persistent blocks.

* Lint.

* Run tests without block cache.

* Try to fix tests.

* Revert "Try to fix tests."

This reverts commit f10c7fc847.

* WIP plot filters.

* pytest.

* Lint.

* Skip cache tests.

* Try to lower the constraints.

* Try to increase timeout.

* Test blockchain.

* Store test.

* Revert to main

* Revert to main

* Revert to main

* Add tests with new interface

* Test.

* Fixtures.

* Fixtures.

* Lint.

* Add first test.

* Improve test.

* Isort.

* merge fix

* fix counter

* Bump test-cache version to soft_fork3.0 DBs

* Enable tests.

* Enable all.

* Syntax.

* Attempt higher timeouts.

* Try to skip big test.

* Revert tests larger timeouts.

* Try to get test to work with 4000.

* Skip 10000 blocks.

* Test.

* Update test_full_node_store.py

* Update test_blockchain.py

* Update test_blockchain.py

* Update tests/conftest.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Update tests/blockchain/test_blockchain.py

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>

* Address some comments.

* Some more comments.

* Use new passes_plot_filter signature.

* Lint.

* Lint.

* Reduce benchmark timeout.

* Skip test.

* Address review comments.

* Delete leftovers.

* Review comments.

* await.

* Revert async call.

* Validate in body too.

* Fix test.

---------

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
2023-07-20 21:25:23 -05:00
Izumi Hoshino
1f9081a225
Added total_effective_plot_size for get_harvesters RPC API (#15770)
* Added `total_effective_plot_size` for `get_harvesters` RPC API

* Fixed lint error

* Change the type of `total_effective_plot_size` in Receiver to `int`

* Fixed lint error

* Fixed lint error 2

* Changed text

* Added test

* Fixed lint error

* Fixed lint error 2

* Fixed a test issue

* Added config.py to tests/cmds
2023-07-18 12:36:12 -05:00
Arvid Norberg
99c55fea67
Announce conditions limit (#15790)
* Test announcement condition limits for soft-fork 3

* run slow wallet-sync tests in plain mode only

---------

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
2023-07-17 13:17:48 -05:00
Kyle Altendorf
ca34240b43
add sql logging config for DataLayer (#15690)
* add sql logging config for DataLayer

* add test

* data_sql.log
2023-07-17 13:15:22 -05:00
Arvid Norberg
ca1147584d
Hard fork 2.0 (#15581)
* add HARD_FORK_2_0 consensus_mode to run tests under hard-fork consensus rules

* hook up soft-fork and hard-fork changes in CLVM

* fix AGG_SIG_* garbage tests for the 2.0 hard-fork

* add test for unknown conditions with cost, which is a hard-fork feature

* add mempool tests for unknown conditions with cost

* add tests for the SOFTFORK condition

* fix benchmark to take LIMIT_ANNOUNCES into account
2023-07-07 13:58:43 -05:00
Amine Khaldi
b498188c0d
Move harvester_farmer_environment to conftest.py (#15590)
Move harvester_farmer_environment to conftest.py.
2023-06-22 11:13:55 -05:00
Arvid Norberg
ad3353d9b4
don't parameterize the blockchain tests on DB version (#15588)
* don't parameterize the blockchain tests on DB version, in order to save CI time

* skip duplicate runs of stores tests based on the consensus mode (i.e. hard-fork or soft-fork testing), since the consensus rules don't affect the store classes. Right now, PLAIN is the only mode we have, but we're about to introduce HARD_FORK_2_0 and SOFT_FORK3 as new consensus modes in the tests
2023-06-22 11:04:10 -05:00
dustinface
a414b41905
tests: Use separate BlockTools in farmer/harvester fixtures (#15510) 2023-06-14 16:09:13 -05:00
dustinface
49140b2b3c
server: Introduce ApiProtocol (#15466)
* server: Introduce `ApiProtocol`

* genericize (#5)

* `ApiProtocol.api_ready` -> `ApiProtocol.ready()`

* Add `ApiProtocol.log` and give APIs separate loggers

* Fix `CrawlerAPI`

* Drop some unrelated removals

* Fix some of the generic hinting

* Revert some changes in `timelord_api.py`

* Fix `CawlerAPI` readiness

* Fix hinting

* Get some `CrawlerAPI` coverage

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-06-13 20:12:25 -05:00
dustinface
d6a2a9aa5d
seeder: Drop CrawlerAPI.__getattr__ (#15489)
* Test sending unknown messages to the crawler

* Drop `CrawlerAPI.__getattr__`

* Use a real crawler service for the test
2023-06-13 15:25:39 -05:00
Matt Hauff
35cc3debe9
Add a path in wallets_prefarm test fixture for generating an RPC client (#15426)
* add a path in wallets_prefarm for generating an RPC client

* Separate into new fixture

* Return wallet services as well
2023-06-06 12:40:17 -05:00
Arvid Norberg
ae0939af79
bump SOFT_FORK3_HEIGHT to align with the next release cycle (#15428) 2023-06-01 21:48:01 -05:00
Amine Khaldi
fbe7daa32e
Convert blockchain_constants's consensus_mode param into a fixture and use it to skip some simulation related tests in HARD_FORK_2_0 mode (#15384)
* Convert blockchain_constants's consensus_mode param into a fixture and use it to run daemon_simulation one at a time.

* Simulation test seems to support only one running at a time too.
2023-05-26 11:31:14 -05:00
Arvid Norberg
ff226e8cb7
make sure tests that use BlockTools use a single source of consensus constants (#15364)
make sure tests that use BlockTools use a single source of consensus constants. This is in preparations for being able to run tests with different sets of constants, e.g. the hard-fork constants
2023-05-23 12:02:28 -05:00
Arvid Norberg
621c75e4ba
Fork infrastructure (#15299)
* remove softfork-logic for 1.7 softfork (which has already activated)

* add new constants for soft-fork3, hard-fork and the other plot-filter adjustments
2023-05-18 20:20:11 -05:00
Arvid Norberg
f100cd2cdf
Softfork2 testing (#15204)
* update tests to use the new softfork2 height

* activate both softforks immediately on testnet
2023-05-04 16:14:15 -05:00
Arvid Norberg
536830b09b
Generator mod cleanup and simplification (#15055)
* remove unused run_generator_mempool() function

* move run_generator_unsafe() into the tests, which is the only place it's used

* remove (somewhat meaningless) setup_generator_args() and create_generator_args()

* remove unused GENERATOR_MOD in mempool_check_conditions.py

* remove redundant get_generator() function

* transition analyze-chain.py to use run_block_generator() and drop dependency on GENERATOR_MOD

* fixup type hints in test_rom.py

* fixup type hints in test_compression.py

* fixup type hints in test_generator_types.py
2023-04-18 16:30:10 -05:00
Arvid Norberg
8dbfc4840a
enable soft-fork2 conditions (incl. ASSERT_BEFORE) (#14733)
* enable soft-fork2

* add blockchain (consensus) test for time-lock conditions (non-ephemeral spend)

* introduce new soft-fork rule to compare ASSERT_SECONDS_* conditions against the previous transaction block's timestamp (to be consistent with ASSERT_HEIGHT_* conditions)

* bump chia_rs. This updates the mempool rules to disallow relative height- and time conditions on ephemeral coin spends

* implement assert_before in mempool_check_time_locks. Extend ephemeral coin test in blockchain with assert_before conditions

* implement support for assert_before conditions in compute_assert_height()

* support assert-before in mempool

* add timelock rule

* address review comments
2023-03-23 11:30:10 -05:00
Arvid Norberg
99ef78029b
disable soft-fork 2 (for now) (#14786)
disable soft-fork 2 (for now) until we properly think-through and settle on how to handle ephemeral coins
2023-03-09 19:13:15 -06:00
Arvid Norberg
4ed31c50d5
Bump chia rs 0.2.4 (#14758)
* fix bug in make_test_conds() test utility

* bump chia_rs to 0.2.4, which preserves assert_seconds_relative 0 in parsing conditions. This allows for the 1.8.0 soft-fork to make the existing time-lock conditions stricter, > instead of >=. This is to match the existing ASSERT_HEIGHT_RELATIVE, which already is >

* fixup separating ENABLE_ASSERT_BEFORE from MEMPOOL_MODE

* Use a low value for SOFT_FORK2_HEIGHT during tests and cover the case before soft-fork2

---------

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
2023-03-07 15:29:54 -06:00
Earle Lowe
6e140f24fb
simplify and improve daemon connection handling (#14662)
* simplify daemon connection handling

* Add tests

* return list of service names during removal

* set remove raises KeyError

* Better test type hints

* Add types and code refactor

* fix test typos

* small type adjustments

* test code improvements

* Better error handling for bad json

* handle proper but unexpected JSON values

* Update chia/daemon/server.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* test improvements

* check for empty thing equivalence

* Improved logging for ConnectionReset

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-03-06 12:49:38 -06:00
Sebastjan Trepca
a38da21ebd
no duplicate offers, handle conflict offers correctly (#14722)
* no duplicate offers, handle conflict offers correctly

* don't sync every offer with a bad spend, only pending ones
2023-03-03 15:13:39 -06:00
dustinface
c098ee9166
full_node: Rename some receive/respond prefixes to add (#14458)
* full_node: Rename `FullNode.resond_block` to `FullNode.add_bock`

The naming `respond_block` is confusing here imo, we have a full node
API method called `respond_block` but here in the `FullNode` class this
method is there to try adding a block to the full nodes's chain so i
think `add_block` is the better choice?

Also change the `respond_block: full_node_protocol.RespondBlock`
parameter to `block: FullBlock` which lets us get rid of many
`full_node_protocol.RespondBlock` wrappings.

* `FullNode.receive_block_batch` -> `FullNode.add_block_batch`

* `receive_unfinished_block` -> `add_unfinished_block` + change parameter

* `FullNode.respond_transaction` -> `FullNode.add_transaction`

* `FullNode.respond_end_of_sub_slot` -> `FullNode.add_end_of_sub_slot`

* `FullNode.respond_compact_vdf` -> `FullNode.add_compact_vdf`

* `respond_compact_proof_of_time` -> `add_compact_proof_of_time`

* `respond_transaction_semaphore` -> `add_transaction_semaphore`
2023-02-23 10:28:29 -06:00
dustinface
0e309d6c6b
tests: Merge some fixtures and drop other redundant ones (#14629)
* Drop redundant fixtures in `test_dl_wallet.py`

* Drop `two_wallet_nodes_five_freeze`

Maybe im just not getting it but the `five_freeze` doesn't sound right 
here to me and the fixture does the same as `two_wallet_nodes` so i 
guess we can just drop it?

* Drop `wallet_and_node` fixture

* Drop both `wallet_node` fixtures

The one in `conftest.py` and the one in `test_dl_wallet.py`.

* Drop both `wallet_node` fixtures

The one in `conftest.py` and the one in `test_dl_wallet.py`.

* Drop `wallet_node_sim_and_wallet` fixture

* Drop `wallet_node_simulator` fixture

* Drop `one_wallet_node` fixture
2023-02-21 18:37:33 -06:00
dustinface
1094ae21fd
tests: Setup 4 nodes in setup_four_nodes (#14630) 2023-02-21 12:26:29 -06:00
Arvid Norberg
1d4a3ecce8
Softfork2 infrastructure (#14625)
* add soft-fork2 (to be introduced in 1.8.0)

* update unknown-opcode test
2023-02-21 12:25:47 -06:00
Adam Kelly
d69ecdda29
Test DID Wallet creation by blockchain singleton update (#14545) 2023-02-13 11:21:32 -06:00
Matt Hauff
59e389968e
Add cost logging to CLVM tests as fixture (#14444) 2023-02-10 15:13:45 -06:00
Kyle Altendorf
6e70868888
move wallet_is_synced/wallets_are_synced to FullNodeSimulator methods (#14403)
* move wallet_is_synced/wallets_are_synced to FullNodeSimulator methods

* black

* gotta catch 'em all
2023-02-10 15:13:23 -06:00
Kyle Altendorf
f346de94ba
refactor test_pool_rpc (#14196)
* preliminary test_pool_rpc refactor

* remove todo

* handle some todos

* one more

* wait for wallets to sync in the fixtures

* drop an extra wallet

* drop unused function
2023-02-08 19:43:43 -06:00
Matt Hauff
97230ca3b0
Add a way to import a SpendSim and SimClient as a contextmanager (#14434) 2023-01-31 19:22:06 -06:00
Arvid Norberg
0f9e4ee41a
infrastructure for upcoming soft-fork (#14302)
soft-fork infrastructure
2023-01-12 03:08:59 -06:00
Earle Lowe
0a0c8920ff
Add a daemon heartbeat setting to config.yaml (#13886)
* make daemon heartbeat configurable and increase default

* Fix up daemon rpc test

* Fix dumb error with parameters

* Restore formatting

* Various updates from feedback

* Update tests/core/test_daemon_rpc.py

use config value for heartbeat

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* black fixes

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-01-06 11:55:20 -06:00
Adam Kelly
407c0ba4a4
Fixes the case when a fee estimation request is made before the FullNode has initialized a peak (#14193)
* Fixes the case when a fee estimation request is made before the FullNode has initialized a peak

* Clarify code by setting defaults early

* Only spawn one full node in test
2022-12-23 11:31:44 -06:00
Kyle Altendorf
55f5e371b3
Block and transaction processing helpers (#11535)
* Block and transaction processing helpers

* flake8

* more and hints

* More

* separate tests of the simulator from tests using the simulator

* move fixture to conftest

* amounts are uint64

* remove unneeded while loop

* backoff instead of fixed sleep times

* .farm_blocks() only forces tx blocks when needed

* fix test_wallet_create_hit_max_send_amount

* fix test_wallet_tx_reorg

* fix test_address_sliding_window

* rename to farm_blocks_to_wallet

* rename to farm_rewards_to_wallet

* undo changes to protocol endpoints

* reword farm_blocks_to_wallet() loop

* rename process_blocks() to farm_blocks_to_puzzlehash() for now

* timeouts

* debug

* debug

* debug... for 3,7

* fix

* up timeout per block to 2 seconds

* increase transaction processing default timeout

* make debug maybe ok to leave around

* tidy

* intersperse sleeps again for wallet progression

* remove debug code

* review tidy

* cheat on uint64() type

* Update tests/simulation/test_simulator.py

* another import fix

* more fixes

* farm_blocks_to_puzzlehash

* more

* tweak

* fix the two-tx-blocks-instead-of-one dilemma

thanks to almog for figuring this out

* fix

* fixup

* rework since it was still broken on 3.7/3.8...

* oops

* yucky timeout

* more timeout

* cleanup todos

* add balance assertion after reorg before resubmission processing

* Apply suggestions from code review

* rename funds to expected_confirmed_balance

* catchup
2022-12-19 16:40:58 -06:00
Arvid Norberg
d330691db1
move mempool tests from full_node to mempool directory (#14056) 2022-12-07 20:49:13 -06:00
xdustinface
0cdce83b57
isort: Remove all tests exceptions and sort the related files 2022-12-05 02:06:00 +01:00
Almog De Paz
efb670cf1b
Setup services mypy (#13932) 2022-11-22 17:01:01 -06:00
Almog De Paz
f85bc15d4b
setup services refactor (#13905)
* always return service

* adjust setup_simulators_and_wallets

* adjust setup_simulators_and_wallets

* refactor setup_simulators_and_wallets
add setup_simulators_and_wallets_service

* revert unneeded change

* fix broken fixtures

* lint, fix data_rpc init_wallet_and_node

* fix two_wallet_nodes fixture

* fix two_nodes_sim_and_wallets_services fixture

* fix one_wallet_node_and_rpc fixture

* lint
2022-11-16 20:40:03 -06:00