Commit Graph

1265 Commits

Author SHA1 Message Date
dustinface
cc61c82682
tests: Improve tests_farmer_harvester_rpc.py (#10023)
* tests: Drop redundant plot creation in `tests_farmer_harvester_rpc.py`

* tests: Drop redundant class

* tests: Move RPC setup into new `environment` fixture

* tests: Split up the single test `test1`
2022-01-31 18:53:21 -08:00
Jeff
b8b7faacfe
Updated tests to use the ready/start/finished model previously added for test_multiple_writers. This reduces timeout-related failures that occur in the CI environment. By waiting for the spawned processes to indicate readiness and wait for a start signal, the test timeouts are better able to target the actual logic being tested rather than flakiness introduced by the OS task scheduler. (#10050) 2022-01-31 18:52:27 -08:00
Arvid Norberg
3d063fea8f
remove the GeneratorArgs type, and instead include the heights and generator programs as separate lists in BlockGenerator. The heights are not necessary when validating blocks, so this makes it easier to omit them in that case. The heights are only used when generating/farming a block. (#10006) 2022-01-31 18:49:39 -08:00
Arvid Norberg
275969ddd1
remove redundant function, calculate_cost_of_program(). The cost is now part of the NPCResult object (#9964) 2022-01-31 18:49:12 -08:00
Arvid Norberg
e49b11d486
use pytest.fixture instead of pytest_asyncio.fixture in cmd test (#10040) 2022-01-31 18:48:58 -08:00
Arvid Norberg
108f72a756
fix DB conversion with duplicate hints in the v1 database (#10041) 2022-01-31 18:48:37 -08:00
Jeff
d136b84498
Fixed Offer history limit and GUI sorting (#10025)
* trade_store: Migrated the `is_my_offer` property from the serialized
trade_record to a new `is_my_offer` column in the trade_records table.
Rewrote get_trades_between to move the query/sorting into the database.
This supports the presentation of offers in the GUI (two lists sorted by
relevance).

RPC: get_all_offers now accepts options to restrict the listing to
completed offers, my offers, taken offers. Added get_offers_count call
to return a breakdown of the number of offers (my offers, taken offers,
total).

CLI: Updated `chia wallet get_offers` to include options to exclude
created/taken offers, as well as an option to reverse the listing. The
-ia|--include-all option has been renamed -ic|--include-completed to
make sense in the context of the --exclude-[my-offers|taken-offers]
options. The full listing is now output, fixing the issue with hitting
the prior paging cap of 50 records.

* Linter fixes

* Updated trade tests. Fixed reverse sort when ordering offers by relevance.

* Make column spec explicit on insert to better support possible future migrations. If a future migration adds a new column with a default value, inserts/updates made by an older client will fail unless the insert explicitly indicates which columns are being updated. By explicitly listing the columns to update, any unspecified columns will have their default value applied.
2022-01-31 18:48:05 -08:00
Matt Hauff
0bf36b0b9d
Fix trade test flakiness (#10048) 2022-01-31 17:10:19 -05:00
Mariano Sorgente
00613bc259
Flaky test in wallet rpc (#10039) 2022-01-31 10:05:32 -05:00
Mariano Sorgente
34edf6e35e
pool_wallet: Deterministic derivation of auth key, and only count poo… (#9922)
* pool_wallet: Deterministic derivation of auth key, and only count poolWallets for indeces

* cli: Use the same limit for cli creation

* farmer: no more auth keys in config

* farmer: handle case of no authentication sk

* farmer: remove async, and fix test

* lint

* test: Add a test for many pool wallets, and user store

* test: Wait for wallet to be synced in test

* wallet: fix startup (no dns servers) and change to server checking of plotnfts

* cli: change to server checking

* tests: fix user store test

* Improve caching

* PR comments

* Remove useless class

* Try to fix test again
2022-01-30 12:31:55 -05:00
Mariano Sorgente
f28fcccfa5
Ms.main tests (#10026)
* Try to fix compact tests

* Don't compactify recent blocks, earlier

* Update test_simulation.py

Increase number of blocks

* Update setup_nodes.py

* Update test_simulation.py

* Update test_daemon.py
2022-01-29 21:37:11 -05:00
Jeff
32df1225a2
Catch TimeoutError in test_multiple_writers. If raised, we'll skip the (#10031)
test. Increased the timeout to 60 seconds.

Tested that a legitimate error in the test will still trigger a failure.
2022-01-29 21:27:12 -05:00
Mariano Sorgente
6e37b74742
Change compact block (#10011)
* Change compact block

* Also recent blocks

* Add bill's rollback code

* Fix v2 issue

* Fix v2 issue blockchain

* Fix tests
2022-01-28 19:19:10 -08:00
Yostra
ca84fa9f8d
Sync improvements (#9974)
* stash

* change log level

* stash changes

* cat tests

* add node id chia show

* make max sibscribe items configurable

* lint

* add logs for limits

* paralelize incluzion valdiation, don't diconnet from remote peers until local peer in synced

* wallet rate limit

* test log lvl, trusted, black

* only validation under semaphore

* flake mypy trade test

* reduce flakyness

* merge fix

* rename

* linting

* validate parallel, apply serial

* reduce falkyness

* lint

* Fix CAT tests

* black

* flaky test, total amount

* fix cat test

* skip existing state

* skip existing

* try other caches

* flake8

Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
2022-01-28 19:16:47 -08:00
Arvid Norberg
b8ada1ceb7
introduce restrictions on generators at a specific height (#9957)
* introduce restrictions on generators at a specific height. disallow division on negative numbers and disallow redundant leading zeros on integer condition arguments (produced by a generator)

* use SOFT_FORK_HEIGHT constant

* there is no need to specify height when validating block in block_creation

* Update tests/core/full_node/test_mempool.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-01-28 12:29:11 -08:00
Chris Marslender
cbbdb6510a
Adding additional data to RPCs for metrics (#9945)
* Add "metrics" as an additional destination for get_blockchain_state changes

* Add compact/uncompact blocks to get_blockchain_state

* Add total mempool cost to blockchain state

* Add hint count to blockchain state

* Add info about max cost, block_cost, and block_fees

* Add mempool min fee to blockchain state

* Add dict for mempool fee estimates

* dont start key with number

* Move max block cost to blockchain state

* Add generic exception logging util

* Catch/log any exceptions in counting hints

* Fix inconsistent key

* Assert row is not none

* Move block count metrics to a new endpoint

* report on block instead of unfinished_block for metrics

* Mark the block event with bool for if transaction block or not

* Add some basic tests for counting hints

* Add some basic tests on counting compact/uncompact blocks

* Fix lint

* Add k-size

* Add timestamp

* Add header_hash + height
2022-01-27 18:42:49 -08:00
Arvid Norberg
5ebcd82c58
simplify run_block() (#9962)
* more tests

* only run the generator once, in run_block(). Prior to this patch, we run it once per spent coin + 1
2022-01-26 16:44:30 -08:00
Chris Marslender
e8309ec884
Use native concurrency settings to cancel duplicate in progress jobs rather than waiting for the job to start to cancel (#9963) 2022-01-26 11:20:31 -06:00
Arvid Norberg
38e8c8c355
simplify cost calculation. It's already computed in rust (#9931) 2022-01-26 09:07:22 -08:00
Jeff
bb1c16cee5
[CLI] chia keys derive [wallet-address | child-key | search] (#9184)
* Added chia keys derive (wallet-address | child-key) to display derived wallet addresses or public keys

* Support for public key (unhardened) derivation

* Fleshed out the search command. Can perform a substring search on wallet addresses, public, and private keys.
Support for deriving private keys

* Default to unhardened derivation. Added --hardened-derivation option. HD paths denote hardened derivation with a 'h' suffix at each index.

* Added --no-progress to reduce output. Exit code indicates search success/failure. Option name cleanup.

* chia keys derive search can now search for multiple terms

* chia keys derive search can be restricted to searching pubkeys|privkeys|addresses|all

* Allow child-key derivation rooted from an arbitrary HD path

* Search derived keys rooted under an arbitrary HD path

* LGTM fixes

* Remove extra master_sk_to_wallet_sk_unhardened definition

* Cleanup and fixed search output

* Cleanup and comments

* Added tests for `chia keys` commands

* Using @pytest_asyncio.fixture to decorate empty_keyring() as suggested in pytest's output.

* Set asyncio-mode to auto to address pytest warning

* See if tests pass without checking exit status

* Added chia init step to tests that direclty invoke the `cli` function

* Explicitly add --root-path option to tests that run the `cli` function.

* Tests fix

* Re-enable checks against exit_code

* More fine tuning to address CI failures

* Changes to address naming and other feedback

* Updated tests
2022-01-25 20:42:16 -08:00
Arvid Norberg
1a84ee8db1
fix mempool issue where it would reject a spend bundle that spends an ephemeral coin with an ASSERT_SECONDS_RELATIVE 0 condition. Such condition is a no-op in the main blockchain validation, so the mempool should also allow it. (#9882) 2022-01-25 11:11:19 -08:00
Arvid Norberg
5375c1a0af
[v2] remove unused counter column in hint table (for v2 schema) (#9778)
* remove unused counter column in hint table (for v2 schema)

* the new db file is expected to be empty. fail if any tables already exist
2022-01-25 07:45:54 -08:00
Earle Lowe
d7cd3fe2cb
run block tool updates (#9807)
* run block tool updates

* pylint fixes

* Add tests

* Update workflows for new tests

* Updates based on comments
2022-01-25 07:44:13 -08:00
Arvid Norberg
77db90ba6a
move member function get_memos() out of SpendBundle and into a free-function living in the wallet code. It does not belong to the full node. Also rename it to indicate that it's expensive (and dangerous) to call (#9907) 2022-01-23 21:45:41 -08:00
Mariano Sorgente
f0a6189f2d
Move BLS validation to other processes during sync (#9746)
* Try putting BLS in multiprocess

* Fix test

* Fix test again

* Fix test again ^ 2

* Don't use cache unless necessary, cleanup of unused code.

* WIP fixing tests

* Blockchain tests passing

* Remaining tests and lint

* More test fixes

* Fix full_node_rpc test

* Fix remaining tests

* Correct boolean

* Don't validate sig if CLVM fails. Also testing signature validation properly

* Simplify block prevalidation API and improve tests

* Fix logic error and remaining tests

* Update comments and change to AssertionError

* Try to improve CI performance

* Fix mempool test, try to fix test_blockchain performance

* Set default argument to fix test

* Fix DB conversion test

* Use cache for validating signatures (#9859)

* Use cache for validating signatures

* comments: fix typo

* Don't rerun the super slow part of the test (#9887)
2022-01-20 14:50:41 -08:00
Matt Hauff
2e3c054c3a
Add remaining offer code to main (#9886)
* Add remaining offer code to main

* Fix compression test

Co-authored-by: Jeff <paninaro@gmail.com>
2022-01-20 14:49:31 -08:00
Arvid Norberg
c780f0978c
add chia function to convert blockchain database to v2 (#9613) 2022-01-19 11:43:56 -08:00
Kyle Altendorf
48cfe11641
Set pools CI test timeout to 45 minutes (#9817)
* Set pools CI test timeout to 45 minutes

There have been several timeout failures recently with successful runs right on the edge at 28 minutes and such.

* pre-commit for workflow rebuild
2022-01-15 16:31:54 -08:00
dustinface
a22ad28edf
tests: Fix plot creation if exclude_final_dir is set (#9646)
With `exclude_final_dir` set (introduced in #9578) the created plot 
don't gets added to `expected_plots` hence the private key stays the 
same for the three plots added with missing keys. This only worked on CI 
because it uses the cached plots there which i created before i 
introduced `exclude_final_dir` in that PR so i also had them locally 
already.
2022-01-13 17:41:44 -08:00
dustinface
25d30a4c72
tests: Fix and improve plot manager caching tests (#9785)
* tests: Fix plot manager caching tests

The new plot manager was compared to itself instead to the one from the 
test environment. This is an old issue introduced in #8875.

* tests: Compare the loaded cache data also, not only the cache size
2022-01-13 17:41:15 -08:00
Arvid Norberg
2851bedbcc
sanity check the blockchain database on startup. If we can't find a peak, we also should not find any coins (#9780) 2022-01-13 14:53:48 -08:00
Arvid Norberg
93d967525b
revert allow-upgrades configuration option. It makes it significantly more complicated to do proper database migrations (#9768) 2022-01-13 14:53:27 -08:00
Yostra
89f15f591c
Merge standalone wallet into main (#9793)
* wallet changes from pac

* cat changes

* pool tests

* pooling tests passing

* offers

* lint

* mempool_mode

* black

* linting

* workflow files

* flake8

* more cleanup

* renamed

* remove obsolete test, don't cast announcement

* memos are not only bytes32

* trade renames

* fix rpcs, block_record

* wallet rpc, recompile settlement clvm

* key derivation

* clvm tests

* lgtm issues and wallet peers

* stash

* rename

* mypy linting

* flake8

* bad initializer

* flaky tests

* Make CAT wallets only create on verified hints (#9651)

* fix clvm tests

* return to log lvl warn

* check puzzle unhardened

* public key, not bytes. api caching change

* precommit changes

* remove unused import

* mypy ci file, tests

* ensure balance before creating a tx

* Remove CAT logic from full node test (#9741)

* Add confirmations and sleeps for wallet (#9742)

* use pool executor

* rever merge mistakes/cleanup

* Fix trade test flakiness (#9751)

* remove precommit

* older version of black

* lint only in super linter

* Make announcements in RPC be objects instead of bytes (#9752)

* Make announcements in RPC be objects instead of bytes

* Lint

* misc hint'ish cleanup (#9753)

* misc hint'ish cleanup

* unremove some ci bits

* Use main cached_bls.py

* Fix bad merge in main_pac (#9774)

* Fix bad merge at 71da0487b9

* Remove unused ignores

* more unused ignores

* Fix bad merge at 3b143e7050

* One more byte32.from_hexstr

* Remove obsolete test

* remove commented out

* remove duplicate payment object

* remove long sync

* remove unused test, noise

* memos type

* bytes32

* make it clear it's a single state at a time

* copy over asset ids from pacr

* file endl linter

* Update chia/server/ws_connection.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-01-13 12:08:32 -08:00
Arvid Norberg
0ba838b7a8
New clvm generator api (#9645)
* use the new run_generator2() and run_chia_program() API of clvm_rs

* bump clvm_rs dependency to 0.1.17
2022-01-11 15:32:43 -08:00
Kyle Altendorf
e498a406a0
Make the hard coded 2 reserved cores configurable (#9709)
* Make the hard coded 2 reserved cores configurable

* add missing reserved_cores parameters

* add missing reserved_cores parameters
2022-01-10 20:58:29 -08:00
Arvid Norberg
7ef619274a
extend more tests to run with both v1 and v2 blockchain database schema (#9656)
* extend more tests to run with both v1 and v2 blockchain database schema

* cleanup db_version fixture

* extend test timeouts

* Update tests/core/full_node/config.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/conftest.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* fixup timeout

* fix typo in TODO comment

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-01-10 20:55:40 -08:00
Jack Nelson
d22976aebe
Add coin announcement to create signed transaction rpc call (#9735)
This will allow pool operators and developers to have more control of the types of transactions they create using the wallet RPC.
2022-01-07 10:16:24 -03:00
Arvid Norberg
659f4f4890
remove (redundant) spent field on CoinRecord. Replace it with a property (#9658) 2021-12-22 18:50:32 -08:00
dustinface
7949033929
tests: Use temp_dir as tmp2_dir, drop temp_dir if canceled (#9601)
If you currently cancel the test during the plot setup phase it just
removes the whole directoy and with it all the test plots for no good
reason. At least in my opinion its just annoying. Sure, you can clone
the `test-cache` repo and copy them over if this happens but i still
think its better to just merge this PR :)
2021-12-20 10:38:43 -08:00
dustinface
86e7d694d2
tests: Avoid unintentional refresh cycles in test_plot_manager.py (#9603) 2021-12-20 10:38:11 -08:00
risner
598a0e8555
colorlog apparently changed, tests fail on colorlog-6.6.0 (#9620)
* Update test_simple_sync_protocol.py

* Update test_wallet_sync.py
2021-12-20 10:37:29 -08:00
Arvid Norberg
f1dcd36c2a
In main chain (#9550)
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
2021-12-17 14:37:45 -08:00
dustinface
2ab146bf2d
plotting|tests|github: Fix and test re-loading/removal of plots where we missed the keys (#9578)
* tests: Allow to use custom keys in `BlockTools.new_plot`

* tests: Create three plots with `G1Element()` as farmer/pool key

But not add them as default plots. Just to have plots in the test cache with different keys from
within and outside the keychain for plot loading tests.

create

* test|github: Bump `test-cache` version to `0.28.0`

* plotting: Drop plots from `no_key_filenames` after the keys appeared

* plotting: Drop plots from `no_key_filenames` if the plot was removed

* tests: Test loading of plots where we miss the keys

* plotting: Use `x.copy()` instead of `list(x)`

* plotting: Add a comment about the missing keys plot removal

* tests: Fix after #9595
2021-12-17 14:36:29 -08:00
Mariano Sorgente
2f86d0eee1
Ms.fix puzzle store (#9606)
* Fix puzzle store to 100 addresses

* Remove unnecessary function call

* Lint
2021-12-17 14:34:27 -08:00
dustinface
234d121c2a
chia|tests|pre-commit: Add missing __init__.py files (#8758)
* chia|tests: Add missing `__init__.py` files

* check and create needed __init__.py files via pre-commit

* plotting|tests: Handle `mypy` issues introduced by the `main` rebase

* pre-commit: Run `build-init-files.py` more verbose

* tests: Ignore `__pycache__` folders in `build-init-files.py`

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* tests: Add shebang, make executable, add description

To `tests/build-init-files.py`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-16 18:52:46 -08:00
dustinface
08b5d68f2c
tests: Move the expected_result_matched assignment out of the loop (#9595)
* tests: Move the `expected_result_matched` assignment out of the loop

This issue was introduced in #9505 and leads to false positives as soon 
as there was one of the attributes in `PlotRefreshResult` checked to be 
valid followed by an invalid one.

* tests: Fix `test_invalid_plots`
2021-12-16 18:52:32 -08:00
dustinface
aabc54bb53
plotting|tests: Fix try/except in PlotManager._refresh_task (#9588)
* plotting: Fix try/except in `PlotManager._refresh_task`

* tests: Move the start refreshing call into `PlotRefreshTester` ctor

It being in `run()` before where it was called every test iteration lead
to not detecting an issue with the try/except introduced in #9409. See
ee1dd1912f4678d7b960b670473bd4b3ffbd890b for the fix.
2021-12-16 15:14:34 -08:00
dustinface
a68ffecea1
plotting|tests: Fix and test removal of invalid plots (#9587)
* plotting: Drop from `failed_to_open_filenames` if the plot was removed

* tests: Test removal from `failed_to_open_filenames`
2021-12-16 15:14:12 -08:00
Arvid Norberg
fa3b90a089
simplify test_block_store (#9594) 2021-12-16 14:53:27 -08:00
Arvid Norberg
b554b0bfa3
simplify test_block_compression (#9590) 2021-12-16 09:44:10 -08:00