Commit Graph

436 Commits

Author SHA1 Message Date
dustinface
c1a9d6c031
cmds|daemon: Improve legacy keyring migration enforcement (#12911)
* Use the daemon for partial migration checks if its running

* Make `migrate_legacy_keyring_interactive` async and drop redundant calls

So that we can await it since 

```
asyncio.run(async_update_daemon_migration_completed_if_running())
```

fails because we now here already have an event loop running.

* Drop redundant `unlocks_keyring` decoration

* Update some comments

* Move the `asyncio` import to make LGTM happy
2022-08-11 15:40:46 -05:00
William Allen
a041576c12
Cherry pick (#12886) (#12890) (#12898)
* skip setproctitle test (#12886)

* remove unittest class from setproctitle test

* actually skip the setproctitle test file

* Combine m1 and intel mac installers to a single workflow (#12890)

* Combine m1 and intel mac installers to a single matrix workflow, since we have native m1 runners now and the jobs are nearly identical

* Fix lint issues

* Fix more lint

* Use setup node

Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2022-08-10 21:10:30 -05:00
dustinface
922523694a
util: Force keyring migration / Deprecate legacy keyring support (#12801)
* util: Force keyring migration / Deprecate legacy keyring support

* Update chia/cmds/keys_funcs.py

Co-authored-by: Jeff <paninaro@gmail.com>

* Update chia/cmds/keys_funcs.py

Co-authored-by: Jeff <paninaro@gmail.com>

Co-authored-by: Jeff <paninaro@gmail.com>
2022-08-08 18:22:27 -05:00
Earle Lowe
8ec6fe9b92
Remove unused ping/pong code (#12779) 2022-08-05 13:27:37 -05:00
Jack Nelson
0dd5391f40
Simulator Final touches (#12703)
* add new rpc and add tests.

I added, get_all_blocks rpc, and expanded the get_all_puzzle_hashes rpc.

* lint pass

* update plot directory for simulator

* return peak height on new block farm

* expand test coverage

* fix plot directory craziness

* simulator sync override

* Add new simulator networktype

* add logging to simulator

* Fix race condition, mempool and confirm

* add reorg seed

* farm block if block 0 sim & fix unsynced for sim

* lint

* oops i forgot to lint

* switch to detecting simulator in network type

also remove inline genesis generation

* fix config loading to use correct variable

Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
2022-08-05 01:23:18 -05:00
dustinface
e5297f411d
util: Unify lockfile handling (#12541)
* util: Unify lockfile handling

This patch:
- Renames `util.lock.lock_by_path` to `util.lock.lock_path` and improves it by using `FileLock`
- Unifies all places where we create lock files to use `chia.util.lock.lock_path`
- Changes the daemon lockfile from `start-daemon.launching` to `daemon.lock` like we have it with all `<service>.lock` in the `run` folder
- Drops the `fastener` dependency

* Move the not keyring related sync tests into `test_lockfile.py`
2022-08-04 12:59:39 -05:00
Mariano Sorgente
c9bebc6c40
Fix more flaky tests, and optimize test_pool_rpc and NFT tests (signed) (#12761)
* Fix asyncio lock created outside async function

* Increase wallet sync timeouts

* Attempt to fix flaky pool tests, remove sleeps

* Remove accidental change

* Completely remove sleeps and optimize

* Remove asyncio import

* More flaky fixes

* Large speed improvements, and test Untrusted also

* Improve perf

* Move wallet_is_synced

* try not starting RPC in setup

* Revert "try not starting RPC in setup"

This reverts commit e8de70e73a2d4b10e0df4a2158e8e9acd9c3afd3.

* Revert accidental changes

* Revert more accidental changes

* Revert cbgui update
2022-08-03 02:28:32 -05:00
Mariano Sorgente
4c7a917905
Increase timeouts to reduce flakiness (#12745) 2022-08-03 00:01:39 -05:00
Arvid Norberg
aa0d3944e5
remove redundant tuple conversions (#12590)
* remove redundant class in block store test

* add test for get_block_records_by_hash

* remove redundant conversions from list to tuple in block_store sqlite calls
2022-08-01 18:14:28 -05:00
Arvid Norberg
92a5f38a86
remove soft fork logic at height 2300000 (#12563)
* remove soft fork logic at height 2300000. The whole chain can be validated with the post-soft-fork rules. Remove testing of pre-soft-fork rules

* fix test_full_sync
2022-08-01 17:32:53 -05:00
Jeff
4b8130ec60
NFT transfer/minting commands now validate the specified addresses (#12520)
* WIP to introduce a wallet AddressType enum to help validate
user-entered addresses for XCH/DID/NFTs etc.

* Support grabbing the current network's address prefix while also
allowing the current value to be updated (in case the config changes)

* Added tests for functionality in/used-by AddressType

* Removed code that would automatically resolve the selected network's address prefix.
is_valid_address() and ensure_valid_address() now require the allowed address types to be passed in.

* Check the length of the decoded address data

* Removed TXCH address type. "txch" address validation now requires a config to be passed into one of the validation functions.

* config is now required when using the address validation functions or calling AddressType.hrp().
root_path_and_config_with_address_prefix --> config_with_address_prefix fixture change.

* Update tests/conftest.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/conftest.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-07-29 12:22:33 -05:00
dustinface
0142129112
util: Free all non-class related functions in FileKeyring (#12655) 2022-07-29 11:35:08 -05:00
Jack Nelson
0bc54ae08b
Move simulator test requirements over to new module (#12521)
* create new test.simulator module

to allow other packages to use the simulator without it crashing.
also tiny QOL fixes

* change user mode sim plot filter

Now only plots matching the user set fingerprint are loaded, this decreases the chance of user error.

* fix small oops in configure

* oops

* fix lint

* isort, who dis

* move folder over to chia/simulator

also change mypy settings again
2022-07-25 14:07:38 -05:00
Freddie Coleman
72c8ba4a29
New RPC block_spends - Get spends for block using transaction generator (#12062)
* New RPC block_spends - Get spends for block using transaction generator (#10446)

* get spends for block using transaction generator

* type annotations and use existing function

* return None on exception

* parse to CoinSpend

* specify return type of get_block_spends in rpc client

* see what can be asserted

* test fix

* flags not necessary as we don't validate

* simplifying as cost is not required as we are not validating

* improve test to cover transaction generator ref_list

* simplify test

* remove unused import

* slight change and cleanup

* lint cleanup

* clean up

* wait until transaction is in mempool

* fix lint

Co-authored-by: Jack Nelson <jack@jacknelson.xyz>

* fix lint

Correct type and run lint checker

* switch to custom clvm

* add new puzzle

* curry and make better

* forgot to set linter to tools dir

* undo run_block changes

* Revert "curry and make better"

The tests do not like it

* correct tests

Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
2022-07-25 09:17:09 -05:00
dustinface
3af40f2d46
util: Fix passphrase hint (#12517)
* Test that passphrase hint stays in the keyring after a re-write

* Make sure the passphrase hint stays in the keychain

Currently the hint gets dropped if the keychain gets re-written after the hint has been set. See the test in the previous commit failing without this commit.

* Fix hint removal
2022-07-22 11:07:50 -05:00
Mariano Sorgente
2de8f7caac
optimize ph generation (~65% faster) (#12467)
* optimize ph generation

* Remove useless check

* More perf improvements

* G1 Elem not bytes

* Fix test

* Fix more tests

* Fix more more tests
2022-07-22 11:05:35 -05:00
Arvid Norberg
de09c3f1eb
make DBWrapper2 test more reliable (#12412) 2022-07-21 14:03:07 -05:00
Arvid Norberg
1fddb7aef1
add new_transaction() to DBWrapper2 which creates a nested transaction, and make write_db() just add writes to the existing transaction (#12275) 2022-07-15 21:07:43 -05:00
Jack Nelson
bd5d4b1392
remove extra bt from as many tests as possible ( speed up tests) (#12290)
* remove extra bt from as many tests as possible

max speed

* wallet bt tools removal

* fix error & add bt_tools to yield

* final changes

L bozo

* something funky going on

* fix issues

* a couple more fixes

* mypy happy

* damn lint

* Revert "something funky going on"

This reverts commit 1fcc9a34e1.
2022-07-13 03:57:32 -05:00
Mariano Sorgente
293bf6e64f
Tx submission idempotance, and prioritize wallet (#12282)
* Tx submission idempotance, and prioritize wallet

* TODO comment
2022-07-07 23:42:17 -05:00
dustinface
60a7581df8
streamable: Turn dataclass_from_dict into streamable_from_dict (#11763)
* Restrict `dataclass_from_dict` to valid streamable classes

They need to have the decorator applied for the required cache being populated.

* `dataclass_from_dict` -> `streamable_from_dict`

* `TestDataclassFromDict` -> `StreamableFromDict`
2022-07-01 13:18:57 -05:00
Mariano Sorgente
7550a13eba
Fix some more flakiness (#12160)
* Fix some more flakiness

* Fixed few other flakes

* Revert debug change

* Fix NFT flaky test

* Try again to fix the wallet RPC test

* More increased timeouts
2022-06-30 20:08:13 -05:00
dustinface
82a83b7ebd
streamable: Introduce Streamable.__post_init__ processing cache (#11730) 2022-06-28 11:49:41 -05:00
dustinface
0fd2dd66fc
rpc: Enable more mypy in chia.rpc.rpc_server (#11928) 2022-06-28 11:42:20 -05:00
Kyle Altendorf
75a287fd4c
not optional - WalletNode.wallet_state_manager, .server, .keychain_proxy (#11981)
* not optional - `WalletNode.wallet_state_manager`, `.server`, `.keychain_proxy`

* correct caught exception type

* remove some more

* more
2022-06-22 12:10:19 -05:00
dustinface
20f5bb22fb
streamable: Improve dataclass_from_dict validation, add more tests (#10652)
* Improve and test `dataclass_from_dict` tuple validation

* Improve and test `dataclass_from_dict` list validation

* Introduce `convert_hex_string`

* Improve and test `dataclass_from_dict` byte type validation

* Improve and test `dataclass_from_dict` unhashable validation

* Improve and test `dataclass_from_dict` primitive validation

* Improve `dataclass_from_dict` validation

* `raise TypeError(...) from e`

* Just `raise`, don't `raise e`

* Use `repr` of `item` not `str`

* `type(item) == f_type` -> `isinstance(item, f_type)`
2022-06-14 12:40:33 -05:00
Arvid Norberg
5ae87189c4
Rust coin (#11807)
* use rust implementation of Coin class, from chia_rs

* bump chia_rs dependency
2022-06-13 07:21:04 -05:00
Mariano Sorgente
ab53d48995
Request header blocks, and new rate limits (#11636)
* new blob block api method integrated into wallet

* direct msg streaming of headers, rename, tests

* perform_handshake call fix

* updated trusted sync with new block header calls

* add max blocks limit to fetch

* added tests for rejected block header msgs

* avoid parsing transactions info if not required

* avoid looking up capabilities setting

* move block tests out of a class

* test fix

* Merge changes

* added docs and increased rate limits

* increased block header request interval from 32 to 128

* remove fetching hashes and use height range

* fetching by height in db v2

* update capabilities, other fixes

* fixed range block header call

* Add type hints

* Start work on optimizing fetch_last_tx_from_peer

* Huge speedup in trusted wallet sync

* Revert unintentional changes

* Fix trade issue

* Improve the code

* Str format

* Optimize handling of farming rewards

* Fix bug

* Performance fixes

* Optimizations to wallet syncing

* Don't return all coins in respond_additions

* Revert concurrency numbers

* More optimization of the caches

* Small optimization in coin_added

* Optimize request_additions significantly by using a cache

* fixes from feedback

* capabilities check fixes

* Increase rate limits to allow 250tps in verification requests

* Start work on rate limits

* New rate limit versioning support

* Revert unrelated changes

* revert return False

* Lint

* Revert cbi

* try tests with trusted peer

* Revert unrelated wallet changes

* Revert more debug changes

* Add test and throw on an error if not found

* Reject invalid requests

* Revert bad change with uint32, and change warning to info

* Parametrize wallet sync test

* Merge and LGTM

* More clean way to choose peers

* Fix lint

* add the new RejectBlockHeaders, RequestBlockHeaders and RespondBlockHeaders to the network protocol regression test and regenerate test files

* Rate limit diffs only

* Improve performance

* Simpler

* Lint

Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
2022-06-11 01:35:41 -05:00
Kyle Altendorf
90516263aa
remove chia.util.path.mkdir() (#10842)
* remove chia.util.path.mkdir()

* remove submodule change

* less tmpdir, more tmp_path
2022-06-08 10:24:40 -05:00
dustinface
4230af1a59
tests: Refactor and improve streamable __post_init__ tests (#11726) 2022-06-03 18:42:05 -05:00
dustinface
a375e3cb8f
farmer: Make Plot{Info|Path}RequestData streamable + use from_json_dict (#11762)
* Make `PlotInfoRequestData` and `PlotPathRequestData` streamable

* Use `from_json_dict` instead of `dataclass_from_dict`

* Recover `PaginatedRequestData(Protocol)`
2022-06-03 12:37:03 -05:00
Arvid Norberg
1aeeffedbe
update unknown condition test (#11746)
* since condition opcode 1 is about to become well defined, use some other opcode to test unknown conditions

* Update tests/core/full_node/test_mempool.py

Co-authored-by: Richard Kiss <him@richardkiss.com>

Co-authored-by: Richard Kiss <him@richardkiss.com>
2022-06-03 12:36:17 -05:00
dustinface
d3baa4c236
Use Plot.from_json_dict instead of dataclass_from_dict (#11760) 2022-06-03 12:35:17 -05:00
dustinface
5c5ab111ad
streamable: Fix default value assignments for dataclass_from_dict (#11732)
* streamable: Use constructor in `dataclass_from_dict`

This fixes default value assignments after #10561 but also leads to less perfomance due to `__post_init__` being called which at least gets mitigated by #11730.

* tests: Test default values with `from_json_dict`

* Convert to `str`, then compare.
2022-06-03 12:14:57 -05:00
Arvid Norberg
3c2fc1947d
upgrade chia-rs to streamable support (#11503)
* upgrade chia-rs to streamable support

* bump chia_rs dependency
2022-06-01 12:56:11 -05:00
Kyle Altendorf
8b2f7ec24c
Add Windows CI (#10681)
* Add Windows CI

* rebuild workflows

* rebuild workflows

* try different activation

* catch up windows workflow template

* debug dir

* hard code dev extra on windows

* it is pytest

* .exe

* -m pytest for windows

* --module pytest

* Revert "debug dir"

This reverts commit 74b5bbfd92.

* debug pkg_resources import on windows

* debug tests.conftest import on windows

* rebuild workflows

* debug

* debug oops

* debug more

* debug more

* Update pytest.ini

* Update pytest.ini

* fix it

* Revert "debug more"

This reverts commit 367ed61dfa.

* Revert "debug more"

This reverts commit 772b6707c6.

* Revert "debug oops"

This reverts commit 5065c1e172.

* Revert "debug"

This reverts commit ecf721e453.

* Revert "debug tests.conftest import on windows"

This reverts commit ea71a51625.

* Revert "debug pkg_resources import on windows"

This reverts commit eb6afb1dfb.

* more warning ignore

* they're regexes...

* --numprocesses 0 for windows ci

* specify each test file individually since pytest on windows does not seem to handle the wildcards

* relative...

* select most recent supported python on windows, support INSTALL_PYTHON_VERSION

* sort test files when generating workflows

* report full python version on windows, actually use the selected version for the env

* oops

* debug dir

* skip timelord-requiring jobs on windows

* oops

* handle wildcard expansion for coverage on windows

* rebuild workflows

* download blocks and plots on windows

* correct and and or in yaml

* -d for Install.ps1

* git config --global core.autocrlf false

* add missing windows plot_sync workflow

* no check resource usage on windows

* delete outdated windows workflows that need the timelord

* catch up windows workflow template

* try again without xdist

* rebuild workflows

* Revert "try again without xdist"

This reverts commit 4bedf5f047.

* maybe now no xdist on windows

* maybe now no xdist on windows

* rebuild workflows

* debug chiapos==1.0.9

* Revert "debug chiapos==1.0.9"

This reverts commit 4b374e128a.

* debug blspy==1.0.9

* catch up windows workflow template

* rebuild workflows

* Various windows CI fixes

* Windows CI fixes and workflow updates

* ignore another warning for the keyring

* pre-commit fixup

* rebuild workflows

* catchup windows test workflows with long_lived/** and release/** setup

* Update test_full_sync.py

* Try without parallelization

* Some other tweaks for windows

* some more timeout tweaks

* more timeout tweaks for windows

* rebuild workflows

* Adjusting more timeouts

* Skip ipv6 test on windows

* More timeout tweaks

* More timeout adjustments

* More time out adjustments

* more windows tweaks

* Try running some things with -n 0

* Fix up windows test template and re-gen workflows

* Try this again

* Try all windows tests with -n 0

* misc update

* Undo some timeout changes

* Adjust timeout

* Increase timeout

* Coverage combine should combine the files in the current directory

* Trying retry test on windows

* timeout adjustment

* use tmp_path

* Few minor updates based on review

* lint fix

* Update tests/build-workflows.py

* Update tests/core/server/test_dos.py

* Try windows with pytest xdist n=2

* Check windows with xdist n=3

* Trying n=2 again

* Adjust timeout

* Update testconfig.py

* Update build-workflows.py

* Update testconfig.py

* Update .pre-commit-config.yaml

* Update testconfig.py

* Update build-workflows.py

* Update build-test-macos

* Update build-test-ubuntu

* Update build-test-windows

* skip

* Apply suggestions from code review

* tidy

* Update test_pool_config.py

Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2022-05-31 14:04:45 -05:00
Mariano Sorgente
8dc1d66cd2
Increase from 180 to 300 (#11707) 2022-05-31 13:40:08 -05:00
William Allen
855fe055fa
Merge pull request #11645 from AmineKhaldi/main_checkpoint_1.4.0_46ebed3
Merge release/1.4.0's 46ebed3 into main
2022-05-27 12:26:50 -05:00
dustinface
5844c6cda2
tests: Avoid using match of pytest.raises together with paths (#11638) 2022-05-26 15:15:30 -07:00
Amine Khaldi
83b49e73a1
Merge branch 'release/1.4.0' into main_checkpoint_1.4.0_46ebed3 2022-05-26 08:51:50 +01:00
dustinface
7142ebcae7
tests: Consolidate farmer/harvester fixtures (#11513)
* tests: Consolidate farmer/harvester fixtures

* Fixup after #11578
2022-05-25 15:15:49 -07:00
arvidn
a328fdcabf
fix jsonify bool 2022-05-25 09:05:47 +02:00
dustinface
dda517ee1a
streamable: Cache convert functions from dataclass_from_dict (#10561)
* streamable: Cache convert functions for dict -> dataclass conversion

* tests: Test `dataclass_from_dict` with non-streamable classes

* `Any` -> `object`

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Move comment into `dataclass_from_dict`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-05-24 13:22:22 -07:00
dustinface
c400d816af
harvester: Validate the path in add_plot_directory (#11625)
Make sure it exists, is a directory and hasn't been added already.
2022-05-24 13:21:44 -07:00
Kyle Altendorf
60c1867a44
async sleep and require connection reset error for DoS test (#11612) 2022-05-23 16:14:04 -07:00
Kyle Altendorf
12ec1a2b3e
add tests.util.misc.assert_maximum_duration for benchmark assertions (#11589)
* add tests.util.misc.assert_maximum_duration for benchmark assertions

* fix some imports

* more

* lint

* correct duration

* hint fixup

* rename to caller_file_and_line()

* set default timer to thread_time()

* extract gc manager and default to disabling

* self calibrate overhead

* wrap none-ness behind .results() method

* add messages

* from __future__ import annotations

* yield out self

* final from typing_extensions

* split it

* just use Future

* tweak

* tweak

* correction in loosely hint checked file

* all future indexing in hints

* union for the hints

* rename

* assert_runtime()

* another rename...

* message -> label

* label all results with test names

* oops
2022-05-23 12:50:48 -07:00
Arvid Norberg
00a2a15844
Coin simplification (#11567)
* factor out as_list() from Coin into free function. Remove unused name_str() from Coin. Minor optimization of hash_coin_ids() for common case.

* extend Coin unit test
2022-05-23 10:43:17 -07:00
Arvid Norberg
3d8081cc0d
restore missing hints being stored as None (instead of 0-length bytes) (#11568) 2022-05-23 10:42:40 -07:00
Mariano Sorgente
817baa3096
Bind port 0 to fix race condition when grabbing available ports (#11578)
* port 0 to fix flakiness

* Try fixing setup_full_system

* Try fixing setup_full_system, and lint

* More attempts to fix

* No more calls to get random ports in setup_nodes

* Revert accidental changes

* Timelord extra arg

* Try with port 0

* Fix daemon test, and lint

* Try without 0.0.0.0

* Back to 0.0.0.0

* Try a few timelord changes to get test running

* Increase timeout again

* Use the correct interface to get the port

* INFO logging to debug issue

* Revert "INFO logging to debug issue"

This reverts commit 7c379e5cca.

* Fix advertised port log

* Add extra log

* Logging back

* Rollback the timelord changes

* Try port 0 timelord

* Revert "Try port 0 timelord"

This reverts commit 4997faf3b2.

* Try full green, change ordering

* Remove unused var

* speed up simulation and cleanup

* Now try without the port config

* Fix a flaky call to get_event_loop

* Try getting the port dynamically

* No dynamic port

* Try changing the ordering

* Try adding a sleep

* Back to what works

* Timelord before vdf clients

* Dynamic port for 1st timelord

* Revert "Dynamic port for 1st timelord"

This reverts commit 0f322a15b7.

* Revert "Timelord before vdf clients"

This reverts commit 3286c34696.

* Revert "Back to what works"

This reverts commit 30380dffb7.

* Revert "Try adding a sleep"

This reverts commit 9212b665a6.

* Revert "Try changing the ordering"

This reverts commit a62597d70d.

* Revert "No dynamic port"

This reverts commit 5d2e15749b.

* Revert "Try getting the port dynamically"

This reverts commit ef9cd75679.

* Revert "Fix a flaky call to get_event_loop"

This reverts commit 01a000fdfb.

* Try one to 0

* Just not 0

* Don't get port dynamically

* Cleanup a bit

* Fix

* Some cleanup work

* Some cleanup work

* Fix daemon test

* Cleanup

* Remove arguments
2022-05-23 08:13:49 -07:00
Mariano Sorgente
b644f37277
Fix a few additional flaky tests (#11597)
* Fix a few additional flaky tests

* Lint

* Update tests/core/full_node/stores/test_full_node_store.py

Co-authored-by: Jeff <jeff@chia.net>

* Fix merge conflict

* Remove unnecessary function

Co-authored-by: Jeff <jeff@chia.net>
2022-05-23 07:55:05 -07:00