Commit Graph

151 Commits

Author SHA1 Message Date
Mariano Sorgente
7fa1861def
Ms.wallet refactor (#10146)
* wallet fixes

* Don't show false positive synched

* Code cleanup and lint

* Fixes

* Revert issue

* Fix reorg issue

* WIP wallet node

* More wallet refactor

* More wallet changes

* More

* Fix subscription bugs

* Fix some tests

* Fix pool tests

* More tweaks

* Lint and small issues

* call update_ui at the correct points

* Small changes

* new peak queue

* Fix peer height issue

* Rollback more for safety, and tweak logging

* Small WSM style fixes

* Change fork point in long_sync

* More fixes with real world testing

* Fix reversed filter

* Fix function name

* Fix coin store bug properly

* Raise CancelledError

* Fix rollback issue

* Lint

* Small fix

* Fix CAT issue

* Fix test trades race condition

* Fix test trades race condition

* Try to reduce flakiness

* Test coin store and fix additional method

* Improve flakiness
2022-02-14 11:28:36 -08:00
Yostra
6315cb4a4b
Try all nodes when fetching a parent. (#10152)
* try all nodes

* lint
2022-02-09 16:19:02 -08:00
Jeff
2314f47ecc
Post a state change when an offer is added or soft-cancelled. More offer (#10153)
state changes may be needed in the future, but for now the GUI will be
able to refresh wallet balances in response to offer soft-cancellation
and addition.
2022-02-08 17:58:59 -08:00
Jeff
67705e0907
Ensure wallet/db directory exists before attempting to migrate a wallet (#10151)
db from standalone_wallet/wallet/db.
2022-02-08 16:51:30 -08:00
Mariano Sorgente
613cdc47f2
Ms.wallet fixes (#10094)
* wallet fixes

* Don't show false positive synched

* Code cleanup and lint

* Fixes

* Revert issue

* Fix reorg issue

Co-authored-by: wjblanke <wjb98672@gmail.com>
2022-02-07 16:38:45 -08:00
Yostra
87aeadc343
Wallet improvements (#10077)
* disonnect from untusted faster, fork point change, pool state handle

* name conflict

* deadlock

* fix inclusion validation for first sub epoch, don't fetch header blocks on every new peak

* lint

* can be none

* revert changes

* stop wallet peers if trusted peer is connected and synced

* remove pool changes

* remove cononfusing log
2022-02-07 16:13:42 -08:00
Jeff
f67ca2c5a3
STANDALONE_ROOT_PATH now expands to a different env variable to avoid (#10114)
conflicting with DEFAULT_ROOT_PATH when CHIA_ROOT is set. When the GUI
launches chia services, CHIA_ROOT is set, which was preventing the
wallet backend from copying existing DBs from the standalone_wallet.
2022-02-07 11:06:36 -08:00
Jeff
eaa6b8beab
Rename migrated standalone_wallet wallet DB to v2 (#10086)
* Rename migrated standalone_wallet wallet db to use '_new.sqlite' suffix

* Removed unnecessary format string

* Wallet DB is now named "v2" instead of "_new"
2022-02-04 09:49:20 -08:00
Mariano Sorgente
3b03872146
Ms.flaky2 (#10080)
* tests: add missing await to fix flaky test

* tests: More flaky pool tests

* WIP

* wallet: apply multiple states per block for pool wallet

* lint
2022-02-02 16:58:15 -05:00
arty
d2f35e627c
Fix some spam when using clvm_tools_rs (#10043) 2022-02-02 10:42:47 -08:00
Mariano Sorgente
5b58ec3848
Ms.wallet cache (#10070)
* wallet: Invalidate caches for reorg

* wallet: lint

* wallet: remove logging

* wallet: simplify logic

* Update chia/wallet/wallet_node.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-02-02 10:05:57 -05:00
Kyle Altendorf
29fad42e0c
await peer.close() in WalletNode.validate_received_state_from_peer() (#10054)
```
farm/chia-blockchain/chia/wallet/wallet_node.py:1058: RuntimeWarning: coroutine 'WSChiaConnection.close' was never awaited
peer.close(9999)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
```
2022-02-01 09:56:35 -08:00
Earle Lowe
fd5e39470d
Do not assert if pool_state is none (#9893) 2022-01-31 19:00:49 -08:00
Arvid Norberg
275969ddd1
remove redundant function, calculate_cost_of_program(). The cost is now part of the NPCResult object (#9964) 2022-01-31 18:49:12 -08:00
Jeff
d136b84498
Fixed Offer history limit and GUI sorting (#10025)
* trade_store: Migrated the `is_my_offer` property from the serialized
trade_record to a new `is_my_offer` column in the trade_records table.
Rewrote get_trades_between to move the query/sorting into the database.
This supports the presentation of offers in the GUI (two lists sorted by
relevance).

RPC: get_all_offers now accepts options to restrict the listing to
completed offers, my offers, taken offers. Added get_offers_count call
to return a breakdown of the number of offers (my offers, taken offers,
total).

CLI: Updated `chia wallet get_offers` to include options to exclude
created/taken offers, as well as an option to reverse the listing. The
-ia|--include-all option has been renamed -ic|--include-completed to
make sense in the context of the --exclude-[my-offers|taken-offers]
options. The full listing is now output, fixing the issue with hitting
the prior paging cap of 50 records.

* Linter fixes

* Updated trade tests. Fixed reverse sort when ordering offers by relevance.

* Make column spec explicit on insert to better support possible future migrations. If a future migration adds a new column with a default value, inserts/updates made by an older client will fail unless the insert explicitly indicates which columns are being updated. By explicitly listing the columns to update, any unspecified columns will have their default value applied.
2022-01-31 18:48:05 -08:00
Mariano Sorgente
34edf6e35e
pool_wallet: Deterministic derivation of auth key, and only count poo… (#9922)
* pool_wallet: Deterministic derivation of auth key, and only count poolWallets for indeces

* cli: Use the same limit for cli creation

* farmer: no more auth keys in config

* farmer: handle case of no authentication sk

* farmer: remove async, and fix test

* lint

* test: Add a test for many pool wallets, and user store

* test: Wait for wallet to be synced in test

* wallet: fix startup (no dns servers) and change to server checking of plotnfts

* cli: change to server checking

* tests: fix user store test

* Improve caching

* PR comments

* Remove useless class

* Try to fix test again
2022-01-30 12:31:55 -05:00
Yostra
ca84fa9f8d
Sync improvements (#9974)
* stash

* change log level

* stash changes

* cat tests

* add node id chia show

* make max sibscribe items configurable

* lint

* add logs for limits

* paralelize incluzion valdiation, don't diconnet from remote peers until local peer in synced

* wallet rate limit

* test log lvl, trusted, black

* only validation under semaphore

* flake mypy trade test

* reduce flakyness

* merge fix

* rename

* linting

* validate parallel, apply serial

* reduce falkyness

* lint

* Fix CAT tests

* black

* flaky test, total amount

* fix cat test

* skip existing state

* skip existing

* try other caches

* flake8

Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
2022-01-28 19:16:47 -08:00
Arvid Norberg
b8ada1ceb7
introduce restrictions on generators at a specific height (#9957)
* introduce restrictions on generators at a specific height. disallow division on negative numbers and disallow redundant leading zeros on integer condition arguments (produced by a generator)

* use SOFT_FORK_HEIGHT constant

* there is no need to specify height when validating block in block_creation

* Update tests/core/full_node/test_mempool.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-01-28 12:29:11 -08:00
Yostra
b08c904ebf
migration (#9948) 2022-01-26 17:19:42 -08:00
Arvid Norberg
7acdf2d30f
avoid converting puzzle_reveal and solution into python representation, just to convert it back again when running it (#9958) 2022-01-26 09:06:41 -08:00
Arvid Norberg
234b31f21a
move CoinSpend.hints() to a free function in the wallet. It does not belong in the full node and can only be called on coins whose program is trusted (#9953) 2022-01-26 08:52:41 -08:00
Arvid Norberg
77db90ba6a
move member function get_memos() out of SpendBundle and into a free-function living in the wallet code. It does not belong to the full node. Also rename it to indicate that it's expensive (and dangerous) to call (#9907) 2022-01-23 21:45:41 -08:00
Juraj Oršulić
515ee9b07e
Spelling nits in p2_delegated_puzzle_or_hidden_puzzle.py (#9913)
* Spelling nits

* One more

* ++
2022-01-23 21:44:25 -08:00
Kyle Altendorf
c1a8dac07c
Set default wallet: dns_servers in code (#9880) 2022-01-21 18:53:48 -08:00
Kyle Altendorf
ee9c7968a8
Pick TX type even when added coin has been spent (#9840) 2022-01-21 09:54:01 -08:00
Matt Hauff
2e3c054c3a
Add remaining offer code to main (#9886)
* Add remaining offer code to main

* Fix compression test

Co-authored-by: Jeff <paninaro@gmail.com>
2022-01-20 14:49:31 -08:00
Kyle Altendorf
2acf03e850
Default connect_to_unknown_peers to True (#9864) 2022-01-18 20:17:44 -08:00
wjblanke
6283f82f85
update sauce (#9816) 2022-01-14 11:59:46 -08:00
Yostra
89f15f591c
Merge standalone wallet into main (#9793)
* wallet changes from pac

* cat changes

* pool tests

* pooling tests passing

* offers

* lint

* mempool_mode

* black

* linting

* workflow files

* flake8

* more cleanup

* renamed

* remove obsolete test, don't cast announcement

* memos are not only bytes32

* trade renames

* fix rpcs, block_record

* wallet rpc, recompile settlement clvm

* key derivation

* clvm tests

* lgtm issues and wallet peers

* stash

* rename

* mypy linting

* flake8

* bad initializer

* flaky tests

* Make CAT wallets only create on verified hints (#9651)

* fix clvm tests

* return to log lvl warn

* check puzzle unhardened

* public key, not bytes. api caching change

* precommit changes

* remove unused import

* mypy ci file, tests

* ensure balance before creating a tx

* Remove CAT logic from full node test (#9741)

* Add confirmations and sleeps for wallet (#9742)

* use pool executor

* rever merge mistakes/cleanup

* Fix trade test flakiness (#9751)

* remove precommit

* older version of black

* lint only in super linter

* Make announcements in RPC be objects instead of bytes (#9752)

* Make announcements in RPC be objects instead of bytes

* Lint

* misc hint'ish cleanup (#9753)

* misc hint'ish cleanup

* unremove some ci bits

* Use main cached_bls.py

* Fix bad merge in main_pac (#9774)

* Fix bad merge at 71da0487b9

* Remove unused ignores

* more unused ignores

* Fix bad merge at 3b143e7050

* One more byte32.from_hexstr

* Remove obsolete test

* remove commented out

* remove duplicate payment object

* remove long sync

* remove unused test, noise

* memos type

* bytes32

* make it clear it's a single state at a time

* copy over asset ids from pacr

* file endl linter

* Update chia/server/ws_connection.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-01-13 12:08:32 -08:00
arty
8610ab704a
Add commentary about sign_coin_spends and p2_delegated_puzzle_or_hidden_puzzle (#9452) 2022-01-10 21:00:24 -08:00
Kyle Altendorf
e498a406a0
Make the hard coded 2 reserved cores configurable (#9709)
* Make the hard coded 2 reserved cores configurable

* add missing reserved_cores parameters

* add missing reserved_cores parameters
2022-01-10 20:58:29 -08:00
Kyle Altendorf
499d01a6c5
Extend the use of AmountWithPuzzlehash(TypedDict) and more uint64() (#9563) 2021-12-17 14:36:48 -08:00
Mariano Sorgente
2f86d0eee1
Ms.fix puzzle store (#9606)
* Fix puzzle store to 100 addresses

* Remove unnecessary function call

* Lint
2021-12-17 14:34:27 -08:00
dustinface
234d121c2a
chia|tests|pre-commit: Add missing __init__.py files (#8758)
* chia|tests: Add missing `__init__.py` files

* check and create needed __init__.py files via pre-commit

* plotting|tests: Handle `mypy` issues introduced by the `main` rebase

* pre-commit: Run `build-init-files.py` more verbose

* tests: Ignore `__pycache__` folders in `build-init-files.py`

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* tests: Add shebang, make executable, add description

To `tests/build-init-files.py`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-16 18:52:46 -08:00
Arvid Norberg
2801d96477
unify the name we use for strict/safe mode (#9545)
* unify the name we use for the special mode we run CLVM in, when accepting transactions into the mempool. We currently call this, strict mode, safe mode, safe and unsafe. Neither of these names are very descriptive. This patch renames this mode to mempool-mode

* import as
2021-12-13 08:38:35 -08:00
Matt Hauff
c6637455b3
Added sorting functionality to /get_transactions (#9278)
* Added sorting functionality to /get_transactions

* Added comment about reversals

* Add determinism and clean up reversal

* Change default back to 50
2021-12-10 15:19:58 -08:00
Kyle Altendorf
44b4d69ad7
Fix some hints (#9484)
* Fix some hints

* remove unused import

* correct AmountWithPuzzlehash.amount to be uint64
2021-12-10 15:00:43 -08:00
Kyle Altendorf
5eeec865e9
stub and use SizedBytes.fromhex() until upstream is released (#9458)
* stub SizedBytes.fromhex() until upstream is released

https://github.com/python/typeshed/pull/6201

* black

* more

* more

* more

* more

* more

* import TYPE_CHECKING via from

* more

* more

* catch up
2021-12-10 14:59:49 -08:00
Juraj Oršulić
9ec44cc2ac
Fix comment typo in the standard wallet puzzle (#9346) 2021-12-10 09:17:06 -08:00
Kyle Altendorf
c5902841de
Just use bytes32() some more (#9472)
* Just use bytes32() some more

* black

* more
2021-12-07 07:55:48 -08:00
Kyle Altendorf
b998193c7e
create and use SizedBytes.from_hexstr() (bytes32) (#9467)
* create and use SizedBytes.from_hexstr() (bytes32)

* another

* more

* more
2021-12-07 07:54:26 -08:00
Kyle Altendorf
8291f0221a
Make the sized bytes types hint compatible (#9369)
* Rework sized bytes for type hinting compatibility

* add a bunch of type: ignores

* this will be handled elsewhere

* noqa E501 instead of changing code

* normalize comment plurality

* @classmethod

* Revert "@classmethod"

This reverts commit 95db80e339.

* add ignore in benchmarks

* just E501 again...

* add some new type: ignores
2021-12-02 09:43:39 -08:00
Chris Ross
2c4aaa1ec9
Resolve v6 addresses (#8861)
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.

* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml

* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.

* Update a couple things for consistency

* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.

* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives.  also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.

* Change the default here to match the default coded into util/network.py.  It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)

* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on

* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.

* Don't test for ip6-localhost, as it's not on all systems.

* Bah.  Forced formatting of commented code...

* Add a type annotation for the addrset variable

* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined.  So, quote them here.

Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-11-22 14:34:03 -08:00
Kyle Altendorf
05f67167fe
Correct some type hints adjacent to bytes32 (#8957)
* Correct some type hints adjacent to bytes32

Note that these errors won't show up until the sized bytes are
rewritten to "support" hinting.  They are too dynamic at the moment.

* drop some bytes32 hints with unneeded Optional

* Apply suggestions from code review

* remove code change, this pr is hints only
2021-11-22 13:16:52 -08:00
Jeff
c4e14f5c78
Peer db new serialization (#9079)
* Serialize/deserialize peer data alongside existing sqlite implementation (to be removed)

* Simplified AddressManagerStore. No longer uses sqlite and is no longer async.

* Removed aiosqlite usage from AddressManagerStore.
Added PeerStoreResolver class to determine the appropriate location for "peers.dat"
Updated initial-config.yaml to include "peers_file_path" default, replacing "peer_db_path" (similar change for "wallet_peers_path")

* Minor comment changes/additions

* Added migration from sqlite peer db.
Made AddressManagerStore's serialization async as it was right at the edge of blocking for too long.

* Minor tweaks to checking for migration

* Removed AddressManagerSQLiteStore class scaffolding

* makePeerDataSerialization now returns bytes instead of a PeerDataSerialization object

* Async file I/O for write_file_async using aiofiles
Added more tests

* Separate out the synchronous part of move_file

* Renamed write_file to files since we're opening up the capabilities a bit

* Update references to write_file

* Renamed test_write_file to test_files

* Tests covering move_file and move_file_async

* Minor refinements to behavior and tests

* Use aiofiles for reading peers.dat

* Added missing mypy typing info for aiofiles. Also added types-PyYAML to dev_dependencies so that `mypy chia tests` doesn't require running with --install-types

* Add types-aiofiles to the linting workflow

* Directory perms can now be passed into write_file_async.
Added an explicit f.flush() followed by os.fsync() after writing the temp file contents.
2021-11-19 11:12:58 -08:00
Mariano Sorgente
8a028c3594
Ms.mempool locking (#9050)
* Prority locking to consensus

* Remove pstats

* Linting

* Do some stuff outside of lock

* Fix startup

* Add log timings

* Try some different locking

* Add limit

* catch excp

* CLVM inside lock

* Try using a semaphore instead

* use events for lock queue

* test

* Add logging for message types

* type

* remove seed

* check new peak waiters

* correct FullNodeAPI self.full_node.new_peak._waiters typo

* correct logging string typos

* only warn about new_peak Waiters if there is at least 1

* remove no-longer-accepted parameter to FullNode.peak_post_processing()

* only warn about respond_transaction Waiters if there is at least 1

* lint

* Change some constants

* Small fix and logging changes

* Put message types outside

* Change some log levels so we can test with info

* More logging

* Increase rate limits but decrease paralelism

* tweaks

* Log dropped tx

* Fix pool rpc test

* Test fixes

* Mempool optimization

* Remove from seen if fails

* Increase queue sizes

* Message types info

* More test and logging

* Small changes to networking just in case

* Decrease logging

* Decrease logging even further

* Decrease logging even further even further

* Decrease logging 3

* Transaction queue

* Don't cancel tasks or close connection

* Cancel tasks on disconnect (for shutdown purposed)

* Fix typo

* Catch cancelled

* Do multiple at a time

* More accurate farmer response time

* More efficiently create tasks

* Increase queue size and priority by fee

* Revert priority

* Don't re-request too many times for dropped TX

* Handle cancelled error so we don't go into a bad state

* Catch cancelled in syncing tasks

* Reduce new_peak_sem to improve performance

* Less bytes conversion

* Missing file, and 2 workers for CLVM

* Validate BLS in a new thread

* tests

* Change semaphore constants

* correct a cancellation triggered exception and assertion

* Fix send_transaction, dont use BaseException, fix tests

* Fix more tests

* only log transaction handler cancellation in debug

* typing in log

* move unfinished validation to diff proc

* it is asyncio.CancelledError

* Add a test for bad signature

* Fix more tests, reduce logging, lint

* One more lint

* blockchain tests, pass bytes directly, single call

* Try to fix rl_wallet failures

* Fix mempool test

* catch everything

* Don't test RL wallet

* Fix more tests and return error code

* Improve error handling in multiprocess

* Add pre-validation time

* Add pre-validation time in logs, and revert pytest.ini changes

* Add log correctly

* Ms.bls cache experiment (#9115)

* Logging for cache

* Less logging

* Return to original plan

* Clean up

* Remove coment

* Remove log

* formalize LockQueue shutdown

* Comments

* Fix blockchain test

* Improve cache

* Remove logs

* Fix sign_coin_spends

* Fix pool wallet

Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
2021-11-04 09:29:05 -07:00
Kyle Altendorf
eb62a28519
Add missing .name() in WalletStateManager.get_filter_additions_removals() (#8958) 2021-11-02 08:44:43 -07:00
Kyle Altendorf
4a12db54a5
Make WalletPuzzleStore.get_derivation_record_for_puzzle_hash() take bytes32 (#8960) 2021-10-29 10:39:06 -07:00
Kyle Altendorf
027aa116a2
Remove tupling of a hash (#8948) 2021-10-29 10:27:39 -07:00
Matt Hauff
3d1aa2b01d
Check that launcher is interesting (#9003)
* Check that launcher is interesting

* Added test

* Lint
2021-10-29 08:43:02 -07:00