Commit Graph

112 Commits

Author SHA1 Message Date
Kyle Altendorf
c5902841de
Just use bytes32() some more (#9472)
* Just use bytes32() some more

* black

* more
2021-12-07 07:55:48 -08:00
Kyle Altendorf
b998193c7e
create and use SizedBytes.from_hexstr() (bytes32) (#9467)
* create and use SizedBytes.from_hexstr() (bytes32)

* another

* more

* more
2021-12-07 07:54:26 -08:00
Kyle Altendorf
8291f0221a
Make the sized bytes types hint compatible (#9369)
* Rework sized bytes for type hinting compatibility

* add a bunch of type: ignores

* this will be handled elsewhere

* noqa E501 instead of changing code

* normalize comment plurality

* @classmethod

* Revert "@classmethod"

This reverts commit 95db80e339.

* add ignore in benchmarks

* just E501 again...

* add some new type: ignores
2021-12-02 09:43:39 -08:00
Chris Ross
2c4aaa1ec9
Resolve v6 addresses (#8861)
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.

* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml

* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.

* Update a couple things for consistency

* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.

* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives.  also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.

* Change the default here to match the default coded into util/network.py.  It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)

* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on

* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.

* Don't test for ip6-localhost, as it's not on all systems.

* Bah.  Forced formatting of commented code...

* Add a type annotation for the addrset variable

* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined.  So, quote them here.

Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-11-22 14:34:03 -08:00
Kyle Altendorf
05f67167fe
Correct some type hints adjacent to bytes32 (#8957)
* Correct some type hints adjacent to bytes32

Note that these errors won't show up until the sized bytes are
rewritten to "support" hinting.  They are too dynamic at the moment.

* drop some bytes32 hints with unneeded Optional

* Apply suggestions from code review

* remove code change, this pr is hints only
2021-11-22 13:16:52 -08:00
Jeff
c4e14f5c78
Peer db new serialization (#9079)
* Serialize/deserialize peer data alongside existing sqlite implementation (to be removed)

* Simplified AddressManagerStore. No longer uses sqlite and is no longer async.

* Removed aiosqlite usage from AddressManagerStore.
Added PeerStoreResolver class to determine the appropriate location for "peers.dat"
Updated initial-config.yaml to include "peers_file_path" default, replacing "peer_db_path" (similar change for "wallet_peers_path")

* Minor comment changes/additions

* Added migration from sqlite peer db.
Made AddressManagerStore's serialization async as it was right at the edge of blocking for too long.

* Minor tweaks to checking for migration

* Removed AddressManagerSQLiteStore class scaffolding

* makePeerDataSerialization now returns bytes instead of a PeerDataSerialization object

* Async file I/O for write_file_async using aiofiles
Added more tests

* Separate out the synchronous part of move_file

* Renamed write_file to files since we're opening up the capabilities a bit

* Update references to write_file

* Renamed test_write_file to test_files

* Tests covering move_file and move_file_async

* Minor refinements to behavior and tests

* Use aiofiles for reading peers.dat

* Added missing mypy typing info for aiofiles. Also added types-PyYAML to dev_dependencies so that `mypy chia tests` doesn't require running with --install-types

* Add types-aiofiles to the linting workflow

* Directory perms can now be passed into write_file_async.
Added an explicit f.flush() followed by os.fsync() after writing the temp file contents.
2021-11-19 11:12:58 -08:00
Mariano Sorgente
8a028c3594
Ms.mempool locking (#9050)
* Prority locking to consensus

* Remove pstats

* Linting

* Do some stuff outside of lock

* Fix startup

* Add log timings

* Try some different locking

* Add limit

* catch excp

* CLVM inside lock

* Try using a semaphore instead

* use events for lock queue

* test

* Add logging for message types

* type

* remove seed

* check new peak waiters

* correct FullNodeAPI self.full_node.new_peak._waiters typo

* correct logging string typos

* only warn about new_peak Waiters if there is at least 1

* remove no-longer-accepted parameter to FullNode.peak_post_processing()

* only warn about respond_transaction Waiters if there is at least 1

* lint

* Change some constants

* Small fix and logging changes

* Put message types outside

* Change some log levels so we can test with info

* More logging

* Increase rate limits but decrease paralelism

* tweaks

* Log dropped tx

* Fix pool rpc test

* Test fixes

* Mempool optimization

* Remove from seen if fails

* Increase queue sizes

* Message types info

* More test and logging

* Small changes to networking just in case

* Decrease logging

* Decrease logging even further

* Decrease logging even further even further

* Decrease logging 3

* Transaction queue

* Don't cancel tasks or close connection

* Cancel tasks on disconnect (for shutdown purposed)

* Fix typo

* Catch cancelled

* Do multiple at a time

* More accurate farmer response time

* More efficiently create tasks

* Increase queue size and priority by fee

* Revert priority

* Don't re-request too many times for dropped TX

* Handle cancelled error so we don't go into a bad state

* Catch cancelled in syncing tasks

* Reduce new_peak_sem to improve performance

* Less bytes conversion

* Missing file, and 2 workers for CLVM

* Validate BLS in a new thread

* tests

* Change semaphore constants

* correct a cancellation triggered exception and assertion

* Fix send_transaction, dont use BaseException, fix tests

* Fix more tests

* only log transaction handler cancellation in debug

* typing in log

* move unfinished validation to diff proc

* it is asyncio.CancelledError

* Add a test for bad signature

* Fix more tests, reduce logging, lint

* One more lint

* blockchain tests, pass bytes directly, single call

* Try to fix rl_wallet failures

* Fix mempool test

* catch everything

* Don't test RL wallet

* Fix more tests and return error code

* Improve error handling in multiprocess

* Add pre-validation time

* Add pre-validation time in logs, and revert pytest.ini changes

* Add log correctly

* Ms.bls cache experiment (#9115)

* Logging for cache

* Less logging

* Return to original plan

* Clean up

* Remove coment

* Remove log

* formalize LockQueue shutdown

* Comments

* Fix blockchain test

* Improve cache

* Remove logs

* Fix sign_coin_spends

* Fix pool wallet

Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
2021-11-04 09:29:05 -07:00
Kyle Altendorf
eb62a28519
Add missing .name() in WalletStateManager.get_filter_additions_removals() (#8958) 2021-11-02 08:44:43 -07:00
Kyle Altendorf
4a12db54a5
Make WalletPuzzleStore.get_derivation_record_for_puzzle_hash() take bytes32 (#8960) 2021-10-29 10:39:06 -07:00
Kyle Altendorf
027aa116a2
Remove tupling of a hash (#8948) 2021-10-29 10:27:39 -07:00
Matt Hauff
3d1aa2b01d
Check that launcher is interesting (#9003)
* Check that launcher is interesting

* Added test

* Lint
2021-10-29 08:43:02 -07:00
Matt Hauff
0e941f5144
Fix launcher name definition (#8971)
* Fix launcher name definition

* Minor update per @yostra
2021-10-26 16:17:33 -07:00
Jeff Cruikshank
f7f44d15ab
Fix long load times when selecting the default wallet key from the GUI. (#8938)
When creating a new WalletNode at launch, the fingerprint will be set to None. After a lengthy initialization task, the logged-in fingerprint was set to None instead of the default private key's fingerprint. If the user then selected that same key in the key selection screen, the same lengthy initialization task would be repeated. This fix will set the logged-in fingerprint to the default key's fingerprint if the fingerprint wasn't originally specified.
2021-10-26 15:31:49 -07:00
Arvid Norberg
1f893907b9
make db_sync setting default to FULL. Extend the configuration to cover all of OFF, NORMAL and OFF (#8836) 2021-10-16 09:02:18 -07:00
Arvid Norberg
61a513869d
make synchronous mode in sqlite database connections configurable (#8753)
* make synchronous mode in sqlite database connections configurabel. Default to enabling synchronous mode, leaving the door open for a more sophisticated heuristic in the future

* Update chia/util/db_synchronous.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/util/initial-config.yaml

Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>
2021-10-08 06:13:06 -07:00
Chris Ross
0a2294841d
Before calling gethostbyname(); check if host is an address (#8765)
* Before calling gethostbyname(), check to see if we already have an address.
That prevents an unnecessary call, and also allows for specifying IPv6
addresses which otherwise cause exceptions here.

* Also recognize addresses when passed to timelord spawn_process, so that
IPv6 addresses can be used.

* Missed importing PeerInfo

* Adjust style to match requirements and existing code

* Cast PeerInfo port to uint16
2021-10-07 16:56:34 -07:00
arty
57110087c8
20210923 clvm tools rs (#8601)
* Allow experimental use of clvm_tools_rs via env var

* Final version of load_clvm.  handle module paths

* Fix some lint hopefully

* Opps

* lint

* More lint

* black format
2021-10-05 10:49:21 -07:00
Arvid Norberg
600a5f90a0
CoinStore benchmark and performance improvements (#8514)
* add coin_store benchmark

* optimize _set_spent in CoinStore

* simplify _add_coin_record, since we won't use it to replace an entry anymore

* use executemany in add_coin_record and set_spent

* disable sqlite sync
2021-09-21 10:43:43 -07:00
Arvid Norberg
31dab2ae5f
Rust only conditions (#8467)
* no need to check announcements in python, it's all done in rust

* remove python condition parsing, use rust unconditionally

* no need to assert_my_amount, it's done in rust

* assert my puzzlehash is done in rust, no need to check it in python

* parent id is checked in rust, no need to do it in python

* my coin ID is checked in rust, no need to do it in python

* reorder condition checks to have to ones we won't see at the end

* no need for the RUST_CONDITION_CHECKER constant anymore

* update pool tests to use get_name_puzzle_condition
2021-09-18 10:25:19 -07:00
Arvid Norberg
b748bb1356
Disable sqlite sync (#8319)
* change sqlite pragma synchronous=FULL to pragma synchronous=NORMAL to improve disk I/O performance. In the case of catastrophic (hardware, power or kernel) failure, our database can be rebuilt

* removed redundant database pragmas in wallet
2021-09-17 10:47:34 -07:00
Arvid Norberg
e4b9db08cd
execute the sqlite pragmas only once, at the lever where the database is opened. Also, say FULL instead of 2, to make it clearer what the pragma is doing (#5993) 2021-09-15 10:57:13 -07:00
Matt Hauff
e18a71376d
Make SpendBundle.debug() use the default genesis challenge by default (#8431) 2021-09-14 10:39:40 -07:00
matt-o-how
0096f755a1
DID Update Merge (#8129)
* fixed the bugs related to DID puzzles

* change test so that wallets recover into other wallets

* fix parent_info fetching when recovering

* fix did_test

* delete did tests related to singleton behaviours as that is tested elsewhere

* linting fixes

* update hash commit for did_innerpuz.clvm

* update DID wallet RPC calls

* more lint fixes

* delete further references to deprecated did_spend function

* fix bug in wallet state manager related to falsely detecting pool create

* added did_rpc test

* make sure amounts are uint64

* lint fixes

* Fix deadlock in DID wallet creation, and introduce create_new_did_wallet API call

* rename solution_to_pool_state

* Remove tests dir from packages

* added potential fix

* Allow getting unconfirmed balance from wallet_state_manager when under lock

* Remove a deadlock from create_new_did_wallet

* Update DID wallet test to use async check for farmed chia in wallet setup

* Fix unsigned arithmetic error

* Make DID wallet tests compatible with WalletStateManager lock checking

* check if removal belongs to the wallet

* unconfirmed

* did test cleanup

* fix temppuzhash to be an innerpuz

* clean up DID Wallet, add test for previously broken case

* added state_change call when coin added
added did_info for transaction sent

* update delete wallet parameters

* add comments to DID innerpuz

* fix duplicating bug with tx store

* re-enable did tests

* fix cc wallet bare raise

* remove unused assignation

* fix bare spend in did_wallet

* fix small bug

* messages are a cons box where the first value denotes type of message

* cc_wallet uses new parameter for get_confired_balance

* updates to the puzzle based upon suggestion by Richard

* update git submodule chia-blockchain-gui to did_branch

* updated gui to did_branch 76f9e6cea9f58a30984580b594631f3ae2679752

* updated gui to 041ac79be4

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: William Blanke <wjb98672@gmail.com>
2021-09-07 12:48:41 -07:00
Yostra
f918669f32
Fix single coin generator (#8185)
* fix single coin generator

* add sha256tree file

* add test for single

* pass it as array

* remove breakpoint
2021-08-24 10:50:40 -07:00
Almog De Paz
c4c14939a5
Sync faster (#7665)
* better concurrency for sync_from_fork_point

* dedup code in wallet

* revert

* lint

* lint

* logs

* lint

* lint

* error handling, async bug fix

* lint

* break fetch if bath failed coroutine

* remove outdated comment

* consolidate duplicate code

* consolidate duplicate code

* notify wallet after each batch

* typo

* add try/catch/finally

* typing, check stale peer

* comment on worker shutdown
2021-08-18 18:26:38 -07:00
Earle Lowe
da5df49f39
New call for logging peer_host (#8108) 2021-08-18 13:02:30 -07:00
Arvid Norberg
5ee182932e
Rust conditions (#7167)
* use rust implementation of condition checker

* enable rust condition checker at specific block height

* bump switch-over height
2021-08-17 14:57:41 -07:00
arty
2ea15bbda3
Add documentation for treehash params (#8092) 2021-08-17 10:53:49 -07:00
Jeff Cruikshank
325d7f8245
Keyring passphrase protection (#7249)
* Moved keyring handling into a KeyringWrapper class

* Update click to 8.0.x for prompt_required support

* Renamed KeyringWrapper to _KeyringWrapper

* Expose password management options on Linux

* CLI support for setting/removing a password

* Global option for specifying the master password

* Cache the password instead of setting on the context

* Password bootstrapping during chia init

* Tidying up _KeyringWraper's interface

* Initial pass migrating the legacy keyring contents

* Encryption/decryption of keyring.yaml contents

* FileKeyring backend encrypts with ChaCha20Poly1305

* Tightened up keyring migration and initialization

* Fixed issues identified by linters

* Remove root_path from Keychain

* Prevent double-migration if setting master passwd

* KeyringWrapper tests are mostly complete

* FileKeyring will now honor the service param

* Tests for get/set/delete password

* Formatting/commenting updates

* Writer lock support with tests - WIP

* keyring.yaml is now watched for modifications

* Reader/Writer lock for get/delete password

* Fixed linter issues

* Reader lock tests

* Formatting update

* Hook up CHIA_ROOT support for KeychainWrapper

* Quick fix to address test failures

* Fixed failures when existing legacy keyring exists

* Fixed test failures caused by reusing the same temp dir

* keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path

* Fixed migration failure when setting a password (not using the default)

* KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False

* The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password.

* Added is_keyring_locked RPC call

* Added 'unlock_keyring' RPC command

* Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class.

* Turn on macOS support for testing keyring passwords

* Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain.

Fixed key reconstruction when sent over RPC.

* Farmer now accesses the keychain over RPC

* Fixes for linter issues and some restructuring to support tests that use setup_nodes.py

* Couple of fixes to unblock the GUI from launching when a keyring password is set

* Added a keychain RPC call for add_private_key()

* Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys

* Check for None when inspecting request arguments

* Run check_keys after unlocking the keyring when the daemon is launched via GUI

* Added check_keys RPC method.
Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details.

* Added get_first_private_key to keychain_server/proxy.
create_plots now uses the keychain proxy when launched from the daemon.

* Added a comment about KeychainProxy in chia plots check

* Workaround import conflict  when importing from 'tests.*' due to fasteners name conflict

* Simulator now uses KeychainProxy if launched by the daemon.
KeychainServer/Proxy now takes keychain user/testing params for testing scenarios.

* Added "set_keyring_passphrase" RPC message

* Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests.

* Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete()

* Silencing file_keyring logging for the moment.

* Updated tests to use test keychains and appropriate BlockTools construction

BlockTools should now be created with create_block_tools(_async) to handle async scenarios.

Updated block_tools to be async compatible

Updated fasteners to fix installation of top-level 'tests' in site-packages

* Added 'remove_keyring_passphrase' RPC message to the daemon

Minor tweak to TempKeyring to default to some test params

* Fixed linter issues

* Remove flake8 ignore statement now that the fasteners module has been updated

* Some initial renaming changes: password -> passphrase

* Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon.

Fixed legacy keyring initialization (for migration scenarios)

* Fixed improperly merged file

* Fixed linter issues.
More renaming.

* Updated spots that were still using an incorrect keychain call

* Renamed use_password_cache, obtain_current_password

* Renamed supports_keyring_password

* Renamed has_master_password

* Renamed has_cached_password, get_cached_master_password

* Linter fixes

* Renamed master_password_is_valid

* Renamed set_cached_master_password

* Renamed set_master_password

* Renamed remove_master_password

* Renamed has_cached_master_password

* Renaming in file_keyring and keyring_wrapper

Updated default keyring payload used for tests

* Renamed get_password
Other renaming updates

* Renamed set_password
Other renaming updates

* Renamed remaining password occurrences (where appropriate)

* password -> passphrase

* Added tests for setting an emoji and Japanese master passphrase

* Attempt to notify the daemon when a keyring passphrase is set/updated/removed

* Missed one password -> passphrase replacement.

* Fixed some file synchronization issues found when running tests on macOS

* Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS.

* Removed logging statements previously added for debugging

* Prompt for keyring passphrase up-front when launching a service.

Changed --have-gui flag to --wait-for-unlock

* Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase.

* Minor test cleanup to deduplicate some code.

* Fixed regression when setting a new master passphrase

* Minor refactoring and docs/commenting updates

* Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends.

* Disabling macOS support (previously added for testing only)

* Disabling passphrase support in preparation for sending out the PR

* Fixed improper merge (vscode didn't save changes during rebase)

* Update chia/cmds/init_funcs.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

* skip_check_keys -> should_check_keys

* Shuffling some imports around to break cycles reported by LGTM

* Handle unlocking the daemon if it's already launched and waiting for unlock.

* Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets.

Linter/reformatting fixes

* Replaced uses_keychain_proxy decorator with a clearer method.

* Cleanup the temp keyring dir using shutil.rmtree()

* Restored self._root_path (had been changed to self.root_path)

* Minor cleanup

* ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails

* Plot key resolution now yields a PlotKeys object which can be passed into create_plots.

De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy.

* Added 'keyring_status' daemon RPC message to support the GUI

* Minor changes relating to PR feedback

* Addressed more PR feedback (mostly type annotations)

* Commented-out macOS file keyring usage. This can be re-enabled for testing purposes.

* Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance.

* Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response.

* Updated type annotations and method signature for set_passphrase to expect a string instead of bytes.

* Fixed Wallet RPC tests

* Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s)

* Fixed test failures in test_pool_rpc

* Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories.

* Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data.

* Second attempt at suppressing LGTM false positives

* Third attempt at addressing LGTM false positives

* Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly.

* Reformatting

* More targeted update of the test config to refresh just the "plot_directories" value

* More LGTM suppressions

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 12:46:55 -07:00
Arvid Norberg
7927a45945
remove initial transaction freeze feature (#7922) 2021-08-03 10:21:37 -07:00
Adam Kelly
d029a2a594
Separate locking and non-locking cases for get_confirmed_balance_for_wallet to allow calling a few wallet_state_manager methods while already under the wallet_state_manager lock (#7790) 2021-08-03 09:55:57 -07:00
dustinface
b2fdeb2291
pre-commit: Add initial config and run it for all files (#7053)
* pre-commit: Add initial `.pre-commit-config.yaml`

* pre-commit: Apply hook `end-of-file-fixer`

* pre-commit: Apply hook `trailing-whitespace`

* pre-commit: Apply hook `debug-statements`

* contrib: Add `pre-commit` section
2021-08-02 10:48:22 -07:00
darkverbito
98f00ef063
Fixed an issue on initial creation of a coloured coin where code always falls into default else clause due to lack of type conversion (#7851) 2021-07-28 22:04:56 -07:00
Almog De Paz
6a0c6a3b63
fix err type in wallet_blockchain.py (#7527) 2021-07-15 07:38:38 -07:00
Matt Hauff
7244df3fe7
Lightweight spend simulator (#7256)
* added a new lightweight simulator and updated a single test to use it

* Implemented full node rpc methods on the node (untested)

* moved locations and separated out the node client

* Lint fix

* Added barebones testing

* python black fixes

* Moved to RAM CoinStore

* Renamed Node to SpendSim and NodeClient to SimClient

* CoinSolution is CoinSpend now
2021-07-13 13:16:18 -07:00
Richard Kiss
6cf29102f9
Rename coin solution to coin spend (#6841)
* Rename `CoinSolution` as `CoinSpend` everywhere.

* Run `black` everywhere.
2021-07-12 16:31:27 -07:00
Almog De Paz
d80d5033fd
move peak look to _start (#7270) 2021-07-09 09:30:50 -07:00
Matt Hauff
7d55c39ab1 Fix load.clvm 2021-07-08 14:12:51 -07:00
Mariano
b1cd26cf5b Fix DB invalid format for additional_coin_spends 2021-07-04 14:48:55 +09:00
Adam Kelly
89f7a4b3d6
Pools mainnet (#7047)
* added clarifying comments

* WIP test

* added WIP test

* Refine genesis challenge. Remove unnecessary pool_puzzle function

* Sign spend. Remove create_member_spend. Rename state transition function to create_travel_spend

* Rename create_member_spend to create_travel_spend

* Add singleton id logging

* Enhance logging for debugging

* renaming

* rephrase inside the puzzle

* fixed signing and added some support functions

* Fix issue with announcement

* Progress spending the singleton

* Fix arguments to pool_state_to_inner_puzzle call

* Fix arguments to pool_state_to_inner_puzzle

* Improve error message when wallet is not running

* Remove misleading message about missing wallet process, when problem is the farmer by making poolnft command error out earlier

* Fix parent coin info bug

* Multiple state transitions in one block

* Lint

* Remove assert

* Fix incorrect p2_singleton_ph calculation (thanks nil00)

* Update waiting room puzzle to accept genesis_challenge

* Update calls to create_waiting

* Go to waiting state from committed state

* Augment debug_spend_bundle

* fix 2 bugs in wallet

* Fix lint

* fix bad_agg_sig bug

* Tests and lint

* remove breakpoint

* fix clvm tests for new hexs and hashs

* Fixed a bug in the coin store that was probably from merging. (#6577)

* Fixed a bug in the coin store that was probably from merging.

* The exception doesn't need to be there

* CI Lint fix

* Added lifecycle tests for pooling drivers (#6610)

* Ms.poolabsorb (#6615)

* Support for absorbing rewards in pools (untested)

* Style improvements

* More work on absorb

* Revert default root and remove log

* Revert small plots

* Use real sub slot iters

* Update types

* debug1

* Fix bugs

* fix output of agg sig log messages

* Make fewer calls to pw_status in test

* remove old comment

* logging and state management

* logging

* small bug fix & rename for accuracy

* format

* Fix types for uncurry function

* lint

* Update test to use exceptions

* Change assumptions about self-pooling in lifecycle test

* Install types for mypy

* Revert "Install types for mypy"

This reverts commit a82dcb712a.

* install types for mypy

* install types for mypy

* More keys

* Remove flags requiring interactive prompts

* Change initial spend to waiting room if self-pooling

* lint

* lint

* linting

* Refactor test

* Use correct value in log message

* update p2_singleton_or_delated_puzhash

* initial version of pool wallet with p2_singleton_or_delay

* run black formatting

* fix rebase wonkiness

* fix announcement code in p2_singleton_or_delayed

* removed redundant defaulting
standardised hexstr handling

* lint fixes

* Fixed pool lifecycle tests to current standards, but discovered tests are not validating signatures

* Signatures validate on this test now although the test still does not check it.

* Lint fix

* Fixed plotnft show and linting errors

* fixed failing farmer/harvester rpc test

* lint fix

* Commenting out some outdated tests

* Updated test coverage

* lint fix

* Some minor P2singleton improvements (#6325)

* Improve some debugging tools.

* Tidy pool clvm.

* Use `SINGLETON_STRUCT`. Remove unused `and` macro.

* Use better name `SINGLETON_MOD_HASH`.

* Finish lifecycle test suite.

* Fixing for merge with chia-blockchain/pools_delayed_puzzle (#72)

Co-authored-by: Matt Hauff <quexington@gmail.com>

* Default delay time was being set incorrectly

* Extracted get_delayed_puz_info_from_launcher_spend to driver code

* Ms.taproot plot2 (#6692)

* Start work on adding taproot to new plots

* Fix issue in block_tools

* new test-cache

* Lint

* DID fixexs

* Fix other tests

* Python black

* Fix full node store test

* Ensure block index <= 128 bits.

* fix test_pool_config test

* fix comments in pool_config and in chialisp files

* self_pool -> pool -> self_pool

* Implement leaving pools

* Fix conflicts with main via mini-rebase

* Fixup rebase mistakes

* Bring in Mariano's node discovery chagnes from pools.dev

* Fix adapters - Thanks Richard

* build tests

* Add pools tests

* Disable DID tests

* farmer|protocol: Some renaming related to the pool protocol

* farmer: Use `None` instead of `{}` and add local `pool_state`

* protocol|farmer: Introduce and use `PoolErrorCode`

* rename: `pool_payout_instructions` -> `payout_instructions`

* refactor: `AuthenticationKeyInfo` -> `authentication_key`

* refactor: Move `launcher_id` up

* rename: Some variable name changes

* rename: `points_balance` -> `points`

* format: Squash aggregation into one line

* farmer: Make `update_pool_state` public

* farmer: Print traceback if `update_pool_state` fails

* farmer: Periodically call `GET /pool_info`, add `_pool_get_pool_info`

* farmer: Add `authentication_token_timeout` to `pool_state`

Fetch it from `GET /pool_info`

* protocol|farmer: Implement support for `GET|POST|PUT /farmer`

* farmer: Make use of `GET|POST /farmer`

- To make the farmer known by the pool
- To update local balance/difficulty from the pool periodically

* farmer|protocol: Adjust `POST /partial` to match the latest spec

* farmer: Hash messages before signing

* pools: Drop unused code

* farmer: Fix aggregation of partial signatures

* farmer: support self pooling, don't pool if url==""

* wallet: return uint64 for delay time, instead of bytes

* pool: add error code for delay time too short

* farmer: cleaner logging when no connection to pool

* farmer: add harvester node id to pool protocol

* Rename method (test fix) and lint fix

* Change errors to warnings (pool communication)

* Remove pool callbacks on a reorg

* farmer: Continue earlier when no pool URL is provided

* farmer: Print method in log

* farmer: Handle exceptions for all pool endpoint calls

* farmer|protocol: Keep track of failed requests to the pool

* farmer: Fix typo which caused issue with pooling

* wallet: simplify solution_to_extra_data

* tests: Comment out DID tests which are not working yet

* Remove DID Wallet test workflows

* Return launcher_id when creating Pool Wallet

* Name p2_singleton_puzzle_hash correctly

* Improve 'test_singleton_lifecycle_fast.py'.

* Make test more robust in the face of asynchronous adversity

* Add commandline cmds for joining and leaving pools

* Fix poolnft leave params

* Remove redundant assignment brought in from main

* Remove unneeded code

* Style and parsimony

* pool_puzzles: Check was wrong, and bad naming

* format: Fix linting

* format: Remove log and rename variable

* pool-wallet: Fix self pooling with multiple pool wallets. Don't remove interested puzzle_hash

* gui: Use pools branch

* format: fix lint

* Remove ununsed code, improve initial_pool_state_from_dict

* farmer: Instantly update the config, when config file changes

* format: Speed up loading of the authentication key

* logging: less annoying logging

* Test pool NFT creation directly to pool

* Test switching pools without self-farming in between

* lint

* pooling: Use integer for protocol version (#6797)

* pooling: Use integer for protocol version

* pooling: Fix import

* Update GUI commit

* Ms.login2 (#6804)

* pooling: Login WIP

* pooling: add RPC for get_link

* dont use timeout

* pooling: rename to get_login_link

* format: remove logging

* Fix SES test

* Required cli argument

Co-authored-by: almog <almogdepaz@gmail.com>

* farmer|protocols: Rename `current_difficulty` for `POST /partial` (#6807)

* Fix to farm summary

* Use target_puzzlehash param name in RPC call

* Pool test coverage (#6782)

* Improvement in test coverage and typing

* Added an extra absorb to the pool lifecycle test (only works when merged with https://github.com/Chia-Network/chia-blockchain/pull/6733)

* Added new drivers for the p2_singleton puzzles

* Added new tests and test coverage for singletons

* organize pools testing directory

* black formatting

* black formatting in venv

* lint fix

* Update CI tests

* Fixing tests post rebase

* lint fix

* Minor readability fix

Co-authored-by: matt <matt@chia.net>

* farmer: Drop `target_puzzle_hash` from `GET /farmer` and `GET /login` (#6816)

* Allow creation of PlotNFTs in self-farming state

* gui: Fix install with more RAM (#6821)

* Allow implicit payout_address in self-pool state, improve error messages and param ergonomics

* print units in non-standard wallets correctly

* Fix farmer import

* Make syncing message in CLI more intuitive like the GUI

* Fix linting and show header hash instead of height

* gui: Update to 725071236eff8c81d5b267dc8eb69d7e03f3df8c

* Revert "Merge"

This reverts commit 23a1e688c5, reversing
changes made to a850246c6f.

* Revert "Revert "Merge""

This reverts commit 680331859f.

* Treat tx_record as Dict. Refactor tx submission

* Also add passed-in coin spends when processing new blocks in reconsider_peak

* Test utilities had moved

* Fix import of moved block_tools

* Potentially fix yaml

* Previously didn't take the right part of this change

* Add -y flag, improve commandline plotnft handling

* Fix typo

* Add -y flag to plotnft create

* pool_wallet: Restore from DB properly

* wallet: ignore bad pool configs

* Reduce memory

* pool_wallet: Add claim command

* pool_wallet: Set transaction records to confirmed

* wallet: Fix bug in transaction cache

* Formatting and remove log

* pool_wallet: CLI balance and improvements to plotnft_funcs.py

* pool_wallet: Simplify, and fix issue with double submission

* pool_wallet: Fix tests

* pool_wallet: Don't allow switching before relative lock height

* update gui

* change to 3000 mem

* Correct sense of -y flag for self-pooling

* cli: Display payout instructions for pool

* pool_wallet: Don't create massive transactions

* cli: Improvements to plotnft

* pool_wallet: Get correct pool state

* pool_wallet: Use last transaction block to prevent condition failure

* Add block height for current state

* Add outstanding unconfirmed transactions to pw_status

* Refine command line plotnft show pending transactions

* Fix tests by using the correct output from pw_status

* Try to fix windows build

* Print expected leave height

* label pool urls

* pool_wallet: Don't include pool 1.75 rewards in total

* wallet: Add RPC and CLI for deleting unconfirmed transactions for a wallet

* pool_wallet: If farming to a pool, show 0 balance in wallet

* pool_wallet: Show error message if invalid state, in CLI

* pool_wallet: Don't allow switching if there are pending unconfirmed transactions

* tests: Clean up pool test logging

* tests: Fix lint

* Changed the pool innerpuzzes (#6802)

* overload solutions for pool_innerpuz parameters

* Fix tests for reduced size puzzles

* deleted messy deprecated test

* Fix lint.

* fix bug where spend types were the wrong way around

* merge with richard's lint fix

* fix wallet bug
remove unnecessary signature
add defun-inline for clarity

* Swap to defun for absorb case
Use cons box for member innerpuz solution

* fix if statement for cons box p1

* remove unnecessary solution arg

* quick innerpuz fix to make tests pass

* Switch to key-value pairs
Undo cons box solution in pool_member inner puzzle

* fix singleton lifecycle test

* added some comments to calrify the meaning on "ps"

* lint fix

* reduce label size, search for label when reconstructing from solution

* no need to keep looping if `p` found

* lint fix

* Removed unecessary defun-inline and changed hyphens to underscores

* Changed created_coin_value_or_0 to an inline function

* Changed morph_condition to an inline function

* Added a comment for odd_cons_m113

* Rename output_odd and odd_output_found

* Add inline functions to document the lineage proof values

* Stager two rewrite

* Added an ASSER_MY_AMOUNT to p2_singleton_or_delayed

* Extract truth functionality to singleton_truths.clib

* Fix tree hashes

* Changed truths to a struct rather than a list.

* fix test_singletons
update did_innerpuz

* recompile did_innerpuz

* fix a log error

* Renamed variable and factored out code per @richardkiss

* lint fix

* switch launcher extra_data to key_value pairs

* fix parsing of new format of extra_data in launcher solution

* fix broken test for new launcher solution format

* remove bare raise

Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>

* Also add passed-in coin spends when processing new blocks in reconsider_peak (#6898)

Co-authored-by: Adam Kelly <aqk>

* Moved debug_spend_bundle and added it to the SpendBundle object (#6840)

* Moved debug_spend_bundle and added it to the SpendBundle object

* Remove problematic typing

* Add testnet config

* wallet: Memory would get corrupted if there was an error (#6902)

* wallet: Memory would get corrupted if there was an error

* wallet: Use block_record

* wallet: Add records in a full fork too

* wallet: remove unnecessary arguments in CC and DID

* add to cache, revert if transaction fails

Co-authored-by: Yostra <straya@chia.net>

* Improve comment

* pool_wallet: Fix driver bug

* wallet: Fix memory corruption

* gui: Update to latest

* Increase memory size

* tests: Add test for absorbing from pool

* small fix in solution_to_extra_data

* Fixed incorrect function name

* pooling: Fix EOS handling in full node

* [pools.testnet9]add post /partial and /farmer header (#6957)

* Update farmer.py

add post header

* Update farmer_api.py

add post header

* Update chia/farmer/farmer.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/farmer/farmer_api.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Fix lint and cleanup farmer.py

* farmer: Fix linting issues (#7010)

* Handle the case of incorrectly formatted PoolState data returned from inner singleton

* wallet: Resubmit transaction if not successful, rename to new_transaction_block_callback (#7008)

* Fix lint in pool_puzzles

* pooling: Fix owner private key lookup, and remove unnecessary argument

* pooling: Clear target state on `delete_unconfirmed_transactions`

* Lint

* Fix non-deterministic test

* Slight cleanup clvm driver code (#7028)

* Return None when a deserialized CLVM structure does not fit the expected format of var-value pair for singleton data

* lint

Co-authored-by: Adam Kelly <aqk>

* Revert "Add testnet config"

This reverts commit 9812442724.

Co-authored-by: matt <matt@chia.net>
Co-authored-by: Adam Kelly <aqk@aqk.im>
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Adam <aqk@Adams-MacBook-Pro.local>
Co-authored-by: Adam Kelly <aqk>
Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: arvidn <arvid@libtorrent.org>
Co-authored-by: willi123yao <willi123yao@gmail.com>
Co-authored-by: arty <art.yerkes@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
Co-authored-by: matt-o-how <48453825+matt-o-how@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: DouCrazy <43004977+lpf763827726@users.noreply.github.com>
2021-06-29 14:21:25 -07:00
Almog De Paz
e5eff54149
wallet short batch sync (#6112)
* short batch sync

* switch log for fixing ci

* switch log level back

* increase test time

* new line

* test fix

* change sem to lock

* change sem to lock

* refactor

* move weight check under lock

* short sync tests

* lint

* remove duplicate code

* remove duplicate code

* add await

* fix
2021-06-29 11:25:54 -07:00
Almog De Paz
748f7825ad
use is_trusted_peer (#6830)
* use is_trusted_peer

* fix indentation

* lint

* dont assume localhost is trusted
2021-06-29 10:33:10 -07:00
Arvid Norberg
0368544c84
validate more aspects of conditions earlier (#6654)
* validate more aspects of conditions earlier, in the process clvm execution is offloaded to

* more tests

* use as_int() in SExp rather than int_from_bytes()

* put condition parser tests in its own test class

* capitalize module-scope variable

* abbreviate enum names in parse_condition_args

* use a more realistic cost per byte in test_rom

* merge parse_coin_id and parse_hash, since they are identical
2021-06-23 09:49:23 -07:00
Nikolaj Kuntner
07f64d2190
Less log getter setup calls (#6123)
* init

* remove whitespace

* docs

* rem double colon

* more type annotation

* retrigger timed out test

Co-authored-by: wjblanke
2021-06-14 09:13:41 -07:00
Florin Chirica
2335beff15
Fix UI refreshes. (#6639)
* Add event for new peak.

* Refresh connections for farmer and wallet.

* Display syncing x/x instead of x/0.

* Refresh UI connections tasks.

* Preventive catch exception inside new_peak related to UI tasks.

* Add back black line.

* Try to optimize refreshes.

* Limit refreshes.

* Try 1s sleep.

* Typo in the comment.

* Try 1.5s
2021-06-12 14:45:23 -07:00
Earle Lowe
ae8497b5ce
Delay peer start for wallet until after backup init (#6671)
* Delay peer start for wallet until after backup init

* Make sure to only start peers once
2021-06-11 14:30:07 -07:00
Arvid Norberg
ca3c8ce407
validate integers in StructStream constructors (#4298)
* validate integers in StructStream constructors

* don't step back sub slots beyond zero

* fix off-by-one in blocks-to-remove

* handle find_fork_point_in_chain() returning -1 in wallet

* fix handling of not finding a shared ancestor in wallet reorg

* terminate clean_block_record() when we reach height 0

* harden int/uint validation in constructor. add comment about int512
2021-06-08 10:45:56 -07:00
Florin Chirica
06a23d7769
Add full node ports for testnet dns. (#6369)
* Add full node ports for testnet dns.

* Lint.
2021-06-08 09:45:53 -07:00
Florin Chirica
59d415fd71
Rename testnet peer dbs. (#6353)
* Rename testnet peer dbs.

* Add selected_network to all constructors.
2021-06-02 15:19:58 -07:00
Benedict Jin
8fcbcd9f22
Remove useless pass in wallet node (#6313) 2021-06-01 22:09:12 -07:00