* replace blspy imports with chia_rs imports for BLS types
* remove blspy-stubs, since we're dropping the blspy dependency. chia_rs has type stubs already
* Limit rate of spammy log messages when farmer is disconnected from pool
* Remove typo
* Use reliably increasing clock
* Make test faster, more reliable
* configure isort to add the future annotations import
* apply the new isort setting
* remove type ignores for new mypy (#13539)
https://pypi.org/project/mypy/0.981/
* another
* provide a skeleton hint protocol for rpc apis
* more
* more self.custom_get_connections = None
* fix RpcApiProtocol.service hinting
* catch up with RpcApiProtocol._state_changed()
* FullNode.server is now non-optional. 'ish
* drop another todo
* custom_get_connections -> get_connections
* just make get_connections() a mandatory method with a helper provided
* lots
* try again
* ._start_with_fingerprint()
* fill out the Service generic parameter in start scripts
* ... instead of pass
* add docstrings to the protocols
* remove no longer needed hasattr/not None checks
* remove outdated comment
* refactor Harvester._is_shutdown to ._shut_down
* farmer: Adjust notifications from the farmer to the UI
- Only send the data for the harvester which actually sent an update
- Notify for each loaded batch during initial loading
* Enable improved farmer/harvester GUI
* farmer|rpc|tests: Implement paginated harvester plot endpoints
* Simplify filtering
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Let the API handle the exceptions
* Simplify the other filtering too
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Simplify count assertions
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Refactor `is_filter_match` to `plot_matches_filter`
And just convert to `Plot` in tests.
* Move `chia.util.misc.KeyValue` to `chia.rpc.farmer_rpc_api.FilterItem`
* Rename `peer_id` to `node_id` to be match `get_harvesters_{summary}`
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* protocols|server: Define new harvester plot refreshing protocol messages
* protocols: Bump `protocol_version` to `0.0.34`
* tests: Introduce `setup_farmer_multi_harvester`
Allows to run a test setup with 1 farmer and mutiple harvesters.
* plotting: Add an initial plot loading indication to `PlotManager`
* plotting|tests: Don't add removed duplicates to `total_result.removed`
`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.
* harvester: Reset `PlotManager` on shutdown
* plot_sync: Implement plot sync protocol
* farmer|harvester: Integrate and enable plot sync
* tests: Implement tests for the plot sync protocol
* farmer|tests: Drop obsolete harvester caching code
* setup: Add `chia.plot_sync` to packages
* plot_sync: Type hints in `DeltaType`
* plot_sync: Drop parameters in `super()` calls
* plot_sync: Introduce `send_response` helper in `Receiver._process`
* plot_sync: Add some parentheses
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* plot_sync: Additional hint for a `Receiver.process_path_list` parameter
* plot_sync: Force named parameters in `Receiver.process_path_list`
* test: Fix fixtures after rebase
* tests: Fix sorting after rebase
* tests: Return type hint for `plot_sync_setup`
* tests: Rename `WSChiaConnection` and move it in the outer scope
* tests|plot_sync: More type hints
* tests: Rework some delta tests
* tests: Drop a `range` and iterate over the list directly
* tests: Use the proper flags to overwrite
* test: More missing duplicates tests
* tests: Drop `ExpectedResult.reset`
* tests: Reduce some asserts
* tests: Add messages to some `assert False` statements
* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`
* tests: Use `secrects` instead of `Crypto.Random`
* Fixes after rebase
* Import from `typing_extensions` to support python 3.7
* Drop task name to support python 3.7
* Introduce `Sender.syncing`, `Sender.connected` and a log about the task
* Add `tests/plot_sync/config.py`
* Align the multi harvester fixture with what we do in other places
* Update the workflows
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Set keychain_proxy to None in await_closed() to support reinitialization.
* Added `shutting_down` param to _await_closed() to control whether the keychain_proxy is closed.
* minor lock scope reduction
* use the lock in tests
* Use the passed root_path in configure CLI command
* switch to lock_and_load_config()
* oops
* cleanup
* make _load_config_maybe_locked() private
* black
* Remove future improvement opportunity TODO comment
* Add configuration locking
Extracted from https://github.com/Chia-Network/chia-blockchain/pull/10631
* note that fasteners will likely be replaced by filelock
* Fix test_multiple_writers on macOS
* create_all_ssl() doesn't need to be inside the config access lock
* add warnings about not using async within get_config_lock() get lock contexts
* no need to pre-touch the lock file
* .yaml.lock instead of just .lock
* test_multiple_writers() is sync
* Revert "add warnings about not using async within get_config_lock() get lock contexts"
This reverts commit 681af3835b.
* reduce lock context size in chia_init()
* use an exit stack in load_config()
* avoid config existence precheck
* only lock around the read in load_config()
* do not raise e, just raise
* tidy new imports
* fix queue empty check in test_config.py
* remove commented out code in test_config.py
* remove unused import
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
* farmer: Bump next update times regardless of the request results
* farmer: Drop additional "success/failure" log logic
We already print the PUT response in `_pool_put_farmer` the other parts
just lead to confusion if the pool didn't implement the PUT correct.
* farmer: Print error responses from the pool with `WARNING` log level
Due to the bug which was fixed in #9922 some pools obviously still have
incorrect authentication public keys from their farmers. This PR is to
make sure that the farmers update their authentication public key on the
pool if they get an `INVALID_SIGNATURE` response error from a `GET
/farmer`.
* pool_wallet: Deterministic derivation of auth key, and only count poolWallets for indeces
* cli: Use the same limit for cli creation
* farmer: no more auth keys in config
* farmer: handle case of no authentication sk
* farmer: remove async, and fix test
* lint
* test: Add a test for many pool wallets, and user store
* test: Wait for wallet to be synced in test
* wallet: fix startup (no dns servers) and change to server checking of plotnfts
* cli: change to server checking
* tests: fix user store test
* Improve caching
* PR comments
* Remove useless class
* Try to fix test again
* farmer: Run a start background task to wait if there are no keys yet
* farmer: Run the handshake in a background task if there are no keys yet
* tests: Yield the service in farmer/harvester setups
* tests: Introduce `start_service` in farmer/harvester setups
* server: Make `ChiaServer` and `Service` restartable
* tests: Test the delayed start/handshake in `test_farmer_harvester.py`
* tests: Move `test_farmer_harvester.py` to `tests/core/farmer_harvester`
Attempt to avoid issues due to the global block tools / shared keychain
between tests
* farmer: Return `bool` in `Farmer.setup_keys`
* tests: Add `__init__.py` to `tests/farmer_harvester`
* farmer|tests: Update cache more frequent for `test_get_harvesters`
* tests: Increase interval between harvester checks
To avoid spamming logs by excessive plot refreshing and cache updates.
* tests: Log error messages if `get_harvesters` returns unexpected data
* tests: Trigger plot refresh on the harvester in `test_get_harvesters`
* tests: Don't assert in `test_refresh_callback`
This lead to a crash of the plot refreshing thread and with this it
silently failed. With this change its at least possible to see which
value is incorrect.
* Create directories with 755 permissions, SSL certs with 644, and keys with 600.
* Check SSL file permissions during chia_init(). Exits if permissions are incorrect.
* Overwrite certs/keys instead of failing to write
* Skip SSL file permission checks on Windows (requires checking ACLs)
* Check SSL file permissions when creating an ssl_context
* Skip check_ssl on Windows.
Handle some SSLInvalidPermissions exceptions.
* Added a few comments
* Added chia init --fix-ssl-permissions option to attempt to fix SSL file permission issues.
Update imported cert permissions when using chia init -c.
Code cleanup/restructuring.
* Return a tuple instead of a list
* LGTM and other minor fixes
* Fixed SSL test breakage when calling ssl_context_for_client. The ca_crt param was being passed in as both the cert and private key, triggering the permission check failure.
* Don't exit if SSL file permissions issues are found
* Tweak the exception types that are raised from traverse_dict
* Moved keyring handling into a KeyringWrapper class
* Update click to 8.0.x for prompt_required support
* Renamed KeyringWrapper to _KeyringWrapper
* Expose password management options on Linux
* CLI support for setting/removing a password
* Global option for specifying the master password
* Cache the password instead of setting on the context
* Password bootstrapping during chia init
* Tidying up _KeyringWraper's interface
* Initial pass migrating the legacy keyring contents
* Encryption/decryption of keyring.yaml contents
* FileKeyring backend encrypts with ChaCha20Poly1305
* Tightened up keyring migration and initialization
* Fixed issues identified by linters
* Remove root_path from Keychain
* Prevent double-migration if setting master passwd
* KeyringWrapper tests are mostly complete
* FileKeyring will now honor the service param
* Tests for get/set/delete password
* Formatting/commenting updates
* Writer lock support with tests - WIP
* keyring.yaml is now watched for modifications
* Reader/Writer lock for get/delete password
* Fixed linter issues
* Reader lock tests
* Formatting update
* Hook up CHIA_ROOT support for KeychainWrapper
* Quick fix to address test failures
* Fixed failures when existing legacy keyring exists
* Fixed test failures caused by reusing the same temp dir
* keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path
* Fixed migration failure when setting a password (not using the default)
* KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False
* The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password.
* Added is_keyring_locked RPC call
* Added 'unlock_keyring' RPC command
* Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class.
* Turn on macOS support for testing keyring passwords
* Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain.
Fixed key reconstruction when sent over RPC.
* Farmer now accesses the keychain over RPC
* Fixes for linter issues and some restructuring to support tests that use setup_nodes.py
* Couple of fixes to unblock the GUI from launching when a keyring password is set
* Added a keychain RPC call for add_private_key()
* Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys
* Check for None when inspecting request arguments
* Run check_keys after unlocking the keyring when the daemon is launched via GUI
* Added check_keys RPC method.
Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details.
* Added get_first_private_key to keychain_server/proxy.
create_plots now uses the keychain proxy when launched from the daemon.
* Added a comment about KeychainProxy in chia plots check
* Workaround import conflict when importing from 'tests.*' due to fasteners name conflict
* Simulator now uses KeychainProxy if launched by the daemon.
KeychainServer/Proxy now takes keychain user/testing params for testing scenarios.
* Added "set_keyring_passphrase" RPC message
* Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests.
* Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete()
* Silencing file_keyring logging for the moment.
* Updated tests to use test keychains and appropriate BlockTools construction
BlockTools should now be created with create_block_tools(_async) to handle async scenarios.
Updated block_tools to be async compatible
Updated fasteners to fix installation of top-level 'tests' in site-packages
* Added 'remove_keyring_passphrase' RPC message to the daemon
Minor tweak to TempKeyring to default to some test params
* Fixed linter issues
* Remove flake8 ignore statement now that the fasteners module has been updated
* Some initial renaming changes: password -> passphrase
* Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon.
Fixed legacy keyring initialization (for migration scenarios)
* Fixed improperly merged file
* Fixed linter issues.
More renaming.
* Updated spots that were still using an incorrect keychain call
* Renamed use_password_cache, obtain_current_password
* Renamed supports_keyring_password
* Renamed has_master_password
* Renamed has_cached_password, get_cached_master_password
* Linter fixes
* Renamed master_password_is_valid
* Renamed set_cached_master_password
* Renamed set_master_password
* Renamed remove_master_password
* Renamed has_cached_master_password
* Renaming in file_keyring and keyring_wrapper
Updated default keyring payload used for tests
* Renamed get_password
Other renaming updates
* Renamed set_password
Other renaming updates
* Renamed remaining password occurrences (where appropriate)
* password -> passphrase
* Added tests for setting an emoji and Japanese master passphrase
* Attempt to notify the daemon when a keyring passphrase is set/updated/removed
* Missed one password -> passphrase replacement.
* Fixed some file synchronization issues found when running tests on macOS
* Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS.
* Removed logging statements previously added for debugging
* Prompt for keyring passphrase up-front when launching a service.
Changed --have-gui flag to --wait-for-unlock
* Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase.
* Minor test cleanup to deduplicate some code.
* Fixed regression when setting a new master passphrase
* Minor refactoring and docs/commenting updates
* Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends.
* Disabling macOS support (previously added for testing only)
* Disabling passphrase support in preparation for sending out the PR
* Fixed improper merge (vscode didn't save changes during rebase)
* Update chia/cmds/init_funcs.py
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
* skip_check_keys -> should_check_keys
* Shuffling some imports around to break cycles reported by LGTM
* Handle unlocking the daemon if it's already launched and waiting for unlock.
* Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets.
Linter/reformatting fixes
* Replaced uses_keychain_proxy decorator with a clearer method.
* Cleanup the temp keyring dir using shutil.rmtree()
* Restored self._root_path (had been changed to self.root_path)
* Minor cleanup
* ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails
* Plot key resolution now yields a PlotKeys object which can be passed into create_plots.
De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy.
* Added 'keyring_status' daemon RPC message to support the GUI
* Minor changes relating to PR feedback
* Addressed more PR feedback (mostly type annotations)
* Commented-out macOS file keyring usage. This can be re-enabled for testing purposes.
* Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance.
* Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response.
* Updated type annotations and method signature for set_passphrase to expect a string instead of bytes.
* Fixed Wallet RPC tests
* Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s)
* Fixed test failures in test_pool_rpc
* Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories.
* Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data.
* Second attempt at suppressing LGTM false positives
* Third attempt at addressing LGTM false positives
* Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly.
* Reformatting
* More targeted update of the test config to refresh just the "plot_directories" value
* More LGTM suppressions
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: wjblanke <wjb98672@gmail.com>
This is to also allow hex strings prefixed with `0x` here which is
required because we prefix the strings in JSON conversion here
4fbc291969/chia/util/json_util.py (L20-L23)
