chia-blockchain/chia/cmds/plots.py

224 lines
7.2 KiB
Python
Raw Normal View History

from __future__ import annotations
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
import asyncio
import logging
import sys
2020-07-03 07:14:10 +03:00
from pathlib import Path
import click
from chia.plotting.util import add_plot_directory, validate_plot_size
2021-03-08 21:50:16 +03:00
DEFAULT_STRIPE_SIZE = 65536
2020-07-03 07:14:10 +03:00
log = logging.getLogger(__name__)
def show_plots(root_path: Path):
chia|tests: Introduce `PlotManager` + some plot loading improvements and fixes (#7848) * harvester|plotting|tests: Introduce `PlotManager` class This moves all plots related and plot-directory related stuff from the harvester into the class `PlotManager`, adjusts all related code accordingly and adds some extra wrappers there. * harvester|plotting|tests: Return how many new plots were loaded * plotting: Fix `failed_to_open_filenames` re-try interval With `< 1200` it just tries it on the next refresh. * plotting: Fix and improve duplicates handling * harvester|plotting: Thread locks for `PlotManager.plots` * chia|tests: Load plots in batches * chia|tests: Move plot refreshing into a separate thread * plotting: Properly handle removed plots And fix tests accordingly. It seems like this fix https://github.com/Chia-Network/chia-blockchain/pull/3350 wasn't really a fix, rather adjusting to allow for a bug? * plotting|harvester|tests: Introduce `PlotRefreshResult` * tests: Expand `test_farmer_harvester_rpc.py` * chia|tests: Move some stuff from `plot_tools.py` into new file `util.py` * refactor: Rename `plot_tools.py` to `manager.py` * chia|tests: Use pure dataclass for `PlotsRefreshParameter` With `uint16` as type saving to config doesn't work, this is a preparation for the next commit. * harvester: Adjust deprecation message, use `info` instead of `warning` * plotting: Fix typo * refactor: Rename `filename` to `file_path` Fits better and does avoid shadowing with filename from outer scope. * chia|tests: Move some methods from `plotting.manager` to `plotting.util` * plotting: Make `refresh_callback` mandatory
2021-08-09 20:25:15 +03:00
from chia.plotting.util import get_plot_directories
print("Directories where plots are being searched for:")
print("Note that subdirectories must be added manually")
print(
"Add with 'chia plots add -d [dir]' and remove with"
+ " 'chia plots remove -d [dir]'"
+ " Scan and check plots with 'chia plots check'"
)
print()
for str_path in get_plot_directories(root_path):
print(f"{str_path}")
@click.group("plots", short_help="Manage your plots")
@click.pass_context
def plots_cmd(ctx: click.Context):
"""Create, add, remove and check your plots"""
2021-03-22 22:36:05 +03:00
from chia.util.chia_logging import initialize_logging
root_path: Path = ctx.obj["root_path"]
2020-07-03 07:14:10 +03:00
if not root_path.is_dir():
raise RuntimeError("Please initialize (or migrate) your config directory with 'chia init'")
2020-07-03 08:19:09 +03:00
initialize_logging("", {"log_stdout": True}, root_path)
@plots_cmd.command("create", short_help="Create plots")
@click.option("-k", "--size", help="Plot size", type=int, default=32, show_default=True)
@click.option("--override-k", help="Force size smaller than 32", default=False, show_default=True, is_flag=True)
@click.option("-n", "--num", help="Number of plots or challenges", type=int, default=1, show_default=True)
@click.option("-b", "--buffer", help="Megabytes for sort/plot buffer", type=int, default=3389, show_default=True)
@click.option("-r", "--num_threads", help="Number of threads to use", type=int, default=2, show_default=True)
@click.option("-u", "--buckets", help="Number of buckets", type=int, default=128, show_default=True)
@click.option(
"-a",
"--alt_fingerprint",
type=int,
default=None,
help="Enter the alternative fingerprint of the key you want to use",
)
@click.option(
"-c",
"--pool_contract_address",
type=str,
default=None,
help="Address of where the pool reward will be sent to. Only used if alt_fingerprint and pool public key are None",
)
@click.option("-f", "--farmer_public_key", help="Hex farmer public key", type=str, default=None)
@click.option("-p", "--pool_public_key", help="Hex public key of pool", type=str, default=None)
@click.option(
"-t",
"--tmp_dir",
help="Temporary directory for plotting files",
type=click.Path(),
default=Path("."),
show_default=True,
)
@click.option("-2", "--tmp2_dir", help="Second temporary directory for plotting files", type=click.Path(), default=None)
@click.option(
"-d",
"--final_dir",
help="Final directory for plots (relative or absolute)",
type=click.Path(),
default=Path("."),
show_default=True,
)
@click.option("-i", "--plotid", help="PlotID in hex for reproducing plots (debugging only)", type=str, default=None)
@click.option("-m", "--memo", help="Memo in hex for reproducing plots (debugging only)", type=str, default=None)
@click.option("-e", "--nobitfield", help="Disable bitfield", default=False, is_flag=True)
@click.option(
"-x", "--exclude_final_dir", help="Skips adding [final dir] to harvester for farming", default=False, is_flag=True
)
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
@click.option(
"-D",
"--connect_to_daemon",
help="Connects to the daemon for keychain operations",
default=False,
is_flag=True,
hidden=True, # -D is only set when launched by the daemon
)
@click.pass_context
def create_cmd(
ctx: click.Context,
size: int,
override_k: bool,
num: int,
buffer: int,
num_threads: int,
buckets: int,
alt_fingerprint: int,
pool_contract_address: str,
farmer_public_key: str,
pool_public_key: str,
tmp_dir: str,
tmp2_dir: str,
final_dir: str,
plotid: str,
memo: str,
nobitfield: bool,
exclude_final_dir: bool,
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
connect_to_daemon: bool,
):
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
from chia.plotting.create_plots import create_plots, resolve_plot_keys
class Params(object):
def __init__(self):
self.size = size
self.num = num
self.buffer = buffer
self.num_threads = num_threads
self.buckets = buckets
2021-03-08 21:50:16 +03:00
self.stripe_size = DEFAULT_STRIPE_SIZE
self.tmp_dir = Path(tmp_dir)
self.tmp2_dir = Path(tmp2_dir) if tmp2_dir else None
self.final_dir = Path(final_dir)
self.plotid = plotid
self.memo = memo
self.nobitfield = nobitfield
root_path: Path = ctx.obj["root_path"]
try:
validate_plot_size(root_path, size, override_k)
except ValueError as e:
print(e)
sys.exit(1)
plot_keys = asyncio.run(
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
resolve_plot_keys(
farmer_public_key,
alt_fingerprint,
pool_public_key,
pool_contract_address,
root_path,
Keyring passphrase protection (#7249) * Moved keyring handling into a KeyringWrapper class * Update click to 8.0.x for prompt_required support * Renamed KeyringWrapper to _KeyringWrapper * Expose password management options on Linux * CLI support for setting/removing a password * Global option for specifying the master password * Cache the password instead of setting on the context * Password bootstrapping during chia init * Tidying up _KeyringWraper's interface * Initial pass migrating the legacy keyring contents * Encryption/decryption of keyring.yaml contents * FileKeyring backend encrypts with ChaCha20Poly1305 * Tightened up keyring migration and initialization * Fixed issues identified by linters * Remove root_path from Keychain * Prevent double-migration if setting master passwd * KeyringWrapper tests are mostly complete * FileKeyring will now honor the service param * Tests for get/set/delete password * Formatting/commenting updates * Writer lock support with tests - WIP * keyring.yaml is now watched for modifications * Reader/Writer lock for get/delete password * Fixed linter issues * Reader lock tests * Formatting update * Hook up CHIA_ROOT support for KeychainWrapper * Quick fix to address test failures * Fixed failures when existing legacy keyring exists * Fixed test failures caused by reusing the same temp dir * keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path * Fixed migration failure when setting a password (not using the default) * KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False * The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password. * Added is_keyring_locked RPC call * Added 'unlock_keyring' RPC command * Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class. * Turn on macOS support for testing keyring passwords * Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain. Fixed key reconstruction when sent over RPC. * Farmer now accesses the keychain over RPC * Fixes for linter issues and some restructuring to support tests that use setup_nodes.py * Couple of fixes to unblock the GUI from launching when a keyring password is set * Added a keychain RPC call for add_private_key() * Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys * Check for None when inspecting request arguments * Run check_keys after unlocking the keyring when the daemon is launched via GUI * Added check_keys RPC method. Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details. * Added get_first_private_key to keychain_server/proxy. create_plots now uses the keychain proxy when launched from the daemon. * Added a comment about KeychainProxy in chia plots check * Workaround import conflict when importing from 'tests.*' due to fasteners name conflict * Simulator now uses KeychainProxy if launched by the daemon. KeychainServer/Proxy now takes keychain user/testing params for testing scenarios. * Added "set_keyring_passphrase" RPC message * Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests. * Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete() * Silencing file_keyring logging for the moment. * Updated tests to use test keychains and appropriate BlockTools construction BlockTools should now be created with create_block_tools(_async) to handle async scenarios. Updated block_tools to be async compatible Updated fasteners to fix installation of top-level 'tests' in site-packages * Added 'remove_keyring_passphrase' RPC message to the daemon Minor tweak to TempKeyring to default to some test params * Fixed linter issues * Remove flake8 ignore statement now that the fasteners module has been updated * Some initial renaming changes: password -> passphrase * Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon. Fixed legacy keyring initialization (for migration scenarios) * Fixed improperly merged file * Fixed linter issues. More renaming. * Updated spots that were still using an incorrect keychain call * Renamed use_password_cache, obtain_current_password * Renamed supports_keyring_password * Renamed has_master_password * Renamed has_cached_password, get_cached_master_password * Linter fixes * Renamed master_password_is_valid * Renamed set_cached_master_password * Renamed set_master_password * Renamed remove_master_password * Renamed has_cached_master_password * Renaming in file_keyring and keyring_wrapper Updated default keyring payload used for tests * Renamed get_password Other renaming updates * Renamed set_password Other renaming updates * Renamed remaining password occurrences (where appropriate) * password -> passphrase * Added tests for setting an emoji and Japanese master passphrase * Attempt to notify the daemon when a keyring passphrase is set/updated/removed * Missed one password -> passphrase replacement. * Fixed some file synchronization issues found when running tests on macOS * Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS. * Removed logging statements previously added for debugging * Prompt for keyring passphrase up-front when launching a service. Changed --have-gui flag to --wait-for-unlock * Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase. * Minor test cleanup to deduplicate some code. * Fixed regression when setting a new master passphrase * Minor refactoring and docs/commenting updates * Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends. * Disabling macOS support (previously added for testing only) * Disabling passphrase support in preparation for sending out the PR * Fixed improper merge (vscode didn't save changes during rebase) * Update chia/cmds/init_funcs.py Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> * skip_check_keys -> should_check_keys * Shuffling some imports around to break cycles reported by LGTM * Handle unlocking the daemon if it's already launched and waiting for unlock. * Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets. Linter/reformatting fixes * Replaced uses_keychain_proxy decorator with a clearer method. * Cleanup the temp keyring dir using shutil.rmtree() * Restored self._root_path (had been changed to self.root_path) * Minor cleanup * ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails * Plot key resolution now yields a PlotKeys object which can be passed into create_plots. De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy. * Added 'keyring_status' daemon RPC message to support the GUI * Minor changes relating to PR feedback * Addressed more PR feedback (mostly type annotations) * Commented-out macOS file keyring usage. This can be re-enabled for testing purposes. * Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance. * Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response. * Updated type annotations and method signature for set_passphrase to expect a string instead of bytes. * Fixed Wallet RPC tests * Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s) * Fixed test failures in test_pool_rpc * Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories. * Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data. * Second attempt at suppressing LGTM false positives * Third attempt at addressing LGTM false positives * Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly. * Reformatting * More targeted update of the test config to refresh just the "plot_directories" value * More LGTM suppressions Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com> Co-authored-by: wjblanke <wjb98672@gmail.com>
2021-08-04 22:46:55 +03:00
log,
connect_to_daemon,
)
)
asyncio.run(create_plots(Params(), plot_keys))
if not exclude_final_dir:
try:
add_plot_directory(root_path, final_dir)
except ValueError as e:
print(e)
@plots_cmd.command("check", short_help="Checks plots")
@click.option("-n", "--num", help="Number of plots or challenges", type=int, default=None)
@click.option(
"-g",
"--grep_string",
help="Shows only plots that contain the string in the filename or directory name",
type=str,
default=None,
)
@click.option("-l", "--list_duplicates", help="List plots with duplicate IDs", default=False, is_flag=True)
@click.option("--debug-show-memo", help="Shows memo to recreate the same exact plot", default=False, is_flag=True)
@click.option("--challenge-start", help="Begins at a different [start] for -n [challenges]", type=int, default=None)
@click.pass_context
def check_cmd(
ctx: click.Context, num: int, grep_string: str, list_duplicates: bool, debug_show_memo: bool, challenge_start: int
):
2021-03-22 22:36:05 +03:00
from chia.plotting.check_plots import check_plots
check_plots(ctx.obj["root_path"], num, challenge_start, grep_string, list_duplicates, debug_show_memo)
@plots_cmd.command("add", short_help="Adds a directory of plots")
@click.option(
"-d",
"--final_dir",
help="Final directory for plots (relative or absolute)",
type=click.Path(),
default=".",
show_default=True,
)
@click.pass_context
def add_cmd(ctx: click.Context, final_dir: str):
chia|tests: Introduce `PlotManager` + some plot loading improvements and fixes (#7848) * harvester|plotting|tests: Introduce `PlotManager` class This moves all plots related and plot-directory related stuff from the harvester into the class `PlotManager`, adjusts all related code accordingly and adds some extra wrappers there. * harvester|plotting|tests: Return how many new plots were loaded * plotting: Fix `failed_to_open_filenames` re-try interval With `< 1200` it just tries it on the next refresh. * plotting: Fix and improve duplicates handling * harvester|plotting: Thread locks for `PlotManager.plots` * chia|tests: Load plots in batches * chia|tests: Move plot refreshing into a separate thread * plotting: Properly handle removed plots And fix tests accordingly. It seems like this fix https://github.com/Chia-Network/chia-blockchain/pull/3350 wasn't really a fix, rather adjusting to allow for a bug? * plotting|harvester|tests: Introduce `PlotRefreshResult` * tests: Expand `test_farmer_harvester_rpc.py` * chia|tests: Move some stuff from `plot_tools.py` into new file `util.py` * refactor: Rename `plot_tools.py` to `manager.py` * chia|tests: Use pure dataclass for `PlotsRefreshParameter` With `uint16` as type saving to config doesn't work, this is a preparation for the next commit. * harvester: Adjust deprecation message, use `info` instead of `warning` * plotting: Fix typo * refactor: Rename `filename` to `file_path` Fits better and does avoid shadowing with filename from outer scope. * chia|tests: Move some methods from `plotting.manager` to `plotting.util` * plotting: Make `refresh_callback` mandatory
2021-08-09 20:25:15 +03:00
from chia.plotting.util import add_plot_directory
try:
add_plot_directory(ctx.obj["root_path"], final_dir)
print(f"Successfully added: {final_dir}")
except ValueError as e:
print(e)
@plots_cmd.command("remove", short_help="Removes a directory of plots from config.yaml")
@click.option(
"-d",
"--final_dir",
help="Final directory for plots (relative or absolute)",
type=click.Path(),
default=".",
show_default=True,
)
@click.pass_context
def remove_cmd(ctx: click.Context, final_dir: str):
chia|tests: Introduce `PlotManager` + some plot loading improvements and fixes (#7848) * harvester|plotting|tests: Introduce `PlotManager` class This moves all plots related and plot-directory related stuff from the harvester into the class `PlotManager`, adjusts all related code accordingly and adds some extra wrappers there. * harvester|plotting|tests: Return how many new plots were loaded * plotting: Fix `failed_to_open_filenames` re-try interval With `< 1200` it just tries it on the next refresh. * plotting: Fix and improve duplicates handling * harvester|plotting: Thread locks for `PlotManager.plots` * chia|tests: Load plots in batches * chia|tests: Move plot refreshing into a separate thread * plotting: Properly handle removed plots And fix tests accordingly. It seems like this fix https://github.com/Chia-Network/chia-blockchain/pull/3350 wasn't really a fix, rather adjusting to allow for a bug? * plotting|harvester|tests: Introduce `PlotRefreshResult` * tests: Expand `test_farmer_harvester_rpc.py` * chia|tests: Move some stuff from `plot_tools.py` into new file `util.py` * refactor: Rename `plot_tools.py` to `manager.py` * chia|tests: Use pure dataclass for `PlotsRefreshParameter` With `uint16` as type saving to config doesn't work, this is a preparation for the next commit. * harvester: Adjust deprecation message, use `info` instead of `warning` * plotting: Fix typo * refactor: Rename `filename` to `file_path` Fits better and does avoid shadowing with filename from outer scope. * chia|tests: Move some methods from `plotting.manager` to `plotting.util` * plotting: Make `refresh_callback` mandatory
2021-08-09 20:25:15 +03:00
from chia.plotting.util import remove_plot_directory
chia|tests: Introduce `PlotManager` + some plot loading improvements and fixes (#7848) * harvester|plotting|tests: Introduce `PlotManager` class This moves all plots related and plot-directory related stuff from the harvester into the class `PlotManager`, adjusts all related code accordingly and adds some extra wrappers there. * harvester|plotting|tests: Return how many new plots were loaded * plotting: Fix `failed_to_open_filenames` re-try interval With `< 1200` it just tries it on the next refresh. * plotting: Fix and improve duplicates handling * harvester|plotting: Thread locks for `PlotManager.plots` * chia|tests: Load plots in batches * chia|tests: Move plot refreshing into a separate thread * plotting: Properly handle removed plots And fix tests accordingly. It seems like this fix https://github.com/Chia-Network/chia-blockchain/pull/3350 wasn't really a fix, rather adjusting to allow for a bug? * plotting|harvester|tests: Introduce `PlotRefreshResult` * tests: Expand `test_farmer_harvester_rpc.py` * chia|tests: Move some stuff from `plot_tools.py` into new file `util.py` * refactor: Rename `plot_tools.py` to `manager.py` * chia|tests: Use pure dataclass for `PlotsRefreshParameter` With `uint16` as type saving to config doesn't work, this is a preparation for the next commit. * harvester: Adjust deprecation message, use `info` instead of `warning` * plotting: Fix typo * refactor: Rename `filename` to `file_path` Fits better and does avoid shadowing with filename from outer scope. * chia|tests: Move some methods from `plotting.manager` to `plotting.util` * plotting: Make `refresh_callback` mandatory
2021-08-09 20:25:15 +03:00
remove_plot_directory(ctx.obj["root_path"], final_dir)
@plots_cmd.command("show", short_help="Shows the directory of current plots")
@click.pass_context
def show_cmd(ctx: click.Context):
show_plots(ctx.obj["root_path"])