* Add "metrics" as an additional destination for get_blockchain_state changes
* Add compact/uncompact blocks to get_blockchain_state
* Add total mempool cost to blockchain state
* Add hint count to blockchain state
* Add info about max cost, block_cost, and block_fees
* Add mempool min fee to blockchain state
* Add dict for mempool fee estimates
* dont start key with number
* Move max block cost to blockchain state
* Add generic exception logging util
* Catch/log any exceptions in counting hints
* Fix inconsistent key
* Assert row is not none
* Move block count metrics to a new endpoint
* report on block instead of unfinished_block for metrics
* Mark the block event with bool for if transaction block or not
* Add some basic tests for counting hints
* Add some basic tests on counting compact/uncompact blocks
* Fix lint
* Add k-size
* Add timestamp
* Add header_hash + height
* Try putting BLS in multiprocess
* Fix test
* Fix test again
* Fix test again ^ 2
* Don't use cache unless necessary, cleanup of unused code.
* WIP fixing tests
* Blockchain tests passing
* Remaining tests and lint
* More test fixes
* Fix full_node_rpc test
* Fix remaining tests
* Correct boolean
* Don't validate sig if CLVM fails. Also testing signature validation properly
* Simplify block prevalidation API and improve tests
* Fix logic error and remaining tests
* Update comments and change to AssertionError
* Try to improve CI performance
* Fix mempool test, try to fix test_blockchain performance
* Set default argument to fix test
* Fix DB conversion test
* Use cache for validating signatures (#9859)
* Use cache for validating signatures
* comments: fix typo
* Don't rerun the super slow part of the test (#9887)
* wallet changes from pac
* cat changes
* pool tests
* pooling tests passing
* offers
* lint
* mempool_mode
* black
* linting
* workflow files
* flake8
* more cleanup
* renamed
* remove obsolete test, don't cast announcement
* memos are not only bytes32
* trade renames
* fix rpcs, block_record
* wallet rpc, recompile settlement clvm
* key derivation
* clvm tests
* lgtm issues and wallet peers
* stash
* rename
* mypy linting
* flake8
* bad initializer
* flaky tests
* Make CAT wallets only create on verified hints (#9651)
* fix clvm tests
* return to log lvl warn
* check puzzle unhardened
* public key, not bytes. api caching change
* precommit changes
* remove unused import
* mypy ci file, tests
* ensure balance before creating a tx
* Remove CAT logic from full node test (#9741)
* Add confirmations and sleeps for wallet (#9742)
* use pool executor
* rever merge mistakes/cleanup
* Fix trade test flakiness (#9751)
* remove precommit
* older version of black
* lint only in super linter
* Make announcements in RPC be objects instead of bytes (#9752)
* Make announcements in RPC be objects instead of bytes
* Lint
* misc hint'ish cleanup (#9753)
* misc hint'ish cleanup
* unremove some ci bits
* Use main cached_bls.py
* Fix bad merge in main_pac (#9774)
* Fix bad merge at 71da0487b9
* Remove unused ignores
* more unused ignores
* Fix bad merge at 3b143e7050
* One more byte32.from_hexstr
* Remove obsolete test
* remove commented out
* remove duplicate payment object
* remove long sync
* remove unused test, noise
* memos type
* bytes32
* make it clear it's a single state at a time
* copy over asset ids from pacr
* file endl linter
* Update chia/server/ws_connection.py
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* stub SizedBytes.fromhex() until upstream is released
https://github.com/python/typeshed/pull/6201
* black
* more
* more
* more
* more
* more
* import TYPE_CHECKING via from
* more
* more
* catch up
* Rework sized bytes for type hinting compatibility
* add a bunch of type: ignores
* this will be handled elsewhere
* noqa E501 instead of changing code
* normalize comment plurality
* @classmethod
* Revert "@classmethod"
This reverts commit 95db80e339.
* add ignore in benchmarks
* just E501 again...
* add some new type: ignores
* initial hack
* crawler
* add pytz
* Checkpoint.
* Catch some bugs.
* Localhost dig working.
* Checkpoint: return only high quality nodes.
* Statistics.
* Try improving finding reliable nodes.
* Bug.
* Move db to memory.
* Timestamp in the last 5 days.
* Increase crawl parameters, 180+ connections per sec.
* Bug.
* Optimize for DNS traffic.
* Prepare for hosting.
* Minimum height.
* Typo.
* Try catch everything.
* dnslib.
* Add db, format code.
* nits.
* No connections for the dns server.
* Rename src -> chia
* Fix some issues with v1.1
* Crawler task pool.
* Optimize closing connections.
* Split crawler and dns server.
* Install instructions.
* Catch startup bug.
* Try a big timeout for lock aquire.
* lint.
* Lint.
* Initial commit extended stats.
* Simplify code.
* Config.
* Correct stats.
* Be more restrictive in crawling.
* Attempt to fix stats bug.
* Add other peers port to config.
* Update README for the config.
* Simplify crawl task.
* Fix bug on restarts.
* Prevent log spamming.
* More spam prevention.
* Fix bug.
* Ipv6 (#1)
* Enable ipv6.
* Fix bug.
* Use numeric codes for QTYPE.
* ANY working.
* More spam prevention.
* Try to improve IPv6 selection.
* Log IPv6 available.
* Try to crawl more aggresive for v6.
* rename dns.py to crawler_dns.py so it doesn't conflict with imported package names
* Remove pytz package off dependencies
* Tidy-up ws_connection.py
* Fix spelling
* Reinstate chia-blockchain readme, with additional lines pertaining to the DNS introducer & crawler
* More detailed info in the README wrt Chia Seeder
* Nit
* More memetic naming of Chia Seeder
* Nit
* Add entry points
* Add entry in packages
* Patch some methods on the upstream server
* Update peer record fields
* Standard library imports first
* Crawler API check
* Reconcile crawl store
* Account for crawler_db_path in config
* Await crawl store load DB and load reliable peers
* Updates to crawler
* Rename to dns_server
* Crawler-specific overrides for the chia server
* Edit comment
* Undo changes to ChiaServer in view of crawler-specific overrides introduced in previous commit
* Nit
* Update service groups
* Expand name maps, mostly
* Fix the init config
* Remove unused import
* total_records unused at this stage
* Remove ios_reliable in peer_reliability table
* Remove row[20] entry
* Split overly long line
* Fix
* Type hint for ns_records
* Reconcile mismatch btw type int and uint64
* Type annotations in crawler
* Check whether crawl store is set
* Remove upnp_list
* Lint
* Chia Seeder CLI
* Lint
* Two white spaces
* 3rd party package import
* Cleaner way to handle overrides for ChiaServer method
* Address linter warnings
* Rename
* Nits
* Fix
* Change port #
* Most chia_seeder commands up and running
* Rename
* Progress of sorts
* Fix
* Improve legibility
* Fix naming
* Fix setup.py
* Lint
* None -> ''
* Remove whitespace
* Rename
* Log ipv6 better. (#9227)
* Log ipv6 better.
* Lint.
* -
* Undo GUI changes
* Another attempt
* GUI changes
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
* plotting|util: Introduce `chia.util.generator_tools.list_to_batches`
* tests: Test `list_to_batches` in `test_list_to_batches.py`
* util|tests: Return an empty iterator for empty input lists
* tests: Test list sizes from 1-10 in `test_valid`
* make the is_peak index smaller
* condition database updates on a configuration option
* drop unused coin_spent index
* drop redundant index on primary key
Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.
* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml
* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.
* Update a couple things for consistency
* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.
* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives. also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.
* Change the default here to match the default coded into util/network.py. It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)
* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on
* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.
* Don't test for ip6-localhost, as it's not on all systems.
* Bah. Forced formatting of commented code...
* Add a type annotation for the addrset variable
* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined. So, quote them here.
Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Serialize/deserialize peer data alongside existing sqlite implementation (to be removed)
* Simplified AddressManagerStore. No longer uses sqlite and is no longer async.
* Removed aiosqlite usage from AddressManagerStore.
Added PeerStoreResolver class to determine the appropriate location for "peers.dat"
Updated initial-config.yaml to include "peers_file_path" default, replacing "peer_db_path" (similar change for "wallet_peers_path")
* Minor comment changes/additions
* Added migration from sqlite peer db.
Made AddressManagerStore's serialization async as it was right at the edge of blocking for too long.
* Minor tweaks to checking for migration
* Removed AddressManagerSQLiteStore class scaffolding
* makePeerDataSerialization now returns bytes instead of a PeerDataSerialization object
* Async file I/O for write_file_async using aiofiles
Added more tests
* Separate out the synchronous part of move_file
* Renamed write_file to files since we're opening up the capabilities a bit
* Update references to write_file
* Renamed test_write_file to test_files
* Tests covering move_file and move_file_async
* Minor refinements to behavior and tests
* Use aiofiles for reading peers.dat
* Added missing mypy typing info for aiofiles. Also added types-PyYAML to dev_dependencies so that `mypy chia tests` doesn't require running with --install-types
* Add types-aiofiles to the linting workflow
* Directory perms can now be passed into write_file_async.
Added an explicit f.flush() followed by os.fsync() after writing the temp file contents.
* Prority locking to consensus
* Remove pstats
* Linting
* Do some stuff outside of lock
* Fix startup
* Add log timings
* Try some different locking
* Add limit
* catch excp
* CLVM inside lock
* Try using a semaphore instead
* use events for lock queue
* test
* Add logging for message types
* type
* remove seed
* check new peak waiters
* correct FullNodeAPI self.full_node.new_peak._waiters typo
* correct logging string typos
* only warn about new_peak Waiters if there is at least 1
* remove no-longer-accepted parameter to FullNode.peak_post_processing()
* only warn about respond_transaction Waiters if there is at least 1
* lint
* Change some constants
* Small fix and logging changes
* Put message types outside
* Change some log levels so we can test with info
* More logging
* Increase rate limits but decrease paralelism
* tweaks
* Log dropped tx
* Fix pool rpc test
* Test fixes
* Mempool optimization
* Remove from seen if fails
* Increase queue sizes
* Message types info
* More test and logging
* Small changes to networking just in case
* Decrease logging
* Decrease logging even further
* Decrease logging even further even further
* Decrease logging 3
* Transaction queue
* Don't cancel tasks or close connection
* Cancel tasks on disconnect (for shutdown purposed)
* Fix typo
* Catch cancelled
* Do multiple at a time
* More accurate farmer response time
* More efficiently create tasks
* Increase queue size and priority by fee
* Revert priority
* Don't re-request too many times for dropped TX
* Handle cancelled error so we don't go into a bad state
* Catch cancelled in syncing tasks
* Reduce new_peak_sem to improve performance
* Less bytes conversion
* Missing file, and 2 workers for CLVM
* Validate BLS in a new thread
* tests
* Change semaphore constants
* correct a cancellation triggered exception and assertion
* Fix send_transaction, dont use BaseException, fix tests
* Fix more tests
* only log transaction handler cancellation in debug
* typing in log
* move unfinished validation to diff proc
* it is asyncio.CancelledError
* Add a test for bad signature
* Fix more tests, reduce logging, lint
* One more lint
* blockchain tests, pass bytes directly, single call
* Try to fix rl_wallet failures
* Fix mempool test
* catch everything
* Don't test RL wallet
* Fix more tests and return error code
* Improve error handling in multiprocess
* Add pre-validation time
* Add pre-validation time in logs, and revert pytest.ini changes
* Add log correctly
* Ms.bls cache experiment (#9115)
* Logging for cache
* Less logging
* Return to original plan
* Clean up
* Remove coment
* Remove log
* formalize LockQueue shutdown
* Comments
* Fix blockchain test
* Improve cache
* Remove logs
* Fix sign_coin_spends
* Fix pool wallet
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
* Expose GUI option to save passphrase (where available)
Prevent GUI from showing the option to remove migrated keys
* Backend support for getting/setting the passphrase hint. The hint data is stored in keyring.yaml in the clear.
* CLI support for showing/setting/removing the passphrase hint
* Tests for setting/retrieving passphrase hints
* Squelch LGTM false positive
* Fixed issue when running without passphrase support enabled
* CLI updates based on feedback
* Initialize passphrase_hint earlier.
* Minor restructuring to address linter complaint.
* Fixes cherry-picked from fix_keyring_tests.
* Logging for the failing test_using_legacy_cryptfilekeyring test
* See if reordering the tests makes a difference.
* Revert "See if reordering the tests makes a difference."
This reverts commit b53807880b.
* Log changes to the _configure_legacy_backend patch
* Fixed patching
* Checking if something isn't cleaned up properly
* Revert "Checking if something isn't cleaned up properly"
This reverts commit ce995ba773.
* Revert "Fixed patching"
This reverts commit 66a70a16ff.
* Revert "Log changes to the _configure_legacy_backend patch"
This reverts commit 26791a40b7.
* Revert "Logging for the failing test_using_legacy_cryptfilekeyring test"
This reverts commit 4fd4873565.
* Move test_keyring_wrapper.py into a standalone test
* Small refactorings/changes per feedback
* make synchronous mode in sqlite database connections configurabel. Default to enabling synchronous mode, leaving the door open for a more sophisticated heuristic in the future
* Update chia/util/db_synchronous.py
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Update chia/util/initial-config.yaml
Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: Nino Uzelac <uzelac.nino@gmail.com>
* Fix to allow migration of keys on macOS
* Fixed file_keyring synchronization test failures on macOS.
Fixed sporadic test failures on macOS when fsevents are delivered for the keyring after deletion.
TempKeyring-based tests now patch supports_os_passphrase_storage() to return False.
* Fixed LGTM issues
* Re-added the cleaning up temp keychain statement. This is being removed in a separate PR.
* Windows keyring passphrase support
* Test tweaks for Windows (mostly to support my slow VM)
* Linter fixes
* Restored prompt_for_passphrase() function which was accidentally removed during a rebase.
* Added CHIA_PASSPHRASE_SUPPORT env variable to allow more testing
* Added 'service' as a Keychain ctor param. Removed 'testing'
* Detect existing keys in the Mac Keychain
* Fix to allow migration of keys on macOS
* Added dump_keyring.py tool to show decrypted contents of keyring.yaml
* Prompt to save passphrase to macOS keychain
* Master passphrase retrieval/removal from the macOS Keychain.
Fixed typos.
* Warn if errSecInteractionNotAllowed is detected when accessing the macOS Keychain
* Fixed file_keyring synchronization test failures on macOS.
Fixed sporadic test failures on macOS when fsevents are delivered for the keyring after deletion.
TempKeyring-based tests now patch supports_os_passphrase_storage() to return False.
* TempKeyring mocks-out legacy_keyring setup to allow tests to succeed on macOS (which could find existing keys in the Keychain)
* Fixed pylint error
* Use with_name instead of with_stem (which is new to Python 3.9)
* Fixed keychain tests that started prompting for the keyring passphrase.
* Fixed LGTM issues
* Re-added the cleaning up temp keychain statement. This is being removed in a separate PR.
* Linter fixes
* Fixed keyring assignment on macOS when passphrase support is disabled.
* Include 'can_save_passphrase' flag in keyring_status response
* More linter fixes
* Fixed determination of the user_passphrase_is_set flag. This was returning true for a newly created keyring without any keys (or passphrase set)
* Removed the tidy_passphrase function per feedback
* Added some comments based on feedback
* Update chia/cmds/passphrase_funcs.py
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
