* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols
* add benchmark for Streamable.to_json_dict()
* Uses the new `from_bytes_unchecked` method in blspy, to improve performance
* Update test
* Fix merge conflict
* Use from_bytes_unchecked in post_init and from_json
* More uniform code
* simplify SizedBytes and StructStream
* lint
* super() !!!
* do not pass parameter up to super().__init__()
* Update chia/util/struct_stream.py
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
* parse fixed-width int data from class name
* add int512 and uint128 .from_bytes(), test .parse() failures
* test serialization against struct.pack()
* use typing_extensions for final
* override ignore
* stop using struct for StructStream
oh the irony
* fixup .to_bytes() to accept parameters again for where we use that
* bring back signed parameter
* format
* adjust tests for new exception
* eliminate custom coding for uint128 and int512
* tidy
* remove unused StructStream.PACK attribute
* add direct tests for parse_metadata_from_name()
* stricter hinting
* remove no-longer-needed typeshed work-around
* apply strict type checking to all touched files
* remove StructStream override of .to_bytes()
* tidy
* types touchup
* add unused parameter comments
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: wjblanke <wjb98672@gmail.com>
* Resend transactions
* Don't recheck transactions more frequently than timeout
* Add wallet resend parameter to config, move timeout code out of tx store, but close to call site
* Add a test for wallet transaction resend
* Add test for wallet retry
* isort new files for precommit and update workflows
* Use correct fixture name
* LGTM - remove unused import
Co-authored-by: Earle Lowe <e.lowe@chia.net>
* farmer|rpc|tests: Implement paginated harvester plot endpoints
* Simplify filtering
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Let the API handle the exceptions
* Simplify the other filtering too
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Simplify count assertions
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Refactor `is_filter_match` to `plot_matches_filter`
And just convert to `Plot` in tests.
* Move `chia.util.misc.KeyValue` to `chia.rpc.farmer_rpc_api.FilterItem`
* Rename `peer_id` to `node_id` to be match `get_harvesters_{summary}`
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* util: Implement `Paginator` class as interface to access a list by pages
* Be less restrictive about page sizes and refactor tests
* Make the pages based of 0 instead of 1 and some more test refactoring
* More tests
* Adjust workflows after rebase
* Introduce `Paginator.create`
* `<=` instead of `- 1`
* daemon: Iterate over a copy of `websockets`
Currently when it fails to send the response to one websocket it skips
the next websocket in list becaue we remove from the list while
iterating.
* util: Iterate over a copy of `peers_with_peak`
Currently when a peer is closed it skips the check for the next peer in
the list becaue we remove from the list while iterating.
* isort: Fix `streamable.py` and `test_streamable.py`
* mypy: Drop `streamable.py` and `test_streamable.py` form exclusion
And fix all the mypy issues.
* Fix `pylint`
* Introduce `ParseFunctionType` and `StreamFunctionType`
* Use `object` instead of `Type[Any]` for `is_type_*` functions
* Some `Any` -> `object`
* Use `typing.overload` for `recurse_jsonify`
* Move some comments
* Drop `Union`, use `Literal` properly
* Explicitly ignore the return of `f_type.parse`
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Merge two `recurse_jsonify` overloads
* Typing for the base definition of `recurse_jsonify`
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* streamable: Merge `strictdataclass` into `Streamable` class
* tests: Test not supported streamable types
* streamable: Reorder decorators
* streamable: Simplify streamable decorator and force correct usage/syntax
* streamable: Just move some stuff around in the file
* streamable: Improve syntax error messages
* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion
* streamable: Use cached fields instead of `__annotations__`
This is now possible after merging `__post_init__` into `Streamable`
* Introduce `DefinitionError` as `StreamableError`
* `/t` -> ` `
* asyncio.get_event_loop() is deprecated in 3.10, stop using it
https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop
> Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().
* black
* minor lock scope reduction
* use the lock in tests
* Use the passed root_path in configure CLI command
* switch to lock_and_load_config()
* oops
* cleanup
* make _load_config_maybe_locked() private
* black
* Remove future improvement opportunity TODO comment
* add inline executor and an option to run single-threaded
* add option to run test_full_sync in single-thread mode, to include block validation in profiles. Also attempt to speed it up by disabling db_sync
* Enable clvm_tools_rs by default
* Re-add clvm_tools dep for now as it provides python idioms for interacting with clvm data
* Take lint formatting
* Adam: Try making this non-parallel
* Try fix for threading issue in tests
* Test whether turning off parallel runs causes things to work (temp)
* Test use of temp files in clvm_tools_rs as a candidate solution for atomic replacement of hex output
* Use proper git+https url scheme (oops)
* Update to candidate 0.1.6 so we can test
* Revert version bump to re-test
* Test whether we can re-enable parallelism
* Attempt to mitigate concurrent test running: return own conception of the compiled output. This will work if the failing path is downstream of recompilation
* fix path to hex file
* Probe for source of 0-length data
* Further exploration: more assertions, hopefully to trigger in the test on ci
* Do an even more paranoid check to verify that we observe a file whose filesystem reported size is much smaller than expected
* Try a heavier handed approach, using heavyweight lockfiles on the filesystem
* Import a simple lockfile implementation and use it to enforce mutual exclusion. Simplify it and remove the unwanted os traffic
* Take lint, precommit advice, bump to clvm_tools_rs 0.1.6 now that it's released
* Fix lint
* While i was working on this, -n auto was on the command line so i think this didn't actually do anything, but reverting my change just in case
* Lint
* label the hashes re: pr
* Add a lock.py for spot exclusivity using the filesystem (re: adam in the pr) and a convenience wrapper that hides the details
* Formatting warning
* Ensure type info is present and do the obvious return of the inner function's result
* Use double quotes (lint)
* Properly balance blank lines
* Lint: alphabetize imports
* One line is required here (lint)
* Remove unnecessary assignment
* Add configuration locking
Extracted from https://github.com/Chia-Network/chia-blockchain/pull/10631
* note that fasteners will likely be replaced by filelock
* Fix test_multiple_writers on macOS
* create_all_ssl() doesn't need to be inside the config access lock
* add warnings about not using async within get_config_lock() get lock contexts
* no need to pre-touch the lock file
* .yaml.lock instead of just .lock
* test_multiple_writers() is sync
* Revert "add warnings about not using async within get_config_lock() get lock contexts"
This reverts commit 681af3835b.
* reduce lock context size in chia_init()
* use an exit stack in load_config()
* avoid config existence precheck
* only lock around the read in load_config()
* do not raise e, just raise
* tidy new imports
* fix queue empty check in test_config.py
* remove commented out code in test_config.py
* remove unused import
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
* Allow CAT autodiscovery
* Add wallet autodiscovery test
* Use dict get for automatically_add_unknown_cats with default false
* Refactor name generation for new CATs into one place
* Remove hardcoded default cat wallet name from wallet rpc test
* initial-config.yaml comment nit
Apply the same pattern as we have for deserialization to serialization.
This avoids all those recursive runtime lookups for "how to stream this
object" which brings a nice speedup:
```
compare: benchmark
mode | µs/iteration old | µs/iteration new | diff %
to_bytes | 447.57 | 193.56 | -56.75
compare: full_block
mode | µs/iteration old | µs/iteration new | diff %
to_bytes | 110.32 | 61.09 | -44.62
```
* spawn (not fork) for multiprocessing
* just append _worker to existing process names
* return properly in getproctitle()
* black
* ignore for unhinted getproctitle()
* Add comments about the setting of the multiprocessing start method
* black==21.12b0 in pre-commit
Match `setup.py`.
* move black from super linter to upload workflow
* black (updated)
* configure so black . works
* --check --diff for black
* Align the wallet node's weight proof timeout with the full node's value.
* Give the wallet config its own weight_proof_timeout value.
* Apply Kyle's cleaner version.
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Apply Kyle's anchors suggestion.
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Add mempool_max_total_cost to RPC
* Add signage_point event
* Fix incorrect crawler RPC port lookup
* Set up initial timelord RPC server + finished_pot_challenge event
* Add new compact proof event
* Add skipping/new_peak to track when fastest or not
* Check for None on change_data
* Add skipping_peak + new_peak to changes for metrics
* Convert chain to value
* Rename iters
* Timelord RPC to 8557 - 8556 is used in simulation tests
* Make tests work with RPC server on timelord
* Change event name to finished_pot
* Use broadcast_farmer object
* Move state changed for `finished_pot` after proofs_finished.append
* Fix type on ips var + add vdf_info and vdf_proof
* fix event name on the state_changed function
* optimize get_block_generator()
* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.
* speed up test_full_block_utils
231ef6faf2/CHANGELOG.md
> Fixed issues where the wallet code would not generate enough addresses when looking for coins, which can result in missed coins due to the address not being checked. Deprecated the config setting initial_num_public_keys_new_wallet. The config setting initial_num_public_keys is now used in all cases.
* Add crawler RPC server
* Generate private keypair for crawler
* Bring over cleanup changes from the last closed PR
* Update the crawler RPC information to be its own subsection within seeder
* Add sleep before crawling to let the daemon connection get set up
* Wait for the actual callback to not be None, instead of just a random sleep interval
* Rework crawler/dns seeder to use the daemon + normal chia start process rather than the old system intended for the standalone repo
* Update configure testnet to work with seeder config
* Add back the crawler/seeder options from the standalone version
* Remove the check for none/sleep. Not needed when this is started by the daemon
* Add real data to the get_peer_counts endpoint
* Lint
* Fix calls to configure from init
* Turns out we still might sometimes move too quick before daemon/state changed callback is ready
* Add peer counts in the state_changed callback method
* Add a setting for peer_connect_timeout in the seeder: section so we can control it just for crawler
* start_seeder
* Pass config/root_path to the DNSServer so it can also use the configured crawler DB Path
* change in () instead of if/or
* Remove unnecessary return
