* new blob block api method integrated into wallet
* direct msg streaming of headers, rename, tests
* perform_handshake call fix
* updated trusted sync with new block header calls
* add max blocks limit to fetch
* added tests for rejected block header msgs
* avoid parsing transactions info if not required
* avoid looking up capabilities setting
* move block tests out of a class
* test fix
* Merge changes
* added docs and increased rate limits
* increased block header request interval from 32 to 128
* remove fetching hashes and use height range
* fetching by height in db v2
* update capabilities, other fixes
* fixed range block header call
* Add type hints
* Start work on optimizing fetch_last_tx_from_peer
* Huge speedup in trusted wallet sync
* Revert unintentional changes
* Fix trade issue
* Improve the code
* Str format
* Optimize handling of farming rewards
* Fix bug
* Performance fixes
* Optimizations to wallet syncing
* Don't return all coins in respond_additions
* Revert concurrency numbers
* More optimization of the caches
* Small optimization in coin_added
* Optimize request_additions significantly by using a cache
* fixes from feedback
* capabilities check fixes
* Increase rate limits to allow 250tps in verification requests
* Start work on rate limits
* New rate limit versioning support
* Revert unrelated changes
* revert return False
* Lint
* Revert cbi
* try tests with trusted peer
* Revert unrelated wallet changes
* Revert more debug changes
* Add test and throw on an error if not found
* Reject invalid requests
* Revert bad change with uint32, and change warning to info
* Parametrize wallet sync test
* Merge and LGTM
* More clean way to choose peers
* Fix lint
* add the new RejectBlockHeaders, RequestBlockHeaders and RespondBlockHeaders to the network protocol regression test and regenerate test files
* Rate limit diffs only
* Improve performance
* Simpler
* Lint
Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: arvidn <arvid@libtorrent.org>
* Add Windows CI
* rebuild workflows
* rebuild workflows
* try different activation
* catch up windows workflow template
* debug dir
* hard code dev extra on windows
* it is pytest
* .exe
* -m pytest for windows
* --module pytest
* Revert "debug dir"
This reverts commit 74b5bbfd92.
* debug pkg_resources import on windows
* debug tests.conftest import on windows
* rebuild workflows
* debug
* debug oops
* debug more
* debug more
* Update pytest.ini
* Update pytest.ini
* fix it
* Revert "debug more"
This reverts commit 367ed61dfa.
* Revert "debug more"
This reverts commit 772b6707c6.
* Revert "debug oops"
This reverts commit 5065c1e172.
* Revert "debug"
This reverts commit ecf721e453.
* Revert "debug tests.conftest import on windows"
This reverts commit ea71a51625.
* Revert "debug pkg_resources import on windows"
This reverts commit eb6afb1dfb.
* more warning ignore
* they're regexes...
* --numprocesses 0 for windows ci
* specify each test file individually since pytest on windows does not seem to handle the wildcards
* relative...
* select most recent supported python on windows, support INSTALL_PYTHON_VERSION
* sort test files when generating workflows
* report full python version on windows, actually use the selected version for the env
* oops
* debug dir
* skip timelord-requiring jobs on windows
* oops
* handle wildcard expansion for coverage on windows
* rebuild workflows
* download blocks and plots on windows
* correct and and or in yaml
* -d for Install.ps1
* git config --global core.autocrlf false
* add missing windows plot_sync workflow
* no check resource usage on windows
* delete outdated windows workflows that need the timelord
* catch up windows workflow template
* try again without xdist
* rebuild workflows
* Revert "try again without xdist"
This reverts commit 4bedf5f047.
* maybe now no xdist on windows
* maybe now no xdist on windows
* rebuild workflows
* debug chiapos==1.0.9
* Revert "debug chiapos==1.0.9"
This reverts commit 4b374e128a.
* debug blspy==1.0.9
* catch up windows workflow template
* rebuild workflows
* Various windows CI fixes
* Windows CI fixes and workflow updates
* ignore another warning for the keyring
* pre-commit fixup
* rebuild workflows
* catchup windows test workflows with long_lived/** and release/** setup
* Update test_full_sync.py
* Try without parallelization
* Some other tweaks for windows
* some more timeout tweaks
* more timeout tweaks for windows
* rebuild workflows
* Adjusting more timeouts
* Skip ipv6 test on windows
* More timeout tweaks
* More timeout adjustments
* More time out adjustments
* more windows tweaks
* Try running some things with -n 0
* Fix up windows test template and re-gen workflows
* Try this again
* Try all windows tests with -n 0
* misc update
* Undo some timeout changes
* Adjust timeout
* Increase timeout
* Coverage combine should combine the files in the current directory
* Trying retry test on windows
* timeout adjustment
* use tmp_path
* Few minor updates based on review
* lint fix
* Update tests/build-workflows.py
* Update tests/core/server/test_dos.py
* Try windows with pytest xdist n=2
* Check windows with xdist n=3
* Trying n=2 again
* Adjust timeout
* Update testconfig.py
* Update build-workflows.py
* Update testconfig.py
* Update .pre-commit-config.yaml
* Update testconfig.py
* Update build-workflows.py
* Update build-test-macos
* Update build-test-ubuntu
* Update build-test-windows
* skip
* Apply suggestions from code review
* tidy
* Update test_pool_config.py
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
* Resend transactions (#11167)
* Resend transactions
* Don't recheck transactions more frequently than timeout
* Add wallet resend parameter to config, move timeout code out of tx store, but close to call site
* Add a test for wallet transaction resend
* Add test for wallet retry
* isort new files for precommit and update workflows
* Use correct fixture name
* LGTM - remove unused import
Co-authored-by: Earle Lowe <e.lowe@chia.net>
* less except -> false (#10219)
* less except -> false
* establish_connection() and .perform_handshake() can just return None
* remove more pointless, and no wrong, assertions
* Write python version error to stderr (#11239)
Co-authored-by: wjblanke <wjb98672@gmail.com>
* simplify SizedBytes and StructStream (#11429)
* simplify SizedBytes and StructStream
* lint
* super() !!!
* do not pass parameter up to super().__init__()
* Update chia/util/struct_stream.py
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
* parse fixed-width int data from class name
* add int512 and uint128 .from_bytes(), test .parse() failures
* test serialization against struct.pack()
* use typing_extensions for final
* override ignore
* stop using struct for StructStream
oh the irony
* fixup .to_bytes() to accept parameters again for where we use that
* bring back signed parameter
* format
* adjust tests for new exception
* eliminate custom coding for uint128 and int512
* tidy
* remove unused StructStream.PACK attribute
* add direct tests for parse_metadata_from_name()
* stricter hinting
* remove no-longer-needed typeshed work-around
* apply strict type checking to all touched files
* remove StructStream override of .to_bytes()
* tidy
* types touchup
* add unused parameter comments
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: wjblanke <wjb98672@gmail.com>
* Allow services to set a non-default max request body size limit (#11516)
* updated bls to 13 (#11529)
* reduce the redundant computations of coin_ids in block_body_validation (#11530)
* add test for streamable -> json conversion (#11527)
* add test for streamable -> json conversion
* fixup test
* Uses the new `from_bytes_unchecked` method in blspy, to improve perfo… (#11463)
* Uses the new `from_bytes_unchecked` method in blspy, to improve performance
* Update test
* Fix merge conflict
* Use from_bytes_unchecked in post_init and from_json
* More uniform code
* rename test files that are missing test_ prefix (#10712)
* rename test files that are missing test_ prefix
* update mypy and isort excludes
* skip test_get_host_addr6() in GitHub Actions under macOS
* rebuild workflows
* Coin Selection Refactor With CAT Coin Selection Refactor (#9975)
* add exact match and best exact match algorithms
* optimize algorithm further
this might be good.
* lint
* fix bad logic
* add final algorithms
* delete lint
* oops
* Update coin_selection.py
* simplify and fix knapsack algoritm
* simplify code and correct logic
* make it way better.
* clarify comments and check for edge cases.
* add comments and stuff
* improve coin selection
addressed comments
Thanks!
* add coin_selection rpc tests.
* clean up and add new unit tests
* undo test changes
* add extra test cases
* move coin_selection to its own function and switch to it for cat and main wallet.
* add cat tests
* lint
* make function align with standards
also removed test
* make test better
* add proper types
* Improve code clarity
* wallet: fix coin selection bugs
* wallet: add an assert just in case
* tests: add some sleeps to reduce flakiness
* Isort
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* fix bad merge
* lint
* fix tests
* address aforementioned changes.
* remove wallet test
* isort
* more tests and fixes
* lint
* rename to amount for coin selection rpc
* fix incase we have no smaller coins
* fix tests + lint
* re add asserts
* oops
missed me.
* lint
* fix test
* Squashed commit of the following:
commit 34a2235de5
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Apr 13 10:09:42 2022 -0400
clarify comment
commit adbf7f4f94
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 20:27:05 2022 -0400
linty lint
commit 5ebc1ac9fd
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 20:17:19 2022 -0400
add failure test and final changes
commit 7e5a21b4c2
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Tue Apr 12 19:35:18 2022 -0400
add descriptions and slim down code
commit 31c95b916d
Merge: d7b91295bd9b0ef5f3
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Mon Apr 11 10:12:05 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit d7b91295b5
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Sun Apr 10 20:31:09 2022 -0400
lint
commit 30dc7c0ab4
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Sun Apr 10 20:25:52 2022 -0400
fix tests
commit 6c8c2e4874
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 15:06:00 2022 -0400
remove duplicate code.
commit 9f79b6f304
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 15:01:10 2022 -0400
address more concerns
commit 67c1b3929f
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 12:59:05 2022 -0400
fix logic error
commit 2d19a53245
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Thu Mar 31 11:47:52 2022 -0400
simplify and de duplicate code
commit 6ab1cc79bb
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 21:34:50 2022 -0400
add function and select individual coin
commit 582c17aa8d
Merge: ce2165942618fbaeba
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 21:14:37 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit ce21659429
Merge: 16aabb3fd6daba28db
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:53:21 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit 16aabb3fd5
Merge: 0b9fc28452286fe426
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:49:02 2022 -0400
Merge branch 'jack-cat-coinselection' into jn_coinselection_dust
commit 0b9fc28455
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:38:12 2022 -0400
lint
commit 62e74c72f4
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 20:34:22 2022 -0400
fix logic and tests
commit e738f44320
Author: Jack Nelson <jack@jacknelson.xyz>
Date: Wed Mar 30 18:52:05 2022 -0400
deal with dust and add tests
* make sure that we do not use any dust
* minor change
* address concerns
* adjust comments
* adjust comment
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* rebuild workflows (#11543)
* remove the cache from CoinStore. It appears the cost of maintaining the cache outweighs the gains. (#11540)
* Keep daemon websocket alive during keyring unlock (#11371)
* When prompting to unlock the keyring during daemon launch, it's possible
that the daemon websocket connection will timeout before the user has
entered their passphrase. We can't quickly detect that the connection
has timed-out, so instead we now reconnect after collecting the
passphrase, unlock the keyring, and then close the old connection.
* mypy fix
* Switch to using asyncio.loop.run_in_executor to collect the passphrase on its own thread.
* Use a new ThreadPoolExecutor configured with 1 worker thread.
* Support searching derived addresses on testnet. (#11449)
* Support searching derived addresses on testnet.
* Added tests
* Optimize code to not perform useless subgroup checks (#11546)
* Optimize code to not perform useless subgroup checks
* Revert less important optimizations
* hints and strict type checking for test_wallet (#11541)
* hints and strict type checking for test_wallet
* fixup
* flip flop
* disable pytest-monitor by default (#11507)
* disable pytest-monitor by default
* take 2
* make recurse_jsonify() work directly on types (#11537)
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols
* add benchmark for Streamable.to_json_dict()
* tests: Split up and improve `test_wallet_rpc.py` (#11552)
This breaks up the one big test into
```
test_send_transaction
test_create_signed_transaction
test_create_signed_transaction_with_coin_announcement
test_create_signed_transaction_with_puzzle_announcement
test_send_transaction_multi
test_get_transactions
test_get_transaction_count
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```
Note that there is still much more room for improvements. Also the
following tests can still be split up more:
```
test_get_transactions
test_cat_endpoints
test_offer_endpoints
test_key_and_address_endpoints
```
* Remove unneeded uint64() intermediate in CAT wallet (#11575)
* Get get_args() and get_origin() from typing_extensions (#11571)
* Get get_args() and get_origin() from typing_extensions
* Update streamable.py
* cleanup
* Fix several flaky tests (#11576)
* Increase some of the timeouts to reduce flakiness
* Experiment with CI for flakiness
* Force more runs
* Add argument name
* Fix flaky mempool test
* Attempt to fix another flaky test
* Fix wallet retry test
* Missing arg
* Lint
* Increase benchmark test time
* Debug
* Debug
* Simplify retry test
* Lint
* More lint
* Return false instead of assert
* No need to check disconnect twice (since we don't break, this was flaky)
* Remove useless changes
* Accidental changes revert
* Accidental changes revert 2
* Accidental changes revert 3
* Revert fixture
* Fix a few additional flaky tests (#11597)
* Fix a few additional flaky tests
* Lint
* Update tests/core/full_node/stores/test_full_node_store.py
Co-authored-by: Jeff <jeff@chia.net>
* Fix merge conflict
* Remove unnecessary function
Co-authored-by: Jeff <jeff@chia.net>
* Bind port 0 to fix race condition when grabbing available ports (#11578)
* port 0 to fix flakiness
* Try fixing setup_full_system
* Try fixing setup_full_system, and lint
* More attempts to fix
* No more calls to get random ports in setup_nodes
* Revert accidental changes
* Timelord extra arg
* Try with port 0
* Fix daemon test, and lint
* Try without 0.0.0.0
* Back to 0.0.0.0
* Try a few timelord changes to get test running
* Increase timeout again
* Use the correct interface to get the port
* INFO logging to debug issue
* Revert "INFO logging to debug issue"
This reverts commit 7c379e5cca.
* Fix advertised port log
* Add extra log
* Logging back
* Rollback the timelord changes
* Try port 0 timelord
* Revert "Try port 0 timelord"
This reverts commit 4997faf3b2.
* Try full green, change ordering
* Remove unused var
* speed up simulation and cleanup
* Now try without the port config
* Fix a flaky call to get_event_loop
* Try getting the port dynamically
* No dynamic port
* Try changing the ordering
* Try adding a sleep
* Back to what works
* Timelord before vdf clients
* Dynamic port for 1st timelord
* Revert "Dynamic port for 1st timelord"
This reverts commit 0f322a15b7.
* Revert "Timelord before vdf clients"
This reverts commit 3286c34696.
* Revert "Back to what works"
This reverts commit 30380dffb7.
* Revert "Try adding a sleep"
This reverts commit 9212b665a6.
* Revert "Try changing the ordering"
This reverts commit a62597d70d.
* Revert "No dynamic port"
This reverts commit 5d2e15749b.
* Revert "Try getting the port dynamically"
This reverts commit ef9cd75679.
* Revert "Fix a flaky call to get_event_loop"
This reverts commit 01a000fdfb.
* Try one to 0
* Just not 0
* Don't get port dynamically
* Cleanup a bit
* Fix
* Some cleanup work
* Some cleanup work
* Fix daemon test
* Cleanup
* Remove arguments
* restore missing hints being stored as None (instead of 0-length bytes) (#11568)
* Coin simplification (#11567)
* factor out as_list() from Coin into free function. Remove unused name_str() from Coin. Minor optimization of hash_coin_ids() for common case.
* extend Coin unit test
* test installer on fedora:36, drop 33
* add tests.util.misc.assert_maximum_duration for benchmark assertions (#11589)
* add tests.util.misc.assert_maximum_duration for benchmark assertions
* fix some imports
* more
* lint
* correct duration
* hint fixup
* rename to caller_file_and_line()
* set default timer to thread_time()
* extract gc manager and default to disabling
* self calibrate overhead
* wrap none-ness behind .results() method
* add messages
* from __future__ import annotations
* yield out self
* final from typing_extensions
* split it
* just use Future
* tweak
* tweak
* correction in loosely hint checked file
* all future indexing in hints
* union for the hints
* rename
* assert_runtime()
* another rename...
* message -> label
* label all results with test names
* oops
* early return from _set_spent function (#11594)
* check for removals before calling function
* move condition to function
* remove redundant condition (#11582)
* fix built-in profiler to work for mac (#11590)
* Bump actions/setup-python from 2 to 3 (#10949)
* Bump actions/setup-python from 2 to 3
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)
---
updated-dependencies:
- dependency-name: actions/setup-python
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fixup templates for python@v3
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
* avoid cancelling release/** and long_lived/** (#11519)
* tests: Some adjustments in `test_offer_endpoints` (#11558)
This is a follow up after #11552 which basically used the initial parts
of `test_cat_endpoints` to make `test_offer_endpoints` working. Now this
PR drops a lot of the stuff we can assume to be not needed in this test
because its tested in `test_cat_endpoints` and it adds some general
improvments to it.
* async sleep and require connection reset error for DoS test (#11612)
* fix jsonify bool
* Remove is not None and length assertion in select_coins() (#11569)
* Remove is not None and length assertion in select_coins()
The `coins is not None` check seems unneeded since `select_coins()` is hinted to return `Set[Coin]`. The length requirement needlessly restricts from being able to request a zero amount. Either situation would either trigger an exception in the next assert, or trigger the assert itself for non-zero amounts requested.
* also remove from cat_wallet.py
* new port 0 stuff
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Evan Graham <me@luakt.net>
Co-authored-by: wjblanke <wjb98672@gmail.com>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: Mariano Sorgente <sorgente711@gmail.com>
Co-authored-by: Jeff <jeff@chia.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: neurosis69 <83925572+neurosis69@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
* add tests.util.misc.assert_maximum_duration for benchmark assertions
* fix some imports
* more
* lint
* correct duration
* hint fixup
* rename to caller_file_and_line()
* set default timer to thread_time()
* extract gc manager and default to disabling
* self calibrate overhead
* wrap none-ness behind .results() method
* add messages
* from __future__ import annotations
* yield out self
* final from typing_extensions
* split it
* just use Future
* tweak
* tweak
* correction in loosely hint checked file
* all future indexing in hints
* union for the hints
* rename
* assert_runtime()
* another rename...
* message -> label
* label all results with test names
* oops
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols
* add benchmark for Streamable.to_json_dict()
* rename test files that are missing test_ prefix
* update mypy and isort excludes
* skip test_get_host_addr6() in GitHub Actions under macOS
* rebuild workflows
* simplify SizedBytes and StructStream
* lint
* super() !!!
* do not pass parameter up to super().__init__()
* Update chia/util/struct_stream.py
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
* parse fixed-width int data from class name
* add int512 and uint128 .from_bytes(), test .parse() failures
* test serialization against struct.pack()
* use typing_extensions for final
* override ignore
* stop using struct for StructStream
oh the irony
* fixup .to_bytes() to accept parameters again for where we use that
* bring back signed parameter
* format
* adjust tests for new exception
* eliminate custom coding for uint128 and int512
* tidy
* remove unused StructStream.PACK attribute
* add direct tests for parse_metadata_from_name()
* stricter hinting
* remove no-longer-needed typeshed work-around
* apply strict type checking to all touched files
* remove StructStream override of .to_bytes()
* tidy
* types touchup
* add unused parameter comments
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: wjblanke <wjb98672@gmail.com>
* util: Implement `Paginator` class as interface to access a list by pages
* Be less restrictive about page sizes and refactor tests
* Make the pages based of 0 instead of 1 and some more test refactoring
* More tests
* Adjust workflows after rebase
* Introduce `Paginator.create`
* `<=` instead of `- 1`
* util: Implement `Paginator` class as interface to access a list by pages
* Be less restrictive about page sizes and refactor tests
* Make the pages based of 0 instead of 1 and some more test refactoring
* More tests
* Adjust workflows after rebase
* Introduce `Paginator.create`
* `<=` instead of `- 1`
* more work on test blockchain
* Optimize test_blockchain.py
* Fix weight proof bug
* Rename variable
* first rc_sub_slot hash bug
* New plots
* try with a new ID
* Run without cache
* Address test blocks and plots preparation.
* Update constant in test_compact_protocol().
* Update this constant too.
* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.
* Fix benchmark test
* Revert mozilla-ca change
* Rebase on main
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
* attempt to checkout test-cache directly to desired location
* rebuild workflows
* maybe we can use CHIA_ROOT
* use CHIA_ROOT to find blocks and plots for tests
* oops
* more informational printing
* oops
* --capture no for debugging
* flake8
* import os
* undo some unrelated changes now covered elsewhere
* undo some debug changes
* rebuild workflows
* Hardcoded SSL test certs/keys
* Added a second set of certs/keys.
Cert/key sets are infinitely cycled-through using get_next_private_ca_cert_and_key() and get_next_nodes_certs_and_keys()
* More cert/key sets and a tool to generate them
* Updated SSL generator to sign with the appropriate root CA.
Fixed linter issues.
* Linter fixes
* Updated generate_ssl_for_nodes() based on feedback
* remove event_loop() fixtures
* flake8
* flake8
* remove sys.exit() from daemon shutdown
* bump full node test timeout. a lot... to see.
* fixup some tests
* back to module scope event loop fixture for test_full_node.py
* Update test_full_node.py
* Iterator...
* for the whole directory
* some fixtures back to module scope for reduced runtime
* back to 40 minute workflow timeouts
* these are being addressed separately
* Better management of KeyringWrapper's keys_root_path when using TempKeyring for tests.
* Move keys_root_path restoration code into `cleanup()`
Added an assert to detect if an unexpected shared KeyringWrapper is injected during a test.
* Conditionally restore keys_root_path for testing
* Use bt fixture
* rebase
* Use local_hostname
* Fix test_json (inheritance from unittest.TestCase)
* Use correct BlockTools fixture for test_simulation
* Pass bt fixture into cost calculation tests
* flake8
* Add missing parameters to test functions
* Fix from rebase issues
* Remove set_shared_instance
* Update comment
* Remove unneeded comments
* Remove unused code
* Remove unused code, run `multiprocessing.set_start_method("spawn")` at correct time.
* Revert unrelated change
* Set daemon_port. Teardown services in correct order.
BIG thanks to Mariano Sorgente for debugging help.
* Add back type signature - rebase issue
* Apply review fixes from Jeff
* Document why we need a later pytest-asyncio version
* Correct type for _configure_legacy_backend
* See what's going on during CI mypy run
* github workflows
* mypy typing
* Remove legacy Keyring create method
* Start daemon first
* Shutdown daemon coroutine properly
* Remove un-needed daemon_port argument
* Set chia-blockchain-gui to hash in main
* Remove connect_to_daemon_port
* Remove code that set "daemon_port" before calling `setup_daemon`
* Remove self_hostname fixture and extra self_hostname global
* Fix two test files that were not importing self_hostname
* self_hostname fixture
* Remove more unused test code
* Simplify fixture
* optimize get_block_generator()
* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.
* speed up test_full_block_utils
* introduce restrictions on generators at a specific height. disallow division on negative numbers and disallow redundant leading zeros on integer condition arguments (produced by a generator)
* use SOFT_FORK_HEIGHT constant
* there is no need to specify height when validating block in block_creation
* Update tests/core/full_node/test_mempool.py
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* unify the name we use for the special mode we run CLVM in, when accepting transactions into the mempool. We currently call this, strict mode, safe mode, safe and unsafe. Neither of these names are very descriptive. This patch renames this mode to mempool-mode
* import as
* Rework sized bytes for type hinting compatibility
* add a bunch of type: ignores
* this will be handled elsewhere
* noqa E501 instead of changing code
* normalize comment plurality
* @classmethod
* Revert "@classmethod"
This reverts commit 95db80e339.
* add ignore in benchmarks
* just E501 again...
* add some new type: ignores
* factor out map of block height to block hash and sub epoch summary from blockchain into its own class, BlockHeightMap
* save and restore the height-to-hash map and height-to-sub epoch summary map to flat files, to imrove startup time. Add unit tests
* add back assert
* Start with farmer protocol.
* Also check bytes.
* Add full node protocol.
* Remove from_bytes() in hardcoded data file.
* Add all protocol data.
* Ignore flake8.
* Run pre-commit.
* Lint.
* Add randomness to bytes32 and uints.
* Run pretest.
* Address deficiency discussed in #8552, add ability to resolve to IPv6
addresses for hostnames.
* If there is no prefer_ipv6 in the config, set it True (per @hoffmang9)
and write it back to config.yaml
* Pass prefer_ipv6 flag to get_host_addr, which required a little digging
for it in a few places that call get_host_addr.
* Update a couple things for consistency
* Move the load_config into Wallet's __init__ so it doesn't get called so
many times as it would in has_full_node.
* Pass None into get_host_addr if there's no preference in config, so we have only that one place where the coded default lives. also fix an oversight where we were building a PeerInfo from a PeerInfo in some cases.
* Change the default here to match the default coded into util/network.py. It seems that github testers can't handle trying to use IPv6 and this may be easier for average users (sadly)
* A test to see if manually creating the server on :: (IP6_ANY) lets tests connect to localhost with IPv6 on
* Revert back to IPv4 default and remove the override inserted into
TCPSite for testing.
* Don't test for ip6-localhost, as it's not on all systems.
* Bah. Forced formatting of commented code...
* Add a type annotation for the addrset variable
* If we don't quote the socket enums, pylint gets upset because it has issues figuring out where/how they're defined. So, quote them here.
Co-authored-by: Chris Ross <cross+chia@distal.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Prority locking to consensus
* Remove pstats
* Linting
* Do some stuff outside of lock
* Fix startup
* Add log timings
* Try some different locking
* Add limit
* catch excp
* CLVM inside lock
* Try using a semaphore instead
* use events for lock queue
* test
* Add logging for message types
* type
* remove seed
* check new peak waiters
* correct FullNodeAPI self.full_node.new_peak._waiters typo
* correct logging string typos
* only warn about new_peak Waiters if there is at least 1
* remove no-longer-accepted parameter to FullNode.peak_post_processing()
* only warn about respond_transaction Waiters if there is at least 1
* lint
* Change some constants
* Small fix and logging changes
* Put message types outside
* Change some log levels so we can test with info
* More logging
* Increase rate limits but decrease paralelism
* tweaks
* Log dropped tx
* Fix pool rpc test
* Test fixes
* Mempool optimization
* Remove from seen if fails
* Increase queue sizes
* Message types info
* More test and logging
* Small changes to networking just in case
* Decrease logging
* Decrease logging even further
* Decrease logging even further even further
* Decrease logging 3
* Transaction queue
* Don't cancel tasks or close connection
* Cancel tasks on disconnect (for shutdown purposed)
* Fix typo
* Catch cancelled
* Do multiple at a time
* More accurate farmer response time
* More efficiently create tasks
* Increase queue size and priority by fee
* Revert priority
* Don't re-request too many times for dropped TX
* Handle cancelled error so we don't go into a bad state
* Catch cancelled in syncing tasks
* Reduce new_peak_sem to improve performance
* Less bytes conversion
* Missing file, and 2 workers for CLVM
* Validate BLS in a new thread
* tests
* Change semaphore constants
* correct a cancellation triggered exception and assertion
* Fix send_transaction, dont use BaseException, fix tests
* Fix more tests
* only log transaction handler cancellation in debug
* typing in log
* move unfinished validation to diff proc
* it is asyncio.CancelledError
* Add a test for bad signature
* Fix more tests, reduce logging, lint
* One more lint
* blockchain tests, pass bytes directly, single call
* Try to fix rl_wallet failures
* Fix mempool test
* catch everything
* Don't test RL wallet
* Fix more tests and return error code
* Improve error handling in multiprocess
* Add pre-validation time
* Add pre-validation time in logs, and revert pytest.ini changes
* Add log correctly
* Ms.bls cache experiment (#9115)
* Logging for cache
* Less logging
* Return to original plan
* Clean up
* Remove coment
* Remove log
* formalize LockQueue shutdown
* Comments
* Fix blockchain test
* Improve cache
* Remove logs
* Fix sign_coin_spends
* Fix pool wallet
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
* use a conftest.py for shared blockchain fixtures
* try delaying import
* Revert "try delaying import"
This reverts commit f86cc305b1.
* debug
* debug
* debug
* create the .chia dir for tests before creating a broken symlink
* go back
* delay more imports
* black
* provide an explanatory TODO
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Fixes cherry-picked from fix_keyring_tests.
* Logging for the failing test_using_legacy_cryptfilekeyring test
* See if reordering the tests makes a difference.
* Revert "See if reordering the tests makes a difference."
This reverts commit b53807880b.
* Log changes to the _configure_legacy_backend patch
* Fixed patching
* Checking if something isn't cleaned up properly
* Revert "Checking if something isn't cleaned up properly"
This reverts commit ce995ba773.
* Revert "Fixed patching"
This reverts commit 66a70a16ff.
* Revert "Log changes to the _configure_legacy_backend patch"
This reverts commit 26791a40b7.
* Revert "Logging for the failing test_using_legacy_cryptfilekeyring test"
This reverts commit 4fd4873565.
* Move test_keyring_wrapper.py into a standalone test
* Small refactorings/changes per feedback
* Added 'service' as a Keychain ctor param. Removed 'testing'
* Detect existing keys in the Mac Keychain
* Fix to allow migration of keys on macOS
* Added dump_keyring.py tool to show decrypted contents of keyring.yaml
* Prompt to save passphrase to macOS keychain
* Master passphrase retrieval/removal from the macOS Keychain.
Fixed typos.
* Warn if errSecInteractionNotAllowed is detected when accessing the macOS Keychain
* Fixed file_keyring synchronization test failures on macOS.
Fixed sporadic test failures on macOS when fsevents are delivered for the keyring after deletion.
TempKeyring-based tests now patch supports_os_passphrase_storage() to return False.
* TempKeyring mocks-out legacy_keyring setup to allow tests to succeed on macOS (which could find existing keys in the Keychain)
* Fixed pylint error
* Use with_name instead of with_stem (which is new to Python 3.9)
* Fixed keychain tests that started prompting for the keyring passphrase.
* Fixed LGTM issues
* Re-added the cleaning up temp keychain statement. This is being removed in a separate PR.
* Linter fixes
* Fixed keyring assignment on macOS when passphrase support is disabled.
* Include 'can_save_passphrase' flag in keyring_status response
* More linter fixes
* Fixed determination of the user_passphrase_is_set flag. This was returning true for a newly created keyring without any keys (or passphrase set)
* Removed the tidy_passphrase function per feedback
* Added some comments based on feedback
* Update chia/cmds/passphrase_funcs.py
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
* no need to check announcements in python, it's all done in rust
* remove python condition parsing, use rust unconditionally
* no need to assert_my_amount, it's done in rust
* assert my puzzlehash is done in rust, no need to check it in python
* parent id is checked in rust, no need to do it in python
* my coin ID is checked in rust, no need to do it in python
* reorder condition checks to have to ones we won't see at the end
* no need for the RUST_CONDITION_CHECKER constant anymore
* update pool tests to use get_name_puzzle_condition
* Moved keyring handling into a KeyringWrapper class
* Update click to 8.0.x for prompt_required support
* Renamed KeyringWrapper to _KeyringWrapper
* Expose password management options on Linux
* CLI support for setting/removing a password
* Global option for specifying the master password
* Cache the password instead of setting on the context
* Password bootstrapping during chia init
* Tidying up _KeyringWraper's interface
* Initial pass migrating the legacy keyring contents
* Encryption/decryption of keyring.yaml contents
* FileKeyring backend encrypts with ChaCha20Poly1305
* Tightened up keyring migration and initialization
* Fixed issues identified by linters
* Remove root_path from Keychain
* Prevent double-migration if setting master passwd
* KeyringWrapper tests are mostly complete
* FileKeyring will now honor the service param
* Tests for get/set/delete password
* Formatting/commenting updates
* Writer lock support with tests - WIP
* keyring.yaml is now watched for modifications
* Reader/Writer lock for get/delete password
* Fixed linter issues
* Reader lock tests
* Formatting update
* Hook up CHIA_ROOT support for KeychainWrapper
* Quick fix to address test failures
* Fixed failures when existing legacy keyring exists
* Fixed test failures caused by reusing the same temp dir
* keyring.yaml now lives in ~/.chia_keys by default. Can be overridden with CHIA_KEYS_ROOT or --keys-root-path
* Fixed migration failure when setting a password (not using the default)
* KeyringWrapper now uses supports_keyring_password to determine if a FileKeyring should be used. Patched tests to work regardless of whether supports_keyring_password return False
* The daemon now takes a --have-gui option that will prevent calling check_keys() during startup. If the keyring is locked, we want the GUI to prompt for the password.
* Added is_keyring_locked RPC call
* Added 'unlock_keyring' RPC command
* Added KeychainProxy and KeychainServer to handle RPC messages related to keyring operations. WalletNode no longer directly accesses the Keychain class.
* Turn on macOS support for testing keyring passwords
* Fixed get_key_for_fingerprint to use the ocal keychain if the platform doesn't need to remotely access the daemon's keychain.
Fixed key reconstruction when sent over RPC.
* Farmer now accesses the keychain over RPC
* Fixes for linter issues and some restructuring to support tests that use setup_nodes.py
* Couple of fixes to unblock the GUI from launching when a keyring password is set
* Added a keychain RPC call for add_private_key()
* Added remaining keychain proxy RPC calls for delete_key_by_fingerprint and delete_all_keys
* Check for None when inspecting request arguments
* Run check_keys after unlocking the keyring when the daemon is launched via GUI
* Added check_keys RPC method.
Fixed deserialization of key entropy in get_all_private_keys. This was preventing the GUI from being able to show key details.
* Added get_first_private_key to keychain_server/proxy.
create_plots now uses the keychain proxy when launched from the daemon.
* Added a comment about KeychainProxy in chia plots check
* Workaround import conflict when importing from 'tests.*' due to fasteners name conflict
* Simulator now uses KeychainProxy if launched by the daemon.
KeychainServer/Proxy now takes keychain user/testing params for testing scenarios.
* Added "set_keyring_passphrase" RPC message
* Reworking KeychainProxy usage to handle local keychain tests and RPC keychain tests.
* Replace my prior usage of asyncio.run() with asyncio.get_event_loop().run_until_complete()
* Silencing file_keyring logging for the moment.
* Updated tests to use test keychains and appropriate BlockTools construction
BlockTools should now be created with create_block_tools(_async) to handle async scenarios.
Updated block_tools to be async compatible
Updated fasteners to fix installation of top-level 'tests' in site-packages
* Added 'remove_keyring_passphrase' RPC message to the daemon
Minor tweak to TempKeyring to default to some test params
* Fixed linter issues
* Remove flake8 ignore statement now that the fasteners module has been updated
* Some initial renaming changes: password -> passphrase
* Fixed wallet RPC issue where get_key_for_fingerprint wasn't awaited-upon.
Fixed legacy keyring initialization (for migration scenarios)
* Fixed improperly merged file
* Fixed linter issues.
More renaming.
* Updated spots that were still using an incorrect keychain call
* Renamed use_password_cache, obtain_current_password
* Renamed supports_keyring_password
* Renamed has_master_password
* Renamed has_cached_password, get_cached_master_password
* Linter fixes
* Renamed master_password_is_valid
* Renamed set_cached_master_password
* Renamed set_master_password
* Renamed remove_master_password
* Renamed has_cached_master_password
* Renaming in file_keyring and keyring_wrapper
Updated default keyring payload used for tests
* Renamed get_password
Other renaming updates
* Renamed set_password
Other renaming updates
* Renamed remaining password occurrences (where appropriate)
* password -> passphrase
* Added tests for setting an emoji and Japanese master passphrase
* Attempt to notify the daemon when a keyring passphrase is set/updated/removed
* Missed one password -> passphrase replacement.
* Fixed some file synchronization issues found when running tests on macOS
* Adjusted timeout values for test_writer_lock_reacquisition_failure for macOS.
* Removed logging statements previously added for debugging
* Prompt for keyring passphrase up-front when launching a service.
Changed --have-gui flag to --wait-for-unlock
* Updated set_keyring_passphrase RPC message to fix optional current_passphrase param when the keyring is using the default passphrase.
* Minor test cleanup to deduplicate some code.
* Fixed regression when setting a new master passphrase
* Minor refactoring and docs/commenting updates
* Renaming password -> passphrase went too far. Keyring backends use password terminology for compatibility with third party backends.
* Disabling macOS support (previously added for testing only)
* Disabling passphrase support in preparation for sending out the PR
* Fixed improper merge (vscode didn't save changes during rebase)
* Update chia/cmds/init_funcs.py
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
* skip_check_keys -> should_check_keys
* Shuffling some imports around to break cycles reported by LGTM
* Handle unlocking the daemon if it's already launched and waiting for unlock.
* Replaced uses_keychain_proxy decorator in farmer.py. Fixed async usage of get_reward_targets.
Linter/reformatting fixes
* Replaced uses_keychain_proxy decorator with a clearer method.
* Cleanup the temp keyring dir using shutil.rmtree()
* Restored self._root_path (had been changed to self.root_path)
* Minor cleanup
* ensure_keychain_proxy() now throws if connect_to_keychain_and_validate() fails
* Plot key resolution now yields a PlotKeys object which can be passed into create_plots.
De-indented test_invalid_icc_sub_slot_vdf to keep git blame tidy.
* Added 'keyring_status' daemon RPC message to support the GUI
* Minor changes relating to PR feedback
* Addressed more PR feedback (mostly type annotations)
* Commented-out macOS file keyring usage. This can be re-enabled for testing purposes.
* Addressed test failures that require multiple keyrings in the same process. Each TempKeyring will now set a custom KeyringWrapper instance.
* Fixed logic for communicating user_passphrase_is_set in the keyring_status RPC response.
* Updated type annotations and method signature for set_passphrase to expect a string instead of bytes.
* Fixed Wallet RPC tests
* Fixed full_node_store tests. BlockTools should be created using the create_block_tools(_async) function(s)
* Fixed test failures in test_pool_rpc
* Fixed test_daemon. After BlockTools.setup_plots is run, the config file needs to be re-read to refresh stale plot_directories.
* Suppressing LGTM false positives regarding passphrase leakage in CLI error output. Seems that LGTM sees MIN_PASSPHRASE_LEN as sensitive data.
* Second attempt at suppressing LGTM false positives
* Third attempt at addressing LGTM false positives
* Removed test_keyring_wrapper param from Keychain ctor. Test setup now sets the keyring_wrapper property directly.
* Reformatting
* More targeted update of the test config to refresh just the "plot_directories" value
* More LGTM suppressions
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: wjblanke <wjb98672@gmail.com>
* validate more aspects of conditions earlier, in the process clvm execution is offloaded to
* more tests
* use as_int() in SExp rather than int_from_bytes()
* put condition parser tests in its own test class
* capitalize module-scope variable
* abbreviate enum names in parse_condition_args
* use a more realistic cost per byte in test_rom
* merge parse_coin_id and parse_hash, since they are identical
* validate integers in StructStream constructors
* don't step back sub slots beyond zero
* fix off-by-one in blocks-to-remove
* handle find_fork_point_in_chain() returning -1 in wallet
* fix handling of not finding a shared ancestor in wallet reorg
* terminate clean_block_record() when we reach height 0
* harden int/uint validation in constructor. add comment about int512
* Start aggsig me
* Pass in genesis challenge
* Update DID, and remove useless functions from coinbase.py
* Unused imports
* Revert GUI
* Use a different constant so forks of chia can easily change it
* util|test: Add misc.py with format_minutes as first method
Format `minutes` into a string with the format:
`unit` and `sub-unit`
where `unit` and `sub-unit` can be:
- years (`unit` only)
- months
- weeks
- days
- hours
- minutes
* cmds: Use `format_minutes` in farm_funcs.summary
* Updated gui with sub block changes
* Move some files, validate fees in mempool
* Support for plots with pool address
* Fix test and new blocks
* chia-blockchain-gui good commit
* changelof
* Fix edge cases with unfinished blocks
* Update gui
* Add test fixtures
