Commit Graph

481 Commits

Author SHA1 Message Date
dustinface
d0c00937b9
full_node: Stop updating wallets during long sync (#16062) 2023-09-05 08:46:17 -05:00
Almog De Paz
d966f3f9e6
Add bad peak cache (#16027)
* add and check bad peak hash

* move check

* refactor, add tests, change cache to Dict

* pr fixes

* fix bad f-string

* refactor chip-0013 errors

* limit cache size, add size to constants

* test cache invalidation fix invalidation bug

* fix off by one

* move cache size to config

* remove bt conf override

* use get_peers_with_peak

* fix expected exception in test_soft_fork4_activation
2023-08-30 13:27:22 -05:00
Arvid Norberg
ec9d0096f4
demote log level for TIMESTAMP_TOO_FAR_IN_FUTURE errors (#16101) 2023-08-29 13:46:57 -05:00
dustinface
22e4a5f9f3
full_node: Drop unused get_fork_point_no_validations (#16107) 2023-08-24 12:38:35 -05:00
dustinface
3b20b4a2f0
full_node: More use of FullNode.get_peers_with_peak (#16045) 2023-08-24 12:38:22 -05:00
Arvid Norberg
7b0bea4043
various typing fixes (#15899)
* add blspy-stubs to improve mypy

* various typing fixes

* avoid cast

* use ValueError exception instead of assert when aborting picking a wallet key
2023-08-17 19:10:40 -05:00
Amine Khaldi
0c0dbd26d0
Merge commit '0101ae11b4e7b7ffd1ec12faa46c643938517aaa' into checkpoint/main_from_release_2.0.0_0101ae11b4e7b7ffd1ec12faa46c643938517aaa 2023-08-14 17:37:44 +01:00
Arvid Norberg
6701f77bc2
fix testnet10 sync-from-scratch (#15967)
soft-fork2 was not a smooth softfork on testnet (but it was on mainnet). This re-introduces the minimum logic for supporting syncing on testnet10
2023-08-11 19:24:15 -05:00
Arvid Norberg
b32128949f
Simplify BlockStore and CoinStore (#15936)
* simplify BlockStore, now that we only support v2

* simplify CoinStore, now that we only support v2

* add test for BlockStore.get_peak()

* remove left-over database v1 support in test_hint_store

* extend test for BlockStore

* remove residual database v2 logic from test_block_height_map
2023-08-11 12:12:16 -05:00
Arvid Norberg
c9e6c9d5cc
fix bug in recent BlockStore optimization patch (#16007)
fix bug in recent BlockStore optimization patch (and fix the issue with the test that allowed it through)
2023-08-10 19:12:11 -05:00
Arvid Norberg
b3fe7c4705
optimize blockchain DB for Chip-13 (#15948)
* optimize blockchain DB for Chip-13

* put plot_filter_info in its own table
2023-08-10 11:51:32 -05:00
Matt Hauff
0101ae11b4
Add ALLOW_BACKREFS as a flag to get_puzzle_and_solution_for_coin in full_node_api.py (#15937)
Add ALLOW_BACKREFS flag to full_node API
2023-08-08 20:37:37 -05:00
Arvid Norberg
c4708072f0
Revert "Add fields to BlockRecord. (#15695)" (#15947)
This reverts commit e84bdb0b30.
2023-08-07 09:32:31 -05:00
Arvid Norberg
1d8d476056
drop support for database schema v1 (#15895)
* don't run tests with database schema v1

* remove support for database schema v1 from BlockStore

* remove support for database schema v1 from CoinStore

* remove support for database schema v1 from HintStore

* remove support for v1 BlockStore schema from blockchain reorg logic

* remove support for database schema v1 from BlockHeightMap

* run block store tests both with and without the cache

* add test with empty blockchain for BlockHeightMap

* fix typo
2023-08-03 14:18:33 -05:00
Arvid Norberg
f0cb47baef
remove softfork2 logic (#15793) 2023-08-01 11:28:16 -05:00
Kyle Altendorf
9a08be7b0b
cleanup basic asyncio.CancelledError uses for 3.8 (#15801) 2023-07-24 20:00:14 -05:00
Matt Hauff
217e81f478
Move some puzzles out of chia.wallet.puzzles (#15822) 2023-07-24 10:39:16 -05:00
Arvid Norberg
eda68c0cdd
solution_generator (#15768)
* use solution_generator() from rust

* farm blocks with backrefs

* update BlockTools to generate block generators serialized with backrefs

* support deserializing programs with backrefs
2023-07-17 10:05:37 -05:00
Kyle Altendorf
2d28d62ab2
move sqlite host parameter limit to db wrapper (#15755)
* move sqlite host parameter limit to db wrapper

* pylint

* E

* oops

* fixup
2023-07-14 19:38:36 -05:00
Florin Chirica
e84bdb0b30
Add fields to BlockRecord. (#15695)
* Add fields to BlockRecord.

* Lint.

* Checkpoint refactor.

* Lint.

* Optimize get_block_records_by_hash.

* Lint.

* Coverage.

* Optimize v1 get_block_record.

* Trailing space.

* Remove async with await.

* Default 400.

* Address review comment.
2023-07-12 12:47:02 -05:00
Arvid Norberg
cecda28e84
Plot filter hard fork (#15336)
* update passed_plot_filter() to take the filter size rather than consensus constants. This allows the filter size to change by block height. make verify_and_get_quality_string() take either the height or filter size

* Add some filter_prefix_bits tests on test blocks.

* Add some filter_prefix_bits tests on simulated farmer and harvester.

* Cache filter prefix bits by challenge chain signage point hash and use that for the the lookups. This allows us to perform plot filter validation.

* Add more cases to verify_and_get_quality_string() unit tests.

* Add some tests for Farmer's respond_signatures.

* Apply Kevin's suggestion to simplify the check for passing plot filter.

* Apply Kevin's suggestions to simplify some test checks and fix a couple typos.

* Apply Kevin's suggestion to send peak height instead of filter prefix bits as part of NewSignagePoint.

* Remove no longer needed filter prefix bit related logic and make height non optional in verify_and_get_quality_string().

---------

Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
2023-07-12 10:26:23 -05:00
Arvid Norberg
1e534d0281
pass consensus constants into get_name_puzzle_conditions(). (#15736)
pass consensus constants into get_name_puzzle_conditions(). This is required for tests that use alternate constants, such as fork activation heights
2023-07-11 09:52:49 -05:00
Arvid Norberg
1d06adc9a6
forward flags in get_puzzle_and_solution_for_coin() (#15708)
support backrefs in the clvm serialization passed in- and out of get_puzzle_solution()
2023-07-07 18:08:15 -05:00
Arvid Norberg
ca1147584d
Hard fork 2.0 (#15581)
* add HARD_FORK_2_0 consensus_mode to run tests under hard-fork consensus rules

* hook up soft-fork and hard-fork changes in CLVM

* fix AGG_SIG_* garbage tests for the 2.0 hard-fork

* add test for unknown conditions with cost, which is a hard-fork feature

* add mempool tests for unknown conditions with cost

* add tests for the SOFTFORK condition

* fix benchmark to take LIMIT_ANNOUNCES into account
2023-07-07 13:58:43 -05:00
Amine Khaldi
a6713258c6
Merge commit 'f90ddf42ad5ff047533ba979b96c4d7f3b21f916' into checkpoint/main_from_release_1.8.2_f90ddf42ad5ff047533ba979b96c4d7f3b21f916 2023-07-06 20:26:03 +01:00
Arvid Norberg
f90ddf42ad
Eos challenge optimization (#15642)
* fix cache miss (#107)

* Increase vdf client timeout (#103)

* Increase vdf client timeout.

* Reset only on new peak.

* Increase next_iters_count to 10 (#118)

* new end of slot ses validation (#99)

* eos validate infused challenge chain hash in all locations, validate deficit

* lint

* add iterations/diff check

* check all cases of ses

* remove asserts log and return none

* remove asserts log and return none

* typo

* fix store tests

* fix store tests

* lint

* remove logs

---------

Co-authored-by: Almog De Paz <almogdepaz@gmail.com>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2023-06-27 15:34:31 -05:00
Arvid Norberg
d1c22134ad
bump chia_rs to 0.2.8 (#15575) 2023-06-21 13:59:22 -05:00
dustinface
f85e68ee22
full_node: Don't send duplicates in register_interest_in_puzzle_hash (#15560)
* Test self hinted coin in puzzle hash subscription

* Don't send duplicates in `register_interest_in_puzzle_hash`

* Some tweaks in the test
2023-06-21 13:11:35 -05:00
dustinface
92b149cd57
full_node: uint32.MAXIMUM_EXCLUSIVE -> uint32.MAXIMUM (#15544) 2023-06-16 11:46:37 -05:00
dustinface
b2498d2917
full_node: Add max_height to CoinStore.get_coin_states_by_ids (#15467) 2023-06-15 19:02:04 -05:00
dustinface
49140b2b3c
server: Introduce ApiProtocol (#15466)
* server: Introduce `ApiProtocol`

* genericize (#5)

* `ApiProtocol.api_ready` -> `ApiProtocol.ready()`

* Add `ApiProtocol.log` and give APIs separate loggers

* Fix `CrawlerAPI`

* Drop some unrelated removals

* Fix some of the generic hinting

* Revert some changes in `timelord_api.py`

* Fix `CawlerAPI` readiness

* Fix hinting

* Get some `CrawlerAPI` coverage

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-06-13 20:12:25 -05:00
dustinface
5e0b8bb1ec
util: Drop chia.util.chunks and use chia.util.to_batches instead (#15418) 2023-06-09 16:35:31 -05:00
dustinface
e3beca2d0d
full_node: More set usage in subscription code (#15420) 2023-06-09 05:11:54 -05:00
Kyle Altendorf
9e8d6ebd71
rework LockQueue as PriorityMutex (#14920)
* explore a simplification of the blockchain priority lock queue

* add some tests

* correct task tracking

* use time.perf_counter for better resolution on windows

* just count an integer for request order retention

* stop using time in the (new) tests as well

* add tests and a couple no covers

* less existing test refactoring

* use a sync PriorityQueue

* switch to deques

* address bugs and simplify priority to deque mapping

* remove unused attribute ._priority_type

* make LockQueu.create() not async

* explain the active element check on wait cancellation

* drop LockQueue._process()

* import final from typing_extensions

* rename LockQueue to PriorityMutex

* remove test from mypy exclusions

* clean up straggling lock references

* ignore test failure case line coverage

* add a monkeypatch test ;[

* remove queued callback feature

* remove todos
2023-06-07 14:08:45 -05:00
Arvid Norberg
f0433d2a47
Weight proof typing (#15316)
* add type hints to test_proof_of_space.py

* add type hints to weight_proof.py
2023-05-22 10:23:33 -05:00
Arvid Norberg
621c75e4ba
Fork infrastructure (#15299)
* remove softfork-logic for 1.7 softfork (which has already activated)

* add new constants for soft-fork3, hard-fork and the other plot-filter adjustments
2023-05-18 20:20:11 -05:00
dustinface
9e9c75912f
full_node: Refactor get_puzzle_and_solution_for_coin (#15323) 2023-05-17 17:34:33 -05:00
Amine Khaldi
b4c161ec57
Merge commit '61ad4dae37b9fd9a1dcd61416d3d7eaae0cc2fe3' into checkpoint/main_from_release_1.8.1_61ad4dae37b9fd9a1dcd61416d3d7eaae0cc2fe3 2023-05-17 11:15:52 +01:00
Arvid Norberg
61ad4dae37
optimize compact proofs (#15304) 2023-05-16 16:45:22 -05:00
Amine Khaldi
e76d9c56e6
Merge commit 'c6863fd92bfaf2838117a38656b0ac0d47d1e970' into checkpoint/main_from_release_1.8.1_c6863fd92bfaf2838117a38656b0ac0d47d1e970 2023-05-12 18:07:31 +01:00
Amine Khaldi
eb898dfe95
Identical spend aggregation (#13799)
* Exploring identical spend aggregation.

* Attempt to address review.

* Revert this.

* Explore spend-based processing instead of the bundle-based one.

* Attempt to address reviews.

* Update against chia_rs and add some improvements.

* Leverage the fact that we're in a function now to perform early exits.

* Explore propagating exceptions.

* Relax the exception handling.

* Add unit tests.

* Add some comments and perform some renames for more readability.

* Refactor tests and split them by scenario.

* Take Arvid's suggestion to simplify the check.

* Improve test readability.

* Make the set explicit instead of computing it with union().

* Run in MEMPOOL_MODE for cost and additions.

* Use int_from_bytes() instead of int.from_bytes().

* Add more unit tests for run_for_cost_and_additions (to cover coin amount 0 as well as a negative amount).

* Don't use as_python() for created coins.

* Account for the cost of create coin conditions and extract additions from the NPCResult.

* Rely on NPCResult for the maximum allowed cost.

* Stop looking for the relevant spend at the first occurrence (then process its created coins).

* Add a unit test for spending a coin in different ways then finding it spent in new peak.

This makes sure all mempool items that would spend that coin get removed.

* Parameterize the test for spending a coin in different ways then finding it spent in new peak, to account for both the optimized and the reorg code paths.

* Keep track of coin spends cost and additions in mempool items in case they won't make it into the block, so we don't rerun that again next time.

* Extend replace by fee tests with eligibility scenarios.

* Refactor find_duplicate_spends into get_deduplication_info and process_mempool_items into create_bundle_from_mempool_items.
Slightly refactor check_removals and create_bundle_from_mempool.

* Cast to uint64 in the return statement.

* Adapt to the recent addition of seq.

* Add unit tests for mempool items' bundle_coin_spends.

* Leverage the recently added make_test_conds when creating this npc_result.

* Pass in bundle_coin_spends and max_cost instead of InternalMempoolItem so that it remains internal.

* Add a test for the scenario where we pick a sub-optimal aggregation by deduplicating on solution A, seen in a relatively higher FPC item, then receiving better cost saving items on solution B after that one.

* Add some end-to-end tests for identical spend aggregation.

* Add a comment documenting why eligible_coin_spends map is ephemeral.
2023-05-12 10:55:53 -05:00
Arvid Norberg
c6863fd92b
don't add transactions to the mempool before it has a valid peak (#15264) 2023-05-12 10:54:35 -05:00
Amine Khaldi
54686f1121
Don't leverage SQLite's generated column feature for fee_per_cost. We'll hopefully reintroduce it again in the future (#15243)
* Don't leverage SQLite's generated column feature for fee_per_cost. We'll hopefully reintroduce it again in the future.

* Lint.
2023-05-09 18:52:09 -05:00
dustinface
a2567f45a9
full_node: Drop peers_with_peak removals (#15165) 2023-05-09 14:46:22 -05:00
Amine Khaldi
ab999d6192
Optimize check_removals() by checking for mempool conflicts all at once (#15202)
* Make conflicting_items a list instead of a set as we're only iterating over it, we're not leveraging set ownership.

This simplifies using can_replace() and also prepares ground to simplify check_removals() next.

* Optimize check_removals by checking for mempool conflicts all at once.
2023-05-04 13:21:18 -05:00
Amine Khaldi
808514a80b
Simplify handling additions and removals in validate_spend_bundle() (#15131)
Simplify handling additions and removals in validate_spend_bundle().
2023-04-27 09:56:14 -05:00
Amine Khaldi
de6fb526b5
Implement get_items_by_coin_ids() for the mempool (#15069)
Implement get_items_by_coin_ids() for the mempool.
2023-04-27 09:55:01 -05:00
Amine Khaldi
cf44101bc6
Use "item" instead of "spend" in mempool's items functions names (#15103)
Use "item" instead of "spend" in mempool's items functions names.

With this we're consistent with mempool's internal items map (_items) and _row_to_item as well as the return types.
2023-04-24 12:45:37 -06:00
Arvid Norberg
536830b09b
Generator mod cleanup and simplification (#15055)
* remove unused run_generator_mempool() function

* move run_generator_unsafe() into the tests, which is the only place it's used

* remove (somewhat meaningless) setup_generator_args() and create_generator_args()

* remove unused GENERATOR_MOD in mempool_check_conditions.py

* remove redundant get_generator() function

* transition analyze-chain.py to use run_block_generator() and drop dependency on GENERATOR_MOD

* fixup type hints in test_rom.py

* fixup type hints in test_compression.py

* fixup type hints in test_generator_types.py
2023-04-18 16:30:10 -05:00
Matt Hauff
e5b94d34a0
Chialisp file extensions (#14996)
* Rename chialisp file extensions

* Add pre-commit check for .clvm files

* Delete sha256tree files

* remove hash files in favor of central dictionary

* Add check for missing files like hex and hash entries

* Enhance clsp pre-commit check

* Actually check hash matches

* Update tools/manage_clvm.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tools/manage_clvm.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Fix Windows file writing

* Fix setup.py package_data fields

* Load hash dict at runtime

* Move away from exception pattern

* Bad equality check

* Minor fixes

* remove trailing whitespace fix

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-04-18 11:27:17 -05:00