Commit Graph

55 Commits

Author SHA1 Message Date
Arvid Norberg
57a59abde0
remove redundant cost field in NPCResults (#17167) 2024-01-02 10:45:23 -07:00
Amine Khaldi
c431c6675c
Merge commit '507899ff19b0d0bbe14bc7ed8a192f5ad0bc7aa2' into checkpoint/main_from_release_2.1.4_507899ff19b0d0bbe14bc7ed8a192f5ad0bc7aa2 2023-12-23 00:24:41 +01:00
Arvid Norberg
507899ff19
Mempool multifetch (#17139)
* update type annotation for CoinStore.get_coin_records to support both List and Set

* update the mempool to fetch multiple coin records per query

* optimize the slow-path of updating the mempool by fetching all coin records up-front, in a single sql query
2023-12-22 16:29:39 -06:00
Kyle Altendorf
4f55ffb1eb
add SpendSim.managed() (#16889) 2023-12-05 13:31:13 -06:00
Amine Khaldi
3c91881324
Fix propagating spent coins IDs to the mempool on successful generator bundle creation in SpendSim's farm_block (#16918) 2023-11-22 09:08:19 -06:00
Arvid Norberg
c0deb18405
remove special case logic for soft-fork 3 activation (#16819) 2023-11-21 17:06:17 -06:00
Amine Khaldi
02bda86e8e
Fix SimBlockRecord construction w.r.t. prev_transaction_block_hash (#16882) 2023-11-21 13:04:48 -06:00
Arvid Norberg
ca85e485e6
transition away from __bytes__ conversion for fixed-size integers (#16701) 2023-10-31 16:57:17 -05:00
Arvid Norberg
fe8ae58650
Get block spends (#16451) 2023-10-10 10:14:21 -05:00
Arvid Norberg
1d8d476056
drop support for database schema v1 (#15895)
* don't run tests with database schema v1

* remove support for database schema v1 from BlockStore

* remove support for database schema v1 from CoinStore

* remove support for database schema v1 from HintStore

* remove support for v1 BlockStore schema from blockchain reorg logic

* remove support for database schema v1 from BlockHeightMap

* run block store tests both with and without the cache

* add test with empty blockchain for BlockHeightMap

* fix typo
2023-08-03 14:18:33 -05:00
Arvid Norberg
f0cb47baef
remove softfork2 logic (#15793) 2023-08-01 11:28:16 -05:00
Arvid Norberg
1e534d0281
pass consensus constants into get_name_puzzle_conditions(). (#15736)
pass consensus constants into get_name_puzzle_conditions(). This is required for tests that use alternate constants, such as fork activation heights
2023-07-11 09:52:49 -05:00
Arvid Norberg
1d06adc9a6
forward flags in get_puzzle_and_solution_for_coin() (#15708)
support backrefs in the clvm serialization passed in- and out of get_puzzle_solution()
2023-07-07 18:08:15 -05:00
dustinface
9e9c75912f
full_node: Refactor get_puzzle_and_solution_for_coin (#15323) 2023-05-17 17:34:33 -05:00
Amine Khaldi
cf44101bc6
Use "item" instead of "spend" in mempool's items functions names (#15103)
Use "item" instead of "spend" in mempool's items functions names.

With this we're consistent with mempool's internal items map (_items) and _row_to_item as well as the return types.
2023-04-24 12:45:37 -06:00
Matt Hauff
e5b94d34a0
Chialisp file extensions (#14996)
* Rename chialisp file extensions

* Add pre-commit check for .clvm files

* Delete sha256tree files

* remove hash files in favor of central dictionary

* Add check for missing files like hex and hash entries

* Enhance clsp pre-commit check

* Actually check hash matches

* Update tools/manage_clvm.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tools/manage_clvm.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Fix Windows file writing

* Fix setup.py package_data fields

* Load hash dict at runtime

* Move away from exception pattern

* Bad equality check

* Minor fixes

* remove trailing whitespace fix

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-04-18 11:27:17 -05:00
Amine Khaldi
f278d9ccf8
Simplify create_bundle_from_mempool's handling of removals (#15056)
Simplify create_bundle_from_mempool's handling of removals.

They're already taken care of by aggregation.
2023-04-17 16:34:22 -05:00
Amine Khaldi
8afba0814d
Remove no longer needed cost_per_byte cases (#14850)
Remove no longer needed cost_per_byte cases.
2023-03-24 18:29:50 -05:00
Arvid Norberg
8dbfc4840a
enable soft-fork2 conditions (incl. ASSERT_BEFORE) (#14733)
* enable soft-fork2

* add blockchain (consensus) test for time-lock conditions (non-ephemeral spend)

* introduce new soft-fork rule to compare ASSERT_SECONDS_* conditions against the previous transaction block's timestamp (to be consistent with ASSERT_HEIGHT_* conditions)

* bump chia_rs. This updates the mempool rules to disallow relative height- and time conditions on ephemeral coin spends

* implement assert_before in mempool_check_time_locks. Extend ephemeral coin test in blockchain with assert_before conditions

* implement support for assert_before conditions in compute_assert_height()

* support assert-before in mempool

* add timelock rule

* address review comments
2023-03-23 11:30:10 -05:00
Arvid Norberg
e195d268fb
simplify the mempool filter to only pass in the spend_bundle name (#14665)
simplify the mempool filter to only pass in the spend_bundle name. That's the only thing we use
2023-02-27 12:57:32 -06:00
Kyle Altendorf
ff5ef6e073
remove leading newlines from indented code blocks (#14653)
* remove leading newlines from indented code blocks

* catch another one
2023-02-23 21:53:48 -06:00
Amine Khaldi
b019fe5962
Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak (#14628)
* Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak.

* Create BenchBlockRecord and use it for benchmarks/mempool.py.

* PR #14611 didn't land yet (keep 3.7 support for now).

* We don't need this guidance anymore.
2023-02-23 01:04:47 -06:00
Arvid Norberg
ed8cee2b4b
Mempool private implementation (#14593)
* fix test asserts to not require the same object, just the same value

* make Mempool's implementation private and give it a public interface

* fixup test that used to count fee *levels* but now count transactions
2023-02-22 12:26:04 -06:00
Matt Hauff
59e389968e
Add cost logging to CLVM tests as fixture (#14444) 2023-02-10 15:13:45 -06:00
Matt Hauff
97230ca3b0
Add a way to import a SpendSim and SimClient as a contextmanager (#14434) 2023-01-31 19:22:06 -06:00
Amine Khaldi
cd4874c584
create_bundle_from_mempool() doesn't need to be async (#14114)
create_bundle_from_mempool() doesn't need to be async.
2022-12-13 17:11:12 -06:00
dustinface
42f6a883ae
isort: Remove the remaining exceptions and sort the related files (#14124) 2022-12-13 17:10:55 -06:00
Adam Kelly
30feeaa45e
Avoid calling spend_bundle.name() twice (#14109) 2022-12-13 00:32:55 -06:00
Amine Khaldi
0cfd19f38d
Simplify the mempool manager instantiation by not making it depend on a coin store (#14015)
* Explore simplifying the mempool manager instantiation.

* Address reviews.
2022-11-29 12:18:09 -06:00
Adam Kelly
5cbc415589
Preserve correct MempoolItem block height when rebuilding mempool (#13951) 2022-11-21 16:28:48 -06:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Kyle Altendorf
a20ee08fce
only compile clvm for developers (#13478)
* only compile clvm for developers

* retain existing behavior for backwards compatibility

* tests/check_clvm.py becomes tools/manage_clvm.py check (and add build subcommand)

* trigger recompilation in pytest runs
2022-09-27 11:21:54 -05:00
Kyle Altendorf
8ed283a821
make DBWrapper2 handle more connection setup (#13510)
* make DBWrapper2 handle more connection setup

* fixup

* make create_connection() both await'able and async with'able

* recover logging
2022-09-26 11:17:20 -05:00
Arvid Norberg
98adae20f2
use get_puzzle_and_solution_for_coin() from chia_rs (#13315) 2022-09-07 02:41:40 -05:00
Mariano Sorgente
8ef4c980b3
Ms.mempool simplify (#13314)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Update after merge with main

* Address comments
2022-09-06 12:32:03 -05:00
Arvid Norberg
163552b8d9
Implement generator_for_single_coin() in python instead of clvm and apply quex' optimization to only computing the puzzle hash if the parent and amount matches (#13237) 2022-08-30 16:30:34 -05:00
Arvid Norberg
1fddb7aef1
add new_transaction() to DBWrapper2 which creates a nested transaction, and make write_db() just add writes to the existing transaction (#12275) 2022-07-15 21:07:43 -05:00
Kyle Altendorf
749162d9fe
hint chia.clvm.spend_sim (#11819)
* hint chia.clvm.spend_sim

* adjust comments

* undo

* Created typo corrected to create
2022-06-29 04:34:41 -05:00
Mariano Sorgente
e0df18f85d
Ms.hint refactor (#11222)
* Some work on hints

* More cleanup and test_generator_tools

* Change relevant calls

* More refactor and cleanup

* More refactor

* Small fix

* Create wrapper object PeakPostProcessingResult

* Lint and small fix

* Fix a hint bug

* Fix hint update_wallets

* Fix test_full_sync

* pre-commit

* Start with PR review comments

* More efficient iteration

* Remove tx_removals_additions_and_hints

* Revert mozilla ca

* Fix issue with startup
2022-05-05 08:19:57 -07:00
Matt Hauff
151fd6476a
rebase and more fixes (#10885) 2022-04-12 10:15:33 -07:00
dustinface
a48fd43100
streamable: Simplify and force correct usage (#10509)
* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `
2022-04-08 18:29:32 -07:00
Arvid Norberg
a100dda37e
new DBWrapper supporting concurrent readers (#10166)
* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument
2022-03-28 11:58:00 -07:00
Matt Hauff
620a1b3f23
add optional persistence to SpendSim (#10780)
* add optional persistence to SpendSim

* Accidental rename
2022-03-22 16:20:32 -07:00
Arvid Norberg
659f4f4890
remove (redundant) spent field on CoinRecord. Replace it with a property (#9658) 2021-12-22 18:50:32 -08:00
dustinface
234d121c2a
chia|tests|pre-commit: Add missing __init__.py files (#8758)
* chia|tests: Add missing `__init__.py` files

* check and create needed __init__.py files via pre-commit

* plotting|tests: Handle `mypy` issues introduced by the `main` rebase

* pre-commit: Run `build-init-files.py` more verbose

* tests: Ignore `__pycache__` folders in `build-init-files.py`

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update tests/build-init-files.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* tests: Add shebang, make executable, add description

To `tests/build-init-files.py`

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2021-12-16 18:52:46 -08:00
Geoff Walmsley
a085d26670
added get_coin_records_by_parent_ids to spend_sim (#9438)
* added get_coin_records_by_parent_ids to spend_sim

* fix linting errors
2021-12-07 09:56:37 -08:00
Kyle Altendorf
c5902841de
Just use bytes32() some more (#9472)
* Just use bytes32() some more

* black

* more
2021-12-07 07:55:48 -08:00
Kyle Altendorf
8291f0221a
Make the sized bytes types hint compatible (#9369)
* Rework sized bytes for type hinting compatibility

* add a bunch of type: ignores

* this will be handled elsewhere

* noqa E501 instead of changing code

* normalize comment plurality

* @classmethod

* Revert "@classmethod"

This reverts commit 95db80e339.

* add ignore in benchmarks

* just E501 again...

* add some new type: ignores
2021-12-02 09:43:39 -08:00
Mariano Sorgente
8a028c3594
Ms.mempool locking (#9050)
* Prority locking to consensus

* Remove pstats

* Linting

* Do some stuff outside of lock

* Fix startup

* Add log timings

* Try some different locking

* Add limit

* catch excp

* CLVM inside lock

* Try using a semaphore instead

* use events for lock queue

* test

* Add logging for message types

* type

* remove seed

* check new peak waiters

* correct FullNodeAPI self.full_node.new_peak._waiters typo

* correct logging string typos

* only warn about new_peak Waiters if there is at least 1

* remove no-longer-accepted parameter to FullNode.peak_post_processing()

* only warn about respond_transaction Waiters if there is at least 1

* lint

* Change some constants

* Small fix and logging changes

* Put message types outside

* Change some log levels so we can test with info

* More logging

* Increase rate limits but decrease paralelism

* tweaks

* Log dropped tx

* Fix pool rpc test

* Test fixes

* Mempool optimization

* Remove from seen if fails

* Increase queue sizes

* Message types info

* More test and logging

* Small changes to networking just in case

* Decrease logging

* Decrease logging even further

* Decrease logging even further even further

* Decrease logging 3

* Transaction queue

* Don't cancel tasks or close connection

* Cancel tasks on disconnect (for shutdown purposed)

* Fix typo

* Catch cancelled

* Do multiple at a time

* More accurate farmer response time

* More efficiently create tasks

* Increase queue size and priority by fee

* Revert priority

* Don't re-request too many times for dropped TX

* Handle cancelled error so we don't go into a bad state

* Catch cancelled in syncing tasks

* Reduce new_peak_sem to improve performance

* Less bytes conversion

* Missing file, and 2 workers for CLVM

* Validate BLS in a new thread

* tests

* Change semaphore constants

* correct a cancellation triggered exception and assertion

* Fix send_transaction, dont use BaseException, fix tests

* Fix more tests

* only log transaction handler cancellation in debug

* typing in log

* move unfinished validation to diff proc

* it is asyncio.CancelledError

* Add a test for bad signature

* Fix more tests, reduce logging, lint

* One more lint

* blockchain tests, pass bytes directly, single call

* Try to fix rl_wallet failures

* Fix mempool test

* catch everything

* Don't test RL wallet

* Fix more tests and return error code

* Improve error handling in multiprocess

* Add pre-validation time

* Add pre-validation time in logs, and revert pytest.ini changes

* Add log correctly

* Ms.bls cache experiment (#9115)

* Logging for cache

* Less logging

* Return to original plan

* Clean up

* Remove coment

* Remove log

* formalize LockQueue shutdown

* Comments

* Fix blockchain test

* Improve cache

* Remove logs

* Fix sign_coin_spends

* Fix pool wallet

Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Yostra <straya@chia.net>
2021-11-04 09:29:05 -07:00
Arvid Norberg
c41ccf6109
improve log formatting (#8543)
* don't use scientific notation for coin_store timing log

* improve logging of timings in add_spendbundle

* improve timing logging of pre_validate_spendbundle. Pass in spend_name

* improve block validation timing logging

* enable coin_store timing logging unconditionally at DEBUG log level

* validation time formatting

* farmer response time formatting
2021-09-21 10:48:44 -07:00