Commit Graph

59 Commits

Author SHA1 Message Date
Arvid Norberg
efcbc0ea4a
fixup block store and coin store benchmarks (#15946) 2023-08-08 02:25:59 -05:00
Kyle Altendorf
42a6d1afa6
remove support for Python 3.7 (#14611)
* remove support for Python 3.7

* python_requires=">=3.8, <4"

* 3.8.1+

* catchup

* remove todos handled in https://github.com/Chia-Network/chia-blockchain/pull/15755
2023-07-14 12:49:58 -05:00
Arvid Norberg
849166361d
bump chia_rs dependency to 0.2.9 (#15760) 2023-07-13 23:19:46 -05:00
Florin Chirica
e84bdb0b30
Add fields to BlockRecord. (#15695)
* Add fields to BlockRecord.

* Lint.

* Checkpoint refactor.

* Lint.

* Optimize get_block_records_by_hash.

* Lint.

* Coverage.

* Optimize v1 get_block_record.

* Trailing space.

* Remove async with await.

* Default 400.

* Address review comment.
2023-07-12 12:47:02 -05:00
Amine Khaldi
7e50f5e0b0
Merge commit '03943a0e51413f1f7fbd1f9ff852fe7259c0658e' into checkpoint/main_from_release_1.8.2_03943a0e51413f1f7fbd1f9ff852fe7259c0658e 2023-06-27 20:18:36 +01:00
Kyle Altendorf
03943a0e51
allow tests/tools/test_full_sync.py::test_full_sync_test to run on 3.7 (#15570)
* allow tests/tools/test_full_sync.py::test_full_sync_test to run on 3.7

* drop dirs_exist_ok=

* more

* less
2023-06-21 18:14:34 -05:00
dustinface
5e0b8bb1ec
util: Drop chia.util.chunks and use chia.util.to_batches instead (#15418) 2023-06-09 16:35:31 -05:00
Kyle Altendorf
2ed75d47f3
more mypy platforms in ci (#12299)
* more mypy platforms in ci

* check not checks

* oh yeah, activate...

* add 3.11

* adjust job names

* actually use the right python version for install

* adjust for 3.7 hinting support

* exclude macos 3.7 and macos arm 3.8

* back to 3.8 for now

* major_dot_minor

* oops

* move ignore comments for 3.7

* and more
2023-04-27 09:51:18 -05:00
Arvid Norberg
2b62f168f6
extend mempool-benchmark (#15038) 2023-04-17 15:44:06 -05:00
Kyle Altendorf
b839dfa0c1
update black to 23.3.0 (#14537)
* unpin black

* Update setup.py

* Update .pre-commit-config.yaml

* black

* black

* 23.3.0

* cleanup
2023-04-04 12:50:58 -05:00
Arvid Norberg
39d5283f1a
add benchmark that tests the mempool over a long time (#14728)
add benchmark that tests the mempool over a long time, churning transactions
2023-03-20 13:16:21 -05:00
Arvid Norberg
4ed31c50d5
Bump chia rs 0.2.4 (#14758)
* fix bug in make_test_conds() test utility

* bump chia_rs to 0.2.4, which preserves assert_seconds_relative 0 in parsing conditions. This allows for the 1.8.0 soft-fork to make the existing time-lock conditions stricter, > instead of >=. This is to match the existing ASSERT_HEIGHT_RELATIVE, which already is >

* fixup separating ENABLE_ASSERT_BEFORE from MEMPOOL_MODE

* Use a low value for SOFT_FORK2_HEIGHT during tests and cover the case before soft-fork2

---------

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
2023-03-07 15:29:54 -06:00
Kyle Altendorf
2d44e510b6
hint_benchmarks (#14738) 2023-03-06 11:50:44 -06:00
Arvid Norberg
9e8dae67b2
Bump chia_rs to 0.2.3. and ASSERT_MY_BIRTH_* (#14720)
* bump chia_rs to version 0.2.3

* add new error codes for assert_my_birth_*

* add new condition codes for ASSERT_MY_BIRTH_*

* add logic for ASSERT_MY_BIRTH_* conditions
2023-03-03 15:12:37 -06:00
Kyle Altendorf
ff5ef6e073
remove leading newlines from indented code blocks (#14653)
* remove leading newlines from indented code blocks

* catch another one
2023-02-23 21:53:48 -06:00
Amine Khaldi
b019fe5962
Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak (#14628)
* Introduce BlockRecordProtocol as a subset of BlockRecord that the mempool manager uses for peak.

* Create BenchBlockRecord and use it for benchmarks/mempool.py.

* PR #14611 didn't land yet (keep 3.7 support for now).

* We don't need this guidance anymore.
2023-02-23 01:04:47 -06:00
Arvid Norberg
1a722be39b
extend and improve the mempool benchmark (#14635) 2023-02-22 12:12:08 -06:00
Arvid Norberg
68c9aafc0a
move SerializedProgram to its own file (#14391)
* move SerializedProgram into its own file (to fixup type annotations)

* fix type annotations for SerializedProgram

* fixup import statements for new SerializedProgram module
2023-01-27 17:20:23 -06:00
Amine Khaldi
cd4874c584
create_bundle_from_mempool() doesn't need to be async (#14114)
create_bundle_from_mempool() doesn't need to be async.
2022-12-13 17:11:12 -06:00
Adam Kelly
30feeaa45e
Avoid calling spend_bundle.name() twice (#14109) 2022-12-13 00:32:55 -06:00
Amine Khaldi
0cfd19f38d
Simplify the mempool manager instantiation by not making it depend on a coin store (#14015)
* Explore simplifying the mempool manager instantiation.

* Address reviews.
2022-11-29 12:18:09 -06:00
Adam Kelly
5cbc415589
Preserve correct MempoolItem block height when rebuilding mempool (#13951) 2022-11-21 16:28:48 -06:00
Adam Kelly
73e9d15cf7
Fee estimation. (#13645) 2022-10-18 09:12:22 -05:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Mariano Sorgente
87a3e93cc4
Ms.mypy enable (#13320)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Enable mypy for a bunch of files

* Update after merge with main

* 99/260 remaining

* Address comments

* Enable more mypy

* Merge conflict mypy file

* More fixes

* pytest.ini and cb-gui

* One more flake8 fix

* Flake8 and tests

* More test fixes

* isort

* Make LGTM happy

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/full_node/full_node_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/util/peer_request_cache.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Apply all suggestions from PR comments

* Revert cb-gui

* 100 back to 1000, minor tweaks

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-09-08 12:57:15 -05:00
Mariano Sorgente
8ef4c980b3
Ms.mempool simplify (#13314)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Update after merge with main

* Address comments
2022-09-06 12:32:03 -05:00
Arvid Norberg
f295d8899b
add benchmark for MempoolManager.add_spendbundle (#12780)
* add benchmark for MempoolManager.add_spendbundle

* log warning if pre_validate_spendbundle takes too long
2022-08-09 18:29:00 -05:00
Arvid Norberg
5ae87189c4
Rust coin (#11807)
* use rust implementation of Coin class, from chia_rs

* bump chia_rs dependency
2022-06-13 07:21:04 -05:00
Arvid Norberg
afae2ad5fe
make recurse_jsonify() work directly on types (#11537)
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols

* add benchmark for Streamable.to_json_dict()
2022-05-18 09:15:03 -07:00
Mariano Sorgente
d1e445fac2
Ms.mypy consensus (#11403)
* Mypy for consensus dir (Except blockchain.py)

* Finish up consensus folder mypy

* Lint

* isort

* lint

* awaitable over coroutine

* less Any

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-05-12 15:21:47 -07:00
Kyle Altendorf
34eed9a3bc
more isort (#11401) 2022-05-05 08:17:38 -07:00
dustinface
a48fd43100
streamable: Simplify and force correct usage (#10509)
* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `
2022-04-08 18:29:32 -07:00
Mariano Sorgente
890c7d3754
Fix remaining linting issues (#10962)
* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change
2022-04-02 13:22:55 -07:00
Arvid Norberg
a100dda37e
new DBWrapper supporting concurrent readers (#10166)
* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument
2022-03-28 11:58:00 -07:00
Arvid Norberg
656d7d94d8
fix type annotations for FullBlock.header_hash and FullBlock.prev_header_hash (#10909) 2022-03-27 12:20:30 -07:00
dustinface
0909fe7672
benchmarks: Implement streamable data comparison (#10433) 2022-03-04 09:51:39 -08:00
dustinface
cdf48f7b29
benchmarks: Implement benchmarks for streamable (#10388)
* benchmarks: Implement benchmarks for streamable

* benchmarks: Collect iterations per time instead of time per iterations

* benchmarks: Add standard deviation to streamable benchs

* benchmarks: Add ns/iteration to streamable benchs

* benchmarks: Move object creation out or the runs loop

* benchmarks: Use `click.Choice` for `--data` and `--mode`

* benchmarks: Its µ

* benchmarks: Improve logging

* benchmarks: Drop unused code

* benchmarks: Use `process_time` as clock

* benchmarks: Add stdev `us/iterations %` + more precission

* benchmarks: Add `--live/--no-live` option to enable live results
2022-02-24 10:19:57 -08:00
Arvid Norberg
34159d3529
optimize get_block_generator() (#10104)
* optimize get_block_generator()

* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.

* speed up test_full_block_utils
2022-02-17 07:32:29 -08:00
Arvid Norberg
0464265734
use monotonic clock in benchmarks (#10242) 2022-02-15 14:42:51 -08:00
dustinface
98971de9d5
pre-commit: Add a new hook to run isort (#8827)
* pre-commit: Add a new hook to run `isort`

* contributing: Add hint about `isort`

* add isort to dev deps, ignore existing .py files, use black profile

* long list to lines not comma delimited

* isort: Update and sort ignore list to match latest `main`

* add bash command line to generate isort extend skip list (#3)

* add bash command line to generate extend skip list

* tidy

* isort: More files to ignore after rebase

* tests: Fix `test_wallet_user_store.py` after rebase

* Some fixes after rebase

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-02-15 07:28:53 -08:00
Kyle Altendorf
d404d0e92b
just put __init__.py files where we have .py files (plus subdirectories) (#10129)
* just put __init__.py files where we have .py files

* and the rest

* remove unused ignore
2022-02-07 13:16:13 -08:00
Arvid Norberg
6872e75a48
benchmark for blockchain.get_block_generator() (#9999) 2022-02-03 16:09:04 -08:00
Arvid Norberg
7fb7a916ee
Simplify FullNode._replace_proof (#10049)
* simplify updating of proofs (when compacting them). This also saves two database lookups before the update

* fixup

* insert or ignore duplicate blocks

* when replacing a weight proof with a compact version, we do so by height. Rather than asking for all blocks at the specific height followed by a linear search for the one with the correct header hash, we can simply lookup the correct header hash and just look up that block. This simplifies the logic by dropping the looping over blocks.

* pass in header_hash to _replace_proof() to avoid the height-to-hash lookup
2022-02-01 22:31:47 -08:00
Mariano Sorgente
6e37b74742
Change compact block (#10011)
* Change compact block

* Also recent blocks

* Add bill's rollback code

* Fix v2 issue

* Fix v2 issue blockchain

* Fix tests
2022-01-28 19:19:10 -08:00
Arvid Norberg
93d967525b
revert allow-upgrades configuration option. It makes it significantly more complicated to do proper database migrations (#9768) 2022-01-13 14:53:27 -08:00
Arvid Norberg
f1dcd36c2a
In main chain (#9550)
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
2021-12-17 14:37:45 -08:00
Arvid Norberg
35cab7cea2
Block store benchmark (#9520)
* include SubEpochSummary objects

* make header_hashes authentic. include benchmarks of lookups on block_store as well
2021-12-09 15:38:59 -08:00
Arvid Norberg
cd80f67d74
[v2] use binary fields in coin_store table (#9455)
* use binary fields on coin_store table

* fix coin_store benchmark format string
2021-12-06 11:22:12 -08:00
Arvid Norberg
546011381d
fix mypy issues in benchmark/block_store.py (#9457) 2021-12-05 09:36:52 -08:00
Arvid Norberg
70d31a8c68
V2 database framework (#9442)
* fix block_store benchmark linting issues

* basic support for database versioning

* update benchmark to run v1 and v2
2021-12-03 13:52:10 -08:00