Commit Graph

37 Commits

Author SHA1 Message Date
Adam Kelly
73e9d15cf7
Fee estimation. (#13645) 2022-10-18 09:12:22 -05:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Mariano Sorgente
87a3e93cc4
Ms.mypy enable (#13320)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Enable mypy for a bunch of files

* Update after merge with main

* 99/260 remaining

* Address comments

* Enable more mypy

* Merge conflict mypy file

* More fixes

* pytest.ini and cb-gui

* One more flake8 fix

* Flake8 and tests

* More test fixes

* isort

* Make LGTM happy

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/full_node/full_node_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/util/peer_request_cache.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Apply all suggestions from PR comments

* Revert cb-gui

* 100 back to 1000, minor tweaks

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-09-08 12:57:15 -05:00
Mariano Sorgente
8ef4c980b3
Ms.mempool simplify (#13314)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Update after merge with main

* Address comments
2022-09-06 12:32:03 -05:00
Arvid Norberg
f295d8899b
add benchmark for MempoolManager.add_spendbundle (#12780)
* add benchmark for MempoolManager.add_spendbundle

* log warning if pre_validate_spendbundle takes too long
2022-08-09 18:29:00 -05:00
Arvid Norberg
5ae87189c4
Rust coin (#11807)
* use rust implementation of Coin class, from chia_rs

* bump chia_rs dependency
2022-06-13 07:21:04 -05:00
Arvid Norberg
afae2ad5fe
make recurse_jsonify() work directly on types (#11537)
* make recurse_jsonify() work directly on types, circumventing the dataclasses.asdict() step. This enables simpler integration of non dataclasses into the Streamable and JSON protocols

* add benchmark for Streamable.to_json_dict()
2022-05-18 09:15:03 -07:00
Mariano Sorgente
d1e445fac2
Ms.mypy consensus (#11403)
* Mypy for consensus dir (Except blockchain.py)

* Finish up consensus folder mypy

* Lint

* isort

* lint

* awaitable over coroutine

* less Any

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-05-12 15:21:47 -07:00
Kyle Altendorf
34eed9a3bc
more isort (#11401) 2022-05-05 08:17:38 -07:00
dustinface
a48fd43100
streamable: Simplify and force correct usage (#10509)
* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `
2022-04-08 18:29:32 -07:00
Mariano Sorgente
890c7d3754
Fix remaining linting issues (#10962)
* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change
2022-04-02 13:22:55 -07:00
Arvid Norberg
a100dda37e
new DBWrapper supporting concurrent readers (#10166)
* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument
2022-03-28 11:58:00 -07:00
Arvid Norberg
656d7d94d8
fix type annotations for FullBlock.header_hash and FullBlock.prev_header_hash (#10909) 2022-03-27 12:20:30 -07:00
dustinface
0909fe7672
benchmarks: Implement streamable data comparison (#10433) 2022-03-04 09:51:39 -08:00
dustinface
cdf48f7b29
benchmarks: Implement benchmarks for streamable (#10388)
* benchmarks: Implement benchmarks for streamable

* benchmarks: Collect iterations per time instead of time per iterations

* benchmarks: Add standard deviation to streamable benchs

* benchmarks: Add ns/iteration to streamable benchs

* benchmarks: Move object creation out or the runs loop

* benchmarks: Use `click.Choice` for `--data` and `--mode`

* benchmarks: Its µ

* benchmarks: Improve logging

* benchmarks: Drop unused code

* benchmarks: Use `process_time` as clock

* benchmarks: Add stdev `us/iterations %` + more precission

* benchmarks: Add `--live/--no-live` option to enable live results
2022-02-24 10:19:57 -08:00
Arvid Norberg
34159d3529
optimize get_block_generator() (#10104)
* optimize get_block_generator()

* add a v1 compatible get_generator() to speed up get_block_generator() with v1 databases. Add test. Add error log in case generator_from_block() fails.

* speed up test_full_block_utils
2022-02-17 07:32:29 -08:00
Arvid Norberg
0464265734
use monotonic clock in benchmarks (#10242) 2022-02-15 14:42:51 -08:00
dustinface
98971de9d5
pre-commit: Add a new hook to run isort (#8827)
* pre-commit: Add a new hook to run `isort`

* contributing: Add hint about `isort`

* add isort to dev deps, ignore existing .py files, use black profile

* long list to lines not comma delimited

* isort: Update and sort ignore list to match latest `main`

* add bash command line to generate isort extend skip list (#3)

* add bash command line to generate extend skip list

* tidy

* isort: More files to ignore after rebase

* tests: Fix `test_wallet_user_store.py` after rebase

* Some fixes after rebase

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-02-15 07:28:53 -08:00
Kyle Altendorf
d404d0e92b
just put __init__.py files where we have .py files (plus subdirectories) (#10129)
* just put __init__.py files where we have .py files

* and the rest

* remove unused ignore
2022-02-07 13:16:13 -08:00
Arvid Norberg
6872e75a48
benchmark for blockchain.get_block_generator() (#9999) 2022-02-03 16:09:04 -08:00
Arvid Norberg
7fb7a916ee
Simplify FullNode._replace_proof (#10049)
* simplify updating of proofs (when compacting them). This also saves two database lookups before the update

* fixup

* insert or ignore duplicate blocks

* when replacing a weight proof with a compact version, we do so by height. Rather than asking for all blocks at the specific height followed by a linear search for the one with the correct header hash, we can simply lookup the correct header hash and just look up that block. This simplifies the logic by dropping the looping over blocks.

* pass in header_hash to _replace_proof() to avoid the height-to-hash lookup
2022-02-01 22:31:47 -08:00
Mariano Sorgente
6e37b74742
Change compact block (#10011)
* Change compact block

* Also recent blocks

* Add bill's rollback code

* Fix v2 issue

* Fix v2 issue blockchain

* Fix tests
2022-01-28 19:19:10 -08:00
Arvid Norberg
93d967525b
revert allow-upgrades configuration option. It makes it significantly more complicated to do proper database migrations (#9768) 2022-01-13 14:53:27 -08:00
Arvid Norberg
f1dcd36c2a
In main chain (#9550)
* add in_main_chain column to full_blocks table

* add test for block_store in_main_chain

* test block_store invariant
2021-12-17 14:37:45 -08:00
Arvid Norberg
35cab7cea2
Block store benchmark (#9520)
* include SubEpochSummary objects

* make header_hashes authentic. include benchmarks of lookups on block_store as well
2021-12-09 15:38:59 -08:00
Arvid Norberg
cd80f67d74
[v2] use binary fields in coin_store table (#9455)
* use binary fields on coin_store table

* fix coin_store benchmark format string
2021-12-06 11:22:12 -08:00
Arvid Norberg
546011381d
fix mypy issues in benchmark/block_store.py (#9457) 2021-12-05 09:36:52 -08:00
Arvid Norberg
70d31a8c68
V2 database framework (#9442)
* fix block_store benchmark linting issues

* basic support for database versioning

* update benchmark to run v1 and v2
2021-12-03 13:52:10 -08:00
Kyle Altendorf
8291f0221a
Make the sized bytes types hint compatible (#9369)
* Rework sized bytes for type hinting compatibility

* add a bunch of type: ignores

* this will be handled elsewhere

* noqa E501 instead of changing code

* normalize comment plurality

* @classmethod

* Revert "@classmethod"

This reverts commit 95db80e339.

* add ignore in benchmarks

* just E501 again...

* add some new type: ignores
2021-12-02 09:43:39 -08:00
Arvid Norberg
a75b639adf
fix typo in block_store benchmark (and missing clvm generator field) (#9433) 2021-12-01 11:31:31 -08:00
Arvid Norberg
fdaadc1ff9
add benchmark for block_store (#9425)
* add benchmark for block_store

* use random instead of secrets
2021-11-30 15:56:11 -08:00
Arvid Norberg
7f2be11da7
print database size in CoinStore benchmark (#9419) 2021-11-29 07:48:42 -08:00
Arvid Norberg
2263f9b399
improve coin_store benchmark (#9354)
Co-authored-by: xorinox <61642896+xorinox@users.noreply.github.com>
2021-11-23 09:53:59 -08:00
Arvid Norberg
6de03d0fe1
Coin store updates (#8818)
* improve coin store benchmark output

* factor out common row_to_coin() in coin_store, and some other slight cleanups
2021-10-18 09:33:20 -07:00
Arvid Norberg
bbc2f76a21
remove unnecessary index in CoinStore & add additional benchmarks (#8641)
* simplify and extend the coin store benchmark to include lookups by coin ID and get_coin_removed_at_height

* remove coin_spent index from coin store table. It's not necessary and slows down updates (and caused one mistaken query plan)
2021-10-12 09:27:47 -07:00
dustinface
2127634aab
benchmarks: Use secrets.token_bytes instead of random.randbytes (#8724)
Latter is only available in python >= 3.9 and lets pre-commit fail for 
lower versions.
2021-10-05 09:47:23 -07:00
Arvid Norberg
600a5f90a0
CoinStore benchmark and performance improvements (#8514)
* add coin_store benchmark

* optimize _set_spent in CoinStore

* simplify _add_coin_record, since we won't use it to replace an entry anymore

* use executemany in add_coin_record and set_spent

* disable sqlite sync
2021-09-21 10:43:43 -07:00