Commit Graph

469 Commits

Author SHA1 Message Date
Adam Kelly
5ba71957d8
Display complete exception info in log file (#14319)
* Display complete exception info in logfile

* Update mempool tests

* Update mempool tests
2023-01-19 15:31:26 -06:00
Kyle Altendorf
69b38bea73
@functools.wraps() for the API request decorators (#14234)
Without this the representation of the resulting methods look like `<function api_request.<locals>.inner.<locals>.wrapper at 0x7ff7ff655990>`.  With, they are identifiable as to the method they wrap such as `<function FullNodeAPI.request_peers at 0x7fcc85d0d990>`.
2023-01-06 16:55:26 -06:00
Chida82
5fd468df15
Add proxy support to Data Layer download client (#14281)
* add proxy

* fix format document

* format document

* fix whitespace

* fix end of file
2023-01-06 14:18:36 -06:00
Earle Lowe
0a0c8920ff
Add a daemon heartbeat setting to config.yaml (#13886)
* make daemon heartbeat configurable and increase default

* Fix up daemon rpc test

* Fix dumb error with parameters

* Restore formatting

* Various updates from feedback

* Update tests/core/test_daemon_rpc.py

use config value for heartbeat

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* black fixes

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-01-06 11:55:20 -06:00
Almog De Paz
18c95da752
add local max subscription config (#14188)
* add local max subscription config

* add and use trusted wallet config
tests

* lint

* unused import

* break early

* dont break because we need to iterate for hints

* fix off by one

* lint
2023-01-02 21:38:38 -06:00
dustinface
e2e103241a
Wrap Union[IPv4Address, IPv6Address] in the class IPAddress (#14136) 2022-12-14 17:11:08 -06:00
dustinface
fbc3127bd5
util: Rework get_host_addr (#14068)
* util: Rework `get_host_addr`

* Require `prefer_ipv6` to be named

* Remove outdated comment

* Improve result selection
2022-12-13 09:48:41 -06:00
dustinface
a40f04012b
server: Replace WSChiaConnection.__getattr__ with call_api (#14052) 2022-12-13 00:35:42 -06:00
Arvid Norberg
377f617d1d
in profile analysis, treat GetQueuedCompletionStatus() as idle state (#14041)
in profiler analysis, treate GetQueuedCompletionStatus() as idle state
2022-12-07 20:48:40 -06:00
dustinface
2e2c297a80
util: Remove legacy keyring support (#13398) 2022-11-18 10:33:18 -06:00
dustinface
d9c97e15e4
isort: Remove all chia.util exceptions and sort the related files (#13939) 2022-11-17 15:32:18 -06:00
Almog De Paz
ac2203eda8
Refactor test utils for import availability outside of repo (#13880)
* refactor for import availability outside of repo

* refactor for import availability outside of repo

* small revert

* newline

* fix imports

* fix pre-commit issues

* types issort, adapt mypy ignore

* redundant imports

* move files from util to simulator

* run pre_commit

* remove setup_nodes/keyring from isort.cfg

* fix asyncio import
2022-11-15 18:51:05 -06:00
Kyle Altendorf
5cf8a291f7
adjust approach to optionality around ApiMetadata (#13888) 2022-11-10 19:58:12 -06:00
dustinface
0e5168b3cf
util: Hint ApiMetadata.message_class with Optional[Type[Streamable]] (#13870)
* util: `message_class: Optional[Any]` -> `message_class: Any` 

Im working towards more strict mypy in `chia.server.ws_connection` and 
this change helps there while (i think) it still retains the same 
behaviour.

* Make it `Optional[Type[Streamable]]`
2022-11-09 14:19:06 -06:00
dustinface
d3e1698cc8
util: Some cleanups for the memory profiler (#13809)
* Make `CallInfo` and `Frame` a `dataclass` and move them to the top level

* Use `click` in `memory_profiler.py`

* Move `mem_profile_task` into `chia.util.profiler`

* Move `chia.util.memory_profiler.py` to `tools/analyze_memory_profile.py`

* Implement `Frame.add`
2022-11-09 14:17:43 -06:00
Kyle Altendorf
8ff73e961a
add LimitedSemaphore (#13642)
* treat Semaphore._waiters as length zero when it is None

We access the `._waiters` private attribute of the Python asyncio
`Semaphore` class.  This was changed in Python 3.10.8 (and other versions)
to be initialized to `None` instead of an empty deque.  Our existing
unconditional length checks failed on the new `None` default.  This seems
to block syncing.

https://github.com/python/cpython/pull/97020

https://github.com/python/cpython/compare/v3.10.7..v3.10.8#diff-0fee1befb15023abc0dad2623effa93a304946796929f6cb445d11a57821e737

Reported traceback:
```python-traceback
2022-10-12T20:03:59.367 full_node full_node_server : INFO Connected with full_node {'host': '65.34.144.6', 'port': 8444}
2022-10-12T20:03:59.370 full_node full_node_server : ERROR Exception: object of type 'NoneType' has no len(), {'host': '65.34.144.6', 'port': 8444}. Traceback (most recent call last):
File "/home/summa/chia-blockchain/chia/server/server.py", line 598, in wrapped_coroutine
result = await coroutine
File "/home/summa/chia-blockchain/chia/full_node/full_node_api.py", line 114, in new_peak
waiter_count = len(self.full_node.new_peak_sem._waiters)
TypeError: object of type 'NoneType' has no len()

2022-10-12T20:03:59.371 full_node full_node_server : ERROR Exception: object of type 'NoneType' has no len() <class 'TypeError'>, closing connection {'host': '65.34.144.6', 'port': 8444}. Traceback (most recent call last):
File "/home/summa/chia-blockchain/chia/server/server.py", line 608, in api_call
response: Optional[Message] = await asyncio.wait_for(wrapped_coroutine(), timeout=timeout)
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/home/summa/chia-blockchain/chia/server/server.py", line 605, in wrapped_coroutine
raise e
File "/home/summa/chia-blockchain/chia/server/server.py", line 598, in wrapped_coroutine
result = await coroutine
File "/home/summa/chia-blockchain/chia/full_node/full_node_api.py", line 114, in new_peak
waiter_count = len(self.full_node.new_peak_sem._waiters)
TypeError: object of type 'NoneType' has no len()

2022-10-12T20:03:59.487 full_node full_node_server : INFO Connection closed: 65.34.144.6, node id: 506fe4c05ce6b72bb707471842e552307c7a547aa9ba981175db5c08fa3e47e6
```

* add LimitedSemaphore
2022-11-06 11:38:25 -06:00
Kyle Altendorf
cb0fbe3208
switch to context manager for task timing (#13473) 2022-11-05 17:46:40 -05:00
Kyle Altendorf
806499cb3f
add task_instrumentation fixture (#13176)
* add task_instrumentation fixture

* fixup

* Update conftest.py
2022-11-03 23:14:21 -05:00
Amine Khaldi
54ce10522b
Merge commit '00aac17996bc0c451133ab45d5f1641f368af959' into checkpoint/main_from_release_1.6.1_00aac17996bc0c451133ab45d5f1641f368af959 2022-11-02 19:07:34 +01:00
Kyle Altendorf
7897c80ed1
only open SQLite log file once per db wrapper (#13754)
* only open SQLite log file once per db wrapper

* actually remember the log file

* actually remember to close the connections...
2022-11-01 15:43:19 -05:00
Kyle Altendorf
00aac17996
strip input before bech32 decoding (#13759)
this avoids leading and trailing whitespace causing offer processing to fail
2022-10-31 10:45:08 -05:00
Sean
a0e70ca73b
Freebsd improvements when checking for legacy key (#13666)
Issue: #13282 Optional[Credential] = keyring.get_credential(keychain_service, current_user) -> AttributeError: 'NoneType' object has no attribute 'get_credential'

Freebsd platform fails to migrate in previous releases or in 1.5.1+ versions it fails to start farmer because there is a NoneType object from keyring.get_credential. This just checks for that when trying to set the legacy_keyring.
2022-10-25 16:46:15 -05:00
Kyle Altendorf
bae4e0c5ce
refactor @api_request decorator and family to use a single decorator (#13565)
* more explicit and complete handling of api decorator data

* fix

* .message_class

* actually, those are different types...

* tweak

* simplify

* learn that functools.wraps copies random attributes

* hack the ~planet~ `@api_request` decorator

* R not T

* more future

* implementation detail renames
2022-10-20 16:10:23 -04:00
Kyle Altendorf
7abc062d0d
test service shutdown response to signals (#13576)
* minimal fixup for daemon signal handling regression

* test daemon shutdown response to signals

* positional

* test datalayer shutdown response to signals

* one parametrized test for the services

* colocate the daemon test

* drop sig

* add sendable_termination_signals

* oops, it was SIGTERM

* wait for the daemon
2022-10-20 16:08:49 -04:00
Kyle Altendorf
5b39550f73
convert datalayer to DBWrapper2 (#13582)
* convert datalayer to DBWrapper2 (all write)

* more read, less write

* remove unneeded connection managers

* and...  close it

* data store now creates its own wrapper

* Drop unused hint DataLayer.batch_update_db_wrapper

* require named arguments for most of `DBWrapper2.create()`
2022-10-03 17:50:12 -05:00
dustinface
c837711bd0
util: Set log levels per handler / Fix the log level in beta mode (#13569)
* util: Set log levels per handler / Fix the log level in beta mode

Prior to this PR the log level from the beta config (`DEBUG`) wasn't 
used. So the beta mode still was depending on the config level of the 
service being `DEBUG`.

This PR refactors the logging setup a bit to make it even possible to 
set a specific log level for each handler which is what we need to make 
the `beta.log` level independent from the normal `debug.log` level.

* Add the error to the log

* Compare to `DEBUG`

* Log the handler also
2022-09-30 17:49:29 -05:00
Kyle Altendorf
2fc65e1178
more explicit and complete handling of api decorator data (#13542)
* more explicit and complete handling of api decorator data

* fix

* .message_class

* actually, those are different types...

* tweak

* simplify

* learn that functools.wraps copies random attributes
2022-09-30 17:47:56 -05:00
dustinface
2aa117b58a
util|rpc: Refactor RpcEnvironment -> WebServer (#13515)
* Move `rpc.rpc_server.RpcEnvironment` -> `util.network.WebServer`

* Improve `WebServer`

* Drop attribute resets

* `black` fix after rebase

* Fix `test_nft_bulk_mint.py`
2022-09-30 17:47:00 -05:00
Kyle Altendorf
61b497a701
add a transactional reader as DBWrapper2.reader() (#13468)
* add a transactional reader as `DBWrapper2.reader()`

* contextlib.AbstractAsyncContextManager

* try a minimal test

* rework test to allow the read to finish so the write can finish the commit after that

* pylint disable

* WAL!!!

* more tests

* if TYPE_CHECKING for typing variable assignments

* future

* cover some more test cases
2022-09-30 17:41:33 -05:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Kyle Altendorf
bbd032e1c9
embrace click 8 (#13559)
* click==8.1.3, black==22.8.0

* black

* click option creator hinting

* ignores for to-be-resolved issue

* remove unused import
2022-09-29 11:24:36 -05:00
Gene Hoffman
1224ef924f
Use codespell to catch some typos (#13557) 2022-09-28 04:27:34 -05:00
Kyle Altendorf
8ed283a821
make DBWrapper2 handle more connection setup (#13510)
* make DBWrapper2 handle more connection setup

* fixup

* make create_connection() both await'able and async with'able

* recover logging
2022-09-26 11:17:20 -05:00
dustinface
77932fcc11
util: Refactor select_port to not load the config (#13514)
* util: Refactor `select_port` to not load the config

* Fix `prefer_ipv6` config lookup

* Use `str2bool`
2022-09-23 17:47:26 -05:00
Kyle Altendorf
a72edd486d
minimal api switch to get_type_hints() (#13507) 2022-09-23 17:33:11 -05:00
dustinface
40538fd942
mypy: Fix some asyncio.Task related ignores (#13487) 2022-09-23 17:19:36 -05:00
Kyle Altendorf
1dbf575a3e
RpcApiProtocol and RpcServiceProtocol (#11522)
* provide a skeleton hint protocol for rpc apis

* more

* more self.custom_get_connections = None

* fix RpcApiProtocol.service hinting

* catch up with RpcApiProtocol._state_changed()

* FullNode.server is now non-optional.  'ish

* drop another todo

* custom_get_connections -> get_connections

* just make get_connections() a mandatory method with a helper provided

* lots

* try again

* ._start_with_fingerprint()

* fill out the Service generic parameter in start scripts

* ... instead of pass

* add docstrings to the protocols

* remove no longer needed hasattr/not None checks

* remove outdated comment

* refactor Harvester._is_shutdown to ._shut_down
2022-09-21 22:49:01 -05:00
Matt Hauff
736b83a933
On chain wallet notifications (#13261)
* initial design

* simplify

* curry amount

* add test for notficiation content

* Add some anti-spam to config

* remove comment function

* flake8

* mypy

* shatree

* add_rpcs

* isort

* lgtm

* isort

* golf a bit

* bad merge?

* add sorting and pagination

* Remove MSG_HASH from clvm

* black

* Decrease spam limit

* Add CLI and minor fix

* add newline
2022-09-21 00:53:31 -05:00
Arvid Norberg
0e4a638a70
Heap profiler (#13399)
* Add heap profiler (enabled via config)

* Update chia/util/task_timing.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* unify name of memory profiler (not heap profiler)

* add timestamp to memory profile output directory

* Update chia/util/memory_profiler.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/util/memory_profiler.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update chia/util/memory_profiler.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-09-16 15:58:32 -05:00
dustinface
fe6358bd65
util: Remove the assigned label if a key gets removed (#13461) 2022-09-15 20:17:31 -05:00
dustinface
837a4798f6
daemon: Implement system metrics logger based on beta config (#12491) 2022-09-14 12:12:24 -05:00
dustinface
f049ca654e
util: Introduce FileKeyringContent (#12672) 2022-09-14 00:49:15 -05:00
dustinface
70fe981fb3
cmds: Implement chia beta (#12389)
* cmds: Implement `chia beta` for the beta test program

* Unhide and document all `beta` subcommands

* Refactor all subcommands

* Introduce `chia beta configure`

* Introduce `chia beta status`

* Test all `chia beta` commands

* Use a separate file logger for beta logs

* Write the plotting call args to the log file

* Sort potential submissions

* Some refactoring around log file log handler creation

* JSON dump the plotting args
2022-09-12 16:57:31 -05:00
Kyle Altendorf
b1f444facc
Merge commit '40c9027bfa70eb68be9cf6535f87d919f505e0e0' into checkpoint/main_from_release_1.6.0_40c9027bfa70eb68be9cf6535f87d919f505e0e0 2022-09-09 20:58:10 -04:00
Amine Khaldi
888da7c722
Merge commit 'd8b4eb13d67214a0b55dc48d5d3ba0c60d1434ff' into checkpoint/main_from_release_1.6.0_d8b4eb13d67214a0b55dc48d5d3ba0c60d1434ff 2022-09-09 20:30:55 +01:00
Amine Khaldi
c03a516a76
Merge commit '1adc4d36b08c21548af6effec73ec57f929b61f2' into checkpoint/main_from_release_1.6.0_1adc4d36b08c21548af6effec73ec57f929b61f2 2022-09-09 16:20:38 +01:00
Mariano Sorgente
87a3e93cc4
Ms.mypy enable (#13320)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Enable mypy for a bunch of files

* Update after merge with main

* 99/260 remaining

* Address comments

* Enable more mypy

* Merge conflict mypy file

* More fixes

* pytest.ini and cb-gui

* One more flake8 fix

* Flake8 and tests

* More test fixes

* isort

* Make LGTM happy

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/full_node/full_node_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/util/peer_request_cache.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Apply all suggestions from PR comments

* Revert cb-gui

* 100 back to 1000, minor tweaks

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-09-08 12:57:15 -05:00
Kyle Altendorf
40c9027bfa
Ignore FileNotFoundError when checking SSL file permissions (#13369)
* Ignore FileNotFoundError when checking SSL file permissions

* Let the natural `FileNotFoundError` propagate
2022-09-07 18:16:26 -05:00
Amine Khaldi
baea40608a
Merge commit '1fdacc75a7ecc8f5169b3c4297c2bf91d99207a5' into checkpoint/main_from_release_1.6.0_1fdacc75a7ecc8f5169b3c4297c2bf91d99207a5 2022-09-07 20:23:55 +01:00
Amine Khaldi
a03a354a7a
Merge commit 'c59bf12f8ec8a0f2c195542af9a689b5e5b12164' into checkpoint/main_from_release_1.6.0_c59bf12f8ec8a0f2c195542af9a689b5e5b12164 2022-09-07 15:46:14 +01:00