Commit Graph

10 Commits

Author SHA1 Message Date
dustinface
49140b2b3c
server: Introduce ApiProtocol (#15466)
* server: Introduce `ApiProtocol`

* genericize (#5)

* `ApiProtocol.api_ready` -> `ApiProtocol.ready()`

* Add `ApiProtocol.log` and give APIs separate loggers

* Fix `CrawlerAPI`

* Drop some unrelated removals

* Fix some of the generic hinting

* Revert some changes in `timelord_api.py`

* Fix `CawlerAPI` readiness

* Fix hinting

* Get some `CrawlerAPI` coverage

---------

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2023-06-13 20:12:25 -05:00
dustinface
d6a2a9aa5d
seeder: Drop CrawlerAPI.__getattr__ (#15489)
* Test sending unknown messages to the crawler

* Drop `CrawlerAPI.__getattr__`

* Use a real crawler service for the test
2023-06-13 15:25:39 -05:00
dustinface
e7f8aa0827
seeder: Hint crawler_api.py (#15498) 2023-06-13 11:25:15 -05:00
dustinface
50d148eadd
Cleanup WSChiaConnection imports/usage (#13868) 2022-11-08 10:10:59 -06:00
Kyle Altendorf
bae4e0c5ce
refactor @api_request decorator and family to use a single decorator (#13565)
* more explicit and complete handling of api decorator data

* fix

* .message_class

* actually, those are different types...

* tweak

* simplify

* learn that functools.wraps copies random attributes

* hack the ~planet~ `@api_request` decorator

* R not T

* more future

* implementation detail renames
2022-10-20 16:10:23 -04:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
Mariano Sorgente
87a3e93cc4
Ms.mypy enable (#13320)
* Remove mempool.additions

* Don't re run the program, and remove program from mempool item

* Removals only stores item ids, and stores a list

* Move pending cache down to prevent cache dos

* Separate validation from adding to pool, and remove mypy exceptions

* Fix bug with replacing

* Add to mypy

* Revert cbgui

* precommit fail

* Properly update the seen dict

* lint error

* Fix mempool bug

* Enable mypy for a bunch of files

* Update after merge with main

* 99/260 remaining

* Address comments

* Enable more mypy

* Merge conflict mypy file

* More fixes

* pytest.ini and cb-gui

* One more flake8 fix

* Flake8 and tests

* More test fixes

* isort

* Make LGTM happy

* Update chia/full_node/block_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/full_node/full_node_store.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/util/peer_request_cache.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Apply all suggestions from PR comments

* Revert cb-gui

* 100 back to 1000, minor tweaks

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-09-08 12:57:15 -05:00
Chris Marslender
7b838239a9
Additional metrics (#11271)
* Remove _set_state_changed_callback from harvester_api.py as its never actually called

In rpc/rpc_server.py:318, what is actually called is rpc_server.rpc_api.service._set_state_changed_callback, so the version of this in harvester.py is what is actually used

* Remove duplicate initial value for state_changed_callback in init

* _state_changed -> state_changed so it can be called from harvester_api as well

* Add farming info state_changed event from harvester

* Add time to farming_info event

* Add [pre_]validation_time to block event

* Remove unused set_state_changed_callback on full_node_api. This is actually called on full_node_rpc_api.service

* Remove unused set_state_changed_callback on crawler_api. This is actually called on crawler_rpc_api.service (crawler)

* Remove unused set_state_changed_callback on farmer_api. This is actually called on farmer_rpc_api.service (farmer), not the api itself

* Add state changed event for submitting a partial

* Lint fixes
2022-04-22 10:53:03 -07:00
Chris Marslender
eb3ed142e0
Crawler RPC (#10141)
* Add crawler RPC server

* Generate private keypair for crawler

* Bring over cleanup changes from the last closed PR

* Update the crawler RPC information to be its own subsection within seeder

* Add sleep before crawling to let the daemon connection get set up

* Wait for the actual callback to not be None, instead of just a random sleep interval

* Rework crawler/dns seeder to use the daemon + normal chia start process rather than the old system intended for the standalone repo

* Update configure testnet to work with seeder config

* Add back the crawler/seeder options from the standalone version

* Remove the check for none/sleep. Not needed when this is started by the daemon

* Add real data to the get_peer_counts endpoint

* Lint

* Fix calls to configure from init

* Turns out we still might sometimes move too quick before daemon/state changed callback is ready

* Add peer counts in the state_changed callback method

* Add a setting for peer_connect_timeout in the seeder: section so we can control it just for crawler

* start_seeder

* Pass config/root_path to the DNSServer so it can also use the configured crawler DB Path

* change in () instead of if/or

* Remove unnecessary return
2022-02-14 11:48:35 -08:00
Gregory
025c45f0b5
Chia Seeder (#8991)
* initial hack

* crawler

* add pytz

* Checkpoint.

* Catch some bugs.

* Localhost dig working.

* Checkpoint: return only high quality nodes.

* Statistics.

* Try improving finding reliable nodes.

* Bug.

* Move db to memory.

* Timestamp in the last 5 days.

* Increase crawl parameters, 180+ connections per sec.

* Bug.

* Optimize for DNS traffic.

* Prepare for hosting.

* Minimum height.

* Typo.

* Try catch everything.

* dnslib.

* Add db, format code.

* nits.

* No connections for the dns server.

* Rename src -> chia

* Fix some issues with v1.1

* Crawler task pool.

* Optimize closing connections.

* Split crawler and dns server.

* Install instructions.

* Catch startup bug.

* Try a big timeout for lock aquire.

* lint.

* Lint.

* Initial commit extended stats.

* Simplify code.

* Config.

* Correct stats.

* Be more restrictive in crawling.

* Attempt to fix stats bug.

* Add other peers port to config.

* Update README for the config.

* Simplify crawl task.

* Fix bug on restarts.

* Prevent log spamming.

* More spam prevention.

* Fix bug.

* Ipv6 (#1)

* Enable ipv6.

* Fix bug.

* Use numeric codes for QTYPE.

* ANY working.

* More spam prevention.

* Try to improve IPv6 selection.

* Log IPv6 available.

* Try to crawl more aggresive for v6.

* rename dns.py to crawler_dns.py so it doesn't conflict with imported package names

* Remove pytz package off dependencies

* Tidy-up ws_connection.py

* Fix spelling

* Reinstate chia-blockchain readme, with additional lines pertaining to the DNS introducer & crawler

* More detailed info in the README wrt Chia Seeder

* Nit

* More memetic naming of Chia Seeder

* Nit

* Add entry points

* Add entry in packages

* Patch some methods on the upstream server

* Update peer record fields

* Standard library imports first

* Crawler API check

* Reconcile crawl store

* Account for crawler_db_path in config

* Await crawl store load DB and load reliable peers

* Updates to crawler

* Rename to dns_server

* Crawler-specific overrides for the chia server

* Edit comment

* Undo changes to ChiaServer in view of crawler-specific overrides introduced in previous commit

* Nit

* Update service groups

* Expand name maps, mostly

* Fix the init config

* Remove unused import

* total_records unused at this stage

* Remove ios_reliable in peer_reliability table

* Remove row[20] entry

* Split overly long line

* Fix

* Type hint for ns_records

* Reconcile mismatch btw type int and uint64

* Type annotations in crawler

* Check whether crawl store is set

* Remove upnp_list

* Lint

* Chia Seeder CLI

* Lint

* Two white spaces

* 3rd party package import

* Cleaner way to handle overrides for ChiaServer method

* Address linter warnings

* Rename

* Nits

* Fix

* Change port #

* Most chia_seeder commands up and running

* Rename

* Progress of sorts

* Fix

* Improve legibility

* Fix naming

* Fix setup.py

* Lint

* None -> ''

* Remove whitespace

* Rename

* Log ipv6 better. (#9227)

* Log ipv6 better.

* Lint.

* -

* Undo GUI changes

* Another attempt

* GUI changes

Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2021-11-27 18:30:25 -08:00