* commonize signal setup
* add signal setup for async handlers
* future
* Update misc.py
* catch up
* logging
* missed one
* context manager and waiting for signal handlers to complete
* correct logging
* fixup
* fixup
* Cleanup seeder & mypy all files
holy crap mother of all tech debt this was horrid
* Make UDP Protocol class a dataclass & separate out functions.
* add TCP protocol class to DNS server
* Fix mypy types & other cleanup
also fix a couple bugs
* Add seeder and crawler tests
* change log levels
* re add db lock timeout
oops
* add edns & edns tests
* fix repeated shutdown on close signal
* fix binding to use both ipv6 and ipv4
whyyyyyyy
* add ipv6 and ipv4 tests + add ipv4 if windows
* server: Introduce `ApiProtocol`
* genericize (#5)
* `ApiProtocol.api_ready` -> `ApiProtocol.ready()`
* Add `ApiProtocol.log` and give APIs separate loggers
* Fix `CrawlerAPI`
* Drop some unrelated removals
* Fix some of the generic hinting
* Revert some changes in `timelord_api.py`
* Fix `CawlerAPI` readiness
* Fix hinting
* Get some `CrawlerAPI` coverage
---------
Co-authored-by: Kyle Altendorf <sda@fstab.net>
<!-- Merging Requirements:
- Please give your PR a title that is release-note friendly
- In order to be merged, you must add the most appropriate category
Label (Added, Changed, Fixed) to your PR
-->
<!-- Explain why this is an improvement (Does this add missing
functionality, improve performance, or reduce complexity?) -->
### Purpose:
remove dead code so the remaining code is more comprehensible
<!-- Does this PR introduce a breaking change? -->
### Current Behavior:
### New Behavior:
<!-- As we aim for complete code coverage, please include details
regarding unit, and regression tests -->
### Testing Notes:
<!-- Attach any visual examples, or supporting evidence (attach any
.gif/video/console output below) -->
Some DNS resolvers (notably 8.8.8.8) have implemented the use of bit 0x20 in
DNS labels to make cache poisoning attacks more difficult.
See https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00 for
more details.
Without this patch, resolution though Google public DNS fails:
$ dig dns-introducer.chia.net @8.8.8.8 +short | wc -l
0
Whereas it works as expected on a seeder which already deployed this patch:
$ dig chia.ctrlaltdel.ch @8.8.8.8 +short | wc -l
32
* Listen on ::0 so we can respond to both ipv4 and ipv6
* Move the initial sleep to the end of the first attempt to load reliable peers, so that if we already have a database, it doesn't take 60 seconds to start responding
* more explicit and complete handling of api decorator data
* fix
* .message_class
* actually, those are different types...
* tweak
* simplify
* learn that functools.wraps copies random attributes
* hack the ~planet~ `@api_request` decorator
* R not T
* more future
* implementation detail renames
* configure isort to add the future annotations import
* apply the new isort setting
* remove type ignores for new mypy (#13539)
https://pypi.org/project/mypy/0.981/
* another
* provide a skeleton hint protocol for rpc apis
* more
* more self.custom_get_connections = None
* fix RpcApiProtocol.service hinting
* catch up with RpcApiProtocol._state_changed()
* FullNode.server is now non-optional. 'ish
* drop another todo
* custom_get_connections -> get_connections
* just make get_connections() a mandatory method with a helper provided
* lots
* try again
* ._start_with_fingerprint()
* fill out the Service generic parameter in start scripts
* ... instead of pass
* add docstrings to the protocols
* remove no longer needed hasattr/not None checks
* remove outdated comment
* refactor Harvester._is_shutdown to ._shut_down
* cmds: Implement `chia beta` for the beta test program
* Unhide and document all `beta` subcommands
* Refactor all subcommands
* Introduce `chia beta configure`
* Introduce `chia beta status`
* Test all `chia beta` commands
* Use a separate file logger for beta logs
* Write the plotting call args to the log file
* Sort potential submissions
* Some refactoring around log file log handler creation
* JSON dump the plotting args
* Remove mempool.additions
* Don't re run the program, and remove program from mempool item
* Removals only stores item ids, and stores a list
* Move pending cache down to prevent cache dos
* Separate validation from adding to pool, and remove mypy exceptions
* Fix bug with replacing
* Add to mypy
* Revert cbgui
* precommit fail
* Properly update the seen dict
* lint error
* Fix mempool bug
* Enable mypy for a bunch of files
* Update after merge with main
* 99/260 remaining
* Address comments
* Enable more mypy
* Merge conflict mypy file
* More fixes
* pytest.ini and cb-gui
* One more flake8 fix
* Flake8 and tests
* More test fixes
* isort
* Make LGTM happy
* Update chia/full_node/block_store.py
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Update chia/full_node/full_node_store.py
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Update chia/wallet/util/peer_request_cache.py
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Apply all suggestions from PR comments
* Revert cb-gui
* 100 back to 1000, minor tweaks
Co-authored-by: Kyle Altendorf <sda@fstab.net>
* Add change_data to _state_changed since the later calls expect it
* Add default change_data of `None` to the rpc_service state_changed method (everywhere else has this default)
* Change data is optional
* Simplify service start configuration loading
* oops
* more oops
* rename `config` to `service_config` as appropriate
* rename `full_config` to `config`
* Remove _set_state_changed_callback from harvester_api.py as its never actually called
In rpc/rpc_server.py:318, what is actually called is rpc_server.rpc_api.service._set_state_changed_callback, so the version of this in harvester.py is what is actually used
* Remove duplicate initial value for state_changed_callback in init
* _state_changed -> state_changed so it can be called from harvester_api as well
* Add farming info state_changed event from harvester
* Add time to farming_info event
* Add [pre_]validation_time to block event
* Remove unused set_state_changed_callback on full_node_api. This is actually called on full_node_rpc_api.service
* Remove unused set_state_changed_callback on crawler_api. This is actually called on crawler_rpc_api.service (crawler)
* Remove unused set_state_changed_callback on farmer_api. This is actually called on farmer_rpc_api.service (farmer), not the api itself
* Add state changed event for submitting a partial
* Lint fixes
* streamable: Merge `strictdataclass` into `Streamable` class
* tests: Test not supported streamable types
* streamable: Reorder decorators
* streamable: Simplify streamable decorator and force correct usage/syntax
* streamable: Just move some stuff around in the file
* streamable: Improve syntax error messages
* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion
* streamable: Use cached fields instead of `__annotations__`
This is now possible after merging `__post_init__` into `Streamable`
* Introduce `DefinitionError` as `StreamableError`
* `/t` -> ` `
* asyncio.get_event_loop() is deprecated in 3.10, stop using it
https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop
> Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().
* black
* Add mempool_max_total_cost to RPC
* Add signage_point event
* Fix incorrect crawler RPC port lookup
* Set up initial timelord RPC server + finished_pot_challenge event
* Add new compact proof event
* Add skipping/new_peak to track when fastest or not
* Check for None on change_data
* Add skipping_peak + new_peak to changes for metrics
* Convert chain to value
* Rename iters
* Timelord RPC to 8557 - 8556 is used in simulation tests
* Make tests work with RPC server on timelord
* Change event name to finished_pot
* Use broadcast_farmer object
* Move state changed for `finished_pot` after proofs_finished.append
* Fix type on ips var + add vdf_info and vdf_proof
* fix event name on the state_changed function
* Add crawler RPC server
* Generate private keypair for crawler
* Bring over cleanup changes from the last closed PR
* Update the crawler RPC information to be its own subsection within seeder
* Add sleep before crawling to let the daemon connection get set up
* Wait for the actual callback to not be None, instead of just a random sleep interval
* Rework crawler/dns seeder to use the daemon + normal chia start process rather than the old system intended for the standalone repo
* Update configure testnet to work with seeder config
* Add back the crawler/seeder options from the standalone version
* Remove the check for none/sleep. Not needed when this is started by the daemon
* Add real data to the get_peer_counts endpoint
* Lint
* Fix calls to configure from init
* Turns out we still might sometimes move too quick before daemon/state changed callback is ready
* Add peer counts in the state_changed callback method
* Add a setting for peer_connect_timeout in the seeder: section so we can control it just for crawler
* start_seeder
* Pass config/root_path to the DNSServer so it can also use the configured crawler DB Path
* change in () instead of if/or
* Remove unnecessary return
* Remove initial writes to DB, since there is nothing new to write at this point
Both of these methods are called later after each crawler iteration. The lines being deleted write all the same data back to the DB right after the initial data was loaded FROM the DB in the "create" method call, which is a waste of time, especially for mainnet where there are hundreds of thousands of records to write back
* Add additional logs around DB writes
* Add a retry around writing to the DB in case something else has a lock on the DB
Metrics processes read from this DB periodically and sometimes things time out just right and we get a lock exception. Without this retry, we'd hit the other exception handler which doesn't allow the loop to continue
* initial hack
* crawler
* add pytz
* Checkpoint.
* Catch some bugs.
* Localhost dig working.
* Checkpoint: return only high quality nodes.
* Statistics.
* Try improving finding reliable nodes.
* Bug.
* Move db to memory.
* Timestamp in the last 5 days.
* Increase crawl parameters, 180+ connections per sec.
* Bug.
* Optimize for DNS traffic.
* Prepare for hosting.
* Minimum height.
* Typo.
* Try catch everything.
* dnslib.
* Add db, format code.
* nits.
* No connections for the dns server.
* Rename src -> chia
* Fix some issues with v1.1
* Crawler task pool.
* Optimize closing connections.
* Split crawler and dns server.
* Install instructions.
* Catch startup bug.
* Try a big timeout for lock aquire.
* lint.
* Lint.
* Initial commit extended stats.
* Simplify code.
* Config.
* Correct stats.
* Be more restrictive in crawling.
* Attempt to fix stats bug.
* Add other peers port to config.
* Update README for the config.
* Simplify crawl task.
* Fix bug on restarts.
* Prevent log spamming.
* More spam prevention.
* Fix bug.
* Ipv6 (#1)
* Enable ipv6.
* Fix bug.
* Use numeric codes for QTYPE.
* ANY working.
* More spam prevention.
* Try to improve IPv6 selection.
* Log IPv6 available.
* Try to crawl more aggresive for v6.
* rename dns.py to crawler_dns.py so it doesn't conflict with imported package names
* Remove pytz package off dependencies
* Tidy-up ws_connection.py
* Fix spelling
* Reinstate chia-blockchain readme, with additional lines pertaining to the DNS introducer & crawler
* More detailed info in the README wrt Chia Seeder
* Nit
* More memetic naming of Chia Seeder
* Nit
* Add entry points
* Add entry in packages
* Patch some methods on the upstream server
* Update peer record fields
* Standard library imports first
* Crawler API check
* Reconcile crawl store
* Account for crawler_db_path in config
* Await crawl store load DB and load reliable peers
* Updates to crawler
* Rename to dns_server
* Crawler-specific overrides for the chia server
* Edit comment
* Undo changes to ChiaServer in view of crawler-specific overrides introduced in previous commit
* Nit
* Update service groups
* Expand name maps, mostly
* Fix the init config
* Remove unused import
* total_records unused at this stage
* Remove ios_reliable in peer_reliability table
* Remove row[20] entry
* Split overly long line
* Fix
* Type hint for ns_records
* Reconcile mismatch btw type int and uint64
* Type annotations in crawler
* Check whether crawl store is set
* Remove upnp_list
* Lint
* Chia Seeder CLI
* Lint
* Two white spaces
* 3rd party package import
* Cleaner way to handle overrides for ChiaServer method
* Address linter warnings
* Rename
* Nits
* Fix
* Change port #
* Most chia_seeder commands up and running
* Rename
* Progress of sorts
* Fix
* Improve legibility
* Fix naming
* Fix setup.py
* Lint
* None -> ''
* Remove whitespace
* Rename
* Log ipv6 better. (#9227)
* Log ipv6 better.
* Lint.
* -
* Undo GUI changes
* Another attempt
* GUI changes
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
