Commit Graph

21 Commits

Author SHA1 Message Date
Kyle Altendorf
90c1441af0
apply pyupgrade --py38-plus --keep-runtime-typing (#14646) 2023-10-24 12:01:23 -05:00
Izumi Hoshino
85d14f561a
Added compression level and harvesting mode to harvester protocol/mes… (#15776)
* Added compression level and harvesting mode to harvester protocol/messages

* Added test

* Fixed lint error
2023-07-18 16:24:56 -05:00
Izumi Hoshino
1f9081a225
Added total_effective_plot_size for get_harvesters RPC API (#15770)
* Added `total_effective_plot_size` for `get_harvesters` RPC API

* Fixed lint error

* Change the type of `total_effective_plot_size` in Receiver to `int`

* Fixed lint error

* Fixed lint error 2

* Changed text

* Added test

* Fixed lint error

* Fixed lint error 2

* Fixed a test issue

* Added config.py to tests/cmds
2023-07-18 12:36:12 -05:00
dustinface
ebc5f3c124
util: Improve list_to_batches (#15415)
* Make `list_to_batches` work with collections instead of lists only

* Move `to_batches` into `chia.util.misc`

* Only support `set` and `list`

* Drop `tests.generator.test_to_batches` exclusion

* Improve type restrictions and be more coverage friendly in tests
2023-06-06 12:00:07 -05:00
dustinface
f0006b5353
server: peer_host + peer_port -> peer_info in WSChiaConnection (#14151) 2023-04-12 11:14:38 -06:00
Kyle Altendorf
3a5c1a5bd5
hint chia.harvester.harvester, harvester_api (#14690)
* hint chia.harvester.harvester

* hint chia.harvester.harvester_api
2023-02-28 11:42:58 -06:00
Kyle Altendorf
c678f00acb
cleanup adjacent string literals (#14650) 2023-02-22 17:16:07 -06:00
dustinface
2706f5995b
server: Enable and fix mypy in ws_connection.py (#13878)
* server: Enable and fix `mypy` in `ws_connection.py`

* Apply suggestions from code review

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Tweak error message

* Tweak formatting

* Make `WSChiaConnection.close_callback` optional

* Tweak assert message

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Don't provide a default for `close_callback`

* Adjust assertion

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-11-12 14:18:11 -06:00
Kyle Altendorf
3b084a165b
configure isort to add the future annotations import (#13327)
* configure isort to add the future annotations import

* apply the new isort setting

* remove type ignores for new mypy (#13539)

https://pypi.org/project/mypy/0.981/

* another
2022-09-30 03:40:22 -05:00
dustinface
40538fd942
mypy: Fix some asyncio.Task related ignores (#13487) 2022-09-23 17:19:36 -05:00
dustinface
60908420b6
plot_sync: Change log level to INFO in Receiver.reset (#13463)
It's not necessarily an error, could be just a harvester restart.
2022-09-16 16:21:51 -05:00
dustinface
3f740d375c
plot_sync: Fix type hints in Receiver._process (#11890)
* plot_sync: Fix type hints in `Receiver._process`

* Move `PlotSyncMessage` and `T_PlotSyncMessage` into `plot_sync.util`
2022-07-06 21:02:09 -05:00
dustinface
a221726770
server: Hint data parameter of make_msg (#11876) 2022-06-13 11:41:22 -05:00
dustinface
3e3dafb0ef
farmer: Improve logging in plot_sync.{receiver|delta} (#11474)
* Add the `node_id` to the logs in `Receiver`

* Log all processed messages

* Log reset calls

* Improve `reciever.Sync` and `delta.Delta` logging
2022-05-24 16:21:56 -07:00
Kyle Altendorf
a9f7cce06b
Skip plot_sync_callback if delta is None, hint Harvester.server (#11620)
* hint Harvester.server as Optional[ChiaServer]

* use a Protocol instead of Callable

* oh right, Optional
2022-05-24 13:21:14 -07:00
dustinface
fc2d6d3db0
harvester: Fix deadlock on disconnect after a repeated plot sync start (#11481)
* Test plot sync start and disconnect while a sync is in progress

* Drop `Receiver._lock`, instead wait for `Receiver.sync_active` to clear

* Properly stop the sender and plot manager refreshing on disconnect

* Poll faster, drop the log

* Reset `_sync_id` at the very end of `_finalize_sync`
2022-05-12 15:16:22 -07:00
dustinface
0d5251c3a7
farmer|gui: Enable paginated plot loading and improved state reporting (#11367)
* farmer: Adjust notifications from the farmer to the UI

- Only send the data for the harvester which actually sent an update
- Notify for each loaded batch during initial loading

* Enable improved farmer/harvester GUI
2022-05-03 15:33:38 -07:00
dustinface
387abc3d8e
farmer|rpc: Some changes to get_harvesters{_summary} RPC endpoints (#11342)
* farmer|rpc: Always add `last_sync_time` in `Receiver.to_dict`

This changes the resonse of `get_harvesters` and
`get_harvesters_summary` to always include that field with value
`None`/`null` if the receiver is not yet synced.

* plot_sync: Add sync state to dict in `Receiver.to_dict`

* Add `total_plot_size` to `Receiver.to_dict`

* Refactor sync object assertions

* Access `_current_sync` directly

* Generate `syncing` data outside
2022-04-29 18:34:14 -07:00
dustinface
270c69eb5d
plot_sync: Introduce receiver.Sync (#11267)
* plot_sync: Introduce `receiver.Sync`

* Use `dataclasses.replace`

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-29 08:16:39 -07:00
dustinface
b693aeb407
farmer|rpc: Introduce get_harvesters_summary RPC endpoint (#11245) 2022-04-28 19:36:21 -07:00
dustinface
ded9f68583
chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00