Commit Graph

739 Commits

Author SHA1 Message Date
William Allen
7b3502d66d
Revert "Add a workflow for scanning python deps" (#13959) 2022-11-18 18:05:51 -06:00
Patrick Maslana
3635df1d2f
Add a workflow for scanning python deps (#13227)
* Add a workflow for scanning python deps

* Removed the requirements.txt file from the commit

* Change to any pull request for running the workflow

* Remove some wording and fix concurrency

* Change python version

* Add a step to pip install requirements.

* Adding skip unresolved when pip installing requirements.txt

* Pip install ignore installed

* Adding a step to update pip

* Fix spacing

* Running each line for the pip install.

* Adding continue-on-error when pip installing

* using skip-unresolved with when monitoring with snyk.

* Remove pip install step and add names to steps

* Removed leading line and set "Cancel-in-progress" for concurrency to true

* Change when the workflow runs to pushes and PRs

* Update .github/workflows/snyk-python-scan.yml

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-11-18 10:42:15 -06:00
Kyle Altendorf
22a1d1b1c2
use aws-actions/configure-aws-credentials@v1-node16 to clear warnings (#13950)
https://github.com/aws-actions/configure-aws-credentials/issues/489#issuecomment-1278145876
2022-11-18 03:25:30 -06:00
Kyle Altendorf
a76336435f
pre-commit -> timeout-minutes: 20 (#13949)
https://github.com/Chia-Network/chia-blockchain/actions/runs/3492945647/jobs/5847234861

Found a hung run that had been going for 5 hours.  A quick look suggests 2-6 minutes is normal with an outlier as high as 11 minutes.  But that was a _quick_ look.
2022-11-18 03:25:02 -06:00
William Allen
282fe9c6ef
Update workflows to use GITHUB_OUTPUT (signed commits) (#13911)
* Update build-windows-installer.yml

Use GITHUB_OUTPUT instead of set-output.

Use PowerShell syntax for PowerShell, Bash Syntax for Bash.

Why?
GitHub is deprecating set-output as per their blog:

https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Signed-off-by: wallentx <william.allentx@gmail.com>

* Use GITHUB_OUTPUT in workflows

Use GITHUB_OUTPUT in workflows instead of set-output.

Why?
GitHub is deprecating set-output as per their blog:

https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Signed-off-by: wallentx <william.allentx@gmail.com>

* Workflow dispatch added to MacOS and Linux ARM

Signed-off-by: wallentx <william.allentx@gmail.com>

* Fix Tests for Windows

Use PowerShell syntax for Windows, Bash for others.

Signed-off-by: wallentx <william.allentx@gmail.com>

* Use setup-python@v4

Update to resolve the set-output warnings

Signed-off-by: wallentx <william.allentx@gmail.com>

* Fix Tests and Benchmarks for Windows

Signed-off-by: wallentx <william.allentx@gmail.com>

* Update test.yml

Fix missed setup-python@v2

Signed-off-by: wallentx <william.allentx@gmail.com>

* Use Chia setup-python wrapper

Signed-off-by: wallentx <william.allentx@gmail.com>

* Revert "Fix Tests and Benchmarks for Windows"

This reverts commit 4fbc6d354b.

Signed-off-by: wallentx <william.allentx@gmail.com>

* Revert "Fix Tests for Windows"

This reverts commit 5860048344.

Signed-off-by: wallentx <william.allentx@gmail.com>

* Use bash for all OS to get pip cache dir

Signed-off-by: wallentx <william.allentx@gmail.com>

* Fixing windows workflow error

* Replacing instances of Out-File

* Fixing ps1 -> bash

* Fixing path when ps1 -> bash

* Escaping Windows path when using bash

* Escaping Windows path when using bash

Signed-off-by: wallentx <william.allentx@gmail.com>
Co-authored-by: zcomputerwiz <zcomputerwiz@users.noreply.github.com>
2022-11-17 21:44:12 -06:00
Chris Marslender
c9758cc476
Trigger sync tests for all release types (#13942)
* Trigger sync tests for all release types

* Temporary fix unit test

Co-authored-by: ytx1991 <t.yu@chia.net>
2022-11-17 20:18:56 -06:00
Kyle Altendorf
26171c2a7f
retryAfter: 5 (24x) for conflict check (#13422)
* retryAfter: 2 for conflict check

* two minutes

* Use 'Chia-Network/actions/label-conflict@main' for conflict check action

Co-authored-by: ChiaMineJP <admin@chiamine.jp>
2022-11-17 03:23:34 -06:00
Chris Marslender
966e67e81b
Set up commit signing for the mozilla CA updates (#13935) 2022-11-16 20:05:17 -06:00
Kyle Altendorf
fe04df9f68
less pip install and activate for windows build (#13845) 2022-11-15 18:47:24 -06:00
William Allen
e29fcaaa01
Adding new PR template, release note config, label exclusion (#13802)
* Adding new templates

* Scoping permissions of third party action

* Modifying label names

* 'Mojifying job name

* Cant parse emojis 😭

* Modified -> Changed

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Jeff <jeff@chia.net>

* Adjusting PR template format

* Adding merging notes to PR template

* Fixing whitespace

Co-authored-by: Jeff <jeff@chia.net>
2022-11-10 23:03:26 -06:00
Kyle Altendorf
15111cf196
use install action in checks and upload workflow (#13846)
* use install action in checks and upload workflow

* correct python version specification
2022-11-10 14:05:37 -06:00
William Allen
304f7ef447
Fixing test Arch install (#13850) 2022-11-07 12:00:38 -06:00
Amine Khaldi
14ab6a3388
Merge commit '0759bc7d4c3cebf211b14e796f6af153b1ed6022' into checkpoint/main_from_release_1.6.1_0759bc7d4c3cebf211b14e796f6af153b1ed6022 2022-11-05 20:50:17 +01:00
William Allen
63fcfd8000
Fix missing bladebit install for DMG build job (#13829) 2022-11-04 21:01:43 -05:00
William Allen
0759bc7d4c
Adding missing env value for osx torrent creation (#13813) 2022-11-03 19:12:31 -05:00
Kyle Altendorf
e882061648
Add 🚨 to the dependency review workflow (#13782) 2022-11-01 15:44:12 -05:00
Kyle Altendorf
3c255f17c4
always use Chia-Network/actions/setup-python (#13779) 2022-11-01 15:43:54 -05:00
William Allen
2eebd6c448
Fix CI publishing (#13725)
* Fix ref check to github.ref instead of github.event.ref, so its always present (#13661)

* Fixing s3 upload conditional for windows builds (#13658)

Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2022-10-27 12:26:46 -05:00
Kyle Altendorf
51b951e957
simpler python exclusions (#13667)
* Revert "Set up json files for python versions, to run a more restricted set on private testing repos (#13641)"

This reverts commit aff3cf5c10.

* hard coded poc

* ${{ 'limited' }}

* env.MATRIX_MODE

* via a the existing configure step

* and ad the parameter

* also define the output

* conditional on not being the main repo
2022-10-23 13:59:08 -05:00
Kyle Altendorf
2ab1c4a65f
Create PRETTY_GOOD_PRACTICES.md (#13600)
* Create PRETTY_GOOD_PRACTICES.md

* trim trailing whitespace

* typo touchups

* correct dataclasses heading format

* correct inheritance heading

* drop nft dataclass conversion example

* consume

* manager

* Update CODEOWNERS

* tab ... ?

* Update PRETTY_GOOD_PRACTICES.md

* Update PRETTY_GOOD_PRACTICES.md
2022-10-20 16:08:18 -04:00
William Allen
2f3bd49d71
Fixing s3 upload conditional for windows builds (#13658) 2022-10-19 14:17:58 -05:00
Chris Marslender
1f19ac2de5
Fix ref check to github.ref instead of github.event.ref, so its always present (#13661) 2022-10-19 14:00:53 -05:00
Chris Marslender
aff3cf5c10
Set up json files for python versions, to run a more restricted set on private testing repos (#13641) 2022-10-19 11:51:26 -05:00
Chris Marslender
440b70a152
Fix secret check (#13597)
* Fix secret check for windows/mac (which have more granular checks because of different secret types)

* Make all the workflows use granular secret checks

* Fix whitespace issues
2022-10-04 17:11:21 -05:00
Chris Marslender
be4ada9b24
Add Bladebit prerelease to prerelease mac builds (#13540)
* Add bladebit for pre-release mac builds. Update pyinstaller to include bladebit if the file exists

* Make plotter inclusion conditional based on if they are present for the build or not

* temp: always get pre-release bladebit plotter for testing

* Only include bladebit on pre-release
2022-09-26 18:35:33 -05:00
William Allen
3023daf435
Changing conditional of start-release (#13501) 2022-09-23 17:31:28 -05:00
Kyle Altendorf
199d989352
Avoid the auto-naming mess by saying more about the coverage job in its name (#13492)
```
Coverage (🐧, ubuntu, Ubuntu, ubuntu-latest, 3.9, 3.9, 3.9, 3.9, 3.9)
```
to
```
🐧 Coverage 3.9
```
2022-09-21 22:50:40 -05:00
William Allen
357075235a
Wallentx/prerelease (#13460)
* Adding workflows to support prerelease

Signed-off-by: wallentx <william.allentx@gmail.com>

* Update installer-version.py

Signed-off-by: wallentx <william.allentx@gmail.com>

* using gh cli to upload release artifact

Signed-off-by: wallentx <william.allentx@gmail.com>

* Fixing whitespace

* Adding gh cli workflow uploads

* Removing CLI artifact upload to release page

* Adding separate release artifact upload for Windows

* Fixing windows installer envar

* Moving torrent release upload step

* Adding needed env for artifact upload

* Removing windows installer conditional for installing AWS CLI

Signed-off-by: wallentx <william.allentx@gmail.com>
2022-09-21 14:11:11 -05:00
Kyle Altendorf
a9cce0f6e0
add final coverage handling (#11778)
* add final coverage handling

* switch to development coverage

* correct url

* only run one directory for now

* only 3.10

* delete windows data for now

* drop the 35% failure point

* recover full test matrix

* Use regular coverage

* Update test.yml

* put coverage reporting into an artifact

* oops

* oopsy

* pytest-cov

* maybe

* maybe help out flakes

* farm the spend bundle...

* some more

* more funds for the did test?

* more...

* report chia/ and tests/ separately

* job_timeout = 45 for core

* tidy and disable diff-cover

* remove job_timeout = 45

* --rcfile=.coveragerc

* Update .github/workflows/test.yml

* revert time and blocks

* assert spend is in mempool, lower times and block count

* remove sleep

* lint

* up timeout

* cat test - farm blocks until spend is included

* add spend check for nft endpoints

* fix nft endpoint test

* remove unused code

* 4

Co-authored-by: almog <almogdepaz@gmail.com>
2022-09-16 14:36:37 -05:00
Kyle Altendorf
c02e57c4aa
Update to conflict check v2.0.1 (#13424)
https://github.com/eps1lon/actions-label-merge-conflict/compare/v2.0.0..v2.0.1
2022-09-12 18:37:26 -05:00
Kyle Altendorf
8aa0e7da90
Update test-single.yml (#13419)
Even the blocks and plots cache appears to finish in its entirety in 15 seconds, so the 1 minute 'segment' (not sure exactly what that means) timeout seems reasonable.

https://github.com/actions/cache/issues/810
2022-09-12 16:52:46 -05:00
Kyle Altendorf
789ee4f415
Add .clvm.hex pre-commit check (#12050)
* Add .clvm.hex pre-commit check

* report all results

* stuff

* maybe pass

* add todo

* explicitly define suffixes

* check for (and remove) extra hex and hash files

* remove garbage failure hex and hash files

* provide runner scripts

* matrix pre-commit checks

* maybe now

* more cross platform

* move it

* oops

* shell:

* maybe

* more configurable

* explicitly specify powershell

* tidy

* remove other stuff

* tidy

* remove existing pytest clvm hex etc checker

* add a trailing newline to decompress_block_spends.clvm.hex.sha256tree

* Update activated.ps1

* git config --global core.autocrlf false

* Update pre-commit.yml

* only check `chia/`

* typing_extensions

* typing_extensions

* removing unused typing import

* move top_level out of the rglob

* catchup dl clvm hex files

* Revert "catchup dl clvm hex files"

This reverts commit cd3d4f70b8.

* now
2022-09-12 16:08:55 -05:00
dustinface
030082e350
github: Drop incorrect/leftover statement in test.yml (#13070) 2022-09-08 13:46:31 -05:00
William Allen
d51d74a63c
Bumping codeQL actions to v2 (#13344) 2022-09-06 14:41:27 -05:00
Chris Marslender
16fb814b22
Don't try to sign mac installers when secrets are unavailable (#13166) 2022-08-25 15:59:58 -05:00
Izumi Hoshino
b3b56a64d1
Replace electron-packager with electron-builder (#12953)
* Install electron-builder

* Replaced electron-packager with electron-builder for windows

* Upgraded electron-builder version for mac

* Replaced electron-packager with electron-builder for Linux and integreated deb/rpm dir

* Updated mac installer job

* Fixed app.asar location

* Fixed dmg name for arm64 mac

* Updated installer-version.py

* Updated Linux(deb) installer CI job

* Updated Linux(rpm) installer CI job

* Fixed Linux(rpm) install CI issue

* Revert to use `electron-packager` instead of `electron-builder` for arm64 .deb installer

* Fixed lint errors

* Fixed rpm installer job issue

* Updated Windows installer CI job

* Fixed rpm installer CI job

* Simplified arm64 build

* Added -y option to `apt install` command

* Added `dmg-license` as npm build dependency for MacOS

* Attempt to set product name appropriately for each platform

* Specified productNames in commands

* Fixed arm64 installer issue

* Stopped to create link '/opt/chia/daemon'

* Updated desktop app name for linux installers

* Updated packageName for linux installers

* Abort build jobs when 'cd' fails

* Updated the GUI submodule ref (main:5e52b7f53478d4f39b4a0f0203f41e31dd53aee9)

Co-authored-by: Jeff Cruikshank <jeff@chia.net>
2022-08-24 20:13:40 -05:00
William Allen
e8662c3c48
dependency-review (#13153) 2022-08-24 17:14:44 -05:00
Chris Marslender
8f7a610fb0
Only try and install aws cli if its not already present, since this is really slow on 10.14 (#13098) 2022-08-22 17:00:12 -05:00
Patrick Maslana
1a673c9453
Change the pattern for running pre-commit. Previously, this was using… (#12891)
* Change the pattern for running pre-commit. Previously, this was using a deprecated action workflow.

* Fix the python version for the - uses: ./.github/actions/install step

* Added --all-files and --verbose to the pre-commit run command

* Precommit is already installed as part of the dev dependencies. Removing the installation step.
2022-08-17 01:00:35 -05:00
Chris Marslender
535d21e679
Combine m1 and intel mac installers to a single workflow (#12890)
* Combine m1 and intel mac installers to a single matrix workflow, since we have native m1 runners now and the jobs are nearly identical

* Fix lint issues

* Fix more lint

* Use setup node
2022-08-10 16:04:59 -05:00
William Allen
a4b0f51f3d
Adding required-reviewers as CODEOWNERS (#12812) 2022-08-09 18:27:32 -05:00
Chris Marslender
87179bd673
Update the arm64 deb installer to also use the 1804 builder container (#12864)
* Update the arm64 deb installer to also use the 1804 builder container

* Bring steps more inline with the other docker workflow

* Fix github_workspace variables and make timeouts the same
2022-08-09 16:48:53 -05:00
Chris Marslender
6b0b7d72c1
Swap intel installer to 10.14 runners (#12735)
* Swap intel installer to 10.14 runners and swap to setup python wrapper forced to pyenv (since github's python versions don't support 10.14)

* Self-hosted cleanup steps

* Update to chia-network notarize cli
2022-08-09 13:10:22 -05:00
Izumi Hoshino
48a8347c83
Fixed an issue where it sometimes tried to remove a non-existent win cert file (#12797) 2022-08-05 13:25:09 -05:00
Izumi Hoshino
4c87c99d15
Improve CI performance (#12523)
* Optimized CI execution times

* Fixed a lint error

* Update GUI cache keys for linux

* Added `git status` and `git submodule` to build_xxx-installer.sh
2022-08-04 12:03:50 -05:00
Patrick Maslana
e7c0bcfcb4
Remove test pypi upload step (#12749) 2022-08-02 07:44:53 -07:00
Kyle Altendorf
a0f27537a9
Mark workspace safe in pre-commit workflow (#12690)
https://github.com/Chia-Network/chia-blockchain/runs/7587001625?check_suite_focus=true
2022-08-01 15:18:44 -05:00
Kyle Altendorf
189790ced2
Drop EOL impish and hirsute (#12559) 2022-07-23 22:51:41 -05:00
Kyle Altendorf
11b210809b
run pre-commit across platforms in ci (#12287)
* run pre-commit across platforms in ci

why?  because we have developers working on multiple platforms.

* pre-clean for pre-commit in ci

* chia setup python action

* fixup

* fix windows specific hinting

* no TypeAlias yet

* debug

* debug

* adjust activated.ps1

* verbose pre-commit

* exit better

* Revert "debug"

This reverts commit 93a100afc7.

* less debug

* less `|`

* Update .pre-commit-config.yaml

* Update activated.ps1
2022-07-15 21:08:07 -05:00
Kyle Altendorf
9a4316f96e
fixup (#12292)
* fixup

* add automated option to install action, default true

* try again

* make Install.ps1 accept -a

* oof

* again

* backup
2022-07-07 23:43:22 -05:00
Kyle Altendorf
e5779eb968
Increate macOS Intel installer build timeout to 55 (#12285)
There are plenty of runs in the high 30s and even low 40s lately.
2022-07-07 20:03:21 -05:00
Kyle Altendorf
51c6590cf5
add install action (#12075)
* add install action

* add the actual action file

* update pre-commit to use the install action

* add prefix for arch -arm64 use

* typo

* outputs:

* copy paste correction

* false

* hmm

* there

* if ("$env:INSTALL_PYTHON_VERSION" -ne "")
2022-07-07 19:56:33 -05:00
Kyle Altendorf
4197b39987
use a variable in the workflow for blocks and plots version (#12209)
* use a variable in the workflow for blocks and plots version

* Update benchmarks.yml
2022-07-06 17:24:47 -05:00
Kyle Altendorf
ee19b7e1fd
move py3createtorrent to development dependencies (#12074) 2022-06-28 11:51:04 -05:00
Amine Khaldi
3c52be6a55
Deduplicate pre-commit Python linting from the super linter. (#12084) 2022-06-27 14:57:14 -05:00
Kyle Altendorf
0aa2019526
make pre-commit mypy run use regular venv (via activated shim) (#12077)
* make pre-commit mypy run use regular venv (via activated shim)

* & not Invoke-Expression

* Update activated.ps1

* install for pre-commit

* Update activated.sh

* Update .pre-commit-config.yaml
2022-06-27 12:13:22 -05:00
Kyle Altendorf
5aaf07a686
run pylint directly (#12100)
* run pylint directly

* fixup
2022-06-27 10:53:04 -05:00
Kyle Altendorf
db04a90eef
add workflow emoji (#11988)
* add workflow emoji

* move from job name to workflow name

* move from test-single to test

* some more
2022-06-21 14:51:01 -05:00
Chris Marslender
36c58e9806
Rename all references to "latest beta" to "latest dev" to match the naming on the website (#11878) 2022-06-13 10:16:23 -05:00
Kyle Altendorf
148b0f01d7
Remove flake8 from superlinter workflow (#11844)
flake8 is already run in CI both via pre-commit and directly.
2022-06-08 23:38:41 -05:00
Kyle Altendorf
458697b519
reusable workflow for a single test workflow (#11722)
* reusable workflow for a single test workflow with coverage collection

* avoid needing typing-extensions to build workflow jobs

* remove incorrect coverage job

* correct enabling of pytest-monitor

* $Env:GITHUB_*

* tweak concurrency

* add explanation for use of the reusable workflow

* name: Generate matrix configuration

* correct timelord installation choice logic

* avoid * to fix coverage combining on windows

* name: Configure matrix

* bring back the conditional run

* less matrix.os.matrix

* hmm

* again

* exclude instead

* fixup JOB_FILE_NAME

* leave the final coverage step for later

* drop configuration.runs in favor of matrix excludes

* fixup os checks

* actions/cache@v3
2022-06-03 16:56:52 -05:00
Arvid Norberg
3c203e335c
add regression test for json formatting of protocol messages (#11770) 2022-06-03 12:34:03 -05:00
Amine Khaldi
ff4e4795e8
Update workflows. 2022-06-01 23:34:33 +01:00
Amine Khaldi
dcb343ada1
Merge release/1.4.0's 8defd89 into main 2022-06-01 23:32:59 +01:00
Amine Khaldi
c924fc7ab2
Merge release/1.4.0's 7b104f3 into main 2022-06-01 18:45:22 +01:00
Amine Khaldi
050c159a7a
Update workflows. 2022-06-01 15:09:53 +01:00
Amine Khaldi
340cecb16c
Merge release/1.4.0's d0de803 into main_checkpoint_1.4.0_d0de803 2022-06-01 10:37:02 +01:00
Amine Khaldi
978e6926f0
Update Windows clvm test workflow. (#11720) 2022-05-31 17:39:06 -05:00
Amine Khaldi
841abe1600
Merge gw_nft_offer_drivers into release/1.4.0 2022-05-31 22:34:29 +01:00
Kyle Altendorf
9e30707fac
Show test output in benchmarks workflow (#11601) 2022-05-31 15:36:22 -05:00
Kyle Altendorf
8b2f7ec24c
Add Windows CI (#10681)
* Add Windows CI

* rebuild workflows

* rebuild workflows

* try different activation

* catch up windows workflow template

* debug dir

* hard code dev extra on windows

* it is pytest

* .exe

* -m pytest for windows

* --module pytest

* Revert "debug dir"

This reverts commit 74b5bbfd92.

* debug pkg_resources import on windows

* debug tests.conftest import on windows

* rebuild workflows

* debug

* debug oops

* debug more

* debug more

* Update pytest.ini

* Update pytest.ini

* fix it

* Revert "debug more"

This reverts commit 367ed61dfa.

* Revert "debug more"

This reverts commit 772b6707c6.

* Revert "debug oops"

This reverts commit 5065c1e172.

* Revert "debug"

This reverts commit ecf721e453.

* Revert "debug tests.conftest import on windows"

This reverts commit ea71a51625.

* Revert "debug pkg_resources import on windows"

This reverts commit eb6afb1dfb.

* more warning ignore

* they're regexes...

* --numprocesses 0 for windows ci

* specify each test file individually since pytest on windows does not seem to handle the wildcards

* relative...

* select most recent supported python on windows, support INSTALL_PYTHON_VERSION

* sort test files when generating workflows

* report full python version on windows, actually use the selected version for the env

* oops

* debug dir

* skip timelord-requiring jobs on windows

* oops

* handle wildcard expansion for coverage on windows

* rebuild workflows

* download blocks and plots on windows

* correct and and or in yaml

* -d for Install.ps1

* git config --global core.autocrlf false

* add missing windows plot_sync workflow

* no check resource usage on windows

* delete outdated windows workflows that need the timelord

* catch up windows workflow template

* try again without xdist

* rebuild workflows

* Revert "try again without xdist"

This reverts commit 4bedf5f047.

* maybe now no xdist on windows

* maybe now no xdist on windows

* rebuild workflows

* debug chiapos==1.0.9

* Revert "debug chiapos==1.0.9"

This reverts commit 4b374e128a.

* debug blspy==1.0.9

* catch up windows workflow template

* rebuild workflows

* Various windows CI fixes

* Windows CI fixes and workflow updates

* ignore another warning for the keyring

* pre-commit fixup

* rebuild workflows

* catchup windows test workflows with long_lived/** and release/** setup

* Update test_full_sync.py

* Try without parallelization

* Some other tweaks for windows

* some more timeout tweaks

* more timeout tweaks for windows

* rebuild workflows

* Adjusting more timeouts

* Skip ipv6 test on windows

* More timeout tweaks

* More timeout adjustments

* More time out adjustments

* more windows tweaks

* Try running some things with -n 0

* Fix up windows test template and re-gen workflows

* Try this again

* Try all windows tests with -n 0

* misc update

* Undo some timeout changes

* Adjust timeout

* Increase timeout

* Coverage combine should combine the files in the current directory

* Trying retry test on windows

* timeout adjustment

* use tmp_path

* Few minor updates based on review

* lint fix

* Update tests/build-workflows.py

* Update tests/core/server/test_dos.py

* Try windows with pytest xdist n=2

* Check windows with xdist n=3

* Trying n=2 again

* Adjust timeout

* Update testconfig.py

* Update build-workflows.py

* Update testconfig.py

* Update .pre-commit-config.yaml

* Update testconfig.py

* Update build-workflows.py

* Update build-test-macos

* Update build-test-ubuntu

* Update build-test-windows

* skip

* Apply suggestions from code review

* tidy

* Update test_pool_config.py

Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2022-05-31 14:04:45 -05:00
Kyle Altendorf
a3e7015d16
more flake8 (#11396) 2022-05-31 14:01:44 -05:00
Amine Khaldi
8e709ae0d3
Merge nft1_soon_tm into release/1.4.0 2022-05-28 23:10:23 +01:00
Amine Khaldi
d0de8038cd
Bring DID/NFT effort into release/1.4.0 (#11662)
* initial commit

* assert coin added in test

* add support functions

* add RPC calls (untestested and surely unworking)
also change innerpuz

* missing comma

* update puzzle clvm hex sha256

* fixes

* add load_wallet for nft_wallet

* add singleton 1.1 and update innerpuz for it

* update wallet for new innerpuz

* use singleton 1.1

* swap DID to singleton 1.1

* Fix DID Wallet and DID Innerpuz

* list hint

* don't push automatically

* change to determine_coin_type

* stash debug

* fix get_hints()

* small fix en route to bigger fix later

* push for straya

* checkpoint

* coin discovered in wallet

* re-enable push of attests because message spends are different

* change a few node calls for rebase and rebase

* black and flake8

* flake8 test formatting

* Add transfer test and some transfer functions

* fix bug where innerpuzzle would not recreate itself properly

* fix get_info for received NFT and reovme breakpoint

* fix messages

* debug commit

* receive and send it back!

* send it back and remove NFTs after they're sent

* fix adding duplicates bug

* mint NFT with RPC calls test

* save newer coin info over older info

* finish RPC tests

* update to new version of singleton
black

* change puzzle to remove need for receive spend when trade_price = 0

* Begin conversion to trade price list

* fix test to new trade_price_list system

* Move towards goals in chialisp

* fix clvm testss

* remove unused code

* more chialisp changes working towards CAT royalties

* much progress on CAT royalties

* round down to even on royalties

* add delay to fix inconsistent tests

* remove unused file

* add assertions and fine tune sleep

* add richer support for URIs / metadata and add periphery functions

* switch to metadata, switch to puzzle announcements

* update innerpuz commit hash

* add clvm files to recomp list

* fix rpc test

* flake8 fixes for tests

* flake 8 stupid fix

* Big change to NFT innerpuz (#11141)

* new innerpuz with passing clvm tests

* more work on url updating

* fix broken add_url tests

* fix nft_wallet generate_nft code

* more progress

* fix wallet and change to auto-send on tradeprice == 0

* flake8 and black

* NFTs are always amount 1
removed some unnecessary new_did info detritus from wallet

* move trade_price_list into tp_solution

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* Update chia/wallet/nft_wallet/nft_puzzles.py

* typing for create_full_puzzle_with_curry_params

* Typing for update_metadata

* Typing for get_trade_prices_list_from_inner_solution

Co-authored-by: Jeff <jeff@chia.net>

* Merge commit with main (9ff3fc993f) (#11137)

* Merged with main (9ff3fc993f)

* Linter and some test fixes

* isort, linting, test fix

* Fixes and bumped the GUI to point to nftdev

* Couple of fixes

* Bumping GUI to 7aa199a4fb

* Bumping GUI to 155866f6d0

* DID changes all in one  (#10559)

* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Handle recovery

* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* add clvm files to recomp list

* fix rpc test

* flake8 fixes for tests

* flake 8 stupid fix

* Bug fix & flake8 fix for NFT tests

* Modify python code

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Generalize the message puzzle

* Add Pubkey as hint

* Receive DID

* Add DID APIs & Tests

* Fix tests

* Test fixes. in_transaction is now passed as a param to the various callers that eventually call save_info.

* Workflow Fix

* Add test config for DID/NFT

* Update workflow test yaml

* Fix install test script

* Fix typo

* Resolve comments

* Fix tests

* Change did_innerpuz and fix wallets for new design (#11196)

* correcting the design of did_innerpuz and related wallet changes

* remove breakpoint comments

* change decimal point accuracy of percentage system

* secure new_amount by fixing it to our current amount

* rename and update comments for new_amount - now my_amount

* rename P2_PUZZLE to simply INNER_PUZZLE

* fix variable re-declaration for flake/merge

* black and flake8 - inclduing wallet_state_manager bug fix

* update RPCs related to add_url
added commented out tests too, but DID needs fixing first

* Fix bugs in transfer case

* Fix pre-commit

* Fix install.sh test for bookworm

Co-authored-by: matt <matt@chia.net>
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
Co-authored-by: matt-o-how <48453825+matt-o-how@users.noreply.github.com>
Co-authored-by: ytx1991 <t.yu@chia.net>

* Merge commit with main 1.3.4 (d154105a6b) (#11276)

* slight simplification of interpreting the bytes received by the timelord. avoid redundant round-trips to strings (#10316)

* move build and twine to be dev deps rather than workflow installs (#10291)

* add db validate function to check consistency of blockchain database (#10398)

* streamable: Cache stream functions (#10419)

Apply the same pattern as we have for deserialization to serialization.

This avoids all those recursive runtime lookups for "how to stream this
object" which brings a nice speedup:

```
compare: benchmark
mode         | µs/iteration old | µs/iteration new | diff %
to_bytes     | 447.57           | 193.56           | -56.75

compare: full_block
mode         | µs/iteration old | µs/iteration new | diff %
to_bytes     | 110.32           | 61.09            | -44.62
```

* Stop assuming `True == 1` (#10396)

* Stop assuming `True == 1`

* Removed unnecessary lines which confuses code readers

* simplify self_hostname. It doesn't need to depend on initial-config.yaml -> create_block_tools() -> global variable bt (#10371)

* Fix balance calculation so it works with future unconfirmed tx (#10447)

* Only update height if we have processed everything before that height. (#10415)

* Only update height if we have processed everything before that height.

* Remove debug return

* Forgot this file

* Better sorting of coin states, and disconnect peer if we are connected to trusted

* Fix disconnect, and don't mutate arguments

* Fix comment

* Update chia/wallet/wallet_node.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/wallet_state_manager.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Address comments

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* GC connections in wallet (#10450)

* Fix the year from 2021 to 2022 (#10462)

* Ms.unused code (#10453)

* Remove old backup stuff

* Remove references to CC (couloured coins) which is deprecated terminology

* removed no longer used import

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Don't load initial data older than 5 days (#10481)

* When loading initial data, don't include IPs older than 5 days in the best timestamp dict

* Don't load version data for hosts older than 5 days

* Update the DMG background image (#10289)

* Update the DMG background image

* Updated with latest design.

* Updated background.tiff with latest design

* Fancier DMG customization support via build_dmg.js

* npm_macos -> npm_macos_m1

* Pass in the app path to build_dmg.js

* Peppering with __init__.py files to satisfy the precommit hook

* Return the fees of an offer via RPC (#10480)

* farmer: Cleanup request retry and some logs (#10484)

* farmer: Bump next update times regardless of the request results

* farmer: Drop additional "success/failure" log logic

We already print the PUT response in `_pool_put_farmer` the other parts 
just lead to confusion if the pool didn't implement the PUT correct.

* farmer: Print error responses from the pool with `WARNING` log level

* Fix method name (#10500)

* updated gui to e2202874e1

* reverted to gui 800a0f6556

* updated gui to 672cf2a74a

* Only rewrite config when there is a difference (#10522)

* Only rewrite config when there is a difference

* Use variable

* Preserve existing pool payout_instructions when creating a PoolWallet (#10507)

* Preserve existing pool payout_instructions when creating a PoolWallet

* Updated the logged string when payout_instructions needs to be generated.

* Tests for update_pool_config

* isort

* Logging change, linter fixes, and more comments.

* Abort trusted sync if a state update fails (#10523)

* Abort trusted sync if a state update fails

* Fix lint

* fix updating of sub-epoch-summary map (part of BlockHeightMap) when the entirey change, including genesis changes (#10486)

* Fix propagation of errors when adding a key with an invalid mnemonic (#10274)

* Fix propagation of error messages originating from the keychain server

* Test that adding key with an invalid mnemonic returns the expected error

* Added daemon tests for the add_private_key RPC. Reverted wallet_rpc_client test as the daemon test is better suited for GUI testing.

* Reformatting updates

* Formatting change as requested by the pre-commit gods.

* catch up test_add_private_key() with dynamic ports (#10530)

* Updated background and icon positions. (#10531)

* improve the picking of free ports for tests (#10491)

* Hide balances until we are synced (#10501)

* make multiprocessing start method configurable (#10528)

* make multiprocessing start method configurable

* forkserver

* corrections

* fixup

* optional

* more optional

* stop attempting anchors in the yaml

* rework config handling

* comment

* Update URL for direct download of windows whl for upnp (#10540)

* Add incoming tx records when cancelling an offer (#10538)

* Add incoming tx records when cancelling an offer

* show fee on all txs

* wallet: Reduce log level for `Pulled from queue` message (#10529)

* pools: Drop redundant `PoolWalletInfo.from_json_dict` (#10524)

It exists the same way in its base class `Streamable`.

* correct multiprocessing start method logging, add python_default (#10547)

* correct multiprocessing start method logging, add python_default

* todo -> regular comment

* correct some comments to refer to CATs (#10548)

* correct some comments to refer to CATs

* one more

* and in a test

* Detect hints correctly in the TX (#10543)

* Detect hints correctly in the TX

* Only support list hints

* Allow CAT autodiscovery, refactor CAT default naming (#10308)

* Allow CAT autodiscovery

* Add wallet autodiscovery test

* Use dict get for automatically_add_unknown_cats with default false

* Refactor name generation for new CATs into one place

* Remove hardcoded default cat wallet name from wallet rpc test

* initial-config.yaml comment nit

* Wallet consistancy (#10532)

* use db transaction, -1 in synced up to height, delete unused funcitons

* use transaction info in key-val-store/pool-store

* cat stores

* db lock

* remove unused lock, set synced not always in transaction

* fix store tests

Co-authored-by: wjblanke <wjb98672@gmail.com>

* benchmarks: Implement streamable data comparison (#10433)

* Resubmit peak to timelord for failure. (#10551)

* Initial commit resubmit peak to timelord.

* Change how toandle exception.

* Add some logging. (#10556)

* Fix `install-gui.sh` (#10460)

* Fixed an issue where running  with gui git branch specified failed

* Fixed an issue where install-gui.sh failed if npm>=7 and NodeJS < 16 were installed

* Fixed inconsistent npm path issue

* Fixed typo

* Add fee to changing pool, and more PlotNFT syncing fixes (#10545)

* Add fee to changing pool, and more aggressive disconnect of untrusted

* Don't publish fee TX

* Lint

* Small plotnft related fixes

* More plotnft fixes

* Apply quexington suggestion

* correct param for in_transaction

* Support get_transaction and get_transactions in plotnft

* Re-add publish_transaction and add comment

* Don't rerun additions

* adding 1.3.0 release notes to changelog (#10578)

* adding 1.3.0 release notes to changelog

* Typos. thx @paninaro

* Adding requested adjustments to changelog

* Require pytest-asyncio>=0.17.0 for @fixture() (#10560)

* Convert tests/core/util/test_streamable.py to use pytest. Remove unneeded class. (#10577)

* make sync fixtures not use async def (#10504)

This is particularly relevant in cases where the scope is not function as
that forces use of a wider scoped event loop fixture as well.

* Fixup some hidden test errors (#10442)

* Touching up changelog (#10584)

* removing known issue that was only applicable to a beta release

* Adding additional fixes to the changelog

* Adding Unreleased section to track upcoming changes

* context manager for socket in find_available_listen_port(), catch OSError (#10567)

* Bump CAT wallet test timeout to 40 minutes (#10605)

* Remove block tools and keychain globals (#10358)

* Use bt fixture

* rebase

* Use local_hostname

* Fix test_json (inheritance from unittest.TestCase)

* Use correct BlockTools fixture for test_simulation

* Pass bt fixture into cost calculation tests

* flake8

* Add missing parameters to test functions

* Fix from rebase issues

* Remove set_shared_instance

* Update comment

* Remove unneeded comments

* Remove unused code

* Remove unused code, run `multiprocessing.set_start_method("spawn")` at correct time.

* Revert unrelated change

* Set daemon_port. Teardown services in correct order.
    BIG thanks to Mariano Sorgente for debugging help.

* Add back type signature - rebase issue

* Apply review fixes from Jeff

* Document why we need a later pytest-asyncio version

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Shutdown daemon coroutine properly

* Remove un-needed daemon_port argument

* Set chia-blockchain-gui to hash in main

* Remove connect_to_daemon_port

* Remove code that set "daemon_port" before calling `setup_daemon`

* Remove self_hostname fixture and extra self_hostname global

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Remove more unused test code

* Simplify  fixture

* a problem with using random.randint() to pick a port is that all processes (running in parallel) are seeded the same, and so pick the same ports at the same time, causing conflicts. This uses proper entropy instead. (#10621)

* `chia keys show` will default to displaying the first observer-derived wallet address. With the addition of the `-d` option, the non-observer derived wallet address can be displayed. (#10615)

* pools: Fix `plotnft claim` command's output (#10609)

If you currently claim rewards `claim_cmd` fails to print the txhash 
with the lookup hint in `submit_tx_with_confirmation`

```
Error performing operation on Plot NFT -f 172057028 wallet id: 12: 
'dict' object has no attribute 'name'
```

Because `submit_tx_with_confirmation` expects a `TransactionRecord` as 
result from its callable parameter `func` but `pw_absorb_rewards` 
returns a dict which includes the `TransactionRecord` as value for the 
key `transaction`. This PR makes sure all other methods used as `func` 
callable have the same return behaviour as `pw_absorb_rewards`. We could 
have adjusted it the other way around (only return `TransactionRecord` 
in `pw_absorb_rewards`) but then we would drop information in the RPC 
client.  


With this PR you get:

```
Do chia wallet get_transaction -f 172057028 -tx 
0x34f74a1ffd9da9a493b78463e635996fd03d4f805ade583acb9764df73355f9c to 
get status
```

* fix initial-config typo - log_maxbytesrotation (#10598)

* Fix invalid DB commit (#10594)

* Fix invalid DB commit

* More fixes

* Add raise

* Fix test

* correct spelling of genrated (#10653)

* fix [Bug] #10569 (#10571)

* Version control (#10479)

* Added version control enforcement to macOS m1

* Added enforced version control

* Added enforce version compliance

* Added enforced version compliance

* Added enforced versioning

* Updating this to include DRY internal action.

* Removed some unintended whitespace.

* Removed some unintended whitespace.

* CI re-run

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.

* plotting: Only lock while actually accessing `PlotManager.plots` (#10675)

* Fix exception when `chia keys migrate` is run without needing migration (#10655)

* Fix exception when `chia keys migrate` is run without needing migration

* Linter fixes

* Slightly different query for V2 DBs for getting compact/uncompact block counts, to ensure the available index is used / avoid a table scan (#10661)

* Check prefix on send_transaction before sending (#10566)

* Check prefix on send_transaction

* Fix failing RPC tests - xch -> txch since tests are a testnet and we enforce valid prefixes with this PR

* Ak.convert fixtures (#10612)

* Use bt fixture

* rebase

* Use local_hostname

* flake8

* Remove set_shared_instance

* Remove unneeded comments

* Revert unrelated change

* Add back type signature - rebase issue

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Set chia-blockchain-gui to hash in main

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Convert all class fixtures to top level functions

* updated gui to cdfa2b9821

* Better management of KeyringWrapper's keys_root_path when using TempKeyring for tests (#10636)

* Better management of KeyringWrapper's keys_root_path when using TempKeyring for tests.

* Move keys_root_path restoration code into `cleanup()`
Added an assert to detect if an unexpected shared KeyringWrapper is injected during a test.

* Conditionally restore keys_root_path for testing

* better TLS1.3 check (#10552)

* better TLS1.3 check

* catch ValueError instead of Exception

* Code simplification and cleanup

* a few nits in comments

* Add configuration locking (#10680)

* Add configuration locking

Extracted from https://github.com/Chia-Network/chia-blockchain/pull/10631

* note that fasteners will likely be replaced by filelock

* Fix test_multiple_writers on macOS

* create_all_ssl() doesn't need to be inside the config access lock

* add warnings about not using async within get_config_lock() get lock contexts

* no need to pre-touch the lock file

* .yaml.lock instead of just .lock

* test_multiple_writers() is sync

* Revert "add warnings about not using async within get_config_lock() get lock contexts"

This reverts commit 681af3835b.

* reduce lock context size in chia_init()

* use an exit stack in load_config()

* avoid config existence precheck

* only lock around the read in load_config()

* do not raise e, just raise

* tidy new imports

* fix queue empty check in test_config.py

* remove commented out code in test_config.py

* remove unused import

Co-authored-by: Jeff Cruikshank <jeff@chia.net>

* fix usage of the bt fixture in conftest fixtures (#10688)

* bump pre-commit mypy to 0.940 (#10672)

* remove some event_loop() fixtures (#10420)

* remove event_loop() fixtures

* flake8

* flake8

* remove sys.exit() from daemon shutdown

* bump full node test timeout.  a lot...  to see.

* fixup some tests

* back to module scope event loop fixture for test_full_node.py

* Update test_full_node.py

* Iterator...

* for the whole directory

* some fixtures back to module scope for reduced runtime

* back to 40 minute workflow timeouts

* these are being addressed separately

* updated gui to c992d07c95

* tests: Add `_PYTEST_RAISE` to fix exception breakpoints with pytest (#10487)

It's currently not possible to have the debuger stop on an uncaucht 
exception when debugging tests. With this patch, adding 
`_PYTEST_RAISE=1` to the environment variables in the pytest 
configuration template fixes this.

* Fixed test failures on Windows. (#10740)

* Convert helper method do_spend from a class method to a function (#10613)

* Remove unused test code (#10614)

* Ak.setup nodes (#10619)

* Remove unused test code

* Centralize fixture uses of setup_n_nodes

* Centralize fixure uses of setup_two_nodes

* Break up setup_nodes into setup_services, for individial services, and setup_nodes, for initializing different simulator configurations

* Sort imports

* more entropy in random listen ports (#10743)

* update chia-blockchain-gui one commit for npm build fix (#10776)

* Updating Changelog for point release (#10781)

* Updating Changelog for point release

* Adding missing changelog items

* run benchmarks separately (#10754)

* run benchmarks separately

* only run benchmarks once, with the most recent python version we support

* Change name to order of returned values. Enforce mandatory naming and inclusion of start_services parameter (#10769)

* cmds: Implement `chia rpc` command (#10763)

* cmds: Implement `chia rpc` command

* Enable "timelord" client + some refactoring to enable "crawler" client

* wallet: Fix `STANDARD_WALLET` creation for `wallet_info.id != 1` (#10757)

* wallet: Optional wallet type parameter for `get_wallets` and `wallet show` (#10739)

* wallet: Add optional `type` parameter to `get_wallets` and `wallet show`

* tests: Use the `type` parameter for `get_wallets` in pool rpc tests

* cmds: Ask for the name of the wallet type in CLI

* harvester: Reuse legacy refresh interval if new params aren't available (#10729)

* mypy 0.941 for pre-commit (#10728)

* Add maker fee to remaining offer RPCs (#10726)

* Add healthcheck endpoint to rpc services (#10718)

* Add healthcheck endpoint to rpc services

* Trailing whitespace ding

* Fix typos lastest > latest (#10720)

* fix typo in command line argument parsing for chia db validate (#10716)

* New RPC get_coin_records_by_hint - Get coins for a given hint (#10715)

* RPC endpoint to retrieve coins by hint

* RPC client update for get_coin_records_by_hint

* start writing tests for get_coin_records_by_hint

* Address linting concerns.

* Address flake8. Fix the get_coin_ids() call.

* convert hint to bytes32

* tests for get_coin_records_by_hint

Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>

* require test-cache repo is found in CI (#10711)

* Issues found in RPC API review (#10702)

* Removed unnecessary substitution

* Recovered  property which was accidentally removed in full node RPC API

* Added backward compatibility to `get_additions_and_removals` full_node RPC API

* Fixed full_node rpc client

* minor followup to config locking (#10696)

* minor lock scope reduction

* use the lock in tests

* Use the passed root_path in configure CLI command (#10694)

* Improve the CI runs w.r.t. timelord installation (#10673)

* Superficial analysis showed that only two test groups require (for now) installing the timelord. This change aims to save us hours of CI running time by simply running the install timelord script only for those test groups, with everything else having it omitted. Dedicated to @hoffmang9

* We don't need these anymore.

* less optional around ssl (#10558)

* less optional

* clean up cruft

* more

* more

* just a little less optional

* cmds: Fix trusted peer hint in `chia wallet show` (#10695)

`config` is the root config here.

* Enable clvm_tools_rs by default (#10554)

* Enable clvm_tools_rs by default

* Re-add clvm_tools dep for now as it provides python idioms for interacting with clvm data

* Take lint formatting

* Adam: Try making this non-parallel

* Try fix for threading issue in tests

* Test whether turning off parallel runs causes things to work (temp)

* Test use of temp files in clvm_tools_rs as a candidate solution for atomic replacement of hex output

* Use proper git+https url scheme (oops)

* Update to candidate 0.1.6 so we can test

* Revert version bump to re-test

* Test whether we can re-enable parallelism

* Attempt to mitigate concurrent test running: return own conception of the compiled output.  This will work if the failing path is downstream of recompilation

* fix path to hex file

* Probe for source of 0-length data

* Further exploration: more assertions, hopefully to trigger in the test on ci

* Do an even more paranoid check to verify that we observe a file whose filesystem reported size is much smaller than expected

* Try a heavier handed approach, using heavyweight lockfiles on the filesystem

* Import a simple lockfile implementation and use it to enforce mutual exclusion.  Simplify it and remove the unwanted os traffic

* Take lint, precommit advice, bump to clvm_tools_rs 0.1.6 now that it's released

* Fix lint

* While i was working on this, -n auto was on the command line so i think this didn't actually do anything, but reverting my change just in case

* Lint

* label the hashes re: pr

* Add a lock.py for spot exclusivity using the filesystem (re: adam in the pr) and a convenience wrapper that hides the details

* Formatting warning

* Ensure type info is present and do the obvious return of the inner function's result

* Use double quotes (lint)

* Properly balance blank lines

* Lint: alphabetize imports

* One line is required here (lint)

* Remove unnecessary assignment

* when creating a new blockchain database implicitly, make it v2 (#10498)

* when creating a new blockchain database implicitly, make it v2

* fix config deadlock

* add select_coins RPC method (#10495)

* add select_coins RPC method

* typing fix

* fix typing, casts

* add RPC coin selection tests

* black formatting

* fix select_coins tests

* improve error messages from chia db upgrade, specifically to help users if the disk is full (#10494)

* more set -o errexit (#10468)

* more set -o errexit -o pipefail

* no pipefail, too fancy for dash at least...

* Bump clvm_tools_rs to fix a problem running as daemon caused by old log message that is now eliminated (#10788)

* when running multiple services in the same process (in tests), don't initialize logging for all of them, and don't set the proctitle of the test (#10686)

* stop helping mkdir() do what it already does (#10802)

* stop helping mkdir() do what it already does

* flake8

* Capitalize display of Rpc -> RPC in `chia show -s` (#10797)

* Remove accidental parameters from calls to setup_simulators_and_wallets and prevent future mistakes (#10770)

* stop using deadsnakes, unless we need it (#10752)

* stop using deadsnakes.  and see...

* only install dead snakes stuff if building the timelord on linux

* small change to fix branch in contributing (#10805)

* small change to fix branch in contributing

* Update CONTRIBUTING.md

* Rename confusing fixtures, especially ones with the same name but dif… (#10772)

* Rename confusing fixtures, especially ones with the same name but different implementation

* rename test_environment to test_plot_environment

* Make it so setup_two_nodes is no longer the name of a fixture and a utility function

* revert premature fixture rename: two_wallet_nodes_start_height_1

* atomic rollback for wallet (#10799)

* atomic rollback for wallet

* Handle cases where one node doesn't have the coin we are looking for (#10829)

* Continue if one node doesn't have the coin

* Pass in coin_state list

* Pass in the single coinstate instead of list

* more simplifications

* run tests in parallel in CI (#10499)

* Fix timelord installation for Debian. (#10841)

* add optional persistence to SpendSim (#10780)

* add optional persistence to SpendSim

* Accidental rename

* remove duplicate event_loop (#10768)

* Adding check for python3.9 alongside python3.10 on Arch (#10363)

* Adding check for python3.9 alongside python3.10 on Arch

* Adjusting install.sh instructions for Arch

* Disabling prescribed python install for Arch

* Setting Arch install script to exit 0 to pass tests

* Adding workflow step for functional Arch install testing

* Adding noconfirm to pacman install command

* Relocating Arch support message for install.sh

* use DEFAULT_ROOT_PATH in tests (#10801)

* Disable the pytest-monitor plugin in CI if not checking results (#10837)

* disable the pytest-monitor plugin if not reporting results

pytest-monitor uses multiprocessing and has caused multiple confusing
issues.  Perhaps it can be adjusted to not use multiprocessing, but
for now lets just isolate the oddities to where we actually use it.

* use a template for resource usage check, similar to timelord install

* hint testconfig.custom_vars

* Check for requesting items when creating an offer (#10864)

* updated gui to 054d7b342e

* Added `-n`/`--new-address` option to `chia wallet get_address` (#10861)

* Added `-n`/`--new-address` option to `chia wallet get_address`

* Formatting fix

* Complemented --new-address with --latest-address per feedback

* Minor output formatting/enhancements for `chia wallet show` (#10863)

* Minor output formatting/enhancements for `chia wallet show`

* Updated format based on internal poll results

* Linter fix and row rearrangement.

* Hardcoded SSL test certs/keys (#10828)

* Hardcoded SSL test certs/keys

* Added a second set of certs/keys.
Cert/key sets are infinitely cycled-through using get_next_private_ca_cert_and_key() and get_next_nodes_certs_and_keys()

* More cert/key sets and a tool to generate them

* Updated SSL generator to sign with the appropriate root CA.
Fixed linter issues.

* Linter fixes

* Updated generate_ssl_for_nodes() based on feedback

* reduce indentation in a few functions in blockchain.py by negating early-exit checks and loop continues (#10872)

* fix typo and index issues in wallet database (#10273)

* fix typo in wallet_puzzle_store

* check some SQL statements

* deduplicate name SQL index

* deduplicate wallet_type index

* deduplicate wallet_id index

* Update appdmg to 0.6.4 to work with macos 12.3 (#10886)

* fixup and enable condition checking tests (#10888)

* fixup and enable tests for the edge cases of absolute timestamp and absolute height conditions in mempool_manager

* Update chia/full_node/full_node_api.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

* Bump colorlog from 5.0.1 to 6.6.0 (#9207)

Bumps [colorlog](https://github.com/borntyping/python-colorlog) from 5.0.1 to 6.6.0.
- [Release notes](https://github.com/borntyping/python-colorlog/releases)
- [Commits](https://github.com/borntyping/python-colorlog/compare/v5.0.1...v6.6.0)

---
updated-dependencies:
- dependency-name: colorlog
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#10505)

* Bump actions/checkout from 2 to 3

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates too

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* Bump github/super-linter from 4.8.1 to 4.9.1 (#10894)

* Bump github/super-linter from 4.8.1 to 4.9.1

Bumps [github/super-linter](https://github.com/github/super-linter) from 4.8.1 to 4.9.1.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.8.1...v4.9.1)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Ignore too-many-function-args in test_type_checking.py

* black

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* fix type annotations for get_block_generator() (#10907)

* fix type annotations for FullBlock.header_hash and FullBlock.prev_header_hash (#10909)

* new DBWrapper supporting concurrent readers (#10166)

* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument

* use rust clvm in Program.run() (#10878)

* remove Program.from_serialized_program

* run the rust clvm implementation (instead of python) even for wallet programs

* Fix flaky trade test (#10921)

* single thread executor (#10919)

* add inline executor and an option to run single-threaded

* add option to run test_full_sync in single-thread mode, to include block validation in profiles. Also attempt to speed it up by disabling db_sync

* await the db commit in the async version of set_db_version (#10906)

* bump pre-commit mypy to v0.942 (#10902)

* bump clvm_tools dependency to make every chia-blockchain installation get the new brun that reports cost accurately (#10880)

* wallet: Drop unused `WalletStateManager.load_wallets` (#10756)

* Switch to integrated lock_and_load_config() context manager (#10698)

* minor lock scope reduction

* use the lock in tests

* Use the passed root_path in configure CLI command

* switch to lock_and_load_config()

* oops

* cleanup

* make _load_config_maybe_locked() private

* black

* Remove future improvement opportunity TODO comment

* move pytest.ini to the root directory (#10892)

* move pytest.ini to the root directory

* pytest.ini: testpaths = tests

https://docs.pytest.org/en/7.1.x/reference/reference.html?highlight=testpaths#confval-testpaths

* set CHIA_ROOT in tests instead of symlinking (#10682)

* attempt to checkout test-cache directly to desired location

* rebuild workflows

* maybe we can use CHIA_ROOT

* use CHIA_ROOT to find blocks and plots for tests

* oops

* more informational printing

* oops

* --capture no for debugging

* flake8

* import os

* undo some unrelated changes now covered elsewhere

* undo some debug changes

* rebuild workflows

* Remove sys.exit() from chia daemon /exit endpoint (#10454)

* asyncio.get_event_loop() is deprecated in 3.10, stop using it (mostly) (#10418)

* asyncio.get_event_loop() is deprecated in 3.10, stop using it

https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop
> Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().

* black

* run tests in CI via coverage (#9704)

* Add coverage (without collection)

* Separate test_block_compression() to avoid coverage-related hangs

* Revert "Separate test_block_compression() to avoid coverage-related hangs"

This reverts commit ebad3d0017.

* multiprocessing.set_start_method("spawn")

* multiprocessing.set_start_method() in conftest.py

* hand hold cc wallet tests

* lint

* spawn for running chia as well

* handle already set start method case

* a bit more timeout for test_multiple_writers

* more timeout for test_writer_lock_blocked_by_readers

* 45 minute tieout for tests/pools/

* 45 minute tieout for tests/pools/

* some more hand holding sleeps

* report coverage in each workflow

only really useful to make sure it is capturing something

* oops

* complete the job name and the JOB_NAME

* better coverage result file names

* reset worker process titles

* rebuild workflows

* rebuild workflows

* black

* black

* rebuild workflows

* push timeouts

* actually include the updated workflows...

* push more workflow timeouts

* parallel=True

* rebuild workflows

* restrict click to < 8.1 for black (#10923)

https://github.com/pallets/click/issues/2225

Doing this instead of updating since updating black will change several
files due to some formatting change.  I would like to take that on
separately from unbreaking CI.

* fixup workflow template merge env duplication (#10925)

* ignore lack of hinting on clvm_tools.binutils.assemble() (#10926)

* Contextualize some store test db names. (#10942)

* Type check values in RL Wallet (#10935)

* Use uint128 for wallet balances (#10936)

* Add more type checks to CAT Wallet (#10934)

* Bump actions/github-script from 4 to 6 (#10246)

* Bump actions/github-script from 4 to 6

Bumps [actions/github-script](https://github.com/actions/github-script) from 4 to 6.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update to github.rest.* for calls to API for compat w/ github-script@v5+

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>

* Bump actions/setup-node from 2.4.1 to 3 (#10506)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2.4.1 to 3.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v2.4.1...v3)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/cache from 2.1.6 to 3 (#10846)

* Bump actions/cache from 2.1.6 to 3

Bumps [actions/cache](https://github.com/actions/cache) from 2.1.6 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.6...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates also

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* Fix trailing bytes shown in CAT asset ID row when using `chia wallet show` (#10924)

* Truncate CAT asset_id output to 32 bytes. A wallet RPC change is needed
to properly separate out the asset ID from the TAIL program returned by
`get_all_wallet_info_entries()`

* Move the truncation to the assignment location

* Consolidate test fixtures (#10778)

* Rename confusing fixtures, especially ones with the same name but different implementation

* revert premature fixture rename: two_wallet_nodes_start_height_1

* Consolidate test fixtures

* Quick fix for improper v2 DB initialization when targeting testnet (#10952)

* bump timing threashold for mempool performance test (#10953)

* Ms.parallel pool t (#10966)

* Try parallel pool tests

* Also change workflow files

* Run less combinations

* Todo for bad test

* Try lower n

* run more tests in parallel on CI (#10960)

* run more tests in parallel on CI

* fix test_farmer_get_harvesters to wait for plots to be loaded before asking about them

* improve error message when a block is missing from the blockchain database (#10958)

* improve error message when a block is missing from the blockchain database

* Update chia/full_node/block_height_map.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Also throw DB error on double spending a coin (#10947)

* Throw error on double spending a coin

* Throw error on double spending a coin

* Improve test

* reorg fixes (#10943)

* when going through a reorg, maintain all chain state until the very end, when the new fork has been fully validated and added

* when rolling back the chain, also rollback the height-to-hash map

* add tests

* Fix the issues in main (failing tests) (#10977)

* Fix one of the issues in test_blockchain

* Only rollback after all async operations are finished

* back to a single option for workflow parallel config (#10979)

* limit test output on CI by dropping -s and -v. Also, only print the 10 slowest tests, instead of all (#10959)

* Ms.flaky gen speed (#10965)

* Flaky test sometimes goes slower than 1 second

* Add sleep to reduce flakiness

* Increase timeout instead of sleeping to hopefully reduce flakiness

* fix test_full_sync.py to only feed the blocks in the main chain to the node (#10974)

* Bump peter-evans/create-pull-request from 3 to 4 (#10950)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v4)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* normalized_to_identity_cc_ip from get_consecutive_blocks was being passed in as overflow_cc_challenge in get_full_block_and_block_record (#10941)

* fix performance tests (#10983)

* Check for vulnerable openssl (#10988)

* Check for vulnerable openssl

* Update OpenSSL on MacOS

* First attempt - openssl Ubuntu 18.04 and 20.04

* place local/bin ahead in PATH

* specify install openssl

* correct path

* run ldconfig

* stop building and check for patched openssl

* spell sudo right by removing it

* Remove openssl building - 1st attempt RHs

* Test Windows OpenSSL version HT @AmineKhaldi

* Non Hobo patch the winstaller for CVE-2022-0778 (#10995)

* apt show not needed (#10997)

* install/upgrade openssl on Arch Linux also (#10999)

* Compile python 3.9.11 which is aware of the openssl issue (#11001)

* install.sh is not upgrading OpenSSL on MacOS (#11003)

* MacOS isn't updating OpenSSL in install.sh

* Exit if no brew on MacOS

* Code the if tree like a pro instead.

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* force index in get_coin_records_by_names (#10987)

* force index in get_coin_records_by_names

* fix lint

* Fix remaining linting issues (#10962)

* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change

* streamable|pools: Fix `Optional` parsing in `dataclass_from_dict` (#10573)

* Test more `Optional` parsing in `dataclass_from_dict`

* Fix optional parsing in `dataclass_from_dict`

* Fix pool wallet / tests

* run_generator2 rust call and compact conditions data structure (#8862)

* use run_generator2 rust call and compact spend bundle conditions data structure pervasively.

* address review comments

* Faster full node tests (#10986)

* Start fast full node tests

* Perf improvement on send_transaction

* Major performance improvement for mempool test

* Speed up another test

* Speed up mempool tests startup

* Lint

* Debug tests

* Try function scope for wallet_nodes

* Update comment

* Force apt to install the things we asked it to (#11047)

* Force apt to install the things we asked it to

* Update .github/workflows/benchmarks.yml

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* github: Drop unused `BUILD_VDF_CLIENT` variables (#11050)

From my understanding this is only used by `chiavdf` source builds which 
happen only if `install-timelord.sh` gets called but it doesn't in the 
addressed cases.

* bump up to 2.1.7 to fix inotify issue resolved by 848 (#11042)

* fix memory leak in test_full_sync (#11004)

* full_node: Drop unused `MempoolManager.constants_json` (#11046)

* simplify some header hash getting and assertions (#11007)

* Remove websockets dependency & do some refactoring (#10611)

* remove old ws

* Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975)

* Adding clean-workspace step to benchmarks (#11063)

* Checkout test blocks and plots for benchmarks workflow (#11068)

* Improve handling of unknown pending balances (likely change from addi… (#10984)

* Improve handling of unknown pending balances (likely change from adding a maker fee).
Minor improvement for fingerprint selection -- enter/return selects the logged-in fingerprint.

* Minor output formatting improvements when showing offer summaries.
Minor wallet key selection improvements.
Added tests for print_offer_summary

* Linter fixes

* isort fix

* Coroutine -> Awaitable

* Removed problematic fee calculation from get_pending_amounts per feedback.

* print average block rate at different block height windows (#11064)

* add -d for Install.ps1 (#11062)

* Set keychain_proxy to None in await_closed() to support reinitialization (#11075)

* Set keychain_proxy to None in await_closed() to support reinitialization.

* Added `shutting_down` param to _await_closed() to control whether the keychain_proxy is closed.

* Significantly speedup preparing test blocks and plots by opting for a release download instead of a shallow git clone, and also by putting a caching layer on top of that. (#11065)

* Bump github/super-linter from 4.9.1 to 4.9.2 (#11067)

Bumps [github/super-linter](https://github.com/github/super-linter) from 4.9.1 to 4.9.2.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.9.1...v4.9.2)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove dead snakes usage from benchmark tests (#11053)

* Handle INSTALL_PYTHON_VERSION in Install.ps1, otherwise search 3.9/3.8/3.7 (#11034)

* Handle INSTALL_PYTHON_VERSION in Install.ps1, otherwise search 3.9/3.8/3.7

* fix python availability check in Install.ps1

* when Install.ps1 does not find an acceptable python, list supported versions in order

* Update Install.ps1

Co-authored-by: Matt Hauff <quexington@gmail.com>

Co-authored-by: Matt Hauff <quexington@gmail.com>

* wallet: Drop `puzzles/genesis_checkers.py` and related puzzles (#10790)

Its all duplicated code and puzzles as far as i can tell, see 
`chia/wallet/puzzles/tails.py`.

* Bump cryptography from 3.4.7 to 36.0.2 (#10787)

Bumps [cryptography](https://github.com/pyca/cryptography) from 3.4.7 to 36.0.2.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/3.4.7...36.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* wallet: Improve logging in `create_more_puzzle_hashes` (#10761)

* wallet: Improve logging in `create_more_puzzle_hashes`

It's pretty spammy currently when scanning puzzle hashes.

* scan -> create, Scanning -> Creating

* `scanning_msg` -> `creating_msg`

* Add /cat_get_unacknowledged API for accessing unknown CATs (#10382)

* Add /cat_get_unacknowledged API for accessing unknown CATs

* Reformat & fix cast issue

* Integration tested & add unit test

* Handle optional uint32

* Reformat

* Reformat

* Reformat

* Merge PR 10308

* Reformat

* Fix concurrent issue

* Add state change notification

* rename API

* Fix failing tests

* Updated state_change name

Co-authored-by: Jeff Cruikshank <jeff@chia.net>

* Increases the probability of connecting to local trusted node (#10633)

* extend tests in test_blockchain to include more conditions, as well as ensuring consensus rules allow unknown condition parameters (#11079)

* chia|tests|github: Implement, integrate and test plot sync protocol (#9695)

* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Add wallentx as additional assignee on mozilla CA update PRs (#11089)

* rebuild workflows (#11092)

* transition to using chia_rs module (#11094)

* Fix the case of claiming a large number of coins (#11038)

* Fix the case of claiming a large number of coins with a fee from a pool wallet

* Revert change to unrelated test

* Set PoolWallet.DEFAULT_MAX_CLAIM_SPENDS to 300

* A few review improvements

* Ms.fast test blockchain (#11051)

* more work on test blockchain

* Optimize test_blockchain.py

* Fix weight proof bug

* Rename variable

* first rc_sub_slot hash bug

* New plots

* try with a new ID

* Run without cache

* Address test blocks and plots preparation.

* Update constant in test_compact_protocol().

* Update this constant too.

* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.

* Fix benchmark test

* Revert mozilla-ca change

* Rebase on main

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>

* can we get by without dead snakes? (#11070)

* can we get by without dead snakes?

* Update install-timelord.sh

* Revert "Update install-timelord.sh"

This reverts commit cba3250b09.

* do not install python dev package for timelords build in ci

it is already there...

* more quotes for sh

* Changelog from 1.3.3 (#11081)

* Updating changelog

* Update appdmg to 0.6.4 to work with macos 12.3 (#10886)

* restrict click to < 8.1 for black

https://github.com/pallets/click/issues/2225

Doing this instead of updating since updating black will change several
files due to some formatting change.  I would like to take that on
separately from unbreaking CI.

* Check for vulnerable openssl (#10988)

* Check for vulnerable openssl

* Update OpenSSL on MacOS

* First attempt - openssl Ubuntu 18.04 and 20.04

* place local/bin ahead in PATH

* specify install openssl

* correct path

* run ldconfig

* stop building and check for patched openssl

* spell sudo right by removing it

* Remove openssl building - 1st attempt RHs

* Test Windows OpenSSL version HT @AmineKhaldi

* Get updated openssl version (#10991)

* Get updated openssl version

* Update pyinstaller

* Fix typo

* lets try this

* Let's try this

* Try this

Co-authored-by: Earle Lowe <e.lowe@chia.net>

* Gh 1.3.3v2 (#11011)

* Non Hobo patch the winstaller for CVE-2022-0778 (#10995)

* install.sh is not upgrading OpenSSL on MacOS (#11003)

* MacOS isn't updating OpenSSL in install.sh

* Exit if no brew on MacOS

* Code the if tree like a pro instead.

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Remove hobo patch

* apt show not needed (#10997)

* install/upgrade openssl on Arch Linux also

* Update CHANGELOG

* revert Arch change backport

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>

* consistently name installer github actions artifact zips (#11096)

* git -C and consistent activation in installer builds (#11098)

* updated gui to d714c21b4e

* only check the version once in installer build workflows (#11099)

* updated gui to 5f8b23fc7d

* updated gui to fccbd3e10d

* streamable: Simplify and force correct usage (#10509)

* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `

* Expose farm_block RPC for simulator (#10830)

* expose farm block api to RPC for simulator

* lint

* pre-commit lint

* Ms.plot load perf2 (#10978)

* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* rebase and more fixes (#10885)

* derivation from just a master public key (#11140)

* Add dependencies macos rhel chiavdf (#11142)

* Added steps, when building the chiavdf wheel for macos and rhel-based systems, to install cmake and/or gmp.

* updated gui to d5b75bcf7a

* Revert "derivation from just a master public key (#11140)" (#11143)

This reverts commit db536c615a.

* Mark the github workspace as safe (#11159)

* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers

* updated gui to 81303fb962

* optimize wallet tool by not caching the puzzle_hash -> derivation index, but caching puzzle_hash -> secret key (which is the lookup we're actually interested in). This avoids duplicating the actual derivation (#11154)

* make listen port colissions in CI less likely (#11164)

* Use get latest release endpoint for plotters, so that we ignore any pre-releases that could be returned by listReleases (#11165)

* Build cli only version of debs (#11166)

* Build cli only version of debs

* Export the vars needed by j2

* Fix paths

* Add symlink to chia in /usr/local/bin/

* Upload the cli only debs to s3

* Add init.py

* Ensure SHA is on the dev build for amd64

* add tool to generate a blockchain with full blocks, as a benchmark (#11146)

* Simplify how the chia symlink is created in the CLI .deb (#11188)

* fix block_tools feature when specifying a list of block references. Also add feature keep_going_until_tx_block. (#11185)

* Fix filename of latest intel dev installer (#11203)

* Add start_crawler and start_seeder to pyinstaller config (#11205)

* Pin mac intel installer to 10.15 (#11209)

* Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)

This reverts commit 93a61eece1.

* Adding changelog (#11223)

* Adding changelog (#11223)

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: ChiaMineJP <admin@chiamine.jp>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: David Barratt <davidbarratt@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
Co-authored-by: Juraj Oršulić <juraj.orsulic@fer.hr>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Johannes Tysiak <vinyl@users.sf.net>
Co-authored-by: Don Kackman <dkackman@gmail.com>
Co-authored-by: austinsirkin <a.sirkin@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
Co-authored-by: Brandon Butler <b.butler@chia.net>
Co-authored-by: Dave <72020697+daverof@users.noreply.github.com>
Co-authored-by: Freddie Coleman <f.coleman@hotmail.co.uk>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
Co-authored-by: arty <art.yerkes@gmail.com>
Co-authored-by: Francesco Truzzi <ftruzzi@users.noreply.github.com>
Co-authored-by: hugepants <hugepants@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: roseiliend <90035993+roseiliend@users.noreply.github.com>
Co-authored-by: Kronus91 <ytx1991@gmail.com>
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: Patrick Maslana <79757486+pmaslana@users.noreply.github.com>

* WIP commit for the new NFT spec

* Update to optimized singleton

* flesh out ownership layer

* nft_transfer_nft: (#11181)

* Introduce some error handling.
* The new_did_inner_hash and trade_price params should be optional for now.

* Add get_confirmed_balance(), get_unconfirmed_balance(), get_spendable_balance(), get_pending_change_balance() and get_max_send_amount() to NFTWallet. This fixes chia wallet show when NFTs are involved. (#11256)

* Needed for DID CLI: (#11268)

* RPC: Add /did_set_wallet_name and /did_get_wallet_name.
* DIDWallet: Add set_name() and get_name().

* Generalize the Offer class to more than CATs

* Remove CAT dependencies from trade_manager

* Fix offer RPC

* isort

* Further generalize the Offer drivers

* Update trade manager with new generalizations

* Fix offer RPC again

* Move outer_puzzles.py

* pivot from string to clvm for dict entries

* add test coverage for driver dict in RPC

* Remove some CAT specific stuff from tm

* Add comments explaining the changes

* Minor fixes

* isort and flake8

* More linting

* Include drivers in offer summary

* Better autodetection of drivers on offer creation

* Forgot to update rpc test

* checkpoint

* fix test and optimise state_layer puz

* rebuild workflows

* Pass Recovery Info in the Transfer case (#11249)

* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Handle recovery

* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* add clvm files to recomp list

* fix rpc test

* flake8 fixes for tests

* flake 8 stupid fix

* Bug fix & flake8 fix for NFT tests

* Modify python code

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Generalize the message puzzle

* Add Pubkey as hint

* Receive DID

* Add DID APIs & Tests

* Fix tests

* Test fixes. in_transaction is now passed as a param to the various callers that eventually call save_info.

* Workflow Fix

* Add test config for DID/NFT

* Update workflow test yaml

* Fix install test script

* Fix typo

* Resolve comments

* Fix tests

* Change did_innerpuz and fix wallets for new design (#11196)

* correcting the design of did_innerpuz and related wallet changes

* remove breakpoint comments

* change decimal point accuracy of percentage system

* secure new_amount by fixing it to our current amount

* rename and update comments for new_amount - now my_amount

* rename P2_PUZZLE to simply INNER_PUZZLE

* fix variable re-declaration for flake/merge

* black and flake8 - inclduing wallet_state_manager bug fix

* update RPCs related to add_url
added commented out tests too, but DID needs fixing first

* Fix bugs in transfer case

* Fix pre-commit

* Fix install.sh test for bookworm

* Pass recovery info in the transfer case

* add clvm files to recomp list

* flake8 fixes for tests

* flake 8 stupid fix

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Pass recovery info in the transfer case

* Fix tests

* Fix security issue of the recovery empty list

* Revert test code

* Allow assign fee when creating the wallet

* reformat

* Add option for pass recovery list

* DID wallet name generation & deduplication

* Fix test

* Remove file reading code

* Fix tests

Co-authored-by: matt <matt@chia.net>
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
Co-authored-by: matt-o-how <48453825+matt-o-how@users.noreply.github.com>
Co-authored-by: ytx1991 <t.yu@chia.net>

* Bumping GUI to 447ac34942

* metadata checkpoint

* fix state layer for metadata updater

* add test for metadata updating

* removed second param for create_ann condition

* forgot to compile clvm

* compiled updated clvm sha256

* * Add fees support to DID wallet creation RPC client. (#11363)

* Initial iteration of the chia wallet did create command. (#11237)

* Add nft_get_nft_info API (#11351)

* Add nft_get_nft_info API

* Refine NFT uncurry code

* Resolve comments

* Resolve comments

Co-authored-by: ytx1991 <t.yu@chia.net>

* almost generates a new nft

* generating nfts

* rpc fix

* DID set wallet name RPC API: Ensure that the wallet type is a DID wallet before proceeding. (#11391)

* Initial iteration of the chia wallet did set_name command. (#11334)

* make NFTWallet a dataclass (#11139)

* make NFTWallet a dataclass

* isort

* flake8

* mypy

* Optional

* Implement nft transfer program

* Resolve comment

* transfer wip

* checkpoint

* basic rpc api working

* Fix data hash issue

* pre commit hooks passing, tests pass

* removed old NFT tests

* workflow files?

* Attempt to fix workflows.

* Workflows?

* Prepare test blocks and plots for NFT wallet tests.

* Reflect the previous commit into workflows.

* clvm compilation fix

* future proof create_coin in nft state layer

* Add RPC client support for the create_new_wallet API. (#11430)

* nft_get_nfts returns a nicer response

* get nfts rpc should now conform to old format

* checkpoint

* Fix tests

* Remove impossible case

* metadata updater working with latest puzzles

* removed unused method

* mypy fix

* Bumping GUI to df86eca99a

* added state change events in nft wallet (#11469)

* added state change events in nft wallet

* Attempt to fix workflows.

Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>

* Bumping GUI to b4d3eebfe3

* fix for clvm compilation fail

* made nft state layer more future proof

* Add `chia.wallet.nft_wallet` to packages in setup.py

* Initial iteration of the NFT0 chia wallet nft create command. (#11476)

* Add wallet RPC client support for the NFT0 nft_mint_nft API. (#11477)

* Automatically create NFT wallet (#11482)

* Create NFT Wallet if there is not a NFT wallet

* Fix pre-commit

* Remove unnecessary log

* fix to merge into main_dids

* Initial iteration of the NFT0 chia nft mint command.

* Initial iteration of the NFT0 chia wallet nft add_uri command.

* Initial iteration of the NFT0 chia wallet nft transfer command.

* Initial iteration of the NFT0 chia wallet nft list command.

* NFT0: Make the wallet ID required for adding a URI.

* Bumping GUI to 725d77abba

* NFT0: Make the wallet ID required for transferring an NFT.

* Add fee option for NFT & limit NFT wallet creation (#11492)

* Add fee option for NFT & limit NFT wallet creation

* Fix pre-commit

* Add fee for update metadata

* Add the ability to set fees for minting NFTs, transferring NFTs and adding URIs to NFTs.

* Fix handling fees on the RPC API side.

* fix for missing minted coins

* updating metadata with multiple uris fix

* Consolidate test_did_rpc.py into test_wallet_rpc.py and make the tests use the wallet RPC client. (#4)

* Update workflows and minor formatting.

* faster tests, other fixes

* potentially fixes the did test failure

Co-authored-by: matt <matt@chia.net>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: matt-o-how <48453825+matt-o-how@users.noreply.github.com>
Co-authored-by: Jeff <jeff@chia.net>
Co-authored-by: Kronus91 <ytx1991@gmail.com>
Co-authored-by: ytx1991 <t.yu@chia.net>
Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: ChiaMineJP <admin@chiamine.jp>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: David Barratt <davidbarratt@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
Co-authored-by: Juraj Oršulić <juraj.orsulic@fer.hr>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Johannes Tysiak <vinyl@users.sf.net>
Co-authored-by: Don Kackman <dkackman@gmail.com>
Co-authored-by: austinsirkin <a.sirkin@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
Co-authored-by: Brandon Butler <b.butler@chia.net>
Co-authored-by: Dave <72020697+daverof@users.noreply.github.com>
Co-authored-by: Freddie Coleman <f.coleman@hotmail.co.uk>
Co-authored-by: arty <art.yerkes@gmail.com>
Co-authored-by: Francesco Truzzi <ftruzzi@users.noreply.github.com>
Co-authored-by: hugepants <hugepants@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: roseiliend <90035993+roseiliend@users.noreply.github.com>
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: Patrick Maslana <79757486+pmaslana@users.noreply.github.com>
Co-authored-by: Sebastjan <trepca@gmail.com>
Co-authored-by: Andreas Greimel <andreas@mintgarden.io>
2022-05-27 13:47:08 -05:00
William Allen
aa95b1e39e
Merge pull request #11565 from AmineKhaldi/offer_generalization_bringup
Quexington's Offer generalization bringup
2022-05-27 12:27:33 -05:00
Amine Khaldi
a37b606c16
Consolidate test_did_rpc.py into test_wallet_rpc.py, make the tests use the wallet RPC client and fix name annotation in create_new_did_wallet(). (#11609) 2022-05-26 17:12:09 -07:00
dependabot[bot]
9f50bb3059
Bump github/super-linter from 4.9.2 to 4.9.3 (#11626)
Bumps [github/super-linter](https://github.com/github/super-linter) from 4.9.2 to 4.9.3.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.9.2...v4.9.3)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-24 16:22:30 -07:00
xdustinface
9aedb714ee
Merge remote-tracking branch 'chia/main' into offer_generalization_bringup 2022-05-24 03:46:51 +02:00
Kyle Altendorf
0f0bc5bda1
avoid cancelling release/** and long_lived/** (#11519) 2022-05-23 16:13:04 -07:00
dependabot[bot]
8ef6a450c6
Bump actions/setup-python from 2 to 3 (#10949)
* Bump actions/setup-python from 2 to 3

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixup templates for python@v3

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
2022-05-23 16:12:23 -07:00
Kyle Altendorf
2c246026c3 test installer on fedora:36, drop 33 2022-05-23 14:44:04 -04:00
geoffwalmsley
560f50996b
fix missing fee for offer maker 2022-05-23 19:16:46 +12:00
Matt Hauff
0592233e9e
pre-commit 2022-05-20 11:57:52 -07:00
xdustinface
3ee55045fd
Merge remote-tracking branch 'chia/main' into pr-main-dids-catchup 2022-05-19 13:02:56 +02:00
Kyle Altendorf
ffc2a42331
disable pytest-monitor by default (#11507)
* disable pytest-monitor by default

* take 2
2022-05-18 09:13:19 -07:00
Kyle Altendorf
c1124d7450
rebuild workflows (#11543) 2022-05-16 13:28:19 -07:00
Kyle Altendorf
f9226fb60d
rename test files that are missing test_ prefix (#10712)
* rename test files that are missing test_ prefix

* update mypy and isort excludes

* skip test_get_host_addr6() in GitHub Actions under macOS

* rebuild workflows
2022-05-16 09:51:00 -07:00
Arvid Norberg
04b543e26a
add test for streamable -> json conversion (#11527)
* add test for streamable -> json conversion

* fixup test
2022-05-15 15:47:17 -07:00
Adam Kelly
298ea9ea20
Resend transactions (#11167)
* Resend transactions

* Don't recheck transactions more frequently than timeout

* Add wallet resend parameter to config, move timeout code out of tx store, but close to call site

* Add a test for wallet transaction resend

* Add test for wallet retry

* isort new files for precommit and update workflows

* Use correct fixture name

* LGTM - remove unused import

Co-authored-by: Earle Lowe <e.lowe@chia.net>
2022-05-14 02:03:10 -07:00
Amine Khaldi
e21108b34c
Merge commit 'e0df18f85d3915f4c947ca2e41c00d60b78f54ba' into main_dids 2022-05-10 19:22:13 +01:00
arty
60f372fcaf
Bump clvm_tools_rs version for clvm stepper and add a test (#11418)
* Bump clvm_tools_rs version for clvm stepper and add a test

* lint

* lint

* flake8

* lint

* lint?

* formatting

* More formatting

* More formatting

* hopefully final lint

* A bit more formal python style for the clvm test

* Add typing imports
2022-05-05 09:14:23 -07:00
Mariano Sorgente
e0df18f85d
Ms.hint refactor (#11222)
* Some work on hints

* More cleanup and test_generator_tools

* Change relevant calls

* More refactor and cleanup

* More refactor

* Small fix

* Create wrapper object PeakPostProcessingResult

* Lint and small fix

* Fix a hint bug

* Fix hint update_wallets

* Fix test_full_sync

* pre-commit

* Start with PR review comments

* More efficient iteration

* Remove tx_removals_additions_and_hints

* Revert mozilla ca

* Fix issue with startup
2022-05-05 08:19:57 -07:00
Kyle Altendorf
cf5113ae87
Remove outdated 3.8 upgrade comment (#11385) 2022-05-05 08:18:36 -07:00
Kyle Altendorf
2daa524aaf
add long_lived/** and release/** to (almost) all workflow push triggers (#11439) 2022-05-05 08:51:36 -06:00
Amine Khaldi
b69323e40e
Reflect the previous commit into workflows. 2022-05-05 11:41:46 +01:00
Amine Khaldi
bddd7e88a5
Workflows? 2022-05-05 11:31:18 +01:00
Amine Khaldi
694b737ccc
Attempt to fix workflows. 2022-05-05 11:23:28 +01:00
Sebastjan
8a32eee2ad workflow files? 2022-05-05 11:26:14 +02:00
Sebastjan
150f2448d6 removed old NFT tests 2022-05-05 11:07:01 +02:00
Arvid Norberg
1b2d119cc3
optimize hash_coin_list() by sorting pure coin_ids, rather than Coins that need to compute its name multiple times during sorting, and also by sorting the actual binary coin names, not hexadecimal strings (#11421) 2022-05-04 07:24:55 -07:00
Kyle Altendorf
d7f39e48c0
Add Windows to wheel check matrix (#11388) 2022-05-02 13:01:08 -07:00
Kyle Altendorf
795f199597
Correct naming of Windows installer artifacts (the GitHub Actions ones) (#11368)
The wrong artifact was renamed in https://github.com/Chia-Network/chia-blockchain/pull/11096.  This corrects that.
2022-04-29 18:36:05 -07:00
dustinface
984c442b5e
util: Implement Paginator class as interface to access a list by pages (#11247)
* util: Implement `Paginator` class as interface to access a list by pages

* Be less restrictive about page sizes and refactor tests

* Make the pages based of 0 instead of 1 and some more test refactoring

* More tests

* Adjust workflows after rebase

* Introduce `Paginator.create`

* `<=` instead of `- 1`
2022-04-29 18:35:47 -07:00
Kyle Altendorf
fd9ff115e3
tidy installer building (#11077)
* tidy installer building

* -d

* fix rpm build

* try for windows

* Install.ps1 -d

* move the install.ps1 call out to the workflow

* Update build_windows.ps1

* Update build_windows.ps1

* debug non-editable install on windows

* platlib and purelib in pyinstaller pathex

* Revert "debug non-editable install on windows"

This reverts commit 7c04168969.

* Revert "platlib and purelib in pyinstaller pathex"

This reverts commit c5c5cdcd81.

* debug

* debug not-editable

* Revert "debug not-editable"

This reverts commit d4373d8e82.

* a bit more similar

* git -C

* find dlls in site-packages?

* oops

* Revert "debug"

This reverts commit 88090f689f.

* consistently name installer github actions artifact zips

* leave SCM_VERSION here

* Update build-windows-installer.yml
2022-04-29 18:34:48 -07:00
Justin England
518bc3966d
adding workflow for checking for conflicts (#11370)
* adding workflow for checking for conflicts

* proper version number

* proper version number syntax?

* not sure tbh

* trying 2.x

* using newly created labels

* adding static 2.0.0 version

* update based on feedback

* removing label removal

* merge_conflict
2022-04-29 19:33:08 -06:00
Kyle Altendorf
521b1ef25d
Merge branch 'main' into main_dids-merge_main_3abcbf1dfa6cfcd5f9f5b47285e8bd5afe35f839 2022-04-27 19:35:44 -04:00
Chris Marslender
ad3259e8fd
Mark workspace safe for arm installers (#11339) 2022-04-27 16:10:41 -07:00
Kyle Altendorf
a813e5906c
Match plurality between macOS and Ubuntu test job names (#11317)
* Match plurality between macOS and Ubuntu test job names

With this, searching for `test /` will match both groups of jobs.

* rebuild workflows
2022-04-27 11:40:31 -07:00
Kyle Altendorf
de0d933b42
rebuild workflows 2022-04-27 13:13:22 -04:00
Kyle Altendorf
28eaf46b3e
Merge branch 'main' into main_dids-merge_cleanup_test 2022-04-27 12:11:15 -04:00
Chris Marslender
3abcbf1dfa
Build installers on push to release branches (#11321) 2022-04-26 16:18:16 -05:00
Kyle Altendorf
7892148bdc
Support for Python 3.10 (#9930)
* Support for Python 3.10

* Update install.sh to block Python 3.11

* websockets to 10.1

* Update workflows for Python 3.10

* single quote 3.10

* Enable fedora:35 (py3.10) installer script testing

* rebuild workflows

* fixup test-install-scripts.yml

* add ignore for distutils deprecation in tests for now

* asyncio.get_event_loop().run_until_complete() -> asyncio.run()

* aiohttp==3.8.1 for python 3.10 support

* use ssl.Purpose.CLIENT_AUTH for ssl_context_for_server()

* rebuild workflows

* use ssl_context_for_client() in BlockTools.get_daemon_ssl_context()

* create a client context for the RpcServer to connect to the daemon

* go back to asyncio.get_event_loop().run_until_complete() for now to recover 3.7

* ignore:There is no current event loop:DeprecationWarning

* Ms.plot load perf2 (#10978)

* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* remove 3.10 avoidance step from debian:bookworm installer testing

* add 3.10 to wheel availability check workflow

* add 3.10 to Install.ps1 supported Python versions for Windows

* add jammy jellyfish to the install script test matrix

* correct ubuntu:jammy job name

* add 22.04 with Python 3.10 to install.sh

Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-26 12:37:01 -07:00
Chris Marslender
a983c4a90b
Upload CLI RPMS to s3, create checksums, etc (#11316) 2022-04-26 13:27:14 -05:00
Kyle Altendorf
f976c07c54
also m1 and arm64 for wheel checks (#11277)
* also m1 and arm64 for wheel checks

* account for self-hosted and pre-setup-python of m1 and arm64 runners

* &&

* python3

* report python version

* use docker on arm64 to get multiple python versions

* flush

* report more system information

* except pycryptodome for now

* more variables, simpler logic

* corrections

* switch to [macos, arm64]

* add python version to job name

* separate os and arch matrix axes

* reorder matrixing

* drop workflow name from job name

* oops

* skip python setup in docker cases

* drop the containers

* Update check_dependency_artifacts.py
2022-04-26 10:48:04 -07:00
Chris Marslender
fd7c3c6363
Update names of installer workflows so they get grouped together in the workflow listing (#11310) 2022-04-26 09:01:21 -06:00
Chris Marslender
d750c901b9
Fix targeting for arm64 to not land on native arm64 mac runners (#11309) 2022-04-25 16:50:38 -05:00
Kyle Altendorf
e584513c70
Add Linux Mint to test matrix (#11295)
* add Linux Mint to test matrix

* : -> / for linux mint dockers

* Prepare Linux Mint

* stop testing linux mint 21 since it reports 20.3

* names, comment, and add 19.2

* mint

* manually install requests

* trailing whitespace
2022-04-25 09:20:08 -07:00
Kyle Altendorf
8460a35466
Debian bookworm now uses 3.10, update the comment (#11278) 2022-04-23 20:11:09 -07:00
Jeff
29e7147c76
Merge commit with main 1.3.4 (d154105a6b) (#11276)
* slight simplification of interpreting the bytes received by the timelord. avoid redundant round-trips to strings (#10316)

* move build and twine to be dev deps rather than workflow installs (#10291)

* add db validate function to check consistency of blockchain database (#10398)

* streamable: Cache stream functions (#10419)

Apply the same pattern as we have for deserialization to serialization.

This avoids all those recursive runtime lookups for "how to stream this
object" which brings a nice speedup:

```
compare: benchmark
mode         | µs/iteration old | µs/iteration new | diff %
to_bytes     | 447.57           | 193.56           | -56.75

compare: full_block
mode         | µs/iteration old | µs/iteration new | diff %
to_bytes     | 110.32           | 61.09            | -44.62
```

* Stop assuming `True == 1` (#10396)

* Stop assuming `True == 1`

* Removed unnecessary lines which confuses code readers

* simplify self_hostname. It doesn't need to depend on initial-config.yaml -> create_block_tools() -> global variable bt (#10371)

* Fix balance calculation so it works with future unconfirmed tx (#10447)

* Only update height if we have processed everything before that height. (#10415)

* Only update height if we have processed everything before that height.

* Remove debug return

* Forgot this file

* Better sorting of coin states, and disconnect peer if we are connected to trusted

* Fix disconnect, and don't mutate arguments

* Fix comment

* Update chia/wallet/wallet_node.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/wallet/wallet_state_manager.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Address comments

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* GC connections in wallet (#10450)

* Fix the year from 2021 to 2022 (#10462)

* Ms.unused code (#10453)

* Remove old backup stuff

* Remove references to CC (couloured coins) which is deprecated terminology

* removed no longer used import

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Don't load initial data older than 5 days (#10481)

* When loading initial data, don't include IPs older than 5 days in the best timestamp dict

* Don't load version data for hosts older than 5 days

* Update the DMG background image (#10289)

* Update the DMG background image

* Updated with latest design.

* Updated background.tiff with latest design

* Fancier DMG customization support via build_dmg.js

* npm_macos -> npm_macos_m1

* Pass in the app path to build_dmg.js

* Peppering with __init__.py files to satisfy the precommit hook

* Return the fees of an offer via RPC (#10480)

* farmer: Cleanup request retry and some logs (#10484)

* farmer: Bump next update times regardless of the request results

* farmer: Drop additional "success/failure" log logic

We already print the PUT response in `_pool_put_farmer` the other parts 
just lead to confusion if the pool didn't implement the PUT correct.

* farmer: Print error responses from the pool with `WARNING` log level

* Fix method name (#10500)

* updated gui to e2202874e1

* reverted to gui 800a0f6556

* updated gui to 672cf2a74a

* Only rewrite config when there is a difference (#10522)

* Only rewrite config when there is a difference

* Use variable

* Preserve existing pool payout_instructions when creating a PoolWallet (#10507)

* Preserve existing pool payout_instructions when creating a PoolWallet

* Updated the logged string when payout_instructions needs to be generated.

* Tests for update_pool_config

* isort

* Logging change, linter fixes, and more comments.

* Abort trusted sync if a state update fails (#10523)

* Abort trusted sync if a state update fails

* Fix lint

* fix updating of sub-epoch-summary map (part of BlockHeightMap) when the entirey change, including genesis changes (#10486)

* Fix propagation of errors when adding a key with an invalid mnemonic (#10274)

* Fix propagation of error messages originating from the keychain server

* Test that adding key with an invalid mnemonic returns the expected error

* Added daemon tests for the add_private_key RPC. Reverted wallet_rpc_client test as the daemon test is better suited for GUI testing.

* Reformatting updates

* Formatting change as requested by the pre-commit gods.

* catch up test_add_private_key() with dynamic ports (#10530)

* Updated background and icon positions. (#10531)

* improve the picking of free ports for tests (#10491)

* Hide balances until we are synced (#10501)

* make multiprocessing start method configurable (#10528)

* make multiprocessing start method configurable

* forkserver

* corrections

* fixup

* optional

* more optional

* stop attempting anchors in the yaml

* rework config handling

* comment

* Update URL for direct download of windows whl for upnp (#10540)

* Add incoming tx records when cancelling an offer (#10538)

* Add incoming tx records when cancelling an offer

* show fee on all txs

* wallet: Reduce log level for `Pulled from queue` message (#10529)

* pools: Drop redundant `PoolWalletInfo.from_json_dict` (#10524)

It exists the same way in its base class `Streamable`.

* correct multiprocessing start method logging, add python_default (#10547)

* correct multiprocessing start method logging, add python_default

* todo -> regular comment

* correct some comments to refer to CATs (#10548)

* correct some comments to refer to CATs

* one more

* and in a test

* Detect hints correctly in the TX (#10543)

* Detect hints correctly in the TX

* Only support list hints

* Allow CAT autodiscovery, refactor CAT default naming (#10308)

* Allow CAT autodiscovery

* Add wallet autodiscovery test

* Use dict get for automatically_add_unknown_cats with default false

* Refactor name generation for new CATs into one place

* Remove hardcoded default cat wallet name from wallet rpc test

* initial-config.yaml comment nit

* Wallet consistancy (#10532)

* use db transaction, -1 in synced up to height, delete unused funcitons

* use transaction info in key-val-store/pool-store

* cat stores

* db lock

* remove unused lock, set synced not always in transaction

* fix store tests

Co-authored-by: wjblanke <wjb98672@gmail.com>

* benchmarks: Implement streamable data comparison (#10433)

* Resubmit peak to timelord for failure. (#10551)

* Initial commit resubmit peak to timelord.

* Change how toandle exception.

* Add some logging. (#10556)

* Fix `install-gui.sh` (#10460)

* Fixed an issue where running  with gui git branch specified failed

* Fixed an issue where install-gui.sh failed if npm>=7 and NodeJS < 16 were installed

* Fixed inconsistent npm path issue

* Fixed typo

* Add fee to changing pool, and more PlotNFT syncing fixes (#10545)

* Add fee to changing pool, and more aggressive disconnect of untrusted

* Don't publish fee TX

* Lint

* Small plotnft related fixes

* More plotnft fixes

* Apply quexington suggestion

* correct param for in_transaction

* Support get_transaction and get_transactions in plotnft

* Re-add publish_transaction and add comment

* Don't rerun additions

* adding 1.3.0 release notes to changelog (#10578)

* adding 1.3.0 release notes to changelog

* Typos. thx @paninaro

* Adding requested adjustments to changelog

* Require pytest-asyncio>=0.17.0 for @fixture() (#10560)

* Convert tests/core/util/test_streamable.py to use pytest. Remove unneeded class. (#10577)

* make sync fixtures not use async def (#10504)

This is particularly relevant in cases where the scope is not function as
that forces use of a wider scoped event loop fixture as well.

* Fixup some hidden test errors (#10442)

* Touching up changelog (#10584)

* removing known issue that was only applicable to a beta release

* Adding additional fixes to the changelog

* Adding Unreleased section to track upcoming changes

* context manager for socket in find_available_listen_port(), catch OSError (#10567)

* Bump CAT wallet test timeout to 40 minutes (#10605)

* Remove block tools and keychain globals (#10358)

* Use bt fixture

* rebase

* Use local_hostname

* Fix test_json (inheritance from unittest.TestCase)

* Use correct BlockTools fixture for test_simulation

* Pass bt fixture into cost calculation tests

* flake8

* Add missing parameters to test functions

* Fix from rebase issues

* Remove set_shared_instance

* Update comment

* Remove unneeded comments

* Remove unused code

* Remove unused code, run `multiprocessing.set_start_method("spawn")` at correct time.

* Revert unrelated change

* Set daemon_port. Teardown services in correct order.
    BIG thanks to Mariano Sorgente for debugging help.

* Add back type signature - rebase issue

* Apply review fixes from Jeff

* Document why we need a later pytest-asyncio version

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Shutdown daemon coroutine properly

* Remove un-needed daemon_port argument

* Set chia-blockchain-gui to hash in main

* Remove connect_to_daemon_port

* Remove code that set "daemon_port" before calling `setup_daemon`

* Remove self_hostname fixture and extra self_hostname global

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Remove more unused test code

* Simplify  fixture

* a problem with using random.randint() to pick a port is that all processes (running in parallel) are seeded the same, and so pick the same ports at the same time, causing conflicts. This uses proper entropy instead. (#10621)

* `chia keys show` will default to displaying the first observer-derived wallet address. With the addition of the `-d` option, the non-observer derived wallet address can be displayed. (#10615)

* pools: Fix `plotnft claim` command's output (#10609)

If you currently claim rewards `claim_cmd` fails to print the txhash 
with the lookup hint in `submit_tx_with_confirmation`

```
Error performing operation on Plot NFT -f 172057028 wallet id: 12: 
'dict' object has no attribute 'name'
```

Because `submit_tx_with_confirmation` expects a `TransactionRecord` as 
result from its callable parameter `func` but `pw_absorb_rewards` 
returns a dict which includes the `TransactionRecord` as value for the 
key `transaction`. This PR makes sure all other methods used as `func` 
callable have the same return behaviour as `pw_absorb_rewards`. We could 
have adjusted it the other way around (only return `TransactionRecord` 
in `pw_absorb_rewards`) but then we would drop information in the RPC 
client.  


With this PR you get:

```
Do chia wallet get_transaction -f 172057028 -tx 
0x34f74a1ffd9da9a493b78463e635996fd03d4f805ade583acb9764df73355f9c to 
get status
```

* fix initial-config typo - log_maxbytesrotation (#10598)

* Fix invalid DB commit (#10594)

* Fix invalid DB commit

* More fixes

* Add raise

* Fix test

* correct spelling of genrated (#10653)

* fix [Bug] #10569 (#10571)

* Version control (#10479)

* Added version control enforcement to macOS m1

* Added enforced version control

* Added enforce version compliance

* Added enforced version compliance

* Added enforced versioning

* Updating this to include DRY internal action.

* Removed some unintended whitespace.

* Removed some unintended whitespace.

* CI re-run

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.

* plotting: Only lock while actually accessing `PlotManager.plots` (#10675)

* Fix exception when `chia keys migrate` is run without needing migration (#10655)

* Fix exception when `chia keys migrate` is run without needing migration

* Linter fixes

* Slightly different query for V2 DBs for getting compact/uncompact block counts, to ensure the available index is used / avoid a table scan (#10661)

* Check prefix on send_transaction before sending (#10566)

* Check prefix on send_transaction

* Fix failing RPC tests - xch -> txch since tests are a testnet and we enforce valid prefixes with this PR

* Ak.convert fixtures (#10612)

* Use bt fixture

* rebase

* Use local_hostname

* flake8

* Remove set_shared_instance

* Remove unneeded comments

* Revert unrelated change

* Add back type signature - rebase issue

* Correct type for _configure_legacy_backend

* See what's going on during CI mypy run

* github workflows

* mypy typing

* Remove legacy Keyring create method

* Start daemon first

* Set chia-blockchain-gui to hash in main

* Fix two test files that were not importing self_hostname

* self_hostname fixture

* Convert all class fixtures to top level functions

* updated gui to cdfa2b9821

* Better management of KeyringWrapper's keys_root_path when using TempKeyring for tests (#10636)

* Better management of KeyringWrapper's keys_root_path when using TempKeyring for tests.

* Move keys_root_path restoration code into `cleanup()`
Added an assert to detect if an unexpected shared KeyringWrapper is injected during a test.

* Conditionally restore keys_root_path for testing

* better TLS1.3 check (#10552)

* better TLS1.3 check

* catch ValueError instead of Exception

* Code simplification and cleanup

* a few nits in comments

* Add configuration locking (#10680)

* Add configuration locking

Extracted from https://github.com/Chia-Network/chia-blockchain/pull/10631

* note that fasteners will likely be replaced by filelock

* Fix test_multiple_writers on macOS

* create_all_ssl() doesn't need to be inside the config access lock

* add warnings about not using async within get_config_lock() get lock contexts

* no need to pre-touch the lock file

* .yaml.lock instead of just .lock

* test_multiple_writers() is sync

* Revert "add warnings about not using async within get_config_lock() get lock contexts"

This reverts commit 681af3835b.

* reduce lock context size in chia_init()

* use an exit stack in load_config()

* avoid config existence precheck

* only lock around the read in load_config()

* do not raise e, just raise

* tidy new imports

* fix queue empty check in test_config.py

* remove commented out code in test_config.py

* remove unused import

Co-authored-by: Jeff Cruikshank <jeff@chia.net>

* fix usage of the bt fixture in conftest fixtures (#10688)

* bump pre-commit mypy to 0.940 (#10672)

* remove some event_loop() fixtures (#10420)

* remove event_loop() fixtures

* flake8

* flake8

* remove sys.exit() from daemon shutdown

* bump full node test timeout.  a lot...  to see.

* fixup some tests

* back to module scope event loop fixture for test_full_node.py

* Update test_full_node.py

* Iterator...

* for the whole directory

* some fixtures back to module scope for reduced runtime

* back to 40 minute workflow timeouts

* these are being addressed separately

* updated gui to c992d07c95

* tests: Add `_PYTEST_RAISE` to fix exception breakpoints with pytest (#10487)

It's currently not possible to have the debuger stop on an uncaucht 
exception when debugging tests. With this patch, adding 
`_PYTEST_RAISE=1` to the environment variables in the pytest 
configuration template fixes this.

* Fixed test failures on Windows. (#10740)

* Convert helper method do_spend from a class method to a function (#10613)

* Remove unused test code (#10614)

* Ak.setup nodes (#10619)

* Remove unused test code

* Centralize fixture uses of setup_n_nodes

* Centralize fixure uses of setup_two_nodes

* Break up setup_nodes into setup_services, for individial services, and setup_nodes, for initializing different simulator configurations

* Sort imports

* more entropy in random listen ports (#10743)

* update chia-blockchain-gui one commit for npm build fix (#10776)

* Updating Changelog for point release (#10781)

* Updating Changelog for point release

* Adding missing changelog items

* run benchmarks separately (#10754)

* run benchmarks separately

* only run benchmarks once, with the most recent python version we support

* Change name to order of returned values. Enforce mandatory naming and inclusion of start_services parameter (#10769)

* cmds: Implement `chia rpc` command (#10763)

* cmds: Implement `chia rpc` command

* Enable "timelord" client + some refactoring to enable "crawler" client

* wallet: Fix `STANDARD_WALLET` creation for `wallet_info.id != 1` (#10757)

* wallet: Optional wallet type parameter for `get_wallets` and `wallet show` (#10739)

* wallet: Add optional `type` parameter to `get_wallets` and `wallet show`

* tests: Use the `type` parameter for `get_wallets` in pool rpc tests

* cmds: Ask for the name of the wallet type in CLI

* harvester: Reuse legacy refresh interval if new params aren't available (#10729)

* mypy 0.941 for pre-commit (#10728)

* Add maker fee to remaining offer RPCs (#10726)

* Add healthcheck endpoint to rpc services (#10718)

* Add healthcheck endpoint to rpc services

* Trailing whitespace ding

* Fix typos lastest > latest (#10720)

* fix typo in command line argument parsing for chia db validate (#10716)

* New RPC get_coin_records_by_hint - Get coins for a given hint (#10715)

* RPC endpoint to retrieve coins by hint

* RPC client update for get_coin_records_by_hint

* start writing tests for get_coin_records_by_hint

* Address linting concerns.

* Address flake8. Fix the get_coin_ids() call.

* convert hint to bytes32

* tests for get_coin_records_by_hint

Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>

* require test-cache repo is found in CI (#10711)

* Issues found in RPC API review (#10702)

* Removed unnecessary substitution

* Recovered  property which was accidentally removed in full node RPC API

* Added backward compatibility to `get_additions_and_removals` full_node RPC API

* Fixed full_node rpc client

* minor followup to config locking (#10696)

* minor lock scope reduction

* use the lock in tests

* Use the passed root_path in configure CLI command (#10694)

* Improve the CI runs w.r.t. timelord installation (#10673)

* Superficial analysis showed that only two test groups require (for now) installing the timelord. This change aims to save us hours of CI running time by simply running the install timelord script only for those test groups, with everything else having it omitted. Dedicated to @hoffmang9

* We don't need these anymore.

* less optional around ssl (#10558)

* less optional

* clean up cruft

* more

* more

* just a little less optional

* cmds: Fix trusted peer hint in `chia wallet show` (#10695)

`config` is the root config here.

* Enable clvm_tools_rs by default (#10554)

* Enable clvm_tools_rs by default

* Re-add clvm_tools dep for now as it provides python idioms for interacting with clvm data

* Take lint formatting

* Adam: Try making this non-parallel

* Try fix for threading issue in tests

* Test whether turning off parallel runs causes things to work (temp)

* Test use of temp files in clvm_tools_rs as a candidate solution for atomic replacement of hex output

* Use proper git+https url scheme (oops)

* Update to candidate 0.1.6 so we can test

* Revert version bump to re-test

* Test whether we can re-enable parallelism

* Attempt to mitigate concurrent test running: return own conception of the compiled output.  This will work if the failing path is downstream of recompilation

* fix path to hex file

* Probe for source of 0-length data

* Further exploration: more assertions, hopefully to trigger in the test on ci

* Do an even more paranoid check to verify that we observe a file whose filesystem reported size is much smaller than expected

* Try a heavier handed approach, using heavyweight lockfiles on the filesystem

* Import a simple lockfile implementation and use it to enforce mutual exclusion.  Simplify it and remove the unwanted os traffic

* Take lint, precommit advice, bump to clvm_tools_rs 0.1.6 now that it's released

* Fix lint

* While i was working on this, -n auto was on the command line so i think this didn't actually do anything, but reverting my change just in case

* Lint

* label the hashes re: pr

* Add a lock.py for spot exclusivity using the filesystem (re: adam in the pr) and a convenience wrapper that hides the details

* Formatting warning

* Ensure type info is present and do the obvious return of the inner function's result

* Use double quotes (lint)

* Properly balance blank lines

* Lint: alphabetize imports

* One line is required here (lint)

* Remove unnecessary assignment

* when creating a new blockchain database implicitly, make it v2 (#10498)

* when creating a new blockchain database implicitly, make it v2

* fix config deadlock

* add select_coins RPC method (#10495)

* add select_coins RPC method

* typing fix

* fix typing, casts

* add RPC coin selection tests

* black formatting

* fix select_coins tests

* improve error messages from chia db upgrade, specifically to help users if the disk is full (#10494)

* more set -o errexit (#10468)

* more set -o errexit -o pipefail

* no pipefail, too fancy for dash at least...

* Bump clvm_tools_rs to fix a problem running as daemon caused by old log message that is now eliminated (#10788)

* when running multiple services in the same process (in tests), don't initialize logging for all of them, and don't set the proctitle of the test (#10686)

* stop helping mkdir() do what it already does (#10802)

* stop helping mkdir() do what it already does

* flake8

* Capitalize display of Rpc -> RPC in `chia show -s` (#10797)

* Remove accidental parameters from calls to setup_simulators_and_wallets and prevent future mistakes (#10770)

* stop using deadsnakes, unless we need it (#10752)

* stop using deadsnakes.  and see...

* only install dead snakes stuff if building the timelord on linux

* small change to fix branch in contributing (#10805)

* small change to fix branch in contributing

* Update CONTRIBUTING.md

* Rename confusing fixtures, especially ones with the same name but dif… (#10772)

* Rename confusing fixtures, especially ones with the same name but different implementation

* rename test_environment to test_plot_environment

* Make it so setup_two_nodes is no longer the name of a fixture and a utility function

* revert premature fixture rename: two_wallet_nodes_start_height_1

* atomic rollback for wallet (#10799)

* atomic rollback for wallet

* Handle cases where one node doesn't have the coin we are looking for (#10829)

* Continue if one node doesn't have the coin

* Pass in coin_state list

* Pass in the single coinstate instead of list

* more simplifications

* run tests in parallel in CI (#10499)

* Fix timelord installation for Debian. (#10841)

* add optional persistence to SpendSim (#10780)

* add optional persistence to SpendSim

* Accidental rename

* remove duplicate event_loop (#10768)

* Adding check for python3.9 alongside python3.10 on Arch (#10363)

* Adding check for python3.9 alongside python3.10 on Arch

* Adjusting install.sh instructions for Arch

* Disabling prescribed python install for Arch

* Setting Arch install script to exit 0 to pass tests

* Adding workflow step for functional Arch install testing

* Adding noconfirm to pacman install command

* Relocating Arch support message for install.sh

* use DEFAULT_ROOT_PATH in tests (#10801)

* Disable the pytest-monitor plugin in CI if not checking results (#10837)

* disable the pytest-monitor plugin if not reporting results

pytest-monitor uses multiprocessing and has caused multiple confusing
issues.  Perhaps it can be adjusted to not use multiprocessing, but
for now lets just isolate the oddities to where we actually use it.

* use a template for resource usage check, similar to timelord install

* hint testconfig.custom_vars

* Check for requesting items when creating an offer (#10864)

* updated gui to 054d7b342e

* Added `-n`/`--new-address` option to `chia wallet get_address` (#10861)

* Added `-n`/`--new-address` option to `chia wallet get_address`

* Formatting fix

* Complemented --new-address with --latest-address per feedback

* Minor output formatting/enhancements for `chia wallet show` (#10863)

* Minor output formatting/enhancements for `chia wallet show`

* Updated format based on internal poll results

* Linter fix and row rearrangement.

* Hardcoded SSL test certs/keys (#10828)

* Hardcoded SSL test certs/keys

* Added a second set of certs/keys.
Cert/key sets are infinitely cycled-through using get_next_private_ca_cert_and_key() and get_next_nodes_certs_and_keys()

* More cert/key sets and a tool to generate them

* Updated SSL generator to sign with the appropriate root CA.
Fixed linter issues.

* Linter fixes

* Updated generate_ssl_for_nodes() based on feedback

* reduce indentation in a few functions in blockchain.py by negating early-exit checks and loop continues (#10872)

* fix typo and index issues in wallet database (#10273)

* fix typo in wallet_puzzle_store

* check some SQL statements

* deduplicate name SQL index

* deduplicate wallet_type index

* deduplicate wallet_id index

* Update appdmg to 0.6.4 to work with macos 12.3 (#10886)

* fixup and enable condition checking tests (#10888)

* fixup and enable tests for the edge cases of absolute timestamp and absolute height conditions in mempool_manager

* Update chia/full_node/full_node_api.py

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>

* Bump colorlog from 5.0.1 to 6.6.0 (#9207)

Bumps [colorlog](https://github.com/borntyping/python-colorlog) from 5.0.1 to 6.6.0.
- [Release notes](https://github.com/borntyping/python-colorlog/releases)
- [Commits](https://github.com/borntyping/python-colorlog/compare/v5.0.1...v6.6.0)

---
updated-dependencies:
- dependency-name: colorlog
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#10505)

* Bump actions/checkout from 2 to 3

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates too

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* Bump github/super-linter from 4.8.1 to 4.9.1 (#10894)

* Bump github/super-linter from 4.8.1 to 4.9.1

Bumps [github/super-linter](https://github.com/github/super-linter) from 4.8.1 to 4.9.1.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.8.1...v4.9.1)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Ignore too-many-function-args in test_type_checking.py

* black

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* fix type annotations for get_block_generator() (#10907)

* fix type annotations for FullBlock.header_hash and FullBlock.prev_header_hash (#10909)

* new DBWrapper supporting concurrent readers (#10166)

* new DBWrapper supporting concurrent readers

* adress review comments

* fixup default database version, when file doesn't exist

* remove unused argument

* use rust clvm in Program.run() (#10878)

* remove Program.from_serialized_program

* run the rust clvm implementation (instead of python) even for wallet programs

* Fix flaky trade test (#10921)

* single thread executor (#10919)

* add inline executor and an option to run single-threaded

* add option to run test_full_sync in single-thread mode, to include block validation in profiles. Also attempt to speed it up by disabling db_sync

* await the db commit in the async version of set_db_version (#10906)

* bump pre-commit mypy to v0.942 (#10902)

* bump clvm_tools dependency to make every chia-blockchain installation get the new brun that reports cost accurately (#10880)

* wallet: Drop unused `WalletStateManager.load_wallets` (#10756)

* Switch to integrated lock_and_load_config() context manager (#10698)

* minor lock scope reduction

* use the lock in tests

* Use the passed root_path in configure CLI command

* switch to lock_and_load_config()

* oops

* cleanup

* make _load_config_maybe_locked() private

* black

* Remove future improvement opportunity TODO comment

* move pytest.ini to the root directory (#10892)

* move pytest.ini to the root directory

* pytest.ini: testpaths = tests

https://docs.pytest.org/en/7.1.x/reference/reference.html?highlight=testpaths#confval-testpaths

* set CHIA_ROOT in tests instead of symlinking (#10682)

* attempt to checkout test-cache directly to desired location

* rebuild workflows

* maybe we can use CHIA_ROOT

* use CHIA_ROOT to find blocks and plots for tests

* oops

* more informational printing

* oops

* --capture no for debugging

* flake8

* import os

* undo some unrelated changes now covered elsewhere

* undo some debug changes

* rebuild workflows

* Remove sys.exit() from chia daemon /exit endpoint (#10454)

* asyncio.get_event_loop() is deprecated in 3.10, stop using it (mostly) (#10418)

* asyncio.get_event_loop() is deprecated in 3.10, stop using it

https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.get_event_loop
> Deprecated since version 3.10: Deprecation warning is emitted if there is no running event loop. In future Python releases, this function will be an alias of get_running_loop().

* black

* run tests in CI via coverage (#9704)

* Add coverage (without collection)

* Separate test_block_compression() to avoid coverage-related hangs

* Revert "Separate test_block_compression() to avoid coverage-related hangs"

This reverts commit ebad3d0017.

* multiprocessing.set_start_method("spawn")

* multiprocessing.set_start_method() in conftest.py

* hand hold cc wallet tests

* lint

* spawn for running chia as well

* handle already set start method case

* a bit more timeout for test_multiple_writers

* more timeout for test_writer_lock_blocked_by_readers

* 45 minute tieout for tests/pools/

* 45 minute tieout for tests/pools/

* some more hand holding sleeps

* report coverage in each workflow

only really useful to make sure it is capturing something

* oops

* complete the job name and the JOB_NAME

* better coverage result file names

* reset worker process titles

* rebuild workflows

* rebuild workflows

* black

* black

* rebuild workflows

* push timeouts

* actually include the updated workflows...

* push more workflow timeouts

* parallel=True

* rebuild workflows

* restrict click to < 8.1 for black (#10923)

https://github.com/pallets/click/issues/2225

Doing this instead of updating since updating black will change several
files due to some formatting change.  I would like to take that on
separately from unbreaking CI.

* fixup workflow template merge env duplication (#10925)

* ignore lack of hinting on clvm_tools.binutils.assemble() (#10926)

* Contextualize some store test db names. (#10942)

* Type check values in RL Wallet (#10935)

* Use uint128 for wallet balances (#10936)

* Add more type checks to CAT Wallet (#10934)

* Bump actions/github-script from 4 to 6 (#10246)

* Bump actions/github-script from 4 to 6

Bumps [actions/github-script](https://github.com/actions/github-script) from 4 to 6.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update to github.rest.* for calls to API for compat w/ github-script@v5+

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>

* Bump actions/setup-node from 2.4.1 to 3 (#10506)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2.4.1 to 3.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v2.4.1...v3)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/cache from 2.1.6 to 3 (#10846)

* Bump actions/cache from 2.1.6 to 3

Bumps [actions/cache](https://github.com/actions/cache) from 2.1.6 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.6...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates also

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>

* Fix trailing bytes shown in CAT asset ID row when using `chia wallet show` (#10924)

* Truncate CAT asset_id output to 32 bytes. A wallet RPC change is needed
to properly separate out the asset ID from the TAIL program returned by
`get_all_wallet_info_entries()`

* Move the truncation to the assignment location

* Consolidate test fixtures (#10778)

* Rename confusing fixtures, especially ones with the same name but different implementation

* revert premature fixture rename: two_wallet_nodes_start_height_1

* Consolidate test fixtures

* Quick fix for improper v2 DB initialization when targeting testnet (#10952)

* bump timing threashold for mempool performance test (#10953)

* Ms.parallel pool t (#10966)

* Try parallel pool tests

* Also change workflow files

* Run less combinations

* Todo for bad test

* Try lower n

* run more tests in parallel on CI (#10960)

* run more tests in parallel on CI

* fix test_farmer_get_harvesters to wait for plots to be loaded before asking about them

* improve error message when a block is missing from the blockchain database (#10958)

* improve error message when a block is missing from the blockchain database

* Update chia/full_node/block_height_map.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Also throw DB error on double spending a coin (#10947)

* Throw error on double spending a coin

* Throw error on double spending a coin

* Improve test

* reorg fixes (#10943)

* when going through a reorg, maintain all chain state until the very end, when the new fork has been fully validated and added

* when rolling back the chain, also rollback the height-to-hash map

* add tests

* Fix the issues in main (failing tests) (#10977)

* Fix one of the issues in test_blockchain

* Only rollback after all async operations are finished

* back to a single option for workflow parallel config (#10979)

* limit test output on CI by dropping -s and -v. Also, only print the 10 slowest tests, instead of all (#10959)

* Ms.flaky gen speed (#10965)

* Flaky test sometimes goes slower than 1 second

* Add sleep to reduce flakiness

* Increase timeout instead of sleeping to hopefully reduce flakiness

* fix test_full_sync.py to only feed the blocks in the main chain to the node (#10974)

* Bump peter-evans/create-pull-request from 3 to 4 (#10950)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v4)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* normalized_to_identity_cc_ip from get_consecutive_blocks was being passed in as overflow_cc_challenge in get_full_block_and_block_record (#10941)

* fix performance tests (#10983)

* Check for vulnerable openssl (#10988)

* Check for vulnerable openssl

* Update OpenSSL on MacOS

* First attempt - openssl Ubuntu 18.04 and 20.04

* place local/bin ahead in PATH

* specify install openssl

* correct path

* run ldconfig

* stop building and check for patched openssl

* spell sudo right by removing it

* Remove openssl building - 1st attempt RHs

* Test Windows OpenSSL version HT @AmineKhaldi

* Non Hobo patch the winstaller for CVE-2022-0778 (#10995)

* apt show not needed (#10997)

* install/upgrade openssl on Arch Linux also (#10999)

* Compile python 3.9.11 which is aware of the openssl issue (#11001)

* install.sh is not upgrading OpenSSL on MacOS (#11003)

* MacOS isn't updating OpenSSL in install.sh

* Exit if no brew on MacOS

* Code the if tree like a pro instead.

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* force index in get_coin_records_by_names (#10987)

* force index in get_coin_records_by_names

* fix lint

* Fix remaining linting issues (#10962)

* FIx remaining linting issues

* Revert type:ignore

* Revert token_bytes change

* streamable|pools: Fix `Optional` parsing in `dataclass_from_dict` (#10573)

* Test more `Optional` parsing in `dataclass_from_dict`

* Fix optional parsing in `dataclass_from_dict`

* Fix pool wallet / tests

* run_generator2 rust call and compact conditions data structure (#8862)

* use run_generator2 rust call and compact spend bundle conditions data structure pervasively.

* address review comments

* Faster full node tests (#10986)

* Start fast full node tests

* Perf improvement on send_transaction

* Major performance improvement for mempool test

* Speed up another test

* Speed up mempool tests startup

* Lint

* Debug tests

* Try function scope for wallet_nodes

* Update comment

* Force apt to install the things we asked it to (#11047)

* Force apt to install the things we asked it to

* Update .github/workflows/benchmarks.yml

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* github: Drop unused `BUILD_VDF_CLIENT` variables (#11050)

From my understanding this is only used by `chiavdf` source builds which 
happen only if `install-timelord.sh` gets called but it doesn't in the 
addressed cases.

* bump up to 2.1.7 to fix inotify issue resolved by 848 (#11042)

* fix memory leak in test_full_sync (#11004)

* full_node: Drop unused `MempoolManager.constants_json` (#11046)

* simplify some header hash getting and assertions (#11007)

* Remove websockets dependency & do some refactoring (#10611)

* remove old ws

* Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975)

* Adding clean-workspace step to benchmarks (#11063)

* Checkout test blocks and plots for benchmarks workflow (#11068)

* Improve handling of unknown pending balances (likely change from addi… (#10984)

* Improve handling of unknown pending balances (likely change from adding a maker fee).
Minor improvement for fingerprint selection -- enter/return selects the logged-in fingerprint.

* Minor output formatting improvements when showing offer summaries.
Minor wallet key selection improvements.
Added tests for print_offer_summary

* Linter fixes

* isort fix

* Coroutine -> Awaitable

* Removed problematic fee calculation from get_pending_amounts per feedback.

* print average block rate at different block height windows (#11064)

* add -d for Install.ps1 (#11062)

* Set keychain_proxy to None in await_closed() to support reinitialization (#11075)

* Set keychain_proxy to None in await_closed() to support reinitialization.

* Added `shutting_down` param to _await_closed() to control whether the keychain_proxy is closed.

* Significantly speedup preparing test blocks and plots by opting for a release download instead of a shallow git clone, and also by putting a caching layer on top of that. (#11065)

* Bump github/super-linter from 4.9.1 to 4.9.2 (#11067)

Bumps [github/super-linter](https://github.com/github/super-linter) from 4.9.1 to 4.9.2.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.9.1...v4.9.2)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove dead snakes usage from benchmark tests (#11053)

* Handle INSTALL_PYTHON_VERSION in Install.ps1, otherwise search 3.9/3.8/3.7 (#11034)

* Handle INSTALL_PYTHON_VERSION in Install.ps1, otherwise search 3.9/3.8/3.7

* fix python availability check in Install.ps1

* when Install.ps1 does not find an acceptable python, list supported versions in order

* Update Install.ps1

Co-authored-by: Matt Hauff <quexington@gmail.com>

Co-authored-by: Matt Hauff <quexington@gmail.com>

* wallet: Drop `puzzles/genesis_checkers.py` and related puzzles (#10790)

Its all duplicated code and puzzles as far as i can tell, see 
`chia/wallet/puzzles/tails.py`.

* Bump cryptography from 3.4.7 to 36.0.2 (#10787)

Bumps [cryptography](https://github.com/pyca/cryptography) from 3.4.7 to 36.0.2.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/3.4.7...36.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* wallet: Improve logging in `create_more_puzzle_hashes` (#10761)

* wallet: Improve logging in `create_more_puzzle_hashes`

It's pretty spammy currently when scanning puzzle hashes.

* scan -> create, Scanning -> Creating

* `scanning_msg` -> `creating_msg`

* Add /cat_get_unacknowledged API for accessing unknown CATs (#10382)

* Add /cat_get_unacknowledged API for accessing unknown CATs

* Reformat & fix cast issue

* Integration tested & add unit test

* Handle optional uint32

* Reformat

* Reformat

* Reformat

* Merge PR 10308

* Reformat

* Fix concurrent issue

* Add state change notification

* rename API

* Fix failing tests

* Updated state_change name

Co-authored-by: Jeff Cruikshank <jeff@chia.net>

* Increases the probability of connecting to local trusted node (#10633)

* extend tests in test_blockchain to include more conditions, as well as ensuring consensus rules allow unknown condition parameters (#11079)

* chia|tests|github: Implement, integrate and test plot sync protocol (#9695)

* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Add wallentx as additional assignee on mozilla CA update PRs (#11089)

* rebuild workflows (#11092)

* transition to using chia_rs module (#11094)

* Fix the case of claiming a large number of coins (#11038)

* Fix the case of claiming a large number of coins with a fee from a pool wallet

* Revert change to unrelated test

* Set PoolWallet.DEFAULT_MAX_CLAIM_SPENDS to 300

* A few review improvements

* Ms.fast test blockchain (#11051)

* more work on test blockchain

* Optimize test_blockchain.py

* Fix weight proof bug

* Rename variable

* first rc_sub_slot hash bug

* New plots

* try with a new ID

* Run without cache

* Address test blocks and plots preparation.

* Update constant in test_compact_protocol().

* Update this constant too.

* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.

* Fix benchmark test

* Revert mozilla-ca change

* Rebase on main

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>

* can we get by without dead snakes? (#11070)

* can we get by without dead snakes?

* Update install-timelord.sh

* Revert "Update install-timelord.sh"

This reverts commit cba3250b09.

* do not install python dev package for timelords build in ci

it is already there...

* more quotes for sh

* Changelog from 1.3.3 (#11081)

* Updating changelog

* Update appdmg to 0.6.4 to work with macos 12.3 (#10886)

* restrict click to < 8.1 for black

https://github.com/pallets/click/issues/2225

Doing this instead of updating since updating black will change several
files due to some formatting change.  I would like to take that on
separately from unbreaking CI.

* Check for vulnerable openssl (#10988)

* Check for vulnerable openssl

* Update OpenSSL on MacOS

* First attempt - openssl Ubuntu 18.04 and 20.04

* place local/bin ahead in PATH

* specify install openssl

* correct path

* run ldconfig

* stop building and check for patched openssl

* spell sudo right by removing it

* Remove openssl building - 1st attempt RHs

* Test Windows OpenSSL version HT @AmineKhaldi

* Get updated openssl version (#10991)

* Get updated openssl version

* Update pyinstaller

* Fix typo

* lets try this

* Let's try this

* Try this

Co-authored-by: Earle Lowe <e.lowe@chia.net>

* Gh 1.3.3v2 (#11011)

* Non Hobo patch the winstaller for CVE-2022-0778 (#10995)

* install.sh is not upgrading OpenSSL on MacOS (#11003)

* MacOS isn't updating OpenSSL in install.sh

* Exit if no brew on MacOS

* Code the if tree like a pro instead.

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Remove hobo patch

* apt show not needed (#10997)

* install/upgrade openssl on Arch Linux also

* Update CHANGELOG

* revert Arch change backport

Co-authored-by: Kyle Altendorf <sda@fstab.net>

Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>

* consistently name installer github actions artifact zips (#11096)

* git -C and consistent activation in installer builds (#11098)

* updated gui to d714c21b4e

* only check the version once in installer build workflows (#11099)

* updated gui to 5f8b23fc7d

* updated gui to fccbd3e10d

* streamable: Simplify and force correct usage (#10509)

* streamable: Merge `strictdataclass` into `Streamable` class

* tests: Test not supported streamable types

* streamable: Reorder decorators

* streamable: Simplify streamable decorator and force correct usage/syntax

* streamable: Just move some stuff around in the file

* streamable: Improve syntax error messages

* mypy: Drop `type_checking.py` and `test_type_checking.py` from exclusion

* streamable: Use cached fields instead of `__annotations__`

This is now possible after merging `__post_init__` into `Streamable`

* Introduce `DefinitionError` as `StreamableError`

* `/t` -> `    `

* Expose farm_block RPC for simulator (#10830)

* expose farm block api to RPC for simulator

* lint

* pre-commit lint

* Ms.plot load perf2 (#10978)

* 2.7 seconds -> 0.45 seconds

* Merge

* Work on create_plots refactor

* Try to fix tests

* Try to fix tests

* Use new functions

* Fix block_tools by adding dir

* Extra argument

* Try to fix cyclic import

* isort

* Drop warning

* Some cleanups around `exclude_final_dir` and directory adding

* Cleanup `min_mainnet_k_size` checks

* Drop unrelated changes

* Fixes after rebase

* Fix cyclic import

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update tests/block_tools.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* rebase and more fixes (#10885)

* derivation from just a master public key (#11140)

* Add dependencies macos rhel chiavdf (#11142)

* Added steps, when building the chiavdf wheel for macos and rhel-based systems, to install cmake and/or gmp.

* updated gui to d5b75bcf7a

* Revert "derivation from just a master public key (#11140)" (#11143)

This reverts commit db536c615a.

* Mark the github workspace as safe (#11159)

* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers

* updated gui to 81303fb962

* optimize wallet tool by not caching the puzzle_hash -> derivation index, but caching puzzle_hash -> secret key (which is the lookup we're actually interested in). This avoids duplicating the actual derivation (#11154)

* make listen port colissions in CI less likely (#11164)

* Use get latest release endpoint for plotters, so that we ignore any pre-releases that could be returned by listReleases (#11165)

* Build cli only version of debs (#11166)

* Build cli only version of debs

* Export the vars needed by j2

* Fix paths

* Add symlink to chia in /usr/local/bin/

* Upload the cli only debs to s3

* Add init.py

* Ensure SHA is on the dev build for amd64

* add tool to generate a blockchain with full blocks, as a benchmark (#11146)

* Simplify how the chia symlink is created in the CLI .deb (#11188)

* fix block_tools feature when specifying a list of block references. Also add feature keep_going_until_tx_block. (#11185)

* Fix filename of latest intel dev installer (#11203)

* Add start_crawler and start_seeder to pyinstaller config (#11205)

* Pin mac intel installer to 10.15 (#11209)

* Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)

This reverts commit 93a61eece1.

* Adding changelog (#11223)

* Adding changelog (#11223)

Co-authored-by: Arvid Norberg <arvid@libtorrent.org>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: ChiaMineJP <admin@chiamine.jp>
Co-authored-by: Mariano Sorgente <3069354+mariano54@users.noreply.github.com>
Co-authored-by: David Barratt <davidbarratt@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: William Blanke <wjb98672@gmail.com>
Co-authored-by: Juraj Oršulić <juraj.orsulic@fer.hr>
Co-authored-by: Yostra <straya@chia.net>
Co-authored-by: Florin Chirica <fchirica96@gmail.com>
Co-authored-by: Richard Kiss <him@richardkiss.com>
Co-authored-by: William Allen <wallentx@users.noreply.github.com>
Co-authored-by: Adam Kelly <338792+aqk@users.noreply.github.com>
Co-authored-by: Johannes Tysiak <vinyl@users.sf.net>
Co-authored-by: Don Kackman <dkackman@gmail.com>
Co-authored-by: austinsirkin <a.sirkin@chia.net>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
Co-authored-by: Brandon Butler <b.butler@chia.net>
Co-authored-by: Dave <72020697+daverof@users.noreply.github.com>
Co-authored-by: Freddie Coleman <f.coleman@hotmail.co.uk>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
Co-authored-by: arty <art.yerkes@gmail.com>
Co-authored-by: Francesco Truzzi <ftruzzi@users.noreply.github.com>
Co-authored-by: hugepants <hugepants@users.noreply.github.com>
Co-authored-by: Jack Nelson <jack@jacknelson.xyz>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Gene Hoffman <30377676+hoffmang9@users.noreply.github.com>
Co-authored-by: roseiliend <90035993+roseiliend@users.noreply.github.com>
Co-authored-by: Kronus91 <ytx1991@gmail.com>
Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: wallentx <william.allentx@gmail.com>
Co-authored-by: Earle Lowe <e.lowe@chia.net>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: Patrick Maslana <79757486+pmaslana@users.noreply.github.com>
2022-04-22 17:36:54 -07:00
Kyle Altendorf
3c48224551
check dependency artifacts (#11243)
* check dependency artifacts

* remove commented out code

* find root path relative to script

* fixup stringy pathy mixup

* Update check_dependency_artifacts.py
2022-04-21 17:00:28 -07:00
Kyle Altendorf
aaf3fbc160
generate file lists for workflows instead of wildcard patterns (#11117) 2022-04-21 16:58:07 -07:00
Chris Marslender
527b30e917
Add cli only rpm (#11236)
* Add cli only rpm

* Ensure rvm (fpm) is loaded before running fpm

* Use full path to fpm, since GHA seems to mess up the PATH in the container

* Add back source and add use ruby-3

* Call rpm script with bash, to see if the rvm script works

* Add --depends for libcrypt.so.1
2022-04-20 21:39:51 -07:00
dustinface
25ab0c90cb
github: Fix plot_sync workflows after #10940 merge (#11241) 2022-04-20 15:27:59 -07:00
Gene Hoffman
797a2fb5fe
Remove 3.9.11 for Windows Installer (#11226)
And replace with just 3.9.x
2022-04-20 11:44:52 -07:00
dependabot[bot]
efbac6a05f
Bump actions/upload-artifact from 2 to 3 (#11144)
* Bump actions/upload-artifact from 2 to 3

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Also update runner_templates

* Mark the github workspace as safe (#11159)

* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2022-04-20 11:40:51 -07:00
Kyle Altendorf
5b044ea497
locate pyinstaller in dev deps rather than build scripts (#11118) 2022-04-20 11:16:53 -07:00
Kyle Altendorf
93da3b727b
Up Windows installer build timeout to 50 minutes (#11107)
Recent runs are mostly low 30s but some are close to 40 and one just timed out.
2022-04-20 11:14:20 -07:00
Kyle Altendorf
b9c9727b75
--module pytest (#10940) 2022-04-20 11:11:08 -07:00
Kronus91
deab050480
DID changes all in one (#10559)
* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Handle recovery

* Chialisp draft

* Modify python code

* Reformat & Fix tests

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* add clvm files to recomp list

* fix rpc test

* flake8 fixes for tests

* flake 8 stupid fix

* Bug fix & flake8 fix for NFT tests

* Modify python code

* Chialisp draft

* Modify python code

* Bug fix & flake8 fix for NFT tests

* Generalize the message puzzle

* Add Pubkey as hint

* Receive DID

* Add DID APIs & Tests

* Fix tests

* Test fixes. in_transaction is now passed as a param to the various callers that eventually call save_info.

* Workflow Fix

* Add test config for DID/NFT

* Update workflow test yaml

* Fix install test script

* Fix typo

* Resolve comments

* Fix tests

* Change did_innerpuz and fix wallets for new design (#11196)

* correcting the design of did_innerpuz and related wallet changes

* remove breakpoint comments

* change decimal point accuracy of percentage system

* secure new_amount by fixing it to our current amount

* rename and update comments for new_amount - now my_amount

* rename P2_PUZZLE to simply INNER_PUZZLE

* fix variable re-declaration for flake/merge

* black and flake8 - inclduing wallet_state_manager bug fix

* update RPCs related to add_url
added commented out tests too, but DID needs fixing first

* Fix bugs in transfer case

* Fix pre-commit

* Fix install.sh test for bookworm

Co-authored-by: matt <matt@chia.net>
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
Co-authored-by: matt-o-how <48453825+matt-o-how@users.noreply.github.com>
Co-authored-by: ytx1991 <t.yu@chia.net>
2022-04-19 17:45:23 -07:00
Chris Marslender
7a123afddb
Fix install.sh test for bookworm (#11227) 2022-04-19 15:43:06 -07:00
Chris Marslender
337dbd4e5c
Revert "Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)" (#11211)
This reverts commit d3e73a75ab.
2022-04-19 12:53:59 -06:00
Chris Marslender
d3e73a75ab
Revert "Pin mac intel installer to 10.15 (#11209)" (#11210)
This reverts commit 93a61eece1.
2022-04-18 15:42:09 -06:00
Chris Marslender
93a61eece1
Pin mac intel installer to 10.15 (#11209) 2022-04-18 15:34:27 -06:00
Chris Marslender
7567b4ea9c
Fix filename of latest intel dev installer (#11203) 2022-04-18 10:18:59 -05:00
Chris Marslender
2fbe062e78
Build cli only version of debs (#11166)
* Build cli only version of debs

* Export the vars needed by j2

* Fix paths

* Add symlink to chia in /usr/local/bin/

* Upload the cli only debs to s3

* Add init.py

* Ensure SHA is on the dev build for amd64
2022-04-14 19:32:26 -05:00
Jeff
e5d048f1de
Merge commit with main (9ff3fc993f) (#11137)
* Merged with main (9ff3fc993f)

* Linter and some test fixes

* isort, linting, test fix

* Fixes and bumped the GUI to point to nftdev

* Couple of fixes
2022-04-14 12:02:42 -07:00
Chris Marslender
3ea3a4dc0c
Use get latest release endpoint for plotters, so that we ignore any pre-releases that could be returned by listReleases (#11165) 2022-04-13 18:06:50 -05:00
Chris Marslender
b3a9459fea
Mark the github workspace as safe (#11159)
* Mark the github workspace as safe

* Move the git config step after git is installed in the test containers
2022-04-13 09:35:42 -06:00
Kyle Altendorf
2d4045b9d2
only check the version once in installer build workflows (#11099) 2022-04-08 13:49:21 -07:00
Kyle Altendorf
e6f60c572e
git -C and consistent activation in installer builds (#11098) 2022-04-08 09:59:59 -07:00
Kyle Altendorf
13512be2b2
consistently name installer github actions artifact zips (#11096) 2022-04-08 09:59:36 -07:00
Kyle Altendorf
415236bf67
can we get by without dead snakes? (#11070)
* can we get by without dead snakes?

* Update install-timelord.sh

* Revert "Update install-timelord.sh"

This reverts commit cba3250b09.

* do not install python dev package for timelords build in ci

it is already there...

* more quotes for sh
2022-04-08 09:58:46 -07:00
Mariano Sorgente
2f9e718073
Ms.fast test blockchain (#11051)
* more work on test blockchain

* Optimize test_blockchain.py

* Fix weight proof bug

* Rename variable

* first rc_sub_slot hash bug

* New plots

* try with a new ID

* Run without cache

* Address test blocks and plots preparation.

* Update constant in test_compact_protocol().

* Update this constant too.

* Revert accidental altering of the gui submodule in ae7e3295f280a591e76c4dffdea75fb74ea5de6f.

* Fix benchmark test

* Revert mozilla-ca change

* Rebase on main

Co-authored-by: almog <almogdepaz@gmail.com>
Co-authored-by: Amine Khaldi <amine.khaldi@reactos.org>
2022-04-08 09:57:59 -07:00
Kyle Altendorf
02a880fbac
rebuild workflows (#11092) 2022-04-07 20:43:27 -07:00
Chris Marslender
c7a89fc425
Add wallentx as additional assignee on mozilla CA update PRs (#11089) 2022-04-07 19:43:14 -05:00
dustinface
ded9f68583
chia|tests|github: Implement, integrate and test plot sync protocol (#9695)
* protocols|server: Define new harvester plot refreshing protocol messages

* protocols: Bump `protocol_version` to `0.0.34`

* tests: Introduce `setup_farmer_multi_harvester`

Allows to run a test setup with 1 farmer and mutiple harvesters.

* plotting: Add an initial plot loading indication to `PlotManager`

* plotting|tests: Don't add removed duplicates to `total_result.removed`

`PlotRefreshResult.removed` should only contain plots that were loaded
properly before they were removed. It shouldn't contain e.g. removed
duplicates or invalid plots since those are synced in an extra sync step
and not as diff but as whole list every time.

* harvester: Reset `PlotManager` on shutdown

* plot_sync: Implement plot sync protocol

* farmer|harvester: Integrate and enable plot sync

* tests: Implement tests for the plot sync protocol

* farmer|tests: Drop obsolete harvester caching code

* setup: Add `chia.plot_sync` to packages

* plot_sync: Type hints in `DeltaType`

* plot_sync: Drop parameters in `super()` calls

* plot_sync: Introduce `send_response` helper in `Receiver._process`

* plot_sync: Add some parentheses

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* plot_sync: Additional hint for a `Receiver.process_path_list` parameter

* plot_sync: Force named parameters in `Receiver.process_path_list`

* test: Fix fixtures after rebase

* tests: Fix sorting after rebase

* tests: Return type hint for `plot_sync_setup`

* tests: Rename `WSChiaConnection` and move it in the outer scope

* tests|plot_sync: More type hints

* tests: Rework some delta tests

* tests: Drop a `range` and iterate over the list directly

* tests: Use the proper flags to overwrite

* test: More missing duplicates tests

* tests: Drop `ExpectedResult.reset`

* tests: Reduce some asserts

* tests: Add messages to some `assert False` statements

* tests: Introduce `ErrorSimulation` enum in `test_sync_simulated.py`

* tests: Use `secrects` instead of `Crypto.Random`

* Fixes after rebase

* Import from `typing_extensions` to support python 3.7

* Drop task name to support python 3.7

* Introduce `Sender.syncing`, `Sender.connected` and a log about the task

* Add `tests/plot_sync/config.py`

* Align the multi harvester fixture with what we do in other places

* Update the workflows

Co-authored-by: Kyle Altendorf <sda@fstab.net>
2022-04-07 17:10:44 -07:00
Kyle Altendorf
c0d346d428
Remove dead snakes usage from benchmark tests (#11053) 2022-04-07 09:18:15 -07:00
dependabot[bot]
42245d74eb
Bump github/super-linter from 4.9.1 to 4.9.2 (#11067)
Bumps [github/super-linter](https://github.com/github/super-linter) from 4.9.1 to 4.9.2.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.9.1...v4.9.2)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-07 09:17:53 -07:00
Amine Khaldi
6026d734cf
Significantly speedup preparing test blocks and plots by opting for a release download instead of a shallow git clone, and also by putting a caching layer on top of that. (#11065) 2022-04-06 20:15:10 -07:00
Kyle Altendorf
45e4c7c156
Checkout test blocks and plots for benchmarks workflow (#11068) 2022-04-05 17:16:38 -05:00
William Allen
6ed61bfbe5
Adding clean-workspace step to benchmarks (#11063) 2022-04-05 15:28:33 -05:00
Amine Khaldi
a2fa8dda01
Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975) 2022-04-05 10:34:57 -07:00
dustinface
4a4b14b78b
github: Drop unused BUILD_VDF_CLIENT variables (#11050)
From my understanding this is only used by `chiavdf` source builds which 
happen only if `install-timelord.sh` gets called but it doesn't in the 
addressed cases.
2022-04-04 18:43:51 -07:00
Adam Kelly
22edd73331
Force apt to install the things we asked it to (#11047)
* Force apt to install the things we asked it to

* Update .github/workflows/benchmarks.yml

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-04-04 17:43:31 -07:00
Gene Hoffman
8cbf96d73c
Non Hobo patch the winstaller for CVE-2022-0778 (#10995) 2022-04-01 16:31:00 -07:00
dependabot[bot]
fc1d52de6a
Bump peter-evans/create-pull-request from 3 to 4 (#10950)
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/v3...v4)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-31 23:04:48 -07:00
Arvid Norberg
a571c78899
limit test output on CI by dropping -s and -v. Also, only print the 10 slowest tests, instead of all (#10959) 2022-03-31 23:01:50 -07:00
Arvid Norberg
bf15087c79
run more tests in parallel on CI (#10960)
* run more tests in parallel on CI

* fix test_farmer_get_harvesters to wait for plots to be loaded before asking about them
2022-03-31 08:23:59 -07:00
Mariano Sorgente
02bf8489d2
Ms.parallel pool t (#10966)
* Try parallel pool tests

* Also change workflow files

* Run less combinations

* Todo for bad test

* Try lower n
2022-03-31 08:23:34 -07:00
dependabot[bot]
817ebe7d6d
Bump actions/cache from 2.1.6 to 3 (#10846)
* Bump actions/cache from 2.1.6 to 3

Bumps [actions/cache](https://github.com/actions/cache) from 2.1.6 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.6...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates also

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
2022-03-29 10:35:06 -07:00
dependabot[bot]
81c60993f6
Bump actions/setup-node from 2.4.1 to 3 (#10506)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2.4.1 to 3.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v2.4.1...v3)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-29 10:34:48 -07:00
dependabot[bot]
9a0d4775ae
Bump actions/github-script from 4 to 6 (#10246)
* Bump actions/github-script from 4 to 6

Bumps [actions/github-script](https://github.com/actions/github-script) from 4 to 6.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update to github.rest.* for calls to API for compat w/ github-script@v5+

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
2022-03-29 10:34:12 -07:00
Kyle Altendorf
fb68bebfc3
fixup workflow template merge env duplication (#10925) 2022-03-28 19:50:42 -07:00
Kyle Altendorf
36bee1b8ca
run tests in CI via coverage (#9704)
* Add coverage (without collection)

* Separate test_block_compression() to avoid coverage-related hangs

* Revert "Separate test_block_compression() to avoid coverage-related hangs"

This reverts commit ebad3d0017.

* multiprocessing.set_start_method("spawn")

* multiprocessing.set_start_method() in conftest.py

* hand hold cc wallet tests

* lint

* spawn for running chia as well

* handle already set start method case

* a bit more timeout for test_multiple_writers

* more timeout for test_writer_lock_blocked_by_readers

* 45 minute tieout for tests/pools/

* 45 minute tieout for tests/pools/

* some more hand holding sleeps

* report coverage in each workflow

only really useful to make sure it is capturing something

* oops

* complete the job name and the JOB_NAME

* better coverage result file names

* reset worker process titles

* rebuild workflows

* rebuild workflows

* black

* black

* rebuild workflows

* push timeouts

* actually include the updated workflows...

* push more workflow timeouts

* parallel=True

* rebuild workflows
2022-03-28 13:22:23 -07:00
Kyle Altendorf
219e815fc4
set CHIA_ROOT in tests instead of symlinking (#10682)
* attempt to checkout test-cache directly to desired location

* rebuild workflows

* maybe we can use CHIA_ROOT

* use CHIA_ROOT to find blocks and plots for tests

* oops

* more informational printing

* oops

* --capture no for debugging

* flake8

* import os

* undo some unrelated changes now covered elsewhere

* undo some debug changes

* rebuild workflows
2022-03-28 13:15:24 -07:00
dependabot[bot]
7c49e29eaf
Bump github/super-linter from 4.8.1 to 4.9.1 (#10894)
* Bump github/super-linter from 4.8.1 to 4.9.1

Bumps [github/super-linter](https://github.com/github/super-linter) from 4.8.1 to 4.9.1.
- [Release notes](https://github.com/github/super-linter/releases)
- [Changelog](https://github.com/github/super-linter/blob/main/docs/release-process.md)
- [Commits](https://github.com/github/super-linter/compare/v4.8.1...v4.9.1)

---
updated-dependencies:
- dependency-name: github/super-linter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Ignore too-many-function-args in test_type_checking.py

* black

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
2022-03-25 22:23:51 -07:00
dependabot[bot]
8781569829
Bump actions/checkout from 2 to 3 (#10505)
* Bump actions/checkout from 2 to 3

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update actions in templates too

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gene Hoffman <hoffmang@hoffmang.com>
2022-03-25 22:23:27 -07:00
Kyle Altendorf
2eb4fdeaee
Disable the pytest-monitor plugin in CI if not checking results (#10837)
* disable the pytest-monitor plugin if not reporting results

pytest-monitor uses multiprocessing and has caused multiple confusing
issues.  Perhaps it can be adjusted to not use multiprocessing, but
for now lets just isolate the oddities to where we actually use it.

* use a template for resource usage check, similar to timelord install

* hint testconfig.custom_vars
2022-03-24 09:30:42 -07:00
William Allen
b1acb5597b
Adding check for python3.9 alongside python3.10 on Arch (#10363)
* Adding check for python3.9 alongside python3.10 on Arch

* Adjusting install.sh instructions for Arch

* Disabling prescribed python install for Arch

* Setting Arch install script to exit 0 to pass tests

* Adding workflow step for functional Arch install testing

* Adding noconfirm to pacman install command

* Relocating Arch support message for install.sh
2022-03-22 16:22:17 -07:00
Arvid Norberg
b5d51c38f8
run tests in parallel in CI (#10499) 2022-03-22 15:04:07 -07:00
Kyle Altendorf
7f84309e70
stop using deadsnakes, unless we need it (#10752)
* stop using deadsnakes.  and see...

* only install dead snakes stuff if building the timelord on linux
2022-03-21 16:27:43 -07:00
Amine Khaldi
6e499e0e62
Improve the CI runs w.r.t. timelord installation (#10673)
* Superficial analysis showed that only two test groups require (for now) installing the timelord. This change aims to save us hours of CI running time by simply running the install timelord script only for those test groups, with everything else having it omitted. Dedicated to @hoffmang9

* We don't need these anymore.
2022-03-17 09:05:33 -07:00
Arvid Norberg
0e956c8b5a
run benchmarks separately (#10754)
* run benchmarks separately

* only run benchmarks once, with the most recent python version we support
2022-03-17 08:36:51 -07:00
austinsirkin
caaf7f40e1
Version control (#10479)
* Added version control enforcement to macOS m1

* Added enforced version control

* Added enforce version compliance

* Added enforced version compliance

* Added enforced versioning

* Updating this to include DRY internal action.

* Removed some unintended whitespace.

* Removed some unintended whitespace.

* CI re-run

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.

* Trying to figure out why it's failing one test.
2022-03-11 15:57:35 -07:00
Arvid Norberg
0e29dbc6d4
Bump CAT wallet test timeout to 40 minutes (#10605) 2022-03-09 03:41:18 -08:00
Kyle Altendorf
43138bccf0
move build and twine to be dev deps rather than workflow installs (#10291) 2022-02-24 20:58:28 -08:00
Chris Marslender
c82840eb69
Fix s3 destination for intel dev installers (#10403) 2022-02-23 17:44:59 -06:00
Chris Marslender
b895de80eb
Update to download.chia.net bucket and use cached url for webseed (#10365) 2022-02-22 12:59:34 -06:00
Chris Marslender
7232b1cd2d
Pin back to windows-2019 for now, since windows-latest is updated to 2022 (#10372) 2022-02-22 09:55:26 -06:00
Kyle Altendorf
e3fb3ce96e
Move black from SuperLinter to upload workflow, use 21.12b0 in pre-commit (#10103)
* black==21.12b0 in pre-commit

Match `setup.py`.

* move black from super linter to upload workflow

* black (updated)

* configure so black . works

* --check --diff for black
2022-02-20 21:33:09 -08:00
Chris Marslender
7ab1152a17
Check if secrets are available for triggering dev docker builds (#10337) 2022-02-20 09:49:39 -06:00
Chris Marslender
17fcd08914
Add trigger workflow on PR for dev docker images (#10305) 2022-02-19 11:28:37 -06:00
Kyle Altendorf
a17e5fc100
Split full node tests directory for parallelism (#10108)
* Split full node tests directory for parallelism

* reduce full node job timeout to 40, duplicate config to new directory

* rebuild workflows

* add missing __init__.py

* update some imports

* oops

* isort

* Revert "isort"

This reverts commit 03d8360342.

* update isort exclude list
2022-02-18 07:51:52 -08:00
Kyle Altendorf
5d2770cbdd
extend pools test workflow timeout to 60 minutes (#10271) 2022-02-16 19:10:37 -08:00
Chris Marslender
20a7078960
Update the GHA build for intel mac to work as the primary release workflow (#10232)
* Update the GHA build for intel mac to work as the primary release workflow

* Add dev hash logic to GHA version of intel mac installer
2022-02-15 15:27:31 -07:00
William Allen
ff88892e73
git short hash in dev artifact (#10195)
* Adding build-linux-arm64-installer changes

* Adding changes to build installers to include short hash in dev artifact

* Fixing s3 dmg path for azure pipelines dev build

* Testing shell: bash for win workflow

* flipping slashes for bash env

* Formatting build windows installer workflow

* Reformatting build windows installer workflow
2022-02-14 22:25:09 -08:00
Kyle Altendorf
bd845f5d3a
Test CLI installer on Rocky (#10134)
* Test CLI installer on Rocky

* Add Rocky support to install.sh

* Update test-install-scripts.yml

* oops

* also sudo for rocky install.sh testing

* fix indentation

* oops
2022-02-14 22:24:19 -08:00
Kyle Altendorf
ae9b7f4e7d
Disable CentOS 8 install script check (#10237) 2022-02-14 22:23:56 -08:00
Kyle Altendorf
fc95c638a3
Update to handle the NodeJS 16 dependency (#9921)
* preliminary testing of in-directory n on ubuntu always

* Use more n chiaminejp (#9971)

* Added n as a local npm dependency

* Fixed an issue where `install.sh` always tried to run `sudo apt install bc -y` even if `bc` is installed already

* Added validations and useful outputs for `start-gui.sh`

* Fixed lint error and use shell functions for readability

* Replace tags with spaces

* Skip installing python39 on RH like OS if it is already installed

* Fixed an issue where start-gui.sh failed silently if venv is not activated

* Suppressed message from pacman

* Support CentOS7

* Fixed typo

* Reduced unnecessary install messages

* Fixed end of file

* Added npm_global/__init__.py to pass CI

* Fixed lint errors

* Install python/sqlite from source on AMZN2. Clear old venv when changing python version on install

* Suppress unnecessary command outputs

* Suppress outputs

* Added centos7/8 to install test

* A minor fix

* Fixed yaml syntax error

* Fixed an issue where test-install-scripts failed in CentOS

Co-authored-by: ChiaMineJP <admin@chiamine.jp>
2022-02-11 18:30:19 -08:00
Patrick Maslana
e0aa80b428
Fix the reviewers names (#10097)
* Fix the reviewers names

* Removed space from the names of the reviewers.
2022-02-03 14:25:02 -07:00
Patrick Maslana
22d664794b
Updated the list of names that will be listed as reviewers for the automated mozilla-ca approval. (#10093) 2022-02-03 11:48:47 -07:00
Chris Marslender
435a077bc6
Allow older jobs to run to completion when on main (#10067) 2022-02-01 19:24:46 -06:00
Chris Marslender
e8309ec884
Use native concurrency settings to cancel duplicate in progress jobs rather than waiting for the job to start to cancel (#9963) 2022-01-26 11:20:31 -06:00
Jeff
bb1c16cee5
[CLI] chia keys derive [wallet-address | child-key | search] (#9184)
* Added chia keys derive (wallet-address | child-key) to display derived wallet addresses or public keys

* Support for public key (unhardened) derivation

* Fleshed out the search command. Can perform a substring search on wallet addresses, public, and private keys.
Support for deriving private keys

* Default to unhardened derivation. Added --hardened-derivation option. HD paths denote hardened derivation with a 'h' suffix at each index.

* Added --no-progress to reduce output. Exit code indicates search success/failure. Option name cleanup.

* chia keys derive search can now search for multiple terms

* chia keys derive search can be restricted to searching pubkeys|privkeys|addresses|all

* Allow child-key derivation rooted from an arbitrary HD path

* Search derived keys rooted under an arbitrary HD path

* LGTM fixes

* Remove extra master_sk_to_wallet_sk_unhardened definition

* Cleanup and fixed search output

* Cleanup and comments

* Added tests for `chia keys` commands

* Using @pytest_asyncio.fixture to decorate empty_keyring() as suggested in pytest's output.

* Set asyncio-mode to auto to address pytest warning

* See if tests pass without checking exit status

* Added chia init step to tests that direclty invoke the `cli` function

* Explicitly add --root-path option to tests that run the `cli` function.

* Tests fix

* Re-enable checks against exit_code

* More fine tuning to address CI failures

* Changes to address naming and other feedback

* Updated tests
2022-01-25 20:42:16 -08:00
Earle Lowe
d7cd3fe2cb
run block tool updates (#9807)
* run block tool updates

* pylint fixes

* Add tests

* Update workflows for new tests

* Updates based on comments
2022-01-25 07:44:13 -08:00
Kyle Altendorf
32a500a88e
Test installers in Dockers (#9896)
* Avoid issue with Ubuntu 21.10+ setuptools>=06

https://github.com/pypa/setuptools/issues/2956
https://gist.github.com/altendky/b852296ac520b343890c55eabc90c880

```
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 290, in set_undefined_options
        setattr(self, dst_option, getattr(src_cmd_obj, src_option))
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__
        raise AttributeError(attr)
    AttributeError: install_layout
```

* corrections

* test installers in various Linux Dockers

* prep docker images with basic tools

* --no-confirm for pacman

* debug

* debug

* debug

* stop installing git

* Revert "stop installing git"

This reverts commit 4b19f1601d.

* checkout after installing git

* pacman wants --noconfirm

* Revert "debug"

This reverts commit e0a2aed041.

* Revert "debug"

This reverts commit 3833da281a.

* Revert "debug"

This reverts commit 705a3c5d6b.

* add bookworm

* add install.sh -a for pacman --noconfirm automated installs

* fix

* fix

* revert install_layout workaround

* Remove =3.9* from debian python3-venv installation

Already discussed this with @wallentx related to the addition in https://github.com/Chia-Network/chia-blockchain/pull/9633/files#diff-043df5bdbf6639d7a77e1d44c5226fd7371e5259a1e4df3a0dd5d64c30dca44fR85.

* add fedora 33 and 34

* nicer naming

* fedora not fedor

* add ubuntu:bionic

* also chia --help

* add amazon

* Amazon wants sudo

* .

* use git-core ppa for git in bionic

* less sudo

* oops
2022-01-23 21:46:16 -08:00
Chris Marslender
bb6e0cb0ac
Build a new version of the main tagged docker image on merge to main (#9888) 2022-01-20 14:12:58 -06:00
Kyle Altendorf
2a6564c0a7
move all linter config to the root (#9830)
This reduces duplication, symlinks, and makes the configurations
available by default for the linting tools.
2022-01-19 11:38:11 -08:00
Kyle Altendorf
48cfe11641
Set pools CI test timeout to 45 minutes (#9817)
* Set pools CI test timeout to 45 minutes

There have been several timeout failures recently with successful runs right on the edge at 28 minutes and such.

* pre-commit for workflow rebuild
2022-01-15 16:31:54 -08:00
Yostra
89f15f591c
Merge standalone wallet into main (#9793)
* wallet changes from pac

* cat changes

* pool tests

* pooling tests passing

* offers

* lint

* mempool_mode

* black

* linting

* workflow files

* flake8

* more cleanup

* renamed

* remove obsolete test, don't cast announcement

* memos are not only bytes32

* trade renames

* fix rpcs, block_record

* wallet rpc, recompile settlement clvm

* key derivation

* clvm tests

* lgtm issues and wallet peers

* stash

* rename

* mypy linting

* flake8

* bad initializer

* flaky tests

* Make CAT wallets only create on verified hints (#9651)

* fix clvm tests

* return to log lvl warn

* check puzzle unhardened

* public key, not bytes. api caching change

* precommit changes

* remove unused import

* mypy ci file, tests

* ensure balance before creating a tx

* Remove CAT logic from full node test (#9741)

* Add confirmations and sleeps for wallet (#9742)

* use pool executor

* rever merge mistakes/cleanup

* Fix trade test flakiness (#9751)

* remove precommit

* older version of black

* lint only in super linter

* Make announcements in RPC be objects instead of bytes (#9752)

* Make announcements in RPC be objects instead of bytes

* Lint

* misc hint'ish cleanup (#9753)

* misc hint'ish cleanup

* unremove some ci bits

* Use main cached_bls.py

* Fix bad merge in main_pac (#9774)

* Fix bad merge at 71da0487b9

* Remove unused ignores

* more unused ignores

* Fix bad merge at 3b143e7050

* One more byte32.from_hexstr

* Remove obsolete test

* remove commented out

* remove duplicate payment object

* remove long sync

* remove unused test, noise

* memos type

* bytes32

* make it clear it's a single state at a time

* copy over asset ids from pacr

* file endl linter

* Update chia/server/ws_connection.py

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: Matt Hauff <quexington@gmail.com>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2022-01-13 12:08:32 -08:00