chia-blockchain/tests
dustinface bc98e6c5d8
tests: Some renaming in plot manager tests to avoid pytest conflicts (#9692)
Having `Test` or `test_` prefixes obviously lets `pytest` think its a
actual test class/method and it
leads to warnings in test logs if it doesn't comply with its rules:

```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:45:
PytestCollectionWarning: cannot collect test class 'TestDirectory'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
or
```
/home/runner/work/chia-blockchain/chia-blockchain/tests/plotting/test_plot_manager.py:135:
PytestCollectionWarning: cannot collect test class 'TestEnvironment'
because it has a __init__ constructor (from:
plotting/test_plot_manager.py)
```
2022-05-31 13:42:11 -05:00
..
blockchain fix some LGTM issues (#11453) 2022-05-12 15:17:24 -07:00
clvm Refine test for __eq__ 2022-05-24 08:33:13 -07:00
core Increase from 180 to 300 (#11707) 2022-05-31 13:40:08 -05:00
farmer_harvester tests: Consolidate farmer/harvester fixtures (#11513) 2022-05-25 15:15:49 -07:00
generator restore missing hints being stored as None (instead of 0-length bytes) (#11568) 2022-05-23 10:42:40 -07:00
plot_sync tests: Some renaming in plot manager tests to avoid pytest conflicts (#9692) 2022-05-31 13:42:11 -05:00
plotting tests: Some renaming in plot manager tests to avoid pytest conflicts (#9692) 2022-05-31 13:42:11 -05:00
pools Bind port 0 to fix race condition when grabbing available ports (#11578) 2022-05-23 08:13:49 -07:00
runner_templates avoid cancelling release/** and long_lived/** (#11519) 2022-05-23 16:13:04 -07:00
simulation Bind port 0 to fix race condition when grabbing available ports (#11578) 2022-05-23 08:13:49 -07:00
tools improve generate_chain (#11242) 2022-04-27 11:43:33 -07:00
util tests: Avoid using match of pytest.raises together with paths (#11638) 2022-05-26 15:15:30 -07:00
wallet Merge remote-tracking branch 'chia/main' into offer_generalization_bringup 2022-05-24 03:46:51 +02:00
weight_proof fix some LGTM issues (#11453) 2022-05-12 15:17:24 -07:00
__init__.py Added tests, improved serialization 2019-08-05 14:38:16 +09:00
block_tools.py Resend transactions (#11167) 2022-05-14 02:03:10 -07:00
build-init-files.py do not generate __init__.py in .pytest_cache (#10270) 2022-02-17 07:27:22 -08:00
build-workflows.py disable pytest-monitor by default (#11507) 2022-05-18 09:13:19 -07:00
check_pytest_monitor_output.py add stress tests with malicious generators (#8570) 2021-09-29 11:41:49 -07:00
check_sql_statements.py fix typo and index issues in wallet database (#10273) 2022-03-25 09:28:06 -07:00
chia-start-sim Rename src to chia. 2021-04-04 21:41:39 -07:00
conftest.py tests: Consolidate farmer/harvester fixtures (#11513) 2022-05-25 15:15:49 -07:00
connection_utils.py less except -> false (#10219) 2022-05-14 02:03:56 -07:00
README.md Improve style consistency in root directory docs files (#4493) 2021-05-12 13:12:30 -07:00
setup_nodes.py tests: Consolidate farmer/harvester fixtures (#11513) 2022-05-25 15:15:49 -07:00
setup_services.py Bind port 0 to fix race condition when grabbing available ports (#11578) 2022-05-23 08:13:49 -07:00
testconfig.py Prepare test blocks and plots only for tests that need them. This saves us a couple more hours of CI running time. (#10975) 2022-04-05 10:34:57 -07:00
time_out_assert.py Timed assertion output simplification (#9290) 2021-11-19 11:14:18 -08:00
wallet_tools.py optimize wallet tool by not caching the puzzle_hash -> derivation index, but caching puzzle_hash -> secret key (which is the lookup we're actually interested in). This avoids duplicating the actual derivation (#11154) 2022-04-13 12:13:25 -07:00

Test CI Job generation

The CI jobs for these tests are managed by build-workflows.py.

If you add a test file, or move one to another directory, please run build-workflows.py. Tests are recognized by the file glob test_*.py. Changing the contents of a file does not require running build-workflows.py.

We currently use github actions. Default runners have two vcpus. The workflows are located in ../.github/workflows/.

The inputs to build-workflows.py are the templates in runner-templates, the file testconfig.py in this directory, and the optional config.py files in some test subdirectories. Files in the template directory ending in include.yml are included in jobs based on the per-directory settings.

The generated workflows are output to ../.github/workflows/.

Each subdirectory below the directories root_test_dirs in testconfig.py becomes a job in the github workflow matrix. If your jobs run too long, simply move some tests into new subdirectories and run build-workflows.py. A workflow built from a parent directory does not include the tests in its subdirectories. The subdirectory jobs do not include the tests from their parents.

testconfig.py

In the top tests directory, testconfig.py contains the application settings and the per-directory default settings.

config.py

Each directory has an optional config.py file, which can override the per-directory default settings.

Per directory settings defaults:

parallel = False
checkout_blocks_and_plots = True
install_timelord = True
job_timeout = 30

Parallel test execution

If you are certain that all the tests in a directory can run in parallel, set parallel = True in config.py inside that directory.

Optional job stages

Set checkout_blocks_and_plots to False to omit checking out the test-cache repo.

Set install_timelord to False to omit the step of installing a Time Lord for your directory's job.

Job Timeout

Set job_timeout to the number of minutes you want the CI system to wait before it kills your job. Add two or three minutes to allow for job setup.