chia-blockchain/chia/pyinstaller.spec
Florin Chirica f550555c7d
Add plotters. (#8497)
* Initial commit add plotters.

* Lint.

* Add progress for bladebit.

* Address some review comments.

* Oops...

* Add install option.

* Change chiapos to fit the old standard.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/install_plotter.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Lint.

* Remove farmerkey as required.

* Chia plotters chiapos works with no arguments.

* Added get_plotters RPC to support the GUI (#8530)

* Added 'get_plotters' daemon RPC. Probes installed/available plotters on behalf of the GUI.

* Linter fix

* Minor type tweak

* Tweaks.

* Run with default arguments all plotters.

* Fix bug.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Change bladebit repo.

* Update chia/plotters/bladebit.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Update chia/plotters/plotters.py

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>

* Re-added the connect-to-daemon hidden option

* Use connect_to_daemon value for madmax and bladebit plot key resolution.

* Updated --tmp_dir, --tmp_dir2, --final_dir options to match old options from chia plots create

* Add CONNECT_TO_DAEMON as a valid option for madmax/bladebit

* Thread multiplier should be an int

* Passing params for madmax to start_plotting. Still needs cleanup/refactoring.

* Update chia/plotters/bladebit.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Update chia/plotters/madmax.py

Co-authored-by: Kyle Altendorf <sda@fstab.net>

* Filename option -z.

* Factor out calling the plotter.

* First attempt refactor install scripts.

* Switch to exec.

* Attempt to fix mypy warning.

* Remove filename.

* Increase RLIMIT_NOFILE for madmax on non-Windows platforms

* Add trailing path separator to madmax tmpdir/tmp2dir/finaldir arguments (required by madmax)

* Fixes to support madmax plotting from the GUI.
Writing output from the plotters now includes a flush to ensure the plotter log (used by the GUI) is updated frequently.

* Handle madmax's tmptoggle option internally when plotting with the GUI

* Formatting and linter fix

* Fixed the -i option for bladebit

* Construct BladeBit plotting options

* Cleanup code for building plotter command line options

* Added a post-processing step after each plotting job completes. Adds the final_dir plot directory as necessary.

* Fix plotter root path

* Reverting prior checkin. Need to figure out how to handle CHIA_ROOT being overridden

* BladeBit support for Windows

* BladeBit's --memory-json option is used to check memory requirements

* Madmax Windows support

* Plotters directory is now under CHIA_ROOT

* Madmax version detection

* BladeBit will default to 0 threads. BladeBit will max-out available threads with this configuration.

* LGTM fixes

* Module definition for chia.plotters to resolve mypy issues with chiapos (package vs chiapos.py)

* Updated BladeBit build script to account for 1.2.0 changes.
Replaced remaining subprocess.run calls with run_command.
Use BladeBit's reported memory requirement instead of hardcoded value.

* Show a disclaimer when using thirdparty plotter

* Test adding mac madmax plotter to the installers

* Get latest madmax from the latest GH release

* Fix bad var name

* m1 madmax

* Add linux/linux arm

* pip install -e for arm installer, so that its consistent with the other platforms when looking for additional files

* madmax + windows

* Get madmax with Invoke-WebRequest

* Use the correct windows slashes

* add madmax to the list of windows binaries

* Check if madmax exists on windows install and move it to site packages if it does

* Make sure windows has .exe extension for madmax

* Update azure to get latest version of madmax from GH releases

* Bladebit for linux/linux arm

* Fix error with binaries.extend

* Bundle bladebit + windows

* Fix download url for bladebit

* Check for bladebit in windows script. move to correct directory if it exists

* Detect and use packaged plotters

* Removed unnecessary import

* Updating the branch to use chiaplotters_gui for installer verification

* Removed a change that was intended for debugging only

* Remove disclaimer

* Updated for new madmax plotter with k33, k34 support.
Updated chia-blockchain-gui submodule

* Fixed typo

* Package the chia_plot_k34 executable

* Boink

* Revert "Boink"

This reverts commit 8d13c07110.

* Additional chia_plot_k34 spots that I missed

* pyinstaller.spec fix for chia_plot_k34

* Windows installer fix for chia_plot_k34.exe

* Restoring chia-blockchain-gui submodule to 047ce16 (as in main)

* Update to chiapos 1.0.6

Co-authored-by: Jeff Cruikshank <paninaro@gmail.com>
Co-authored-by: Jeff Cruikshank <jeff@chia.net>
Co-authored-by: Kyle Altendorf <sda@fstab.net>
Co-authored-by: Chris Marslender <chrismarslender@gmail.com>
Co-authored-by: Earle Lowe <30607889+emlowe@users.noreply.github.com>
2021-10-28 15:37:46 -07:00

199 lines
4.8 KiB
Python

# -*- mode: python ; coding: utf-8 -*-
import importlib
import pathlib
import platform
from pkg_resources import get_distribution
from PyInstaller.utils.hooks import collect_submodules, copy_metadata
THIS_IS_WINDOWS = platform.system().lower().startswith("win")
THIS_IS_MAC = platform.system().lower().startswith("darwin")
ROOT = pathlib.Path(importlib.import_module("chia").__file__).absolute().parent.parent
def solve_name_collision_problem(analysis):
"""
There is a collision between the `chia` file name (which is the executable)
and the `chia` directory, which contains non-code resources like `english.txt`.
We move all the resources in the zipped area so there is no
need to create the `chia` directory, since the names collide.
Fetching data now requires going into a zip file, so it will be slower.
It's best if files that are used frequently are cached.
A sample large compressible file (1 MB of `/dev/zero`), seems to be
about eight times slower.
Note that this hack isn't documented, but seems to work.
"""
zipped = []
datas = []
for data in analysis.datas:
if str(data[0]).startswith("chia/"):
zipped.append(data)
else:
datas.append(data)
# items in this field are included in the binary
analysis.zipped_data = zipped
# these items will be dropped in the root folder uncompressed
analysis.datas = datas
keyring_imports = collect_submodules("keyring.backends")
# keyring uses entrypoints to read keyring.backends from metadata file entry_points.txt.
keyring_datas = copy_metadata("keyring")[0]
version_data = copy_metadata(get_distribution("chia-blockchain"))[0]
block_cipher = None
SERVERS = [
"wallet",
"full_node",
"harvester",
"farmer",
"introducer",
"timelord",
]
# TODO: collapse all these entry points into one `chia_exec` entrypoint that accepts the server as a parameter
entry_points = ["chia.cmds.chia"] + [f"chia.server.start_{s}" for s in SERVERS]
hiddenimports = []
hiddenimports.extend(entry_points)
hiddenimports.extend(keyring_imports)
binaries = [
(
f"{ROOT}/madmax/chia_plot",
"madmax"
),
(
f"{ROOT}/madmax/chia_plot_k34",
"madmax"
)
]
if not THIS_IS_MAC:
binaries.extend([
(
f"{ROOT}/bladebit/bladebit",
"bladebit"
)
])
if THIS_IS_WINDOWS:
hiddenimports.extend(["win32timezone", "win32cred", "pywintypes", "win32ctypes.pywin32"])
# this probably isn't necessary
if THIS_IS_WINDOWS:
entry_points.extend(["aiohttp", "chia.util.bip39"])
if THIS_IS_WINDOWS:
chia_mod = importlib.import_module("chia")
dll_paths = ROOT / "*.dll"
binaries = [
(
dll_paths,
".",
),
(
"C:\\Windows\\System32\\msvcp140.dll",
".",
),
(
"C:\\Windows\\System32\\vcruntime140_1.dll",
".",
),
(
f"{ROOT}\\madmax\\chia_plot.exe",
"madmax"
),
(
f"{ROOT}\\madmax\\chia_plot_k34.exe",
"madmax"
),
(
f"{ROOT}\\bladebit\\bladebit.exe",
"bladebit"
),
]
datas = []
datas.append((f"{ROOT}/chia/util/english.txt", "chia/util"))
datas.append((f"{ROOT}/chia/util/initial-config.yaml", "chia/util"))
datas.append((f"{ROOT}/chia/wallet/puzzles/*.hex", "chia/wallet/puzzles"))
datas.append((f"{ROOT}/chia/ssl/*", "chia/ssl"))
datas.append((f"{ROOT}/mozilla-ca/*", "mozilla-ca"))
datas.append(version_data)
pathex = []
def add_binary(name, path_to_script, collect_args):
analysis = Analysis(
[path_to_script],
pathex=pathex,
binaries=binaries,
datas=datas,
hiddenimports=hiddenimports,
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False,
)
solve_name_collision_problem(analysis)
binary_pyz = PYZ(analysis.pure, analysis.zipped_data, cipher=block_cipher)
binary_exe = EXE(
binary_pyz,
analysis.scripts,
[],
exclude_binaries=True,
name=name,
debug=False,
bootloader_ignore_signals=False,
strip=False,
)
collect_args.extend(
[
binary_exe,
analysis.binaries,
analysis.zipfiles,
analysis.datas,
]
)
COLLECT_ARGS = []
add_binary("chia", f"{ROOT}/chia/cmds/chia.py", COLLECT_ARGS)
add_binary("daemon", f"{ROOT}/chia/daemon/server.py", COLLECT_ARGS)
for server in SERVERS:
add_binary(f"start_{server}", f"{ROOT}/chia/server/start_{server}.py", COLLECT_ARGS)
COLLECT_KWARGS = dict(
strip=False,
upx_exclude=[],
name="daemon",
)
coll = COLLECT(*COLLECT_ARGS, **COLLECT_KWARGS)